最小二乘法线性拟合

发表于 更新于 分类于 ， 阅读次数：

概述

最小二乘法是曲线拟合的常用方法，使用该方法对匹配函数的选取非常重要。 青岛

理论

  1. 假设拟合的多项式为:

\[ y = a_0 + a_1*x +a_2*x^2 + ...+a_m*x^m \]

其中，m代表多项式的阶数。

  1. 离散点与该多项式的平方和\(F(a_0,a_1,,a_m)\)为，其中n代表采样点数：

\[ F(a_0,a_1,,a_m) = \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)]^2 \]

  1. 最小二乘法的思想是求平方和函数\(F(a_0,a_1,,a_m)\)的最小值，而对于二次方程求最小值的问题，常见的思路就是对方程求导，倒数为零的点，及为方程的极值点。

  2. 下面对\(F(a_0,a_1,,a_m)\)分别求\(a_i\)偏导数，得：

\[ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] = 0 \\ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] x_i= 0\\ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] x_i^2= 0\\ \ldots\\ -2 \sum_{i=0}^n[y_i - (a_0 + a_1*x_i +a_2*x_i^2 + ...+a_m*x_i^m)] x_i^m= 0 \]

  1. 整理得

\[ a_0\sum_{i=0}^{n}+a_1\sum_{i=0}^{n} x_i+ a_2\sum_{i=0}^{n} x_i^2+...+a_m\sum_{i=0}^{n}x_i^m = \sum_{i=0}^{n}y_i\\ a_0\sum_{i=0}^{n} x_i+a_1\sum_{i=0}^{n} x_i^2+ a_2\sum_{i=0}^{n} x_i^3+...+a_m\sum_{i=0}^{n}x_i^{m+1} = \sum_{i=0}^{n}y_ix_i\\ \ldots\\ a_0\sum_{i=0}^{n} x_i^m+a_1\sum_{i=0}^{n} x_i^{m+1}+ a_2\sum_{i=0}^{n} x_i^{m+2}+...+a_m\sum_{i=0}^{n}x_i^{2m} = \sum_{i=0}^{n}y_ix_i^m \]

  1. 使用矩阵形式表示为：

\[ \left[ \begin{matrix} \sum_{i=0}^{n} & \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}x_i^2&\ldots&\sum_{i=0}^{n}x_i^m\\ \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}x_i^2&\sum_{i=0}^{n}x_i^3&\ldots&\sum_{i=0}^{n}x_i^{m+1}\\ \sum_{i=0}^{n}x_i^2&\sum_{i=0}^{n}x_i^3&\sum_{i=0}^{n}x_i^4&\ldots&\sum_{i=0}^{n}x_i^{m+2}\\ \ldots&\ldots&\ldots&\ldots&\ldots\\ \sum_{i=0}^{n}x_i^{m}&\sum_{i=0}^{n}x_i^{m+1}&\sum_{i=0}^{n}x_i^{m+2}&\ldots&\sum_{i=0}^{n}x_i^{2m} \end{matrix} \right] \left[ \begin{matrix} a_0\\ a_1\\ a_2\\ \ldots\\ a_m \end{matrix} \right]=\\ \left[ \begin{matrix} \sum_{i=0}^{n}y_i\\ \sum_{i=0}^{n}y_ix_i\\ \sum_{i=0}^{n}y_ix_i^2\\ \ldots\\ \sum_{i=0}^{n}y_ix_i^m\\ \end{matrix} \right] \]

  1. 下面就是求解一次线性方程，常用的方法是使用高斯消元法，也可以使用求矩阵的秩或者求解逆矩阵等方式求解。

高斯消元法求解线性方程

原则

  • 两个方程互换解不变
  • 一个方程乘以非零K,解不变
  • 一个方程乘以非零K,加上另一个方程解不变

一阶线性函数

假设多项式得阶数m为2，则上述矩阵方程为：

\[ \left[ \begin{matrix} \sum_{i=0}^{n} & \sum_{i=0}^{n}x_i\\ \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}x_i^2& \end{matrix} \right] \left[ \begin{matrix} a_0\\ a_1 \end{matrix} \right]=\\ \left[ \begin{matrix} \sum_{i=0}^{n}y_i\\ \sum_{i=0}^{n}y_ix_i \end{matrix} \right] \]

写成行列式的形式为：

\[ \begin{vmatrix} \sum_{i=0}^{n} & \sum_{i=0}^{n}x_i&\sum_{i=0}^{n}y_i\\ \sum_{i=0}^{n}x_i & \sum_{i=0}^{n}x_i^2&\sum_{i=0}^{n}y_ix_i\\ \end{vmatrix} \]

第一行乘以系数\(\sum_{i=0}^{n}\)，第二行乘以系数\(\sum_{i=0}^{n}x_i\)

\[ \begin{vmatrix} 1 & \frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}} & \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}}\\ 1 & \frac{\sum_{i=0}^{n}x_i^2}{\sum_{i=0}^{n}x_i} & \frac{\sum_{i=0}^{n}y_ix_i}{\sum_{i=0}^{n}x_i} \end{vmatrix} \]

第二行减去第一行得

\[ \begin{vmatrix} 1 & \frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}} & \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}}\\ 0 & \frac{\sum_{i=0}^{n}x_i^2}{\sum_{i=0}^{n}x_i} -\frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}} & \frac{\sum_{i=0}^{n}y_ix_i}{\sum_{i=0}^{n}x_i} - \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}} \end{vmatrix} \]

通过上式可得\(a_1\)为

\[ a_1 = \frac{\frac{\sum_{i=0}^{n}y_ix_i}{\sum_{i=0}^{n}x_i} - \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}}}{\frac{\sum_{i=0}^{n}x_i^2}{\sum_{i=0}^{n}x_i} -\frac{\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}}} =\frac{\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i} \]

同理将\(a_1\)回代入得：

\[ a_0 = \frac{\sum_{i=0}^{n}y_i}{\sum_{i=0}^{n}} - \frac{(\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}x_i}{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}}\\ =\frac{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}y_i -(\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}x_i}{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}}\\ =\frac{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n}\sum_{i=0}^{n}y_i -\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}\sum_{i=0}^{n}x_i}{(\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i)*\sum_{i=0}^{n}}\\ =\frac{\sum_{i=0}^{n}x_i^2\sum_{i=0}^{n}y_i -\sum_{i=0}^{n}y_ix_i\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i} \]

故最终可得

\[ a_0 =\frac{\sum_{i=0}^{n}x_i^2\sum_{i=0}^{n}y_i -\sum_{i=0}^{n}y_ix_i\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i} \]

\[ a_1 =\frac{\sum_{i=0}^{n}y_ix_i*\sum_{i=0}^{n}-\sum_{i=0}^{n}y_i*\sum_{i=0}^{n}x_i}{\sum_{i=0}^{n}x_i^2*\sum_{i=0}^{n} - \sum_{i=0}^{n}x_i*\sum_{i=0}^{n}x_i} \]