求解最小二乘问题的带动量的Gauss-Seidel方法

Gauss-Seidel Method with Momentum for Solving Least-Squares Problems

最小二乘问题是重要的数学与统计模型,广泛用于回归分析、参数估计、最优控制和数据拟合等领域。基于古典的Gauss-Seidel方法,推导了求解最小二乘问题的迭代格式。结合Gauss-Seidel方法和Polyak's Heavy-Ball技术,提出了动量型Gauss-Seidel方法的算法框架。根据贪婪的策略选择指标,建立了贪婪的动量型Gauss-Seidel方法的线性收敛性。最后,数值实验表明贪婪的动量型Gauss-Seidel方法在迭代步数和计算时间方面均优于贪婪的Gauss-Seidel方法。

The least-squares problem is an important mathematical and statistical model,which is widely used in regression analysis,parameter estimation,optimal control and data fitting.Based on the classical Gauss-Seidel method,the iterative scheme for solving the least-squares problem is deduced.Combining Gauss-Seidel method and Polyaks Heavy-Ball technique,an algorithm framework of Gauss-Seidel method with momentum is proposed.The linear convergence of the greedy Gauss-Seidel method with momentum is established by selecting the column index according to the greedy strategy.Finally,numerical experiments show that the greedy Gauss-Seidel method with momentum outperforms the greedy Gauss-Seidel method in terms of iteration steps and computation time.