嵌入Stabilized Barzilai-Borwein步长的一阶随机优化算法研究

Research on First-order Stochastic Optimization Algorithm for Embedded with Stabilized Barzilai-Borwein Step Size

近年来,机器学习发展迅速,在广泛应用于各个领域的同时取得了许多理论突破。它为系统提供访问数据的能力,通过从过去的经验中学习和改进来解决复杂问题,使机器能够执行认知功能。文中将基于改进的Stabilized Barzilai-Borwein(SBB)方法自动计算步长,与SVRG结合形成新的算法SVRG-SBB,并从理论上证明新算法收敛,能够有效地解决机器学习中的常见问题。

In recent years,machine learning is developing rapidly and has made many theoretical breakthroughs while being widely applied in various fields.It provides the system with the ability to access data,solve complex problems by learning from past experience and improving,and enable the machine to perform cognitive functions.In this paper,the step size is automatically calculated based on the improved Stabilized Barzilai-Borwein(SBB) method,which is combined with SVRG to form a new algorithm SVRG-SBB.It is proved theoretically that the new algorithm converges and can effectively solve the common problems in machine learning.