非线性规划U—D分解方法及其在神经网络训练中的应用

U-D FACTORIZATION-BASED NONLINEAR PROGRAMMING METHOD AND ITS APPLICATION IN NEURAL NETWORK TRAINING

提出一种有效的Ｕ－Ｄ分解ＤＦＰ和ＢＦＧＳ算法．该算法解决了Ｈ阵的正定性问题，保证了算法的数值稳定性，并大大提高了计算效率．对Ｈ阵的计算量分析表明，该算法的计算效率比普通方法高２０％，比普通平方根方法高０．４ｎ（ｎ为Ｈ阵维数）倍．神经网络训练的应用表明，新算法比普通ＤＰＰ和ＢＦＧＳ方法更有效、更准确．

To solve convergence rate problems of often used DFP and BFCS methods, the stable construction of inverse Hassian matrix are presented. To get high numerical stability and computational efficiency, U-D factorization-based DFP and BFGS algorithms are developed. In the new methods the positive definiteness of the inverse matrix H is ensured and both the stability and convergence of the algorithm is improved. By using rank-one U-D factorization updates of H, the numerical accuracy and efficiency are increased. Operational counts for computing H show that the efficiency of the new algorithm is increased by 20% and the storages of matrix H is reduced by 50%. Results of several numerical example show that the optimization problems can be solved by using the programming methods presented in this paper and accurate results may be obtained.