极大似然最大熵概率密度估计及其优化解法

Estimation and Optimization of MLE Maximum Entropy Probability Density

针对经典最大熵概率密度估计中拉格朗日乘子计算目前存在高度非线性、计算精度不高或有时难以收敛等问题,提出了一种"最大似然+逐次优化"的方法。基于最大似然估计法,推导建立了简化的拉格朗日优化函数;在此基础上,基于样本原点矩约束,提出了逐次寻优算法。根据优化过程不稳定,重新推导了拉格朗日乘子的线性变换公式,避免矩阵求逆运算引起的奇异现象。针对几种常见的概率分布类型及可靠性问题,采用极大似然最大熵概率密度估计法与经典型最大熵概率密度估计法分别计算概率密度及可靠度的对比表明:极大似然最大熵概率密度估计法的优化函数非线性程度低,形式简单,而且"极大似然最大熵概率密度估计+逐次优化法计算"精度高,收敛性好。

Aiming at high nonlinearity, low computational accuracy or hard convergence of Lagrangian multiplier calculation in the probability density function estimation by the classic maximum entropy method, a new method combining the maximum likelihood estimation （MLE） maximum entropy proba bility density method with the sequential updating method is proposed in this paper. Lagrangian optimi zation function with low nonlinearity is established on the basis of MIRE. Furthermore, the sequential updating method is proposed which is constrained by the sample origin moments. Because of unsteady in the process of optimization, the transformation formula of Lagrangian multiplier is deduced again to avoid singularity phenomenon caused by matrix inversion. By analyzing several common distribution and reliability issues using the MLE maximum entropy probability density method and the classic maximum entropy probability density method, it is found that the MLE maximum entropy probability density method has advantage of low nonlinearity and simple form in the optimization function, while the new combination method does well in computational accuracy and optimization convergence.