A local minimum is frequently encountered in the training of back propagation neural networks (BPNN), which sharply slows the training process. In this paper, an analysis of the formation of local minima is presented,...A local minimum is frequently encountered in the training of back propagation neural networks (BPNN), which sharply slows the training process. In this paper, an analysis of the formation of local minima is presented, and an improved genetic algorithm (GA) is introduced to overcome local minima. The Sigmoid function is generally used as the activation function of BPNN nodes. It is the flat characteristic of the Sigmoid function that results in the formation of local minima. In the improved GA, pertinent modifications are made to the evaluation function and the mutation model. The evaluation of the solution is associated with both the training error and gradient. The sensitivity of the error function to network parameters is used to form a self adapting mutation model. An example of industrial application shows the advantage of the improved GA to overcome local minima.展开更多
影响图模型选择中存在数据依赖性、计算复杂性和非概率关系问题.通过对影响图结构进行分解,提出PS-EM 算法对影响图的概率结构部分进行模型选择.给出一种 BP 神经网络,通过对局部效用函数的学习实现效用结构部分的模型选择,并引入权重...影响图模型选择中存在数据依赖性、计算复杂性和非概率关系问题.通过对影响图结构进行分解,提出PS-EM 算法对影响图的概率结构部分进行模型选择.给出一种 BP 神经网络,通过对局部效用函数的学习实现效用结构部分的模型选择,并引入权重阈值来避免过拟合.PS-EM 算法是在 SEM 算法中引入一种融合先验知识的MDL 评分标准来降低传统 MDL 评分对数据的依赖性,并通过将参数学习和结构评分分开计算提高计算效率.算法比较的结果显示 PS-EM 比标准 SEM 的时间性能好、对数据依赖性小,且效用部分的结构选择易于实现.展开更多
文摘A local minimum is frequently encountered in the training of back propagation neural networks (BPNN), which sharply slows the training process. In this paper, an analysis of the formation of local minima is presented, and an improved genetic algorithm (GA) is introduced to overcome local minima. The Sigmoid function is generally used as the activation function of BPNN nodes. It is the flat characteristic of the Sigmoid function that results in the formation of local minima. In the improved GA, pertinent modifications are made to the evaluation function and the mutation model. The evaluation of the solution is associated with both the training error and gradient. The sensitivity of the error function to network parameters is used to form a self adapting mutation model. An example of industrial application shows the advantage of the improved GA to overcome local minima.
文摘影响图模型选择中存在数据依赖性、计算复杂性和非概率关系问题.通过对影响图结构进行分解,提出PS-EM 算法对影响图的概率结构部分进行模型选择.给出一种 BP 神经网络,通过对局部效用函数的学习实现效用结构部分的模型选择,并引入权重阈值来避免过拟合.PS-EM 算法是在 SEM 算法中引入一种融合先验知识的MDL 评分标准来降低传统 MDL 评分对数据的依赖性,并通过将参数学习和结构评分分开计算提高计算效率.算法比较的结果显示 PS-EM 比标准 SEM 的时间性能好、对数据依赖性小,且效用部分的结构选择易于实现.
