Generalization properties of restricted Boltzmann machine for short-range order

A biased sampling algorithm for the restricted Boltzmann machine(RBM) is proposed, which allows generating configurations with a conserved quantity. To validate the method, a study of the short-range order in binary alloys with positive and negative exchange interactions is carried out. The network is trained on the data collected by Monte–Carlo simulations for a simple Ising-like binary alloy model and used to calculate the Warren–Cowley short-range order parameter and other thermodynamic properties. We demonstrate that the proposed method allows us not only to correctly reproduce the order parameters for the alloy concentration at which the network was trained, but can also predict them for any other concentrations.

Restricted Boltzmann machine: Recent advances and mean-field theory

This review deals with restricted Boltzmann machine(RBM) under the light of statistical physics.The RBM is a classical family of machine learning(ML) models which played a central role in the development of deep learning.Viewing it as a spin glass model and exhibiting various links with other models of statistical physics,we gather recent results dealing with mean-field theory in this context.First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM,leading in particular to identify a compositional phase where a small number of features or modes are combined to form complex patterns.Then we discuss recent works either able to devise mean-field based learning algorithms;either able to reproduce generic aspects of the learning process from some ensemble dynamics equations or/and from linear stability arguments.

Sustainable Investment Forecasting of Power Grids Based on theDeep Restricted Boltzmann Machine Optimized by the Lion Algorithm

This paper proposes a new power grid investment prediction model based on the deep restricted Boltzmann machine(DRBM)optimized by the Lion algorithm(LA).Firstly,two factors including transmission and distribution price reform(TDPR)and 5G station construction were comprehensively incorporated into the consideration of influencing factors,and the fuzzy threshold method was used to screen out critical influencing factors.Then,the LA was used to optimize the parameters of the DRBM model to improve the model’s prediction accuracy,and the model was trained with the selected influencing factors and investment.Finally,the LA-DRBM model was used to predict the investment of a power grid enterprise,and the final prediction result was obtained by modifying the initial result with the modifying factors.The LA-DRBMmodel compensates for the deficiency of the singlemodel,and greatly improves the investment prediction accuracy of the power grid.In this study,a power grid enterprise was taken as an example to carry out an empirical analysis to prove the validity of the model,and a comparison with the RBM,support vector machine(SVM),back propagation neural network(BPNN),and regression model was conducted to verify the superiority of the model.The conclusion indicates that the proposed model has a strong generalization ability and good robustness,is able to abstract the combination of low-level features into high-level features,and can improve the efficiency of the model’s calculations for investment prediction of power grid enterprises.

Classification of steel based on laser-induced breakdown spectroscopy combined with restricted Boltzmann machine and support vector machine

In recent years,a laser-induced breakdown spectrometer(LIBS)combined with machine learning has been widely developed for steel classification.However,the much redundant information of LIBS spectra increases the computation complexity for classification.In this work,restricted Boltzmann machines(RBM)and principal component analysis(PCA)were used for dimension reduction of datasets,respectively.Then,a support vector machine(SVM)was adopted to process feature information.Two models(RBM-SVM and PCA-SVM)are compared in terms of performance.After optimization,the accuracy of the RBM-SVM model can achieve 100%,and the maximum dimension reduction time is 33.18 s,which is nearly half of that of the PCA model(53.19 s).These results preliminarily indicate that LIBS combined with RBM-SVM has great potential in the real-time classification of steel.

采用Reichardt运动检测器和Boltzmann Machine神经网络的运动计算

建立了一个探讨灵长类视皮层从V1区到MT区的运动信息加工原理的计算模型.这个过程的突出特征是视觉运动信息经过了从局部检测进步到整体感知.模型的第一层由用于抽提运动模式的局部速度以及结构性质的Reichardt运动检测器组成.进一步的加工是通过Boltzmann Machine神经网络来实现的.这种网络的学习算法具有局部更新的显著性质.在学习阶段,网络不断地修改联结权重以形成对于记录在网络的显单元上的环境结构的内部表征.此处的结构是显示给网络的各个局部运动二雏矢量场与对应的模式运动真实方向的组合.网络的训练受到一些网络参数的影响:总体结果是,随着学习样本数的增加,该网络的正确判断率不断提高,从而表明,无论运动模式朝向如何,该网络都能够判断出它的真实运动方向.

Green's function Monte Carlo method combined with restricted Boltzmann machine approach to the frustrated J_(1)–J_(2)Heisenberg model

Restricted Boltzmann machine(RBM)has been proposed as a powerful variational ansatz to represent the ground state of a given quantum many-body system.On the other hand,as a shallow neural network,it is found that the RBM is still hardly able to capture the characteristics of systems with large sizes or complicated interactions.In order to find a way out of the dilemma,here,we propose to adopt the Green's function Monte Carlo(GFMC)method for which the RBM is used as a guiding wave function.To demonstrate the implementation and effectiveness of the proposal,we have applied the proposal to study the frustrated J_(1)-J_(2)Heisenberg model on a square lattice,which is considered as a typical model with sign problem for quantum Monte Carlo simulations.The calculation results demonstrate that the GFMC method can significantly further reduce the relative error of the ground-state energy on the basis of the RBM variational results.This encourages to combine the GFMC method with other neural networks like convolutional neural networks for dealing with more models with sign problem in the future.

FPGA Implementation of a Scalable and Highly Parallel Architecture for Restricted Boltzmann Machines

Restricted Boltzmann Machines (RBMs) are an effective model for machine learning;however, they require a significant amount of processing time. In this study, we propose a highly parallel, highly flexible architecture that combines small and completely parallel RBMs. This proposal addresses problems associated with calculation speed and exponential increases in circuit scale. We show that this architecture can optionally respond to the trade-offs between these two problems. Furthermore, our FPGA implementation performs at a 134 times processing speed up factor with respect to a conventional CPU.

Data-driven optimization study of the multi-relaxation-time lattice Boltzmann method for solid-liquid phase change

Sharp phase interfaces and accurate temperature distributions are important criteria in the simulation of solid-liquid phase changes.The multi-relaxation-time lattice Boltzmann method(MRT-LBM)shows great numerical performance during simulation;however,the value method of the relaxation parameters needs to be specified.Therefore,in this study,a random forest(RF)model is used to discriminate the importance of different relaxation parameters to the convergence,and a support vector machine(SVM)is used to explore the decision boundary of the convergent samples in each dimensional model.The results show that the convergence of the samples is consistent with the sign of the decision number,and two types of the numerical deviations appear,i.e.,the phase mushy zone and the non-physical heat transfer.The relaxation parameters chosen on the decision boundary can further suppress the numerical bias and improve numerical accuracy.

受限玻尔兹曼机及其变体研究综述

受限玻尔兹曼机作为学习数据分布和提取内在特征的典型概率图模型,是深度学习领域重要的基础模型。近年来,通过改进受限玻尔兹曼机的模型结构和能量函数得到众多新兴模型,即受限玻尔兹曼机变体,可以进一步提升模型的特征提取性能。研究受限玻尔兹曼机及其变体能够显著促进深度学习领域的发展,实现大数据时代海量信息的快速提取。基于此,对近年来受限玻尔兹曼机及其变体的相关研究进行系统回顾,并创新性地从训练算法改进、模型结构改进、模型深层融合研究和模型相关最新应用4个方面进行全面综述。其中,重点梳理受限玻尔兹曼机训练算法和变体模型的发展史。最后,讨论受限玻尔兹曼机及其变体领域的现存难点与挑战,对主要研究工作进行总结与展望。

基于投票机制的暖通空调空气处理单元传感器故障诊断

现有暖通空调(HVAC)空气处理单元(AHU)的故障诊断研究往往是集中式的。少量的分布式方法大多需要求解大量耗时的优化问题,使得无法及时完成故障诊断。针对以上挑战,该文提出一种基于投票机制的分布式故障诊断方法。在该方法中,建立一个玻尔兹曼机来描述传感器网络,通过传感器之间的相互投票来确定玻尔兹曼机的边权值,基于边权值对玻尔兹曼机的状态也就是传感器的状态进行迭代,从而定位传感器的故障。设计了一种基于欧氏距离的投票策略确定投票值。开发了一种方法,通过在玻尔兹曼机中增加一个额外的节点来重置其权值矩阵,在将玻尔兹曼机对称化的同时,保持原来各传感器之间的投票关系,以保证玻尔兹曼机状态的迭代收敛。该方法不需要求解大量的优化问题,相较于当前的分布式方法计算量小。使用ASHRAE Project RP-1312提供的实际数据对所提方法进行验证。实验结果表明所提方法可以精确且高效地诊断出空气处理单元传感器的偏差故障和漂移故障。

基于Boltzmann机的矿产靶区预测

矿产靶区预测是一种从统计单元集合中识别找矿靶区的非线性模式识别过程,可以利用Boltzmann机能够对外部刺激进行编码和重建的功能,实现基于Boltzmann机的矿产靶区非线性统计预测过程.鉴于此,笔者定义了面向矿产靶区预测的三层Boltzmann机模型,模型输入层神经元数目等于找矿证据数目,输出层只有一个神经元,隐藏层神经元数目由用户根据矿产靶区预测的精度要求确定;模型应用Hebbian编码和模拟退火算法相结合的随机学习算法进行训练,根据学习训练后模型输入层与隐藏层神经元之间的连接权确定找矿证据的权系数;根据证据权系数和统计单元证据组合特征计算单元成矿有利度,圈定找矿靶区.在GDAL数字图像输入输出函数库基础上,用VC++语言开发了面向栅格数据的矿产靶区预测Boltzmann机算法程序并应用于新疆阿勒泰地区的矿产靶区预测研究.结果表明,Boltzmann机模型预测的统计单元成矿有利度能够正确反映研究区已知矿床(点)的空间分布规律,因此,基于Boltzmann机的矿产靶区非线性统计预测模型是有效的.

用遗传算法优化Boltzmann机

Boltzmann机是一种应用广泛的随机神经网络。它通过模拟退火算法进行网络学习,能取得一个全局或接近全局最优的最优值;通过期望网络模式和实际学习得到的网络模式比较来调节网络的权值,使网络能尽可能地达到或逼近期望的网络模式。将遗传算法运用到Boltzmann机的网络学习中,在对BM机编码后,通过选择、交叉和变异等遗传操作算子对网络进行训练,调整网络的权值,使适应度函数值大的网络保留下来,最终使网络达到期望的模式。通过实例验证,这是一种简单可行的调节网络权值的方法。

用改进的Paik型Boltzmann机实现图像复原

为解决传统的Boltzmann机方法不仅容易陷入局部最小点,而且收敛速度慢问题,对传统的Boltzmann机进行了改进。将Paik′s算法与Boltzmann机结合,使串行模式推广到并行模式以加快收敛速度;使用亚单位步长增进技术增加计算精度;最后,为折中收敛速度与收敛精度这一对矛盾,采用了自适应步长策略。对算法的改进进行了理论验证、收敛性分析并对残差变化进行了讨论。实验表明,该方法能够收敛到全局最优,复原结果的峰值信噪比比改进的Boltzmann机法获得的峰值信噪比高0.5～0.8dB,且收敛速度仅为该方法的1/3,证明了本文提出的改进的Paik型Boltzmann机对图像复原是有效的。

基于判别性无穷模糊受限玻尔兹曼机模型的HRRP序列识别

针对雷达高分辨率距离像(High Resolution Range Profiles, HRRP)序列数据在受干扰或背景杂波严重时目标识别性能较差的问题,提出一种判别性无穷模糊受限玻尔兹曼机(Discriminative Infinite Fuzzy Re-stricted Boltzmann Machine, Dis-iFRBM)模型。该模型结合判别性受限玻尔兹曼机分类和无穷受限玻尔兹曼机模型复杂度自适应特点,汲取模糊神经网络在低信噪比环境下提取特征更稳定的优点,将模型参数从实数扩展为模糊参数,实现了对HRRP序列数据原始特征的更稳定提取以及对雷达目标的更稳健识别。通过对多个HRRP序列的识别实验,验证了Dis-iFRBM的识别稳定性以及鲁棒性,与其他模型的对比实验验证了所提模型在“噪声”环境中的有效性和优越性。

受限制Boltzmann机深度置信网络与手写数字识别

手写数字因为每个人的书写习惯不同而差异很大。手写数字快速有效的识别一直是图像识别领域的热门话题。提出了一种基于受限制Boltzmann机组成的深度置信网络应用于手写数字识别的算法。对训练样本图片中的手写数字进行定位,分块,计算每块的特征值;用由受限制Boltzmann机组成的深度置信网络对样本特征值进行无监督学习;使用训练好的神经网络对待测样本进行深度特征提取。结果表明:该算法可行,相对于浅层学习,深度学习在图像特征提取上有了很大提高。

基于DBN网络的滚动轴承故障诊断

为了提高滚动轴承故障诊断的准确率,提出基于DBN网络的滚动轴承故障诊断方法。针对浅层神经网络难以从振动信号中提取深层故障特征且易陷入维度灾难等技术难点,结合深度置信网络(DBN)能够处理高维非线性数据和有效提取故障特征的特点,建立基于DBN网络的滚动轴承故障诊断模型。通过验证分析,确定了DBN的隐含层层数、最佳数据类型、激活函数等网络参数,为DBN网络参数的设置提供一种新的方法与思路。并对受限玻尔兹曼机(RBM)的重构能力进行了验证。将DBN网络与BP、ELM、PNN等浅层神经网络进行了对比分析,结果表明DBN网络具有较高的诊断精度与较强的稳定性,证明了DBN网络在滚动轴承故障诊断中的有效性。

用户行为驱动偏好代理模型辅助的交互式个性化进化搜索算法

随着互联网用户数量迅猛增长,产生了大量用户生成内容,出现了信息过载现象.考虑用户生成数据,建立用户兴趣偏好模型,同时联合交互式进化计算,提出用户行为驱动偏好代理模型辅助的交互式个性化进化搜索算法,帮助用户从海量搜索空间中搜寻符合用户潜在需求和兴趣偏好的项目或内容.利用用户交互行为、评分数据和项目类别信息,构建基于受限玻尔兹曼机的用户偏好感知模型,抽取用户偏好特征;从进化优化的角度,设计基于用户偏好的代理模型及其进化策略,生成含用户偏好的新个体,并预测进化个体适应值,引导进化优化过程;根据新增用户生成内容和模型管理机制,动态更新各模型,及时跟踪用户偏好,顺利完成个性化进化搜索.通过大量真实世界数据集的实验,验证了所提算法处理动态个性化搜索和推荐任务的可行性及有效性.

应用Boltzmann机实现离散线性二次型动态优化

为了求解线性二次型的动态优化控制问题,提出了一种基于随机神经网络Boltzmann机求解线性二次型动态优化控制问题的方法,将系统的性能指标转化成Boltzmann机的能量函数,将控制序列与网络的神经元相对应,Boltzmann机的稳态对应的就是最优的控制序列.理论研究表明,对于任意时变多变量的线性二次型,都能找到一个相应的。Boltzmann机,它的能量函数与二次型的性能指标等价.仿真试验表明,应用Boltzmann机可实现任意时变多变量系统的线性二次型动态优化控制.

基于MPSR和IRBM的电力系统中长期负荷预测

针对电力系统中长期负荷波动大及不确定因素导致负荷预测误差较大的问题,提出了一种基于多变量相空间重构(MPSR)和改进受限波尔兹曼机(IRBM)的电力系统中长期负荷预测方法。首先,利用多元线性回归分析方法分析天气因素与电负荷之间的相关性,并将其与电负荷序列组成多变量时间序列;然后,利用C-C法确定每一时间序列的最优嵌入维数和时间延迟,实现多变量相空间重构;最后,采用多变量相空间重构建立的数据集训练电力系统负荷预测模型,同时利用梯度优化法对参数进行优化,得到预测模型。结果表明:相比长短期记忆神经网络和粒子群优化BP神经网络,所提出的预测方法有较高的精准度。

Monitoring and diagnosis of complex production process based on free energy of Gaussian–Bernoulli restricted Boltzmann machine

Online monitoring and diagnosis of production processes face great challenges due to the nonlinearity and multivariate of complex industrial processes.Traditional process monitoring methods employ kernel function or multilayer neural networks to solve the nonlinear mapping problem of data.However,the above methods increase the model complexity and are not interpretable,leading to difficulties in subsequent fault recognition/diagnosis/location.A process monitoring and diagnosis method based on the free energy of Gaussian-Bernoulli restricted Boltzmann machine(GBRBM-FE)was proposed.Firstly,a GBRBM network was established to make the probability distribution of the reconstructed data as close as possible to the probability distribution of the raw data.On this basis,the weights and biases in GBRBM network were used to construct F statistics,which represents the free energy of the sample.The smaller the energy of the sample is,the more normal the sample is.Therefore,F statistics can be used to monitor the production process.To diagnose fault variables,the F statistic for each sample was decomposed to obtain the Fv statistic for each variable.By analyzing the deviation degree between the corresponding variables of abnormal samples and normal samples,the cause of process abnormalities can be accurately located.The application of converter steelmaking process demonstrates that the proposed method outperforms the traditional methods,in terms of fault monitoring and diagnosis performance.