This review deals with restricted Boltzmann machine(RBM) under the light of statistical physics.The RBM is a classical family of machine learning(ML) models which played a central role in the development of deep learn...This review deals with restricted Boltzmann machine(RBM) under the light of statistical physics.The RBM is a classical family of machine learning(ML) models which played a central role in the development of deep learning.Viewing it as a spin glass model and exhibiting various links with other models of statistical physics,we gather recent results dealing with mean-field theory in this context.First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM,leading in particular to identify a compositional phase where a small number of features or modes are combined to form complex patterns.Then we discuss recent works either able to devise mean-field based learning algorithms;either able to reproduce generic aspects of the learning process from some ensemble dynamics equations or/and from linear stability arguments.展开更多
Restricted Boltzmann Machines (RBMs) are an effective model for machine learning;however, they require a significant amount of processing time. In this study, we propose a highly parallel, highly flexible architecture...Restricted Boltzmann Machines (RBMs) are an effective model for machine learning;however, they require a significant amount of processing time. In this study, we propose a highly parallel, highly flexible architecture that combines small and completely parallel RBMs. This proposal addresses problems associated with calculation speed and exponential increases in circuit scale. We show that this architecture can optionally respond to the trade-offs between these two problems. Furthermore, our FPGA implementation performs at a 134 times processing speed up factor with respect to a conventional CPU.展开更多
Time series forecasting research area mainly focuses on developing effective forecasting models toimprove prediction accuracy. An ensemble model composed of autoregressive integrated movingaverage （ARIMA）, artificia...Time series forecasting research area mainly focuses on developing effective forecasting models toimprove prediction accuracy. An ensemble model composed of autoregressive integrated movingaverage （ARIMA）, artificial neural network （ANN）, restricted Boltzmann machines （RBM）, anddiscrete wavelet transform （DWT） is presented in this paper. In the proposed model, DWT firstdecomposes time series into approximation and detail. Then Khashei and Bijari＇s model, which is anensemble model of ARIMA and ANN, is applied to the approximation and detail to extract their bothlinear and nonlinear components and fit the relationship between the components as a function insteadof additive relationship. Furthermore, RBM is used to perform pre-training for generating initialweights and biases based on inputs feature for ANN. Finally, the forecasted approximation and detailare combined to obtain final forecasting. The forecasting capability of the proposed model is testedwith three well-known time series： sunspot, Canadian lynx, exchange rate time series. The predictionperformance is compared to the other six forecasting models. The results indicate that the proposedmodel gives the best performance in all three data sets and all three measures （i.e. MSE, MAE andMAPE）.展开更多
基金supported by the Comunidad de Madrid and the Complutense University of Madrid (Spain) through the Atracción de Talento program (Ref. 2019-T1/TIC-13298)
文摘This review deals with restricted Boltzmann machine(RBM) under the light of statistical physics.The RBM is a classical family of machine learning(ML) models which played a central role in the development of deep learning.Viewing it as a spin glass model and exhibiting various links with other models of statistical physics,we gather recent results dealing with mean-field theory in this context.First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM,leading in particular to identify a compositional phase where a small number of features or modes are combined to form complex patterns.Then we discuss recent works either able to devise mean-field based learning algorithms;either able to reproduce generic aspects of the learning process from some ensemble dynamics equations or/and from linear stability arguments.
文摘Restricted Boltzmann Machines (RBMs) are an effective model for machine learning;however, they require a significant amount of processing time. In this study, we propose a highly parallel, highly flexible architecture that combines small and completely parallel RBMs. This proposal addresses problems associated with calculation speed and exponential increases in circuit scale. We show that this architecture can optionally respond to the trade-offs between these two problems. Furthermore, our FPGA implementation performs at a 134 times processing speed up factor with respect to a conventional CPU.
文摘Time series forecasting research area mainly focuses on developing effective forecasting models toimprove prediction accuracy. An ensemble model composed of autoregressive integrated movingaverage （ARIMA）, artificial neural network （ANN）, restricted Boltzmann machines （RBM）, anddiscrete wavelet transform （DWT） is presented in this paper. In the proposed model, DWT firstdecomposes time series into approximation and detail. Then Khashei and Bijari＇s model, which is anensemble model of ARIMA and ANN, is applied to the approximation and detail to extract their bothlinear and nonlinear components and fit the relationship between the components as a function insteadof additive relationship. Furthermore, RBM is used to perform pre-training for generating initialweights and biases based on inputs feature for ANN. Finally, the forecasted approximation and detailare combined to obtain final forecasting. The forecasting capability of the proposed model is testedwith three well-known time series： sunspot, Canadian lynx, exchange rate time series. The predictionperformance is compared to the other six forecasting models. The results indicate that the proposedmodel gives the best performance in all three data sets and all three measures （i.e. MSE, MAE andMAPE）.
文摘面向用户生成内容(User generated content,UGC)的进化搜索在大数据及个性化服务领域已引起广泛关注,其关键在于基于多源异构用户生成内容构建用户认知偏好模型,进而设计高效的进化搜索机制.针对此,提出融合注意力机制(Attention mechanism,AM)的受限玻尔兹曼机(Restricted Boltzmann machine,RBM)偏好认知代理模型构建机制,并应用于交互式分布估计算法(Interactive estimation of distribution algorithm,IEDA),设计含用户生成内容的个性化进化搜索策略.基于用户群体提供的文本评论,以及搜索物品的类别文本,构建无监督受限玻尔兹曼机模型提取广义特征;设计注意力机制,融合广义特征,获取对用户认知偏好高度相关特征的集成;利用该特征再次训练受限玻尔兹曼机,实现对用户偏好认知代理模型的构建;根据用户偏好认知代理模型,给出交互式分布估计算法概率更新模型以及物品适应度评价函数,实现物品个性化进化搜索.算法在亚马逊个性化搜索实例的应用验证了用户认知偏好模型的可靠性,以及个性化进化搜索的有效性.
