Generalization properties of restricted Boltzmann machine for short-range order

A biased sampling algorithm for the restricted Boltzmann machine(RBM) is proposed, which allows generating configurations with a conserved quantity. To validate the method, a study of the short-range order in binary alloys with positive and negative exchange interactions is carried out. The network is trained on the data collected by Monte–Carlo simulations for a simple Ising-like binary alloy model and used to calculate the Warren–Cowley short-range order parameter and other thermodynamic properties. We demonstrate that the proposed method allows us not only to correctly reproduce the order parameters for the alloy concentration at which the network was trained, but can also predict them for any other concentrations.

Sustainable Investment Forecasting of Power Grids Based on theDeep Restricted Boltzmann Machine Optimized by the Lion Algorithm

This paper proposes a new power grid investment prediction model based on the deep restricted Boltzmann machine(DRBM)optimized by the Lion algorithm(LA).Firstly,two factors including transmission and distribution price reform(TDPR)and 5G station construction were comprehensively incorporated into the consideration of influencing factors,and the fuzzy threshold method was used to screen out critical influencing factors.Then,the LA was used to optimize the parameters of the DRBM model to improve the model’s prediction accuracy,and the model was trained with the selected influencing factors and investment.Finally,the LA-DRBM model was used to predict the investment of a power grid enterprise,and the final prediction result was obtained by modifying the initial result with the modifying factors.The LA-DRBMmodel compensates for the deficiency of the singlemodel,and greatly improves the investment prediction accuracy of the power grid.In this study,a power grid enterprise was taken as an example to carry out an empirical analysis to prove the validity of the model,and a comparison with the RBM,support vector machine(SVM),back propagation neural network(BPNN),and regression model was conducted to verify the superiority of the model.The conclusion indicates that the proposed model has a strong generalization ability and good robustness,is able to abstract the combination of low-level features into high-level features,and can improve the efficiency of the model’s calculations for investment prediction of power grid enterprises.

Restricted Boltzmann machine: Recent advances and mean-field theory

This review deals with restricted Boltzmann machine(RBM) under the light of statistical physics.The RBM is a classical family of machine learning(ML) models which played a central role in the development of deep learning.Viewing it as a spin glass model and exhibiting various links with other models of statistical physics,we gather recent results dealing with mean-field theory in this context.First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM,leading in particular to identify a compositional phase where a small number of features or modes are combined to form complex patterns.Then we discuss recent works either able to devise mean-field based learning algorithms;either able to reproduce generic aspects of the learning process from some ensemble dynamics equations or/and from linear stability arguments.

Classification of steel based on laser-induced breakdown spectroscopy combined with restricted Boltzmann machine and support vector machine

In recent years,a laser-induced breakdown spectrometer(LIBS)combined with machine learning has been widely developed for steel classification.However,the much redundant information of LIBS spectra increases the computation complexity for classification.In this work,restricted Boltzmann machines(RBM)and principal component analysis(PCA)were used for dimension reduction of datasets,respectively.Then,a support vector machine(SVM)was adopted to process feature information.Two models(RBM-SVM and PCA-SVM)are compared in terms of performance.After optimization,the accuracy of the RBM-SVM model can achieve 100%,and the maximum dimension reduction time is 33.18 s,which is nearly half of that of the PCA model(53.19 s).These results preliminarily indicate that LIBS combined with RBM-SVM has great potential in the real-time classification of steel.

Green's function Monte Carlo method combined with restricted Boltzmann machine approach to the frustrated J_(1)–J_(2)Heisenberg model

Restricted Boltzmann machine(RBM)has been proposed as a powerful variational ansatz to represent the ground state of a given quantum many-body system.On the other hand,as a shallow neural network,it is found that the RBM is still hardly able to capture the characteristics of systems with large sizes or complicated interactions.In order to find a way out of the dilemma,here,we propose to adopt the Green's function Monte Carlo(GFMC)method for which the RBM is used as a guiding wave function.To demonstrate the implementation and effectiveness of the proposal,we have applied the proposal to study the frustrated J_(1)-J_(2)Heisenberg model on a square lattice,which is considered as a typical model with sign problem for quantum Monte Carlo simulations.The calculation results demonstrate that the GFMC method can significantly further reduce the relative error of the ground-state energy on the basis of the RBM variational results.This encourages to combine the GFMC method with other neural networks like convolutional neural networks for dealing with more models with sign problem in the future.

FPGA Implementation of a Scalable and Highly Parallel Architecture for Restricted Boltzmann Machines

Restricted Boltzmann Machines (RBMs) are an effective model for machine learning;however, they require a significant amount of processing time. In this study, we propose a highly parallel, highly flexible architecture that combines small and completely parallel RBMs. This proposal addresses problems associated with calculation speed and exponential increases in circuit scale. We show that this architecture can optionally respond to the trade-offs between these two problems. Furthermore, our FPGA implementation performs at a 134 times processing speed up factor with respect to a conventional CPU.

Monitoring and diagnosis of complex production process based on free energy of Gaussian–Bernoulli restricted Boltzmann machine

Online monitoring and diagnosis of production processes face great challenges due to the nonlinearity and multivariate of complex industrial processes.Traditional process monitoring methods employ kernel function or multilayer neural networks to solve the nonlinear mapping problem of data.However,the above methods increase the model complexity and are not interpretable,leading to difficulties in subsequent fault recognition/diagnosis/location.A process monitoring and diagnosis method based on the free energy of Gaussian-Bernoulli restricted Boltzmann machine(GBRBM-FE)was proposed.Firstly,a GBRBM network was established to make the probability distribution of the reconstructed data as close as possible to the probability distribution of the raw data.On this basis,the weights and biases in GBRBM network were used to construct F statistics,which represents the free energy of the sample.The smaller the energy of the sample is,the more normal the sample is.Therefore,F statistics can be used to monitor the production process.To diagnose fault variables,the F statistic for each sample was decomposed to obtain the Fv statistic for each variable.By analyzing the deviation degree between the corresponding variables of abnormal samples and normal samples,the cause of process abnormalities can be accurately located.The application of converter steelmaking process demonstrates that the proposed method outperforms the traditional methods,in terms of fault monitoring and diagnosis performance.

A new item-based deep network structure using a restricted Boltzmann machine for collaborative filtering

The collaborative filtering(CF) technique has been widely used recently in recommendation systems. It needs historical data to give predictions. However, the data sparsity problem still exists. We propose a new item-based restricted Boltzmann machine(RBM) approach for CF and use the deep multilayer RBM network structure, which alleviates the data sparsity problem and has excellent ability to extract features. Each item is treated as a single RBM, and different items share the same weights and biases. The parameters are learned layer by layer in the deep network. The batch gradient descent algorithm with minibatch is used to increase the convergence speed. The new feature vector discovered by the multilayer RBM network structure is very effective in predicting a rating and achieves a better result. Experimental results on the data set of MovieL ens show that the item-based multilayer RBM approach achieves the best performance, with a mean absolute error of 0.6424 and a root-mean-square error of 0.7843.

AN ENSEMBLE MODEL OF ARIMA AND ANN WITH RESTRICTED BOLTZMANN MACHINE BASED ON DECOMPOSITION OF DISCRETE WAVELET TRANSFORM FOR TIME SERIES FORECASTING

Time series forecasting research area mainly focuses on developing effective forecasting models toimprove prediction accuracy. An ensemble model composed of autoregressive integrated movingaverage （ARIMA）, artificial neural network （ANN）, restricted Boltzmann machines （RBM）, anddiscrete wavelet transform （DWT） is presented in this paper. In the proposed model, DWT firstdecomposes time series into approximation and detail. Then Khashei and Bijari＇s model, which is anensemble model of ARIMA and ANN, is applied to the approximation and detail to extract their bothlinear and nonlinear components and fit the relationship between the components as a function insteadof additive relationship. Furthermore, RBM is used to perform pre-training for generating initialweights and biases based on inputs feature for ANN. Finally, the forecasted approximation and detailare combined to obtain final forecasting. The forecasting capability of the proposed model is testedwith three well-known time series： sunspot, Canadian lynx, exchange rate time series. The predictionperformance is compared to the other six forecasting models. The results indicate that the proposedmodel gives the best performance in all three data sets and all three measures （i.e. MSE, MAE andMAPE）.

Emotion recognition from thermal infrared images using deep Boltzmann machine

Facial expression and emotion recognition from thermal infrared images has attracted more and more attentions in recent years. However, the features adopted in current work are either temperature statistical parameters extracted from the facial regions of interest or several hand-crafted features that are commonly used in visible spectrum. Till now there are no image features specially designed for thermal infrared images. In this paper, we propose using the deep Boltzmann machine to learn thermal features for emotion recognition from thermal infrared facial images. First, the face is located and normalized from the thermal infrared im- ages. Then, a deep Boltzmann machine model composed of two layers is trained. The parameters of the deep Boltzmann machine model are further fine-tuned for emotion recognition after pre-tralning of feature learning. Comparative experimental results on the NVIE database demonstrate that our approach outperforms other approaches using temperature statistic features or hand-crafted features borrowed from visible domain. The learned features from the forehead, eye, and mouth are more effective for discriminating valence dimension of emotion than other facial areas. In addition, our study shows that adding unlabeled data from other database during training can also improve feature learning performance.

Low Impedance Fault Identification and Classification Based on Boltzmann Machine Learning for HVDC Transmission Systems

Identification and classification of DC faults are considered as fundamentals of DC grid protection.A sudden rise of DC fault current must be identified and classified to immediately operate the corresponding interrupting mechanism.In this paper,the Boltzmann machine learning(BML)approach is proposed for identification and classification of DC faults using travelling waves generated at fault point in voltage source converter based high-voltage direct current(VSC-HVDC)transmission system.An unsupervised way of feature extraction is performed on the frequency spectrum of the travelling waves.Binomial class logistic regression(BCLR)classifies the HVDC transmission system into faulty and healthy states.The proposed technique reduces the time for fault identification and classification because of reduced tagged data with few characteristics.Therefore,the faults near or at converter stations are readily identified and classified.The performance of the proposed technique is assessed via simulations developed in MATLAB/Simulink and tested for pre-fault and post-fault data both at VSC1 and VSC2,respectively.Moreover,the proposed technique is supported by analyzing the root mean square error to show practicality and realization with reduced computations.

Realizing number recognition with simulated quantum semi-restricted Boltzmann machine

Quantum machine learning based on quantum algorithms may achieve an exponential speedup over classical algorithms in dealing with some problems such as clustering.In this paper,we use the method of training the lower bound of the average log likelihood function on the quantum Boltzmann machine(QBM)to recognize the handwritten number datasets and compare the training results with classical models.We find that,when the QBM is semi-restricted,the training results get better with fewer computing resources.This shows that it is necessary to design a targeted algorithm to speed up computation and save resources.

Boltzmann machines with clusters of stochastic binary units

The original restricted Boltzmann machines(RBMs)are extended by replacing the binary visible and hidden variables with clusters of binary units,and a new learning algorithm for training deep Boltzmann machine of this new variant is proposed.The sum of binary units of each cluster is approximated by a Gaussian distribution.Experiments demonstrate that the proposed Boltzmann machines can achieve good performance in the MNIST handwritten digital recognition task.

Intrusion detection model based on deep belief nets

This paper focuses on the intrusion classification of huge amounts of data in a network intrusion detection system. An intrusion detection model based on deep belief nets （DBN） is proposed to conduct intrusion detection,and the principles regarding DBN are discussed.The DBN is composed of a multiple unsupervised restricted Boltzmann machine （RBM） and a supervised back propagation （BP）network.First,the DBN in the proposed model is pre-trained in a fast and greedy way,and each RBM is trained by the contrastive divergence algorithm.Secondly,the whole network is fine-tuned by the supervised BP algorithm,which is employed for classifying the low-dimensional features of the intrusion data generated by the last RBM layer simultaneously.The experimental results on the KDD CUP 1999 dataset demonstrate that the DBN using the RBM network with three or more layers outperforms the self-organizing maps （SOM）and neural network （NN）in intrusion classification.Therefore,the DBN is an efficient approach for intrusion detection in high-dimensional space.

Damage identification of steel truss bridges based on deep belief network

To improve the accuracy and anti-noise ability of the structural damage identification method,a bridge damage identification method is proposed based on a deep belief network(DBN).The output vector is used to establish the nonlinear mapping relationship between the mode shape and structural damage.The hidden layer of the DBN is trained through a layer-by-layer pre-training.Finally,the backpropagation algorithm is used to fine-tune the entire network.The method is validated using a numerical model of a steel truss bridge.The results show that under the influence of noise and modeling uncertainty,the damage identification method based on the DBN can identify the accurate damage location and degree identification compared with the traditional damage identification method based on an artificial neural network.

Heart Disease Classification Using Multiple K-PCA and Hybrid Deep Learning Approach

One of the severe health problems and the most common types of heartdisease (HD) is Coronary heart disease (CHD). Due to the lack of a healthy lifestyle, HD would cause frequent mortality worldwide. If the heart attack occurswithout any symptoms, it cannot be cured by an intelligent detection system.An effective diagnosis and detection of CHD should prevent human casualties.Moreover, intelligent systems employ clinical-based decision support approachesto assist physicians in providing another option for diagnosing and detecting HD.This paper aims to introduce a heart disease prediction model including phaseslike (i) Feature extraction, (ii) Feature selection, and (iii) Classification. At first,the feature extraction process is carried out, where the features like a time-domainindex, frequency-domain index, geometrical domain features, nonlinear features,WT features, signal energy, skewness, entropy, kurtosis features are extractedfrom the input ECG signal. The curse of dimensionality becomes a severe issue.This paper provides the solution for this issue by introducing a new ModifiedPrincipal Component Analysis known as Multiple Kernel-based PCA for dimensionality reduction. Furthermore, the dimensionally reduced feature set is thensubjected to a classification process, where the hybrid classifier combining bothRecurrent Neural Network (RNN) and Restricted Boltzmann Machine (RBM)is used. At last, the performance analysis of the adopted scheme is compared overother existing schemes in terms of specific measures.

Cumulative gain and lift charts for model performance assessment in mineral potential mapping

Model performance assessment is a key procedure for mineral potential mapping, but the correspond-ing research achievements are seldom reported in literature. Cumulative gain and lift charts are well known in the data mining community specialized in marketing and sales applications and widely used in customer chum prediction for model performance assessment. In this paper, they are introduced into the field of mineral poten-tial mapping for model performance assessment. These two charts can be viewed as a graphic representation of the advantage of using a predictive model to choose mineral targets. A cumulative gain curve can represent how much a predictive model is superior to a random guess in mineral target prediction. A lift chart can express how much more likely the mineral targets predicted by a model are deposit-bearing ones than those by a random se-lection. As an illustration, the cumulative gain and lift charts are applied to measure the performance of weights of evidence, logistic regression,restricted Boltzmann machine, and multilayer perceptron in mineral potential mapping in the Altay district in northern Xinjiang in China. The results show that the cumulative gain and lift charts can visually reveal that the first three models perform well while the last one performs poorly. Thus, the cumulative gain and lift charts can serve as a graphic tool for model performance assessment in mineral potential mapping.

Boltzmann Machine Method forSolving Mixed Integer Bilevel Programming Problem

This paper presents the mathematical model of mixed integer bilevel programming problem. The properties of the problem are discussed. An algorithm which combines the Boltzmann machine and two-level iterative computation is also proposed to solve it. It has been shown by the initial simulated results that the algorithm is efficient.

Post-COVID effect on heart after recovery based on hybrid EfficientNet-DBN with multilevel classification using ECG images

The highly contagious and dangerous virus known as COVID-19 is caused by severe acute respiratory syndrome(SARS),which is disseminated worldwide.Heart disease is a major cause of death worldwide.One year after the COVID-19 pandemic,the risk of heart issues is considerable.Such heart issues involve irregular heartbeats,heart failure,which is the inability of the heart to pump correctly,and coronary disease,which builds up in the arteries and causes restriction in blood flow,heart attacks,etc.Therefore,classifying this disease at an early stage is crucial.Hence,post-COVID effect on the heart after recovery with multilevel classification was performed using the hybrid EfficientNet+Deep Belief Network(EfficientNet+DBN)designed in this study.The input image obtained from the dataset was delivered for binary image conversion and then allowed for feature extraction.Subsequently,first-level disease classification was performed using the hybrid EfficientNet+DBN to detect whether the disease was in normal or abnormal conditions.If it is categorized as abnormal,a second-level classification is performed using EfficientNet-DBN,which classifies myocardial infarction and COVID-19 patients.The Pearson's correlation coefficient was used for the post-COVID correlation study.The experimental outcome showed that EfficientNet-DBN attained a maximum accuracy of 89.80%,sensitivity of 94.70%,and specificity of 91.80%.

Velocity forecasts using a combined deep learning model in hybrid electric vehicles with V2V and V2I communication

Vehicle velocity forecast is an important clue in improving the performance of energy management in hybrid electric vehicles(HEV). This paper presents a new combined model for predicting vehicle’s velocity time series. The main features of the model are to combine the feature extraction capability of deep restricted Boltzmann machines(DBM) and sequence pattern predicting capability of bidirectional long short-term memory(BLSTM). Hence, the model is named as DBMBLSTM. In addition, the DRMBLSTM model utilizes the vehicle driving information and roadside infrastructure information provided respectively through vehicle-to-vehicle(V2V) and vehicle-to-infrastructure(V2I) communication channels to predict vehicle velocity at various length of prediction horizon. Furthermore, the predictions results of this study are compared with the state of the art of vehicle velocity forecasts. The root mean square error(RMSE) is used as an evaluation criteria of predictions accuracy. Finally,these compared prediction model are applied in model predictive control(MPC) energy management strategy for the verifications of fuel economy improvement of a HEV. Simulation results confirm that the proposed combined deep learning model performs better than other five prediction methods. Therefore, it is a means of arriving at a reliable forecast model for HEV.