Uncertainty quantification of predicting stable structures for high-entropy alloys using Bayesian neural networks

High entropy alloys(HEAs)have excellent application prospects in catalysis because of their rich components and configuration space.In this work,we develop a Bayesian neural network(BNN)based on energies calculated with density functional theory to search the configuration space of the CoNiRhRu HEA system.The BNN model was developed by considering six independent features of Co-Ni,Co-Rh,CoRu,Ni-Rh,Ni-Ru,and Rh-Ru in different shells and energies of structures as the labels.The root mean squared error of the energy predicted by BNN is 1.37 me V/atom.Moreover,the influence of feature periodicity on the energy of HEA in theoretical calculations is discussed.We found that when the neural network is optimized to a certain extent,only using the accuracy indicator of root mean square error to evaluate model performance is no longer accurate in some scenarios.More importantly,we reveal the importance of uncertainty quantification for neural networks to predict new structures of HEAs with proper confidence based on BNN.

Study of Deuteron Separation Energy Based on Bayesian Neural Network Approach

Deuteron separation energy is not only the basis for validating the nuclear mass models and nucleon-nucleon interaction potential,but also can determine the stability of a nuclide to certain extent.Bayesian neural network(BNN)approach,which has strong predictive power and can naturally give theoretical errors of predicted values,had been successfully applied to study the different kinds of separations except the deuteron separation.In this paper,several typical nuclear mass models,such as macroscopic model BW2,macroscopic-microscopic model WS4,and microscopic model HFB-31,are chosen to study the deuteron separation energy combining BNN approach.The root-mean-square deviations of these models are partly reduced.In addition,the inclusion of physical parameters related to the pair and shell effects in the input layer can further improve the theoretical accuracy for the deuteron separation energy.The results show that the theoretical predictions are more reliable as more physical features of BNN approach are included.

Nonlinear inversion of electrical resistivity imaging using pruning Bayesian neural networks

Conventional artificial neural networks used to solve electrical resistivity imaging （ERI） inversion problem suffer from overfitting and local minima. To solve these problems, we propose to use a pruning Bayesian neural network （PBNN） nonlinear inversion method and a sample design method based on the K-medoids clustering algorithm. In the sample design method, the training samples of the neural network are designed according to the prior information provided by the K-medoids clustering results; thus, the training process of the neural network is well guided. The proposed PBNN, based on Bayesian regularization, is used to select the hidden layer structure by assessing the effect of each hidden neuron to the inversion results. Then, the hyperparameter αk, which is based on the generalized mean, is chosen to guide the pruning process according to the prior distribution of the training samples under the small-sample condition. The proposed algorithm is more efficient than other common adaptive regularization methods in geophysics. The inversion of synthetic data and field data suggests that the proposed method suppresses the noise in the neural network training stage and enhances the generalization. The inversion results with the proposed method are better than those of the BPNN, RBFNN, and RRBFNN inversion methods as well as the conventional least squares inversion.

Improvement of the Bayesian neural network to study the photoneutron yield cross sections

This work is an attempt to improve the Bayesian neural network (BNN) for studying photoneutron yield cross sections as a function of the charge number Z, mass number A, and incident energy ε. The BNN was improved in terms of three aspects:numerical parameters, input layer, and network structure. First, by minimizing the deviations between the predictions and data, the numerical parameters, including the hidden layer number, hidden node number, and activation function, were selected. It was found that the BNN with three hidden layers, 10 hidden nodes, and sigmoid activation function provided the smallest deviations. Second, based on known knowledge,such as the isospin dependence and shape effect, the optimal ground-state properties were selected as input neurons. Third, the Lorentzian function was applied to map the hidden nodes to the output cross sections, and the empirical formula of the Lorentzian parameters was applied to link some of the input nodes to the output cross sections. It was found that the last two aspects improved the predictions and avoided overfitting, especially for the axially deformed nucleus.

Reinforcement Learning of Molecule Optimization with Bayesian Neural Networks

Creating new molecules with desired properties is a fundamental and challenging problem in chemistry. Reinforcement learning (RL) has shown its utility in this area where the target chemical property values can serve as a reward signal. At each step of making a new molecule, the RL agent learns selecting an action from a list of many chemically valid actions for a given molecule, implying a great uncertainty associated with its learning. In a traditional implementation of deep RL algorithms, deterministic neural networks are typically employed, thus allowing the agent to choose one action from one sampled action at each step. In this paper, we proposed a new strategy of applying Bayesian neural networks to RL to reduce uncertainty so that the agent can choose one action from a pool of sampled actions at each step, and investigated its benefits in molecule design. Our experiments suggested the Bayesian approach could create molecules of desirable chemical quality while maintained their diversity, a very difficult goal to achieve in machine learning of molecules. We further exploited their diversity by using them to train a generative model to yield more novel drug-like molecules, which were absent in the training molecules as we know novelty is essential for drug candidate molecules. In conclusion, Bayesian approach could offer a balance between exploitation and exploration in RL, and a balance between optimization and diversity in molecule design.

Uncertainties of nuclear level density estimated using Bayesian neural networks

Nuclear level density(NLD)is a critical parameter for understanding nuclear reactions and the structure of atomic nuclei;however,accurate estimation of NLD is challenging owing to limitations inherent in both experimental measurements and theoretical models.This paper presents a sophisticated approach using Bayesian neural networks(BNNs)to analyze NLD across a wide range of models.It uniquely incorporates the assessment of model uncertainties.The application of BNNs demonstrates remarkable success in accurately predicting NLD values when compared to recent experimental data,confirming the effectiveness of our methodology.The reliability and predictive power of the BNN approach not only validates its current application but also encourages its integration into future analyses of nuclear reaction cross sections.

Precise machine learning models for fragment production in projectile fragmentation reactions using Bayesian neural networks

Machine learning models are constructed to predict fragment production cross sections in projectile fragmentation(PF)reactions using Bayesian neural network(BNN)techniques.The massive learning for BNN models is based on 6393 fragments from 53 measured projectile fragmentation reactions.A direct BNN model and physical guiding BNN via FRACS parametrization(BNN+FRACS)model have been constructed to predict the fragment cross section in projectile fragmentation reactions.It is verified that the BNN and BNN+FRACS models can reproduce a wide range of fragment productions in PF reactions with incident energies from 40 MeV/u to 1 GeV/u,reaction systems with projectile nuclei from^40 Ar to^208 Pb,and various target nuclei.The high precision of the BNN and BNN+FRACS models makes them applicable for the low production rate of extremely rare isotopes in future PF reactions with large projectile nucleus asymmetry in the new generation of radioactive nuclear beam factories.

Magnetic moment predictions of odd-A nuclei with the Bayesian neural network approach

The Bayesian neural network approach has been employed to improve the nuclear magnetic moment predictions of odd-A nuclei.The Schmidt magnetic moment obtained from the extreme single-particle shell model makes large root-mean-square(rms)deviations from data,i.e.,0.949μN and 1.272μN for odd-neutron nuclei and odd-proton nuclei,respectively.By including the dependence of the nuclear spin and Schmidt magnetic moment,the machine-learning approach precisely describes the magnetic moments of odd-A uclei with rms deviations of 0.036μN for odd-neutron nuclei and 0.061μN for odd-proton nuclei.Furthermore,the evolution of magnetic moments along isotopic chains,including the staggering and sudden jump trend,which are difficult to describe using nuclear models,have been well reproduced by the Bayesian neural network(BNN)approach.The magnetic moments of doubly closed-shell±1 nuclei,for example,isoscalar and isovector magnetic moments,have been well studied and compared with the corresponding non-relativistic and relativistic calculations.

Neural Network-Based Performance Index Model for Enterprise Goals Simulation and Forecasting

Enterprise Information System management has become an increasingly vital factor for many firms. Several organizations have encountered problems when attempting to evaluate organizational performance. Measurement of performance metrics is a key challenge for a huge number of firms. In order to preserve relevance and adaptability in competitive markets, it has become essential to respond proactively to complex events through informed decision-making that is supported by technology. Therefore, the objective of this study was to apply neural networks to the modeling, simulation, and forecasting of the effects of the performance indicators of Enterprise Information Systems on the achievement of corporate objectives and value creation. A set of quantifiable and sizeable conditionally independent associations were derived using a simplified joint probability distribution technique. Bayesian Neural Networks were utilized to describe the link between random variables (features) and to concisely and easily specify the joint probability distribution. The research demonstrated that Bayesian networks could effectively explore complex logical linkages by employing probability to represent uncertainty and probabilistic rules;and by applying impact models from Bayesian taxonomies to achieve learning and reasoning processes.

Applying Bayesian neural networks to identify pion,kaon and proton in BESⅡ

The Monte-Carlo samples of pion, kaon and proton generated from 0.3 GeV/c to 1.2 GeV/c by the ‘tester＇ generator from SIMBES which are used to simulate the detector of BES Ⅱ are identified with the Bayesian neural networks （BNN）. The pion identification and misidentification efficiencies are obviously better at high momentum region using BNN than the methods of χ^2 analysis of dE/dX and TOF information. The kaon identification and misidentification efficiencies are obviously better from 0.3 GeV/c to 1.2 GeV/c using BNN than the methods of X2 analysis. The proton identification and misidentification efficiencies using BNN are basically consistent with the ones of χ^2 analysis. The anti-proton identification and misidentification efficiencies are better below 0.6 GeV/c using BNN than the methods of χ^2 analysis.

Separable Shadow Hamiltonian Hybrid Monte Carlo for Bayesian Neural Network Inference in wind speed forecasting

Accurate wind speed and consequently wind power forecasts form a critical enabling tool for large scale wind energy adoption.Probabilistic machine learning models such as Bayesian Neural Network(BNN)models are often preferred in the forecasting task as they facilitate estimates of predictive uncertainty and automatic relevance determination(ARD).Hybrid Monte Carlo(HMC)is widely used to perform asymptotically exact inference of the network parameters.A significant limitation to the increased adoption of HMC in inference for large scale machine learning systems is the exponential degradation of the acceptance rates and the corresponding effective sample sizes with increasing model dimensionality due to numerical integration errors.This paper presents a solution to this problem by sampling from a modified or shadow Hamiltonian that is conserved to a higher-order by the leapfrog integrator.BNNs trained using Separable Shadow Hamiltonian Hybrid Monte Carlo(S2HMC)are used to forecast one hour ahead wind speeds on the Wind Atlas for South Africa(WASA)datasets.Experimental results find that S2HMC yields higher effective sample sizes than the competing HMC.The predictive performance of S2HMC and HMC based BNNs is found to be similar.We further perform hierarchical inference for BNN parameters by combining the S2HMC sampler with Gibbs sampling of hyperparameters for relevance determination.A generalisable ARD committee framework is introduced to synthesise the various sampler ARD outputs into robust feature selections.Experimental results show that this ARD committee approach selects features of high predictive information value.Further,the results show that dimensionality reduction performed through this approach improves the sampling performance of samplers that suffer from random walk behaviour such as Metropolis–Hastings(MH).

Application of Bayesian regularized BP neural network model for analysis of aquatic ecological data—A case study of chlorophyll-a prediction in Nanzui water area of Dongting Lake

Bayesian regularized BP neural network（BRBPNN） technique was applied in the chlorophyll-α prediction of Nanzui water area in Dongting Lake. Through BP network interpolation method, the input and output samples of the network were obtained. After the selection of input variables using stepwise/multiple linear regression method in SPSS i1.0 software, the BRBPNN model was established between chlorophyll-α and environmental parameters, biological parameters. The achieved optimal network structure was 3-11-1 with the correlation coefficients and the mean square errors for the training set and the test set as 0.999 and 0.000？8426, 0.981 and 0.0216 respectively. The sum of square weights between each input neuron and the hidden layer of optimal BRBPNN models of different structures indicated that the effect of individual input parameter on chlorophyll- α declined in the order of alga amount 〉 secchi disc depth（SD） 〉 electrical conductivity （EC）. Additionally, it also demonstrated that the contributions of these three factors were the maximal for the change of chlorophyll-α concentration, total phosphorus（TP） and total nitrogen（TN） were the minimal. All the results showed that BRBPNN model was capable of automated regularization parameter selection and thus it may ensure the excellent generation ability and robustness. Thus, this study laid the foundation for the application of BRBPNN model in the analysis of aquatic ecological data（chlorophyll-α prediction） and the explanation about the effective eutrophication treatment measures for Nanzui water area in Dongting Lake.

ECGAN:Translate Real World to Cartoon Style Using Enhanced Cartoon Generative Adversarial Network

Visual illustration transformation from real-world to cartoon images is one of the famous and challenging tasks in computer vision.Image-to-image translation from real-world to cartoon domains poses issues such as a lack of paired training samples,lack of good image translation,low feature extraction from the previous domain images,and lack of high-quality image translation from the traditional generator algorithms.To solve the above-mentioned issues,paired independent model,high-quality dataset,Bayesian-based feature extractor,and an improved generator must be proposed.In this study,we propose a high-quality dataset to reduce the effect of paired training samples on the model’s performance.We use a Bayesian Very Deep Convolutional Network(VGG)-based feature extractor to improve the performance of the standard feature extractor because Bayesian inference regu-larizes weights well.The generator from the Cartoon Generative Adversarial Network(GAN)is modified by introducing a depthwise convolution layer and channel attention mechanism to improve the performance of the original generator.We have used the Fréchet inception distance(FID)score and user preference score to evaluate the performance of the model.The FID scores obtained for the generated cartoon and real-world images are 107 and 76 for the TCC style,and 137 and 57 for the Hayao style,respectively.User preference score is also calculated to evaluate the quality of generated images and our proposed model acquired a high preference score compared to other models.We achieved stunning results in producing high-quality cartoon images,demonstrating the proposed model’s effectiveness in transferring style between authentic images and cartoon images.

Comparative efficacy and safety of cognitive enhancers for treating vascular cognitive impairment: systematic review and Bayesian network meta-analysis

Objective: To assess and compare the clinical efficacy and safety of cognitive enhancers(donepezil, galantamine, rivastigmine, and memantine) on cognition, behavior, function, and global status in patients with vascular cognitive impairment.Data sources: The initial literature search was performed with PubMed, EMBASE, the Cochrane Methodology Register, the Cochrane Central Register of Controlled Trials, and Cumulative Index to Nursing & Allied Health(CINAHL) from inception to January 2018 for studies regarding donepezil, galantamine, rivastigmine, and memantine for treatment of vascular cognitive impairment.Data selection: Randomized controlled trials on donepezil, galantamine, rivastigmine, and memantine as monotherapy in the treatment of vascular cognitive impairment were included. A Bayesian network meta-analysis was conducted. Outcome measures: Efficacy was assessed by changes in scores of the Alzheimer's Disease Assessment Scale, cognitive subscale, Mini-Mental State Examination, Neuropsychiatric Inventory scores and Clinician's Interview-Based Impression of Change Scale Plus Caregiver's Input, Activities of Daily Living, the Clinical Dementia Rating scale. Safety was evaluated by mortality, total adverse events(TAEs), serious adverse events(SAEs), nausea, vomiting. diarrhea, or cerebrovascular accidents(CVAs). Results: After screening 1717 citations, 12 randomized controlled trials were included. Donepezil and rivastigmine(mean difference(e) = –0.77, 95% confidence interval(CI): 0.25–1.32; MD = 1.05, 95% CI: 0.18–1.79) were significantly more effective than placebo in reducing Mini-Mental State Examination scores. Donepezil, galantamine, and memantine(MD = –1.30, 95% CI: –2.27 to –0.42; MD = –1.67, 95% CI: –3.36 to –0.06; MD = –2.27, 95% CI: –3.91 to –0.53) showed superior benefits on the Alzheimer's Disease Assessment Scale–cognitive scores compared with placebo. Memantine(MD = 2.71, 95% CI: 1.05–7.29) improved global status(Clinician's Interview-Based Impression of Change Scale Plus Caregiver's Input) more than the placebo. Safety results revealed that donepezil 10 mg(odds ratio(OR) = 3.04, 95% CI: 1.86–5.41) contributed to higer risk of adverse events than placebo. Galantamine(OR = 5.64, 95% CI: 1.31–26.71) increased the risk of nausea. Rivastigmine(OR = 16.80, 95% CI: 1.78–319.26) increased the risk of vomiting. No agents displayed a significant risk of serious adverse events, mortality, cerebrovascular accidents, or diarrhea.Conclusion: We found significant efficacy of donepezil, galantamine, and memantine on cognition. Memantine can provide significant efficacy in global status. They are all safe and well tolerated.

Winning Probability Estimation Based on Improved Bradley-Terry Model and Bayesian Network for Aircraft Carrier Battle

To provide a decision-making aid for aircraft carrier battle,the winning probability estimation based on Bradley-Terry model and Bayesian network is presented. Firstly,the armed forces units of aircraft carrier are classified into three types,which are aircraft,ship and submarine. Then,the attack ability value and defense ability value for each type of armed forces are estimated by using BP neural network,whose training results of sample data are consistent with the estimation results. Next,compared the assessment values through an improved Bradley-Terry model and constructed a Bayesian network to do the global assessment,the winning probabilities of both combat sides are obtained. Finally,the winning probability estimation for a navy battle is given to illustrate the validity of the proposed scheme.

Time Series Forecasting with Multiple Deep Learners: Selection from a Bayesian Network

Considering the recent developments in deep learning, it has become increasingly important to verify what methods are valid for the prediction of multivariate time-series data. In this study, we propose a novel method of time-series prediction employing multiple deep learners combined with a Bayesian network where training data is divided into clusters using K-means clustering. We decided how many clusters are the best for K-means with the Bayesian information criteria. Depending on each cluster, the multiple deep learners are trained. We used three types of deep learners: deep neural network (DNN), recurrent neural network (RNN), and long short-term memory (LSTM). A naive Bayes classifier is used to determine which deep learner is in charge of predicting a particular time-series. Our proposed method will be applied to a set of financial time-series data, the Nikkei Average Stock price, to assess the accuracy of the predictions made. Compared with the conventional method of employing a single deep learner to acquire all the data, it is demonstrated by our proposed method that F-value and accuracy are improved.

Using Radial Neural Network to Predict the Ultimate Moment of a Reinforced Concrete Beam Reinforced with Composites

This article is intended as a proposal for a numerical model for the prediction of the ultimate moment of a reinforced concrete beam reinforced with composite materials based on neural networks, which are classified in the artificial intelligence method. In this work, a RBF network or radial basis function type model was created and tested. The validation of the RBF architecture consists in judging its predictive capacity by using the weights and biases computed during the training, to apply them to another database which did not participate to the training and testing of the model. So, with Bayesian regularization, a maximum error of 0.0813 Tm in absolute value was found between the targets and predicted outputs. The value of the mean square error MSE = 1.1106 * 10-4 allowed us to quantify and justify the prediction performance of this network. Through this article, RBF network model was justified perform and can be used and exploited by our engineers with a high reliability rate.

基于贝叶斯单源域领域泛化算法的天然气管道故障智能诊断

基于深度学习算法的故障智能诊断模型已被广泛应用于天然气管道运输安全领域,然而管道通常处于准稳态,使得训练集中的故障样本量受限。为此,针对天然气管道故障诊断中因训练集故障样本量有限,导致难以准确诊断的问题,提出了一种基于贝叶斯单源域领域泛化(BSDG)算法,部署了一种攻击防御策略,通过在攻击阶段明确伪目标域增强路径,并在防御阶段引导模型参数的后验分布向伪域样本得分更高的方向调整,增强模型在面对不同域扰动时的适应性和鲁棒性。研究结果表明:(1)基于贝叶斯网络建立的非定向攻击模型确保伪域样本既保留了与源域的相关性,又引入了足够的域差异来模拟潜在的目标域,由此提升了多源域和单源域设置下的领域泛化诊断准确率;(2)测试结果显示,BSDG算法在多源域泛化任务及两项单源域泛化任务中,相较于性能最优的对比算法,其准确率分别提高了9.79%、5.09%和27.98%;(3)裕度差异损失通过在学习决策边界的过程中引入不确定性,令分类器可以灵活且有效应对频繁的分布变化,显著性测试结果表明BSDG算法在多数场景下显著优于先进对比算法;(4)贝叶斯神经网络通过在权重上引入不确定性,有效提升了BSDG算法的泛化稳定性。结论认为,BSDG算法通过使用基于贝叶斯推理的攻击防御策略,有效扩展了源域模型的决策边界,解决了实际场景数据匮乏导致的深度神经网络泛化能力差的问题,为样本受限情形下的天然气管道故障诊断模型设计提供了理论支撑。

一种贝叶斯网络的卫星姿态系统故障诊断方法

姿态控制系统是卫星系统中重要的组成部分,由于其高昂的造价,发生故障会引发恶劣的影响。随着航天科技的发展,卫星姿态控制系统也逐渐复杂,其可能发生故障的概率也随之增大。针对传统神经网络故障诊断结果缺少置信度、鲁棒性较差以及易发生过拟合的缺点,在对贝叶斯统计和深度学习理论研究的基础上,提出了一种基于贝叶斯线性层与贝叶斯卷积层的Bayesian Le Net结合的网络模型。通过对卫星姿态控制系统飞轮部件的故障数据分析和处理,进而采用该模型对故障仿真,并与贝叶斯全连接神经网络与传统Le Net进行对比,实验结果表明:在飞轮可能发生的三种故障前提下,上述网络模型准确率较高,过拟合现象较轻。验证了上述网络模型的有效性。