Sustainable Investment Forecasting of Power Grids Based on theDeep Restricted Boltzmann Machine Optimized by the Lion Algorithm

This paper proposes a new power grid investment prediction model based on the deep restricted Boltzmann machine(DRBM)optimized by the Lion algorithm(LA).Firstly,two factors including transmission and distribution price reform(TDPR)and 5G station construction were comprehensively incorporated into the consideration of influencing factors,and the fuzzy threshold method was used to screen out critical influencing factors.Then,the LA was used to optimize the parameters of the DRBM model to improve the model’s prediction accuracy,and the model was trained with the selected influencing factors and investment.Finally,the LA-DRBM model was used to predict the investment of a power grid enterprise,and the final prediction result was obtained by modifying the initial result with the modifying factors.The LA-DRBMmodel compensates for the deficiency of the singlemodel,and greatly improves the investment prediction accuracy of the power grid.In this study,a power grid enterprise was taken as an example to carry out an empirical analysis to prove the validity of the model,and a comparison with the RBM,support vector machine(SVM),back propagation neural network(BPNN),and regression model was conducted to verify the superiority of the model.The conclusion indicates that the proposed model has a strong generalization ability and good robustness,is able to abstract the combination of low-level features into high-level features,and can improve the efficiency of the model’s calculations for investment prediction of power grid enterprises.

FPGA Implementation of a Scalable and Highly Parallel Architecture for Restricted Boltzmann Machines

Restricted Boltzmann Machines (RBMs) are an effective model for machine learning;however, they require a significant amount of processing time. In this study, we propose a highly parallel, highly flexible architecture that combines small and completely parallel RBMs. This proposal addresses problems associated with calculation speed and exponential increases in circuit scale. We show that this architecture can optionally respond to the trade-offs between these two problems. Furthermore, our FPGA implementation performs at a 134 times processing speed up factor with respect to a conventional CPU.

Intrusion detection model based on deep belief nets

This paper focuses on the intrusion classification of huge amounts of data in a network intrusion detection system. An intrusion detection model based on deep belief nets （DBN） is proposed to conduct intrusion detection,and the principles regarding DBN are discussed.The DBN is composed of a multiple unsupervised restricted Boltzmann machine （RBM） and a supervised back propagation （BP）network.First,the DBN in the proposed model is pre-trained in a fast and greedy way,and each RBM is trained by the contrastive divergence algorithm.Secondly,the whole network is fine-tuned by the supervised BP algorithm,which is employed for classifying the low-dimensional features of the intrusion data generated by the last RBM layer simultaneously.The experimental results on the KDD CUP 1999 dataset demonstrate that the DBN using the RBM network with three or more layers outperforms the self-organizing maps （SOM）and neural network （NN）in intrusion classification.Therefore,the DBN is an efficient approach for intrusion detection in high-dimensional space.

融合谱-空域信息的DBM高光谱图像分类方法

在高光谱图像分类问题中,提取能够有效表达地物特征的信息是分类方法中的关键问题。为了提高高光谱图像分类精度,提出一种基于深度玻尔兹曼机的高光谱图像分类方法。该方法首先对高光谱图像数据进行主成分分析法白化处理,并提取像元的空域信息,与像元光谱信息组成综合的谱-空域信息;然后通过多层深度玻尔兹曼机模型从像元的谱-空域信息中提取深层次类别特征;最后通过逻辑回归模型对所提取特征进行分类。这种深度玻尔兹曼机模型能够利用数据的先验知识对高维数据进行特征提取,并且所提取的特征内在地表示了地物的空间结构和光谱特征。实验结果表明,这种方法能够有效地提高高光谱图像的分类精度。

受限制Boltzmann机深度置信网络与手写数字识别

手写数字因为每个人的书写习惯不同而差异很大。手写数字快速有效的识别一直是图像识别领域的热门话题。提出了一种基于受限制Boltzmann机组成的深度置信网络应用于手写数字识别的算法。对训练样本图片中的手写数字进行定位,分块,计算每块的特征值;用由受限制Boltzmann机组成的深度置信网络对样本特征值进行无监督学习;使用训练好的神经网络对待测样本进行深度特征提取。结果表明:该算法可行,相对于浅层学习,深度学习在图像特征提取上有了很大提高。

DBM-ELM深层网络模型

使用深层限制波尔兹曼机实现高维数据非线性降维,再结合极速学习机算法,提出了一种复合的DBMELM深层网络模型.该模型在复杂高维数据的分类问题上,能较好的将高维数据简化到低维空间,进而得到较好的分类效果,实现复杂函数的表示.最后在人脸和手写数字识别实验上得到了很好的证明.

基于MQDF-DBM模型的脱机手写体汉字识别

针对脱机手写体汉字识别准确率较低的问题,提出一种基于修正的二次判别函数(Modified Quadratic Discriminant Function,MQDF)与深度玻尔兹曼机(Deep Boltzmann Machine,DBM)的分类器级联模型。该模型的主要思想是MQDF和DBM在特征提取和分类机制上可以相辅相成。先用MQDF进行识别并得出结果,同时计算该结果的一个广义置信度。若置信度满足要求,则将识别结果作为最终结果输出,否则结合DBM进行二次识别,得到最终识别结果。实验结果表明,使用MQDF-DBM模型可以获得比单独使用MQDF和DBM模型更高的识别准确率,且识别速度比DBM更快。

基于先验知识与DBM采样的类不平衡用电数据分类方法

智能电网建设过程中现有客户标签体系不够完善,针对海量用户用电数据的分类管理中带有标签的样本数据量小以及类不平衡分布的问题,提出了一种基于先验知识与深度玻尔兹曼机(DBM)采样的不平衡用电数据分类方法。首先,提取负荷曲线的特征,建立采样原则,利用先验知识和DBM对负荷曲线进行采样。然后,将采样数据通过极限学习机(ELM)网络进行训练。最后以爱尔兰用户用电数据为数据源,通过与原始非采样、随机过采样、合成少数类过采样技术(SMOTE)的对比性实验分析结果表明,所提出的基于先验知识与DBM采样的不平衡用电数据分类方法能够更好地对类不平衡用电数据集进行分类,实现用户用电行为的分析,有效支撑用户侧错峰避峰工作。

基于DBM的电力投诉工单分类的应用研究

提出了基于深度玻尔兹曼机的电力投诉工单识别分类模型。首先对投诉工单数据进行数据清洗,对处理后的数据使用结巴分词算法进行分词并制作字典,再使用词袋模型对所分词向量化处理提取文本特征。进一步地,通过TF-IDF算法找出关键词以及余弦相似度计算训练、测试文档间的相似度;最后使用深度玻尔兹曼机对投诉工单进行分类。实验证明,分类的准确度达到80%,有效地缓解电力部门的工作压力,提高工作效率。

An Acoustic Events Recognition for Robotic Systems Based on a Deep Learning Method

In this paper, we provide a new approach to classify and recognize the acoustic events for multiple autonomous robots systems based on the deep learning mechanisms. For disaster response robotic systems, recognizing certain acoustic events in the noisy environment is very effective to perform a given operation. As a new approach, trained deep learning networks which are constructed by RBMs, classify the acoustic events from input waveform signals. From the experimental results, usefulness of our approach is discussed and verified.

Damage identification of steel truss bridges based on deep belief network

To improve the accuracy and anti-noise ability of the structural damage identification method,a bridge damage identification method is proposed based on a deep belief network(DBN).The output vector is used to establish the nonlinear mapping relationship between the mode shape and structural damage.The hidden layer of the DBN is trained through a layer-by-layer pre-training.Finally,the backpropagation algorithm is used to fine-tune the entire network.The method is validated using a numerical model of a steel truss bridge.The results show that under the influence of noise and modeling uncertainty,the damage identification method based on the DBN can identify the accurate damage location and degree identification compared with the traditional damage identification method based on an artificial neural network.

Completeness Problem of the Deep Neural Networks

Hornik, Stinchcombe & White have shown that the multilayer feed forward networks with enough hidden layers are universal approximators. Roux & Bengio have proved that adding hidden units yield a strictly improved modeling power, and Restricted Boltzmann Machines (RBM) are universal approximators of discrete distributions. In this paper, we provide yet another proof. The advantage of this new proof is that it will lead to several new learning algorithms. We prove that the Deep Neural Networks implement an expansion and the expansion is complete. First, we briefly review the basic Boltzmann Machine and that the invariant distributions of the Boltzmann Machine generate Markov chains. We then review the θ-transformation and its completeness, i.e. any function can be expanded by θ-transformation. We further review ABM (Attrasoft Boltzmann Machine). The invariant distribution of the ABM is a θ-transformation;therefore, an ABM can simulate any distribution. We discuss how to convert an ABM into a Deep Neural Network. Finally, by establishing the equivalence between an ABM and the Deep Neural Network, we prove that the Deep Neural Network is complete.

受限玻尔兹曼机及其变体研究综述

受限玻尔兹曼机作为学习数据分布和提取内在特征的典型概率图模型,是深度学习领域重要的基础模型。近年来,通过改进受限玻尔兹曼机的模型结构和能量函数得到众多新兴模型,即受限玻尔兹曼机变体,可以进一步提升模型的特征提取性能。研究受限玻尔兹曼机及其变体能够显著促进深度学习领域的发展,实现大数据时代海量信息的快速提取。基于此,对近年来受限玻尔兹曼机及其变体的相关研究进行系统回顾,并创新性地从训练算法改进、模型结构改进、模型深层融合研究和模型相关最新应用4个方面进行全面综述。其中,重点梳理受限玻尔兹曼机训练算法和变体模型的发展史。最后,讨论受限玻尔兹曼机及其变体领域的现存难点与挑战,对主要研究工作进行总结与展望。

基于DBN网络的滚动轴承故障诊断

为了提高滚动轴承故障诊断的准确率,提出基于DBN网络的滚动轴承故障诊断方法。针对浅层神经网络难以从振动信号中提取深层故障特征且易陷入维度灾难等技术难点,结合深度置信网络(DBN)能够处理高维非线性数据和有效提取故障特征的特点,建立基于DBN网络的滚动轴承故障诊断模型。通过验证分析,确定了DBN的隐含层层数、最佳数据类型、激活函数等网络参数,为DBN网络参数的设置提供一种新的方法与思路。并对受限玻尔兹曼机(RBM)的重构能力进行了验证。将DBN网络与BP、ELM、PNN等浅层神经网络进行了对比分析,结果表明DBN网络具有较高的诊断精度与较强的稳定性,证明了DBN网络在滚动轴承故障诊断中的有效性。

A new item-based deep network structure using a restricted Boltzmann machine for collaborative filtering

The collaborative filtering(CF) technique has been widely used recently in recommendation systems. It needs historical data to give predictions. However, the data sparsity problem still exists. We propose a new item-based restricted Boltzmann machine(RBM) approach for CF and use the deep multilayer RBM network structure, which alleviates the data sparsity problem and has excellent ability to extract features. Each item is treated as a single RBM, and different items share the same weights and biases. The parameters are learned layer by layer in the deep network. The batch gradient descent algorithm with minibatch is used to increase the convergence speed. The new feature vector discovered by the multilayer RBM network structure is very effective in predicting a rating and achieves a better result. Experimental results on the data set of MovieL ens show that the item-based multilayer RBM approach achieves the best performance, with a mean absolute error of 0.6424 and a root-mean-square error of 0.7843.

Emotion recognition from thermal infrared images using deep Boltzmann machine

Facial expression and emotion recognition from thermal infrared images has attracted more and more attentions in recent years. However, the features adopted in current work are either temperature statistical parameters extracted from the facial regions of interest or several hand-crafted features that are commonly used in visible spectrum. Till now there are no image features specially designed for thermal infrared images. In this paper, we propose using the deep Boltzmann machine to learn thermal features for emotion recognition from thermal infrared facial images. First, the face is located and normalized from the thermal infrared im- ages. Then, a deep Boltzmann machine model composed of two layers is trained. The parameters of the deep Boltzmann machine model are further fine-tuned for emotion recognition after pre-tralning of feature learning. Comparative experimental results on the NVIE database demonstrate that our approach outperforms other approaches using temperature statistic features or hand-crafted features borrowed from visible domain. The learned features from the forehead, eye, and mouth are more effective for discriminating valence dimension of emotion than other facial areas. In addition, our study shows that adding unlabeled data from other database during training can also improve feature learning performance.

基于MLP-DBN模型的构造煤分布预测策略分析

构造煤分布情况对煤矿开采活动的安全具有重要意义,为了实现构造煤分布情况的准确预测,以构造煤层的地震属性信息特点为依据,提出了基于深度置信网和多层感知器的构造煤分布预测模型。实验结果显示,多层感知器—深度置信网模型在噪声数据集和无噪声数据集中的拟合度分别为0.965、0.996。与其他模型相比,多层感知器—深度置信网模型平均决定系数和平均解释方差得分分别为0.963、0.87,均高于其他模型;平均均方误差和平均均方根误差分别为0.006、0.078,均低于其他模型。上述结果表明,基于MLP-DBN的构造煤分布预测模型能更准确地对构造煤分布情况进行预测,预测结果与实际情况的拟合度更高,为煤层瓦斯的超前治理提供了有力支持。

基于深度学习的太阳能电池片表面缺陷检测方法

目前对太阳能电池片的缺陷检测仍依赖人工完成,很难通过传统的CCD成像系统自动识别.作为一种多层神经网络学习算法,深度学习因对输入样本数据强大的特征提取能力而受到广泛关注.文中提出一种基于深度学习的太阳能电池片表面缺陷检测方法,该方法首先根据样本特征建立深度置信网络(DBN),并训练获取网络的初始权值;然后通过BP算法微调网络参数,取得训练样本到无缺陷模板之间的映射关系;最后利用重构图像与缺陷图像之间的对比关系,实现测试样本的缺陷检测.实验表明DBN能较好地建立上述映射关系,且准确、快速地进行缺陷检测.

深度学习研究综述

鉴于深度学习在学术界和工业界的重要性,依据数据流向对目前有代表性的深度学习算法进行归纳和总结,综述了不同类型深度网络的结构及特点.首先介绍了深度学习的概念;然后根据深度学习算法的结构特征,概述了前馈深度网络、反馈深度网络和双向深度网络3类主流深度学习算法的网络结构和训练方法;最后介绍了深度学习算法在不同数据处理中的最新应用现状及其发展趋势.可以看到:深度学习在不同应用领域都取得了明显的优势,但仍存在需要进一步探索的问题,如无标记数据的特征学习、网络模型规模与训练速度精度之间的权衡、与其他方法的融合等.

受限波尔兹曼机

受限波尔兹曼机(restricted Boltzmann machines,RBM)是一类具有两层结构、对称连接且无自反馈的随机神经网络模型,层间全连接,层内无连接.近年来,随着RBM的快速学习算法一对比散度的出现,机器学习界掀起了研究RBM理论及应用的热潮.实践表明,RBM是一种有效的特征提取方法,用于初始化前馈神经网络可明显提高泛化能力,堆叠多个RBM组成的深度信念网络能提取更抽象的特征.鉴于RBM的优点及其在深度学习中的广泛应用,本文对RBM的基本模型、学习算法、参数设置、评估方法、变形算法等进行了详细介绍,最后探讨了RBM在未来值得研究的方向.