A Short-Term Climate Prediction Model Based on a Modular Fuzzy Neural Network

In terms of the modular fuzzy neural network (MFNN) combining fuzzy c-mean (FCM) cluster and single-layer neural network, a short-term climate prediction model is developed. It is found from modeling results that the MFNN model for short-term climate prediction has advantages of simple structure, no hidden layer and stable network parameters because of the assembling of sound functions of the self-adaptive learning, association and fuzzy information processing of fuzzy mathematics and neural network methods. The case computational results of Guangxi flood season (JJA) rainfall show that the mean absolute error (MAE) and mean relative error (MRE) of the prediction during 1998-2002 are 68.8 mm and 9.78%, and in comparison with the regression method, under the conditions of the same predictors and period they are 97.8 mm and 12.28% respectively. Furthermore, it is also found from the stability analysis of the modular model that the change of the prediction results of independent samples with training times in the stably convergent interval of the model is less than 1.3 mm. The obvious oscillation phenomenon of prediction results with training times, such as in the common back-propagation neural network (BPNN) model, does not occur, indicating a better practical application potential of the MFNN model.

Conditional Random Field Tracking Model Based on a Visual Long Short Term Memory Network

In dense pedestrian tracking,frequent object occlusions and close distances between objects cause difficulty when accurately estimating object trajectories.In this study,a conditional random field tracking model is established by using a visual long short term memory network in the three-dimensional(3D)space and the motion estimations jointly performed on object trajectory segments.Object visual field information is added to the long short term memory network to improve the accuracy of the motion related object pair selection and motion estimation.To address the uncertainty of the length and interval of trajectory segments,a multimode long short term memory network is proposed for the object motion estimation.The tracking performance is evaluated using the PETS2009 dataset.The experimental results show that the proposed method achieves better performance than the tracking methods based on the independent motion estimation.

Decision Technique of Solar Radiation Prediction Applying Recurrent Neural Network for Short-Term Ahead Power Output of Photovoltaic System

In recent years, introduction of a renewable energy source such as solar energy is expected. However, solar radiation is not constant and power output of photovoltaic (PV) system is influenced by weather conditions. It is difficult for getting to know accurate power output of PV system. In order to forecast the power output of PV system as accurate as possible, this paper proposes a decision technique of forecasting model for short-term-ahead power output of PV system based on solar radiation prediction. Application of Recurrent Neural Network (RNN) is shown for solar radiation prediction in this paper. The proposed method in this paper does not require complicated calculation, but mathematical model with only useful weather data. The validity of the proposed RNN is confirmed by comparing simulation results of solar radiation forecasting with that obtained from other

Estimation of unloading relaxation depth of Baihetan Arch Dam foundation using long-short term memory network

The unloading relaxation caused by excavation for construction of high arch dams is an important factor influencing the foundation’s integrity and strength.To evaluate the degree of unloading relaxation,the long-short term memory(LSTM)network was used to estimate the depth of unloading relaxation zones on the left bank foundation of the Baihetan Arch Dam.Principal component analysis indicates that rock charac-teristics,the structural plane,the protection layer,lithology,and time are the main factors.The LSTM network results demonstrate the unloading relaxation characteristics of the left bank,and the relationships with the factors were also analyzed.The structural plane has the most significant influence on the distribution of unloading relaxation zones.Compared with massive basalt,the columnar jointed basalt experiences a more significant unloading relaxation phenomenon with a clear time effect,with the average unloading relaxation period being 50 d.The protection layer can effectively reduce the unloading relaxation depth by approximately 20%.

CONVERGENCE OF ONLINE GRADIENT METHOD WITH A PENALTY TERM FOR FEEDFORWARD NEURAL NETWORKS WITH STOCHASTIC INPUTS

Online gradient algorithm has been widely used as a learning algorithm for feedforward neural network training. In this paper, we prove a weak convergence theorem of an online gradient algorithm with a penalty term, assuming that the training examples are input in a stochastic way. The monotonicity of the error function in the iteration and the boundedness of the weight are both guaranteed. We also present a numerical experiment to support our results.

Deep Learning Network for Energy Storage Scheduling in Power Market Environment Short-Term Load Forecasting Model

In the electricity market,fluctuations in real-time prices are unstable,and changes in short-term load are determined by many factors.By studying the timing of charging and discharging,as well as the economic benefits of energy storage in the process of participating in the power market,this paper takes energy storage scheduling as merely one factor affecting short-term power load,which affects short-term load time series along with time-of-use price,holidays,and temperature.A deep learning network is used to predict the short-term load,a convolutional neural network(CNN)is used to extract the features,and a long short-term memory(LSTM)network is used to learn the temporal characteristics of the load value,which can effectively improve prediction accuracy.Taking the load data of a certain region as an example,the CNN-LSTM prediction model is compared with the single LSTM prediction model.The experimental results show that the CNN-LSTM deep learning network with the participation of energy storage in dispatching can have high prediction accuracy for short-term power load forecasting.

Term Structure of Interest Rates Based on Artificial Neural Network

In light of the nonlinear approaching capability of artificial neural networks （ ANN）, the term structure of interest rates is predicted using The generalized regression neural network （GRNN） and back propagation （BP） neural networks models. The prediction performance is measured with US interest rate data. Then, RBF and BP models are compared with Vasicek＇s model and Cox-Ingersoll-Ross （CIR） model. The comparison reveals that neural network models outperform Vasicek＇s model and CIR model, which are more precise and closer to the real market situation.

Long Short-Term Memory Recurrent Neural Network-Based Acoustic Model Using Connectionist Temporal Classification on a Large-Scale Training Corpus

A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.

Online Gradient Methods with a Punishing Term for Neural Networks

Online gradient methods are widely used for training the weight of neural networks and for other engineering computations. In certain cases, the resulting weight may become very large, causing difficulties in the implementation of the network by electronic circuits. In this paper we introduce a punishing term into the error function of the training procedure to prevent this situation. The corresponding convergence of the iterative training procedure and the boundedness of the weight sequence are proved. A supporting numerical example is also provided.

结合太阳辐射量计算与CNN-LSTM组合的光伏功率预测方法研究

为了提高模型预测性能,提出一种综合太阳辐射模型及深度学习的光伏功率预测模型。首先,利用太阳辐射机理建立太阳辐射模型(SRM),估算出水平面上总辐射值,再由斜面辐照度转换方法计算出光伏组件所接收的斜面辐射值。其次,通过皮尔逊相关分析法筛选出对光伏功率影响较大的主要因素,将斜面辐射计算值及主要影响因素作为输入,采用卷积神经网络(CNN)和长短期记忆网络(LSTM)建立光伏功率SRM-CNN-LSTM预测模型。分别利用春夏秋冬四季典型日的数据开展对比实验,结果表明:与几种其他方法相比,该文方法具有更好的预测效果。

Application of preoperative artificial neural network based on blood biomarkers and clinicopathological parameters for predicting longterm survival of patients with gastric cancer

BACKGROUND Because of the powerful abilities of self-learning and handling complex biological information,artificial neural network(ANN)models have been widely applied to disease diagnosis,imaging analysis,and prognosis prediction.However,there has been no trained preoperative ANN(preope-ANN)model to preoperatively predict the prognosis of patients with gastric cancer(GC).AIM To establish a neural network model that can predict long-term survival of GC patients before surgery to evaluate the tumor condition before the operation.METHODS The clinicopathological data of 1608 GC patients treated from January 2011 to April 2015 at the Department of Gastric Surgery,Fujian Medical University Union Hospital were analyzed retrospectively.The patients were randomly divided into a training set(70%)for establishing a preope-ANN model and a testing set(30%).The prognostic evaluation ability of the preope-ANN model was compared with that of the American Joint Commission on Cancer(8th edition)clinical TNM(cTNM)and pathological TNM(pTNM)staging through the receiver operating characteristic curve,Akaike information criterion index,Harrell's C index,and likelihood ratio chi-square.RESULTS We used the variables that were statistically significant factors for the 3-year overall survival as input-layer variables to develop a preope-ANN in the training set.The survival curves within each score of the preope-ANN had good discrimination(P<0.05).Comparing the preope-ANN model,cTNM,and pTNM in both the training and testing sets,the preope-ANN model was superior to cTNM in predictive discrimination(C index),predictive homogeneity(likelihood ratio chi-square),and prediction accuracy(area under the curve).The prediction efficiency of the preope-ANN model is similar to that of pTNM.CONCLUSION The preope-ANN model can accurately predict the long-term survival of GC patients,and its predictive efficiency is not inferior to that of pTNM stage.

Dynamic Analysis of Fractional-Order Fuzzy BAM Neural Networks with Delays in the Leakage Terms

In this paper, based on the theory of fractional-order calculus, we obtain some sufficient conditions for the uniform stability of fractional-order fuzzy BAM neural networks with delays in the leakage terms. Moreover, the existence, uniqueness and stability of its equilibrium point are also proved. A numerical example is presented to demonstrate the validity and feasibility of the proposed results.

基于神经网络模型的水平井破裂压力预测方法

破裂压力是井身结构设计的基础依据,也是水力压裂设备选型和方案设计的基础参数,通常采用测井解释获取破裂压力剖面,但其存在参数准确获取难、计算过程繁琐、普适性较差、计算精度低等问题,机器学习提供了一种解决这些问题的新方法。为此,以测井数据作为输入参数,采用4种不同的神经网络模型,建立水平井测井数据与破裂压力间的非线性关系,通过测试集预测结果的对比分析,优选出最佳的神经网络模型,并优化模型网络结构和超参数,实现水平井破裂压力的直接预测。研究结果表明:1)破裂压力与井斜角、横波时差和纵波时差表现为极强相关性,与井深、岩性密度和补偿中子表现为强相关性,与井径和自然伽马表现为弱相关性;2)不同组合的测井参数对模型预测结果具有显著影响,最优输入参数为井斜角、横波时差、纵波时差、井深、岩性密度和补偿中子;3)对比多层感知机、深度神经网络、循环神经网络和长短期记忆神经网络(LSTM)模型,发现LSTM模型的预测效果最佳;4)优化了LSTM模型的网络结构及超参数,优化后破裂压力预测的平均绝对百分比误差为0.106%、决定系数为0.996。LSTM模型能够有效构建水平井测井参数与破裂压力之间的非线性关系,可以实现水平井破裂压力的准确预测,对于准确预测破裂压力、简化破裂压力计算过程、推广机器学习在石油工程领域的应用具有重要的作用。

The Role of Rare Terms in Enhancing the Performance of Polynomial Networks Based Text Categorization

In this paper, the role of rare or infrequent terms in enhancing the accuracy of English Text Categorization using Polynomial Networks (PNs) is investigated. To study the impact of rare terms in enhancing the accuracy of PNs-based text categorization, different term reduction criteria as well as different term weighting schemes were experimented on the Reuters Corpus using PNs. Each term weighting scheme on each reduced term set was tested once keeping the rare terms and another time removing them. All the experiments conducted in this research show that keeping rare terms substantially improves the performance of Polynomial Networks in Text Categorization, regardless of the term reduction method, the number of terms used in classification, or the term weighting scheme adopted.

基于CNN-LSTM的机床滚动轴承性能退化趋势和寿命预测

滚动轴承作为机床主轴的关键部件,其剩余寿命预测直接决定着整机设备的剩余寿命。若不能及时地预知滚动轴承的健康状态或损伤情况,不仅会影响维修策略的制定,还会造成级联故障,易造成机床灾难性的事故。针对大数据下滚动轴承振动信号的自适应故障特征提取和智能诊断问题,构建卷积神经网络和长短期记忆网络(CNN-LSTM)相结合的寿命预测模型,它可以避免人工参与的影响,实现网络的互补优势。对滚动轴承的退化状态以及剩余寿命进行预测,并与卷积神经网络(CNN)、长短时记忆神经网络(LSTM)进行对比实验。结果表明:所提方法CNN-LSTM有着较高的预测准确度。

基于多源信息融合和WOA-CNN-LSTM的外脚手架隐患分类预警研究

面对施工现场外脚手架隐患信息的多样性,传统的基于传感器监测的单一信号预警研究存在容错力不佳、含有信息有限等问题。针对施工现场外脚手架“图像+监测”数据,提出一种基于数据层和特征层信息融合的脚手架隐患分类预警方法。首先,利用Revit三维建模软件建立外脚手架实体模型,对不同初始隐患下的外脚手架进行有限元分析,划分隐患预警等级;其次,利用无迹卡尔曼滤波算法(Unscented Kalman Filter,UKF)及卷积长短时记忆网络(Convolutional Neural Network-Long Short Term Memory Network,CNN-LSTM)实现脚手架同类信息数据层融合及异类信息特征层融合;最后,通过实时收集西安市某在建项目落地式双排扣件式钢管脚手架隐患信息,对其进行分类预警,并使用鲸鱼优化算法(Whale Optimization Algorithm,WOA)对CNN-LSTM网络进行参数优化,发现隐藏节点个数为30、学习率为0.0072、正则化系数为1×10^(-4)时分类效果最佳,优化后预警精度达到了91.4526%。通过可视化WOA-CNN-LSTM、CNN-LSTM、CNN-SVM(Support Vector Machine,支持向量机)及CNN-GRU(Gate Recurrent Unit,门控循环单元)分类预警结果,证实了优化后的CNN-LSTM网络在脚手架分类预警方面的优越性。

Short-Term Load Forecasting Using Radial Basis Function Neural Network

An accurate short-term forecasting method for load of electric power system can help the electric power system’s operator to reduce the risk of unreliability of electricity supply. This paper proposed a radial basis function (RBF) neural network method to forecast the short-term load of electric power system. To demonstrate the effectiveness of the proposed method, the method is tested on the practical load data information of the Tai power system. The good agreements between the realistic values and forecasting values are obtained;the numerical results show that the proposed forecasting method is accurate and reliable.

基于ResNet-LSTM的航空发动机性能异常检测方法

为了实现数据驱动的航空发动机性能异常的智能检测,提出了一种基于残差网络(ResNet)-长短期记忆网络(LSTM)的发动机性能异常检测方法。采用发动机性能数据图像化方法,在数据降维的同时,完备保留数据的关联特征和时序特征;以残差单元构建发动机性能异常检测模型,在加深网络结构的同时,消除深层网络梯度消失问题,提高发动机性能图像空间关联特征的提取能力。同时,引入LSTM,提出基于ResNet-LSTM的发动机性能异常检测模型,通过ResNet与LSTM的融合,强化异常检测模型对时序特征的提取,提升发动机性能异常检测的准确率;通过发动机运行数据进行验证。结果表明:在训练集上,该方法的异常检测准确率为94.95%,比基于ResNet18、ResNet34、ResNet50异常检测模型的分别提高10.87%、8.00%、3.23%;在测试集上,该方法的异常检测准确率为92.15%,比基于ResNet18、ResNet34、ResNet50异常检测模型的分别提高11.81%、9.45%、3.78%。

Dynamic Resource Allocation in LTE Radio Access Network Using Machine Learning Techniques

Current LTE networks are experiencing significant growth in the number of users worldwide. The use of data services for online browsing, e-learning, online meetings and initiatives such as smart cities means that subscribers stay connected for long periods, thereby saturating a number of signalling resources. One of such resources is the Radio Resource Connected (RRC) parameter, which is allocated to eNodeBs with the aim of limiting the number of connected simultaneously in the network. The fixed allocation of this parameter means that, depending on the traffic at different times of the day and the geographical position, some eNodeBs are saturated with RRC resources (overused) while others have unused RRC resources. However, as these resources are limited, there is the problem of their underutilization (non-optimal utilization of resources at the eNodeB level) due to static allocation (manual configuration of resources). The objective of this paper is to design an efficient machine learning model that will take as input some key performance indices (KPIs) like traffic data, RRC, simultaneous users, etc., for each eNodeB per hour and per day and accurately predict the number of needed RRC resources that will be dynamically allocated to them in order to avoid traffic and financial losses to the mobile network operator. To reach this target, three machine learning algorithms have been studied namely: linear regression, convolutional neural networks and long short-term memory (LSTM) to train three models and evaluate them. The model trained with the LSTM algorithm gave the best performance with 97% accuracy and was therefore implemented in the proposed solution for RRC resource allocation. An interconnection architecture is also proposed to embed the proposed solution into the Operation and maintenance network of a mobile network operator. In this way, the proposed solution can contribute to developing and expanding the concept of Self Organizing Network (SON) used in 4G and 5G networks.

基于BERT-BiLSTM-CRF模型的畜禽疫病文本分词研究

针对畜禽疫病文本语料匮乏、文本内包含大量疫病名称及短语等未登录词问题,提出了一种结合词典匹配的BERT-BiLSTM-CRF畜禽疫病文本分词模型。以羊疫病为研究对象,构建了常见疫病文本数据集,将其与通用语料PKU结合,利用BERT(Bidirectional encoder representation from transformers)预训练语言模型进行文本向量化表示;通过双向长短时记忆网络(Bidirectional long short-term memory network,BiLSTM)获取上下文语义特征;由条件随机场(Conditional random field,CRF)输出全局最优标签序列。基于此,在CRF层后加入畜禽疫病领域词典进行分词匹配修正,减少在分词过程中出现的疫病名称及短语等造成的歧义切分,进一步提高了分词准确率。实验结果表明,结合词典匹配的BERT-BiLSTM-CRF模型在羊常见疫病文本数据集上的F1值为96.38%,与jieba分词器、BiLSTM-Softmax模型、BiLSTM-CRF模型、未结合词典匹配的本文模型相比,分别提升11.01、10.62、8.3、0.72个百分点,验证了方法的有效性。与单一语料相比,通用语料PKU和羊常见疫病文本数据集结合的混合语料,能够同时对畜禽疫病专业术语及疫病文本中常用词进行准确切分,在通用语料及疫病文本数据集上F1值都达到95%以上,具有较好的模型泛化能力。该方法可用于畜禽疫病文本分词。