Estimation of unloading relaxation depth of Baihetan Arch Dam foundation using long-short term memory network

The unloading relaxation caused by excavation for construction of high arch dams is an important factor influencing the foundation’s integrity and strength.To evaluate the degree of unloading relaxation,the long-short term memory(LSTM)network was used to estimate the depth of unloading relaxation zones on the left bank foundation of the Baihetan Arch Dam.Principal component analysis indicates that rock charac-teristics,the structural plane,the protection layer,lithology,and time are the main factors.The LSTM network results demonstrate the unloading relaxation characteristics of the left bank,and the relationships with the factors were also analyzed.The structural plane has the most significant influence on the distribution of unloading relaxation zones.Compared with massive basalt,the columnar jointed basalt experiences a more significant unloading relaxation phenomenon with a clear time effect,with the average unloading relaxation period being 50 d.The protection layer can effectively reduce the unloading relaxation depth by approximately 20%.

Online multi-target intelligent tracking using a deep long-short term memory network

Multi-target tracking is facing the difficulties of modeling uncertain motion and observation noise.Traditional tracking algorithms are limited by specific models and priors that may mismatch a real-world scenario.In this paper,considering the model-free purpose,we present an online Multi-Target Intelligent Tracking(MTIT)algorithm based on a Deep Long-Short Term Memory(DLSTM)network for complex tracking requirements,named the MTIT-DLSTM algorithm.Firstly,to distinguish trajectories and concatenate the tracking task in a time sequence,we define a target tuple set that is the labeled Random Finite Set(RFS).Then,prediction and update blocks based on the DLSTM network are constructed to predict and estimate the state of targets,respectively.Further,the prediction block can learn the movement trend from the historical state sequence,while the update block can capture the noise characteristic from the historical measurement sequence.Finally,a data association scheme based on Hungarian algorithm and the heuristic track management strategy are employed to assign measurements to targets and adapt births and deaths.Experimental results manifest that,compared with the existing tracking algorithms,our proposed MTIT-DLSTM algorithm can improve effectively the accuracy and robustness in estimating the state of targets appearing at random positions,and be applied to linear and nonlinear multi-target tracking scenarios.

Feature identification in complex fluid flows by convolutional neural networks

Recent advancements have established machine learning's utility in predicting nonlinear fluid dynamics,with predictive accuracy being a central motivation for employing neural networks.However,the pattern recognition central to the networks function is equally valuable for enhancing our dynamical insight into the complex fluid dynamics.In this paper,a single-layer convolutional neural network(CNN)was trained to recognize three qualitatively different subsonic buffet flows(periodic,quasi-periodic and chaotic)over a high-incidence airfoil,and a near-perfect accuracy was obtained with only a small training dataset.The convolutional kernels and corresponding feature maps,developed by the model with no temporal information provided,identified large-scale coherent structures in agreement with those known to be associated with buffet flows.Sensitivity to hyperparameters including network architecture and convolutional kernel size was also explored.The coherent structures identified by these models enhance our dynamical understanding of subsonic buffet over high-incidence airfoils over a wide range of Reynolds numbers.

基于CNN-NLSTM的脑电信号注意力状态分类方法

通过脑电信号进行注意力状态检测,对扩大脑-机接口技术的应用范围具有重要意义。为了提高注意力状态的分类准确率,该文提出一种基于CNN-NLSTM的脑电信号分类模型。首先采用Welch方法获得脑电信号的功率谱密度特征并将其表示为二维灰度图像。然后使用卷积神经网络从灰度图像中学习表征注意力状态的特征,并将相关特征输入到嵌套长短时记忆神经网络依次获得所有时间步骤的注意力特征。最后将两个网络依次连接来构建深度学习框架进行注意力状态分类。实验结果表明,该文所提出的模型通过进行多次5-折交叉验证评估后得到89.26%的平均分类准确率和90.40%的最大分类准确率,与其他模型相比具有更好的分类效果和稳定性。

ST-Trader:A Spatial-Temporal Deep Neural Network for Modeling Stock Market Movement

Stocks that are fundamentally connected with each other tend to move together.Considering such common trends is believed to benefit stock movement forecasting tasks.However,such signals are not trivial to model because the connections among stocks are not physically presented and need to be estimated from volatile data.Motivated by this observation,we propose a framework that incorporates the inter-connection of firms to forecast stock prices.To effectively utilize a large set of fundamental features,we further design a novel pipeline.First,we use variational autoencoder(VAE)to reduce the dimension of stock fundamental information and then cluster stocks into a graph structure(fundamentally clustering).Second,a hybrid model of graph convolutional network and long-short term memory network(GCN-LSTM)with an adjacency graph matrix(learnt from VAE)is proposed for graph-structured stock market forecasting.Experiments on minute-level U.S.stock market data demonstrate that our model effectively captures both spatial and temporal signals and achieves superior improvement over baseline methods.The proposed model is promising for other applications in which there is a possible but hidden spatial dependency to improve time-series prediction.

基于设备特征多层优选和CNN⁃NLSTM模型的非侵入式负荷分解

非侵入式负荷分解技术可以有效挖掘用户侧设备信息,是电网开展用户负荷互动响应的基础。针对目前非侵入式负荷分解模型适应性较差及准确率较低等问题,提出一种基于设备特征多层优选的非侵入式负荷分解模型。首先,针对设备运行特性设计自适应滑动数据窗,进而获取到更加完整的设备功率片段,同时调整网络输入输出维度;其次,通过融合浅层卷积神经网络(CNN)与两层嵌套长短时记忆网络(NLSTM)提取并加深设备特征;然后,将其输入到改进的注意力机制中,通过调配特征权重,获得最优的设备特征序列;最后,在REDD数据集上进行实验分析,通过对设备特征多层选择、加深与复用在减小训练时间的同时,显著地提升负荷分解的准确率。

MSCNN-LSTM Model for Predicting Return Loss of the UHF Antenna in HF-UHF RFID Tag Antenna

High-frequency(HF)and ultrahigh-frequency(UHF)dual-band radio frequency identification(RFID)tags with both near-field and farfield communication can meet different application scenarios.However,it is time-consuming to calculate the return loss of a UHF antenna in a dualband tag antenna using electromagnetic(EM)simulators.To overcome this,the present work proposes a model of a multi-scale convolutional neural network stacked with long and short-term memory(MSCNN-LSTM)for predicting the return loss of UHF antennas instead of EM simulators.In the proposed MSCNN-LSTM,the MSCNN has three branches,which include three convolution layers with different kernel sizes and numbers.Therefore,MSCNN can extract fine-grain localized information of the antenna and overall features.The LSTM can effectively learn the EM characteristics of different structures of the antenna to improve the prediction accuracy of the model.Experimental results show that the mean absolute error(0.0073),mean square error(0.00032),and root mean square error(0.01814)of theMSCNNLSTM are better than those of other prediction methods.In predicting the return loss of 100UHFantennas,compared with the simulation time of 4800 s for High Frequency Structure Simulator(HFSS),MSCNN-LSTM takes only 0.927519 s under the premise of ensuring prediction accuracy,significantly reducing the calculation time,which provides a basis for the rapid design of HF-UHF RFID tag antenna.ThenMSCNN-LSTM is used to determine the dimensions of the UHF antenna quickly.The return loss of the designed dualband RFID tag antenna is−58.76 and−22.63 dB at 13.56 and 915 MHz,respectively,achieving the desired goal.

基于SVMD-MASSA-NLSTM的台区短期负荷预测

为提高电力短期负荷预测的精度,提出了一种基于逐次变分模态分解、改进的自适应麻雀搜索算法和嵌套长短期记忆网络的混合深度学习模型。首先,使用SVMD将原始数据分解为一定数量的模态函数和残差分量。其次,采用混沌逆向学习技术增加种群多样性,利用动态自适应权重,平衡开采和勘探能力。采用自适应螺旋搜索方法修改了SSA的发现者和追随者的位置更新公式,拓宽了个体搜索空间,提高了算法的全局搜索能力。然后,利用改进的SSA优化NLSTM模型的参数,即隐藏神经元数和学习率,将优化后的NLSTM应用于分解后的模态分量。最后对各模态分量的预测结果进行汇总,得到负荷预测结果。实验结果表明,文中所提模型较其他模型拥有更好的预测性能。

基于注意力CNLSTM模型的新闻文本分类

结合卷积神经网络(CNN)和嵌套长短期记忆网络(NLSTM)2种模型,基于注意力机制提出一个用于文本表示和分类的CNLSTM模型。采用CNN提取短语序列的特征表示,利用NLSTM学习文本的特征表示,引入注意力机制突出关键短语以优化特征提取的过程。在3个公开新闻数据集中进行性能测试,结果表明,该模型的分类准确率分别为96.87 %、95.43 %和97.58 %,其性能比baseline方法有显著提高。

基于交互式特征融合的嵌套命名实体识别

现有命名实体识别模型在字嵌入过程中多采用字符向量、字向量等不同单词表示向量的拼接或累加方式提取信息,未考虑不同单词表示特征之间的相互依赖关系,导致单词内部特征信息获取不足。提出一种基于交互式特征融合的嵌套命名实体识别模型,通过交互的方式构建不同特征之间的通信桥梁,以捕获多特征之间的依赖关系。采用交互机制得到包含不同单词表示信息的字嵌入向量,基于双向长短时记忆网络提取单词的表示特征,并对不同单词的表示特征进行交互,捕获特征之间的相互依赖关系。为进一步提取序列特征的上下文信息,采用基于特征交互的多头注意力机制捕获句子上下文的依赖关系。在此基础上,采用二元序列标记法过滤非实体区域,得到粗粒度候选区间,并对其进行细粒度划分以判断实体类别。实验结果表明,该模型的召回率和F1值为72.4%和71.2%,相比现有的嵌套命名实体识别模型,F1值平均提高了1.72%。

Short-term feeding behaviour sound classification method for sheep using LSTM networks

A deep learning approach using long-short term memory(LSTM)networks was implemented in this study to classify the sound of short-term feeding behaviour of sheep,including biting,chewing,bolus regurgitation,and rumination chewing.The original acoustic signal was split into sound episodes using an endpoint detection method,where the thresholds of short-term energy and average zero-crossing rate were utilized.A discrete wavelet transform(DWT),Mel-frequency cepstral,and principal-component analysis(PCA)were integrated to extract the dimensionally reduced DWT based Mel-frequency cepstral coefficients(denoted by PW_MFCC)for each sound episode.Then,LSTM networks were employed to train classifiers for sound episode category classification.The performances of the LSTM classifiers with original Mel-frequency cepstral coefficients(MFCC),DWT based MFCC(denoted by W_MFCC),and PW_MFCC as the input feature coefficients were compared.Comparison results demonstrated that the introduction of DWT improved the classifier performance effectively,and PCA reduced the computational overhead without degrading classifier performance.The overall accuracy and comprehensive F1-score of the PW_MFCC based LSTM classifier were 94.97%and 97.41%,respectively.The classifier established in this study provided a foundation for an automatic identification system for sick sheep with abnormal feeding and rumination behaviour pattern.

Enhanced Accuracy for Motor Imagery Detection Using Deep Learning for BCI

Brain-Computer Interface(BCI)is a system that provides a link between the brain of humans and the hardware directly.The recorded brain data is converted directly to the machine that can be used to control external devices.There are four major components of the BCI system:acquiring signals,preprocessing of acquired signals,features extraction,and classification.In traditional machine learning algorithms,the accuracy is insignificant and not up to the mark for the classification of multi-class motor imagery data.The major reason for this is,features are selected manually,and we are not able to get those features that give higher accuracy results.In this study,motor imagery(MI)signals have been classified using different deep learning algorithms.We have explored two different methods:Artificial Neural Network(ANN)and Long Short-Term Memory(LSTM).We test the classification accuracy on two datasets:BCI competition III-dataset IIIa and BCI competition IV-dataset IIa.The outcome proved that deep learning algorithms provide greater accuracy results than traditional machine learning algorithms.Amongst the deep learning classifiers,LSTM outperforms the ANN and gives higher classification accuracy of 96.2%.

分层区域穷举的中文嵌套命名实体识别方法

嵌套命名实体之间蕴含着丰富的语义关系与结构信息,开发能够准确识别嵌套命名实体的算法具有重要研究意义。针对现有的中文嵌套命名实体数据集中存在错标漏标以及现有识别方法大多忽略嵌套实体内部信息关联关系而导致准确性下降的问题,结合自动生成与手动标注的方法构建新的中文嵌套命名实体数据集NEPD,在此基础上,设计一种利用分层区域穷举的中文嵌套命名实体识别模型。该模型通过遍历文本组合实体,获取低层编码层的词嵌入信息;其次,为使邻接编码层之间实现信息交换,将低层编码层的词嵌入信息融入高层编码层;最后,利用多层解码层使长度为L的命名实体仅在第L层预测,有效防止错误传播现象发生从而提高识别准确度。实验结果表明,在没有外部知识资源的情况下,LREM模型在嵌套命名实体与非嵌套命名实体上的识别F1值分别达到87.19%和86.27%,其中非嵌套命名实体识别的F1值比传统的BiLSTM+CRF模型提升1.18%,验证了该模型的可靠性。

A hybrid Wavelet-CNN-LSTM deep learning model for short-term urban water demand forecasting

Short-term water demand forecasting provides guidance on real-time water allocation in the water supply network, which help water utilities reduce energy cost and avoid potential accidents. Although a variety of methods have been proposed to improve forecast accuracy, it is still difficult for statistical models to learn the periodic patterns due to the chaotic nature of the water demand data with high temporal resolution. To overcome this issue from the perspective of improving data predictability, we proposed a hybrid Wavelet-CNN-LSTM model, that combines time-frequency decomposition characteristics of Wavelet Multi-Resolution Analysis (MRA) and implement it into an advanced deep learning model, CNN-LSTM. Four models - ANN, Conv1D, LSTM, GRUN - are used to compare with Wavelet-CNN-LSTM, and the results show that Wavelet-CNN-LSTM outperforms the other models both in single-step and multi-steps prediction. Besides, further mechanistic analysis revealed that MRA produce significant effect on improving model accuracy.

CPPCNDL: Crude oil price prediction using complex network and deep learning algorithms

Crude oil price prediction is a challenging task in oil producing countries.Its price is among the most complex and tough to model because fluctuations of price of crude oil are highly irregular,nonlinear and varies dynamically with high uncertainty.This paper proposed a hybrid model for crude oil price prediction that uses the complex network analysis and long short-term memory(LSTM)of the deep learning algorithms.The complex network analysis tool called the visibility graph is used to map the dataset on a network and K-core centrality was employed to extract the non-linearity features of crude oil and reconstruct the dataset.The complex network analysis is carried out in order to preprocess the original data to extract the non-linearity features and to reconstruct the data.Thereafter,LSTM was employed to model the reconstructed data.To verify the result,we compared the empirical results with other research in the literature.The experiments show that the proposed model has higher accuracy,and is more robust and reliable.

Double LSTM Structure for Network Traffic Flow Prediction

The network traffic prediction is important for service quality control in computer network.The performance of the traditional prediction method significantly degrades for the burst short-term flow.In view of the problem,this paper proposes a double LSTMs structure,one of which acts as the main flow predictor,another as the detector of the time the burst flow starts at.The two LSTM units can exchange information about their internal states,and the predictor uses the detector’s information to improve the accuracy of the prediction.A training algorithm is developed specially to train the structure offline.To obtain the prediction online,a pulse series is used as a simulant of the burst event.A simulation experiment is designed to test performance of the predictor.The results of the experiment show that the prediction accuracy of the double LSTM structure is significantly improved,compared with the traditional single LSTM structure.

基于残差NLSTM网络和注意力机制的航空发动机剩余使用寿命预测

针对长短期记忆(LSTM)网络对于多维数据特征识别和提取上存在不足的问题,在其改进模型嵌套式长短期记忆(NLSTM)网络的基础上,提出了一种基于注意力机制和残差NLSTM网络的剩余使用寿命预测方法。该方法将双层NLSTM网络代替残差块中的主网络,保留捷径连接中的卷积神经网络结构,既能充分提取时序特征又能保证有用数据在网络层中的跳层传递,并融入注意力机制构建多层残差网络,注意力机制的使用能够选择出对预测结果有重要影响的信息,有效提高预测的准确率。在航空发动机退化实验数据集上进行实验分析,结果表明:所述方法能有效建立监测数据与发动机健康状态之间的关系,剩余使用寿命预测误差较未改进残差结构方法平均降低10.8%,比未融入注意力机制方法平均降低18.9%,有效提高了预测精度。

A novel predict-prevention quality control method of multi-stage manufacturing process towards zero defect manufacturing

Zero defection manufacturing (ZDM) is the pursuit of the manufacturing industry. However, there is a lack of the implementation method of ZDM in the multi-stage manufacturing process (MMP). Implementing ZDM and controlling product quality in MMP remains an urgent problem in intelligent manufacturing. A novel predict-prevention quality control method in MMP towards ZDM is proposed, including quality characteristics monitoring, key quality characteristics prediction, and assembly quality optimization. The stability of the quality characteristics is detected by analyzing the distribution of quality characteristics. By considering the correlations between different quality characteristics, a deep supervised long-short term memory (SLSTM) prediction network is built for time series prediction of quality characteristics. A long-short term memory-genetic algorithm (LSTM-GA) network is proposed to optimize the assembly quality. By utilizing the proposed quality control method in MMP, unqualified products can be avoided, and ZDM of MMP is implemented. Extensive empirical evaluations on the MMP of compressors validate the applicability and practicability of the proposed method.

基于焦点损失函数的嵌套长短时记忆网络心电信号分类研究

心电图(ECG)可直观地反映人体心脏生理电活动,在心律失常检测与分类领域中具有重要意义。针对ECG数据中类别不平衡对心律失常分类带来的消极作用,本文提出一种用于不平衡ECG信号分类的嵌套长短时记忆网络(NLSTM)模型。搭建NLSTM学习并记忆复杂信号中的时序特征,利用焦点损失函数(focal loss)降低易识别样本的权重;然后采用残差注意力机制(residual attention mechanism),根据各类别特征重要性修改已分配权值,解决样本不平衡问题;再采用合成过采样技术算法(SMOTE)对麻省理工学院与贝斯以色列医院心律失常(MIT-BIH-AR)数据库进行简单的人工过采样处理,进一步增加模型的分类准确率,最终应用MIT-BIHAR数据库对上述算法进行实验验证。实验结果表明,所提方法能有效地解决ECG信号中样本不平衡、特征不突出的问题,模型的总体准确率达到98.34%,较大地提升对少数类样本的识别和分类效果,为心律失常辅助诊断提供可行的新方法。

基于嵌套长短期记忆网络的机械装备剩余使用寿命预测方法

剩余使用寿命(remaining useful life,RUL)预测是保障机械装备可靠性、可用性和安全性的重要技术.本文提出一种基于嵌套长短期记忆(nested long short-term memory,NLSTM)网络的机械装备RUL预测方法,它通过融合多传感器监测信号,实现对机械装备RUL的精确预测.区别于普通LSTM网络,NLSTM将存储单元进一步加深,将一个LSTM神经元结构嵌套在原有LSTM的存储空间中,实现对多传感器时间序列信号中长期依赖性的深度捕捉.本文使用涡扇发动机和加工刀具两个实验案例来验证NLSTM的预测性能;从涡扇发动机案例验证可知,相比于LSTM,NLSTM的预测性能在两个指标上分别整体提升了4.66%和15.18%,且NLSTM的预测结果也优于文献中的其他先进方法;从加工刀具案例验证可知,NLSTM的预测结果在六个刀具上的预测结果均优于LSTM.