An efficient stock market prediction model using hybrid feature reduction method based on variational autoencoders and recursive feature elimination

In this study,the hourly directions of eight banking stocks in Borsa Istanbul were predicted using linear-based,deep-learning(LSTM)and ensemble learning(Light-GBM)models.These models were trained with four different feature sets and their performances were evaluated in terms of accuracy and F-measure metrics.While the first experiments directly used the own stock features as the model inputs,the second experiments utilized reduced stock features through Variational AutoEncoders(VAE).In the last experiments,in order to grasp the effects of the other banking stocks on individual stock performance,the features belonging to other stocks were also given as inputs to our models.While combining other stock features was done for both own(named as allstock_own)and VAE-reduced(named as allstock_VAE)stock features,the expanded dimensions of the feature sets were reduced by Recursive Feature Elimination.As the highest success rate increased up to 0.685 with allstock_own and LSTM with attention model,the combination of allstock_VAE and LSTM with the attention model obtained an accuracy rate of 0.675.Although the classification results achieved with both feature types was close,allstock_VAE achieved these results using nearly 16.67%less features compared to allstock_own.When all experimental results were examined,it was found out that the models trained with allstock_own and allstock_VAE achieved higher accuracy rates than those using individual stock features.It was also concluded that the results obtained with the VAE-reduced stock features were similar to those obtained by own stock features.

Estimation of unloading relaxation depth of Baihetan Arch Dam foundation using long-short term memory network

The unloading relaxation caused by excavation for construction of high arch dams is an important factor influencing the foundation’s integrity and strength.To evaluate the degree of unloading relaxation,the long-short term memory(LSTM)network was used to estimate the depth of unloading relaxation zones on the left bank foundation of the Baihetan Arch Dam.Principal component analysis indicates that rock charac-teristics,the structural plane,the protection layer,lithology,and time are the main factors.The LSTM network results demonstrate the unloading relaxation characteristics of the left bank,and the relationships with the factors were also analyzed.The structural plane has the most significant influence on the distribution of unloading relaxation zones.Compared with massive basalt,the columnar jointed basalt experiences a more significant unloading relaxation phenomenon with a clear time effect,with the average unloading relaxation period being 50 d.The protection layer can effectively reduce the unloading relaxation depth by approximately 20%.

基于LSTM-文本分析的量化选股模型

随着国民生活水平的提高,越来越多的人投身于股票市场.为了科学有效地量化选股,通过将量化投资、深度学习及文本分析进行有机结合,来建立量化选股模型.首先,通过文本分析筛选出基本面利好的股票;然后,通过长短期记忆(long-short term memory,LSTM)选出预测准确度良好的股票;最后,预测所选出的股票在未来几天的股价趋势.在实证分析方面,通过本模型对部分股票进行运算,选取预测效果较好的股票:赢合科技.

多时间尺度小波变换和LSTM自编码器电弧故障检测方法

在光伏发电系统中,电弧故障检测是维持系统安全运行的关键问题。以往的电弧故障检测方法大多基于单时间尺度的故障特征,然而单一时间尺度特征往往会受到环境变化的干扰,导致检测精度降低,针对这一问题,提出一种多时间尺度小波和长短时记忆(LSTM)自编码器电弧故障检测方法,该方法首先在机理分析的基础上找到电弧3个特性,即电弧初始阶段电流发生突变、燃弧阶段电流均值降低、燃弧阶段高频分量变大。再基于上述电弧特性进行小波变换提取对应多尺度特征,然后使用LSTM自编码器进行端到端的自动检测。与以往方法不同,该方法提取了电弧特性的多种时间尺度特征,增加了故障信号的检测依据,降低了受外界干扰时检测结果出现误报漏报的可能性。理论分析和实验结果表明,所提出的方法降低了故障电弧检测的误报率,提高了其准确率。

Feature identification in complex fluid flows by convolutional neural networks

Recent advancements have established machine learning's utility in predicting nonlinear fluid dynamics,with predictive accuracy being a central motivation for employing neural networks.However,the pattern recognition central to the networks function is equally valuable for enhancing our dynamical insight into the complex fluid dynamics.In this paper,a single-layer convolutional neural network(CNN)was trained to recognize three qualitatively different subsonic buffet flows(periodic,quasi-periodic and chaotic)over a high-incidence airfoil,and a near-perfect accuracy was obtained with only a small training dataset.The convolutional kernels and corresponding feature maps,developed by the model with no temporal information provided,identified large-scale coherent structures in agreement with those known to be associated with buffet flows.Sensitivity to hyperparameters including network architecture and convolutional kernel size was also explored.The coherent structures identified by these models enhance our dynamical understanding of subsonic buffet over high-incidence airfoils over a wide range of Reynolds numbers.

Online multi-target intelligent tracking using a deep long-short term memory network

Multi-target tracking is facing the difficulties of modeling uncertain motion and observation noise.Traditional tracking algorithms are limited by specific models and priors that may mismatch a real-world scenario.In this paper,considering the model-free purpose,we present an online Multi-Target Intelligent Tracking(MTIT)algorithm based on a Deep Long-Short Term Memory(DLSTM)network for complex tracking requirements,named the MTIT-DLSTM algorithm.Firstly,to distinguish trajectories and concatenate the tracking task in a time sequence,we define a target tuple set that is the labeled Random Finite Set(RFS).Then,prediction and update blocks based on the DLSTM network are constructed to predict and estimate the state of targets,respectively.Further,the prediction block can learn the movement trend from the historical state sequence,while the update block can capture the noise characteristic from the historical measurement sequence.Finally,a data association scheme based on Hungarian algorithm and the heuristic track management strategy are employed to assign measurements to targets and adapt births and deaths.Experimental results manifest that,compared with the existing tracking algorithms,our proposed MTIT-DLSTM algorithm can improve effectively the accuracy and robustness in estimating the state of targets appearing at random positions,and be applied to linear and nonlinear multi-target tracking scenarios.

Soil NOx Emission Prediction via Recurrent Neural Networks

This paper presents designing sequence-to-sequence recurrent neural network(RNN)architectures for a novel study to predict soil NOx emissions,driven by the imperative of understanding and mitigating environmental impact.The study utilizes data collected by the Environmental Protection Agency(EPA)to develop two distinct RNN predictive models:one built upon the long-short term memory(LSTM)and the other utilizing the gated recurrent unit(GRU).These models are fed with a combination of historical and anticipated air temperature,air moisture,and NOx emissions as inputs to forecast future NOx emissions.Both LSTM and GRU models can capture the intricate pulse patterns inherent in soil NOx emissions.Notably,the GRU model emerges as the superior performer,surpassing the LSTM model in predictive accuracy while demonstrating efficiency by necessitating less training time.Intriguingly,the investigation into varying input features reveals that relying solely on past NOx emissions as input yields satisfactory performance,highlighting the dominant influence of this factor.The study also delves into the impact of altering input series lengths and training data sizes,yielding insights into optimal configurations for enhanced model performance.Importantly,the findings promise to advance our grasp of soil NOx emission dynamics,with implications for environmental management strategies.Looking ahead,the anticipated availability of additional measurements is poised to bolster machine-learning model efficacy.Furthermore,the future study will explore physical-based RNNs,a promising avenue for deeper insights into soil NOx emission prediction.

Nonlinear Dynamic System Identification of ARX Model for Speech Signal Identification

System Identification becomes very crucial in the field of nonlinear and dynamic systems or practical systems.As most practical systems don’t have prior information about the system behaviour thus,mathematical modelling is required.The authors have proposed a stacked Bidirectional Long-Short Term Memory(Bi-LSTM)model to handle the problem of nonlinear dynamic system identification in this paper.The proposed model has the ability of faster learning and accurate modelling as it can be trained in both forward and backward directions.The main advantage of Bi-LSTM over other algorithms is that it processes inputs in two ways:one from the past to the future,and the other from the future to the past.In this proposed model a backward-running Long-Short Term Memory(LSTM)can store information from the future along with application of two hidden states together allows for storing information from the past and future at any moment in time.The proposed model is tested with a recorded speech signal to prove its superiority with the performance being evaluated through Mean Square Error(MSE)and Root Means Square Error(RMSE).The RMSE and MSE performances obtained by the proposed model are found to be 0.0218 and 0.0162 respectively for 500 Epochs.The comparison of results and further analysis illustrates that the proposed model achieves better performance over other models and can obtain higher prediction accuracy along with faster convergence speed.

MSCNN-LSTM Model for Predicting Return Loss of the UHF Antenna in HF-UHF RFID Tag Antenna

High-frequency(HF)and ultrahigh-frequency(UHF)dual-band radio frequency identification(RFID)tags with both near-field and farfield communication can meet different application scenarios.However,it is time-consuming to calculate the return loss of a UHF antenna in a dualband tag antenna using electromagnetic(EM)simulators.To overcome this,the present work proposes a model of a multi-scale convolutional neural network stacked with long and short-term memory(MSCNN-LSTM)for predicting the return loss of UHF antennas instead of EM simulators.In the proposed MSCNN-LSTM,the MSCNN has three branches,which include three convolution layers with different kernel sizes and numbers.Therefore,MSCNN can extract fine-grain localized information of the antenna and overall features.The LSTM can effectively learn the EM characteristics of different structures of the antenna to improve the prediction accuracy of the model.Experimental results show that the mean absolute error(0.0073),mean square error(0.00032),and root mean square error(0.01814)of theMSCNNLSTM are better than those of other prediction methods.In predicting the return loss of 100UHFantennas,compared with the simulation time of 4800 s for High Frequency Structure Simulator(HFSS),MSCNN-LSTM takes only 0.927519 s under the premise of ensuring prediction accuracy,significantly reducing the calculation time,which provides a basis for the rapid design of HF-UHF RFID tag antenna.ThenMSCNN-LSTM is used to determine the dimensions of the UHF antenna quickly.The return loss of the designed dualband RFID tag antenna is−58.76 and−22.63 dB at 13.56 and 915 MHz,respectively,achieving the desired goal.

An Improved Time Feedforward Connections Recurrent Neural Networks

Recurrent Neural Networks(RNNs)have been widely applied to deal with temporal problems,such as flood forecasting and financial data processing.On the one hand,traditional RNNs models amplify the gradient issue due to the strict time serial dependency,making it difficult to realize a long-term memory function.On the other hand,RNNs cells are highly complex,which will signifi-cantly increase computational complexity and cause waste of computational resources during model training.In this paper,an improved Time Feedforward Connections Recurrent Neural Networks(TFC-RNNs)model was first proposed to address the gradient issue.A parallel branch was introduced for the hidden state at time t−2 to be directly transferred to time t without the nonlinear transforma-tion at time t−1.This is effective in improving the long-term dependence of RNNs.Then,a novel cell structure named Single Gate Recurrent Unit(SGRU)was presented.This cell structure can reduce the number of parameters for RNNs cell,consequently reducing the computational complexity.Next,applying SGRU to TFC-RNNs as a new TFC-SGRU model solves the above two difficulties.Finally,the performance of our proposed TFC-SGRU was verified through sev-eral experiments in terms of long-term memory and anti-interference capabilities.Experimental results demonstrated that our proposed TFC-SGRU model can cap-ture helpful information with time step 1500 and effectively filter out the noise.The TFC-SGRU model accuracy is better than the LSTM and GRU models regarding language processing ability.

基于对抗迁移学习模型的环境类虚假投诉检测

为实现环境类虚假投诉举报检测,提出一种基于对抗迁移学习方法的虚假投诉举报检测模型。首先,以长短期记忆(long-short term memory,LSTM)网络为特征抽取器抽取微博谣言(源域)和投诉举报文本(目标域)的共享特征;然后,使用对抗学习方法进行领域适配,将源域特征和目标域特征进行特征对齐;最后,由分类器输出分类结果,并由分类损失和领域适配损失共同更新网络参数。通过模型对比实验和消融实验可知,模型的F_(1)达到了79.61%。结果表明,对抗迁移学习模型具有较好的性能,适合应用在环境类虚假投诉举报检测任务中。

ST-Trader:A Spatial-Temporal Deep Neural Network for Modeling Stock Market Movement

Stocks that are fundamentally connected with each other tend to move together.Considering such common trends is believed to benefit stock movement forecasting tasks.However,such signals are not trivial to model because the connections among stocks are not physically presented and need to be estimated from volatile data.Motivated by this observation,we propose a framework that incorporates the inter-connection of firms to forecast stock prices.To effectively utilize a large set of fundamental features,we further design a novel pipeline.First,we use variational autoencoder(VAE)to reduce the dimension of stock fundamental information and then cluster stocks into a graph structure(fundamentally clustering).Second,a hybrid model of graph convolutional network and long-short term memory network(GCN-LSTM)with an adjacency graph matrix(learnt from VAE)is proposed for graph-structured stock market forecasting.Experiments on minute-level U.S.stock market data demonstrate that our model effectively captures both spatial and temporal signals and achieves superior improvement over baseline methods.The proposed model is promising for other applications in which there is a possible but hidden spatial dependency to improve time-series prediction.

A Spatial-Temporal Attention Model for Human Trajectory Prediction

Human trajectory prediction is essential and promising in many related applications. This is challenging due to the uncertainty of human behaviors, which can be influenced not only by himself, but also by the surrounding environment. Recent works based on long-short term memory(LSTM) models have brought tremendous improvements on the task of trajectory prediction. However, most of them focus on the spatial influence of humans but ignore the temporal influence. In this paper, we propose a novel spatial-temporal attention(ST-Attention) model,which studies spatial and temporal affinities jointly. Specifically,we introduce an attention mechanism to extract temporal affinity,learning the importance for historical trajectory information at different time instants. To explore spatial affinity, a deep neural network is employed to measure different importance of the neighbors. Experimental results show that our method achieves competitive performance compared with state-of-the-art methods on publicly available datasets.

A Visual-Based Gesture Prediction Framework Applied in Social Robots

In daily life,people use their hands in various ways for most daily activities.There are many applications based on the position,direction,and joints of the hand,including gesture recognition,gesture prediction,robotics and so on.This paper proposes a gesture prediction system that uses hand joint coordinate features collected by the Leap Motion to predict dynamic hand gestures.The model is applied to the NAO robot to verify the effectiveness of the proposed method.First of all,in order to reduce jitter or jump generated in the process of data acquisition by the Leap Motion,the Kalman filter is applied to the original data.Then some new feature descriptors are introduced.The length feature,angle feature and angular velocity feature are extracted from the filtered data.These features are fed into the long-short time memory recurrent neural network(LSTM-RNN)with different combinations.Experimental results show that the combination of coordinate,length and angle features achieves the highest accuracy of 99.31%,and it can also run in real time.Finally,the trained model is applied to the NAO robot to play the finger-guessing game.Based on the predicted gesture,the NAO robot can respond in advance.

Enhanced Accuracy for Motor Imagery Detection Using Deep Learning for BCI

Brain-Computer Interface(BCI)is a system that provides a link between the brain of humans and the hardware directly.The recorded brain data is converted directly to the machine that can be used to control external devices.There are four major components of the BCI system:acquiring signals,preprocessing of acquired signals,features extraction,and classification.In traditional machine learning algorithms,the accuracy is insignificant and not up to the mark for the classification of multi-class motor imagery data.The major reason for this is,features are selected manually,and we are not able to get those features that give higher accuracy results.In this study,motor imagery(MI)signals have been classified using different deep learning algorithms.We have explored two different methods:Artificial Neural Network(ANN)and Long Short-Term Memory(LSTM).We test the classification accuracy on two datasets:BCI competition III-dataset IIIa and BCI competition IV-dataset IIa.The outcome proved that deep learning algorithms provide greater accuracy results than traditional machine learning algorithms.Amongst the deep learning classifiers,LSTM outperforms the ANN and gives higher classification accuracy of 96.2%.

Customer Churn Prediction Model Based on User Behavior Sequences

Customer churn prediction model refers to a certain algorithm model that can predict in advance whether the current subscriber will terminate the contract with the current operator in the future.Many scholars currently introduce different depth models for customer churn prediction research,but deep modeling research on the features of historical behavior sequences generated by users over time is lacked.In this paper,a customer churn prediction model based on user behavior sequences is proposed.In this method,a long-short term memory(LSTM)network is introduced to learn the overall interest preferences of user behavior sequences.And the multi-headed attention mechanism is used to learn the collaborative information between multiple behaviors of users from multiple perspectives and to carry out the capture of information about various features of users.Experimentally validated on a real telecom dataset,the method has better prediction performance and further enhances the capability of the customer churn prediction system.

Using the CVP Traffic Detection Model at Road-Section Applies to Traffic Information Collection and Monitor—the Case Study

This paper proposes a using Cellular-Based Vehicle Probe(CVP)at road-section(RS)method to detect and setup a model for traffic flow information(info)collection and monitor.There are multiple traffic collection devices including CVP,ETC-Based Vehicle Probe(EVP),Vehicle Detector(VD),and CCTV as traffic resources to serve as road condition info for predicting the traffic jam problem,monitor and control.The main project has been applied at Tai#2 Ghee-Jing roadway connects to Wan-Li section as a trial field on fiscal year of 2017-2018.This paper proposes a man-flow turning into traffic-flow with Long-Short Time Memory(LTSM)from recurrent neural network(RNN)model.We also provide a model verification and validation methodology with RNN for cross verification of system performance.

Prediction of solar activities:Sunspot numbers and solar magnetic synoptic maps

The evolution of solar magnetic fields is significant for understanding and predicting solar activities.And our knowledge of solar magnetic fields largely depends on the photospheric magnetic field.In this paper,based on the spherical harmonic expansion of the photospheric magnetic field observed by Wilcox Solar Observatory,we analyze the time series of spherical harmonic coefficients and predict Sunspot Number as well as synoptic maps for Solar Cycle 25.We find that solar maximum years have complex short-period disturbances,and the time series of coefficient g_(7)~0 is nearly in-phase with Sunspot Number,which may be related to solar meridional circulation.Utilizing Long Short-Term Memory networks(LSTM),our prediction suggests that the maximum of Solar Cycle 25 is likely to occur in June 2024 with an error of 8 months,the peak sunspot number may be 166.9±22.6,and the next solar minimum may occur around January 2031.By incorporating Empirical Mode Decomposition,we enhance our forecast of synoptic maps truncated to Order 5,validating their relative reliability.This prediction not only addresses a gap in forecasting the global distribution of the solar magnetic field but also holds potential reference value for forthcoming solar observation plans.

A hybrid Wavelet-CNN-LSTM deep learning model for short-term urban water demand forecasting

Short-term water demand forecasting provides guidance on real-time water allocation in the water supply network, which help water utilities reduce energy cost and avoid potential accidents. Although a variety of methods have been proposed to improve forecast accuracy, it is still difficult for statistical models to learn the periodic patterns due to the chaotic nature of the water demand data with high temporal resolution. To overcome this issue from the perspective of improving data predictability, we proposed a hybrid Wavelet-CNN-LSTM model, that combines time-frequency decomposition characteristics of Wavelet Multi-Resolution Analysis (MRA) and implement it into an advanced deep learning model, CNN-LSTM. Four models - ANN, Conv1D, LSTM, GRUN - are used to compare with Wavelet-CNN-LSTM, and the results show that Wavelet-CNN-LSTM outperforms the other models both in single-step and multi-steps prediction. Besides, further mechanistic analysis revealed that MRA produce significant effect on improving model accuracy.

Trajectory prediction in pipeline form for intercepting hypersonic gliding vehicles based on LSTM

The interception problem of Hypersonic Gliding Vehicles(HGVs)has been an important aspect of missile defense systems.In order to provide interceptors with accurate information of target trajectory,a model based on an improved Long Short-Time Memory(LSTM)network for trajectory prediction pipeline is proposed for the interception of a skip gliding hypersonic target.Firstly,for trajectory prediction required by intercepting guidance laws,the altitude,velocity and velocity direction of the target are formulated in the form of analytic functions,consisting of linear decay terms and amplitude decay sinusoidal terms.Then,the dynamic characteristics of the model parameters are analyzed,and the target trajectory prediction pipeline is proposed with the prediction error considered.Finally,an improved LSTM network is designed to estimate parameters in a dynamically-updated manner,and estimation results are used for the calculation of the final trajectory prediction pipeline.The proposed prediction algorithm provides information on the velocity vector for midcourse guidance with the effect of prediction errors on interception taken into account.Simulation is conducted and the results show the high accuracy of the algorithm in HGVs’trajectory prediction which is conducive to increasing the interception success rate.