TSCND:Temporal Subsequence-Based Convolutional Network with Difference for Time Series Forecasting

Time series forecasting plays an important role in various fields, such as energy, finance, transport, and weather. Temporal convolutional networks (TCNs) based on dilated causal convolution have been widely used in time series forecasting. However, two problems weaken the performance of TCNs. One is that in dilated casual convolution, causal convolution leads to the receptive fields of outputs being concentrated in the earlier part of the input sequence, whereas the recent input information will be severely lost. The other is that the distribution shift problem in time series has not been adequately solved. To address the first problem, we propose a subsequence-based dilated convolution method (SDC). By using multiple convolutional filters to convolve elements of neighboring subsequences, the method extracts temporal features from a growing receptive field via a growing subsequence rather than a single element. Ultimately, the receptive field of each output element can cover the whole input sequence. To address the second problem, we propose a difference and compensation method (DCM). The method reduces the discrepancies between and within the input sequences by difference operations and then compensates the outputs for the information lost due to difference operations. Based on SDC and DCM, we further construct a temporal subsequence-based convolutional network with difference (TSCND) for time series forecasting. The experimental results show that TSCND can reduce prediction mean squared error by 7.3% and save runtime, compared with state-of-the-art models and vanilla TCN.

AFSTGCN:Prediction for multivariate time series using an adaptive fused spatial-temporal graph convolutional network

The prediction for Multivariate Time Series(MTS)explores the interrelationships among variables at historical moments,extracts their relevant characteristics,and is widely used in finance,weather,complex industries and other fields.Furthermore,it is important to construct a digital twin system.However,existing methods do not take full advantage of the potential properties of variables,which results in poor predicted accuracy.In this paper,we propose the Adaptive Fused Spatial-Temporal Graph Convolutional Network(AFSTGCN).First,to address the problem of the unknown spatial-temporal structure,we construct the Adaptive Fused Spatial-Temporal Graph(AFSTG)layer.Specifically,we fuse the spatial-temporal graph based on the interrelationship of spatial graphs.Simultaneously,we construct the adaptive adjacency matrix of the spatial-temporal graph using node embedding methods.Subsequently,to overcome the insufficient extraction of disordered correlation features,we construct the Adaptive Fused Spatial-Temporal Graph Convolutional(AFSTGC)module.The module forces the reordering of disordered temporal,spatial and spatial-temporal dependencies into rule-like data.AFSTGCN dynamically and synchronously acquires potential temporal,spatial and spatial-temporal correlations,thereby fully extracting rich hierarchical feature information to enhance the predicted accuracy.Experiments on different types of MTS datasets demonstrate that the model achieves state-of-the-art single-step and multi-step performance compared with eight other deep learning models.

A Time Series Intrusion Detection Method Based on SSAE,TCN and Bi-LSTM

In the fast-evolving landscape of digital networks,the incidence of network intrusions has escalated alarmingly.Simultaneously,the crucial role of time series data in intrusion detection remains largely underappreciated,with most systems failing to capture the time-bound nuances of network traffic.This leads to compromised detection accuracy and overlooked temporal patterns.Addressing this gap,we introduce a novel SSAE-TCN-BiLSTM(STL)model that integrates time series analysis,significantly enhancing detection capabilities.Our approach reduces feature dimensionalitywith a Stacked Sparse Autoencoder(SSAE)and extracts temporally relevant features through a Temporal Convolutional Network(TCN)and Bidirectional Long Short-term Memory Network(Bi-LSTM).By meticulously adjusting time steps,we underscore the significance of temporal data in bolstering detection accuracy.On the UNSW-NB15 dataset,ourmodel achieved an F1-score of 99.49%,Accuracy of 99.43%,Precision of 99.38%,Recall of 99.60%,and an inference time of 4.24 s.For the CICDS2017 dataset,we recorded an F1-score of 99.53%,Accuracy of 99.62%,Precision of 99.27%,Recall of 99.79%,and an inference time of 5.72 s.These findings not only confirm the STL model’s superior performance but also its operational efficiency,underpinning its significance in real-world cybersecurity scenarios where rapid response is paramount.Our contribution represents a significant advance in cybersecurity,proposing a model that excels in accuracy and adaptability to the dynamic nature of network traffic,setting a new benchmark for intrusion detection systems.

Deep Learning for Financial Time Series Prediction:A State-of-the-Art Review of Standalone and HybridModels

Financial time series prediction,whether for classification or regression,has been a heated research topic over the last decade.While traditional machine learning algorithms have experienced mediocre results,deep learning has largely contributed to the elevation of the prediction performance.Currently,the most up-to-date review of advanced machine learning techniques for financial time series prediction is still lacking,making it challenging for finance domain experts and relevant practitioners to determine which model potentially performs better,what techniques and components are involved,and how themodel can be designed and implemented.This review article provides an overview of techniques,components and frameworks for financial time series prediction,with an emphasis on state-of-the-art deep learning models in the literature from2015 to 2023,including standalonemodels like convolutional neural networks(CNN)that are capable of extracting spatial dependencies within data,and long short-term memory(LSTM)that is designed for handling temporal dependencies;and hybrid models integrating CNN,LSTM,attention mechanism(AM)and other techniques.For illustration and comparison purposes,models proposed in recent studies are mapped to relevant elements of a generalized framework comprised of input,output,feature extraction,prediction,and related processes.Among the state-of-the-artmodels,hybrid models like CNNLSTMand CNN-LSTM-AM in general have been reported superior in performance to stand-alone models like the CNN-only model.Some remaining challenges have been discussed,including non-friendliness for finance domain experts,delayed prediction,domain knowledge negligence,lack of standards,and inability of real-time and highfrequency predictions.The principal contributions of this paper are to provide a one-stop guide for both academia and industry to review,compare and summarize technologies and recent advances in this area,to facilitate smooth and informed implementation,and to highlight future research directions.

Convolutional neural networks for time series classification

Time series classification is an important task in time series data mining, and has attracted great interests and tremendous efforts during last decades. However, it remains a challenging problem due to the nature of time series data: high dimensionality, large in data size and updating continuously. The deep learning techniques are explored to improve the performance of traditional feature-based approaches. Specifically, a novel convolutional neural network (CNN) framework is proposed for time series classification. Different from other feature-based classification approaches, CNN can discover and extract the suitable internal structure to generate deep features of the input time series automatically by using convolution and pooling operations. Two groups of experiments are conducted on simulated data sets and eight groups of experiments are conducted on real-world data sets from different application domains. The final experimental results show that the proposed method outperforms state-of-the-art methods for time series classification in terms of the classification accuracy and noise tolerance. © 1990-2011 Beijing Institute of Aerospace Information.

基于VMD-TCN-GRU模型的水质预测研究

为充分挖掘水质数据在短时震荡中的变化特征,提升预测模型的精度,提出一种基于VMD(变分模态分解)、TCN(卷积时间神经网络)及GRU(门控循环单元)组成的混合水质预测模型,采用VMD-TCN-GRU模型对汾河水库出水口高锰酸盐指数进行预测,并与此类研究中常见的SVR(支持向量回归)、LSTM(长短期记忆神经网络)、TCN和CNN-LSTM(卷积神经网络-长短期记忆神经网络)这4种模型预测结果对比表明:VMD-TCN-GRU模型能更好挖掘水质数据在短时震荡过程中的特征信息,提升水质预测精度;VMD-TCN-GRU模型的MAE(平均绝对误差)、RMSE(均方根误差)下降,R^(2)(确定系数)提高,其MAE、RMSE、R^(2)分别为0.0553、0.0717、0.9351;其预测性能优越,预测精度更高且拥有更强的泛化能力,可以应用于汾河水质预测。

An Improved Granulated Convolutional Neural Network Data Analysis Model for COVID-19 Prediction

As COVID-19 poses a major threat to people’s health and economy,there is an urgent need for forecasting methodologies that can anticipate its trajectory efficiently.In non-stationary time series forecasting jobs,there is frequently a hysteresis in the anticipated values relative to the real values.The multilayer deep-time convolutional network and a feature fusion network are combined in this paper’s proposal of an enhanced Multilayer Deep Time Convolutional Neural Network(MDTCNet)for COVID-19 prediction to address this problem.In particular,it is possible to record the deep features and temporal dependencies in uncertain time series,and the features may then be combined using a feature fusion network and a multilayer perceptron.Last but not least,the experimental verification is conducted on the prediction task of COVID-19 real daily confirmed cases in the world and the United States with uncertainty,realizing the short-term and long-term prediction of COVID-19 daily confirmed cases,and verifying the effectiveness and accuracy of the suggested prediction method,as well as reducing the hysteresis of the prediction results.

基于Swin-TCN融合网络的运动视频理解的研究

针对运动比赛中运动员行为识别的问题,提出一种基于Swin Transformer和TCN(时域卷积网络)的混合网络模型。首先对视频进行预处理得到视频帧,使用Swin模块提取视频帧中人体动作相关的空间特征,将输出的特征图的空间尺寸压缩,纵向拼接后交给TCN模块提取视频中人体动作的时序特征。为提高通道对于行为识别结果的贡献度,在TCN残差块中加入通道注意力机制,经过分类模块后得到最终结果。实验结果显示,该模型在UCF101上动作识别准确率达到了89.7%。

Multi-dimension and multi-modal rolling mill vibration prediction model based on multi-level network fusion

Mill vibration is a common problem in rolling production,which directly affects the thickness accuracy of the strip and may even lead to strip fracture accidents in serious cases.The existing vibration prediction models do not consider the features contained in the data,resulting in limited improvement of model accuracy.To address these challenges,this paper proposes a multi-dimensional multi-modal cold rolling vibration time series prediction model(MDMMVPM)based on the deep fusion of multi-level networks.In the model,the long-term and short-term modal features of multi-dimensional data are considered,and the appropriate prediction algorithms are selected for different data features.Based on the established prediction model,the effects of tension and rolling force on mill vibration are analyzed.Taking the 5th stand of a cold mill in a steel mill as the research object,the innovative model is applied to predict the mill vibration for the first time.The experimental results show that the correlation coefficient(R^(2))of the model proposed in this paper is 92.5%,and the root-mean-square error(RMSE)is 0.0011,which significantly improves the modeling accuracy compared with the existing models.The proposed model is also suitable for the hot rolling process,which provides a new method for the prediction of strip rolling vibration.

基于CEEMDAN和TCN的变压器油中溶解气体含量预测

准确预测油中溶解气体含量的变化趋势,对变压器的状态评价和寿命评估有着积极的作用。为了提高油中溶解气体预测的准确性,文中提出一种基于自适应噪声完备集合经验模态分解(complete ensemble empirical mode decomposition with adaptive noise,CEEMDAN)和时间卷积网络(time convolution network,TCN)的油中溶解气体预测方法。首先,通过CEEMDAN方法将油中溶解气体含量的原始序列分解为多个本征模态分量,并将其中的稳定分量与非稳定分量分离;其次,对本征模态分量分别建立TCN并预测未来趋势变化;最后,叠加TCN对各个本征模态分量的预测结果,重构得到原始序列的预测结果。实例分析表明,该预测方法的均方根误差、平均绝对误差、最大误差分别为1.01μL/L、1.53μL/L、5.54μL/L,相较于未采用CEEMDAN算法时分别减小了53.47%、41.18%、13.36%;在使用CEEMDAN的情况下,对比常用的递归神经网络,3种误差均最小。且对比现有油中溶解气体预测方法,文中提出的油中溶解气体预测方法具有更高的预测精度,可以为制定状态检修策略提供更有效的支撑。

基于多尺度LDTW和TCN的空间负荷预测方法

空间负荷预测为合理建设和使用变电站、馈线等提供了重要的指导,成为配电网规划中不可或缺的一部分。配电网规划的精细化产生了大量高分辨率的负荷数据,社会的快速发展使得地块的用电特征日趋复杂。当前的空间负荷预测没有充分考虑负荷数据之间的时间特性,且在预测过程中也未考虑到不同类型地块间可能存在的负荷峰值出现时间不一致问题。为此,提出一种空间负荷预测方法,通过基于多尺度限制对齐路径长度(LDTW)的谱聚类分析用户的负荷曲线在形状上的相似性,并提取不同地块的典型用电行为,以进一步分类确定同类型地块对应的同时率。多尺度LDTW通过限制序列之间匹配步长的上限来抑制病态匹配的产生,提高曲线相似性的综合评估能力。根据聚类结果筛选适合待预测区域的训练样本并构建基于时间卷积网络(TCN)的回归预测模型,将预测结果基于地块各自的同时率进行聚合,实现空间负荷预测。实验结果表明:该方法加强了对负荷曲线形状的分析和对不同类型地块同时率的区分,在聚类方面,DBI指数达到0.57,VI指数达到0.31;在预测方面,相对误差达到1.93%,决定系数达到0.941,相比其他典型方法均取得了较大改善。

基于情感分析的TCN-LSTM的股价预测

针对传统股价预测模型未考虑市场投资者情绪对股票价格的影响以及难以较好的处理股价预测任务的问题,提出融合情感特征的时序深度学习模型BERT-TCN-LSTM。首先,对从股吧爬取投资者的评论信息进行情绪分析,提取出每日情绪的平均值作为模型的输入;其次,将每日情感均值与股票价格数据、技术指标输入构建的TCN-LSTM模型中进行训练;最后,在沪深300以及四只个股股票数据的数据集上进行实验。结果表明,相较于时间卷积网络(TCN)、LSTM和CNN-LSTM,BERT-TCN-LSTM在沪深300数据集上的平均绝对误差(MAE)平均降低了54%。BERT-TCN-LSTM模型可以有效提升股票价格预测的精度。

基于IAPSO-Holt-TCN的时序瓦斯浓度预测模型

为了提升时序瓦斯浓度预测精度,提出1种基于特征组合(FCIH)-时间卷积网络(TCN)的预测模型。首先,基于粒子群(PSO)算法重构惯性权重和加速因子,设计自适应粒子群(IAPSO)寻优算法;然后,利用IAPSO优化霍尔特指数平滑(Holt)相关超参数,应用Holt生成时序瓦斯浓度的水平、趋势分量,并与时序瓦斯浓度历史数据构成特征组合,以此获取具有高度预测性的特征;接着,基于构建的特征组合,搭建FCIH-TCN时序瓦斯浓度预测框架;最后,采用多个模型进行对比实验。研究结果表明:使用IAPSO后,Holt预测模型的平均绝对误差下降0.019;FCIH作为模型输入有效提高LSTM、GRU及TCN模型的预测精度;FCIH-TCN的RMSE为0.05,MAE为0.035,其预测精度优于其他对比模型。研究结果可为矿井瓦斯灾害防治提供参考。

Accurate Multi-Scale Feature Fusion CNN for Time Series Classification in Smart Factory

Time series classification(TSC)has attracted various attention in the community of machine learning and data mining and has many successful applications such as fault detection and product identification in the process of building a smart factory.However,it is still challenging for the efficiency and accuracy of classification due to complexity,multi-dimension of time series.This paper presents a new approach for time series classification based on convolutional neural networks(CNN).The proposed method contains three parts:short-time gap feature extraction,multi-scale local feature learning,and global feature learning.In the process of short-time gap feature extraction,large kernel filters are employed to extract the features within the short-time gap from the raw time series.Then,a multi-scale feature extraction technique is applied in the process of multi-scale local feature learning to obtain detailed representations.The global convolution operation with giant stride is to obtain a robust and global feature representation.The comprehension features used for classifying are a fusion of short time gap feature representations,local multi-scale feature representations,and global feature representations.To test the efficiency of the proposed method named multi-scale feature fusion convolutional neural networks(MSFFCNN),we designed,trained MSFFCNN on some public sensors,device,and simulated control time series data sets.The comparative studies indicate our proposed MSFFCNN outperforms other alternatives,and we also provided a detailed analysis of the proposed MSFFCNN.

Remaining Useful Life Prediction of Aeroengine Based on Principal Component Analysis and One-Dimensional Convolutional Neural Network

In order to directly construct the mapping between multiple state parameters and remaining useful life(RUL),and reduce the interference of random error on prediction accuracy,a RUL prediction model of aeroengine based on principal component analysis(PCA)and one-dimensional convolution neural network(1D-CNN)is proposed in this paper.Firstly,multiple state parameters corresponding to massive cycles of aeroengine are collected and brought into PCA for dimensionality reduction,and principal components are extracted for further time series prediction.Secondly,the 1D-CNN model is constructed to directly study the mapping between principal components and RUL.Multiple convolution and pooling operations are applied for deep feature extraction,and the end-to-end RUL prediction of aeroengine can be realized.Experimental results show that the most effective principal component from the multiple state parameters can be obtained by PCA,and the long time series of multiple state parameters can be directly mapped to RUL by 1D-CNN,so as to improve the efficiency and accuracy of RUL prediction.Compared with other traditional models,the proposed method also has lower prediction error and better robustness.

A production prediction method of single well in water flooding oilfield based on integrated temporal convolutional network model

Since the oil production of single well in water flooding reservoir varies greatly and is hard to predict, an oil production prediction method of single well based on temporal convolutional network(TCN) is proposed and verified. This method is started from data processing, the correspondence between water injectors and oil producers is determined according to the influence radius of the water injectors, the influence degree of a water injector on an oil producer in the month concerned is added as a model feature, and a Random Forest(RF) model is built to fill the dynamic data of water flooding. The single well history is divided into 4 stages according to its water cut, that is, low water cut, middle water cut, high water cut and extra-high water cut stages. In each stage, a TCN based prediction model is established, hyperparameters of the model are optimized by the Sparrow Search Algorithm(SSA). Finally, the models of the 4 stages are integrated into one whole-life model of the well for production prediction. The application of this method in Daqing Oilfield, NE China shows that:(1) Compared with conventional data processing methods, the data obtained by this processing method are more close to the actual production, and the data set obtained is more authentic and complete.(2) The TCN model has higher prediction accuracy than other 11 models such as Long Short Term Memory(LSTM).(3) Compared with the conventional full-life-cycle models, the model of integrated stages can significantly reduce the error of production prediction.

基于XGBoost和TCN-Attention的棉花价格多影响因素选择及预测

棉花价格受多种因素影响而复杂多变,通过选择合适的数据特征和预测模型可提高棉花价格预测精度.本文以棉花日现货价格数据为研究目标,采集了供需关系、国际市场、宏观经济、产业链这4个方面的9项影响因素作为特征,使用极限梯度提升(XGBoost)算法对棉花价格影响因素进行特征评估筛选,选取其中5项特征后,采用引入注意力机制(Attention)的时间卷积网络(TCN) TCN-Attention、TCN、LSTM、GRU等模型对棉花价格进行预测.通过消融实验和对比实验,结果表明:(1)经过XGBoost特征筛选后, TCN-Attention价格预测的平均绝对误差(MAE)和均方根误差(RMSE)为41.47和58.76,与未筛选相比分别降低了77.57%和76.49%.(2)与TCN、LSTM、GRU相比,本文提出的TCN-Attention模型预测结果更准确, MAE和RMSE均降低50%以上,运行时间较LSTM、GRU缩短60%.

基于VMD和改进TCN的短期光伏发电功率预测

光伏发电功率存在波动性,且光伏出力易受各种气象特征影响,传统TCN网络容易过度强化空间特性而弱化个体特性。针对上述问题,文中提出一种基于VMD和改进TCN的短期光伏发电功率预测模型。通过VMD将原始光伏发电功率时间序列分解为若干不同频率的模态分量,将各个模态分量以及相对应的气象数据输入至改进TCN网络进行建模学习。利用中心频率法确定VMD的最优分解模态分解个数。在传统TCN预测模型的基础上,使用DropBlock正则化取代Dropout正则化以达到抑制卷积层中信息协同的效果,并引入注意力机制自主挖掘并突出关键气象输入特征的影响,量化各气象因素对光伏发电的影响,从而提高预测精度。以江苏省某光伏电站真实数据为例进行仿真实验,结果表明所提预测方法的RMSE为0.62 MW,MAPE为2.03%。

基于STL-LSTM-TCN模型的短期负荷预测方法

为提高模型的预测精度,引入STL算法将负荷序列分解为周期分量、趋势分量、残差分量,利用各分量训练LSTM和TCN模型。在得到LSTM和TCN模型的预测后,为进一步提高模型的预测精度,构建了集成模型对LSTM模型和TCN模型的预测值进行集成。以西班牙电力负荷数据集为例,对所提负荷预测方法进行了验证。实验结果表明,STL算法和集成模型的引入均提高了模型的预测精度,基于STL-LSTM-TCN的预测方法相较于LSTM、TCN、STL-LSTM、STL-TCN,其MAPE分别降低了2.8664%、2.1229%、0.37%、0.1%,所提负荷预测方法的预测误差最低,验证了所提预测方法的有效性与合理性。

基于SE-TCN的一维低采样卫星帆板温度遥测数据插补方法

针对因入境时间短、组帧错误等原因导致的卫星帆板温度遥测数据缺失问题,提出一种基于引入注意力机制的时间卷积网络(SE-TCN)的自回归预测方法。温度遥测数据可看作是具有较强规律性的渐周期信号,采用SE-TCN对历史数据到未来数据的映射进行拟合完成缺失值的插补,同时为表征对实际缺失数据集的插补效果,增加评价指标的计算方式,有效解决了使用物理模型仿真和统计学方法插值偏差过大,及无法计算实际插值效果的问题。与长短时记忆网络和时间卷积网络等模型相比,SE-TCN在测试集和实际缺失数据集上均得到了更好的插值效果。