TSCND:Temporal Subsequence-Based Convolutional Network with Difference for Time Series Forecasting

Time series forecasting plays an important role in various fields, such as energy, finance, transport, and weather. Temporal convolutional networks (TCNs) based on dilated causal convolution have been widely used in time series forecasting. However, two problems weaken the performance of TCNs. One is that in dilated casual convolution, causal convolution leads to the receptive fields of outputs being concentrated in the earlier part of the input sequence, whereas the recent input information will be severely lost. The other is that the distribution shift problem in time series has not been adequately solved. To address the first problem, we propose a subsequence-based dilated convolution method (SDC). By using multiple convolutional filters to convolve elements of neighboring subsequences, the method extracts temporal features from a growing receptive field via a growing subsequence rather than a single element. Ultimately, the receptive field of each output element can cover the whole input sequence. To address the second problem, we propose a difference and compensation method (DCM). The method reduces the discrepancies between and within the input sequences by difference operations and then compensates the outputs for the information lost due to difference operations. Based on SDC and DCM, we further construct a temporal subsequence-based convolutional network with difference (TSCND) for time series forecasting. The experimental results show that TSCND can reduce prediction mean squared error by 7.3% and save runtime, compared with state-of-the-art models and vanilla TCN.

AFSTGCN:Prediction for multivariate time series using an adaptive fused spatial-temporal graph convolutional network

The prediction for Multivariate Time Series(MTS)explores the interrelationships among variables at historical moments,extracts their relevant characteristics,and is widely used in finance,weather,complex industries and other fields.Furthermore,it is important to construct a digital twin system.However,existing methods do not take full advantage of the potential properties of variables,which results in poor predicted accuracy.In this paper,we propose the Adaptive Fused Spatial-Temporal Graph Convolutional Network(AFSTGCN).First,to address the problem of the unknown spatial-temporal structure,we construct the Adaptive Fused Spatial-Temporal Graph(AFSTG)layer.Specifically,we fuse the spatial-temporal graph based on the interrelationship of spatial graphs.Simultaneously,we construct the adaptive adjacency matrix of the spatial-temporal graph using node embedding methods.Subsequently,to overcome the insufficient extraction of disordered correlation features,we construct the Adaptive Fused Spatial-Temporal Graph Convolutional(AFSTGC)module.The module forces the reordering of disordered temporal,spatial and spatial-temporal dependencies into rule-like data.AFSTGCN dynamically and synchronously acquires potential temporal,spatial and spatial-temporal correlations,thereby fully extracting rich hierarchical feature information to enhance the predicted accuracy.Experiments on different types of MTS datasets demonstrate that the model achieves state-of-the-art single-step and multi-step performance compared with eight other deep learning models.

A Time Series Intrusion Detection Method Based on SSAE,TCN and Bi-LSTM

In the fast-evolving landscape of digital networks,the incidence of network intrusions has escalated alarmingly.Simultaneously,the crucial role of time series data in intrusion detection remains largely underappreciated,with most systems failing to capture the time-bound nuances of network traffic.This leads to compromised detection accuracy and overlooked temporal patterns.Addressing this gap,we introduce a novel SSAE-TCN-BiLSTM(STL)model that integrates time series analysis,significantly enhancing detection capabilities.Our approach reduces feature dimensionalitywith a Stacked Sparse Autoencoder(SSAE)and extracts temporally relevant features through a Temporal Convolutional Network(TCN)and Bidirectional Long Short-term Memory Network(Bi-LSTM).By meticulously adjusting time steps,we underscore the significance of temporal data in bolstering detection accuracy.On the UNSW-NB15 dataset,ourmodel achieved an F1-score of 99.49%,Accuracy of 99.43%,Precision of 99.38%,Recall of 99.60%,and an inference time of 4.24 s.For the CICDS2017 dataset,we recorded an F1-score of 99.53%,Accuracy of 99.62%,Precision of 99.27%,Recall of 99.79%,and an inference time of 5.72 s.These findings not only confirm the STL model’s superior performance but also its operational efficiency,underpinning its significance in real-world cybersecurity scenarios where rapid response is paramount.Our contribution represents a significant advance in cybersecurity,proposing a model that excels in accuracy and adaptability to the dynamic nature of network traffic,setting a new benchmark for intrusion detection systems.

基于TCN和迁移学习的混凝土坝变形预测方法

混凝土坝变形测点数据丢失或者新增测点测量时间太短都会导致这部分测点的数据量不足,使得变形预测精度受到影响。为了提高这些小数据量测点的变形预测精度,提出了将时域卷积网络(TCN)与迁移学习相结合的变形预测方法。以数据量充足的测点为源域,以缺少数据的测点为目标域,将在源域上训练好的TCN模型的结构和参数迁移到目标域模型中,固定其中的冻结层参数,利用目标域中的数据对目标域模型可调层参数进行调整。同时,采用动态时间规整选择与目标域数据序列相似度最高的监测数据作为最佳源域数据,提升迁移学习效果。工程实例分析表明:迁移学习后的目标域模型的均方根误差和平均绝对误差与利用足量数据训练的TCN模型的预测误差相比,差异仅分别为1.73%和8.09%,小数据量情况下TCN预测模型的精度得到了提高。

基于参数自适应SVR和VMD-TCN的水电机组劣化趋势预测

针对水电机组难以利用实时监测数据对机组劣化状态进行有效评估,以及水电机组不同运行工况对运行状态指标趋势预测模型参数影响显著的问题,提出一种基于参数自适应支持向量回归机(SVR)、变分模态分解(VMD)和时间卷积网络(TCN)的水电机组劣化趋势预测方法;首先按照功率和水头将机组运行工况细化为若干典型工况,在此基础上采用改进天鹰算法建立SVR模型,对各个工况下的预测参数进行寻优,建立起工况与最优参数的数据;再通过神经网络对工况和最优预测参数进行拟合,构建出映射两者复杂关系的非线性函数,然后将构建出的映射关系加入到传统的SVR中,实现适应于水电机组工况变化的自适应SVR健康模型;其次,根据健康模型输出的标准值和监测数据,计算出劣化趋势序列;最后,考虑到劣化趋势序列的非线性因素,建立了一个基于VMD-TCN的时间序列预测模型,以实现对劣化趋势的准确预测。并设计多组对比实验,验证所提出模型的精度更高,时间更快。

基于VMD-TCN-GRU模型的水质预测研究

为充分挖掘水质数据在短时震荡中的变化特征,提升预测模型的精度,提出一种基于VMD(变分模态分解)、TCN(卷积时间神经网络)及GRU(门控循环单元)组成的混合水质预测模型,采用VMD-TCN-GRU模型对汾河水库出水口高锰酸盐指数进行预测,并与此类研究中常见的SVR(支持向量回归)、LSTM(长短期记忆神经网络)、TCN和CNN-LSTM(卷积神经网络-长短期记忆神经网络)这4种模型预测结果对比表明:VMD-TCN-GRU模型能更好挖掘水质数据在短时震荡过程中的特征信息,提升水质预测精度;VMD-TCN-GRU模型的MAE(平均绝对误差)、RMSE(均方根误差)下降,R^(2)(确定系数)提高,其MAE、RMSE、R^(2)分别为0.0553、0.0717、0.9351;其预测性能优越,预测精度更高且拥有更强的泛化能力,可以应用于汾河水质预测。

基于ARIMA-TCN混合模型的高速铁路时间同步方法

列控系统作为高速铁路的核心系统,保持其系统的时间同步对于行车安全至关重要。针对现有时间同步方法易受时变上下行传输时延、随机时钟跳变等影响,导致主从时钟偏移估计不准确的问题,提出一种基于差分自回归移动平均-时域卷积神经网络(ARIMA-TCN)混合模型的高速铁路时间同步方法。首先,根据上下行链路传输速率的不对称比,建立高速铁路时钟的数学理论和实际观测模型。然后,使用拉依达准则识别处理跳变异常值,完成实际时间序列的预处理。再次,使用ARIMA模型平滑时间序列中不确定时延带来的噪声抖动,获得平稳的时间序列。最后,通过提出的注意力增强TCN模型进行预测补偿,完成时钟偏移的补偿校正。通过实验仿真,得到基站区间内位置、基站间距以及车速对高速铁路时间同步的影响性分析。实验结果表明:与对比方法相比,所提方法补偿后的均方根误差较最小二乘法减少了75%、较最大似然估计方法误差减少了44.4%,较BP神经网络方法误差减少了16.7%,验证所提方法具有更低的同步误差和更高的同步精度。

基于TCN的跟网/构网混合型新能源场站并网系统小干扰稳定性快速评估

为支撑跟网/构网混合型新能源场站中机组控制方式快速切换,实现可适应电网强度变化的新能源场站安全稳定运行,提出基于时间卷积神经网络(temporal convolutional network,TCN)的跟网/构网混合型新能源场站并网系统小干扰稳定性快速评估方法。首先,构建跟网/构网混合型新能源场站聚合阻抗模型,通过特征值计算得到并网系统小干扰稳定裕度。然后,以并网系统短路比和新能源场站跟网/构网控制方式信息作为输入特征,以并网系统小干扰稳定裕度和阻尼比作为输出特征,训练TCN得到混合型新能源场站并网系统小干扰稳定性快速评估模型。经过训练的模型可根据短路比和跟网/构网混合型新能源场站中各机组的控制方式快速输出对应的小干扰稳定裕度和阻尼比。最后,以一个含10台风电机组的新能源场站为对象进行算例分析。结果表明:所提TCN方法相比于长短期记忆神经网络方法,在小干扰稳定裕度和阻尼比预测上的平均绝对百分比误差分别降低16.76%、14.75%;所提方法的计算耗时相对于特征值计算方法降低98.54%,从而验证所提小干扰稳定性快速评估方法的准确性与时效性。

融合距离阈值和双向TCN的时空注意力行人轨迹预测模型

为解决因缺乏部分行人建模思想、缺少时间维度的全局视野和忽略行人交互模式多样性,而导致交互建模不充分、低预测精度等问题,提出基于Social-STGCNN(social spatio-temporal graph convolutional neural network)的改进模型STG-DTBTA(spatio-temporal graph distance threshold Bi-TCN attention)。首先,构建PPM(partial pedestrian module)模块,对不满足距离阈值等约束条件的行人交互连接剪枝以去噪。其次,引入时空注意力机制,空间注意力动态分配交互权重,并设置多个注意力头以处理交互多样性问题;时间注意力捕捉时序数据的时间依赖关系。最后,采用双向TCN增加全局视野以捕捉轨迹数据中的动态模式和趋势,并采用门控机制融合双向特征。在ETH和UCY数据集上的实验结果表明,与Social-STGCNN相比,STG-DTBTA在维持参数量与推理时间接近的情况下,ADE平均降低8%,FDE平均降低16%。STG-DTBTA具有良好的交互建模能力、模型性能和预测效果。

基于STE-TCN的中短期电力负荷预测

目的 针对传统电力负荷预测模型对长序列预测精度低的问题,提出一种结合跳级卷积连接与时间编码网络的新型时序卷积神经网络(TCN)模型——STE-TCN模型。方法 首先对TCN模型加入跨周期的膨胀卷积通道(Skip-convolution)提取电力数据周期信息;再进行特征融合得到Skip-TCN网络,使网络抓取周期规律,增加信息利用长度;最后设计日期编码网络(Time encoding network)捕捉生活周期和季节性特征,与Skip-TCN进行特征融合得到STE-TCN模型,实现对电力负荷数据长序列预测。结果 实验表明:在与TCN模型和传统时序网络的对比下,Skip-TCN的预测精度均有提升,在预测长度更长的测试上提升尤为明显。结论 实验结果验证了通过对更长跨度时序关系的捕捉,STE-TCN网络改进方法有效提升了对长序列电力负荷的预测精度。

基于MsTCN-Transformer模型的轴承剩余使用寿命预测研究

剩余使用寿命(remaining useful life, RUL)预测是PHM的核心问题之一,复杂的运行工况往往导致设备部件经历不同的故障退化过程,给RUL准确预测带来了巨大挑战。为此,提出了一种多尺度时间卷积网络(multi-scale temporal convolutional network, MsTCN)与Transformer(MsTCN-Transformer)融合模型用于变工况下滚动轴承RUL预测。该方法设计了一种新的多尺度膨胀因果卷积单元(multi-scale dilated causal convolution unit, MsDCCU),能够自适应地挖掘滚动轴承全寿命信号中固有的时序特征信息;然后构建了基于自注意力机制的Transformer网络模型,在克服预测序列记忆力退化的基础上,准确学习时序特征与轴承RUL之间的映射关系。此外,通过对轴承不同故障退化阶段所提取的时序特征可视化分析,验证了所提方法在变工况下所提取的时序特征泛化性较好。多种工况条件下滚动轴承RUL预测试验表明,所提方法能够较为准确地实现变工况下轴承的RUL预测,相比当前多种方法RUL预测结果准确性更高。

基于Bi-TCN-LSTM的滚动轴承剩余使用寿命预测方法

由于时间卷积网络(temporal convolutional networks, TCN)感知场不足,轴承的关键退化信息常常被忽略,导致轴承剩余使用寿命(remaining useful life, RUL)预测结果不佳;而长短期记忆网络(long short-term memory, LSTM)随着数据量及序列长度的增加,长期依赖问题仍可能得不到很好解决。因此,课题组提出了一种基于双向时间卷积网络和长短期记忆(Bi-TCN-LSTM)的滚动轴承寿命预测方法。首先对多传感器数据进行归一化并做融合处理,然后采用Bi-TCN-LSTM进行数据特征提取与深度学习,其中对TCN模块引入卷积注意力机制(convolutional attention module, CAM),将LSTM的3个门简化为1个门,有效加快了预测模型学习的速度并提高了预测模型的精确度;采用IEEE PHM 2012轴承数据集作为实验数据集,进行了RUL预测实验。结果表明:与其他先进的预测模型相比,Bi-TCN-LSTM方法预测结果的误差相对较低,预测性能较好。

Convolutional neural networks for time series classification

Time series classification is an important task in time series data mining, and has attracted great interests and tremendous efforts during last decades. However, it remains a challenging problem due to the nature of time series data: high dimensionality, large in data size and updating continuously. The deep learning techniques are explored to improve the performance of traditional feature-based approaches. Specifically, a novel convolutional neural network (CNN) framework is proposed for time series classification. Different from other feature-based classification approaches, CNN can discover and extract the suitable internal structure to generate deep features of the input time series automatically by using convolution and pooling operations. Two groups of experiments are conducted on simulated data sets and eight groups of experiments are conducted on real-world data sets from different application domains. The final experimental results show that the proposed method outperforms state-of-the-art methods for time series classification in terms of the classification accuracy and noise tolerance. © 1990-2011 Beijing Institute of Aerospace Information.

结合CNN与TCN神经网络的滚动轴承寿命预测

针对滚动轴承振动信号特征提取不充分、过于依赖人工特征提取及预测精度低等问题,提出了CNN-TCN-Attention网络模型预测方法。该方法选取滚动轴承振动信号作为输入通过增强顶帽算子(EAVGH)对信号进行特征增强,运用卷积神经网络(CNN)来提取信号中的深层特征,并构建TCN-Attention模型对滚动轴承剩余寿命进行预测。将注意力机制与时间卷积网络相结合可以有效的提高模型预测精度,通过轴承寿命实验数据进行验证,CNN-TCN-Attention预测模型能有效的提取滚动轴承振动信号中的深层特征,并且具有较高的预测精度。

基于焦点损失的ATCN-GRU语音情感识别

为了改善RNN的空间信息丢失和CNN忽略时序信息的问题,引入了时间卷积网络TCN,将上述网络与双向门控循环单元Bi-GRU以及注意力机制组合构建了声学模型ATCN-GRU来进一步提高语音情感识别的性能,并通过加入焦点损失改善EMODB和IEMOCAP数据库训练样本不平均导致的识别结果不均衡问题。首先,通过TCN残差块从手工提取的特征中选取最具有代表性和鲁棒性的特征;其次,利用Bi-GRU模型学习语音样本的上下文相关信息,并利用注意力机制学习模型的输入序列与输出序列之间的关联程度,从而给予有效信息更多关注;最后,通过Softmax层对情感进行分类。相较于前人的研究成果,模型ATCN-GRU取得了更好的识别性能:在CASIA、EMODB以及IEMOCAP三个数据库上分别取得了88.17%、85.98%和65.83%的平均准确率;引入焦点损失后,EMODB和IEMOCAP数据库上的平均准确率分别达到了86.26%和66.30%。

An Improved Granulated Convolutional Neural Network Data Analysis Model for COVID-19 Prediction

As COVID-19 poses a major threat to people’s health and economy,there is an urgent need for forecasting methodologies that can anticipate its trajectory efficiently.In non-stationary time series forecasting jobs,there is frequently a hysteresis in the anticipated values relative to the real values.The multilayer deep-time convolutional network and a feature fusion network are combined in this paper’s proposal of an enhanced Multilayer Deep Time Convolutional Neural Network(MDTCNet)for COVID-19 prediction to address this problem.In particular,it is possible to record the deep features and temporal dependencies in uncertain time series,and the features may then be combined using a feature fusion network and a multilayer perceptron.Last but not least,the experimental verification is conducted on the prediction task of COVID-19 real daily confirmed cases in the world and the United States with uncertainty,realizing the short-term and long-term prediction of COVID-19 daily confirmed cases,and verifying the effectiveness and accuracy of the suggested prediction method,as well as reducing the hysteresis of the prediction results.

Motor Fault Diagnosis Based on Short-time Fourier Transform and Convolutional Neural Network

With the rapid development of mechanical equipment, the mechanical health monitoring field has entered the era of big data. However, the method of manual feature extraction has the disadvantages of low efficiency and poor accuracy, when handling big data. In this study, the research object was the asynchronous motor in the drivetrain diagnostics simulator system. The vibration signals of different fault motors were collected. The raw signal was pretreated using short time Fourier transform （STFT） to obtain the corresponding time-frequency map. Then, the feature of the time-frequency map was adap- tively extracted by using a convolutional neural network （CNN）. The effects of the pretreatment method, and the hyper parameters of network diagnostic accuracy, were investigated experimentally. The experimental results showed that the influence of the preprocessing method is small, and that the batch-size is the main factor affecting accuracy and training efficiency. By investigating feature visualization, it was shown that, in the case of big data, the extracted CNN features can represent complex mapping relationships between signal and health status, and can also overcome the prior knowledge and engineering experience requirement for feature extraction, which is used by tra- ditional diagnosis methods. This paper proposes a new method, based on STFT and CNN, which can complete motor fault diagnosis tasks more intelligently and accurately.

基于改进TCN的上扣扭矩序列数据分类

在油气开发领域,油套管安装后的密封性能检测尤为重要。其中,上扣过程中产生的扭矩序列数据可以作为油套管密封性的评判依据,用来判断上扣是否合格。为了利用上扣扭矩序列数据信息进行油套管密封性的识别分类,首先基于TCN网络模型结构,再融入位置编码机制和自注意力机制,搭建了一种新的网络模型,即PSE-TCN网络。通过比较不同策略下的结果准确率,展示了模型学习的过程,通过与其他网络模型进行对比,验证了本方法的有效性。实验结果表明,PSE-TCN相较于其他经典网络模型和一些改进后的TCN网络模型,扭矩序列识别精度有较大提升,在自制UCR_whorl数据集上,模型识别准确率达到93.41%。

基于TCN和残差自注意力的变工况下滚动轴承剩余寿命迁移预测

针对变工况环境下采集到的滚动轴承寿命状态数据存在特征分布差异,深度神经网络模型泛化能力差的问题,结合时间卷积网络(temporal convolutional neural network,TCN)和残差自注意力机制提出了一种端到端的滚动轴承剩余寿命(remaining useful life,RUL)迁移预测方法。首先,将传感器采集到的一维时域信号利用短时傅里叶变换转换为频域信号;其次,剩余寿命迁移预测网络通用特征提取层采用残差自注意力TCN网络,该网络在较好提取时间序列信息的同时,进一步通过残差自注意力机制捕获轴承局部退化特征,增强模型的迁移特征提取能力;再次,采用提出的联合领域自适应策略匹配变工况下滚动轴承寿命状态数据特征分布差异,实现不同工况下轴承寿命状态知识的迁移预测;最后,在公开的滚动轴承全寿命数据集上进行试验验证,结果表明所提方法能有效实现变工况下的滚动轴承RUL预测,并获得较好的预测性能。

基于TCN-自适应的地下洞室围岩变形异常数据识别

水电站地下洞室围岩变形数据具有变化不确定、序列样本短等特点,传统的异常识别方法漏识率、误判率较高。随着智能技术的发展,通过神经网络建立更加可靠的异常识别方法是目前研究的热点,而传统的神经网络存在时序关联性不强和计算模型庞杂等问题。为此,提出了基于时域卷积神经网络(TCN)及标准自适应的地下洞室异常数据识别算法,该算法利用TCN技术,考虑序列的前后关系,建立了更为可靠的序列模型;同时针对地下洞室监测数据特征,通过考虑误差中位数、数据波动和仪器精度3个方面,突现自适应匹配最优识别准则。将该算法应用在叶巴滩水电站地下洞室围岩变形的异常数据识别中,证明了其可有效避免梯度爆炸、消失,模型耗时较长等问题,极大地提高了异常值分析效率和识别率。相关经验可供类似工程异常监测数据识别时借鉴。