TSCND:Temporal Subsequence-Based Convolutional Network with Difference for Time Series Forecasting

Time series forecasting plays an important role in various fields, such as energy, finance, transport, and weather. Temporal convolutional networks (TCNs) based on dilated causal convolution have been widely used in time series forecasting. However, two problems weaken the performance of TCNs. One is that in dilated casual convolution, causal convolution leads to the receptive fields of outputs being concentrated in the earlier part of the input sequence, whereas the recent input information will be severely lost. The other is that the distribution shift problem in time series has not been adequately solved. To address the first problem, we propose a subsequence-based dilated convolution method (SDC). By using multiple convolutional filters to convolve elements of neighboring subsequences, the method extracts temporal features from a growing receptive field via a growing subsequence rather than a single element. Ultimately, the receptive field of each output element can cover the whole input sequence. To address the second problem, we propose a difference and compensation method (DCM). The method reduces the discrepancies between and within the input sequences by difference operations and then compensates the outputs for the information lost due to difference operations. Based on SDC and DCM, we further construct a temporal subsequence-based convolutional network with difference (TSCND) for time series forecasting. The experimental results show that TSCND can reduce prediction mean squared error by 7.3% and save runtime, compared with state-of-the-art models and vanilla TCN.

Using BlazePose on Spatial Temporal Graph Convolutional Networks for Action Recognition

The ever-growing available visual data(i.e.,uploaded videos and pictures by internet users)has attracted the research community’s attention in the computer vision field.Therefore,finding efficient solutions to extract knowledge from these sources is imperative.Recently,the BlazePose system has been released for skeleton extraction from images oriented to mobile devices.With this skeleton graph representation in place,a Spatial-Temporal Graph Convolutional Network can be implemented to predict the action.We hypothesize that just by changing the skeleton input data for a different set of joints that offers more information about the action of interest,it is possible to increase the performance of the Spatial-Temporal Graph Convolutional Network for HAR tasks.Hence,in this study,we present the first implementation of the BlazePose skeleton topology upon this architecture for action recognition.Moreover,we propose the Enhanced-BlazePose topology that can achieve better results than its predecessor.Additionally,we propose different skeleton detection thresholds that can improve the accuracy performance even further.We reached a top-1 accuracy performance of 40.1%on the Kinetics dataset.For the NTU-RGB+D dataset,we achieved 87.59%and 92.1%accuracy for Cross-Subject and Cross-View evaluation criteria,respectively.

基于TCN和迁移学习的混凝土坝变形预测方法

混凝土坝变形测点数据丢失或者新增测点测量时间太短都会导致这部分测点的数据量不足,使得变形预测精度受到影响。为了提高这些小数据量测点的变形预测精度,提出了将时域卷积网络(TCN)与迁移学习相结合的变形预测方法。以数据量充足的测点为源域,以缺少数据的测点为目标域,将在源域上训练好的TCN模型的结构和参数迁移到目标域模型中,固定其中的冻结层参数,利用目标域中的数据对目标域模型可调层参数进行调整。同时,采用动态时间规整选择与目标域数据序列相似度最高的监测数据作为最佳源域数据,提升迁移学习效果。工程实例分析表明:迁移学习后的目标域模型的均方根误差和平均绝对误差与利用足量数据训练的TCN模型的预测误差相比,差异仅分别为1.73%和8.09%,小数据量情况下TCN预测模型的精度得到了提高。

基于ARIMA-TCN混合模型的高速铁路时间同步方法

列控系统作为高速铁路的核心系统,保持其系统的时间同步对于行车安全至关重要。针对现有时间同步方法易受时变上下行传输时延、随机时钟跳变等影响,导致主从时钟偏移估计不准确的问题,提出一种基于差分自回归移动平均-时域卷积神经网络(ARIMA-TCN)混合模型的高速铁路时间同步方法。首先,根据上下行链路传输速率的不对称比,建立高速铁路时钟的数学理论和实际观测模型。然后,使用拉依达准则识别处理跳变异常值,完成实际时间序列的预处理。再次,使用ARIMA模型平滑时间序列中不确定时延带来的噪声抖动,获得平稳的时间序列。最后,通过提出的注意力增强TCN模型进行预测补偿,完成时钟偏移的补偿校正。通过实验仿真,得到基站区间内位置、基站间距以及车速对高速铁路时间同步的影响性分析。实验结果表明:与对比方法相比,所提方法补偿后的均方根误差较最小二乘法减少了75%、较最大似然估计方法误差减少了44.4%,较BP神经网络方法误差减少了16.7%,验证所提方法具有更低的同步误差和更高的同步精度。

基于VMD-TCN-GRU模型的水质预测研究

为充分挖掘水质数据在短时震荡中的变化特征,提升预测模型的精度,提出一种基于VMD(变分模态分解)、TCN(卷积时间神经网络)及GRU(门控循环单元)组成的混合水质预测模型,采用VMD-TCN-GRU模型对汾河水库出水口高锰酸盐指数进行预测,并与此类研究中常见的SVR(支持向量回归)、LSTM(长短期记忆神经网络)、TCN和CNN-LSTM(卷积神经网络-长短期记忆神经网络)这4种模型预测结果对比表明:VMD-TCN-GRU模型能更好挖掘水质数据在短时震荡过程中的特征信息,提升水质预测精度;VMD-TCN-GRU模型的MAE(平均绝对误差)、RMSE(均方根误差)下降,R^(2)(确定系数)提高,其MAE、RMSE、R^(2)分别为0.0553、0.0717、0.9351;其预测性能优越,预测精度更高且拥有更强的泛化能力,可以应用于汾河水质预测。

基于TCN-BiLSTM网络的电力电缆故障诊断

为了提升电力电缆故障诊断技术的准确率,解决传统电力电缆诊断过程中操作复杂、可靠性低和精准度不够等问题,提出了一种基于TCN和BiLSTM的电力电缆故障诊断方法。该方法的核心是使用Matlab/Simulink搭建三相电缆的仿真模型,按照电缆的实际参数设置模型,然后提取出电缆的四种短路故障:单相接地短路、双相接地短路、双相相间短路以及三相短路的电压信号。构建电缆故障样本集,搭建TCN和BiLSTM网络对电缆故障信号进行特征提取和序列捕捉,通过与TCN网络和CNN-BiLSTM网络进行实验对比,以及对从淮南某煤矿采集到的数据进行验证,证明该方法对电缆故障诊断具有良好的性能。

Training-based symbol detection with temporal convolutional neural network in single-polarized optical communication system

In order to reduce the physical impairment caused by signal distortion,in this paper,we investigate symbol detection with Deep Learning(DL)methods to improve bit-error performance in the optical communication system.Many DL-based methods have been applied to such systems to improve bit-error performance.Referring to the speech-to-text method of automatic speech recognition,this paper proposes a signal-to-symbol method based on DL and designs a receiver for symbol detection on single-polarized optical communications modes.To realize this detection method,we propose a non-causal temporal convolutional network-assisted receiver to detect symbols directly from the baseband signal,which specifically integrates most modules of the receiver.Meanwhile,we adopt three training approaches for different signal-to-noise ratios.We also apply a parametric rectified linear unit to enhance the noise robustness of the proposed network.According to the simulation experiments,the biterror-rate performance of the proposed method is close to or even superior to that of the conventional receiver and better than the recurrent neural network-based receiver.

A Spatio-Temporal Heterogeneity Data Accuracy Detection Method Fused by GCN and TCN

Spatio-temporal heterogeneous data is the database for decisionmaking in many fields,and checking its accuracy can provide data support for making decisions.Due to the randomness,complexity,global and local correlation of spatiotemporal heterogeneous data in the temporal and spatial dimensions,traditional detection methods can not guarantee both detection speed and accuracy.Therefore,this article proposes a method for detecting the accuracy of spatiotemporal heterogeneous data by fusing graph convolution and temporal convolution networks.Firstly,the geographic weighting function is introduced and improved to quantify the degree of association between nodes and calculate the weighted adjacency value to simplify the complex topology.Secondly,design spatiotemporal convolutional units based on graph convolutional neural networks and temporal convolutional networks to improve detection speed and accuracy.Finally,the proposed method is compared with three methods,ARIMA,T-GCN,and STGCN,in real scenarios to verify its effectiveness in terms of detection speed,detection accuracy and stability.The experimental results show that the RMSE,MAE,and MAPE of this method are the smallest in the cases of simple connectivity and complex connectivity degree,which are 13.82/12.08,2.77/2.41,and 16.70/14.73,respectively.Also,it detects the shortest time of 672.31/887.36,respectively.In addition,the evaluation results are the same under different time periods of processing and complex topology environment,which indicates that the detection accuracy of this method is the highest and has good research value and application prospects.

Temporal Convolutional Network for Speech Bandwidth Extension

In the field of speech bandwidth exten-sion,it is difficult to achieve high speech quality based on the shallow statistical model method.Although the application of deep learning has greatly improved the extended speech quality,the high model complex-ity makes it infeasible to run on the client.In order to tackle these issues,this paper proposes an end-to-end speech bandwidth extension method based on a temporal convolutional neural network,which greatly reduces the complexity of the model.In addition,a new time-frequency loss function is designed to en-able narrowband speech to acquire a more accurate wideband mapping in the time domain and the fre-quency domain.The experimental results show that the reconstructed wideband speech generated by the proposed method is superior to the traditional heuris-tic rule based approaches and the conventional neu-ral network methods for both subjective and objective evaluation.

多尺度分解下GRU-TCN集成的动力电池剩余使用寿命预测方法

精准预测动力电池的剩余使用寿命(remaining useful life,RUL)能够提前规避因电池过度使用带来的风险,为退役电池的二次利用提供决策依据,提升电池第二寿命的利用率。为了降低动力电池RUL预测任务中噪声和容量回升现象导致的非线性特征对RUL预测精度的影响,提出了一种基于集合经验模态分解(ensemble empirical mode decomposition,EEMD)、门控循环单元网络(gated recurrent unit,GRU)和时序卷积网络(temporal convolutional networks,TCN)集成的动力电池RUL预测模型。首先,使用EEMD对原始数据进行分解,动力电池容量衰退过程中由噪声和容量回升现象导致的非线性特征被分解到高频分量,而原始容量数据的主要趋势被分解到低频分量。其次,再使用GRU和TCN网络分别对高频分量和低频分量进行预测。最后,使用Attention对预测结果进行集成。在NASA数据集上的实验结果表明,本工作提出的集成模型的预测精度和对非线性特征的拟合程度都优于其他单一模型和其他同类型模型,最大平均绝对误差和最大均方根误差分别在0.52%和0.74%内,绝对误差在1个循环周期内,证明本模型有较好的RUL预测能力。

TC-Net:A Modest&Lightweight Emotion Recognition System Using Temporal Convolution Network

Speech signals play an essential role in communication and provide an efficient way to exchange information between humans and machines.Speech Emotion Recognition(SER)is one of the critical sources for human evaluation,which is applicable in many real-world applications such as healthcare,call centers,robotics,safety,and virtual reality.This work developed a novel TCN-based emotion recognition system using speech signals through a spatial-temporal convolution network to recognize the speaker’s emotional state.The authors designed a Temporal Convolutional Network(TCN)core block to recognize long-term dependencies in speech signals and then feed these temporal cues to a dense network to fuse the spatial features and recognize global information for final classification.The proposed network extracts valid sequential cues automatically from speech signals,which performed better than state-of-the-art(SOTA)and traditional machine learning algorithms.Results of the proposed method show a high recognition rate compared with SOTAmethods.The final unweighted accuracy of 80.84%,and 92.31%,for interactive emotional dyadic motion captures(IEMOCAP)and berlin emotional dataset(EMO-DB),indicate the robustness and efficiency of the designed model.

基于TCN和残差自注意力的变工况下滚动轴承剩余寿命迁移预测

针对变工况环境下采集到的滚动轴承寿命状态数据存在特征分布差异,深度神经网络模型泛化能力差的问题,结合时间卷积网络(temporal convolutional neural network,TCN)和残差自注意力机制提出了一种端到端的滚动轴承剩余寿命(remaining useful life,RUL)迁移预测方法。首先,将传感器采集到的一维时域信号利用短时傅里叶变换转换为频域信号;其次,剩余寿命迁移预测网络通用特征提取层采用残差自注意力TCN网络,该网络在较好提取时间序列信息的同时,进一步通过残差自注意力机制捕获轴承局部退化特征,增强模型的迁移特征提取能力;再次,采用提出的联合领域自适应策略匹配变工况下滚动轴承寿命状态数据特征分布差异,实现不同工况下轴承寿命状态知识的迁移预测;最后,在公开的滚动轴承全寿命数据集上进行试验验证,结果表明所提方法能有效实现变工况下的滚动轴承RUL预测,并获得较好的预测性能。

融合2维卷积与注意力以预测PM_(2.5)浓度的S-TCN模型

针对传统预测模型对PM_(2.5)浓度预测精度较低、可解释性差的缺陷,提出一种融合2维卷积层(2D convolution)和注意力层的时空卷积网络预测模型(spatio-2D-temporal convolutional networks attention, S-2D-TCNA)。选取北京市2014年5月1日~2015年4月30日的36个监测站点逐小时空气质量和气象数据,通过对多个站点时空相关性分析,将符合相关性阈值的监测站数据输入至卷积进行升维再降维的处理方式,得出具有时空序列的输入特征;将注意力融入时间卷积网络预测模型,用于预测未来1 h的中心监测站PM_(2.5)浓度。在模型训练优化参数过程中,通过Adam来训练深度学习模型的参数,然后使用贝叶斯优化来调整模型的超参数,这种方法能找到模型的最佳参数,使其均方根误差、平均绝对误差分别减少3.791%和5.576%,拟合优度增大0.67%;在质量方面,所提出的S-Conv2D-TCNA模型均方根误差、平均绝对误差和拟合优度分别为16.020 9、10.610 0和0.942 8,该预测模型在准确性和稳定性方面优于基线模型。结果表明,该预测模型空气污染的预警、区域预防和控制方面大有可为。

基于TCN-A模型的高效查询负载预测算法

针对大数据查询领域中出现的由于查询负载随时间动态变化且难以有效预测所导致的数据库管理系统无法及时优化的问题,提出了一种基于新型时间序列预测模型的查询负载预测算法。首先,该算法采用过滤、时域间隔划分以及查询负载构造等技术对原始的历史用户查询进行预处理,得到便于网络模型分析处理的查询负载序列。其次,所提算法以时间卷积神经网络为核心构建时序预测模型,提取查询负载数据的历史变化趋势及自相关性特征,高效地实现时序预测;同时,融入设计的时域注意力机制,对查询负载序列进行重要性加权,保证模型的分析计算效率,提升算法的预测性能。最后,基于上述时序预测模型,充分利用查询间隔时间完成对未来查询负载的精确预测,使得数据库管理系统得以预先实现自身性能调优,以适应工作负载的动态变化。实验结果表明,设计的查询负载预测算法在多个评价指标中均表现出良好的预测性能,并且能够在查询时间间隔内更加精确地预测未来查询负载的变化。

基于Bi-TCN-LSTM的滚动轴承剩余使用寿命预测方法

由于时间卷积网络(temporal convolutional networks, TCN)感知场不足,轴承的关键退化信息常常被忽略,导致轴承剩余使用寿命(remaining useful life, RUL)预测结果不佳;而长短期记忆网络(long short-term memory, LSTM)随着数据量及序列长度的增加,长期依赖问题仍可能得不到很好解决。因此,课题组提出了一种基于双向时间卷积网络和长短期记忆(Bi-TCN-LSTM)的滚动轴承寿命预测方法。首先对多传感器数据进行归一化并做融合处理,然后采用Bi-TCN-LSTM进行数据特征提取与深度学习,其中对TCN模块引入卷积注意力机制(convolutional attention module, CAM),将LSTM的3个门简化为1个门,有效加快了预测模型学习的速度并提高了预测模型的精确度;采用IEEE PHM 2012轴承数据集作为实验数据集,进行了RUL预测实验。结果表明:与其他先进的预测模型相比,Bi-TCN-LSTM方法预测结果的误差相对较低,预测性能较好。

A multi-source information fusion layer counting method for penetration fuze based on TCN-LSTM

When employing penetration ammunition to strike multi-story buildings,the detection methods using acceleration sensors suffer from signal aliasing,while magnetic detection methods are susceptible to interference from ferromagnetic materials,thereby posing challenges in accurately determining the number of layers.To address this issue,this research proposes a layer counting method for penetration fuze that incorporates multi-source information fusion,utilizing both the temporal convolutional network(TCN)and the long short-term memory(LSTM)recurrent network.By leveraging the strengths of these two network structures,the method extracts temporal and high-dimensional features from the multi-source physical field during the penetration process,establishing a relationship between the multi-source physical field and the distance between the fuze and the target plate.A simulation model is developed to simulate the overload and magnetic field of a projectile penetrating multiple layers of target plates,capturing the multi-source physical field signals and their patterns during the penetration process.The analysis reveals that the proposed multi-source fusion layer counting method reduces errors by 60% and 50% compared to single overload layer counting and single magnetic anomaly signal layer counting,respectively.The model's predictive performance is evaluated under various operating conditions,including different ratios of added noise to random sample positions,penetration speeds,and spacing between target plates.The maximum errors in fuze penetration time predicted by the three modes are 0.08 ms,0.12 ms,and 0.16 ms,respectively,confirming the robustness of the proposed model.Moreover,the model's predictions indicate that the fitting degree for large interlayer spacings is superior to that for small interlayer spacings due to the influence of stress waves.

基于MsTCN-Transformer模型的轴承剩余使用寿命预测研究

剩余使用寿命(remaining useful life, RUL)预测是PHM的核心问题之一,复杂的运行工况往往导致设备部件经历不同的故障退化过程,给RUL准确预测带来了巨大挑战。为此,提出了一种多尺度时间卷积网络(multi-scale temporal convolutional network, MsTCN)与Transformer(MsTCN-Transformer)融合模型用于变工况下滚动轴承RUL预测。该方法设计了一种新的多尺度膨胀因果卷积单元(multi-scale dilated causal convolution unit, MsDCCU),能够自适应地挖掘滚动轴承全寿命信号中固有的时序特征信息;然后构建了基于自注意力机制的Transformer网络模型,在克服预测序列记忆力退化的基础上,准确学习时序特征与轴承RUL之间的映射关系。此外,通过对轴承不同故障退化阶段所提取的时序特征可视化分析,验证了所提方法在变工况下所提取的时序特征泛化性较好。多种工况条件下滚动轴承RUL预测试验表明,所提方法能够较为准确地实现变工况下轴承的RUL预测,相比当前多种方法RUL预测结果准确性更高。

一种基于TCN-LGBM的航空发动机气路故障诊断方法

长时间工作在高温高压、强振动等恶劣气路环境下的航空发动机经常面临部件疲劳、腐蚀和性能退化的问题,且其故障诊断时序逻辑性不强、故障参数耦合较深等特点十分明显,为此提出一种基于时间卷积神经网络(Temporal Convolutional Network,TCN)和轻量级梯度提升机(Light Gradient Boosting Machine,LGBM)的航空发动机气路故障诊断方法。故障诊断分为故障特征提取和分类诊断两个过程:引入TCN框架,在保证故障数据训练时序逻辑的基础上,实现对远层历史信息和当前层信息的特征融合构建,融合通道注意力机制增强了高质量特征的权重;基于LGBM模型实现对特征的快速分类,利用贝叶斯方法实现对模型超参数的快速优化。以基于PROOSIS软件建模的某军用小涵道比涡扇发动机故障仿真数据为例,对6种故障模式进行诊断识别。仿真结果表明了所提方法的有效性;通过与其他模型对比体现了该方法的优越性。

基于SA-TCN的轴承短期故障预测方法

【目的】轴承是制造业中的核心零部件之一,其健康状况决定着主机设备的安全性,对轴承进行短期故障预测可以有效预防生产设备故障的发生。【方法】针对轴承短期故障预测未能实现端到端的问题,提出一种基于时序卷积网络(temporal convolutional network,TCN)的短期故障预测方法。该网络能通过当前时刻监测到的数据直接输出轴承最终会发生的故障类型以及下一时刻将要处于的劣化阶段。此外,提出软阈值注意力机制(soft threshold with attention mechanism,SA)解决轴承工作环境存在背景噪声或采集数据的过程中有噪声干扰的问题;在短期故障预测过程中,注意力机制根据TCN网络的预测目标自适应生成软阈值,软阈值作用于TCN提取到的时空特征,以达到降低噪声影响的目的。【结果】实验结果表明所提算法准确率高,具有较高的实际工程应用价值。

融合TA-TCN和迁移学习的滚动轴承寿命预测

针对在实际工业生产中,滚动轴承由于数据量少导致剩余寿命预测的准确度不高的问题,提出了一种时序注意力(temporal attention, TA)优化的时间卷积神经网络(time convolutional networks, TCN)与迁移学习相结合的剩余寿命预测方法。首先,通过互补集合经验模态分解(complementary ensemble empirical mode decomposition, CEEMD)将原始特征向量分解为一组子序列分量,突出特征信号、降低噪声干扰;然后,将子序列分量输入搭建好的TCN模型并添加TA进行优化,深度挖掘深度特征与退化曲线关系;最后,引入迁移学习,利用源域数据进行训练和少量目标域数据进行参数微调,得到目标网络模型。经实例验证,所提模型的稳定性、预测精度相对于其它对比模型有所提升,且在异工况条件下依然有着良好的预测能力。