Defect Detection Model Using Time Series Data Augmentation and Transformation

Time-series data provide important information in many fields,and their processing and analysis have been the focus of much research.However,detecting anomalies is very difficult due to data imbalance,temporal dependence,and noise.Therefore,methodologies for data augmentation and conversion of time series data into images for analysis have been studied.This paper proposes a fault detection model that uses time series data augmentation and transformation to address the problems of data imbalance,temporal dependence,and robustness to noise.The method of data augmentation is set as the addition of noise.It involves adding Gaussian noise,with the noise level set to 0.002,to maximize the generalization performance of the model.In addition,we use the Markov Transition Field(MTF)method to effectively visualize the dynamic transitions of the data while converting the time series data into images.It enables the identification of patterns in time series data and assists in capturing the sequential dependencies of the data.For anomaly detection,the PatchCore model is applied to show excellent performance,and the detected anomaly areas are represented as heat maps.It allows for the detection of anomalies,and by applying an anomaly map to the original image,it is possible to capture the areas where anomalies occur.The performance evaluation shows that both F1-score and Accuracy are high when time series data is converted to images.Additionally,when processed as images rather than as time series data,there was a significant reduction in both the size of the data and the training time.The proposed method can provide an important springboard for research in the field of anomaly detection using time series data.Besides,it helps solve problems such as analyzing complex patterns in data lightweight.

Multivariate Time Series Anomaly Detection Based on Spatial-Temporal Network and Transformer in Industrial Internet of Things

In the Industrial Internet of Things(IIoT),sensors generate time series data to reflect the working state.When the systems are attacked,timely identification of outliers in time series is critical to ensure security.Although many anomaly detection methods have been proposed,the temporal correlation of the time series over the same sensor and the state(spatial)correlation between different sensors are rarely considered simultaneously in these methods.Owing to the superior capability of Transformer in learning time series features.This paper proposes a time series anomaly detection method based on a spatial-temporal network and an improved Transformer.Additionally,the methods based on graph neural networks typically include a graph structure learning module and an anomaly detection module,which are interdependent.However,in the initial phase of training,since neither of the modules has reached an optimal state,their performance may influence each other.This scenario makes the end-to-end training approach hard to effectively direct the learning trajectory of each module.This interdependence between the modules,coupled with the initial instability,may cause the model to find it hard to find the optimal solution during the training process,resulting in unsatisfactory results.We introduce an adaptive graph structure learning method to obtain the optimal model parameters and graph structure.Experiments on two publicly available datasets demonstrate that the proposed method attains higher anomaly detection results than other methods.

基于双向稀疏Transformer的多变量时序分类模型

针对多变量时序(Multivariate Time Series,MTS)分类中长序列数据难以捕捉时序特征的问题,提出一种基于双向稀疏Transformer的时序分类模型BST(Bidirectional Sparse Transformer),提高了MTS分类任务的准确度.BST模型使用Transformer框架,构建了一种基于活跃度得分的双向稀疏注意力机制.基于KL散度构建活跃度评价函数,并将评价函数的非对称问题转变为对称权重问题.据此,对原有查询矩阵、键值矩阵进行双向稀疏化,从而降低原Transformer模型中自注意力机制运算的时间复杂度.实验结果显示,BST模型在9个长序列数据集上取得最高平均排名,在临界差异图中领先第2名35.7%,对于具有强时序性的乙醇浓度数据集(Ethanol Concentration,EC),分类准确率提高30.9%.

基于Swin Transformer与GRU的低温贮藏番茄成熟度识别与时序预测研究

面向绿熟番茄采后持续转熟特征,适时调温是满足不同成熟度番茄适宜贮运温度需求的关键,而果实成熟度自动识别与动态预测则是实现温度适时调控的基础条件。本文基于Swin Transformer与改进GRU提出了一种番茄成熟度识别与时序动态预测模型,首先通过融合番茄两侧图像获取番茄表观全局红色总占比,构建不同成熟番茄图像数据集,并基于迁移学习优化Swin Transformer模型初始权重配置,实现番茄成熟度分类识别;其次,周期性采集不同储藏温度(4、9、14℃)下番茄图像数据,结合番茄初始颜色特征与贮藏环境信息,构建基于Swin Transformer与GRU的番茄成熟度时序预测模型,并融合时间注意力模块优化模型预测精度;最后,对比分析不同模型预测结果,验证本研究所提模型的准确性与优越性。结果表明,番茄成熟度正确识别率为95.783%,相比VGG16、AlexNet、ResNet50模型,模型正确识别率分别提升2.83%、3.35%、12.34%。番茄成熟度时序预测均方误差(MSE)为0.225,相比原始GRU、LSTM、BiGRU模型MSE最高降低29.46%。本研究为兼顾番茄成熟度实现贮藏温度柔性适时调控提供了关键理论基础。

基于Transformer和GAN的多元时间序列异常检测方法

在时序数据分析中,异常检测是最为成熟的应用之一。它在量化交易、网络安全检测、自动驾驶和大型工业设备日常维护等现实领域广泛应用。随着业务组合的复杂性和时序数据量的增加,传统的人工和简单算法方法很难判断异常点。针对上述问题,对现有的检测方法进行了改进,提出了一种基于Transformer和生成式对抗网络(Generative Adversarial Networks,GAN)的时间序列异常检测模型,利用改进后的Transformer对时间序列的空间特征进行提取,并使用基于异常分数的异常检测算法和对抗训练以获得稳定性和准确性。模型采用自监督训练的方式,避免了需要手动标注异常数据的麻烦,同时减少了数据集对于监督模型训练的依赖。通过实验验证,本文提出的基于Transformer的时间序列异常检测模型在准确率上与先进的基于Transformer的模型相当,并且表现优于多元时间序列的大型数据集上的监督训练和传统异常检测方法。因此,该模型在实际应用中具有较好的潜力。

基于LSTM与Transformer的地面沉降智能预测方法研究——以上海市为例

受地面沉降严重威胁到生命财产安全的人口已达19%,开展地面沉降模拟预测对防灾减灾具有非常重要的现实意义。针对现有地面沉降预测在模型参数难以获取、单一深度学习方法在预测精度低等方面的局限性,本文提出了集成大模型核心技术的地面沉降预测方法。首先,从地面沉降模拟预测的顶层设计,提出了基于深度学习的地面沉降预测包括算力层、数据层、模型层、评估层与应用层的总体架构;其次,基于LSTM与Transformer提出了地面沉降预测的实用方法;最后,利用上海的地面沉降数据进行了实验研究。结果表明:深度学习技术可以在地面沉降模拟预测中取得较好的结果,多模型法对地面沉降变化不大、回弹、变化较大均可进行预测,iTransformer模型对地面沉降变化较小的情况预测效果较好;在微量地面沉降时代,利用大模型的核心技术Transformer可以取得较高的精度。

Generalized canonical transformation for second-order Birkhoffian systems on time scales

The theory of time scales,which unifies continuous and discrete analysis,provides a powerful mathematical tool for the study of complex dynamic systems.It enables us to understand more clearly the essential problems of continuous systems and discrete systems as well as other complex systems.In this paper,the theory of generalized canonical transformation for second-order Birkhoffian systems on time scales is proposed and studied,which extends the canonical transformation theory of Hamilton canonical equations.First,the condition of generalized canonical transformation for the second-order Birkhoffian system on time scales is established.Second,based on this condition,six basic forms of generalized canonical transformation for the second-order Birkhoffian system on time scales are given.Also,the relationships between new variables and old variables for each of these cases are derived.In the end,an example is given to show the application of the results.

基于Transformer的矿井内因火灾时间序列预测方法

传统的基于机器学习的矿井内因火灾预测方法尽管具备一定的预测能力,然而在处理复杂的多变量数据时不能有效捕捉数据间的全局依赖关系,导致预测精度较低。针对上述问题,提出了一种基于Transformer的矿井内因火灾时间序列预测方法。首先,采用Hampel滤波器和拉格朗日插值法对数据进行异常值检测和缺失值填补。然后,利用Transformer的自注意力机制对时间序列数据进行特征提取及趋势预测。最后,通过调节滑动窗口的大小与步长,在不同的时间步长和预测长度下对模型进行不同时间维度的训练。结合气体分析法将矿井火灾产生的标志性气体(CO,O_(2),N_(2),CO_(2),C_(2)H_(2),C_(2)H4,C_(2)H_(6))作为模型输入变量,其中CO作为模型输出的目标变量,O_(2),N_(2),CO_(2),C_(2)H_(2),C_(2)H4,C_(2)H_(6)作为模型输入的协变量。选取陕煤集团柠条塔煤矿S1206回风隅角火灾预警的束管数据进行实验验证,结果表明:①对CO进行单变量预测和多变量预测,多变量预测相比单变量预测有着更高的预测精度,说明多变量预测能通过捕捉序列间的相关性提高模型的预测精度。②当时间步长固定时,基于Transformer的矿井内因火灾预测模型的预测精度随着预测长度的增加而下降。当预测长度固定时,模型的预测精度随时间步长增加而提高。③Transformer算法的预测精度较长短时记忆(LSTM)算法和循环神经网络(RNN)算法分别提高了7.1%~12.6%和20.9%~24.9%。

NONLINEAR TIME TRANSFORMATION METHOD FOR STRONG NONLINEAR OSCILLATION SYSTEMS

In this paper,a nonlinear time transformation method is presented for the analysis of strong nonlinear oscillation systems.This method can be used to study the limit cycle behavior of the autonomous systems and to analyze the forced vibration of a strong nonlinear system.

基于Transformer-LSTM网络的轴承寿命预测

轴承是旋转机械设备中的重要部件,由于工况、材质、加工方式等原因,轴承寿命时长相差许多。传统的并行或串行神经网络预测方式,对数据集有一定要求。因此,需要一种能够适用于不同数据长短的轴承剩余使用寿命预测网络。为此提出了一种能够预测不同寿命时长的Transformer-LSTM串并行神经网络预测模型。通过将Transformer解码层进行重构,并与长短期记忆时序神经网络(long short-term memory,LSTM)网络结构融合,实现轴承寿命数据的串并行预测处理。试验结果表明Transformer-LSTM神经网络能够精准预测长、中、短不同寿命时长的轴承失效时间,具有较强的模型泛化能力,提升轴承寿命预测精度与模型的泛化能力。

Vehicle Density Prediction in Low Quality Videos with Transformer Timeseries Prediction Model(TTPM)

Recent advancement in low-cost cameras has facilitated surveillance in various developing towns in India.The video obtained from such surveillance are of low quality.Still counting vehicles from such videos are necessity to avoid traf-fic congestion and allows drivers to plan their routes more precisely.On the other hand,detecting vehicles from such low quality videos are highly challenging with vision based methodologies.In this research a meticulous attempt is made to access low-quality videos to describe traffic in Salem town in India,which is mostly an un-attempted entity by most available sources.In this work profound Detection Transformer(DETR)model is used for object(vehicle)detection.Here vehicles are anticipated in a rush-hour traffic video using a set of loss functions that carry out bipartite coordinating among estimated and information acquired on real attributes.Every frame in the traffic footage has its date and time which is detected and retrieved using Tesseract Optical Character Recognition.The date and time extricated and perceived from the input image are incorporated with the length of the recognized objects acquired from the DETR model.This furnishes the vehicles report with timestamp.Transformer Timeseries Prediction Model(TTPM)is proposed to predict the density of the vehicle for future prediction,here the regular NLP layers have been removed and the encoding temporal layer has been modified.The proposed TTPM error rate outperforms the existing models with RMSE of 4.313 and MAE of 3.812.

基于MsTCN-Transformer模型的轴承剩余使用寿命预测研究

剩余使用寿命(remaining useful life, RUL)预测是PHM的核心问题之一,复杂的运行工况往往导致设备部件经历不同的故障退化过程,给RUL准确预测带来了巨大挑战。为此,提出了一种多尺度时间卷积网络(multi-scale temporal convolutional network, MsTCN)与Transformer(MsTCN-Transformer)融合模型用于变工况下滚动轴承RUL预测。该方法设计了一种新的多尺度膨胀因果卷积单元(multi-scale dilated causal convolution unit, MsDCCU),能够自适应地挖掘滚动轴承全寿命信号中固有的时序特征信息;然后构建了基于自注意力机制的Transformer网络模型,在克服预测序列记忆力退化的基础上,准确学习时序特征与轴承RUL之间的映射关系。此外,通过对轴承不同故障退化阶段所提取的时序特征可视化分析,验证了所提方法在变工况下所提取的时序特征泛化性较好。多种工况条件下滚动轴承RUL预测试验表明,所提方法能够较为准确地实现变工况下轴承的RUL预测,相比当前多种方法RUL预测结果准确性更高。

Predicting Wavelet-Transformed Stock Prices Using a Vanishing Gradient Resilient Optimized Gated Recurrent Unit with a Time Lag

The development of accurate prediction models continues to be highly beneficial in myriad disciplines. Deep learning models have performed well in stock price prediction and give high accuracy. However, these models are largely affected by the vanishing gradient problem escalated by some activation functions. This study proposes the use of the Vanishing Gradient Resilient Optimized Gated Recurrent Unit (OGRU) model with a scaled mean Approximation Coefficient (AC) time lag which should counter slow convergence, vanishing gradient and large error metrics. This study employed the Rectified Linear Unit (ReLU), Hyperbolic Tangent (Tanh), Sigmoid and Exponential Linear Unit (ELU) activation functions. Real-life datasets including the daily Apple and 5-minute Netflix closing stock prices were used, and they were decomposed using the Stationary Wavelet Transform (SWT). The decomposed series formed a decomposed data model which was compared to an undecomposed data model with similar hyperparameters and different default lags. The Apple daily dataset performed well with a Default_1 lag, using an undecomposed data model and the ReLU, attaining 0.01312, 0.00854 and 3.67 minutes for RMSE, MAE and runtime. The Netflix data performed best with the MeanAC_42 lag, using decomposed data model and the ELU achieving 0.00620, 0.00487 and 3.01 minutes for the same metrics.

基于变量选择与Transformer模型的中长期电力负荷预测方法

准确且有效的负荷预测对于电力系统的实时运行和调度非常重要。提出了一种融合变量选择与稀疏Transformer模型的预测方法,将静态变量和时序变量作为输入,充分发挥静态变量在全局时间范围内的信息增强作用,基于门控机制设计变量分权组件,根据变量与预测结果的相关性,赋予变量不同的权重。设计了双层编码结构,进行时序特征提取,对注意力进行稀疏处理,通过多变量输入对未来时刻负荷进行预测。基于真实电力负荷数据的实验表明,本文模型能够提高中长期负荷预测精度和效率。

Dispersion Relations in Diffraction in Time

In agreement with Titchmarsh’s theorem, we prove that dispersion relations are just the Fourier-transform of the identity, g(x′)=±Sgn(x′)g(x′), which defines the property of being a truncated functions at the origin. On the other hand, we prove that the wave-function of a generalized diffraction in time problem is just the Fourier-transform of a truncated function. Consequently, the existence of dispersion relations for the diffraction in time wave-function follows. We derive these explicit dispersion relations.

基于Transformer与改进记忆机制的用电量预测研究

近年来我国经济的高速发展对电力配置提出了更高要求,实现电力资源的高效配置需要更加精准的用电量预测。随着人工智能、机器学习等技术的发展,高效精准的用电量预测成为可能。目前该领域普遍使用Long Short-Term Memory (LSTM)及其变种模型,但准确度相对较低。文中提出了一种基于改进记忆机制与Transformer的用电量预测模型,使用Transformer编码输入,提出了一种新型记忆机制来执行预测。实验表明该方法相较随机森林回归和LSTM及其变种模型,一周内平均误差分别下降9.05%与5.32%,模型收敛速度更快且具有较好的泛化性能。

Modeling of Incubation Time for Austenite to Ferrite Phase Transformation

On the basis of the classical nucleation theory, a new model of incubation time for austenite to ferrite transformation has been developed, in which the effect of deformation on austenite has been taken into consideration. To prove the precision of modeling, ferrite transformation starting temperature （Ar3） has been calculated using the Scheil＇s additivity rule, and the Ar3 values were measured using a Gleeble 1500 thermomechanical simulator. The Ar3 values provided by the modeling method coincide with the measured ones, indicating that the model is precise in oredicting the incubation time for austenite to ferrite transformation in hot deformed steels.

Arrival time measurements of first arrival phases P and PKIKP using the method of fixed scale wavelet transformation ratio

The arrival times of first teleseismic phases are difficult to be measured precisely because of slowly and gradually changed onsets and weak amplitudes. The arrival times measured manually are usually behind the real ones. In this paper, using the ratio method of fixed scale wavelet transformations improved by us, the arrival times for the first arrival phases (such as P and PKIKP) at the teleseismic and far-teleseismic distances were measured. The results are reasonable and reliable based on the analysis and discussion of the reliabilities and errors.

基于GAT的Transformer多维时间序列异常检测

针对多维时序数据中多个时间和多个变量之间的复杂依赖关系,无法准确地识别出少量异常点问题,提出一种基于GAT的Transformer多变量时间序列异常检测方法。首先,将特征转换为嵌入向量表示;然后,引入图注意力机制自适应地学习不同时间和不同变量之间复杂的依赖关系;最后,将原始数据与GAT层的输出拼接,输入带有位置编码的Transformer编码器,通过计算异常分数并设定阈值判定异常情况。结果表明,所提模型可以有效地检测出时序数据中的异常。

Condition for Successful Square Transformation in Time Series Modeling

In this study we establish the probability density function of the square transformed left-truncated N(1,σ2) error component of the multiplicative time series model and the functional expressions for its mean and variance. Furthermore the mean and variance of the square transformed left-truncated N(1,σ2) error component and those of the untransformed component were compared for the purpose of establishing the interval for σ where the properties of the two distributions are approximately the same in terms of equality of means and normality. From the results of the study, it was established that the two distributions are normally distributed and have means ≌1.0 correct to 1 dp in the interval 0 σ , hence a successful square transformation where necessary is achieved for values of σ such that 0 σ .