Power Quality Disturbance Identification Basing on Adaptive Kalman Filter andMulti-Scale Channel Attention Fusion Convolutional Network

In light of the prevailing issue that the existing convolutional neural network(CNN)power quality disturbance identification method can only extract single-scale features,which leads to a lack of feature information and weak anti-noise performance,a new approach for identifying power quality disturbances based on an adaptive Kalman filter(KF)and multi-scale channel attention(MS-CAM)fused convolutional neural network is suggested.Single and composite-disruption signals are generated through simulation.The adaptive maximum likelihood Kalman filter is employed for noise reduction in the initial disturbance signal,and subsequent integration of multi-scale features into the conventional CNN architecture is conducted.The multi-scale features of the signal are captured by convolution kernels of different sizes so that the model can obtain diverse feature expressions.The attention mechanism(ATT)is introduced to adaptively allocate the extracted features,and the features are fused and selected to obtain the new main features.The Softmax classifier is employed for the classification of power quality disturbances.Finally,by comparing the recognition accuracy of the convolutional neural network(CNN),the model using the attention mechanism,the bidirectional long-term and short-term memory network(MS-Bi-LSTM),and the multi-scale convolutional neural network(MSCNN)with the attention mechanism with the proposed method.The simulation results demonstrate that the proposed method is higher than CNN,MS-Bi-LSTM,and MSCNN,and the overall recognition rate exceeds 99%,and the proposed method has significant classification accuracy and robust classification performance.This achievement provides a new perspective for further exploration in the field of power quality disturbance classification.

An Intelligent Framework for Resilience Recovery of FANETs with Spatio-Temporal Aggregation and Multi-Head Attention Mechanism

Due to the time-varying topology and possible disturbances in a conflict environment,it is still challenging to maintain the mission performance of flying Ad hoc networks(FANET),which limits the application of Unmanned Aerial Vehicle(UAV)swarms in harsh environments.This paper proposes an intelligent framework to quickly recover the cooperative coveragemission by aggregating the historical spatio-temporal network with the attention mechanism.The mission resilience metric is introduced in conjunction with connectivity and coverage status information to simplify the optimization model.A spatio-temporal node pooling method is proposed to ensure all node location features can be updated after destruction by capturing the temporal network structure.Combined with the corresponding Laplacian matrix as the hyperparameter,a recovery algorithm based on the multi-head attention graph network is designed to achieve rapid recovery.Simulation results showed that the proposed framework can facilitate rapid recovery of the connectivity and coverage more effectively compared to the existing studies.The results demonstrate that the average connectivity and coverage results is improved by 17.92%and 16.96%,respectively compared with the state-of-the-art model.Furthermore,by the ablation study,the contributions of each different improvement are compared.The proposed model can be used to support resilient network design for real-time mission execution.

Structured Multi-Head Attention Stock Index Prediction Method Based Adaptive Public Opinion Sentiment Vector

The present study examines the impact of short-term public opinion sentiment on the secondary market,with a focus on the potential for such sentiment to cause dramatic stock price fluctuations and increase investment risk.The quantification of investment sentiment indicators and the persistent analysis of their impact has been a complex and significant area of research.In this paper,a structured multi-head attention stock index prediction method based adaptive public opinion sentiment vector is proposed.The proposedmethod utilizes an innovative approach to transform numerous investor comments on social platforms over time into public opinion sentiment vectors expressing complex sentiments.It then analyzes the continuous impact of these vectors on the market through the use of aggregating techniques and public opinion data via a structured multi-head attention mechanism.The experimental results demonstrate that the public opinion sentiment vector can provide more comprehensive feedback on market sentiment than traditional sentiment polarity analysis.Furthermore,the multi-head attention mechanism is shown to improve prediction accuracy through attention convergence on each type of input information separately.Themean absolute percentage error(MAPE)of the proposedmethod is 0.463%,a reduction of 0.294% compared to the benchmark attention algorithm.Additionally,the market backtesting results indicate that the return was 24.560%,an improvement of 8.202% compared to the benchmark algorithm.These results suggest that themarket trading strategy based on thismethod has the potential to improve trading profits.

Posture Detection of Heart Disease Using Multi-Head Attention Vision Hybrid(MHAVH)Model

Cardiovascular disease is the leading cause of death globally.This disease causes loss of heart muscles and is also responsible for the death of heart cells,sometimes damaging their functionality.A person’s life may depend on receiving timely assistance as soon as possible.Thus,minimizing the death ratio can be achieved by early detection of heart attack(HA)symptoms.In the United States alone,an estimated 610,000 people die fromheart attacks each year,accounting for one in every four fatalities.However,by identifying and reporting heart attack symptoms early on,it is possible to reduce damage and save many lives significantly.Our objective is to devise an algorithm aimed at helping individuals,particularly elderly individuals living independently,to safeguard their lives.To address these challenges,we employ deep learning techniques.We have utilized a vision transformer(ViT)to address this problem.However,it has a significant overhead cost due to its memory consumption and computational complexity because of scaling dot-product attention.Also,since transformer performance typically relies on large-scale or adequate data,adapting ViT for smaller datasets is more challenging.In response,we propose a three-in-one steam model,theMulti-Head Attention Vision Hybrid(MHAVH).Thismodel integrates a real-time posture recognition framework to identify chest pain postures indicative of heart attacks using transfer learning techniques,such as ResNet-50 and VGG-16,renowned for their robust feature extraction capabilities.By incorporatingmultiple heads into the vision transformer to generate additional metrics and enhance heart-detection capabilities,we leverage a 2019 posture-based dataset comprising RGB images,a novel creation by the author that marks the first dataset tailored for posture-based heart attack detection.Given the limited online data availability,we segmented this dataset into gender categories(male and female)and conducted testing on both segmented and original datasets.The training accuracy of our model reached an impressive 99.77%.Upon testing,the accuracy for male and female datasets was recorded at 92.87%and 75.47%,respectively.The combined dataset accuracy is 93.96%,showcasing a commendable performance overall.Our proposed approach demonstrates versatility in accommodating small and large datasets,offering promising prospects for real-world applications.

Multi-Head Attention Spatial-Temporal Graph Neural Networks for Traffic Forecasting

Accurate traffic prediction is crucial for an intelligent traffic system (ITS). However, the excessive non-linearity and complexity of the spatial-temporal correlation in traffic flow severely limit the prediction accuracy of most existing models, which simply stack temporal and spatial modules and fail to capture spatial-temporal features effectively. To improve the prediction accuracy, a multi-head attention spatial-temporal graph neural network (MSTNet) is proposed in this paper. First, the traffic data is decomposed into unique time spans that conform to positive rules, and valuable traffic node attributes are mined through an adaptive graph structure. Second, time and spatial features are captured using a multi-head attention spatial-temporal module. Finally, a multi-step prediction module is used to achieve future traffic condition prediction. Numerical experiments were conducted on an open-source dataset, and the results demonstrate that MSTNet performs well in spatial-temporal feature extraction and achieves more positive forecasting results than the baseline methods.

Combing Type-Aware Attention and Graph Convolutional Networks for Event Detection

Event detection(ED)is aimed at detecting event occurrences and categorizing them.This task has been previously solved via recognition and classification of event triggers(ETs),which are defined as the phrase or word most clearly expressing event occurrence.Thus,current approaches require both annotated triggers as well as event types in training data.Nevertheless,triggers are non-essential in ED,and it is time-wasting for annotators to identify the“most clearly”word from a sentence,particularly in longer sentences.To decrease manual effort,we evaluate event detectionwithout triggers.We propose a novel framework that combines Type-aware Attention and Graph Convolutional Networks(TA-GCN)for event detection.Specifically,the task is identified as a multi-label classification problem.We first encode the input sentence using a novel type-aware neural network with attention mechanisms.Then,a Graph Convolutional Networks(GCN)-based multilabel classification model is exploited for event detection.Experimental results demonstrate the effectiveness.

Traffic Sign Recognition for Autonomous Vehicle Using Optimized YOLOv7 and Convolutional Block Attention Module

The infrastructure and construction of roads are crucial for the economic and social development of a region,but traffic-related challenges like accidents and congestion persist.Artificial Intelligence(AI)and Machine Learning(ML)have been used in road infrastructure and construction,particularly with the Internet of Things(IoT)devices.Object detection in Computer Vision also plays a key role in improving road infrastructure and addressing trafficrelated problems.This study aims to use You Only Look Once version 7(YOLOv7),Convolutional Block Attention Module(CBAM),the most optimized object-detection algorithm,to detect and identify traffic signs,and analyze effective combinations of adaptive optimizers like Adaptive Moment estimation(Adam),Root Mean Squared Propagation(RMSprop)and Stochastic Gradient Descent(SGD)with the YOLOv7.Using a portion of German traffic signs for training,the study investigates the feasibility of adopting smaller datasets while maintaining high accuracy.The model proposed in this study not only improves traffic safety by detecting traffic signs but also has the potential to contribute to the rapid development of autonomous vehicle systems.The study results showed an impressive accuracy of 99.7%when using a batch size of 8 and the Adam optimizer.This high level of accuracy demonstrates the effectiveness of the proposed model for the image classification task of traffic sign recognition.

Using Recurrent Neural Network Structure and Multi-Head Attention with Convolution for Fraudulent Phone Text Recognition

Fraud cases have been a risk in society and people’s property security has been greatly threatened.In recent studies,many promising algorithms have been developed for social media offensive text recognition as well as sentiment analysis.These algorithms are also suitable for fraudulent phone text recognition.Compared to these tasks,the semantics of fraudulent words are more complex and more difficult to distinguish.Recurrent Neural Networks(RNN),the variants ofRNN,ConvolutionalNeuralNetworks(CNN),and hybrid neural networks to extract text features are used by most text classification research.However,a single network or a simple network combination cannot obtain rich characteristic knowledge of fraudulent phone texts relatively.Therefore,a new model is proposed in this paper.In the fraudulent phone text,the knowledge that can be learned by the model includes the sequence structure of sentences,the correlation between words,the correlation of contextual semantics,the feature of keywords in sentences,etc.The new model combines a bidirectional Long-Short Term Memory Neural Network(BiLSTM)or a bidirectional Gate Recurrent United(BiGRU)and a Multi-Head attention mechanism module with convolution.A normalization layer is added after the output of the final hidden layer.BiLSTM or BiGRU is used to build the encoding and decoding layer.Multi-head attention mechanism module with convolution(MHAC)enhances the ability of the model to learn global interaction information and multi-granularity local interaction information in fraudulent sentences.A fraudulent phone text dataset is produced by us in this paper.The THUCNews data sets and fraudulent phone text data sets are used in experiments.Experiment results show that compared with the baseline model,the proposed model(LMHACL)has the best experiment results in terms of Accuracy,Precision,Recall,and F1 score on the two data sets.And the performance indexes on fraudulent phone text data sets are all above 0.94.

基于CNN-BiGRU-Attention的短期电力负荷预测

针对目前电力负荷数据随机性强,影响因素复杂,传统单一预测模型精度低的问题,结合卷积神经网络(Convolutional neural network,CNN)、双向门控循环单元(Bi-directional gated recurrent unit,BiGRU)以及注意力机制(Attention)在短期电力负荷预测上的不同优点,提出一种基于CNN-BiGRU-Attention的混合预测模型。该方法首先通过CNN对历史负荷和气象数据进行初步特征提取,然后利用BiGRU进一步挖掘特征数据间时序关联,再引入注意力机制,对BiGRU输出状态给与不同权重,强化关键特征,最后完成负荷预测。试验结果表明,该模型的平均绝对百分比误差(Mean absolute percentage error,MAPE)、均方根误差(Root mean square error,RMSE)、判定系数(R-square,R~2)分别为0.167%、0.057%、0.993,三项指标明显优于其他模型,具有更高的预测精度和稳定性,验证了模型在短期负荷预测中的优势。

基于CNN-LSTM-Attention的月生活需水预测研究

需水预测是进行水资源配置的重要部分,对于水资源合理开发利用和社会可持续发展有重要指导意义.本文以陕西省为研究区,结合大数据分析法,提出一种基于CNN-LSTM-Attention的月生活需水预测模型.首先,通过卷积神经网络(convolutional neural networks,CNN)提取数据动态变化特征,然后利用长短期记忆(long short-term memory,LSTM)网络对提取的特征进行学习训练,最后使用注意力(attention)机制分配LSTM隐含层不同权重,预测月生活需水量并对比实际数据.结果表明,CNN-LSTM-Attention模型的相对平均误差值和决定系数(R2)分别为2.54%、0.95,满足预测精度需求,相比于LSTM模型预测精度更高.进一步证明了模型预测的合理性,可为陕西省水资源规划提供指导.

融合RoBERTa-GCN-Attention的隐喻识别与情感分类模型

在隐喻识别与隐喻情感分类任务的联合研究中,现有多任务学习模型存在对隐喻语料中的上下文语义信息和句法结构信息提取不够准确,并且缺乏对粗细两种粒度信息同时捕捉的问题.针对第1个问题,首先改进了传统的RoBERTa模型,在原有的自注意力机制中引入上下文信息,以此提取上下文中重要的隐喻语义特征;其次在句法依存树上使用图卷积网络提取隐喻句中的句法结构信息.针对第2个问题,使用双层注意力机制,分别聚焦于单词和句子层面中对隐喻识别和情感分类有贡献的特征信息.在两类任务6个数据集上的对比实验结果表明,该模型相比基线模型性能均有提升.

基于SSA-CG-Attention模型的多因素采煤工作面涌水量预测

矿井工作面涌水量预测对确保矿山安全、优化资源配置、提高工作效率等都具有重要作用。为提高预测结果的准确性和稳定性,基于钻孔水位和微震能量数据与涌水量的强关联性,选择其作为多因素特征变量,提出SSA-CG-Attention多因素矿井工作面涌水量预测模型。该模型在门控循环单元(GatedRecurrentUnit,GRU)提取时序特征的基础上,与卷积神经网络(ConvolutionalNeuralNet-work,CNN)融合形成新的网络结构提取数据的有效非线性局部特征,并且加入注意力机制(Atten-tion),在预测过程中将注意力集中在输入元素上,提高模型的准确性。最后通过麻雀搜索算法(Spar-row Search Algorithm,SSA)优化模型参数,避免局部最优解的问题。将提出的模型分别与传统的BP神经网络、LSTM、GRU单因素涌水量预测模型以及MLP、SLP、SVR、LSTM、GRU、SSA-LSTM、SSA-GRU多因素涌水量预测模型的预测结果进行对比分析,结果表明:SSA算法以最少迭代次数快速寻优,避免了局部最优解的缺陷;SSA-CG-Attention多因素涌水量预测模型整体预测指标绝对误差(E_(MA))、均方根误差(E_(RMS))以及平均绝对百分比误差(E_(MAP))分别为5.24 m^(3)/h、7.25 m^(3)/h、6%,指标方差和为8.90。相较于其他预测模型预测精度更高,相较于单因素涌水量预测模型,多因素涌水量预测模型预测结果更加稳定。研究结果为矿井工作面涌水量预测提供了新的思路与方法,对矿井工作面涌水量预测及防控有着借鉴与指导作用,具有一定的理论价值和现实意义。

基于Coordinate Attention和空洞卷积的异物识别

在我国工厂的工业化生产中,带式运输机占有重要的地位,但是在其运输物料的过程中,常有木板、金属管、大型金属片等混入物料中,从而对带式运输机的传送带造成损毁,引起巨大的经济损失.为了检测出传送带上的不规则异物,设计了一种新的异物检测方法.针对传统异物检测方法中存在的对于图像特征提取能力不足以及网络感受野相对较小的问题,我们提出了一种基于coordinate attention和空洞卷积的单阶段异物识别方法.首先,网络利用coordinate attention机制,使网络更加关注图像的空间信息,并对图像中的重要特征进行了增强,增强了网络的性能;其次,在网络提取多尺度特征的部分,将原网络的静态卷积变为空洞卷积,有效减少了常规卷积造成的信息损失;除此之外,我们还使用了新的损失函数,进一步提高了网络的性能.实验结果证明,我们提出的网络能有效识别出传送带上的异物,较好地完成异物检测任务.

基于PCA和TCN-Attention的重载铁路钢轨剥离伤损退化趋势预测

重载铁路在高强度的运输过程中极易导致钢轨产生剥离、磨耗等伤损影响行车安全,为了保证铁路的安全运行,对钢轨的伤损状态监测和预测是非常重要的。然而,目前钢轨伤损检测方法主要以人工道路巡检为主,检测结果存在主观性强、伤损程度量化难、伤损退化趋势预测难等问题。针对现有问题,提出一种基于主成分分析(principal component analysis,PCA)和TCN-Attention(temporal convolutional networks with attention)的重载铁路钢轨剥离伤损退化趋势预测新方法。首先,从钢轨剥离伤损振动信号中提取时域、频域特征,并采用PCA对提取到的高维特征进行降维;其次,利用时序样本间特征的差异性,构建出钢轨剥离伤损退化指标描述退化趋势性,解决伤损状态度量难的问题;利用TCN网络模型结合Attention机制对有效特征的关注提升模型的预测精度;最后,利用某铁路机务段采集的钢轨从正常到出现损伤直至失效的全生命周期振动数据,对所提方法的有效性进行验证,实验结果表明:所提出的方法能准确地预测钢轨剥离伤损的退化趋势。

基于AVMD-CNN-GRU-Attention的超短期风功率预测研究

为提高超短期风功率的预测精度,提出一种改进的基于变分模态分解的卷积神经网络(AVMD-CNN)、门控循环单元(GRU)和注意力机制(Attention)的超短期风功率预测模型。首先利用改进的VMD将风功率序列分解为K个子模态;然后将各子模态利用样本熵(SE)和中心频率进行分类,根据分类结果对各子模态分别给定归一化方式,并按SE值分别输入到GRU-Attention和CNN-GRU-Attention模型中进行训练和预测;最后将各子模态预测结果叠加得到最终结果,从而完成超短期风功率预测。以决定系数(R^(2))、平均绝对误差(MAE)、均方根误差(RMSE)以及平均绝对百分比误差(MAPE)为精度评估指标,实际算例表明,所提出模型的R^(2)较文中其他方法平均提高12.06%,MAE、RMSE以及MAPE分别平均降低59.36%、62.49%和48.34%,具有较高的预测精度。

基于多传感器信息融合和CNN-BIGRU-Attention模型的液压防水阀故障诊断方法

在建筑工程领域,尤其是在工程搅拌设备中,涉及到复杂液压工作介质,液压防水阀往往会出现不同程度的故障。此外,恶劣的工作环境和复杂的噪声背景使得对液压防水阀的故障进行诊断变得困难。为了解决这一难题,提出了一种基于多传感器信息融合和卷积神经网络-双向门控循环单元-自注意力机制(CNN-BIGRU-Attention)模型的防水阀故障诊断方法。首先,考虑到单一传感器振动信号难以充分表达故障特征,该方法使用了3个传感器采集含噪声的振动信号,并进行了必要的预处理;其次,提取了信号的16个时域特征、5个频域特征以及3个时频域特征,并利用熵权法进行了特征融合,达到了增强特征的目的;然后,将融合的多维特征集输入到CNN-BIGRU-Attention模型中进行了特征识别;最后,利用实际的液压防水阀故障诊断实验,验证了该方法的有效性。研究结果表明:采用多传感器提取的特征更为全面,信息融合有助于捕捉更完整的隐藏数据特征,从而显著提高诊断的准确率;相较于其他特征识别方法,采用CNN-BIGRU-Attention模型的液压防水阀故障诊断准确率可分别提高6.7%、4.6%和14.2%,达到了96.86%,证明了该方法的有效性。该方法将先进的机器学习技术与实际工程应用相结合,为建筑工程问题提供了一种新颖、有效的解决方案。

基于CNN融合PGW-Attention的金属表面缺陷识别方法

针对分散和细小的金属表面缺陷检测方法,卷积神经网络(CNN)缺乏全局特征捕捉能力,在识别氧化颗粒、裂纹和划痕等缺陷时易发生漏检和特征丢失,Transformer能够捕捉图像全局信息,但全局计算导致较高的计算成本。为实现高效且精准的金属表面缺陷识别,将CNN的局部特征提取能力与Transformer的全局建模能力有效融合,提出了一种基于深度可分离卷积(DW-Conv)融合池化网格窗口注意力机制(PGW-Attention)的金属表面缺陷识别网络架构(DPG-Transformer)。在自建金属缺陷数据集(ST-DET)和公开金属缺陷数据集(NEU-CLS)上对该方法进行了实验验证,DPG-Transformer的缺陷识别准确率分别为99.3%和99.6%,在准确率、计算量和浮点计算量等指标上优于多种经典网络。此外,在可视化实验中,DPG-Transformer显示出比CNN模型更全面地腐蚀和氧化皮的缺陷特征提取能力,并能比Transformer模型更加精准地关注到细长裂纹和划痕的全局缺陷特征。实验结果表明,该方法可以降低Transformer模型的计算量和复杂度,同时能够更全面、精准地提取到金属表面缺陷特征,是一种更切合实际应用的金属表面缺陷检测方法。

基于CNN-Attention-LSTM的大坝变形预测模型

【目的】预测大坝变形以规避风险是大坝变形监测的重点,一个可靠的预测模型可以洞察大坝未来变形趋势。为了更好地预测大坝的变形,提高预测精度和计算效率,【方法】提出了一种基于卷积神经网络(CNN)、注意力机制(Attention)和长短时记忆网络(LSTM)的大坝监测模型。CNN从监测数据中提取特征,LSTM更好地从时间序列数据中学习,并在此CNN-LSTM模型的基础上,耦合深度学习算法Attention机制,突出特征对输入效果的影响,在不影响模型精度的前提下提高计算速度,进一步提高模型预测精度与稳定性。同时,结合工程实例进行了应用分析。【结果】结果显示,所建模型能够精确预测大坝变形,在各点位测试集上平均R2、MAE、RMSE、MSE和MAPE分别为0.989 mm、0.337 mm、0.469 mm、0.252 mm和13.918%。【结论】结果表明:所建模型具有较好的变形预测能力和适用性,相较于CNN、LSTM、CNN-LSTM、Attention-LSTM模型,该模型具有较好的MAE、RMSE、MSE、MAPE和R2等指标,并提高了计算效率,更适合大坝变形的预测。

基于CNN-BiGRU-ATTENTION的船舶航迹预测方法研究

为了维护海上交通安全,实时掌握船舶航行动态,国际海事组织(International Maritime Organization,IMO)要求船舶必须配备船舶自动识别系统(Automatic Identification System,AIS),而AIS数据包含了船舶的许多航行特征,可以预测船舶的未来走向,便于海上交管部门的管理。根据船舶AIS数据,提出了基于船舶航迹聚类和CNN-BiGRU-ATTENTION模型的船舶航迹预测方法,构建了基于卷积神经网络(CNN)、双向门控循环单元(BiGRU)和注意力机制(ATTENTION)的船舶航迹预测模型。用CNN抽取AIS数据潜在特征,利用BiGRU提取时序数据历史和未来的信息,结合注意力机制来突出关键特征,从而预测船舶航迹。实验结果表明,基于CNN-BiGRU-ATTENTION模型的船舶航迹预测准确度更高,在经度、纬度、航向和航速的预测上,CNN-BiGRU-ATTENTION模型的平均绝对误差(MAE)和均方根误差(RMSE)相比BiGRU和LSTM更低。

基于CNN-LSTM-Attention网络的河南省冬小麦产量预测

为探讨利用时空建模的深度学习方法提高大区域冬小麦产量预测精度的可行性,从县级冬小麦产量预测角度出发,使用卷积神经网络(convolutional neural networks,CNN)从气候和土壤数据中提取与作物产量密切相关的特征数据,利用注意力机制(attention mechanism)捕捉特征数据之间的相互依赖性,最后将重新加权的特征与长短期记忆网络(long short-term memory network,LSTM)从年产量中捕获到的时间依赖性结合来预测县级冬小麦产量。结果表明,注意力机制模块能够有效地考虑到从CNN中提取的特征之间的相对重要性;模型RMSE为686.82 kg·hm^(-2),相较于支持向量机(SVR)、深度全连接(DFNN)和随机森林(RF)模型分别降低了43%、30%和67%,且R^(2)在0.755以上,MAPE低于14.11%,预测精度均优于传统方法。这说明将注意力机制、CNN和LSTM结合建立的预测模型具有良好的泛化能力和空间平稳性,可用于大区域冬小麦产量预测。