The Short-Term Prediction ofWind Power Based on the Convolutional Graph Attention Deep Neural Network

The fluctuation of wind power affects the operating safety and power consumption of the electric power grid and restricts the grid connection of wind power on a large scale.Therefore,wind power forecasting plays a key role in improving the safety and economic benefits of the power grid.This paper proposes a wind power predicting method based on a convolutional graph attention deep neural network with multi-wind farm data.Based on the graph attention network and attention mechanism,the method extracts spatial-temporal characteristics from the data of multiple wind farms.Then,combined with a deep neural network,a convolutional graph attention deep neural network model is constructed.Finally,the model is trained with the quantile regression loss function to achieve the wind power deterministic and probabilistic prediction based on multi-wind farm spatial-temporal data.A wind power dataset in the U.S.is taken as an example to demonstrate the efficacy of the proposed model.Compared with the selected baseline methods,the proposed model achieves the best prediction performance.The point prediction errors(i.e.,root mean square error(RMSE)and normalized mean absolute percentage error(NMAPE))are 0.304 MW and 1.177%,respectively.And the comprehensive performance of probabilistic prediction(i.e.,con-tinuously ranked probability score(CRPS))is 0.580.Thus,the significance of multi-wind farm data and spatial-temporal feature extraction module is self-evident.

A Multi-View Gait Recognition Method Using Deep Convolutional Neural Network and Channel Attention Mechanism

In many existing multi-view gait recognition methods based on images or video sequences,gait sequences are usually used to superimpose and synthesize images and construct energy-like template.However,information may be lost during the process of compositing image and capture EMG signals.Errors and the recognition accuracy may be introduced and affected respectively by some factors such as period detection.To better solve the problems,a multi-view gait recognition method using deep convolutional neural network and channel attention mechanism is proposed.Firstly,the sliding time window method is used to capture EMG signals.Then,the back-propagation learning algorithm is used to train each layer of convolution,which improves the learning ability of the convolutional neural network.Finally,the channel attention mechanism is integrated into the neural network,which will improve the ability of expressing gait features.And a classifier is used to classify gait.As can be shown from experimental results on two public datasets,OULP and CASIA-B,the recognition rate of the proposed method can be achieved at 88.44%and 97.25%respectively.As can be shown from the comparative experimental results,the proposed method has better recognition effect than several other newer convolutional neural network methods.Therefore,the combination of convolutional neural network and channel attention mechanism is of great value for gait recognition.

Facial Expression Recognition Using Enhanced Convolution Neural Network with Attention Mechanism

Facial Expression Recognition(FER)has been an interesting area of research in places where there is human-computer interaction.Human psychol-ogy,emotions and behaviors can be analyzed in FER.Classifiers used in FER have been perfect on normal faces but have been found to be constrained in occluded faces.Recently,Deep Learning Techniques(DLT)have gained popular-ity in applications of real-world problems including recognition of human emo-tions.The human face reflects emotional states and human intentions.An expression is the most natural and powerful way of communicating non-verbally.Systems which form communications between the two are termed Human Machine Interaction(HMI)systems.FER can improve HMI systems as human expressions convey useful information to an observer.This paper proposes a FER scheme called EECNN(Enhanced Convolution Neural Network with Atten-tion mechanism)to recognize seven types of human emotions with satisfying results in its experiments.Proposed EECNN achieved 89.8%accuracy in classi-fying the images.

Deep Neural Network Based Spam Email Classification Using Attention Mechanisms

Spam emails pose a threat to individuals. The proliferation of spam emails daily has rendered traditional machine learning and deep learning methods for screening them ineffective and inefficient. In our research, we employ deep neural networks like RNN, LSTM, and GRU, incorporating attention mechanisms such as Bahdanua, scaled dot product (SDP), and Luong scaled dot product self-attention for spam email filtering. We evaluate our approach on various datasets, including Trec spam, Enron spam emails, SMS spam collections, and the Ling spam dataset, which constitutes a substantial custom dataset. All these datasets are publicly available. For the Enron dataset, we attain an accuracy of 99.97% using LSTM with SDP self-attention. Our custom dataset exhibits the highest accuracy of 99.01% when employing GRU with SDP self-attention. The SMS spam collection dataset yields a peak accuracy of 99.61% with LSTM and SDP attention. Using the GRU (Gated Recurrent Unit) alongside Luong and SDP (Structured Self-Attention) attention mechanisms, the peak accuracy of 99.89% in the Ling spam dataset. For the Trec spam dataset, the most accurate results are achieved using Luong attention LSTM, with an accuracy rate of 99.01%. Our performance analyses consistently indicate that employing the scaled dot product attention mechanism in conjunction with gated recurrent neural networks (GRU) delivers the most effective results. In summary, our research underscores the efficacy of employing advanced deep learning techniques and attention mechanisms for spam email filtering, with remarkable accuracy across multiple datasets. This approach presents a promising solution to the ever-growing problem of spam emails.

Dynamic interwell connectivity analysis of multi-layer waterflooding reservoirs based on an improved graph neural network

The analysis of interwell connectivity plays an important role in the formulation of oilfield development plans and the description of residual oil distribution. In fact, sandstone reservoirs in China's onshore oilfields generally have the characteristics of thin and many layers, so multi-layer joint production is usually adopted. It remains a challenge to ensure the accuracy of splitting and dynamic connectivity in each layer of the injection-production wells with limited field data. The three-dimensional well pattern of multi-layer reservoir and the relationship between injection-production wells can be equivalent to a directional heterogeneous graph. In this paper, an improved graph neural network is proposed to construct an interacting process mimics the real interwell flow regularity. In detail, this method is used to split injection and production rates by combining permeability, porosity and effective thickness, and to invert the dynamic connectivity in each layer of the injection-production wells by attention mechanism.Based on the material balance and physical information, the overall connectivity from the injection wells,through the water injection layers to the production layers and the output of final production wells is established. Meanwhile, the change of well pattern caused by perforation, plugging and switching of wells at different times is achieved by updated graph structure in spatial and temporal ways. The effectiveness of the method is verified by a combination of reservoir numerical simulation examples and field example. The method corresponds to the actual situation of the reservoir, has wide adaptability and low cost, has good practical value, and provides a reference for adjusting the injection-production relationship of the reservoir and the development of the remaining oil.

Improved Convolutional Neural Network for Traffic Scene Segmentation

In actual traffic scenarios,precise recognition of traffic participants,such as vehicles and pedestrians,is crucial for intelligent transportation.This study proposes an improved algorithm built on Mask-RCNN to enhance the ability of autonomous driving systems to recognize traffic participants.The algorithmincorporates long and shortterm memory networks and the fused attention module(GSAM,GCT,and Spatial Attention Module)to enhance the algorithm’s capability to process both global and local information.Additionally,to increase the network’s initial operation stability,the original network activation function was replaced with Gaussian error linear unit.Experiments were conducted using the publicly available Cityscapes dataset.Comparing the test results,it was observed that the revised algorithmoutperformed the original algorithmin terms of AP_(50),AP_(75),and othermetrics by 8.7%and 9.6%for target detection and 12.5%and 13.3%for segmentation.

Lightweight Cross-Modal Multispectral Pedestrian Detection Based on Spatial Reweighted Attention Mechanism

Multispectral pedestrian detection technology leverages infrared images to provide reliable information for visible light images, demonstrating significant advantages in low-light conditions and background occlusion scenarios. However, while continuously improving cross-modal feature extraction and fusion, ensuring the model’s detection speed is also a challenging issue. We have devised a deep learning network model for cross-modal pedestrian detection based on Resnet50, aiming to focus on more reliable features and enhance the model’s detection efficiency. This model employs a spatial attention mechanism to reweight the input visible light and infrared image data, enhancing the model’s focus on different spatial positions and sharing the weighted feature data across different modalities, thereby reducing the interference of multi-modal features. Subsequently, lightweight modules with depthwise separable convolution are incorporated to reduce the model’s parameter count and computational load through channel-wise and point-wise convolutions. The network model algorithm proposed in this paper was experimentally validated on the publicly available KAIST dataset and compared with other existing methods. The experimental results demonstrate that our approach achieves favorable performance in various complex environments, affirming the effectiveness of the multispectral pedestrian detection technology proposed in this paper.

Power Quality Disturbance Identification Basing on Adaptive Kalman Filter andMulti-Scale Channel Attention Fusion Convolutional Network

In light of the prevailing issue that the existing convolutional neural network(CNN)power quality disturbance identification method can only extract single-scale features,which leads to a lack of feature information and weak anti-noise performance,a new approach for identifying power quality disturbances based on an adaptive Kalman filter(KF)and multi-scale channel attention(MS-CAM)fused convolutional neural network is suggested.Single and composite-disruption signals are generated through simulation.The adaptive maximum likelihood Kalman filter is employed for noise reduction in the initial disturbance signal,and subsequent integration of multi-scale features into the conventional CNN architecture is conducted.The multi-scale features of the signal are captured by convolution kernels of different sizes so that the model can obtain diverse feature expressions.The attention mechanism(ATT)is introduced to adaptively allocate the extracted features,and the features are fused and selected to obtain the new main features.The Softmax classifier is employed for the classification of power quality disturbances.Finally,by comparing the recognition accuracy of the convolutional neural network(CNN),the model using the attention mechanism,the bidirectional long-term and short-term memory network(MS-Bi-LSTM),and the multi-scale convolutional neural network(MSCNN)with the attention mechanism with the proposed method.The simulation results demonstrate that the proposed method is higher than CNN,MS-Bi-LSTM,and MSCNN,and the overall recognition rate exceeds 99%,and the proposed method has significant classification accuracy and robust classification performance.This achievement provides a new perspective for further exploration in the field of power quality disturbance classification.

Acoustic Emission Recognition Based on a Three-Streams Neural Network with Attention

Acoustic emission(AE)is a nondestructive real-time monitoring technology,which has been proven to be a valid way of monitoring dynamic damage to materials.The classification and recognition methods of the AE signals of the rotor are mostly focused on machine learning.Considering that the huge success of deep learning technologies,where the Recurrent Neural Network(RNN)has been widely applied to sequential classification tasks and Convolutional Neural Network(CNN)has been widely applied to image recognition tasks.A novel three-streams neural network(TSANN)model is proposed in this paper to deal with fault detection tasks.Based on residual connection and attention mechanism,each stream of the model is able to learn the most informative representation from Mel Frequency Cepstrum Coefficient(MFCC),Tempogram,and short-time Fourier transform(STFT)spectral respectively.Experimental results show that,in comparison with traditional classification methods and single-stream CNN networks,TSANN achieves the best overall performance and the classification error rate is reduced by up to 50%,which demonstrates the availability of the model proposed.

HDAM:Heuristic Difference Attention Module for Convolutional Neural Networks

The attention mechanism is one of the most important priori knowledge to enhance convolutional neural networks.Most attention mechanisms are bound to the convolutional layer and use local or global contextual information to recalibrate the input.This is a popular attention strategy design method.Global contextual information helps the network to consider the overall distribution,while local contextual information is more general.The contextual information makes the network pay attention to the mean or maximum value of a particular receptive field.Different from the most attention mechanism,this article proposes a novel attention mechanism with the heuristic difference attention module(HDAM).HDAM’s input recalibration is based on the difference between the local and global contextual information instead of the mean and maximum values.At the same time,to make different layers have amore suitable local receptive field sizes and increase the flexibility of the local receptive field design,we use genetic algorithm to heuristically produce local receptive fields.First,HDAM extracts the mean value of the global and local receptive fields as the corresponding contextual information.Then the difference between the global and local contextual information is calculated.Finally,HDAM uses this difference to recalibrate the input.In addition,we use the heuristic ability of genetic algorithm to search for the local receptive field size of each layer.Our experiments on CIFAR-10 and CIFAR-100 show that HDAM can use fewer parameters than other attention mechanisms to achieve higher accuracy.We implement HDAM with the Python library,Pytorch,and the code and models will be publicly available.

Point Cloud Classification Network Based on Graph Convolution and Fusion Attention Mechanism

The classification of point cloud data is the key technology of point cloud data information acquisition and 3D reconstruction, which has a wide range of applications. However, the existing point cloud classification methods have some shortcomings when extracting point cloud features, such as insufficient extraction of local information and overlooking the information in other neighborhood features in the point cloud, and not focusing on the point cloud channel information and spatial information. To solve the above problems, a point cloud classification network based on graph convolution and fusion attention mechanism is proposed to achieve more accurate classification results. Firstly, the point cloud is regarded as a node on the graph, the k-nearest neighbor algorithm is used to compose the graph and the information between points is dynamically captured by stacking multiple graph convolution layers;then, with the assistance of 2D experience of attention mechanism, an attention mechanism which has the capability to integrate more attention to point cloud spatial and channel information is introduced to increase the feature information of point cloud, aggregate local useful features and suppress useless features. Through the classification experiments on ModelNet40 dataset, the experimental results show that compared with PointNet network without considering the local feature information of the point cloud, the average classification accuracy of the proposed model has a 4.4% improvement and the overall classification accuracy has a 4.4% improvement. Compared with other networks, the classification accuracy of the proposed model has also been improved.

Superimposed Attention Mechanism-Based CNN Network for Reading Comprehension and Question Answering

In recent years, end-to-end models have been widely used in the fields of machine comprehension (MC) and question answering (QA). Recurrent neural network (RNN) or convolutional neural network (CNN) is combined with attention mechanism to construct models to improve their accuracy. However, a single attention mechanism does not fully express the meaning of the text. In this paper, recurrent neural network is replaced with the convolutional neural network to process the text, and a superimposed attention mechanism is proposed. The model was constructed by combining a convolutional neural network with a superimposed attention mechanism. It shows that good results are achieved on the Stanford question answering dataset (SQuAD).

基于CNN-BiGRU-Attention的短期电力负荷预测

针对目前电力负荷数据随机性强,影响因素复杂,传统单一预测模型精度低的问题,结合卷积神经网络(Convolutional neural network,CNN)、双向门控循环单元(Bi-directional gated recurrent unit,BiGRU)以及注意力机制(Attention)在短期电力负荷预测上的不同优点,提出一种基于CNN-BiGRU-Attention的混合预测模型。该方法首先通过CNN对历史负荷和气象数据进行初步特征提取,然后利用BiGRU进一步挖掘特征数据间时序关联,再引入注意力机制,对BiGRU输出状态给与不同权重,强化关键特征,最后完成负荷预测。试验结果表明,该模型的平均绝对百分比误差(Mean absolute percentage error,MAPE)、均方根误差(Root mean square error,RMSE)、判定系数(R-square,R~2)分别为0.167%、0.057%、0.993,三项指标明显优于其他模型,具有更高的预测精度和稳定性,验证了模型在短期负荷预测中的优势。

基于CNN-LSTM-Attention的月生活需水预测研究

需水预测是进行水资源配置的重要部分,对于水资源合理开发利用和社会可持续发展有重要指导意义.本文以陕西省为研究区,结合大数据分析法,提出一种基于CNN-LSTM-Attention的月生活需水预测模型.首先,通过卷积神经网络(convolutional neural networks,CNN)提取数据动态变化特征,然后利用长短期记忆(long short-term memory,LSTM)网络对提取的特征进行学习训练,最后使用注意力(attention)机制分配LSTM隐含层不同权重,预测月生活需水量并对比实际数据.结果表明,CNN-LSTM-Attention模型的相对平均误差值和决定系数(R2)分别为2.54%、0.95,满足预测精度需求,相比于LSTM模型预测精度更高.进一步证明了模型预测的合理性,可为陕西省水资源规划提供指导.

基于SABO-GRU-Attention的锂电池SOC估计

提出一种基于SABO-GRU-Attention(subtraction average based optimizer-gate recurrent unitattention)的锂电池SOC(state of charge)估计方法。采用基于平均减法优化算法自适应更新GRU神经网络的超参数,融合SE(squeeze and excitation)注意力机制自适应分配各通道权重,提高学习效率。对马里兰大学电池数据集进行预处理,输入电压、电流参数,进行锂电池充放电仿真实验,并搭建锂电池荷电状态实验平台进行储能锂电池充放电实验。结果表明,提出的SOC神经网络估计模型明显优于LSTM、GRU以及PSO-GRU等模型,具有较高的估计精度与应用价值。

融合RoBERTa-GCN-Attention的隐喻识别与情感分类模型

在隐喻识别与隐喻情感分类任务的联合研究中,现有多任务学习模型存在对隐喻语料中的上下文语义信息和句法结构信息提取不够准确,并且缺乏对粗细两种粒度信息同时捕捉的问题.针对第1个问题,首先改进了传统的RoBERTa模型,在原有的自注意力机制中引入上下文信息,以此提取上下文中重要的隐喻语义特征;其次在句法依存树上使用图卷积网络提取隐喻句中的句法结构信息.针对第2个问题,使用双层注意力机制,分别聚焦于单词和句子层面中对隐喻识别和情感分类有贡献的特征信息.在两类任务6个数据集上的对比实验结果表明,该模型相比基线模型性能均有提升.

Attention-based long short-term memory fully convolutional network for chemical process fault diagnosis

A correct and timely fault diagnosis is important for improving the safety and reliability of chemical processes. With the advancement of big data technology, data-driven fault diagnosis methods are being extensively used and still have considerable potential. In recent years, methods based on deep neural networks have made significant breakthroughs, and fault diagnosis methods for industrial processes based on deep learning have attracted considerable research attention. Therefore, we propose a fusion deeplearning algorithm based on a fully convolutional neural network(FCN) to extract features and build models to correctly diagnose all types of faults. We use long short-term memory(LSTM) units to expand our proposed FCN so that our proposed deep learning model can better extract the time-domain features of chemical process data. We also introduce the attention mechanism into the model, aimed at highlighting the importance of features, which is significant for the fault diagnosis of chemical processes with many features. When applied to the benchmark Tennessee Eastman process, our proposed model exhibits impressive performance, demonstrating the effectiveness of the attention-based LSTM FCN in chemical process fault diagnosis.

An Improved Attention Parameter Setting Algorithm Based on Award Learning Mechanism

The setting of attention parameters plays a role in the performance of synergetic neural network based on PFAP model. This paper first analyzes the attention parameter setting algorithm based on award-penalty learning mechanism. Then, it presents an improved algorithm to overcome its drawbacks. The experimental results demonstrate that the novel algorithm is better than the original one under the same circumstances.

基于SSA-CG-Attention模型的多因素采煤工作面涌水量预测

矿井工作面涌水量预测对确保矿山安全、优化资源配置、提高工作效率等都具有重要作用。为提高预测结果的准确性和稳定性,基于钻孔水位和微震能量数据与涌水量的强关联性,选择其作为多因素特征变量,提出SSA-CG-Attention多因素矿井工作面涌水量预测模型。该模型在门控循环单元(GatedRecurrentUnit,GRU)提取时序特征的基础上,与卷积神经网络(ConvolutionalNeuralNet-work,CNN)融合形成新的网络结构提取数据的有效非线性局部特征,并且加入注意力机制(Atten-tion),在预测过程中将注意力集中在输入元素上,提高模型的准确性。最后通过麻雀搜索算法(Spar-row Search Algorithm,SSA)优化模型参数,避免局部最优解的问题。将提出的模型分别与传统的BP神经网络、LSTM、GRU单因素涌水量预测模型以及MLP、SLP、SVR、LSTM、GRU、SSA-LSTM、SSA-GRU多因素涌水量预测模型的预测结果进行对比分析,结果表明:SSA算法以最少迭代次数快速寻优,避免了局部最优解的缺陷;SSA-CG-Attention多因素涌水量预测模型整体预测指标绝对误差(E_(MA))、均方根误差(E_(RMS))以及平均绝对百分比误差(E_(MAP))分别为5.24 m^(3)/h、7.25 m^(3)/h、6%,指标方差和为8.90。相较于其他预测模型预测精度更高,相较于单因素涌水量预测模型,多因素涌水量预测模型预测结果更加稳定。研究结果为矿井工作面涌水量预测提供了新的思路与方法,对矿井工作面涌水量预测及防控有着借鉴与指导作用,具有一定的理论价值和现实意义。

Narrow Pooling Clothing Classification Based on Attention Mechanism

In recent years,with the rapid development of e-commerce,people need to classify the wide variety and a large number of clothing images appearing on e-commerce platforms.In order to solve the problems of long time consumption and unsatisfactory classification accuracy arising from the classification of a large number of clothing images,researchers have begun to exploit deep learning techniques instead of traditional learning methods.The paper explores the use of convolutional neural networks(CNNs)for feature learning to enhance global feature information interactions by adding an improved hybrid attention mechanism(HAM)that fully utilizes feature weights in three dimensions:channel,height,and width.Moreover,the improved pooling layer not only captures local feature information,but also fuses global and local information to improve the misclassification problem that occurs between similar categories.Experiments on the Fashion-MNIST and DeepFashion datasets show that the proposed method significantly improves the accuracy of clothing classification(93.62%and 67.9%)compared with residual network(ResNet)and convolutional block attention module(CBAM).