Aerial target threat assessment based on gated recurrent unit and self-attention mechanism

Aerial threat assessment is a crucial link in modern air combat, whose result counts a great deal for commanders to make decisions. With the consideration that the existing threat assessment methods have difficulties in dealing with high dimensional time series target data, a threat assessment method based on self-attention mechanism and gated recurrent unit(SAGRU) is proposed. Firstly, a threat feature system including air combat situations and capability features is established. Moreover, a data augmentation process based on fractional Fourier transform(FRFT) is applied to extract more valuable information from time series situation features. Furthermore, aiming to capture key characteristics of battlefield evolution, a bidirectional GRU and SA mechanisms are designed for enhanced features.Subsequently, after the concatenation of the processed air combat situation and capability features, the target threat level will be predicted by fully connected neural layers and the softmax classifier. Finally, in order to validate this model, an air combat dataset generated by a combat simulation system is introduced for model training and testing. The comparison experiments show the proposed model has structural rationality and can perform threat assessment faster and more accurately than the other existing models based on deep learning.

Prediction and scheduling of multi-energy microgrid based on BiGRU self-attention mechanism and LQPSO

To predict renewable energy sources such as solar power in microgrids more accurately,a hybrid power prediction method is presented in this paper.First,the self-attention mechanism is introduced based on a bidirectional gated recurrent neural network(BiGRU)to explore the time-series characteristics of solar power output and consider the influence of different time nodes on the prediction results.Subsequently,an improved quantum particle swarm optimization(QPSO)algorithm is proposed to optimize the hyperparameters of the combined prediction model.The final proposed LQPSO-BiGRU-self-attention hybrid model can predict solar power more effectively.In addition,considering the coordinated utilization of various energy sources such as electricity,hydrogen,and renewable energy,a multi-objective optimization model that considers both economic and environmental costs was constructed.A two-stage adaptive multi-objective quantum particle swarm optimization algorithm aided by a Lévy flight,named MO-LQPSO,was proposed for the comprehensive optimal scheduling of a multi-energy microgrid system.This algorithm effectively balances the global and local search capabilities and enhances the solution of complex nonlinear problems.The effectiveness and superiority of the proposed scheme are verified through comparative simulations.

Clothing Parsing Based on Multi-Scale Fusion and Improved Self-Attention Mechanism

Due to the lack of long-range association and spatial location information,fine details and accurate boundaries of complex clothing images cannot always be obtained by using the existing deep learning-based methods.This paper presents a convolutional structure with multi-scale fusion to optimize the step of clothing feature extraction and a self-attention module to capture long-range association information.The structure enables the self-attention mechanism to directly participate in the process of information exchange through the down-scaling projection operation of the multi-scale framework.In addition,the improved self-attention module introduces the extraction of 2-dimensional relative position information to make up for its lack of ability to extract spatial position features from clothing images.The experimental results based on the colorful fashion parsing dataset(CFPD)show that the proposed network structure achieves 53.68%mean intersection over union(mIoU)and has better performance on the clothing parsing task.

Hierarchical multihead self-attention for time-series-based fault diagnosis

Fault diagnosis is important for maintaining the safety and effectiveness of chemical process.Considering the multivariate,nonlinear,and dynamic characteristic of chemical process,many time-series-based data-driven fault diagnosis methods have been developed in recent years.However,the existing methods have the problem of long-term dependency and are difficult to train due to the sequential way of training.To overcome these problems,a novel fault diagnosis method based on time-series and the hierarchical multihead self-attention(HMSAN)is proposed for chemical process.First,a sliding window strategy is adopted to construct the normalized time-series dataset.Second,the HMSAN is developed to extract the time-relevant features from the time-series process data.It improves the basic self-attention model in both width and depth.With the multihead structure,the HMSAN can pay attention to different aspects of the complicated chemical process and obtain the global dynamic features.However,the multiple heads in parallel lead to redundant information,which cannot improve the diagnosis performance.With the hierarchical structure,the redundant information is reduced and the deep local time-related features are further extracted.Besides,a novel many-to-one training strategy is introduced for HMSAN to simplify the training procedure and capture the long-term dependency.Finally,the effectiveness of the proposed method is demonstrated by two chemical cases.The experimental results show that the proposed method achieves a great performance on time-series industrial data and outperforms the state-of-the-art approaches.

Self-Attention Spatio-Temporal Deep Collaborative Network for Robust FDIA Detection in Smart Grids

False data injection attack(FDIA)can affect the state estimation of the power grid by tampering with the measured value of the power grid data,and then destroying the stable operation of the smart grid.Existing work usually trains a detection model by fusing the data-driven features from diverse power data streams.Data-driven features,however,cannot effectively capture the differences between noisy data and attack samples.As a result,slight noise disturbances in the power grid may cause a large number of false detections for FDIA attacks.To address this problem,this paper designs a deep collaborative self-attention network to achieve robust FDIA detection,in which the spatio-temporal features of cascaded FDIA attacks are fully integrated.Firstly,a high-order Chebyshev polynomials-based graph convolution module is designed to effectively aggregate the spatio information between grid nodes,and the spatial self-attention mechanism is involved to dynamically assign attention weights to each node,which guides the network to pay more attention to the node information that is conducive to FDIA detection.Furthermore,the bi-directional Long Short-Term Memory(LSTM)network is introduced to conduct time series modeling and long-term dependence analysis for power grid data and utilizes the temporal selfattention mechanism to describe the time correlation of data and assign different weights to different time steps.Our designed deep collaborative network can effectively mine subtle perturbations from spatiotemporal feature information,efficiently distinguish power grid noise from FDIA attacks,and adapt to diverse attack intensities.Extensive experiments demonstrate that our method can obtain an efficient detection performance over actual load data from New York Independent System Operator(NYISO)in IEEE 14,IEEE 39,and IEEE 118 bus systems,and outperforms state-of-the-art FDIA detection schemes in terms of detection accuracy and robustness.

基于DCNN网络及Self-Attention-BiGRU机制的轴承剩余寿命预测

深度神经网络在剩余寿命预测(RUL)领域得到了广泛的应用。传统的滚动轴承寿命预测模型存在预测精确度较低、鲁棒性较弱的问题。为了进一步提升预测模型的精确度以及鲁棒性,提出了一种融合深度卷积神经网络(DCNN)、双向门控循环单元(BiGRU)以及自注意力机制(Self-Attention)三种模块的滚动轴承剩余使用寿命预测模型。首先,利用DCNN网络对原始振动信号的时域特征、频域特征进行了提取;然后,使用不确定量化的方法对提取到的特征进行了评价和筛选,利用筛选过后的特征构建了新的替代特征集;最后,利用Self-Attention-BiGRU网络对轴承的剩余使用寿命进行了预测,并在IEEE PHM2012数据集上进行了验证。实验结果表明:相较于BiGRU、GRU和BiLSTM三种模型的预测结果,基于DCNN及Self-Attention-BiGRU方法的预测结果最优,两项误差值:平均绝对误差(MAE)、均方根误差(RMSE)最低,其中工况一的一号轴承RUL预测的MAE值相较于BiGRU、GRU以及BiLSTM网络分别下降了7.0%、7.4%和6.5%,RMSE值相较于其他三种模型分别下降了7.6%、8.4%和6.9%,预测的Score值最高,分值为0.985。通过不同数据集的划分,证明了该方法在轴承RUL预测时的强鲁棒性。实验结果验证了基于DCNN网络及Self-Attention-BiGRU模型在轴承剩余使用寿命预测中的有效性。

Sentiment classification model for bullet screen based on self-attention mechanism

With the development of short video industry,video and bullet screen have become important ways to spread public opinions.Public attitudes can be timely obtained through emotional analysis on bullet screen,which can also reduce difficulties in management of online public opinions.A convolutional neural network model based on multi-head attention is proposed to solve the problem of how to effectively model relations among words and identify key words in emotion classification tasks with short text contents and lack of complete context information.Firstly,encode word positions so that order information of input sequences can be used by the model.Secondly,use a multi-head attention mechanism to obtain semantic expressions in different subspaces,effectively capture internal relevance and enhance dependent relationships among words,as well as highlight emotional weights of key emotional words.Then a dilated convolution is used to increase the receptive field and extract more features.On this basis,the above multi-attention mechanism is combined with a convolutional neural network to model and analyze the seven emotional categories of bullet screens.Testing from perspectives of model and dataset,experimental results can validate effectiveness of our approach.Finally,emotions of bullet screens are visualized to provide data supports for hot event controls and other fields.

Keyphrase Generation Based on Self-Attention Mechanism

Keyphrase greatly provides summarized and valuable information.This information can help us not only understand text semantics,but also organize and retrieve text content effectively.The task of automatically generating it has received considerable attention in recent decades.From the previous studies,we can see many workable solutions for obtaining keyphrases.One method is to divide the content to be summarized into multiple blocks of text,then we rank and select the most important content.The disadvantage of this method is that it cannot identify keyphrase that does not include in the text,let alone get the real semantic meaning hidden in the text.Another approach uses recurrent neural networks to generate keyphrases from the semantic aspects of the text,but the inherently sequential nature precludes parallelization within training examples,and distances have limitations on context dependencies.Previous works have demonstrated the benefits of the self-attention mechanism,which can learn global text dependency features and can be parallelized.Inspired by the above observation,we propose a keyphrase generation model,which is based entirely on the self-attention mechanism.It is an encoder-decoder model that can make up the above disadvantage effectively.In addition,we also consider the semantic similarity between keyphrases,and add semantic similarity processing module into the model.This proposed model,which is demonstrated by empirical analysis on five datasets,can achieve competitive performance compared to baseline methods.

Growth mechanism of atomic-layer-deposited TiAlC metal gate based on TiCl4 and TMA precursors

TiAIC metal gate for the metal-oxide-semiconductor field-effect-transistor （MOSFET） is grown by the atorr/ic layer deposition method using TiCI4 and AI（CH3） 3 （TMA） as precursors. It is found that the major PrOduct of the TIC14 and TMA reaction is TiA1C, and the components of C and A1 are found to increase with higher growth temperature. The reaction mechanism is investigated by using x-ray photoemission spectroscopy （XPS）, Fourier transform infrared spectroscopy （FFIR）, and scanning electron microscope （SEM）. The reaction mechanism is as follows. Ti is generated through the reduction of TiCI4 by TMA. The reductive behavior of TMA involves the formation of ethane. The Ti from the reduction of TIC14 by TMA reacts with ethane easily forming heterogenetic TiCH2, TiCH=CH2 and TiC fragments. In addition, TMA thermally decomposes, driving A1 into the TiC film and leading to TiA1C formation. With the growth temperature increasing, TMA decomposes more severely, resulting in more C and A1 in the TiA1C film. Thus, the film composition can be controlled by the growth temperature to a certain extent.

Fake News Detection Based on Cross-Modal Message Aggregation and Gated Fusion Network

Social media has become increasingly significant in modern society,but it has also turned into a breeding ground for the propagation of misleading information,potentially causing a detrimental impact on public opinion and daily life.Compared to pure text content,multmodal content significantly increases the visibility and share ability of posts.This has made the search for efficient modality representations and cross-modal information interaction methods a key focus in the field of multimodal fake news detection.To effectively address the critical challenge of accurately detecting fake news on social media,this paper proposes a fake news detection model based on crossmodal message aggregation and a gated fusion network(MAGF).MAGF first uses BERT to extract cumulative textual feature representations and word-level features,applies Faster Region-based ConvolutionalNeuralNetwork(Faster R-CNN)to obtain image objects,and leverages ResNet-50 and Visual Geometry Group-19(VGG-19)to obtain image region features and global features.The image region features and word-level text features are then projected into a low-dimensional space to calculate a text-image affinity matrix for cross-modal message aggregation.The gated fusion network combines text and image region features to obtain adaptively aggregated features.The interaction matrix is derived through an attention mechanism and further integrated with global image features using a co-attention mechanism to producemultimodal representations.Finally,these fused features are fed into a classifier for news categorization.Experiments were conducted on two public datasets,Twitter and Weibo.Results show that the proposed model achieves accuracy rates of 91.8%and 88.7%on the two datasets,respectively,significantly outperforming traditional unimodal and existing multimodal models.

Deep Learning Applied to Computational Mechanics:A Comprehensive Review,State of the Art,and the Classics

Three recent breakthroughs due to AI in arts and science serve as motivation:An award winning digital image,protein folding,fast matrix multiplication.Many recent developments in artificial neural networks,particularly deep learning(DL),applied and relevant to computational mechanics(solid,fluids,finite-element technology)are reviewed in detail.Both hybrid and pure machine learning(ML)methods are discussed.Hybrid methods combine traditional PDE discretizations with ML methods either(1)to help model complex nonlinear constitutive relations,(2)to nonlinearly reduce the model order for efficient simulation(turbulence),or(3)to accelerate the simulation by predicting certain components in the traditional integration methods.Here,methods(1)and(2)relied on Long-Short-Term Memory(LSTM)architecture,with method(3)relying on convolutional neural networks.Pure ML methods to solve(nonlinear)PDEs are represented by Physics-Informed Neural network(PINN)methods,which could be combined with attention mechanism to address discontinuous solutions.Both LSTM and attention architectures,together with modern and generalized classic optimizers to include stochasticity for DL networks,are extensively reviewed.Kernel machines,including Gaussian processes,are provided to sufficient depth for more advanced works such as shallow networks with infinite width.Not only addressing experts,readers are assumed familiar with computational mechanics,but not with DL,whose concepts and applications are built up from the basics,aiming at bringing first-time learners quickly to the forefront of research.History and limitations of AI are recounted and discussed,with particular attention at pointing out misstatements or misconceptions of the classics,even in well-known references.Positioning and pointing control of a large-deformable beam is given as an example.

融合CNN-BiGRU和注意力机制的网络入侵检测模型

为提高网络入侵检测模型特征提取能力和分类准确率,提出了一种融合双向门控循环单元(CNN-BiGRU)和注意力机制的网络入侵检测模型.使用CNN有效提取流量数据集中的非线性特征;双向门控循环单元(BiGRU)提取数据集中的时序特征,最后融合注意力机制对不同类型流量数据通过加权的方式进行重要程度的区分,从而整体提高该模型特征提取与分类的性能.实验结果表明:其整体精确率比双向长短期记忆网络(BiLSTM)模型提升了2.25%.K折交叉验证结果表明:该模型泛化性能良好,避免了过拟合现象的发生,印证了该模型的有效性与合理性.

基于双层注意力和深度自编码器的时间序列异常检测模型

目前时间序列通常具有弱周期性以及高度复杂的相关性特征,传统的时间序列异常检测方法难以检测此类异常。针对这一问题,提出了一种新的无监督时间序列异常检测模型(DA-CBG-AE)。首先,使用新型滑动窗口方法,针对时间序列周期性设置滑动窗口大小;其次,采用卷积神经网络提取时间序列高维度空间特征;然后,提出具有堆叠式Dropout双向门循环单元网络作为自编码器的基本结构,从而捕捉时间序列的相关性特征;最后,引入双层注意力机制,进一步提取特征,选择更加关键的时间序列,从而提高异常检测准确率。为了验证该模型的有效性,将DA-CBG-AE与6种基准模型在8个数据集上进行比较。最终的实验结果表明,DA-CBG-AE获得了最优的F1值(0.863),并且其检测性能相比最新的基准模型Tad-GAN高出25.25%。

基于注意力机制和特征融合的股票预测方法

基于人工智能在金融数据中的应用,提出了一种新的股票预测方法,称为AFG。AFG使用位置编码和时间编码获取股票数据的位置信息和时间信息,然后通过门控循环单元和多头自注意力机制对股票数据分别进行特征提取。在将两类股票特征融合之后,由全连接层导出最终的股票预测曲线。

基于Electra预训练模型并融合依存关系的中文事件检测模型

事件检测是信息提取领域的一个重要研究方向。现存的事件检测模型受到语言模型训练目标的限制,只能被动地获取词与词之间的依赖关系,使得模型在训练的过程中过多地关注与训练目标不相关的成分,从而导致检测结果错误。以往的研究表明,充分理解上下文信息对于基于深度学习的事件检测技术至关重要。因此,在Electra预训练模型的基础上,引入KVMN网络来捕捉单词之间的依赖关系,以增强单词的语义特征,并采用了一种门控机制来加权这些特征。然后,为了解决中文事件检测中模型识别错误决策的问题,在输入中加入负样本,对不同样本加入不同程度的噪声,使模型学习更好的嵌入表示,有效提高了模型对未知样本的泛化能力。最后,在公共数据集LEVEN上的实验结果表明,该方法优于现有方法,取得了93.43%的F1值。

门控机制的图像分类网络

为了提取更具表达能力和区分度的重点特征,减少网络传递时关键特征的流失,提高神经网络图像分类能力,提出一种新的门控机制图像分类网络(image classification Network of Gating Mechanism,GMNet).首先,使用门控卷积提取浅层特征,通过门控机制选择性地进行卷积操作,提高网络对原始图像关键特征的提取能力;其次,设计了一种插值门控卷积(Interpolation Gated Convolution,IGC)模块,利用Lanczos插值与门控卷积相结合,强化浅层特征的同时提取更具区分度的特征,提高特征的非线性表达能力;然后,设计了大核门控注意力机制(Large kernel Gated Attention Mechanism,LGAM)模块,将大核注意力与门控卷积相融合,实现了特征的选择性增强和选择性融合,提高关键区域特征的贡献度;最后,将大核门控注意力机制模块嵌入到残差分支中,让模型更有效地学习输入数据的特征和上下文信息,减少关键特征在网络信息传递时流失,提高网络的分类能力.本文方法在图像数据集CIFAR-10、CIFAR100、SVHN、Imagenette、Imagewoof上分别达到了97.05%、83.68%、97.68%、90.60%、83.05%的分类准确率,与当前先进的方法相比分别平均提高了3.26%、7.08%、3.44%、2.65%、5.02%.与现有主流网络模型相较,本文门控机制图像分类网络能够增强特征的非线性表达能力,提取更具表达能力和区分度的重点特征,减少关键特征流失,提高关键区域特征的贡献度,有效地提高神经网络图像分类能力.

基于射击时机的坦克射击门分析及改进

针对坦克的射击门控制逻辑,分析现有射击门在坦克高速行进条件下在射击时机判断上的不足,提出基于预测机制的射击门改进方法,该射击门在考虑射击时延和炮口切向速度影响的情况下,去除了身管必须满足进入简单射击门的角度约束,可更大限度地把握有效射击时机,数字仿真结果验证了改进射击门的优越性。

基于CNN-BiGRU-Attention的短期电力负荷预测

针对目前电力负荷数据随机性强,影响因素复杂,传统单一预测模型精度低的问题,结合卷积神经网络(Convolutional neural network,CNN)、双向门控循环单元(Bi-directional gated recurrent unit,BiGRU)以及注意力机制(Attention)在短期电力负荷预测上的不同优点,提出一种基于CNN-BiGRU-Attention的混合预测模型。该方法首先通过CNN对历史负荷和气象数据进行初步特征提取,然后利用BiGRU进一步挖掘特征数据间时序关联,再引入注意力机制,对BiGRU输出状态给与不同权重,强化关键特征,最后完成负荷预测。试验结果表明,该模型的平均绝对百分比误差(Mean absolute percentage error,MAPE)、均方根误差(Root mean square error,RMSE)、判定系数(R-square,R~2)分别为0.167%、0.057%、0.993,三项指标明显优于其他模型,具有更高的预测精度和稳定性,验证了模型在短期负荷预测中的优势。

基于注意力机制的CNN-BiLSTM的IGBT剩余使用寿命预测

针对绝缘栅双极型晶体管(IGBT)可靠性问题,提出了一种融合卷积神经网络(CNN)、双向长短期记忆(BiLSTM)网络和注意力机制的剩余使用寿命(RUL)预测模型,可用于IGBT的寿命预测。模型中使用CNN提取特征参数,BiLSTM提取时序信息,注意力机制加权处理特征参数。使用IGBT加速老化数据集对提出的模型进行验证。结果表明,对比自回归差分移动平均(ARIMA)、长短期记忆(LSTM)、多层LSTM(Multi-LSTM)、 BiLSTM预测模型,在均方根误差和决定系数等评价指标方面该模型的性能最优。验证了提出的寿命预测模型对IGBT失效预测是有效的。

环核苷酸门控离子通道及其功能的研究进展

环核苷酸门控(CNG)离子通道是一种非选择性的四聚体阳离子通道,可以直接被细胞内信使小分子——环核苷酸活化,是钙离子进入细胞的主要通道之一。CNG通道蛋白由6种不同基因编码:4个A亚单位和2个B亚单位。CNG离子通道的活性可被钙离子/钙调素(Ca2+/CaM)及磷酸化或膜上磷酸肌醇作用调节,从而改变细胞内钙离子浓度,参与多种生物学功能的调控。自从在视杆细胞中发现CNG离子通道以来,经历了对其生理功能、克隆相关基因、理解调控方式、解析晶体结构和开发相关的基因治疗方法等研究过程,在视觉和嗅觉感觉神经元(OSNs)的信号转导中发挥着重要作用。现就CNG离子通道的功能、结构、调控机制及其与相关疾病关系等方面进行简要综述,以期为CNG离子通道相关疾病的治疗提供理论依据。