融合Self-Attention机制和n-gram卷积核的印尼语复合名词自动识别方法研究

针对印尼语复合名词短语自动识别,提出一种融合Self-Attention机制、n-gram卷积核的神经网络和统计模型相结合的方法,改进现有的多词表达抽取模型。在现有SHOMA模型的基础上,使用多层CNN和Self-Attention机制进行改进。对Universal Dependencies公开的印尼语数据进行复合名词短语自动识别的对比实验,结果表明:TextCNN+Self-Attention+CRF模型取得32.20的短语多词识别F1值和32.34的短语单字识别F1值,比SHOMA模型分别提升了4.93%和3.04%。

融合self-attention机制的卷积神经网络文本分类模型

传统的文本分类算法采用词向量表示文本,忽视了上下文语境中词义的变化.本文通过引入self-attention机制处理词向量,提出一种卷积神经网络模型与关键词提取技术相结合的文本分类模型.该模型对文档进行self-attention操作,以抽取关键信息,构建文档特征图,根据卷积神经网络模型和关键词提取技术实现特征向量的分类.在真实数据集上进行性能分析,并与循环神经网络模型、长短时记忆网络模型进行比较,结果表明该分类模型有效地提高了分类的准确性.

基于DCNN网络及Self-Attention-BiGRU机制的轴承剩余寿命预测

深度神经网络在剩余寿命预测(RUL)领域得到了广泛的应用。传统的滚动轴承寿命预测模型存在预测精确度较低、鲁棒性较弱的问题。为了进一步提升预测模型的精确度以及鲁棒性,提出了一种融合深度卷积神经网络(DCNN)、双向门控循环单元(BiGRU)以及自注意力机制(Self-Attention)三种模块的滚动轴承剩余使用寿命预测模型。首先,利用DCNN网络对原始振动信号的时域特征、频域特征进行了提取;然后,使用不确定量化的方法对提取到的特征进行了评价和筛选,利用筛选过后的特征构建了新的替代特征集;最后,利用Self-Attention-BiGRU网络对轴承的剩余使用寿命进行了预测,并在IEEE PHM2012数据集上进行了验证。实验结果表明:相较于BiGRU、GRU和BiLSTM三种模型的预测结果,基于DCNN及Self-Attention-BiGRU方法的预测结果最优,两项误差值:平均绝对误差(MAE)、均方根误差(RMSE)最低,其中工况一的一号轴承RUL预测的MAE值相较于BiGRU、GRU以及BiLSTM网络分别下降了7.0%、7.4%和6.5%,RMSE值相较于其他三种模型分别下降了7.6%、8.4%和6.9%,预测的Score值最高,分值为0.985。通过不同数据集的划分,证明了该方法在轴承RUL预测时的强鲁棒性。实验结果验证了基于DCNN网络及Self-Attention-BiGRU模型在轴承剩余使用寿命预测中的有效性。

基于Self-Attention-BiLSTM网络的西瓜种苗叶片氮磷钾含量高光谱检测方法

元素含量无损检测技术可以为植物生长发育的环境精准调控提供关键实时数据。以西瓜苗为例,提出了一种基于图谱特征融合的氮磷钾含量深度学习检测方法。首先,使用高光谱仪拍摄西瓜苗叶片的高光谱图像,使用连续流动化学分析仪测定叶片的3种元素含量。然后,采用基线偏移校正(BOC)叠加高斯平滑滤波(GF)的光谱预处理方法和随机森林算法(RF)建立预测模型,基于竞争性自适应重加权采样(CARS)和连续投影算法(SPA)2种算法初步筛选出特征波长,再综合考虑波长数和建模精度设计了一种最优波长评价方法,将波长数进一步减少到3~4个。最后,提取使用U-Net网络分割的彩色图像颜色和纹理特征,和光谱反射率特征一起作为输入,基于自注意力机制-双向长短时记忆(Self-Attention-BiLSTM)网络构建了3种元素含量的预测模型。实验结果表明,氮磷钾含量预测的R2分别为0.961、0.954、0.958,RMSE分别为0.294%、0.262%、0.196%,实现了很好的建模效果。使用该模型对另2个品种西瓜进行测试,R2超过0.899、RMSE小于0.498%,表明该模型具有很好的泛化性。该高光谱建模方法使用少量波长光谱即实现了高精度检测,在精度和效率上达成了很好的平衡,为后续便携式高光谱检测装备开发奠定了理论基础。

Hierarchical multihead self-attention for time-series-based fault diagnosis

Fault diagnosis is important for maintaining the safety and effectiveness of chemical process.Considering the multivariate,nonlinear,and dynamic characteristic of chemical process,many time-series-based data-driven fault diagnosis methods have been developed in recent years.However,the existing methods have the problem of long-term dependency and are difficult to train due to the sequential way of training.To overcome these problems,a novel fault diagnosis method based on time-series and the hierarchical multihead self-attention(HMSAN)is proposed for chemical process.First,a sliding window strategy is adopted to construct the normalized time-series dataset.Second,the HMSAN is developed to extract the time-relevant features from the time-series process data.It improves the basic self-attention model in both width and depth.With the multihead structure,the HMSAN can pay attention to different aspects of the complicated chemical process and obtain the global dynamic features.However,the multiple heads in parallel lead to redundant information,which cannot improve the diagnosis performance.With the hierarchical structure,the redundant information is reduced and the deep local time-related features are further extracted.Besides,a novel many-to-one training strategy is introduced for HMSAN to simplify the training procedure and capture the long-term dependency.Finally,the effectiveness of the proposed method is demonstrated by two chemical cases.The experimental results show that the proposed method achieves a great performance on time-series industrial data and outperforms the state-of-the-art approaches.

SMSTracker:A Self-Calibration Multi-Head Self-Attention Transformer for Visual Object Tracking

Visual object tracking plays a crucial role in computer vision.In recent years,researchers have proposed various methods to achieve high-performance object tracking.Among these,methods based on Transformers have become a research hotspot due to their ability to globally model and contextualize information.However,current Transformer-based object tracking methods still face challenges such as low tracking accuracy and the presence of redundant feature information.In this paper,we introduce self-calibration multi-head self-attention Transformer(SMSTracker)as a solution to these challenges.It employs a hybrid tensor decomposition self-organizing multihead self-attention transformermechanism,which not only compresses and accelerates Transformer operations but also significantly reduces redundant data,thereby enhancing the accuracy and efficiency of tracking.Additionally,we introduce a self-calibration attention fusion block to resolve common issues of attention ambiguities and inconsistencies found in traditional trackingmethods,ensuring the stability and reliability of tracking performance across various scenarios.By integrating a hybrid tensor decomposition approach with a self-organizingmulti-head self-attentive transformer mechanism,SMSTracker enhances the efficiency and accuracy of the tracking process.Experimental results show that SMSTracker achieves competitive performance in visual object tracking,promising more robust and efficient tracking systems,demonstrating its potential to providemore robust and efficient tracking solutions in real-world applications.

A Self-Attention Based Dynamic Resource Management for Satellite-Terrestrial Networks

The satellite-terrestrial networks possess the ability to transcend geographical constraints inherent in traditional communication networks,enabling global coverage and offering users ubiquitous computing power support,which is an important development direction of future communications.In this paper,we take into account a multi-scenario network model under the coverage of low earth orbit(LEO)satellite,which can provide computing resources to users in faraway areas to improve task processing efficiency.However,LEO satellites experience limitations in computing and communication resources and the channels are time-varying and complex,which makes the extraction of state information a daunting task.Therefore,we explore the dynamic resource management issue pertaining to joint computing,communication resource allocation and power control for multi-access edge computing(MEC).In order to tackle this formidable issue,we undertake the task of transforming the issue into a Markov decision process(MDP)problem and propose the self-attention based dynamic resource management(SABDRM)algorithm,which effectively extracts state information features to enhance the training process.Simulation results show that the proposed algorithm is capable of effectively reducing the long-term average delay and energy consumption of the tasks.

An Aerial Target Recognition Algorithm Based on Self-Attention and LSTM

In the application of aerial target recognition,on the one hand,the recognition error produced by the single measurement of the sensor is relatively large due to the impact of noise.On the other hand,it is difficult to apply machine learning methods to improve the intelligence and recognition effect due to few or no actual measurement samples.Aiming at these problems,an aerial target recognition algorithm based on self-attention and Long Short-Term Memory Network(LSTM)is proposed.LSTM can effectively extract temporal dependencies.The attention mechanism calculates the weight of each input element and applies the weight to the hidden state of the LSTM,thereby adjusting the LSTM’s attention to the input.This combination retains the learning ability of LSTM and introduces the advantages of the attention mechanism,making the model have stronger feature extraction ability and adaptability when processing sequence data.In addition,based on the prior information of the multidimensional characteristics of the target,the three-point estimation method is adopted to simulate an aerial target recognition dataset to train the recognition model.The experimental results show that the proposed algorithm achieves more than 91%recognition accuracy,lower false alarm rate and higher robustness compared with the multi-attribute decision-making(MADM)based on fuzzy numbers.

Missing Value Imputation for Radar-Derived Time-Series Tracks of Aerial Targets Based on Improved Self-Attention-Based Network

The frequent missing values in radar-derived time-series tracks of aerial targets(RTT-AT)lead to significant challenges in subsequent data-driven tasks.However,the majority of imputation research focuses on random missing(RM)that differs significantly from common missing patterns of RTT-AT.The method for solving the RM may experience performance degradation or failure when applied to RTT-AT imputation.Conventional autoregressive deep learning methods are prone to error accumulation and long-term dependency loss.In this paper,a non-autoregressive imputation model that addresses the issue of missing value imputation for two common missing patterns in RTT-AT is proposed.Our model consists of two probabilistic sparse diagonal masking self-attention(PSDMSA)units and a weight fusion unit.It learns missing values by combining the representations outputted by the two units,aiming to minimize the difference between the missing values and their actual values.The PSDMSA units effectively capture temporal dependencies and attribute correlations between time steps,improving imputation quality.The weight fusion unit automatically updates the weights of the output representations from the two units to obtain a more accurate final representation.The experimental results indicate that,despite varying missing rates in the two missing patterns,our model consistently outperforms other methods in imputation performance and exhibits a low frequency of deviations in estimates for specific missing entries.Compared to the state-of-the-art autoregressive deep learning imputation model Bidirectional Recurrent Imputation for Time Series(BRITS),our proposed model reduces mean absolute error(MAE)by 31%~50%.Additionally,the model attains a training speed that is 4 to 8 times faster when compared to both BRITS and a standard Transformer model when trained on the same dataset.Finally,the findings from the ablation experiments demonstrate that the PSDMSA,the weight fusion unit,cascade network design,and imputation loss enhance imputation performance and confirm the efficacy of our design.

Self-Attention Spatio-Temporal Deep Collaborative Network for Robust FDIA Detection in Smart Grids

False data injection attack(FDIA)can affect the state estimation of the power grid by tampering with the measured value of the power grid data,and then destroying the stable operation of the smart grid.Existing work usually trains a detection model by fusing the data-driven features from diverse power data streams.Data-driven features,however,cannot effectively capture the differences between noisy data and attack samples.As a result,slight noise disturbances in the power grid may cause a large number of false detections for FDIA attacks.To address this problem,this paper designs a deep collaborative self-attention network to achieve robust FDIA detection,in which the spatio-temporal features of cascaded FDIA attacks are fully integrated.Firstly,a high-order Chebyshev polynomials-based graph convolution module is designed to effectively aggregate the spatio information between grid nodes,and the spatial self-attention mechanism is involved to dynamically assign attention weights to each node,which guides the network to pay more attention to the node information that is conducive to FDIA detection.Furthermore,the bi-directional Long Short-Term Memory(LSTM)network is introduced to conduct time series modeling and long-term dependence analysis for power grid data and utilizes the temporal selfattention mechanism to describe the time correlation of data and assign different weights to different time steps.Our designed deep collaborative network can effectively mine subtle perturbations from spatiotemporal feature information,efficiently distinguish power grid noise from FDIA attacks,and adapt to diverse attack intensities.Extensive experiments demonstrate that our method can obtain an efficient detection performance over actual load data from New York Independent System Operator(NYISO)in IEEE 14,IEEE 39,and IEEE 118 bus systems,and outperforms state-of-the-art FDIA detection schemes in terms of detection accuracy and robustness.

Prediction and scheduling of multi-energy microgrid based on BiGRU self-attention mechanism and LQPSO

To predict renewable energy sources such as solar power in microgrids more accurately,a hybrid power prediction method is presented in this paper.First,the self-attention mechanism is introduced based on a bidirectional gated recurrent neural network(BiGRU)to explore the time-series characteristics of solar power output and consider the influence of different time nodes on the prediction results.Subsequently,an improved quantum particle swarm optimization(QPSO)algorithm is proposed to optimize the hyperparameters of the combined prediction model.The final proposed LQPSO-BiGRU-self-attention hybrid model can predict solar power more effectively.In addition,considering the coordinated utilization of various energy sources such as electricity,hydrogen,and renewable energy,a multi-objective optimization model that considers both economic and environmental costs was constructed.A two-stage adaptive multi-objective quantum particle swarm optimization algorithm aided by a Lévy flight,named MO-LQPSO,was proposed for the comprehensive optimal scheduling of a multi-energy microgrid system.This algorithm effectively balances the global and local search capabilities and enhances the solution of complex nonlinear problems.The effectiveness and superiority of the proposed scheme are verified through comparative simulations.

Stroke Electroencephalogram Data Synthesizing through Progressive Efficient Self-Attention Generative Adversarial Network

Early and timely diagnosis of stroke is critical for effective treatment,and the electroencephalogram(EEG)offers a low-cost,non-invasive solution.However,the shortage of high-quality patient EEG data often hampers the accuracy of diagnostic classification methods based on deep learning.To address this issue,our study designed a deep data amplification model named Progressive Conditional Generative Adversarial Network with Efficient Approximating Self Attention(PCGAN-EASA),which incrementally improves the quality of generated EEG features.This network can yield full-scale,fine-grained EEG features from the low-scale,coarse ones.Specially,to overcome the limitations of traditional generative models that fail to generate features tailored to individual patient characteristics,we developed an encoder with an effective approximating self-attention mechanism.This encoder not only automatically extracts relevant features across different patients but also reduces the computational resource consumption.Furthermore,the adversarial loss and reconstruction loss functions were redesigned to better align with the training characteristics of the network and the spatial correlations among electrodes.Extensive experimental results demonstrate that PCGAN-EASA provides the highest generation quality and the lowest computational resource usage compared to several existing approaches.Additionally,it significantly improves the accuracy of subsequent stroke classification tasks.

Aerial target threat assessment based on gated recurrent unit and self-attention mechanism

Aerial threat assessment is a crucial link in modern air combat, whose result counts a great deal for commanders to make decisions. With the consideration that the existing threat assessment methods have difficulties in dealing with high dimensional time series target data, a threat assessment method based on self-attention mechanism and gated recurrent unit(SAGRU) is proposed. Firstly, a threat feature system including air combat situations and capability features is established. Moreover, a data augmentation process based on fractional Fourier transform(FRFT) is applied to extract more valuable information from time series situation features. Furthermore, aiming to capture key characteristics of battlefield evolution, a bidirectional GRU and SA mechanisms are designed for enhanced features.Subsequently, after the concatenation of the processed air combat situation and capability features, the target threat level will be predicted by fully connected neural layers and the softmax classifier. Finally, in order to validate this model, an air combat dataset generated by a combat simulation system is introduced for model training and testing. The comparison experiments show the proposed model has structural rationality and can perform threat assessment faster and more accurately than the other existing models based on deep learning.

数字经济赋能乡村生态农产品价值实现的典型模式与形成机制分析

【目的】生态农产品价值实现是多元化推进乡村振兴的关键举措,探索数字经济赋能乡村生态农产品价值实现的典型模式和形成机制有利于培育壮大乡村绿色发展新动能,促进和美乡村建设和农民增收。【方法】通过对全国数字化先进县的典型案例进行梳理,从其关键做法,主要成效和价值体现,以及何以实现等方面系统分析数字经济赋能乡村生态农产品价值实现的典型模式和形成机制。【结果】“互联网+现代农业”融合发展、产学研合作,以及物联网与大数据技术应用是案例中数字经济赋能乡村生态农产品价值实现的3种典型模式。在此基础上,得出数字经济赋能乡村生态农产品价值实现的3条形成路径,即数字经济通过推动产业升级,为乡村生态农产品价值实现提供驱动力;数字经济通过提高乡村农产品流通效率,为乡村生态农产品价值实现创造良好的条件;数字经济通过创新商业模式,为乡村生态农产品价值实现建立有效路径。【结论】目前乡村数字经济发展仍处于较低水平,相应的政策措施和管理机制应该联合政府、企业和社会多方力量,聚焦在数字基础设施建设和人才培养等方面,推动乡村数字技术渗透的广度和深度,促进乡村生态农产品价值实现。

干旱形成机制与预测理论方法及其灾害风险特征研究进展与展望

在全球变暖背景下,干旱事件发生的频率和强度不断增大、影响不断加重,干旱发生规律的异常性和机制的复杂性也更为突出,对干旱形成机制、预测理论方法及灾害风险变化规律等方面都提出了新的挑战,也制约了当前干旱预测、预警及其灾害防控能力的提高。近年来,在国家重点基础研究发展计划(973计划)课题等多个国家级项目支持下,已在干旱灾害形成机制与预测理论方法及其风险特征方面取得了一系列新成果。通过动力诊断、数值模拟和田间试验等方法,开展了干旱形成的多因子协同作用和多尺度叠加机制、干旱致灾过程的逐阶递进特征,以及干旱灾害风险分布演化的主控因素等方面的研究。对如下几方面的新进展进行了系统总结归纳:(1)厘清了全球变暖背景下青藏高原热力、海温、夏季风、遥相关等多因子对干旱形成的作用机制。(2)发现了降水亏缺时间尺度和农作物不同生长阶段的干旱敏感性规律。(3)揭示了变暖背景下典型区域干旱灾害风险分布及其变异的新特征;构建了干旱灾害风险新概念模型。(4)研发了东亚季风区的季节和次季节干旱集成预测系统。在总结归纳已取得研究成果的基础上,对未来干旱形成机制及其灾害风险科学研究进行了展望,提出了5个重点研究方向:(1)多因子联动及其多尺度叠加效应对干旱形成的影响;(2)系统整合人类活动和决策以及相关反馈的气候模式研究;(3)揭示陆-气耦合和大气环流协同作用对干旱的影响;(4)认识干旱灾害对粮食安全和生态安全影响的关键过程;(5)提高不同气候情景下干旱预估的准确度。

线粒体在肺癌发生中的作用机制及治疗研究进展

肺癌具有高发生率、高侵袭性和高致亡率的特点,其发生发展受多方面因素影响。线粒体作为普遍存在于人体内的细胞器,通过调节细胞代谢、信号转导、氧化应激和基因组不稳定性等过程,影响肺癌的发生和发展。本文总结近年来有关线粒体靶向药物、线粒体转移和线粒体基因疗法等治疗肺癌的研究进展,探讨线粒体治疗肺癌的原理和前景,以期为肺癌的治疗提供新的思路。

探讨氧化应激与变应性鼻炎发病机制的相关性及中药防治策略

变应性鼻炎(allergic rhinitis,AR)是由免疫球蛋白E(IgE)介导的由吸入变应原诱发的鼻黏膜炎症性疾病,涉及鼻腔细胞和炎症细胞、细胞因子、介质和细胞黏附分子的各种成分参与过敏性鼻炎的过程。AR的主要临床表现为鼻痒、鼻塞、流鼻涕、打喷嚏以及嗅觉减退。目前西医的治疗手段主要包括鼻/口服皮质类固醇、白三烯受体拮抗剂、抗组胺药、肥大细胞稳定剂和短期鼻充血减轻剂,但不良反应较明显,患者精神压力大,因此深入探索变应性鼻炎的作用机制与治疗策略具有重要的临床意义。氧化应激是机体在内外环境损害下产生活性氧(ROS)所引起的疾病生理和病理异常,氧化应激加剧了炎性细胞因子的释放,与AR的发病机制密切相关,参与信号通路转导、细胞凋亡、自噬和线粒体功能障碍等病理状态,是促进AR病情发展的重要因素。传统中医药对于临床治疗过敏性疾病已有上千年历史,近年来中医药防治变应性鼻炎的临床成效也得到大量研究结果的肯定,基于氧化应激途径,进一步阐明了中药单体、活性成分及其复方可凭借多靶点、多组分和多途径的生物学优势对AR起到整体性保护作用。作者通过分析近年来国内外中医药相关研究进展,对氧化应激在AR中的防治作用及机制进行综述,以期为中医药临床防治AR的作用机制研究提供进一步的理论依据。

2023年12月18日甘肃积石山6.2级地震震源机制解

使用区域台网资料,采用CAP方法反演了2023年12月18日甘肃省积石山6.2级地震的震源机制解,结果显示,该地震为逆冲型地震,与GCMT、 GFZ和USGS的震源机制解基本一致。震源机制解节面Ⅱ的走向与积石山东缘断裂大致相同。

中国页岩油勘探开发面临的挑战与高效运营机制研究

中国页岩油资源丰富,并在多个盆地取得重大勘探开发突破,已成为石油战略接替新领域,但页岩油勘探开发时间相对较短,顶层的战略规划与政策导向尚未明确,存在勘探突破难、开发成本高和组织运营不畅等问题。为此,调研剖析了中美页岩油经营理念、宏观环境、资源配置、生产运行、科技水平和信息化程度等现状,深入思考中国页岩油勘探开发的痛点、难点和阻点,认为当前中国页岩油勘探开发主要面临理念思路、技术能力、运营管理和绿色发展等4大挑战。围绕中国能源战略,提出了实现中国页岩油高效运营的对策建议:谋划稳中求进的页岩油发展战略,构建市场机制下多主体融合的战略合作共同体,打造多兵种协作的生产运行新模式,建立迭代式创新的科技发展新机制,建立数智化赋能的信息支撑新范式,开创绿色低碳化的产业发展新格局,营造高契合友好的外部运营新环境。

深、浅部煤层气地质条件差异性及其形成机制

深部煤层气资源丰富、开发前景广阔,但对其与浅部煤层气地质条件的内在联系研究尚不够深入。从煤层形成演化角度出发,以鄂尔多斯盆地上古生界煤层为例,总结了煤层深埋深藏型、深埋浅藏型及浅埋浅藏型3种埋深演化模式。系统分析了深部和浅部煤层在温压条件与含气性、地应力与渗透率特征、变质程度与含水性等方面的差异性及其形成机制。研究指出,受埋深与演化过程影响,深部和浅部煤储层温度最多相差100℃以上,储层压力最大相差40 MPa左右,导致由浅部向深部,气体赋存状态以吸附气为主转变为吸附气与游离气共存,地应力场由水平应力主导转化为垂向应力主导,煤储层孔隙率、渗透率及含水性逐渐降低。明确了深部煤层气的典型特点,即:在高温高压条件下,以吸附态和游离态共存于一定深度以下煤储层中的甲烷气体,该类煤储层在垂向应力为主导的作用下,孔裂隙空间极度压缩,含水极少且矿化度极高,内生微裂隙为主要渗流通道。基于含气性临界深度和地应力场转换深度的不一致性,指出浅部向深部煤层演化过程中存在过渡区,该区内呈现出非典型深部煤层气的特点,或深部煤层气和浅部煤层气地质条件共存的情况,在勘探开发过程中,应具体分析,制定针对性开发方案,以实现浅部与深部煤层气的高效协同开发。