结合LSTM和Self‑Attention的滚动轴承剩余使用寿命预测方法

为了构建准确表征滚动轴承退化过程的趋势性健康度指标,提高滚动轴承剩余使用寿命(Remaining Useful Life,RUL)的预测精度,提出了一种结合长短期记忆(Long‑Short Term Memory,LSTM)和自注意力(Self‑Attention)机制的神经网络模型(LSTM‑SA)用于滚动轴承RUL预测。利用包络解调获得原始信号的包络谱,再将包络谱分段并计算对应频段的皮尔逊相关系数,得到具有单调性和趋势性的退化特征;将退化特征归一化处理后作为LSTM‑SA模型的输入,并利用LSTM自适应提取退化特征时间上的内部相关性以及Self‑Attention对关键信息的筛选,消除无用信息的干扰,挖掘深层次特征,构建健康度指标并得到退化曲线;确定失效阈值,利用最小二乘法拟合退化曲线,预测寿命失效点,实现滚动轴承的RUL预测。在PHM2012数据集上的实验结果表明,所提出的方法相比于其他文献,平均绝对误差分别降低了43.18%,62.57%和59.44%,平均得分分别提高了10.87%,45.71%和34.21%;在工程实际数据中的实验结果表明,所提出方法的平均预测误差分别比Standard‑RNN和CNN方法降低了39.58%和74.86%。

基于Self-Attention-BiLSTM网络的西瓜种苗叶片氮磷钾含量高光谱检测方法

元素含量无损检测技术可以为植物生长发育的环境精准调控提供关键实时数据。以西瓜苗为例,提出了一种基于图谱特征融合的氮磷钾含量深度学习检测方法。首先,使用高光谱仪拍摄西瓜苗叶片的高光谱图像,使用连续流动化学分析仪测定叶片的3种元素含量。然后,采用基线偏移校正(BOC)叠加高斯平滑滤波(GF)的光谱预处理方法和随机森林算法(RF)建立预测模型,基于竞争性自适应重加权采样(CARS)和连续投影算法(SPA)2种算法初步筛选出特征波长,再综合考虑波长数和建模精度设计了一种最优波长评价方法,将波长数进一步减少到3~4个。最后,提取使用U-Net网络分割的彩色图像颜色和纹理特征,和光谱反射率特征一起作为输入,基于自注意力机制-双向长短时记忆(Self-Attention-BiLSTM)网络构建了3种元素含量的预测模型。实验结果表明,氮磷钾含量预测的R2分别为0.961、0.954、0.958,RMSE分别为0.294%、0.262%、0.196%,实现了很好的建模效果。使用该模型对另2个品种西瓜进行测试,R2超过0.899、RMSE小于0.498%,表明该模型具有很好的泛化性。该高光谱建模方法使用少量波长光谱即实现了高精度检测,在精度和效率上达成了很好的平衡,为后续便携式高光谱检测装备开发奠定了理论基础。

Application of the improved dung beetle optimizer,muti-head attention and hybrid deep learning algorithms to groundwater depth prediction in the Ningxia area,China

Due to the lack of accurate data and complex parameterization,the prediction of groundwater depth is a chal-lenge for numerical models.Machine learning can effectively solve this issue and has been proven useful in the prediction of groundwater depth in many areas.In this study,two new models are applied to the prediction of groundwater depth in the Ningxia area,China.The two models combine the improved dung beetle optimizer(DBO)algorithm with two deep learning models:The Multi-head Attention-Convolution Neural Network-Long Short Term Memory networks(MH-CNN-LSTM)and the Multi-head Attention-Convolution Neural Network-Gated Recurrent Unit(MH-CNN-GRU).The models with DBO show better prediction performance,with larger R(correlation coefficient),RPD(residual prediction deviation),and lower RMSE(root-mean-square error).Com-pared with the models with the original DBO,the R and RPD of models with the improved DBO increase by over 1.5%,and the RMSE decreases by over 1.8%,indicating better prediction results.In addition,compared with the multiple linear regression model,a traditional statistical model,deep learning models have better prediction performance.

基于模糊逻辑的FBiLSTM-Attention短期负荷预测

针对电力负荷数据由于受多种因素的影响具有高度不确定性的问题,将负荷数据的不确定性与深度学习算法相结合,提出了一种基于模糊逻辑的FBiLSTM-Attention短期负荷预测模型,以提高负荷预测的精度。首先,对原始数据进行数据预处理,包括缺失值填充、相关性分析及数据归一化;其次,通过K-Means聚类将每个特征的数据转换成模糊规则引入模糊逻辑的处理,同时,模型结构方面采用双向长短期记忆网络(BiLSTM)和注意力机制(Attention);最后,对所提方法和传统的LSTM与BiLSTM-Attention模型的预测结果进行对比。结果表明,结合了模糊逻辑的模型精确度和鲁棒性都有了明显的提升,具有更好的预测性能。所提模型可以有效提高处理不确定性数据的能力,为负荷预测研究提供了参考。

基于DCNN网络及Self-Attention-BiGRU机制的轴承剩余寿命预测

深度神经网络在剩余寿命预测(RUL)领域得到了广泛的应用。传统的滚动轴承寿命预测模型存在预测精确度较低、鲁棒性较弱的问题。为了进一步提升预测模型的精确度以及鲁棒性,提出了一种融合深度卷积神经网络(DCNN)、双向门控循环单元(BiGRU)以及自注意力机制(Self-Attention)三种模块的滚动轴承剩余使用寿命预测模型。首先,利用DCNN网络对原始振动信号的时域特征、频域特征进行了提取;然后,使用不确定量化的方法对提取到的特征进行了评价和筛选,利用筛选过后的特征构建了新的替代特征集;最后,利用Self-Attention-BiGRU网络对轴承的剩余使用寿命进行了预测,并在IEEE PHM2012数据集上进行了验证。实验结果表明:相较于BiGRU、GRU和BiLSTM三种模型的预测结果,基于DCNN及Self-Attention-BiGRU方法的预测结果最优,两项误差值:平均绝对误差(MAE)、均方根误差(RMSE)最低,其中工况一的一号轴承RUL预测的MAE值相较于BiGRU、GRU以及BiLSTM网络分别下降了7.0%、7.4%和6.5%,RMSE值相较于其他三种模型分别下降了7.6%、8.4%和6.9%,预测的Score值最高,分值为0.985。通过不同数据集的划分,证明了该方法在轴承RUL预测时的强鲁棒性。实验结果验证了基于DCNN网络及Self-Attention-BiGRU模型在轴承剩余使用寿命预测中的有效性。

基于Attention-LSTM的短期电力负荷预测

电力负荷预测的准确性受到多种因素的干扰,如气候变化、经济发展以及区域差异等,这些因素使得电力负荷呈现出显著的不稳定性和复杂的非线性特征,从而增加了提高预测精度的难度。为了应对这一挑战,本文创新性地引入了一种结合自注意力机制与长短期记忆网络(LSTM)的预测方法。通过在美国某一地区的实际用电负荷数据验证模型,实验结果表明,该方法的决定系数(R2)为0.96,平均绝对误差(MAE)为0.023,均方根误差(RMSE)为0.029,提升了预测的准确性。这不仅证明了所提模型在提高电力负荷预测精度方面的有效性,也为其在船舶电力负荷预测的应用奠定了一定的基础。

ConvNeXt网络及Stacked BiLSTM-Self-Attention在轴承剩余寿命预测中的应用

在滚动轴承剩余使用寿命预测方面,采用传统方法时存在鲁棒性差、精度低等各种问题。近些年来深度学习的发展为解决这些问题提供了新的思路。为了进一步提高对轴承寿命的预测精度,提出了一种基于ConvNeXt网络、堆叠双向长短时记忆网络(SBiLSTM)和自注意力机制(Self-Attention)的滚动轴承寿命预测方法。首先,采用连续小波变换(CWT)构造了振动信号的时频图,以更好地捕捉信号的时域和频域特征;然后,将得到的时频图输入到构建的ConvNeXt网络中,通过卷积、池化和层归一化等操作,对时频图的关键特征进行了提取;最后,将提取后的特征输入到SBiLSTM-Self-Attention模块中,进一步提取了时序信息和特征权重分配数据,利用PHM2012挑战数据集进行了验证,通过实验分析了该方法的均方根误差(RMSE)和平均绝对误差(MAE)。研究结果表明:相较于现有技术方法,该方法的平均RMSE为0.031;与其他三种方法,即卷积神经网络(CNN)、深度残差双向门控循环单元(DRN-BiGRU)和深度卷积自注意力双向门控循环单元(DCNN-Self-Attention-BiGRU)相比,其平均RMSE值分别下降了79%、74%和55%,MAE值分别下降了78%、73%和53%,说明该方法在滚动轴承剩余寿命预测中有较好的性能。

RepBoTNet-CESA:An Alzheimer’s Disease Computer Aided Diagnosis Method Using Structural Reparameterization BoTNet and Cubic Embedding Self Attention

Various deep learning models have been proposed for the accurate assisted diagnosis of early-stage Alzheimer’s disease(AD).Most studies predominantly employ Convolutional Neural Networks(CNNs),which focus solely on local features,thus encountering difficulties in handling global features.In contrast to natural images,Structural Magnetic Resonance Imaging(sMRI)images exhibit a higher number of channel dimensions.However,during the Position Embedding stage ofMulti Head Self Attention(MHSA),the coded information related to the channel dimension is disregarded.To tackle these issues,we propose theRepBoTNet-CESA network,an advanced AD-aided diagnostic model that is capable of learning local and global features simultaneously.It combines the advantages of CNN networks in capturing local information and Transformer networks in integrating global information,reducing computational costs while achieving excellent classification performance.Moreover,it uses the Cubic Embedding Self Attention(CESA)proposed in this paper to incorporate the channel code information,enhancing the classification performance within the Transformer structure.Finally,the RepBoTNet-CESA performs well in various AD-aided diagnosis tasks,with an accuracy of 96.58%,precision of 97.26%,and recall of 96.23%in the AD/NC task;an accuracy of 92.75%,precision of 92.84%,and recall of 93.18%in the EMCI/NC task;and an accuracy of 80.97%,precision of 83.86%,and recall of 80.91%in the AD/EMCI/LMCI/NC task.This demonstrates that RepBoTNet-CESA delivers outstanding outcomes in various AD-aided diagnostic tasks.Furthermore,our study has shown that MHSA exhibits superior performance compared to conventional attention mechanisms in enhancing ResNet performance.Besides,the Deeper RepBoTNet-CESA network fails to make further progress in AD-aided diagnostic tasks.

TianXing:A Linear Complexity Transformer Model with Explicit Attention Decay for Global Weather Forecasting

In this paper,we introduce TianXing,a transformer-based data-driven model designed with physical augmentation for skillful and efficient global weather forecasting.Previous data-driven transformer models such as Pangu-Weather,FengWu,and FuXi have emerged as promising alternatives for numerical weather prediction in weather forecasting.However,these models have been characterized by their substantial computational resource consumption during training and limited incorporation of explicit physical guidance in their modeling frameworks.In contrast,TianXing applies a linear complexity mechanism that ensures proportional scalability with input data size while significantly diminishing GPU resource demands,with only a marginal compromise in accuracy.Furthermore,TianXing proposes an explicit attention decay mechanism in the linear attention derived from physical insights to enhance its forecasting skill.The mechanism can reweight attention based on Earth's spherical distances and learned sparse multivariate coupling relationships,promptingTianXing to prioritize dynamically relevant neighboring features.Finally,to enhance its performance in mediumrange forecasting,TianXing employs a stacked autoregressive forecast algorithm.Validation of the model's architecture is conducted using ERA5 reanalysis data at a 5.625°latitude-longitude resolution,while a high-resolution dataset at 0.25°is utilized for training the actual forecasting model.Notably,the TianXing exhibits excellent performance,particularly in the Z500(geopotential height)and T850(temperature)fields,surpassing previous data-driven models and operational fullresolution models such as NCEP GFS and ECMWF IFS,as evidenced by latitude-weighted RMSE and ACC metrics.Moreover,the TianXing has demonstrated remarkable capabilities in predicting extreme weather events,such as typhoons.

A Generative Adversarial Network with an Attention Spatiotemporal Mechanism for Tropical Cyclone Forecasts

Tropical cyclones(TCs)are complex and powerful weather systems,and accurately forecasting their path,structure,and intensity remains a critical focus and challenge in meteorological research.In this paper,we propose an Attention Spatio-Temporal predictive Generative Adversarial Network(AST-GAN)model for predicting the temporal and spatial distribution of TCs.The model forecasts the spatial distribution of TC wind speeds for the next 15 hours at 3-hour intervals,emphasizing the cyclone's center,high wind-speed areas,and its asymmetric structure.To effectively capture spatiotemporal feature transfer at different time steps,we employ a channel attention mechanism for feature selection,enhancing model performance and reducing parameter redundancy.We utilized High-Resolution Weather Research and Forecasting(HWRF)data to train our model,allowing it to assimilate a wide range of TC motion patterns.The model is versatile and can be applied to various complex scenarios,such as multiple TCs moving simultaneously or TCs approaching landfall.Our proposed model demonstrates superior forecasting performance,achieving a root-mean-square error(RMSE)of 0.71 m s^(-1)for overall wind speed and 2.74 m s^(-1)for maximum wind speed when benchmarked against ground truth data from HWRF.Furthermore,the model underwent optimization and independent testing using ERA5reanalysis data,showcasing its stability and scalability.After fine-tuning on the ERA5 dataset,the model achieved an RMSE of 1.33 m s^(-1)for wind speed and 1.75 m s^(-1)for maximum wind speed.The AST-GAN model outperforms other state-of-the-art models in RMSE on both the HWRF and ERA5 datasets,maintaining its superior performance and demonstrating its effectiveness for spatiotemporal prediction of TCs.

Self-potential inversion based on Attention U-Net deep learning network

Landfill leaks pose a serious threat to environmental health,risking the contamination of both groundwater and soil resources.Accurate investigation of these sites is essential for implementing effective prevention and control measures.The self-potential(SP)stands out for its sensitivity to contamination plumes,offering a solution for monitoring and detecting the movement and seepage of subsurface pollutants.However,traditional SP inversion techniques heavily rely on precise subsurface resistivity information.In this study,we propose the Attention U-Net deep learning network for rapid SP inversion.By incorporating an attention mechanism,this algorithm effectively learns the relationship between array-style SP data and the location and extent of subsurface contaminated sources.We designed a synthetic landfill model with a heterogeneous resistivity structure to assess the performance of Attention U-Net deep learning network.Additionally,we conducted further validation using a laboratory model to assess its practical applicability.The results demonstrate that the algorithm is not solely dependent on resistivity information,enabling effective locating of the source distribution,even in models with intricate subsurface structures.Our work provides a promising tool for SP data processing,enhancing the applicability of this method in the field of near-subsurface environmental monitoring.

Mobile Crowdsourcing Task Allocation Based on Dynamic Self-Attention GANs

Crowdsourcing technology is widely recognized for its effectiveness in task scheduling and resource allocation.While traditional methods for task allocation can help reduce costs and improve efficiency,they may encounter challenges when dealing with abnormal data flow nodes,leading to decreased allocation accuracy and efficiency.To address these issues,this study proposes a novel two-part invalid detection task allocation framework.In the first step,an anomaly detection model is developed using a dynamic self-attentive GAN to identify anomalous data.Compared to the baseline method,the model achieves an approximately 4%increase in the F1 value on the public dataset.In the second step of the framework,task allocation modeling is performed using a twopart graph matching method.This phase introduces a P-queue KM algorithm that implements a more efficient optimization strategy.The allocation efficiency is improved by approximately 23.83%compared to the baseline method.Empirical results confirm the effectiveness of the proposed framework in detecting abnormal data nodes,enhancing allocation precision,and achieving efficient allocation.

Fusion of Convolutional Self-Attention and Cross-Dimensional Feature Transformationfor Human Posture Estimation

Human posture estimation is a prominent research topic in the fields of human-com-puter interaction,motion recognition,and other intelligent applications.However,achieving highaccuracy in key point localization,which is crucial for intelligent applications,contradicts the lowdetection accuracy of human posture detection models in practical scenarios.To address this issue,a human pose estimation network called AT-HRNet has been proposed,which combines convolu-tional self-attention and cross-dimensional feature transformation.AT-HRNet captures significantfeature information from various regions in an adaptive manner,aggregating them through convolu-tional operations within the local receptive domain.The residual structures TripNeck and Trip-Block of the high-resolution network are designed to further refine the key point locations,wherethe attention weight is adjusted by a cross-dimensional interaction to obtain more features.To vali-date the effectiveness of this network,AT-HRNet was evaluated using the COCO2017 dataset.Theresults show that AT-HRNet outperforms HRNet by improving 3.2%in mAP,4.0%in AP75,and3.9%in AP^(M).This suggests that AT-HRNet can offer more beneficial solutions for human posture estimation.

基于Self-Attention与Bi-LSTM的大学生情感倾向研究

针对基于词向量的网络模型性能过分依赖分词准确性的问题,提出基于FastText字向量表示方法结合SelfAttention与BiLSTM的大学生情感分析方法(character-SATT-BiLSTM)。使用fasttext模型生成文本字向量,通过双向长短时记忆模型提取上下文语义特征,利用自注意力机制强化关键信息,最后使用Softmax分类器判断情感类别。实验结果显示,字向量文本表示方法比词向量更适合论坛文本情感分类,同时character-SATT-BiLSTM相比characterLSTM、character-BiLSTM等模型的效果更优,分类性能分别提高了6%和3%。

引入上下文信息和Attention Gate的GUS-YOLO遥感目标检测算法

目前基于通用YOLO系列的遥感目标检测算法存在并未充分利用图像的全局上下文信息,在特征融合金字塔部分并未充分考虑缩小融合特征之间的语义鸿沟、抑制冗余信息干扰的缺点。在结合YOLO算法优点的基础上提出GUS-YOLO算法,其拥有一个能够充分利用全局上下文信息的骨干网络Global Backbone。除此之外,该算法在融合特征金字塔自顶向下的结构中引入Attention Gate模块,可以突出必要的特征信息,抑制冗余信息。另外,为Attention Gate模块设计了最佳的网络结构,提出了网络的特征融合结构U-Net。最后,为克服ReLU函数可能导致模型梯度不再更新的问题,该算法将Attention Gate模块的激活函数升级为可学习的SMU激活函数,提高模型鲁棒性。在NWPU VHR-10遥感数据集上,该算法相较于YOLOV7算法取得宽松指标mAP^(0.50)1.64个百分点和严格指标mAP^(0.75)9.39个百分点的性能提升。相较于目前主流的七种检测算法,该算法取得较好的检测性能。

基于CNN-LSTM-Attention的气井井筒积液诊断

为了解决传统积液诊断模型存在的诸多问题,如选择缺乏定性标准、计算结果差异大以及无法满足实际工程需求等,提出一种基于神经网络的气井井筒积液诊断方法,该模型将卷积神经网络(CNN)和长短期记忆网络(LSTM)的结合,使其能够有效捕捉气井在不同工况下的动态特征,增强模型对复杂数据的处理能力,并在此基础上引入注意力机制自动聚焦于输入数据中最相关的信息,从而提升特征的权重。在实验中,使用真实气井生产相关数据集,对比分析多个模型与所提出的CNN-LSTM-Attention模型的相关性能指标。实验结果显示,所提模型的准确率高达97.6%,多次试验结果方差值明显优于其他深度学习模型和传统方法。这一显著的性能提升,验证了模型的有效性,并对气田生产具有一定的指导作用。

基于IGWO-Attention-GRU的短期电力负荷预测模型

为了提高短期电力负荷的预测精度,针对电力负荷序列波动性强、复杂性高的特点,综合考虑气象因素及日期类型的影响,文中提出一种基于改进灰狼优化算法(IGWO)优化Attention-GRU网络的短期电力负荷预测模型。首先,构建Attention-GRU网络;其次,对灰狼优化算法(GWO)进行改进,并利用IGWO寻找Attention-GRU网络的超参数;最后,使用IGWO-Attention-GRU模型在电力负荷数据集上进行实验,并与多种预测模型进行比较。实验结果表明,IGWO-Attention-GRU模型的MAPE、RMSE和MAE值均为各种预测模型中最低,验证了IGWO-Attention-GRU模型的优越性。

基于SABO-GRU-Attention的锂电池SOC估计

提出一种基于SABO-GRU-Attention(subtraction average based optimizer-gate recurrent unitattention)的锂电池SOC(state of charge)估计方法。采用基于平均减法优化算法自适应更新GRU神经网络的超参数,融合SE(squeeze and excitation)注意力机制自适应分配各通道权重,提高学习效率。对马里兰大学电池数据集进行预处理,输入电压、电流参数,进行锂电池充放电仿真实验,并搭建锂电池荷电状态实验平台进行储能锂电池充放电实验。结果表明,提出的SOC神经网络估计模型明显优于LSTM、GRU以及PSO-GRU等模型,具有较高的估计精度与应用价值。

融合self-attention机制的卷积神经网络文本分类模型

传统的文本分类算法采用词向量表示文本,忽视了上下文语境中词义的变化.本文通过引入self-attention机制处理词向量,提出一种卷积神经网络模型与关键词提取技术相结合的文本分类模型.该模型对文档进行self-attention操作,以抽取关键信息,构建文档特征图,根据卷积神经网络模型和关键词提取技术实现特征向量的分类.在真实数据集上进行性能分析,并与循环神经网络模型、长短时记忆网络模型进行比较,结果表明该分类模型有效地提高了分类的准确性.

基于XGBoost-WOA-BiLSTM-Attention的公共建筑暖通空调能耗预测研究

为在双碳目标下实现节能减排,降低能源成本,提出一种基于BiLSTM的公共建筑暖通空调能耗预测模型。在BiLSTM模型基础上,使用XGBoost算法对输入特征进行选择,剔除冗余特征,得到最佳模型输入特征;然后利用WOA优化算法对添加了Attention机制的BiLSTM模型中的6个超参数进行优化,将得到的最优参数代入BiLSTM-Attention神经网络中进行预测,并与BiLSTM模型、BiLSTM-Attention模型和WOA-BiLSTM-Attention模型进行对比。结果表明,所提出的XGBoost-WOA-BiLSTM-Attention模型的RMSE、MAE、R2分别为0.0106、0.006、0.9991,优于其他模型,且相对于持续模型在均方根误差RMSE上提升了98%,为降低公共建筑暖通空调能耗研究提供了参考。