Posture Detection of Heart Disease Using Multi-Head Attention Vision Hybrid(MHAVH)Model

Cardiovascular disease is the leading cause of death globally.This disease causes loss of heart muscles and is also responsible for the death of heart cells,sometimes damaging their functionality.A person’s life may depend on receiving timely assistance as soon as possible.Thus,minimizing the death ratio can be achieved by early detection of heart attack(HA)symptoms.In the United States alone,an estimated 610,000 people die fromheart attacks each year,accounting for one in every four fatalities.However,by identifying and reporting heart attack symptoms early on,it is possible to reduce damage and save many lives significantly.Our objective is to devise an algorithm aimed at helping individuals,particularly elderly individuals living independently,to safeguard their lives.To address these challenges,we employ deep learning techniques.We have utilized a vision transformer(ViT)to address this problem.However,it has a significant overhead cost due to its memory consumption and computational complexity because of scaling dot-product attention.Also,since transformer performance typically relies on large-scale or adequate data,adapting ViT for smaller datasets is more challenging.In response,we propose a three-in-one steam model,theMulti-Head Attention Vision Hybrid(MHAVH).Thismodel integrates a real-time posture recognition framework to identify chest pain postures indicative of heart attacks using transfer learning techniques,such as ResNet-50 and VGG-16,renowned for their robust feature extraction capabilities.By incorporatingmultiple heads into the vision transformer to generate additional metrics and enhance heart-detection capabilities,we leverage a 2019 posture-based dataset comprising RGB images,a novel creation by the author that marks the first dataset tailored for posture-based heart attack detection.Given the limited online data availability,we segmented this dataset into gender categories(male and female)and conducted testing on both segmented and original datasets.The training accuracy of our model reached an impressive 99.77%.Upon testing,the accuracy for male and female datasets was recorded at 92.87%and 75.47%,respectively.The combined dataset accuracy is 93.96%,showcasing a commendable performance overall.Our proposed approach demonstrates versatility in accommodating small and large datasets,offering promising prospects for real-world applications.

An Intelligent Framework for Resilience Recovery of FANETs with Spatio-Temporal Aggregation and Multi-Head Attention Mechanism

Due to the time-varying topology and possible disturbances in a conflict environment,it is still challenging to maintain the mission performance of flying Ad hoc networks(FANET),which limits the application of Unmanned Aerial Vehicle(UAV)swarms in harsh environments.This paper proposes an intelligent framework to quickly recover the cooperative coveragemission by aggregating the historical spatio-temporal network with the attention mechanism.The mission resilience metric is introduced in conjunction with connectivity and coverage status information to simplify the optimization model.A spatio-temporal node pooling method is proposed to ensure all node location features can be updated after destruction by capturing the temporal network structure.Combined with the corresponding Laplacian matrix as the hyperparameter,a recovery algorithm based on the multi-head attention graph network is designed to achieve rapid recovery.Simulation results showed that the proposed framework can facilitate rapid recovery of the connectivity and coverage more effectively compared to the existing studies.The results demonstrate that the average connectivity and coverage results is improved by 17.92%and 16.96%,respectively compared with the state-of-the-art model.Furthermore,by the ablation study,the contributions of each different improvement are compared.The proposed model can be used to support resilient network design for real-time mission execution.

Structured Multi-Head Attention Stock Index Prediction Method Based Adaptive Public Opinion Sentiment Vector

The present study examines the impact of short-term public opinion sentiment on the secondary market,with a focus on the potential for such sentiment to cause dramatic stock price fluctuations and increase investment risk.The quantification of investment sentiment indicators and the persistent analysis of their impact has been a complex and significant area of research.In this paper,a structured multi-head attention stock index prediction method based adaptive public opinion sentiment vector is proposed.The proposedmethod utilizes an innovative approach to transform numerous investor comments on social platforms over time into public opinion sentiment vectors expressing complex sentiments.It then analyzes the continuous impact of these vectors on the market through the use of aggregating techniques and public opinion data via a structured multi-head attention mechanism.The experimental results demonstrate that the public opinion sentiment vector can provide more comprehensive feedback on market sentiment than traditional sentiment polarity analysis.Furthermore,the multi-head attention mechanism is shown to improve prediction accuracy through attention convergence on each type of input information separately.Themean absolute percentage error(MAPE)of the proposedmethod is 0.463%,a reduction of 0.294% compared to the benchmark attention algorithm.Additionally,the market backtesting results indicate that the return was 24.560%,an improvement of 8.202% compared to the benchmark algorithm.These results suggest that themarket trading strategy based on thismethod has the potential to improve trading profits.

An attention-based teacher-student model for multivariate short-term landslide displacement prediction incorporating weather forecast data

Predicting the displacement of landslide is of utmost practical importance as the landslide can pose serious threats to both human life and property.However,traditional methods have the limitation of random selection in sliding window selection and seldom incorporate weather forecast data for displacement prediction,while a single structural model cannot handle input sequences of different lengths at the same time.In order to solve these limitations,in this study,a new approach is proposed that utilizes weather forecast data and incorporates the maximum information coefficient(MIC),long short-term memory network(LSTM),and attention mechanism to establish a teacher-student coupling model with parallel structure for short-term landslide displacement prediction.Through MIC,a suitable input sequence length is selected for the LSTM model.To investigate the influence of rainfall on landslides during different seasons,a parallel teacher-student coupling model is developed that is able to learn sequential information from various time series of different lengths.The teacher model learns sequence information from rainfall intensity time series while incorporating reliable short-term weather forecast data from platforms such as China Meteorological Administration(CMA)and Reliable Prognosis(https://rp5.ru)to improve the model’s expression capability,and the student model learns sequence information from other time series.An attention module is then designed to integrate different sequence information to derive a context vector,representing seasonal temporal attention mode.Finally,the predicted displacement is obtained through a linear layer.The proposed method demonstrates superior prediction accuracies,surpassing those of the support vector machine(SVM),LSTM,recurrent neural network(RNN),temporal convolutional network(TCN),and LSTM-Attention models.It achieves a mean absolute error(MAE)of 0.072 mm,root mean square error(RMSE)of 0.096 mm,and pearson correlation coefficients(PCCS)of 0.85.Additionally,it exhibits enhanced prediction stability and interpretability,rendering it an indispensable tool for landslide disaster prevention and mitigation.

HybridGAD: Identification of AI-Generated Radiology Abstracts Based on a Novel Hybrid Model with Attention Mechanism

Class Title:Radiological imaging method a comprehensive overview purpose.This GPT paper provides an overview of the different forms of radiological imaging and the potential diagnosis capabilities they offer as well as recent advances in the field.Materials and Methods:This paper provides an overview of conventional radiography digital radiography panoramic radiography computed tomography and cone-beam computed tomography.Additionally recent advances in radiological imaging are discussed such as imaging diagnosis and modern computer-aided diagnosis systems.Results:This paper details the differences between the imaging techniques the benefits of each and the current advances in the field to aid in the diagnosis of medical conditions.Conclusion:Radiological imaging is an extremely important tool in modern medicine to assist in medical diagnosis.This work provides an overview of the types of imaging techniques used the recent advances made and their potential applications.

Stock Price Prediction Based on the Bi-GRU-Attention Model

The stock market, as one of the hotspots in the financial field, forms a data system with a huge volume of data and complex relationships between various factors, making stock price prediction an area of keen interest for further in-depth mining and research. Mathematical statistics methods struggle to deal with nonlinear relationships in practical applications, making it difficult to explore deep information about stocks. Meanwhile, machine learning methods, particularly neural network models and composite models, which have achieved outstanding results in other fields, are being applied to the stock market with significant results. However, researchers have found that these methods do not grasp the essential information of the data as well as expected. In response to these issues, researchers are exploring better neural network models and combining them with other methods to analyze stock data. Thus, this paper proposes the ABiGRU composite model, which combines the attention mechanism and bidirectional gated recurrent unit (GRU) that can effectively extract data features for stock price prediction research. Models such as LSTM, GRU, and Bi-LSTM are selected for comparative experiments. To ensure the credibility and representativeness of the research data, daily stock price indices of BYD are chosen for closing price prediction studies across different models. The results show that the ABiGRU model has a lower prediction error and better fitting effect on three index-based stock prices, enhancing the learning efficiency of the neural network model and demonstrating good prediction stability. This suggests that the ABiGRU model is highly adaptable for stock price prediction.

Multi-Head Attention Spatial-Temporal Graph Neural Networks for Traffic Forecasting

Accurate traffic prediction is crucial for an intelligent traffic system (ITS). However, the excessive non-linearity and complexity of the spatial-temporal correlation in traffic flow severely limit the prediction accuracy of most existing models, which simply stack temporal and spatial modules and fail to capture spatial-temporal features effectively. To improve the prediction accuracy, a multi-head attention spatial-temporal graph neural network (MSTNet) is proposed in this paper. First, the traffic data is decomposed into unique time spans that conform to positive rules, and valuable traffic node attributes are mined through an adaptive graph structure. Second, time and spatial features are captured using a multi-head attention spatial-temporal module. Finally, a multi-step prediction module is used to achieve future traffic condition prediction. Numerical experiments were conducted on an open-source dataset, and the results demonstrate that MSTNet performs well in spatial-temporal feature extraction and achieves more positive forecasting results than the baseline methods.

基于CNN-BiGRU-Attention的短期电力负荷预测

针对目前电力负荷数据随机性强,影响因素复杂,传统单一预测模型精度低的问题,结合卷积神经网络(Convolutional neural network,CNN)、双向门控循环单元(Bi-directional gated recurrent unit,BiGRU)以及注意力机制(Attention)在短期电力负荷预测上的不同优点,提出一种基于CNN-BiGRU-Attention的混合预测模型。该方法首先通过CNN对历史负荷和气象数据进行初步特征提取,然后利用BiGRU进一步挖掘特征数据间时序关联,再引入注意力机制,对BiGRU输出状态给与不同权重,强化关键特征,最后完成负荷预测。试验结果表明,该模型的平均绝对百分比误差(Mean absolute percentage error,MAPE)、均方根误差(Root mean square error,RMSE)、判定系数(R-square,R~2)分别为0.167%、0.057%、0.993,三项指标明显优于其他模型,具有更高的预测精度和稳定性,验证了模型在短期负荷预测中的优势。

基于CNN-BiLSTM-Attention的三峡库区滑坡地表位移预测研究

地表位移预测在滑坡监测预警中具有重要意义,建立稳定可靠的滑坡位移预测模型是关键。本文基于卷积神经网络和注意力机制的滑坡位移预测方法,并以三峡库区黄泥巴蹬坎滑坡为例进行了验证。本文综合分析了该滑坡长达8年的降雨量、库水位和地表位移等监测数据,建立了结合卷积神经网络(convolutional neural network,CNN)、双向长短期记忆(bidirectional long short-term memory,BiLSTM)网络和注意力机制(attention)的CNN-BiLSTM-Attention深度学习组合预测模型,采用了适应性学习率和正则化技术进行模型训练,提高了模型的泛化能力同时避免过拟合,并与传统LSTM模型进行对比验证。结果表明:相较于传统的机器学习和神经网络方法,该模型在滑坡位移预测精度上取得了显著提升,预测模型拟合优度(R^(2))达0.989,平均绝对百分比误差(MAPE)仅为0.059。

基于Attention-LSTM时序模型的机械钻速预测方法

针对石油天然气开采成本优化的需求,聚焦钻井作业中提速的关键技术,即机械钻速的预测。传统预测方法通常只考虑瞬时工程参数对机械钻速的影响,而未充分考虑钻井作业的时序性以及机械钻速在时间序列上的相关性。提出一种结合时序特征的机械钻速预测模型,该模型基于Attention-LSTM架构。通过Attention机制,模型有效捕捉了工程参数与机械钻速之间的相关性,并利用LSTM网络提取参数的时序特征,增强了模型对时间依赖性的捕捉能力。实验结果证实,所提模型相较于传统深度神经网络在预测精度上有显著提升。添加的Attention机制进一步提升了模型的解释性、训练效率及预测准确性。采用实际油田钻井数据对提出的方法进行了验证,并与现有几种机械钻速预测模型进行了对比分析,证明了本文方法在准确性、可靠性及解释性方面的优势。

基于Attention-UNet网络的速度模型构建方法研究

随着油气资源的不断勘探开发,相对易开采的油气矿逐渐建成,地震勘探的研究重点也向地下更深、构造更复杂的区域转移。目前,传统的地震速度建模方法在稳定性、准确性和计算效率方面都面临挑战。因此,笔者利用将地震数据映射到速度模型的思路,提出了一种基于Attention-UNet网络的深度学习速度建模方法。采用的这种方法利用有限差分正演得到反射波形数据,将反射波形数据和对应的速度模型(标签)对作为Attention-UNet网络的输入,建立地震数据和速度模型之间的映射关系。网络训练后对新输入的地震数据进行速度模型的估计。数值实验结果表明,与传统的FWI相比,笔者提出的方法表现出良好的性能;基于Attention-UNet网络模型训练完成后,不需要经过大量的计算,就可以快速执行与训练集中速度结构相似的地下结构的速度建模,这比传统方法计算效率更高。该方法在建立大量速度模型时具有很好的推广价值。

基于CNN-LSTM-Attention模型的沁河流域径流模拟及未来多情景预测

为提升深度学习模型对变化环境下流域的径流模拟精度,以沁河流域为例,构建了基于卷积神经网络(CNN)、长短期记忆网络(LSTM)和注意力机制(Attention)的CNN-LSTM-Attention耦合模型,加入多种优化算法,结合第六次国际耦合模式比较计划CMIP6中的BCC-CSM2-MR气候模式并考虑多种情景,应用于流域的径流模拟和预测,同时比较了多种深度学习模型的模拟精度。结果表明:CNN-LSTM-Attention模型在沁河流域表现出了较好的径流模拟效果,模拟精度均优于其他深度学习模型,纳什效率系数(NSE)为0.883,均方根误差(RMSE)为2.317,平均绝对误差(MAE)为1.098;不同气候变化情景下,沁河流域在2025—2050年的年径流量均呈现缓慢衰减趋势且波动程度较大,尤其在SSP1-2.6情景下,径流量衰减和波动程度突出。研究可为深度学习模型在人水关系智能化计算模拟领域的应用提供新思路,并为流域后续的水资源开发利用和管理提供科学参考价值。

引入卷积块注意力模块的Attention U-Net木材表面裂纹检测方法

木材缺陷会影响木材的使用价值和使用期限,其中木材表面裂纹是严重影响木材外观质量和机械强度的一种木材缺陷。对木材表面裂纹的检测可以尽快发现此类缺陷木材,或为后续处理提供依据。针对现有的人工检测和自动化检测木材表面裂纹效率低、成本高、漏检率高等问题,采用引入卷积块注意力模块(convolutional block attention module,CBAM)的Attention U-Net深度学习模型对木材表面裂纹图像进行语义分割,从而达到木材表面裂纹检测的目的。引入的CBAM模块包含通道注意力机制和空间注意力机制,分别用于捕捉通道间的依赖关系和像素级的空间关系,该模块被添加到Attention U-Net网络的编码阶段,以增加感兴趣区域的权重并抑制冗余信息。最后,通过消融试验验证了Attention U-Net中加入CBAM对分割性能的提升。采用像素准确率(PA)、类别像素准确率(CPA)、召回率(Recall)、Dice系数、交并比(IoU)和平均交并比(MIoU)等语义分割评价指标评价各模型的优劣,并确定最佳模型及其参数。在自制木材表面数据集的裂纹分割中,使用AdamW优化器引入CBAM的Attention U-Net的PA、木材裂纹Recall、木材裂纹Dice系数、木材裂纹IoU、MIoU分别比使用SGD优化器的Attention U-Net原始模型提高了0.11%,4.14%,2.96%,3.58%和1.84%。结果表明,使用AdamW优化器引入CBAM的Attention U-Net能够较好地分割背景和木材表面裂纹,区分节点、表面纹理和木材裂纹,并将节点和表面纹理分割为背景。

基于多传感器信息融合和CNN-BIGRU-Attention模型的液压防水阀故障诊断方法

在建筑工程领域,尤其是在工程搅拌设备中,涉及到复杂液压工作介质,液压防水阀往往会出现不同程度的故障。此外,恶劣的工作环境和复杂的噪声背景使得对液压防水阀的故障进行诊断变得困难。为了解决这一难题,提出了一种基于多传感器信息融合和卷积神经网络-双向门控循环单元-自注意力机制(CNN-BIGRU-Attention)模型的防水阀故障诊断方法。首先,考虑到单一传感器振动信号难以充分表达故障特征,该方法使用了3个传感器采集含噪声的振动信号,并进行了必要的预处理;其次,提取了信号的16个时域特征、5个频域特征以及3个时频域特征,并利用熵权法进行了特征融合,达到了增强特征的目的;然后,将融合的多维特征集输入到CNN-BIGRU-Attention模型中进行了特征识别;最后,利用实际的液压防水阀故障诊断实验,验证了该方法的有效性。研究结果表明:采用多传感器提取的特征更为全面,信息融合有助于捕捉更完整的隐藏数据特征,从而显著提高诊断的准确率;相较于其他特征识别方法,采用CNN-BIGRU-Attention模型的液压防水阀故障诊断准确率可分别提高6.7%、4.6%和14.2%,达到了96.86%,证明了该方法的有效性。该方法将先进的机器学习技术与实际工程应用相结合,为建筑工程问题提供了一种新颖、有效的解决方案。

基于CB-Attention的JavaScript恶意混淆代码检测方法

当今JavaScript代码混淆方法日益多样,现有检测方法在对混淆代检测时会出现漏报和误报的情况,为解决该问题,提出一种基于CB-Attention的JavaScript恶意代码检测方法。由SDPCNN模型和BiLSTM+Attention模型构成,SDPCNN对短距离间的语义特征信息进行提取,BiLSTM+Attention获取JavaScript代码中长距离间的语义信息特征。为验证所提方法的有效性,将该方法与其它方法进行对比,对比结果表明,该方法具有较好的检测效果,F1-Score可达98.78%。

基于Conv-Attention-MLP的新能源汽车电池异常检测方法

在新能源汽车电池异常信号的检测方面,针对传统异常检测方法对多个维度电池数据检测精度低、泛化性差等问题,提出一种融合了卷积(Conv)、自注意力机制(Attention)和多层感知机(Multilayer Perceptron,MLP)的Conv-Attention-MLP深度神经网络方法。该方法首先对多个维度的时序数据使用特征卷积进行降维,再利用多头自注意力机制(MultiHead-Self-Attention,MSA)对数据的时间不同步以及数据逻辑之间的重建模,最后经过多层感知机(MLP)的连续线性映射与非线性运算,产生复杂的分段映射,以达到更好的拟合效果。实现了对多个维度电池数据的有效检测,提高了模型对于多样化数据的准确性和稳健性。实验结果表明:相较于传统异常检测方法,在vloongs数据集上Conv-Attention-MLP模型表现效果更加优越,展现出更高精度和更强的鲁棒性。

基于双向LSTM-Attention模型的火电厂负荷预测研究

准确预测电厂负荷可指导火电厂制定发电计划和调度安排,有利于降低能源成本和污染物排放,对电厂的经济性和环保性有重要意义。本文提出一种基于双向LSTM-Attention的火电厂负荷预测方法。首先,通过皮尔逊系数筛选出关键特征变量;其次利用双向长短期记忆网络提取关键变量之间的长期依赖关系与短期变化特征,最后融合注意力权重机制以进一步突出关键时序信息,进而实现负荷的准确预测。以某在役600 MW超临界机组为对象进行验证。结果表明:相较于单向LSTM、双向LSTM、单向LSTM-Attention,本文所提方法的决定系数R^(2)、均方根误差S_(RMSE)和平均绝对误差S_(MAE)均为最优,分别为0.9566、16.3159、13.5043,能更准确地捕捉到负荷快速波动的趋势,为电厂的负荷预测和能源管理提供可行的方法。

高速公路施工控制区动态交通流预测的LSTM-BiGRU-Attention模型

为提前准确预知高速公路施工控制区交通流变化趋势,解决交通流时间序列中的长期依赖问题,文中建立了高速公路施工控制区动态交通流预测的LSTM-BiGRU-Attention模型。首先,将预处理后的动态交通流数据集按时间步长顺序输入到LSTM网络,对交通流信息建模和学习。然后,引入BiGRU和Attention机制以更好地捕捉上下文信息和提供更具针对性的权重分配。最后,将构建的LSTM-BiGRU-Attention模型与其他模型进行交通流预测对比,评估模型性能。实验以G35济广高速公路某施工控制区交通运行情况为案例进行研究,结果显示该模型的平均绝对误差MAE为1.91,均方根误差RMSE为2.83,决定系数R^(2)为0.79,平均绝对百分数误差MAPE为3.23。对比其他模型,LSTM-BiGRU-Attention模型的4个评估指标均有所下降,说明该模型可为高速公路施工控制区提供更加精准的预测。

基于Attention注意力机制下的鱼群韧性研究

生物集群运动可以自组织实现群体涌现行为,但是在外界因素的影响之下,生物集群是否能够保持韧性,单体是否能够根据当前信息重新交互仍然面临巨大的挑战。在攻击体的干扰之下,本文根据红鼻剪刀鱼的运动数据,设计Attention注意力模型。模型考虑到单体鱼与攻击体和周围邻居鱼之间的信息交互,预测下一时刻单体出现的具体位置,说明Attention注意力模型能够使生物的集群韧性与自组织运动保持一致。实验结果表明,所提的Attention注意力模型能够较好的解释生物集群韧性,增强生物集群韧性的鲁棒性和灵活性,为复杂系统解释内外部之间的联系提供了有力的支撑,该方法对生物控制领域的分布式管理也有很好的借鉴作用。

Using Recurrent Neural Network Structure and Multi-Head Attention with Convolution for Fraudulent Phone Text Recognition

Fraud cases have been a risk in society and people’s property security has been greatly threatened.In recent studies,many promising algorithms have been developed for social media offensive text recognition as well as sentiment analysis.These algorithms are also suitable for fraudulent phone text recognition.Compared to these tasks,the semantics of fraudulent words are more complex and more difficult to distinguish.Recurrent Neural Networks(RNN),the variants ofRNN,ConvolutionalNeuralNetworks(CNN),and hybrid neural networks to extract text features are used by most text classification research.However,a single network or a simple network combination cannot obtain rich characteristic knowledge of fraudulent phone texts relatively.Therefore,a new model is proposed in this paper.In the fraudulent phone text,the knowledge that can be learned by the model includes the sequence structure of sentences,the correlation between words,the correlation of contextual semantics,the feature of keywords in sentences,etc.The new model combines a bidirectional Long-Short Term Memory Neural Network(BiLSTM)or a bidirectional Gate Recurrent United(BiGRU)and a Multi-Head attention mechanism module with convolution.A normalization layer is added after the output of the final hidden layer.BiLSTM or BiGRU is used to build the encoding and decoding layer.Multi-head attention mechanism module with convolution(MHAC)enhances the ability of the model to learn global interaction information and multi-granularity local interaction information in fraudulent sentences.A fraudulent phone text dataset is produced by us in this paper.The THUCNews data sets and fraudulent phone text data sets are used in experiments.Experiment results show that compared with the baseline model,the proposed model(LMHACL)has the best experiment results in terms of Accuracy,Precision,Recall,and F1 score on the two data sets.And the performance indexes on fraudulent phone text data sets are all above 0.94.