Recommendation Algorithm Integrating CNN and Attention System in Data Extraction

With the rapid development of the Internet globally since the 21st century,the amount of data information has increased exponentially.Data helps improve people’s livelihood and working conditions,as well as learning efficiency.Therefore,data extraction,analysis,and processing have become a hot issue for people from all walks of life.Traditional recommendation algorithm still has some problems,such as inaccuracy,less diversity,and low performance.To solve these problems and improve the accuracy and variety of the recommendation algorithms,the research combines the convolutional neural networks(CNN)and the attention model to design a recommendation algorithm based on the neural network framework.Through the text convolutional network,the input layer in CNN has transformed into two channels:static ones and non-static ones.Meanwhile,the self-attention system focuses on the system so that data can be better processed and the accuracy of feature extraction becomes higher.The recommendation algorithm combines CNN and attention system and divides the embedding layer into user information feature embedding and data name feature extraction embedding.It obtains data name features through a convolution kernel.Finally,the top pooling layer obtains the length vector.The attention system layer obtains the characteristics of the data type.Experimental results show that the proposed recommendation algorithm that combines CNN and the attention system can perform better in data extraction than the traditional CNN algorithm and other recommendation algorithms that are popular at the present stage.The proposed algorithm shows excellent accuracy and robustness.

CNN Channel Attention Intrusion Detection SystemUsing NSL-KDD Dataset

Intrusion detection systems(IDS)are essential in the field of cybersecurity because they protect networks from a wide range of online threats.The goal of this research is to meet the urgent need for small-footprint,highly-adaptable Network Intrusion Detection Systems(NIDS)that can identify anomalies.The NSL-KDD dataset is used in the study;it is a sizable collection comprising 43 variables with the label’s“attack”and“level.”It proposes a novel approach to intrusion detection based on the combination of channel attention and convolutional neural networks(CNN).Furthermore,this dataset makes it easier to conduct a thorough assessment of the suggested intrusion detection strategy.Furthermore,maintaining operating efficiency while improving detection accuracy is the primary goal of this work.Moreover,typical NIDS examines both risky and typical behavior using a variety of techniques.On the NSL-KDD dataset,our CNN-based approach achieves an astounding 99.728%accuracy rate when paired with channel attention.Compared to previous approaches such as ensemble learning,CNN,RBM(Boltzmann machine),ANN,hybrid auto-encoders with CNN,MCNN,and ANN,and adaptive algorithms,our solution significantly improves intrusion detection performance.Moreover,the results highlight the effectiveness of our suggested method in improving intrusion detection precision,signifying a noteworthy advancement in this field.Subsequent efforts will focus on strengthening and expanding our approach in order to counteract growing cyberthreats and adjust to changing network circumstances.

An Image Fingerprint and Attention Mechanism Based Load Estimation Algorithm for Electric Power System

With the rapid development of electric power systems,load estimation plays an important role in system operation and planning.Usually,load estimation techniques contain traditional,time series,regression analysis-based,and machine learning-based estimation.Since the machine learning-based method can lead to better performance,in this paper,a deep learning-based load estimation algorithm using image fingerprint and attention mechanism is proposed.First,an image fingerprint construction is proposed for training data.After the data preprocessing,the training data matrix is constructed by the cyclic shift and cubic spline interpolation.Then,the linear mapping and the gray-color transformation method are proposed to form the color image fingerprint.Second,a convolutional neural network(CNN)combined with an attentionmechanism is proposed for training performance improvement.At last,an experiment is carried out to evaluate the estimation performance.Compared with the support vector machine method,CNN method and long short-term memory method,the proposed algorithm has the best load estimation performance.

引入上下文信息和Attention Gate的GUS-YOLO遥感目标检测算法

目前基于通用YOLO系列的遥感目标检测算法存在并未充分利用图像的全局上下文信息,在特征融合金字塔部分并未充分考虑缩小融合特征之间的语义鸿沟、抑制冗余信息干扰的缺点。在结合YOLO算法优点的基础上提出GUS-YOLO算法,其拥有一个能够充分利用全局上下文信息的骨干网络Global Backbone。除此之外,该算法在融合特征金字塔自顶向下的结构中引入Attention Gate模块,可以突出必要的特征信息,抑制冗余信息。另外,为Attention Gate模块设计了最佳的网络结构,提出了网络的特征融合结构U-Net。最后,为克服ReLU函数可能导致模型梯度不再更新的问题,该算法将Attention Gate模块的激活函数升级为可学习的SMU激活函数,提高模型鲁棒性。在NWPU VHR-10遥感数据集上,该算法相较于YOLOV7算法取得宽松指标mAP^(0.50)1.64个百分点和严格指标mAP^(0.75)9.39个百分点的性能提升。相较于目前主流的七种检测算法,该算法取得较好的检测性能。

基于XGBoost-WOA-BiLSTM-Attention的公共建筑暖通空调能耗预测研究

为在双碳目标下实现节能减排,降低能源成本,提出一种基于BiLSTM的公共建筑暖通空调能耗预测模型。在BiLSTM模型基础上,使用XGBoost算法对输入特征进行选择,剔除冗余特征,得到最佳模型输入特征;然后利用WOA优化算法对添加了Attention机制的BiLSTM模型中的6个超参数进行优化,将得到的最优参数代入BiLSTM-Attention神经网络中进行预测,并与BiLSTM模型、BiLSTM-Attention模型和WOA-BiLSTM-Attention模型进行对比。结果表明,所提出的XGBoost-WOA-BiLSTM-Attention模型的RMSE、MAE、R2分别为0.0106、0.006、0.9991,优于其他模型,且相对于持续模型在均方根误差RMSE上提升了98%,为降低公共建筑暖通空调能耗研究提供了参考。

基于Coordinate Attention和空洞卷积的异物识别

在我国工厂的工业化生产中,带式运输机占有重要的地位,但是在其运输物料的过程中,常有木板、金属管、大型金属片等混入物料中,从而对带式运输机的传送带造成损毁,引起巨大的经济损失.为了检测出传送带上的不规则异物,设计了一种新的异物检测方法.针对传统异物检测方法中存在的对于图像特征提取能力不足以及网络感受野相对较小的问题,我们提出了一种基于coordinate attention和空洞卷积的单阶段异物识别方法.首先,网络利用coordinate attention机制,使网络更加关注图像的空间信息,并对图像中的重要特征进行了增强,增强了网络的性能;其次,在网络提取多尺度特征的部分,将原网络的静态卷积变为空洞卷积,有效减少了常规卷积造成的信息损失;除此之外,我们还使用了新的损失函数,进一步提高了网络的性能.实验结果证明,我们提出的网络能有效识别出传送带上的异物,较好地完成异物检测任务.

基于ALBERT-Seq2Seq-Attention模型的数字化档案多标签分类

针对现有的数字化档案多标签分类方法存在分类标签之间缺少关联性的问题,提出一种用于档案多标签分类的深层神经网络模型ALBERT-Seq2Seq-Attention.该模型通过ALBERT(A Little BERT)预训练语言模型内部多层双向的Transfomer结构获取进行文本特征向量的提取,并获得上下文语义信息;将预训练提取的文本特征作为Seq2Seq-Attention(Sequence to Sequence-Attention)模型的输入序列,构建标签字典以获取多标签间的关联关系.将分类模型在3种数据集上分别进行对比实验,结果表明:模型分类的效果F1值均超过90%.该模型不仅能提高档案文本的多标签分类效果,也能关注标签之间的相关关系.

Attention Markets of Blockchain-Based Decentralized Autonomous Organizations

The attention is a scarce resource in decentralized autonomous organizations(DAOs),as their self-governance relies heavily on the attention-intensive decision-making process of“proposal and voting”.To prevent the negative effects of pro-posers’attention-capturing strategies that contribute to the“tragedy of the commons”and ensure an efficient distribution of attention among multiple proposals,it is necessary to establish a market-driven allocation scheme for DAOs’attention.First,the Harberger tax-based attention markets are designed to facilitate its allocation via continuous and automated trading,where the individualized Harberger tax rate(HTR)determined by the pro-posers’reputation is adopted.Then,the Stackelberg game model is formulated in these markets,casting attention to owners in the role of leaders and other competitive proposers as followers.Its equilibrium trading strategies are also discussed to unravel the intricate dynamics of attention pricing.Moreover,utilizing the single-round Stackelberg game as an illustrative example,the existence of Nash equilibrium trading strategies is demonstrated.Finally,the impact of individualized HTR on trading strategies is investigated,and results suggest that it has a negative correlation with leaders’self-accessed prices and ownership duration,but its effect on their revenues varies under different conditions.This study is expected to provide valuable insights into leveraging attention resources to improve DAOs’governance and decision-making process.

基于CNN-LSTM-Attention和自回归的混合水位预测模型

【目的】水位预测对交通运输、农业以及防洪措施具有重要影响。精确的水位值可用于提升水道运输的安全及效率、降低洪水风险,同时也是保障区域可持续发展的必要条件。【方法】提出一种CRANet的混合水位预测模型,以卷积神经网络(CNN)、长短期记忆网络(LSTM)、注意力机制以及自回归(AR)组件为基础,旨在应对时间序列数据中存在的线性与非线性问题,缓解自回归及ARIMA模型的缺陷。其应用不仅在于为航运调度提供决策支撑,加强导航安全效率,同样能提升防洪减灾的能力。其中,CNN和LSTM组件有效地针对数据集内的局部和全局关系进行捕捉,AR组件则能充分考虑数据的时间序列特性。同时,通过注意力机制,模型能够优先考虑相关特性,提高预测效果。【结果】研究成果所提出的模型已成功应用于中国西江梧州站的水位预测,在测试集上预测未来3 h级别水位的MAE、RMSE和R^(2)分别为0.086、0.114 5和0.950 8。【结论】结果表明所提出的CRANet模型在水位预测方面的高可用性、准确度与稳健性,相较于AR、SVR、CNN、LSTM等模型具有更优的MAE、RMSE和R^(2)。

An Underwater Target Detection Algorithm Based on Attention Mechanism and Improved YOLOv7

For underwater robots in the process of performing target detection tasks,the color distortion and the uneven quality of underwater images lead to great difficulties in the feature extraction process of the model,which is prone to issues like error detection,omission detection,and poor accuracy.Therefore,this paper proposed the CER-YOLOv7(CBAM-EIOU-RepVGG-YOLOv7)underwater target detection algorithm.To improve the algorithm’s capability to retain valid features from both spatial and channel perspectives during the feature extraction phase,we have added a Convolutional Block Attention Module(CBAM)to the backbone network.The Reparameterization Visual Geometry Group(RepVGG)module is inserted into the backbone to improve the training and inference capabilities.The Efficient Intersection over Union(EIoU)loss is also used as the localization loss function,which reduces the error detection rate and missed detection rate of the algorithm.The experimental results of the CER-YOLOv7 algorithm on the UPRC(Underwater Robot Prototype Competition)dataset show that the mAP(mean Average Precision)score of the algorithm is 86.1%,which is a 2.2%improvement compared to the YOLOv7.The feasibility and validity of the CER-YOLOv7 are proved through ablation and comparison experiments,and it is more suitable for underwater target detection.

融合MacBERT和Talking⁃Heads Attention实体关系联合抽取模型

针对现有的医学文本关系抽取任务模型在训练过程中存在语义理解能力不足,可能导致关系抽取的效果不尽人意的问题,文中提出一种融合MacBERT和Talking⁃Heads Attention的实体关系联合抽取模型。该模型首先利用MacBERT语言模型来获取动态字向量表达,MacBERT作为改进的BERT模型,能够减少预训练和微调阶段之间的差异,从而提高模型的泛化能力;然后,将这些动态字向量表达输入到双向门控循环单元(BiGRU)中,以便提取文本的上下文特征。BiGRU是一种改进的循环神经网络(RNN),具有更好的长期依赖捕获能力。在获取文本上下文特征之后,使用Talking⁃Heads Attention来获取全局特征。Talking⁃Heads Attention是一种自注意力机制,可以捕获文本中不同位置之间的关系,从而提高关系抽取的准确性。实验结果表明,与实体关系联合抽取模型GRTE相比,该模型F1值提升1%,precision值提升0.4%,recall值提升1.5%。

基于Attention机制和递归思想的LSTM车辆轨迹预测

针对现有车辆轨迹预测模型在长时预测方面准确性不足的问题,基于Attention机制和递归思想的长短时记忆网络(long short-term memory, LSTM)构建了一种新型的车辆轨迹预测模型,即ATT-LSTM(RE)模型,使用编码器–解码器架构更精确地预测车辆未来的行驶轨迹。研究结果表明,模型意图识别的准确率为91.7%,F1分数、召回率、精确率均在0.872~0.977之间;1 s、2 s、3 s、4 s、5 s的终点轨迹预测的均方根误差为0.52 m、1.07 m、1.69 m、2.58 m、3.31 m,优于同类型模型。

基于双向LSTM-Attention模型的火电厂负荷预测研究

准确预测电厂负荷可指导火电厂制定发电计划和调度安排,有利于降低能源成本和污染物排放,对电厂的经济性和环保性有重要意义。本文提出一种基于双向LSTM-Attention的火电厂负荷预测方法。首先,通过皮尔逊系数筛选出关键特征变量;其次利用双向长短期记忆网络提取关键变量之间的长期依赖关系与短期变化特征,最后融合注意力权重机制以进一步突出关键时序信息,进而实现负荷的准确预测。以某在役600 MW超临界机组为对象进行验证。结果表明:相较于单向LSTM、双向LSTM、单向LSTM-Attention,本文所提方法的决定系数R^(2)、均方根误差S_(RMSE)和平均绝对误差S_(MAE)均为最优,分别为0.9566、16.3159、13.5043,能更准确地捕捉到负荷快速波动的趋势,为电厂的负荷预测和能源管理提供可行的方法。

基于Attention-BiLSTM混合模型的月尺度降水量预测

降水受到多种气象因素的影响,从而导致降水预测精度不高。针对这个问题,在考虑影响降水的多个气象因素基础上,通过Attention机制赋予各种气象因素不同的权重,结合双向长短期记忆神经网络(BiLSTM),提出了改进的Attention-BiLSTM混合模型去实现月尺度降水量的预测。以江西省南昌气象站为例,将1989—2018年的逐月降水量与逐月气象因素(气温、蒸发量、气压等)观测资料作为模型输入数据,通过Attention机制识别出各种气象因素的权重,从而提高BiLSTM模型对降水量的预测性能。结果表明:Attention-BiLSTM混合模型可有效地提高降水量预测的精度;通过Attention机制的修正,显著地改善了原有的BiLSTM模型降水量预测值偏低的问题。

Causal temporal graph attention network for fault diagnosis of chemical processes

Fault detection and diagnosis(FDD)plays a significant role in ensuring the safety and stability of chemical processes.With the development of artificial intelligence(AI)and big data technologies,data-driven approaches with excellent performance are widely used for FDD in chemical processes.However,improved predictive accuracy has often been achieved through increased model complexity,which turns models into black-box methods and causes uncertainty regarding their decisions.In this study,a causal temporal graph attention network(CTGAN)is proposed for fault diagnosis of chemical processes.A chemical causal graph is built by causal inference to represent the propagation path of faults.The attention mechanism and chemical causal graph were combined to help us notice the key variables relating to fault fluctuations.Experiments in the Tennessee Eastman(TE)process and the green ammonia(GA)process showed that CTGAN achieved high performance and good explainability.

An enhanced method for predicting and analysing forest fires using an attention-based CNN model

Prediction,prevention,and control of forest fires are crucial on at all scales.Developing effective fire detection systems can aid in their control.This study proposes a novel CNN(convolutional neural network)using an attention blocks module which combines an attention module with numerous input layers to enhance the performance of neural networks.The suggested model focuses on predicting the damage affected/burned areas due to possible wildfires and evaluating the multilateral interactions between the pertinent factors.The results show the impacts of CNN using attention blocks for feature extraction and to better understand how ecosystems are affected by meteorological factors.For selected meteorological data,RMSE 12.08 and MAE 7.45 values provide higher predictive power for selecting relevant and necessary features to provide optimal performance with less operational and computational costs.These findings show that the suggested strategy is reliable and effective for planning and managing fire-prone regions as well as for predicting forest fire damage.

MicroRNAs as potential biomarkers for diagnosis of attention deficit hyperactivity disorder

Inappropriate levels of hyperactivity,impulsivity,and inattention characterize attention deficit hyperactivity disorder,a common childhood-onset neuropsychiatric disorder.The cognitive function and learning ability of children with attention deficit hyperactivity disorder are affected,and these symptoms may persist to adulthood if they are not treated.The diagnosis of attention deficit hyperactivity disorder is only based on symptoms and objective tests for attention deficit hyperactivity disorder are missing.Treatments for attention deficit hyperactivity disorder in children include medications,behavior therapy,counseling,and education services which can relieve many of the symptoms of attention deficit hyperactivity disorder but cannot cure it.There is a need for a molecular biomarker to distinguish attention deficit hyperactivity disorder from healthy subjects and other neurological conditions,which would allow for an earlier and more accurate diagnosis and appropriate treatment to be initiated.Abnormal expression of microRNAs is connected to brain development and disease and could provide novel biomarkers for the diagnosis and prognosis of attention deficit hyperactivity disorder.The recent studies reviewed had performed microRNA profiling in whole blood,white blood cells,blood plasma,and blood serum of children with attention deficit hyperactivity disorder.A large number of microRNAs were dysregulated when compared to healthy controls and with some overlap between individual studies.From the studies that had included a validation set of patients and controls,potential candidate biomarkers for attention deficit hyperactivity disorder in children could be miR-140-3p,let-7g-5p,-30e-5p,-223-3p,-142-5p,-486-5p,-151a-3p,-151a-5p,and-126-5p in total white blood cells,and miR-4516,-6090,-4763-3p,-4281,-4466,-101-3p,-130a-3p,-138-5p,-195-5p,and-106b-5p in blood serum.Further studies are warranted with children and adults with attention deficit hyperactivity disorder,and consideration should be given to utilizing rat models of attention deficit hyperactivity disorder.Animal studies could be used to confirm microRNA findings in human patients and to test the effects of targeting specific microRNAs on disease progression and behavior.

An attention-based teacher-student model for multivariate short-term landslide displacement prediction incorporating weather forecast data

Predicting the displacement of landslide is of utmost practical importance as the landslide can pose serious threats to both human life and property.However,traditional methods have the limitation of random selection in sliding window selection and seldom incorporate weather forecast data for displacement prediction,while a single structural model cannot handle input sequences of different lengths at the same time.In order to solve these limitations,in this study,a new approach is proposed that utilizes weather forecast data and incorporates the maximum information coefficient(MIC),long short-term memory network(LSTM),and attention mechanism to establish a teacher-student coupling model with parallel structure for short-term landslide displacement prediction.Through MIC,a suitable input sequence length is selected for the LSTM model.To investigate the influence of rainfall on landslides during different seasons,a parallel teacher-student coupling model is developed that is able to learn sequential information from various time series of different lengths.The teacher model learns sequence information from rainfall intensity time series while incorporating reliable short-term weather forecast data from platforms such as China Meteorological Administration(CMA)and Reliable Prognosis(https://rp5.ru)to improve the model’s expression capability,and the student model learns sequence information from other time series.An attention module is then designed to integrate different sequence information to derive a context vector,representing seasonal temporal attention mode.Finally,the predicted displacement is obtained through a linear layer.The proposed method demonstrates superior prediction accuracies,surpassing those of the support vector machine(SVM),LSTM,recurrent neural network(RNN),temporal convolutional network(TCN),and LSTM-Attention models.It achieves a mean absolute error(MAE)of 0.072 mm,root mean square error(RMSE)of 0.096 mm,and pearson correlation coefficients(PCCS)of 0.85.Additionally,it exhibits enhanced prediction stability and interpretability,rendering it an indispensable tool for landslide disaster prevention and mitigation.

Automatic recognition of landslides based on YOLOv7 and attention mechanism

Landslide disasters comprise the majority of geological incidents on slopes,posing severe threats to the safety of human lives and property while exerting a significant impact on the geological environment.The rapid identification of landslides is important for disaster prevention and control;however,currently,landslide identification relies mainly on the manual interpretation of remote sensing images.Manual interpretation and feature recognition methods are time-consuming,labor-intensive,and challenging when confronted with complex scenarios.Consequently,automatic landslide recognition has emerged as a pivotal avenue for future development.In this study,a dataset comprising 2000 landslide images was constructed using open-source remote sensing images and datasets.The YOLOv7 model was enhanced using data augmentation algorithms and attention mechanisms.Three optimization models were formulated to realize automatic landslide recognition.The findings demonstrate the commendable performance of the optimized model in automatic landslide recognition,achieving a peak accuracy of 95.92%.Subsequently,the optimized model was applied to regional landslide identification,co-seismic landslide identification,and landslide recognition at various scales,all of which showed robust recognition capabilities.Nevertheless,the model exhibits limitations in detecting small targets,indicating areas for refining the deep-learning algorithms.The results of this research offer valuable technical support for the swift identification,prevention,and mitigation of landslide disasters.

Image Inpainting Technique Incorporating Edge Prior and Attention Mechanism

Recently,deep learning-based image inpainting methods have made great strides in reconstructing damaged regions.However,these methods often struggle to produce satisfactory results when dealing with missing images with large holes,leading to distortions in the structure and blurring of textures.To address these problems,we combine the advantages of transformers and convolutions to propose an image inpainting method that incorporates edge priors and attention mechanisms.The proposed method aims to improve the results of inpainting large holes in images by enhancing the accuracy of structure restoration and the ability to recover texture details.This method divides the inpainting task into two phases:edge prediction and image inpainting.Specifically,in the edge prediction phase,a transformer architecture is designed to combine axial attention with standard self-attention.This design enhances the extraction capability of global structural features and location awareness.It also balances the complexity of self-attention operations,resulting in accurate prediction of the edge structure in the defective region.In the image inpainting phase,a multi-scale fusion attention module is introduced.This module makes full use of multi-level distant features and enhances local pixel continuity,thereby significantly improving the quality of image inpainting.To evaluate the performance of our method.comparative experiments are conducted on several datasets,including CelebA,Places2,and Facade.Quantitative experiments show that our method outperforms the other mainstream methods.Specifically,it improves Peak Signal-to-Noise Ratio(PSNR)and Structure Similarity Index Measure(SSIM)by 1.141~3.234 db and 0.083~0.235,respectively.Moreover,it reduces Learning Perceptual Image Patch Similarity(LPIPS)and Mean Absolute Error(MAE)by 0.0347~0.1753 and 0.0104~0.0402,respectively.Qualitative experiments reveal that our method excels at reconstructing images with complete structural information and clear texture details.Furthermore,our model exhibits impressive performance in terms of the number of parameters,memory cost,and testing time.