Self-potential inversion based on Attention U-Net deep learning network

Landfill leaks pose a serious threat to environmental health,risking the contamination of both groundwater and soil resources.Accurate investigation of these sites is essential for implementing effective prevention and control measures.The self-potential(SP)stands out for its sensitivity to contamination plumes,offering a solution for monitoring and detecting the movement and seepage of subsurface pollutants.However,traditional SP inversion techniques heavily rely on precise subsurface resistivity information.In this study,we propose the Attention U-Net deep learning network for rapid SP inversion.By incorporating an attention mechanism,this algorithm effectively learns the relationship between array-style SP data and the location and extent of subsurface contaminated sources.We designed a synthetic landfill model with a heterogeneous resistivity structure to assess the performance of Attention U-Net deep learning network.Additionally,we conducted further validation using a laboratory model to assess its practical applicability.The results demonstrate that the algorithm is not solely dependent on resistivity information,enabling effective locating of the source distribution,even in models with intricate subsurface structures.Our work provides a promising tool for SP data processing,enhancing the applicability of this method in the field of near-subsurface environmental monitoring.

Decoding topological XYZ^(2) codes with reinforcement learning based on attention mechanisms

Quantum error correction, a technique that relies on the principle of redundancy to encode logical information into additional qubits to better protect the system from noise, is necessary to design a viable quantum computer. For this new topological stabilizer code-XYZ^(2) code defined on the cellular lattice, it is implemented on a hexagonal lattice of qubits and it encodes the logical qubits with the help of stabilizer measurements of weight six and weight two. However topological stabilizer codes in cellular lattice quantum systems suffer from the detrimental effects of noise due to interaction with the environment. Several decoding approaches have been proposed to address this problem. Here, we propose the use of a state-attention based reinforcement learning decoder to decode XYZ^(2) codes, which enables the decoder to more accurately focus on the information related to the current decoding position, and the error correction accuracy of our reinforcement learning decoder model under the optimisation conditions can reach 83.27% under the depolarizing noise model, and we have measured thresholds of 0.18856 and 0.19043 for XYZ^(2) codes at code spacing of 3–7 and 7–11, respectively. our study provides directions and ideas for applications of decoding schemes combining reinforcement learning attention mechanisms to other topological quantum error-correcting codes.

A Multi-Feature Learning Model with Enhanced Local Attention for Vehicle Re-Identification

Vehicle re-identification(ReID)aims to retrieve the target vehicle in an extensive image gallery through its appearances from various views in the cross-camera scenario.It has gradually become a core technology of intelligent transportation system.Most existing vehicle re-identification models adopt the joint learning of global and local features.However,they directly use the extracted global features,resulting in insufficient feature expression.Moreover,local features are primarily obtained through advanced annotation and complex attention mechanisms,which require additional costs.To solve this issue,a multi-feature learning model with enhanced local attention for vehicle re-identification(MFELA)is proposed in this paper.The model consists of global and local branches.The global branch utilizes both middle and highlevel semantic features of ResNet50 to enhance the global representation capability.In addition,multi-scale pooling operations are used to obtain multiscale information.While the local branch utilizes the proposed Region Batch Dropblock(RBD),which encourages the model to learn discriminative features for different local regions and simultaneously drops corresponding same areas randomly in a batch during training to enhance the attention to local regions.Then features from both branches are combined to provide a more comprehensive and distinctive feature representation.Extensive experiments on VeRi-776 and VehicleID datasets prove that our method has excellent performance.

An Improved Attention Parameter Setting Algorithm Based on Award Learning Mechanism

The setting of attention parameters plays a role in the performance of synergetic neural network based on PFAP model. This paper first analyzes the attention parameter setting algorithm based on award-penalty learning mechanism. Then, it presents an improved algorithm to overcome its drawbacks. The experimental results demonstrate that the novel algorithm is better than the original one under the same circumstances.

Multi-Head Attention Graph Network for Few Shot Learning

The majority of existing graph-network-based few-shot models focus on a node-similarity update mode.The lack of adequate information intensies the risk of overtraining.In this paper,we propose a novel Multihead Attention Graph Network to excavate discriminative relation and fulll effective information propagation.For edge update,the node-level attention is used to evaluate the similarities between the two nodes and the distributionlevel attention extracts more in-deep global relation.The cooperation between those two parts provides a discriminative and comprehensive expression for edge feature.For node update,we embrace the label-level attention to soften the noise of irrelevant nodes and optimize the update direction.Our proposed model is veried through extensive experiments on two few-shot benchmark MiniImageNet and CIFAR-FS dataset.The results suggest that our method has a strong capability of noise immunity and quick convergence.The classication accuracy outperforms most state-of-the-art approaches.

An Efficient Indoor Localization Based on Deep Attention Learning Model

Indoor localization methods can help many sectors,such as healthcare centers,smart homes,museums,warehouses,and retail malls,improve their service areas.As a result,it is crucial to look for low-cost methods that can provide exact localization in indoor locations.In this context,imagebased localization methods can play an important role in estimating both the position and the orientation of cameras regarding an object.Image-based localization faces many issues,such as image scale and rotation variance.Also,image-based localization’s accuracy and speed(latency)are two critical factors.This paper proposes an efficient 6-DoF deep-learning model for image-based localization.This model incorporates the channel attention module and the Scale PyramidModule(SPM).It not only enhances accuracy but also ensures the model’s real-time performance.In complex scenes,a channel attention module is employed to distinguish between the textures of the foregrounds and backgrounds.Our model adapted an SPM,a feature pyramid module for dealing with image scale and rotation variance issues.Furthermore,the proposed model employs two regressions(two fully connected layers),one for position and the other for orientation,which increases outcome accuracy.Experiments on standard indoor and outdoor datasets show that the proposed model has a significantly lower Mean Squared Error(MSE)for both position and orientation.On the indoor 7-Scenes dataset,the MSE for the position is reduced to 0.19 m and 6.25°for the orientation.Furthermore,on the outdoor Cambridge landmarks dataset,the MSE for the position is reduced to 0.63 m and 2.03°for the orientation.According to the findings,the proposed approach is superior and more successful than the baseline methods.

Irregularly sampled seismic data interpolation via wavelet-based convolutional block attention deep learning

Seismic data interpolation,especially irregularly sampled data interpolation,is a critical task for seismic processing and subsequent interpretation.Recently,with the development of machine learning and deep learning,convolutional neural networks(CNNs)are applied for interpolating irregularly sampled seismic data.CNN based approaches can address the apparent defects of traditional interpolation methods,such as the low computational efficiency and the difficulty on parameters selection.However,current CNN based methods only consider the temporal and spatial features of irregularly sampled seismic data,which fail to consider the frequency features of seismic data,i.e.,the multi-scale features.To overcome these drawbacks,we propose a wavelet-based convolutional block attention deep learning(W-CBADL)network for irregularly sampled seismic data reconstruction.We firstly introduce the discrete wavelet transform(DWT)and the inverse wavelet transform(IWT)to the commonly used U-Net by considering the multi-scale features of irregularly sampled seismic data.Moreover,we propose to adopt the convolutional block attention module(CBAM)to precisely restore sampled seismic traces,which could apply the attention to both channel and spatial dimensions.Finally,we adopt the proposed W-CBADL model to synthetic and pre-stack field data to evaluate its validity and effectiveness.The results demonstrate that the proposed W-CBADL model could reconstruct irregularly sampled seismic data more effectively and more efficiently than the state-of-the-art contrastive CNN based models.

Multi-Task Deep Learning with Task Attention for Post-Click Conversion Rate Prediction

Online advertising has gained much attention on various platforms as a hugely lucrative market.In promoting content and advertisements in real life,the acquisition of user target actions is usually a multi-step process,such as impres-sion→click→conversion,which means the process from the delivery of the recommended item to the user’s click to the final conversion.Due to data sparsity or sample selection bias,it is difficult for the trained model to achieve the business goal of the target campaign.Multi-task learning,a classical solution to this pro-blem,aims to generalize better on the original task given several related tasks by exploiting the knowledge between tasks to share the same feature and label space.Adaptively learned task relations bring better performance to make full use of the correlation between tasks.We train a general model capable of captur-ing the relationships between various tasks on all existing active tasks from a meta-learning perspective.In addition,this paper proposes a Multi-task Attention Network(MAN)to identify commonalities and differences between tasks in the feature space.The model performance is improved by explicitly learning the stacking of task relationships in the label space.To illustrate the effectiveness of our method,experiments are conducted on Alibaba Click and Conversion Pre-diction(Ali-CCP)dataset.Experimental results show that the method outperforms the state-of-the-art multi-task learning methods.

Low-Cost Real-Time Automated Optical Inspection Using Deep Learning and Attention Map

The recent trends in Industry 4.0 and Internet of Things have encour-aged many factory managers to improve inspection processes to achieve automa-tion and high detection rates.However,the corresponding cost results of sample tests are still used for quality control.A low-cost automated optical inspection system that can be integrated with production lines to fully inspect products with-out adjustments is introduced herein.The corresponding mechanism design enables each product to maintain afixed position and orientation during inspec-tion to accelerate the inspection process.The proposed system combines image recognition and deep learning to measure the dimensions of the thread and iden-tify its defects within 20 s,which is lower than the production-line productivity per 30 s.In addition,the system is designed to be used for monitoring production lines and equipment status.The dimensional tolerance of the proposed system reaches 0.012 mm,and a 100%accuracy is achieved in terms of the defect reso-lution.In addition,an attention-based visualization approach is utilized to verify the rationale for the use of the convolutional neural network model and identify the location of thread defects.

Attention-Guided Organized Perception and Learning of Object Categories Based on Probabilistic Latent Variable Models

This paper proposes a probabilistic model of object category learning in conjunction with attention-guided organized perception. This model consists of a model of attention-guided organized perception of object segments on Markov random fields and a model of learning object categories based on a probabilistic latent component analysis. In attention guided organized perception, concurrent figure-ground segmentation is performed on dynamically-formed Markov random fields around salient preattentive points and co-occurring segments are grouped in the neighborhood of selective attended segments. In object category learning, a set of classes of each object category is obtained based on the probabilistic latent component analysis with the variable number of classes from bags of features of segments extracted from images which contain the categorical objects in context and an object category is represented by a composite of object classes. Through experiments using two image data sets, it is shown that the model learns a probabilistic structure of intra-categorical composition and inter-categorical difference of object categories and achieves high performance in object category recognition.

COVAD: Content-oriented video anomaly detection using a self attention-based deep learning model

Background Video anomaly detection has always been a hot topic and has attracted increasing attention.Many of the existing methods for video anomaly detection depend on processing the entire video rather than considering only the significant context. Method This paper proposes a novel video anomaly detection method called COVAD that mainly focuses on the region of interest in the video instead of the entire video. Our proposed COVAD method is based on an autoencoded convolutional neural network and a coordinated attention mechanism,which can effectively capture meaningful objects in the video and dependencies among different objects. Relying on the existing memory-guided video frame prediction network, our algorithm can significantly predict the future motion and appearance of objects in a video more effectively. Result The proposed algorithm obtained better experimental results on multiple datasets and outperformed the baseline models considered in our analysis. Simultaneously, we provide an improved visual test that can provide pixel-level anomaly explanations.

Towards Mining Public Opinion: An Attention-Based Long Short Term Memory Network Using Transfer Learning

The Internet provides a large number of tools and resources, such as social media sites, online newsgroups, blogs, electronic forums, virtual communities, and online travel sites, for consumers to express their views or opinions regarding various issues. These opinions can help organizations like tourism to improve their products and services for their consumers. Opinion mining refers to a process of identifying emotions by applying Natural Language Processing (NLP) techniques to a pool of texts. This paper mainly focuses on mining public opinion from the hotel reviews domain. To do so, we proposed a novel technique called the Attention-Based Long Short Term Memory (Attention-LSTM) Network using a transfer learning approach. We empirically analyzed several machine learning and deep learning methods and observed our proposed technique provided an adequate performance for mining public opinion in the hotel reviews domain.

融合RoBERTa-GCN-Attention的隐喻识别与情感分类模型

在隐喻识别与隐喻情感分类任务的联合研究中,现有多任务学习模型存在对隐喻语料中的上下文语义信息和句法结构信息提取不够准确,并且缺乏对粗细两种粒度信息同时捕捉的问题.针对第1个问题,首先改进了传统的RoBERTa模型,在原有的自注意力机制中引入上下文信息,以此提取上下文中重要的隐喻语义特征;其次在句法依存树上使用图卷积网络提取隐喻句中的句法结构信息.针对第2个问题,使用双层注意力机制,分别聚焦于单词和句子层面中对隐喻识别和情感分类有贡献的特征信息.在两类任务6个数据集上的对比实验结果表明,该模型相比基线模型性能均有提升.

基于GA-VMD与CNN-BiLSTM-Attention模型的区域碳排放交易价格预测研究

准确的碳价预测可为碳排放权交易市场监管者和投资者提供决策依据与参考。本文提出了基于GA-VMD降噪分解及CNN-BiLSTM-Attention混合模型的碳价预测方法,并选取湖北碳市场2014年4月2日到2022年6月15日1857个交易日的数据进行分析:首先通过遗传算法改进变分模态分解(GA-VMD)将原始碳价序列分解为平稳的本征模态函数(IMF)分量,降低数据噪音;随后构建CNN-BiLSTM-Attention混合模型对各IMF分量进行预测。其中,卷积神经网络(CNN)可提取影响碳价多个特征,双向长短期记忆网络(BiLSTM)可实现时间序列信息提取,注意力机制(Attention)可突出某个关键输入对输出的影响。本文将预测出的各IMF分量集合成碳价序列,并提出12个模型,分为3个组进行剥离分析,结果显示GA-VMD-CNN-BiLSTM-Attention的预测结果最好。另外,为给市场参与者提供更多信息,本文在确定性预测的基础上加入区间预测,以便提前测量碳市场的波动性。

Enhancing Deep Learning Soil Moisture Forecasting Models by Integrating Physics-based Models

Accurate soil moisture(SM)prediction is critical for understanding hydrological processes.Physics-based(PB)models exhibit large uncertainties in SM predictions arising from uncertain parameterizations and insufficient representation of land-surface processes.In addition to PB models,deep learning(DL)models have been widely used in SM predictions recently.However,few pure DL models have notably high success rates due to lacking physical information.Thus,we developed hybrid models to effectively integrate the outputs of PB models into DL models to improve SM predictions.To this end,we first developed a hybrid model based on the attention mechanism to take advantage of PB models at each forecast time scale(attention model).We further built an ensemble model that combined the advantages of different hybrid schemes(ensemble model).We utilized SM forecasts from the Global Forecast System to enhance the convolutional long short-term memory(ConvLSTM)model for 1–16 days of SM predictions.The performances of the proposed hybrid models were investigated and compared with two existing hybrid models.The results showed that the attention model could leverage benefits of PB models and achieved the best predictability of drought events among the different hybrid models.Moreover,the ensemble model performed best among all hybrid models at all forecast time scales and different soil conditions.It is highlighted that the ensemble model outperformed the pure DL model over 79.5%of in situ stations for 16-day predictions.These findings suggest that our proposed hybrid models can adequately exploit the benefits of PB model outputs to aid DL models in making SM predictions.

An Underwater Target Detection Algorithm Based on Attention Mechanism and Improved YOLOv7

For underwater robots in the process of performing target detection tasks,the color distortion and the uneven quality of underwater images lead to great difficulties in the feature extraction process of the model,which is prone to issues like error detection,omission detection,and poor accuracy.Therefore,this paper proposed the CER-YOLOv7(CBAM-EIOU-RepVGG-YOLOv7)underwater target detection algorithm.To improve the algorithm’s capability to retain valid features from both spatial and channel perspectives during the feature extraction phase,we have added a Convolutional Block Attention Module(CBAM)to the backbone network.The Reparameterization Visual Geometry Group(RepVGG)module is inserted into the backbone to improve the training and inference capabilities.The Efficient Intersection over Union(EIoU)loss is also used as the localization loss function,which reduces the error detection rate and missed detection rate of the algorithm.The experimental results of the CER-YOLOv7 algorithm on the UPRC(Underwater Robot Prototype Competition)dataset show that the mAP(mean Average Precision)score of the algorithm is 86.1%,which is a 2.2%improvement compared to the YOLOv7.The feasibility and validity of the CER-YOLOv7 are proved through ablation and comparison experiments,and it is more suitable for underwater target detection.

融合MacBERT和Talking⁃Heads Attention实体关系联合抽取模型

针对现有的医学文本关系抽取任务模型在训练过程中存在语义理解能力不足,可能导致关系抽取的效果不尽人意的问题,文中提出一种融合MacBERT和Talking⁃Heads Attention的实体关系联合抽取模型。该模型首先利用MacBERT语言模型来获取动态字向量表达,MacBERT作为改进的BERT模型,能够减少预训练和微调阶段之间的差异,从而提高模型的泛化能力;然后,将这些动态字向量表达输入到双向门控循环单元(BiGRU)中,以便提取文本的上下文特征。BiGRU是一种改进的循环神经网络(RNN),具有更好的长期依赖捕获能力。在获取文本上下文特征之后,使用Talking⁃Heads Attention来获取全局特征。Talking⁃Heads Attention是一种自注意力机制,可以捕获文本中不同位置之间的关系,从而提高关系抽取的准确性。实验结果表明,与实体关系联合抽取模型GRTE相比,该模型F1值提升1%,precision值提升0.4%,recall值提升1.5%。

基于Attention和残差网络的非侵入式负荷监测

非侵入式负荷监测(non-intrusive load monitoring,NILM)可以从家庭电能表的总功率读数,估算出各用电器的功率。由于对于同一类用电器,其状态种类、各状态持续时长、各状态的功率波形都不同,这使得基于特征工程和聚类的模型的泛化能力不强;回归模型的分解功率难以迅速跟踪真实功率。针对这些问题,文中将回归问题转化为在序列每个时刻的多分类问题,并提出基于Attention和残差网络的非侵入式负荷监测模型。该模型基于具有编码器和解码器的seq2seq框架,首先通过嵌入矩阵将高维稀疏one-hot向量映射为低维稠密向量;在编码部分,通过双向GRU从前后两个方向提取序列信息,引入Attention机制计算序列中当前时刻最重要的信息,引入残差连接学习残差部分输入输出之间的差异;在解码部分,用回归层组合BiGRU解码结果,取经过softmax函数处理的最大概率功率类别作为结果。该模型在选取REFIT数据集中表现良好,其中测试集与训练集完全独立,表明训练好的模型可以直接应用在新的住宅用户中。

基于CNN-LSTM-Attention模型的沁河流域径流模拟及未来多情景预测

为提升深度学习模型对变化环境下流域的径流模拟精度,以沁河流域为例,构建了基于卷积神经网络(CNN)、长短期记忆网络(LSTM)和注意力机制(Attention)的CNN-LSTM-Attention耦合模型,加入多种优化算法,结合第六次国际耦合模式比较计划CMIP6中的BCC-CSM2-MR气候模式并考虑多种情景,应用于流域的径流模拟和预测,同时比较了多种深度学习模型的模拟精度。结果表明:CNN-LSTM-Attention模型在沁河流域表现出了较好的径流模拟效果,模拟精度均优于其他深度学习模型,纳什效率系数(NSE)为0.883,均方根误差(RMSE)为2.317,平均绝对误差(MAE)为1.098;不同气候变化情景下,沁河流域在2025—2050年的年径流量均呈现缓慢衰减趋势且波动程度较大,尤其在SSP1-2.6情景下,径流量衰减和波动程度突出。研究可为深度学习模型在人水关系智能化计算模拟领域的应用提供新思路,并为流域后续的水资源开发利用和管理提供科学参考价值。

引入卷积块注意力模块的Attention U-Net木材表面裂纹检测方法

木材缺陷会影响木材的使用价值和使用期限,其中木材表面裂纹是严重影响木材外观质量和机械强度的一种木材缺陷。对木材表面裂纹的检测可以尽快发现此类缺陷木材,或为后续处理提供依据。针对现有的人工检测和自动化检测木材表面裂纹效率低、成本高、漏检率高等问题,采用引入卷积块注意力模块(convolutional block attention module,CBAM)的Attention U-Net深度学习模型对木材表面裂纹图像进行语义分割,从而达到木材表面裂纹检测的目的。引入的CBAM模块包含通道注意力机制和空间注意力机制,分别用于捕捉通道间的依赖关系和像素级的空间关系,该模块被添加到Attention U-Net网络的编码阶段,以增加感兴趣区域的权重并抑制冗余信息。最后,通过消融试验验证了Attention U-Net中加入CBAM对分割性能的提升。采用像素准确率(PA)、类别像素准确率(CPA)、召回率(Recall)、Dice系数、交并比(IoU)和平均交并比(MIoU)等语义分割评价指标评价各模型的优劣,并确定最佳模型及其参数。在自制木材表面数据集的裂纹分割中,使用AdamW优化器引入CBAM的Attention U-Net的PA、木材裂纹Recall、木材裂纹Dice系数、木材裂纹IoU、MIoU分别比使用SGD优化器的Attention U-Net原始模型提高了0.11%,4.14%,2.96%,3.58%和1.84%。结果表明,使用AdamW优化器引入CBAM的Attention U-Net能够较好地分割背景和木材表面裂纹,区分节点、表面纹理和木材裂纹,并将节点和表面纹理分割为背景。