Multi-Scale Attention-Based Deep Neural Network for Brain Disease Diagnosis

Whole brain functional connectivity(FC)patterns obtained from resting-state functional magnetic resonance imaging(rs-fMRI)have been widely used in the diagnosis of brain disorders such as autism spectrum disorder(ASD).Recently,an increasing number of studies have focused on employing deep learning techniques to analyze FC patterns for brain disease classification.However,the high dimensionality of the FC features and the interpretation of deep learning results are issues that need to be addressed in the FC-based brain disease classification.In this paper,we proposed a multi-scale attention-based deep neural network(MSA-DNN)model to classify FC patterns for the ASD diagnosis.The model was implemented by adding a flexible multi-scale attention(MSA)module to the auto-encoder based backbone DNN,which can extract multi-scale features of the FC patterns and change the level of attention for different FCs by continuous learning.Our model will reinforce the weights of important FC features while suppress the unimportant FCs to ensure the sparsity of the model weights and enhance the model interpretability.We performed systematic experiments on the large multi-sites ASD dataset with both ten-fold and leaveone-site-out cross-validations.Results showed that our model outperformed classical methods in brain disease classification and revealed robust intersite prediction performance.We also localized important FC features and brain regions associated with ASD classification.Overall,our study further promotes the biomarker detection and computer-aided classification for ASD diagnosis,and the proposed MSA module is flexible and easy to implement in other classification networks.

MSSTNet:Multi-scale facial videos pulse extraction network based on separable spatiotemporal convolution and dimension separable attention

Background The use of remote photoplethysmography(rPPG)to estimate blood volume pulse in a noncontact manner has been an active research topic in recent years.Existing methods are primarily based on a singlescale region of interest(ROI).However,some noise signals that are not easily separated in a single-scale space can be easily separated in a multi-scale space.Also,existing spatiotemporal networks mainly focus on local spatiotemporal information and do not emphasize temporal information,which is crucial in pulse extraction problems,resulting in insufficient spatiotemporal feature modelling.Methods Here,we propose a multi-scale facial video pulse extraction network based on separable spatiotemporal convolution(SSTC)and dimension separable attention(DSAT).First,to solve the problem of a single-scale ROI,we constructed a multi-scale feature space for initial signal separation.Second,SSTC and DSAT were designed for efficient spatiotemporal correlation modeling,which increased the information interaction between the long-span time and space dimensions;this placed more emphasis on temporal features.Results The signal-to-noise ratio(SNR)of the proposed network reached 9.58dB on the PURE dataset and 6.77dB on the UBFC-rPPG dataset,outperforming state-of-the-art algorithms.Conclusions The results showed that fusing multi-scale signals yielded better results than methods based on only single-scale signals.The proposed SSTC and dimension-separable attention mechanism will contribute to more accurate pulse signal extraction.

Few-shot image recognition based on multi-scale features prototypical network

In order to improve the models capability in expressing features during few-shot learning,a multi-scale features prototypical network(MS-PN)algorithm is proposed.The metric learning algo-rithm is employed to extract image features and project them into a feature space,thus evaluating the similarity between samples based on their relative distances within the metric space.To sufficiently extract feature information from limited sample data and mitigate the impact of constrained data vol-ume,a multi-scale feature extraction network is presented to capture data features at various scales during the process of image feature extraction.Additionally,the position of the prototype is fine-tuned by assigning weights to data points to mitigate the influence of outliers on the experiment.The loss function integrates contrastive loss and label-smoothing to bring similar data points closer and separate dissimilar data points within the metric space.Experimental evaluations are conducted on small-sample datasets mini-ImageNet and CUB200-2011.The method in this paper can achieve higher classification accuracy.Specifically,in the 5-way 1-shot experiment,classification accuracy reaches 50.13%and 66.79%respectively on these two datasets.Moreover,in the 5-way 5-shot ex-periment,accuracy of 66.79%and 85.91%are observed,respectively.

Disease Recognition of Apple Leaf Using Lightweight Multi-Scale Network with ECANet

To solve the problem of difficulty in identifying apple diseases in the natural environment and the low application rate of deep learning recognition networks,a lightweight ResNet(LW-ResNet)model for apple disease recognition is proposed.Based on the deep residual network(ResNet18),the multi-scale feature extraction layer is constructed by group convolution to realize the compression model and improve the extraction ability of different sizes of lesion features.By improving the identity mapping structure to reduce information loss.By introducing the efficient channel attention module(ECANet)to suppress noise from a complex background.The experimental results show that the average precision,recall and F1-score of the LW-ResNet on the test set are 97.80%,97.92%and 97.85%,respectively.The parameter memory is 2.32 MB,which is 94%less than that of ResNet18.Compared with the classic lightweight networks SqueezeNet and MobileNetV2,LW-ResNet has obvious advantages in recognition performance,speed,parameter memory requirement and time complexity.The proposed model has the advantages of low computational cost,low storage cost,strong real-time performance,high identification accuracy,and strong practicability,which can meet the needs of real-time identification task of apple leaf disease on resource-constrained devices.

Stacked Attention Networks for Referring Expressions Comprehension

Referring expressions comprehension is the task of locating the image region described by a natural language expression,which refer to the properties of the region or the relationships with other regions.Most previous work handles this problem by selecting the most relevant regions from a set of candidate regions,when there are many candidate regions in the set these methods are inefficient.Inspired by recent success of image captioning by using deep learning methods,in this paper we proposed a framework to understand the referring expressions by multiple steps of reasoning.We present a model for referring expressions comprehension by selecting the most relevant region directly from the image.The core of our model is a recurrent attention network which can be seen as an extension of Memory Network.The proposed model capable of improving the results by multiple computational hops.We evaluate the proposed model on two referring expression datasets:Visual Genome and Flickr30k Entities.The experimental results demonstrate that the proposed model outperform previous state-of-the-art methods both in accuracy and efficiency.We also conduct an ablation experiment to show that the performance of the model is not getting better with the increase of the attention layers.

A survey of fine-grained visual categorization based on deep learning

Deep learning has achieved excellent results in various tasks in the field of computer vision,especially in fine-grained visual categorization.It aims to distinguish the subordinate categories of the label-level categories.Due to high intra-class variances and high inter-class similarity,the fine-grained visual categorization is extremely challenging.This paper first briefly introduces and analyzes the related public datasets.After that,some of the latest methods are reviewed.Based on the feature types,the feature processing methods,and the overall structure used in the model,we divide them into three types of methods:methods based on general convolutional neural network(CNN)and strong supervision of parts,methods based on single feature processing,and meth-ods based on multiple feature processing.Most methods of the first type have a relatively simple structure,which is the result of the initial research.The methods of the other two types include models that have special structures and training processes,which are helpful to obtain discriminative features.We conduct a specific analysis on several methods with high accuracy on pub-lic datasets.In addition,we support that the focus of the future research is to solve the demand of existing methods for the large amount of the data and the computing power.In terms of tech-nology,the extraction of the subtle feature information with the burgeoning vision transformer(ViT)network is also an important research direction.

Deep learning-based automated grading of visual impairment in cataract patients using fundus images

Cataract is the leading cause of visual impairment globally.The scarcity and uneven distribution of ophthalmologists seriously hinder early visual impairment grading for cataract patients in the clin-ic.In this study,a deep learning-based automated grading system of visual impairment in cataract patients is proposed using a multi-scale efficient channel attention convolutional neural network(MECA_CNN).First,the efficient channel attention mechanism is applied in the MECA_CNN to extract multi-scale features of fundus images,which can effectively focus on lesion-related regions.Then,the asymmetric convolutional modules are embedded in the residual unit to reduce the infor-mation loss of fine-grained features in fundus images.In addition,the asymmetric loss function is applied to address the problem of a higher false-negative rate and weak generalization ability caused by the imbalanced dataset.A total of 7299 fundus images derived from two clinical centers are em-ployed to develop and evaluate the MECA_CNN for identifying mild visual impairment caused by cataract(MVICC),moderate to severe visual impairment caused by cataract(MSVICC),and nor-mal sample.The experimental results demonstrate that the MECA_CNN provides clinically meaning-ful performance for visual impairment grading in the internal test dataset:MVICC(accuracy,sensi-tivity,and specificity;91.3%,89.9%,and 92%),MSVICC(93.2%,78.5%,and 96.7%),and normal sample(98.1%,98.0%,and 98.1%).The comparable performance in the external test dataset is achieved,further verifying the effectiveness and generalizability of the MECA_CNN model.This study provides a deep learning-based practical system for the automated grading of visu-al impairment in cataract patients,facilitating the formulation of treatment strategies in a timely man-ner and improving patients’vision prognosis.

一种基于多模态特征提取的医学视觉问答方法

随着深度学习在医疗领域的快速发展,医学视觉问答(Med-VQA)吸引了研究人员的广泛关注.现有的Med-VQA方法大都使用权重参数共享的同一特征提取网络对多模态医学影像进行特征提取,在一定程度上忽略了不同模态医学影像的差异性特征,导致对特定模态特征提取时引入其它模态的噪声特征,使得模型难以关注到不同模态医学影像中的关键特征.针对上述问题,本文提出一种基于多模态特征提取的医学视觉问答方法.首先,对医学影像进行模态识别,根据模态标签指导输入参数不共享的特征提取网络以获得不同模态影像的差异性特征;然后,设计了一种面向Med-VQA的卷积降噪模块以降低医学影像不同模态特征的噪声信息;最后,采用空间与通道注意力模块进一步增强不同模态差异性特征的关注度.在Med-VQA公共数据集Slake上得到的实验结果表明,本文提出方法能有效提高Med-VQA的准确率.

面向视觉问答的上下文感知多模态交互网络

近年来,视觉问答已经引起研究人员的广泛关注。现有的方法通过视觉与语言模态之间的密集交互以捕捉两种模态之间的高层语义信息,然而这些方法仅单独考虑单个词与视觉区域之间的关系,忽略了上下文信息来计算模态之间的依存关系。针对此问题,该文提出了一种上下文感知的多模态交互网络,通过融合上下文信息增强模态内与模态间的信息交互,提高视觉问答的推理能力。该文在大规模基准数据集VQA v2.0上进行了一系列对比实验与消融实验,实验结果表明,该方法在视觉问答任务上能够取得比当前主流的方法更高的准确率。

基于注意力卷积神经网络的视觉里程计

传统的视觉里程计(visual odometry,VO)要求图像含有大量的纹理信息,且求解过程较为复杂。针对以上问题提出基于注意力卷积神经网络的视觉里程计,对相机进行端到端的位姿估计,利用注意力机制提高模型估计轨迹的精度。首先,使用注意力-卷积神经网络(convolutional neural networks,CNN)模块提取图像特征;然后,将特征输入到门控循环单元(gated recurrent unit,GRU)学习图像的时序连接性;最后,通过全连接层降维输出相机位姿。在KITTI数据集上完成实验,并与其他方法进行对比,结果表明卷积网络中加入注意力机制可以有效提高轨迹估计的精度,且误差低于其他视觉里程计算法。

基于YOLOv8改进的脑癌检测算法

自动检测脑部肿瘤在磁共振成像中的位置是一个复杂、繁重的任务,需要耗费大量时间和资源。传统识别方案经常出现误解、遗漏和误导的情况,从而影响患者的治疗进度,对患者的生命安全产生影响。为了进一步提高鉴定的效果,引入了4项关键改进措施。首先,采用了高效的多尺度注意力EMA(Efficient Multi-scale Attention),这种方法既可以对全局信息进行编码,也可以对信息进行重新校准,同时通过并行的分支输出特征进行跨维度的交互,使信息进一步聚合。其次,引入了BiFPN(Bidirectional Feature Pyramid Network)模块,并对其结构进行改进,以便缩短每一次检测所需要的时间,同时提升图像识别效果。然后采用MDPIoU损失函数和Mish激活函数进行改进,进一步提高检测的准确度。最后进行仿真实验,实验结果表明,改进的YOLOv8算法在脑癌检测中的精确率、召回率、平均精度均值均有提升,其中Precision提高了4.48%,Recall提高了2.64%,mAP@0.5提高了2.6%,mAP@0.5:0.9提高了7.0%。

基于二阶注意力的Siamese网络视觉跟踪算法

为提升基于Siamese网络视觉跟踪算法的特征表达能力和判别能力,以获得更好的跟踪性能,提出了一种轻量级的基于二阶注意力的Siamese网络视觉跟踪算法。使用轻量级VGG-Net作为Siamese网络的主干,获取目标的深度特征;在Siamese网络的末端并行使用所提残差二阶池化网络和二阶空间注意力网络,获取具有通道相关性的二阶注意力特征和具有空间相关性的二阶注意力特征;使用残差二阶通道注意力特征和二阶空间注意力特征,通过双分支响应策略实现视觉跟踪。利用GOT-10k数据集对所提算法进行端到端的训练,并在OTB100和VOT2018数据集上进行验证。实验结果表明:所提算法的跟踪性能取得了显著提升,与基准算法SiamFC相比,在OTB100数据集上,精度和成功率分别提高了0.100和0.096,在VOT2018数据集上,预期平均重叠率(EAO)提高了0.077,跟踪速度达到了48帧/s。

基于迁移学习和残差网络的SSVEP信号识别

针对脑电信号中的稳态视觉诱发电位(SSVEP)信号目标识别难以适应个体差异、识别稳定性差、精度低的难题,提出了一种参数共享迁移学习的残差网络SSVEP信号识别方法。首先,利用离散小波变换将多通道SSVEP信号转化为小波系数,并与变换前信号构成特征矩阵作为输入特征集,提升特征提取的丰富性;其次,建立融合空间注意力机制的残差网络,利用清华大学脑—机接口提供的两个SSVEP信号数据集,包括105名被试,进行跨任务的迁移训练,把源域上训练完成的网络逐模块迁移至目标网络以获取合适的迁移模块,迁移后连接2层残差块和模式识别单元得到跨个体差异识别结果。实验结果显示,在1s时间窗口,训练与测试使用被试无交集情况下,测试集的总识别率达到84.2%,提升了脑电信号识别的个体适应性,验证了提出的方法在提高SSVEP信号识别的稳健性和准确性上具有优势。

通道加权下的双判别GAN超分辨率网络

针对现有基于生成对抗网络的单图超分辨率重构方法特征利用率不足,生成图像包含少量无意义噪声的问题,提出一种基于通道注意力机制的双判别生成对抗网络。通过对生成网络中密集残差块进行通道加权,优化网络的特征利用率。同时在对抗网络中对生成图像进行像素域和特征域的双重判别,促使生成网络产生更丰富的结构特征和高频信息。实验结果表明,与现有的SRGAN、ESRGAN两种算法相比,该算法能够重构出感官质量更高的图像。

基于注意力机制的增强视觉追踪网络

为提升传统Transformer结构追踪器的性能并解决与注意力机制结合的问题,提出一种具有注意力机制的Transformer结构视觉跟踪器(称为EVOTA)。提出一个具有局部跨通道的交互策略的通道注意力模块,通过显式建模通道之间的相互依赖关系实现自适应校准通道方向的特征响应。受神经科学理论启发,提出一个能量函数分析神经网络中每个神经元的重要性,进一步推断其相应三维权重。在多个基准数据集上,EVOTA的性能优于多种优秀的追踪器。

基于MTF可视化和改进DenseNet神经网络的电能质量扰动识别算法

针对传统电能质量扰动(power quality disturbances,PQDs)分类器人工选取特征过程复杂、精细化程度不足的问题,提出一种基于马尔科夫迁移场(Markov translate filed,MTF)可视化和改进密集卷积网络(dense convolu-tional networks,DenseNet)的PQDs识别新方法。首先将一维PQD信号经MTF映射为二维图像,接着将图像输入到具有新型通道注意力机制的改进DenseNet中,最后训练网络自行从海量样本中提取特征,实现PQDs信号的正确识别。算例结果表明:在无噪声和信噪比为20、30 dB情况下,所提改进DenseNet能有效克服传统方法中主观性强、抗噪性能差等特征缺点,可以更好地提取复合PQD特征信息,对复合PQD识别率高。

基于马尔可夫转移场和轻量级网络的非侵入式负荷识别

负荷识别是非侵入式负荷监测(non-intrusive loadmonitoring,NILM)的关键一步。针对目前识别方法存在网络参数量大、识别率低的局限性,提出了一种基于马尔可夫转移场(Markov transition field,MTF)和轻量级网络的非侵入式负荷识别方法。首先,利用归一化后的电压电流计算马尔可夫状态转移矩阵,在时域上排列每个状态转移概率构建MTF。其次,对MTF降采样以适应神经网络的学习,利用伪彩色编码技术得到RGB彩色图像。最后,在轻量级网络Shuffle Net V2中加入SimAM无参注意力模块作为特征提取网络,以较少的参数量实现负荷分类识别。使用公共数据集PLAID和WHITED对所提方法进行实验,结果表明,SimAM-ShuffleNetV2在两个数据集的识别准确率分别达到了98.99%和99.22%,参数量分别为0.37 M和0.41 M,比现有的方法具有更高的识别准确率和更少的参数量,验证了所提方法的有效性和优越性。

基于抽象关系场景图的图像情感识别

图像情感识别是通过分析视觉刺激来预测人类情感的抽象过程。现有方法大多缺乏对对象间关系以及对象与场景间相互作用的关注,并且对象间复杂多样的关系难以得到充分利用,进而导致难以正确对图像情感进行预测。为解决上述问题,提出一种基于抽象关系场景图的图像情感识别方法。首先,构建对象和属性检测器来提取图像中对象及其属性的特征。其次,使用对象特征推理对象间的亲密度和抽象关系特征,进而构建抽象关系场景图。再次,提出抽象关系图卷积网络来推理抽象关系场景图。最后,设计渐进式注意力机制对多个对象特征进行融合,以得到图像的整体对象特征。在FI、EmotionRoI和Twitter I公开数据集上的试验结果表明,该方法的分类准确率优于现有方法。

基于多任务学习的复杂城市遥感图像道路提取

提出一种新的遥感图像道路提取框架,旨在利用从道路边缘检测中获得的知识来提高道路提取的准确性。研究中引入了一个融合多尺度信息和视觉注意力机制的多尺度视觉注意力模块,并构建了一个级联特征融合模块以集成网络在不同尺度上的预测结果。在此基础上,构建了一个包含编码器和解码器的多尺度视觉注意网络(MSVANet)。同时,提出一个多任务学习框架,该框架结合了MSVANet,并采用粒子群优化算法对多任务学习框架的两个学习率超参数的自动选取进行优化。RNBD数据集的训练和测试结果表明,所提方法在各种分割精度指标和泛化能力方面均优于其他道路提取方法。

基于双重注意力残差模块的低照度图像增强

低照度图像增强(Low Light Image Enhancement,LLIE)是将光照不足条件下获取的图像恢复成正常曝光的图像,基于深度学习的LLIE算法常用堆叠卷积或上/下采样的方式设计,这样缺少相关语义信息的指导,导致增强后的图像存在噪声增多、色彩失真、细节丢失等问题。为此,本文提出一种基于双重注意力残差模块的LLIE算法。该算法提出融合双重注意力单元的残差模块(Dual Attention Residual Block,DA-ResBlock),在通道域和空间域提供的语义信息引导下,通过多级串联的DA-ResBlock对有效特征进行稳定提取,并且使用跳跃链接与卷积神经网络来恢复图像细节信息。此外,使用复合损失函数对增强任务进行约束。最后,在2个真实图像的公共数据集上与近几年主流算法进行对比。实验结果表明,本文算法在主观视觉上在有效提高图像亮度的同时,更好地抑制了噪声、恢复了图像色彩与细节纹理,客观评价上在PSNR、SSIM、LPIPS这3个指标上均优于所对比的主流算法。