Unsupervised multi-modal image translation based on the squeeze-and-excitation mechanism and feature attention module

The unsupervised multi-modal image translation is an emerging domain of computer vision whose goal is to transform an image from the source domain into many diverse styles in the target domain.However,the multi-generator mechanism is employed among the advanced approaches available to model different domain mappings,which results in inefficient training of neural networks and pattern collapse,leading to inefficient generation of image diversity.To address this issue,this paper introduces a multi-modal unsupervised image translation framework that uses a generator to perform multi-modal image translation.Specifically,firstly,the domain code is introduced in this paper to explicitly control the different generation tasks.Secondly,this paper brings in the squeeze-and-excitation(SE)mechanism and feature attention(FA)module.Finally,the model integrates multiple optimization objectives to ensure efficient multi-modal translation.This paper performs qualitative and quantitative experiments on multiple non-paired benchmark image translation datasets while demonstrating the benefits of the proposed method over existing technologies.Overall,experimental results have shown that the proposed method is versatile and scalable.

AF-Net:A Medical Image Segmentation Network Based on Attention Mechanism and Feature Fusion

Medical image segmentation is an important application field of computer vision in medical image processing.Due to the close location and high similarity of different organs in medical images,the current segmentation algorithms have problems with mis-segmentation and poor edge segmentation.To address these challenges,we propose a medical image segmentation network(AF-Net)based on attention mechanism and feature fusion,which can effectively capture global information while focusing the network on the object area.In this approach,we add dual attention blocks(DA-block)to the backbone network,which comprises parallel channels and spatial attention branches,to adaptively calibrate and weigh features.Secondly,the multi-scale feature fusion block(MFF-block)is proposed to obtain feature maps of different receptive domains and get multi-scale information with less computational consumption.Finally,to restore the locations and shapes of organs,we adopt the global feature fusion blocks(GFF-block)to fuse high-level and low-level information,which can obtain accurate pixel positioning.We evaluate our method on multiple datasets(the aorta and lungs dataset),and the experimental results achieve 94.0%in mIoU and 96.3%in DICE,showing that our approach performs better than U-Net and other state-of-art methods.

Multiscale feature learning and attention mechanism for infrared and visible image fusion

Current fusion methods for infrared and visible images tend to extract features at a single scale,which results in insufficient detail and incomplete feature preservation.To address these issues,we propose an infrared and visible image fusion network based on a multiscale feature learning and attention mechanism(MsAFusion).A multiscale dilation convolution framework is employed to capture image features across various scales and broaden the perceptual scope.Furthermore,an attention network is introduced to enhance the focus on salient targets in infrared images and detailed textures in visible images.To compensate for information loss during convolution,jump connections are utilized during the image reconstruction phase.The fusion process utilizes a combined loss function consisting of pixel loss and gradient loss for unsupervised fusion of infrared and visible images.Extensive experiments on the dataset of electricity facilities demonstrate that our proposed method outperforms nine state-of-theart methods in terms of visual perception and four objective evaluation metrics.

Attention Guided Food Recognition via Multi-Stage Local Feature Fusion

The task of food image recognition,a nuanced subset of fine-grained image recognition,grapples with substantial intra-class variation and minimal inter-class differences.These challenges are compounded by the irregular and multi-scale nature of food images.Addressing these complexities,our study introduces an advanced model that leverages multiple attention mechanisms and multi-stage local fusion,grounded in the ConvNeXt architecture.Our model employs hybrid attention(HA)mechanisms to pinpoint critical discriminative regions within images,substantially mitigating the influence of background noise.Furthermore,it introduces a multi-stage local fusion(MSLF)module,fostering long-distance dependencies between feature maps at varying stages.This approach facilitates the assimilation of complementary features across scales,significantly bolstering the model’s capacity for feature extraction.Furthermore,we constructed a dataset named Roushi60,which consists of 60 different categories of common meat dishes.Empirical evaluation of the ETH Food-101,ChineseFoodNet,and Roushi60 datasets reveals that our model achieves recognition accuracies of 91.12%,82.86%,and 92.50%,respectively.These figures not only mark an improvement of 1.04%,3.42%,and 1.36%over the foundational ConvNeXt network but also surpass the performance of most contemporary food image recognition methods.Such advancements underscore the efficacy of our proposed model in navigating the intricate landscape of food image recognition,setting a new benchmark for the field.

Improved Blending Attention Mechanism in Visual Question Answering

Visual question answering(VQA)has attracted more and more attention in computer vision and natural language processing.Scholars are committed to studying how to better integrate image features and text features to achieve better results in VQA tasks.Analysis of all features may cause information redundancy and heavy computational burden.Attention mechanism is a wise way to solve this problem.However,using single attention mechanism may cause incomplete concern of features.This paper improves the attention mechanism method and proposes a hybrid attention mechanism that combines the spatial attention mechanism method and the channel attention mechanism method.In the case that the attention mechanism will cause the loss of the original features,a small portion of image features were added as compensation.For the attention mechanism of text features,a selfattention mechanism was introduced,and the internal structural features of sentences were strengthened to improve the overall model.The results show that attention mechanism and feature compensation add 6.1%accuracy to multimodal low-rank bilinear pooling network.

Video expression recognition based on frame-level attention mechanism

Facial expression recognition(FER) in video has attracted the increasing interest and many approaches have been made.The crucial problem of classifying a given video sequence into several basic emotions is how to fuse facial features of individual frames.In this paper, a frame-level attention module is integrated into an improved VGG-based frame work and a lightweight facial expression recognition method is proposed.The proposed network takes a sub video cut from an experimental video sequence as its input and generates a fixed-dimension representation.The VGG-based network with an enhanced branch embeds face images into feature vectors.The frame-level attention module learns weights which are used to adaptively aggregate the feature vectors to form a single discriminative video representation.Finally, a regression module outputs the classification results.The experimental results on CK+and AFEW databases show that the recognition rates of the proposed method can achieve the state-of-the-art performance.

Single-Image Dehazing Based on Two-Stream Convolutional Neural Network

The haze weather environment leads to the deterioration of the visual effect of the image,and it is difficult to carry out the work of the advanced vision task.Therefore,dehazing the haze image is an important step before the execution of the advanced vision task.Traditional dehazing algorithms achieve image dehazing by improving image brightness and contrast or constructing artificial priors such as color attenuation priors and dark channel priors.However,the effect is unstable when dealing with complex scenes.In the method based on convolutional neural network,the image dehazing network of the encoding and decoding structure does not consider the difference before and after the dehazing image,and the image spatial information is lost in the encoding stage.In order to overcome these problems,this paper proposes a novel end-to-end two-stream convolutional neural network for single-image dehazing.The network model is composed of a spatial information feature stream and a highlevel semantic feature stream.The spatial information feature stream retains the detailed information of the dehazing image,and the high-level semantic feature stream extracts the multi-scale structural features of the dehazing image.A spatial information auxiliary module is designed and placed between the feature streams.This module uses the attention mechanism to construct a unified expression of different types of information and realizes the gradual restoration of the clear image with the semantic information auxiliary spatial information in the dehazing network.A parallel residual twicing module is proposed,which performs dehazing on the difference information of features at different stages to improve the model’s ability to discriminate haze images.The peak signal-to-noise ratio(PSNR)and structural similarity are used to quantitatively evaluate the similarity between the dehazing results of each algorithm and the original image.The structure similarity and PSNR of the method in this paper reached 0.852 and 17.557dB on the HazeRD dataset,which were higher than existing comparison algorithms.On the SOTS dataset,the indicators are 0.955 and 27.348dB,which are sub-optimal results.In experiments with real haze images,this method can also achieve excellent visual restoration effects.The experimental results show that the model proposed in this paper can restore desired visual effects without fog images,and it also has good generalization performance in real haze scenes.

基于YOLOv5s和超声图像的儿童肠套叠特征检测模型

为帮助医生快速寻找到儿童腹部超声中肠套叠的病变特征并实现肠套叠超声诊后数据的快速质检,文章将目标检测算法应用于儿童腹部超声图像检测肠套叠“同心圆”征.首先探索了基于YOLOv5s的儿童肠套叠检测模型,发现该模型检测肠套叠“同心圆”征的精确度、召回率、F 1分数、mAP@0.5、FPS以及参数量等方面均优于Faster RCNN.进一步,为解决肉眼难以观察的“同心圆”征的检测问题,使用双向特征金字塔网络,并将注意力机制加入YOLOv5s网络,形成基于YOLOv5s_BiFPN_SE框架的儿童肠套叠“同心圆”征检测模型.该模型检测的精确率、召回率、F 1分数、mAP@0.5分别达到了91.33%、90.73%、91.03%、88.77%,性能更优于YOLOv5s.

基于细粒度特征的面料图像检索

面料图像检索对于纺织工厂面料库存和样品管理意义重大,但面料外观的多样性以及织物纹理的精细性,使得在面料检索时面料的特征提取较困难。该研究提出一种基于细粒度特征的面料图像检索算法。该算法使用坐标注意(coordinate attention,CA)模块来提取图像的精准位置信息,并将缩放系数法用于在宽度和高度方面整体缩放MobileNetV3的网络结构以减少模型参数数量,达到减少网络训练时间的目的。据此筛选出提取面料图像细粒度特征的最佳模型,在面料图像数据集(fabric image dataset,FID)上进行面料检索实验。结果表明,该算法有效提高了面料图像细粒度特征提取的准确性,检索精度达到91.82%,浮点运算数达到175.34 MB。检索精度比MobileNetV3原模型提高了13.49个百分点,同时减少了网络训练时间,速度提高了25.14%。该算法具有实际应用价值。

基于改进的YOLOv5安全帽佩戴检测算法

针对安全帽佩戴检测中存在的误检和漏检的问题,提出一种基于YOLOv5模型改进的安全帽佩戴检测算法。改进模型引入多尺度加权特征融合网络,即在YOLOv5的网络结构中增加一个浅层检测尺度,并引入特征权重进行加权融合,构成新的四尺检测结构,有效地提升图像浅层特征的提取及融合能力;在YOLOv5的Neck网络的BottleneckCSP结构中加入SENet模块,使模型更多地关注目标信息忽略背景信息;针对大分辨率的图像,添加图像切割层,避免多倍下采样造成的小目标特征信息大量丢失。对YOLOv5模型进行改进之后,通过自制的安全帽数据集进行训练检测,mAP和召回率分别达到97.06%、92.54%,与YOLOv5相比较分别提升了4.74%和4.31%。实验结果表明:改进的YOLOv5算法可有效提升安全帽佩戴的检测性能,能够准确识别施工人员的安全帽佩戴情况,从而大大降低施工现场的安全风险。

基于多分支空谱特征增强的高光谱图像分类

为了解决高光谱图像自身及分类过程中噪声干扰大、空间-光谱特征信息提取不足以及有限样本下分类性能不佳等问题,提出一种基于多分支空谱特征增强的高光谱图像分类模型SSFE-MBACNN。首先,利用多分支特征提取模块分别提取浅层空谱特征和深层空间特征信息,并引入注意力机制抑制噪声干扰。其次,设计一种改进多尺度空谱特征提取融合模块及结合双池化和空洞卷积的空间特征增强模块实现空谱特征增强,减少模型参数量和提高分类性能。最后,用全局平均池化层代替全连接层,进一步降低参数量,缓解模型过拟合问题。实验结果表明,在Indian Pines(10%训练样本)、Pavia University (5%训练样本)和Salinas(1%训练样本)数据集分别取得了0.990 7、0.997 5和0.994 7的总体分类精度。SSFE-MBACNN不仅能充分利用空谱特征信息,而且在有限样本下也取得了优秀的分类性能,明显高于其他对比方法。

基于多尺度残差注意力网络的水下图像增强

针对水下图像由水的散射、吸收引起的色偏、色弱、信息丢失问题,提出了一种基于多尺度残差注意力网络的水下图像增强算法。该网络引入了改进的UNet3+-Avg结构与注意力机制,设计出多尺度密集特征提取模块与残差注意力恢复模块,以及由Charbonnier损失和边缘损失相结合的联合损失函数,使该网络得以学习到多个尺度的丰富特征,在改善图像色彩的同时又可保留大量的物体边缘信息。增强后图像的平均峰值信噪比(PSNR)达到23.63 dB、结构相似度(SSIM)达到0.93。与其他水下图像增强网络的对比实验结果表明,由该网络所增强的图像在主观感受与客观评价上都取得了显著的效果。

基于注意力机制与光照感知网络的红外与可见光图像融合

部分图像融合方法未充分考虑图像环境的光照状况,导致融合图像中出现红外目标亮度不足以及整体画面亮度较低,从而影响纹理细节的清晰度.为解决上述问题,提出一种基于注意力机制与光照感知网络相结合的红外与可见光图像融合算法.首先,在训练融合网络之前利用光照感知网络计算当前场景是日间或夜间的概率,将其运用至融合网络损失函数中,用以指导融合网络训练;然后,在网络的特征提取部分采用空间注意力机制和深度可分离卷积对源图像进行特征提取,得到空间显著信息后,输入卷积神经网络(CNN)以提取深度特征;最后,将深度特征信息进行拼接用于图像重建,进而得到最终的融合图像.实验结果表明:本文方法的互信息(MI)、视觉保真度(VIF)、平均梯度(AG)、融合质量(Qabf)与空间频率(SF)较对比方法分别平均提高39.33%、11.29%、26.27%、47.11%和39.01%;融合后的图像能够有效保留红外目标亮度,且包含丰富的纹理细节信息.

改进YOLOv5的无人机航拍图像目标检测算法

针对无人机航拍图像目标检测中目标尺度多样、相似目标众多、目标聚集导致的目标漏检、误检问题,提出了改进YOLOv5的无人机航拍图像目标检测算法DA-YOLO。提出由特征图注意力生成器和动态权重学习模块组成的多尺度动态特征加权融合网络,特征图注意力生成器融合处理不同尺度目标更重要的特征,权重学习模块自适应地调节对不同尺度目标特征的学习,该网络可增强在目标尺度多样下的辨识度从而降低目标漏检。设计一种并行选择性注意力机制(PSAM)添加到特征提取网络中,该模块通过动态融合空间信息和通道信息,加强特征的表达获得更优质的特征图,提高网络对相似目标的区分能力以减少误检。使用Soft-NMS代替YOLOv5中采用的非极大值抑制(NMS)以改善目标聚集场景下的漏检、误检。实验结果表明,改进算法在VisDrone数据集上检测精度达到37.79%,相比于YOLOv5s算法精度提高了5.59个百分点,改进后的算法可以更好地应用于无人机航拍图像目标检测中。

结合对抗训练和特征混合的孪生网络防御模型

神经网络模型容易受到对抗样本攻击。针对当前防御方法侧重改进模型结构或模型仅使用对抗训练方法导致防御类型单一且损害模型分类能力、效率低下的问题,提出结合对抗训练和特征混合训练孪生神经网络模型(SS-ResNet18)的方法。该方法通过线性插值混合训练集样本数据,使用残差注意力模块搭建孪生网络模型,将PGD对抗样本和正常样本输入不同分支网络进行训练。在特征空间互换相邻样本部分输入特征以增强网络抗干扰能力,结合对抗损失和分类损失作为网络整体损失函数并对其进行标签平滑。在CIFAR-10和SVHN数据集上进行实验,该方法在白盒攻击下表现出优异的防御性能,黑盒攻击下模型对PGD、JSMA等对抗样本的防御成功率均在80%以上;同时,SS-ResNet18模型时间花销仅为子空间对抗训练方法的二分之一。实验结果表明,SS-ResNet18模型能防御多种对抗样本攻击,与现有防御方法相比,其鲁棒性强且训练耗时较短。

纺织品车缝线迹分割网络

针对织物车缝线缝制工艺多样,造成其种类多、形态不定以及缝线与织物纹理近似性等导致车缝线准确分割困难等问题,提出一个基于多尺度特征的纺织品车缝线迹分割网络.首先采用融合注意力机制的残差网络提取其位置信息;然后使用增强特征金字塔模块,充分利用多尺度特征得到预候选区域的语义信息,将其融合后经过筛选得到车缝线候选区域;最后经过全卷积网络实现车缝线的分割.在真实纺织品车缝线数据集SewTrace上进行实验的结果表明,所提网络对纺织品车缝线迹分割的均值平均精度为0.96,计算量为1.5G;在具有相似特征的公开数据集CrackForest,CRKWH100和Kolektor上与其他同类网络进行实验的结果表明,该网络的均值平均精度分别达到0.85,0.89和0.89,分割精度和预测速度指标优于其他同类网络,证明其能够有效地提高线形目标分割精度.

局部注意力引导下的全局池化残差分类网络

大部分注意力机制虽然能增强图像特征,但没有考虑局部特征的关联性影响特征整体的问题。针对以上问题,本文提出局部注意力引导下的全局池化残差分类网络(MSLENet)。MSLENet的基线网络为ResNet34,首先改变首层结构,保留图像重要信息;其次提出多分割局部增强注意力机制(MSLE)模块,MSLE模块将图像整体分割成多个小图像,增强每个小图像的局部特征,通过特征组交互的方式将局部重要特征引导到全局特征中;最后提出池化残差(PR)模块来处理ResNet残差结构丢失信息的问题,提高各层之间的信息利用率。实验结果表明,MSLENet通过增强局部特征的关联性,在多个数据集上均有良好的效果,有效地提高了网络的表达能力。

一种基于航拍红外图像的光伏热斑故障分类检测方法

针对航拍光伏红外图像热斑检测方法中小目标特征易丢失问题,提出一种光伏热斑故障分类检测方法。首先将多头自注意力机制结合CSPNet结构进行改进,提出CSPMAT网络,再将其引入New CSP-Darknet网络,构建CSPMAT-Darknet模型,实现了光伏组件热斑定位及分类。实验结果表明:该模型在小目标检测任务中的性能显著提升,且在目标尺寸差异较大的故障分类检测任务中,均值平均精度达到82.92%,提高了13.97个百分点,具有良好的检测精度和泛化能力。

注意力机制的混合卷积高光谱图像分类方法

高光谱图像以其高分辨率的空间和光谱信息在军事、航空航天及民用等遥感领域均有重要应用,具有重要的研究意义。深度学习具有学习能力强、覆盖范围广及可移植性强的优势,成为目前高精度高光谱图像分类技术研究的热点。其中卷积神经网络(CNN)因强大的特征提取能力广泛应用于高光谱图像分类方法研究中,取得了有效的研究成果,但该类方法通常单独基于2D-CNN或3D-CNN进行,针对高光谱图像的单一特征,一是不能充分利用高光谱数据本身完整的特征信息;二是虽然相应提取网络局部特征优化性好,但是整体泛化能力不足,在深度挖掘HSI的空间和光谱信息方面存在局限性。鉴于此,提出了基于注意力机制的混合卷积神经网络模型(HybridSN_AM),使用主成分分析法对高光谱图像进行降维,采用卷积神经网络作为分类模型的主体,通过注意力机制筛选出更有区分度的特征,使模型能够提取到更精确、更核心的空间-光谱信息,实现高光谱图像的高精度分类。对Indian Pines(IP)、University of Pavia(UP)和Salinas(SA)三个数据集进行了应用实验,结果表明,基于该模型的目标图像总体分类精度、平均分类精度和Kappa系数均高于98.14%、97.17%、97.87%。与常规HybridSN模型对比表明,HybridSN_AM模型在三个数据集上的分类精度分别提升了0.89%、0.07%和0.73%。有效解决了高光谱图像空间-光谱特征提取与融合的难题,提高HSI分类的精度,具有较强的泛化能力,充分验证了注意力机制结合混合卷积神经网络在高光谱图像分类中的有效性和可行性,对高光谱图像分类技术的发展及应用具有重要的科学价值。

基于注意力特征增强的投影图像光度补偿方法

目前,投影补偿算法已经取得了良好的研究成果,但大部分的投影图像色彩补偿研究,忽略了色彩传递函数建模过程中的光学部分,导致对色彩传递函数的建模精确度不高。同时,针对投影图像色彩补偿过程中,出现的加深网络导致提取特征信息丢失的现象,大多数的深度学习网络优化设计较少。基于以上问题,本文提出了一种基于注意力特征增强的投影图像光度补偿方法。该方法通过增加网络深度来提取带有彩色纹理投影表面的特征信息,同时采用深度学习来拟合复杂的复合辐射传递函数,以解决传统光度补偿方法存在的问题,提高了投影图像的质量和色彩,进一步消除了对高质量投影幕的依赖。本文所提方法的投影图像光度补偿结果,在峰值信噪比(PSNR)、均方根误差(RMSE)和结构相似性(SSIM)三项评价指标上,均好于其他对比算法,与CompenNet系列方法相比,本文所提方法在PSNR评价指标上最高提升5717、在RMSE评价指标上最高减少14968,在SSIM评价指标上最高提升2893。