A Lightweight Convolutional Neural Network with Hierarchical Multi-Scale Feature Fusion for Image Classification

Convolutional neural networks (CNNs) are widely used in image classification tasks, but their increasing model size and computation make them challenging to implement on embedded systems with constrained hardware resources. To address this issue, the MobileNetV1 network was developed, which employs depthwise convolution to reduce network complexity. MobileNetV1 employs a stride of 2 in several convolutional layers to decrease the spatial resolution of feature maps, thereby lowering computational costs. However, this stride setting can lead to a loss of spatial information, particularly affecting the detection and representation of smaller objects or finer details in images. To maintain the trade-off between complexity and model performance, a lightweight convolutional neural network with hierarchical multi-scale feature fusion based on the MobileNetV1 network is proposed. The network consists of two main subnetworks. The first subnetwork uses a depthwise dilated separable convolution (DDSC) layer to learn imaging features with fewer parameters, which results in a lightweight and computationally inexpensive network. Furthermore, depthwise dilated convolution in DDSC layer effectively expands the field of view of filters, allowing them to incorporate a larger context. The second subnetwork is a hierarchical multi-scale feature fusion (HMFF) module that uses parallel multi-resolution branches architecture to process the input feature map in order to extract the multi-scale feature information of the input image. Experimental results on the CIFAR-10, Malaria, and KvasirV1 datasets demonstrate that the proposed method is efficient, reducing the network parameters and computational cost by 65.02% and 39.78%, respectively, while maintaining the network performance compared to the MobileNetV1 baseline.

Attention Guided Multi Scale Feature Fusion Network for Automatic Prostate Segmentation

The precise and automatic segmentation of prostate magnetic resonance imaging(MRI)images is vital for assisting doctors in diagnosing prostate diseases.In recent years,many advanced methods have been applied to prostate segmentation,but due to the variability caused by prostate diseases,automatic segmentation of the prostate presents significant challenges.In this paper,we propose an attention-guided multi-scale feature fusion network(AGMSF-Net)to segment prostate MRI images.We propose an attention mechanism for extracting multi-scale features,and introduce a 3D transformer module to enhance global feature representation by adding it during the transition phase from encoder to decoder.In the decoder stage,a feature fusion module is proposed to obtain global context information.We evaluate our model on MRI images of the prostate acquired from a local hospital.The relative volume difference(RVD)and dice similarity coefficient(DSC)between the results of automatic prostate segmentation and ground truth were 1.21%and 93.68%,respectively.To quantitatively evaluate prostate volume on MRI,which is of significant clinical significance,we propose a unique AGMSF-Net.The essential performance evaluation and validation experiments have demonstrated the effectiveness of our method in automatic prostate segmentation.

Ship recognition based on HRRP via multi-scale sparse preserving method

In order to extract the richer feature information of ship targets from sea clutter, and address the high dimensional data problem, a method termed as multi-scale fusion kernel sparse preserving projection(MSFKSPP) based on the maximum margin criterion(MMC) is proposed for recognizing the class of ship targets utilizing the high-resolution range profile(HRRP). Multi-scale fusion is introduced to capture the local and detailed information in small-scale features, and the global and contour information in large-scale features, offering help to extract the edge information from sea clutter and further improving the target recognition accuracy. The proposed method can maximally preserve the multi-scale fusion sparse of data and maximize the class separability in the reduced dimensionality by reproducing kernel Hilbert space. Experimental results on the measured radar data show that the proposed method can effectively extract the features of ship target from sea clutter, further reduce the feature dimensionality, and improve target recognition performance.

Feature Fusion-Based Deep Learning Network to Recognize Table Tennis Actions

A system for classifying four basic table tennis strokes using wearable devices and deep learning networks is proposed in this study.The wearable device consisted of a six-axis sensor,Raspberry Pi 3,and a power bank.Multiple kernel sizes were used in convolutional neural network(CNN)to evaluate their performance for extracting features.Moreover,a multiscale CNN with two kernel sizes was used to perform feature fusion at different scales in a concatenated manner.The CNN achieved recognition of the four table tennis strokes.Experimental data were obtained from20 research participants who wore sensors on the back of their hands while performing the four table tennis strokes in a laboratory environment.The data were collected to verify the performance of the proposed models for wearable devices.Finally,the sensor and multi-scale CNN designed in this study achieved accuracy and F1 scores of 99.58%and 99.16%,respectively,for the four strokes.The accuracy for five-fold cross validation was 99.87%.This result also shows that the multi-scale convolutional neural network has better robustness after fivefold cross validation.

Grasp Detection with Hierarchical Multi-Scale Feature Fusion and Inverted Shuffle Residual

Grasp detection plays a critical role for robot manipulation.Mainstream pixel-wise grasp detection networks with encoder-decoder structure receive much attention due to good accuracy and efficiency.However,they usually transmit the high-level feature in the encoder to the decoder,and low-level features are neglected.It is noted that low-level features contain abundant detail information,and how to fully exploit low-level features remains unsolved.Meanwhile,the channel information in high-level feature is also not well mined.Inevitably,the performance of grasp detection is degraded.To solve these problems,we propose a grasp detection network with hierarchical multi-scale feature fusion and inverted shuffle residual.Both low-level and high-level features in the encoder are firstly fused by the designed skip connections with attention module,and the fused information is then propagated to corresponding layers of the decoder for in-depth feature fusion.Such a hierarchical fusion guarantees the quality of grasp prediction.Furthermore,an inverted shuffle residual module is created,where the high-level feature from encoder is split in channel and the resultant split features are processed in their respective branches.By such differentiation processing,more high-dimensional channel information is kept,which enhances the representation ability of the network.Besides,an information enhancement module is added before the encoder to reinforce input information.The proposed method attains 98.9%and 97.8%in image-wise and object-wise accuracy on the Cornell grasping dataset,respectively,and the experimental results verify the effectiveness of the method.

Multi-Scale Feature Fusion Model for Bridge Appearance Defect Detection

Although the Faster Region-based Convolutional Neural Network(Faster R-CNN)model has obvious advantages in defect recognition,it still cannot overcome challenging problems,such as time-consuming,small targets,irregular shapes,and strong noise interference in bridge defect detection.To deal with these issues,this paper proposes a novel Multi-scale Feature Fusion(MFF)model for bridge appearance disease detection.First,the Faster R-CNN model adopts Region Of Interest(ROl)pooling,which omits the edge information of the target area,resulting in some missed detections and inaccuracies in both detecting and localizing bridge defects.Therefore,this paper proposes an MFF based on regional feature Aggregation(MFF-A),which reduces the missed detection rate of bridge defect detection and improves the positioning accuracy of the target area.Second,the Faster R-CNN model is insensitive to small targets,irregular shapes,and strong noises in bridge defect detection,which results in a long training time and low recognition accuracy.Accordingly,a novel Lightweight MFF(namely MFF-L)model for bridge appearance defect detection using a lightweight network EfficientNetV2 and a feature pyramid network is proposed,which fuses multi-scale features to shorten the training speed and improve recognition accuracy.Finally,the effectiveness of the proposed method is evaluated on the bridge disease dataset and public computational fluid dynamic dataset.

Vehicle color recognition based on smooth modulation neural network with multi-scale feature fusion

Vehicle Color Recognition(VCR)plays a vital role in intelligent traffic management and criminal investigation assistance.However,the existing vehicle color datasets only cover 13 classes,which can not meet the current actual demand.Besides,although lots of efforts are devoted to VCR,they suffer from the problem of class imbalance in datasets.To address these challenges,in this paper,we propose a novel VCR method based on Smooth Modulation Neural Network with Multi-Scale Feature Fusion(SMNN-MSFF).Specifically,to construct the benchmark of model training and evaluation,we first present a new VCR dataset with 24 vehicle classes,Vehicle Color-24,consisting of 10091 vehicle images from a 100-hour urban road surveillance video.Then,to tackle the problem of long-tail distribution and improve the recognition performance,we propose the SMNN-MSFF model with multiscale feature fusion and smooth modulation.The former aims to extract feature information from local to global,and the latter could increase the loss of the images of tail class instances for training with class-imbalance.Finally,comprehensive experimental evaluation on Vehicle Color-24 and previously three representative datasets demonstrate that our proposed SMNN-MSFF outperformed state-of-the-art VCR methods.And extensive ablation studies also demonstrate that each module of our method is effective,especially,the smooth modulation efficiently help feature learning of the minority or tail classes.Vehicle Color-24 and the code of SMNN-MSFF are publicly available and can contact the author to obtain.

Bidirectional parallel multi-branch convolution feature pyramid network for target detection in aerial images of swarm UAVs

In this paper,based on a bidirectional parallel multi-branch feature pyramid network(BPMFPN),a novel one-stage object detector called BPMFPN Det is proposed for real-time detection of ground multi-scale targets by swarm unmanned aerial vehicles(UAVs).First,the bidirectional parallel multi-branch convolution modules are used to construct the feature pyramid to enhance the feature expression abilities of different scale feature layers.Next,the feature pyramid is integrated into the single-stage object detection framework to ensure real-time performance.In order to validate the effectiveness of the proposed algorithm,experiments are conducted on four datasets.For the PASCAL VOC dataset,the proposed algorithm achieves the mean average precision(mAP)of 85.4 on the VOC 2007 test set.With regard to the detection in optical remote sensing(DIOR)dataset,the proposed algorithm achieves 73.9 mAP.For vehicle detection in aerial imagery(VEDAI)dataset,the detection accuracy of small land vehicle(slv)targets reaches 97.4 mAP.For unmanned aerial vehicle detection and tracking(UAVDT)dataset,the proposed BPMFPN Det achieves the mAP of 48.75.Compared with the previous state-of-the-art methods,the results obtained by the proposed algorithm are more competitive.The experimental results demonstrate that the proposed algorithm can effectively solve the problem of real-time detection of ground multi-scale targets in aerial images of swarm UAVs.

Face anti-spoofing based on multi-modal and multi-scale features fusion

Face anti-spoofing is used to assist face recognition system to judge whether the detected face is real face or fake face. In the traditional face anti-spoofing methods, features extracted by hand are used to describe the difference between living face and fraudulent face. But these handmade features do not apply to different variations in an unconstrained environment. The convolutional neural network(CNN) for face deceptions achieves considerable results. However, most existing neural network-based methods simply use neural networks to extract single-scale features from single-modal data, while ignoring multi-scale and multi-modal information. To address this problem, a novel face anti-spoofing method based on multi-modal and multi-scale features fusion(MMFF) is proposed. Specifically, first residual network(Resnet)-34 is adopted to extract features of different scales from each modality, then these features of different scales are fused by feature pyramid network(FPN), finally squeeze-and-excitation fusion(SEF) module and self-attention network(SAN) are combined to fuse features from different modalities for classification. Experiments on the CASIA-SURF dataset show that the new method based on MMFF achieves better performance compared with most existing methods.

基于多特征融合自动编码器的增量式入侵检测

针对增量式入侵检测算法由于对旧知识产生灾难性遗忘而导致对旧类别数据分类准确率不高的问题,本文提出了一种基于非对称式多特征融合自动编码器(asymmetric multi-feature fusion auto-encoder, AMAE)和全连接分类神经网络(classification deep neural network, C-DNN)的增量式入侵检测算法(ImFace).在增量学习阶段, ImFace会为每一批新的数据集训练一个AMAE模型和C-DNN模型.同时,本文通过使用变分自动编码器(variational auto-encoder, VAE)对数据进行过采样的方式来改善由于数据集不平衡而导致C-DNN对某些类别数据的检测能力不足的问题.在检测阶段, ImFace将输入数据经过所有AMAE和C-DNN,然后将AMAE的结果作为置信度来选择某一个C-DNN的输出结果作为最终结果.本文使用CICIDS2017数据集来检验ImFace算法的有效性.实验结果表明, ImFace算法不仅能够保留对旧类别的分类能力,同时对新类别的数据也有很高的检测准确率.

基于改进ShuffleNet v1的服装图像分类算法

针对服装图像分类模型体积较大,缺少细分类的问题,提出基于改进ShuffleNet v1的服装图像分类算法。该算法以ShuffleNet v1为基础,通过优化模块的堆叠次数和网络层通道数来降低模型的计算量,满足算法的实时性要求;嵌入通道和空间注意力模块,使得模型关注重要的特征信息,抑制无用的特征信息;设计非对称多尺度特征融合模块,加强模型的特征提取能力。结果表明:所提算法在自建的衬衫服装数据集中准确率为88.31%,分别高于ShuffleNet v1、ShuffleNet v2、MobileNet v2和ResNet50模型2.77%、3.69%、1.98%、0.62%;所提算法在DeepFashion的部分数据集中也取得了不错的效果,验证了所提算法的有效性与通用性;与基础模型相比,所提模型的参数量仅为0.73M,模型参数量减少了约60%,实现了模型准确率和推理速度的提升。

基于残差学习的非对称卷积神经网络图像去噪方法

为了获得更清晰的图片,更好地去除图像中的噪声,对目前去噪效果较好的基于残差学习技术的深度卷积神经网络去噪算法(DnCNN)进行改进,提出了DnACNN模型,该模型通过将常规的方形卷积核替换为一组非对称卷积核来增强方形卷积核的骨架部分,从而提高特征提取精度和性能;并在测试阶段进行权值融合,从而减少参数、简化模型,确保不增加任何额外的推理时间。实验表明该方法在不增加任何额外推理时间的前提下能更有效地去除图像中的噪声,与目前主流去噪方法相比,能更好地捕获特征,提升特征丰富性,获得了更高的峰值信噪比,且能够更多地保留细节纹理。

面向遥感图像的多阶段特征融合目标检测方法

遥感图像目标具有多尺度、大横纵比、多角度等特性,给传统的目标检测方法带来了新的挑战.针对现有方法应用于目标尺度小、横纵比例不均衡的遥感图像时存在的精度下降问题,提出一种基于多阶段特征融合的目标检测方法MF2M(Multi-stage Feature Fusion Method).该方法在一阶段对特征图通道进行组合拆分,再采用卷积拼接的融合方式聚合通道维度的特征,从而强化输出的目标空间轮廓信息;二阶段设计多比例的非对称卷积块,增强大横纵比目标的高维全局特征,改善目标与检测框匹配粗糙的问题,同时利用串并行相结合的处理方式减少冗余卷积参数,加速网络收敛.在DOTA(Dataset for Object deTection in Aerial images)数据集上的实验结果表明,基准方法引入MF2M后,在保证检测速度的前提下精度指标mAP提高至76.44%,结果验证了所提算法的有效性与可靠性.

基于注意力机制和不对称卷积的目标跟踪算法

一般孪生网络跟踪算法中目标模板不会更新,模板分支与搜索分支在计算时相互独立,无法进行鲁棒跟踪,使用深度互相关来融合两分支的特征有着容易被干扰物欺骗、激活通道数少、对目标边界的分辨能力较弱,且不能充分受益于大规模的离线训练,为此提出一种基于注意力机制和不对称卷积的目标跟踪算法。设计增强注意力网络增强和传递分支信息。采用不对称卷积来代替深度互相关,使用有效的参数学习如何更好地互相关。所提算法在OTB100、LaSOT、VOT2019上做了对比实验,实验结果表明,所提算法表现较好,性能优于现有的多个先进跟踪器。

Multi-source Remote Sensing Image Registration Based on Contourlet Transform and Multiple Feature Fusion

Image registration is an indispensable component in multi-source remote sensing image processing. In this paper, we put forward a remote sensing image registration method by including an improved multi-scale and multi-direction Harris algorithm and a novel compound feature. Multi-scale circle Gaussian combined invariant moments and multi-direction gray level co-occurrence matrix are extracted as features for image matching. The proposed algorithm is evaluated on numerous multi-source remote sensor images with noise and illumination changes. Extensive experimental studies prove that our proposed method is capable of receiving stable and even distribution of key points as well as obtaining robust and accurate correspondence matches. It is a promising scheme in multi-source remote sensing image registration.

Modulation recognition network of multi-scale analysis with deep threshold noise elimination

To improve the accuracy of modulated signal recognition in variable environments and reduce the impact of factors such as lack of prior knowledge on recognition results,researchers have gradually adopted deep learning techniques to replace traditional modulated signal processing techniques.To address the problem of low recognition accuracy of the modulated signal at low signal-to-noise ratios,we have designed a novel modulation recognition network of multi-scale analysis with deep threshold noise elimination to recognize the actually collected modulated signals under a symmetric cross-entropy function of label smoothing.The network consists of a denoising encoder with deep adaptive threshold learning and a decoder with multi-scale feature fusion.The two modules are skip-connected to work together to improve the robustness of the overall network.Experimental results show that this method has better recognition accuracy at low signal-to-noise ratios than previous methods.The network demonstrates a flexible self-learning capability for different noise thresholds and the effectiveness of the designed feature fusion module in multi-scale feature acquisition for various modulation types.

基于非对称增强注意力与特征交叉融合的行人重识别方法

针对现有的行人重识别方法提取到的特征信息充分性与辨识性不足导致检索精度低的问题,提出一种基于非对称增强注意力与特征交叉融合的行人重识别方法。首先,构建非对称增强注意力模块,通过多重池化聚合的跨邻域通道交互注意力增强显著特征表示,使网络聚焦于图像中的行人区域;其次,考虑到网络各层特征间的差异性与关联性,构建特征交叉融合模块,利用交叉融合方式实现同层不同级特征的跨层级融合,进而实现多尺度融合;最后,水平切分输出特征以获取局部特征,从而实现在特定区域上描述行人。在Market1501、DukeMTMC-reID与CUHK03这3个公开数据集上对提出的方法进行了验证,首位命中率(Rank-1)分别达到了93.5%、85.1%和64.3%,证明了该方法在提升行人重识别性能上具有优越性。

融合低层和高层特征图表示的图像显著性检测算法

为了有效地利用不同层次特征的互补性,提高鲁棒性,提出一种融合低层和高层特征的图表示的图像显著性算法.首先以超像素为结点构图,通过高层特征和底层特征差异定义该图的点和边的权重;然后根据该图模型构造不对称转移概率矩阵,并利用Markov随机游走算法进行求解,得到初始显著性图;最后结合中心先验及改进的边界先验得到最终的图像显著性结果.在4个公共数据集上与10种方法进行比较与分析,验证了该算法的有效性.

一种基于非对称决策融合的行人检测方法

行人检测一直是计算机视觉中的热门话题,其重要应用包括视觉监视、机器人技术和汽车驾驶等。而当行人暂时保持静止时,运动信息将消失,运动特征与外观特征均不稳定,故一般特征融合方法在该情况中无法获得良好效果。针对于此类问题,论文提出了一种非对称特征融合方法来处理此类不稳定信息源的融合问题,经实验表明,该方法具有较好的检测效果。

基于小波域和时域的视频质量评价

目前多数视频质量评价算法将视频所有帧的图像质量平均值作为整个视频的质量,但该方式只考虑空间图像质量,忽略视频固有时域上的特性,因而无法准确地描述客观视频质量评价和主观评价的相关性。为此,结合视频的时域特性,提出一种改进的视频质量评价算法。该算法将视频帧图像分为边缘区域与平滑区域,分别对2个区域进行小波变换,并利用小波系数求得各个区域视频帧的图像质量度量值,进行加权后得出视频单帧图像质量的度量值,对连续单帧图像进行时域融合,从而求得整个视频的质量度量值。在LIVE视频数据库上的实验结果表明,该算法与人类主观评价结果具有较好的一致性,斯皮尔曼相关系数达到0.788 5。