A Lightweight Convolutional Neural Network with Hierarchical Multi-Scale Feature Fusion for Image Classification

Convolutional neural networks (CNNs) are widely used in image classification tasks, but their increasing model size and computation make them challenging to implement on embedded systems with constrained hardware resources. To address this issue, the MobileNetV1 network was developed, which employs depthwise convolution to reduce network complexity. MobileNetV1 employs a stride of 2 in several convolutional layers to decrease the spatial resolution of feature maps, thereby lowering computational costs. However, this stride setting can lead to a loss of spatial information, particularly affecting the detection and representation of smaller objects or finer details in images. To maintain the trade-off between complexity and model performance, a lightweight convolutional neural network with hierarchical multi-scale feature fusion based on the MobileNetV1 network is proposed. The network consists of two main subnetworks. The first subnetwork uses a depthwise dilated separable convolution (DDSC) layer to learn imaging features with fewer parameters, which results in a lightweight and computationally inexpensive network. Furthermore, depthwise dilated convolution in DDSC layer effectively expands the field of view of filters, allowing them to incorporate a larger context. The second subnetwork is a hierarchical multi-scale feature fusion (HMFF) module that uses parallel multi-resolution branches architecture to process the input feature map in order to extract the multi-scale feature information of the input image. Experimental results on the CIFAR-10, Malaria, and KvasirV1 datasets demonstrate that the proposed method is efficient, reducing the network parameters and computational cost by 65.02% and 39.78%, respectively, while maintaining the network performance compared to the MobileNetV1 baseline.

Attention Guided Multi Scale Feature Fusion Network for Automatic Prostate Segmentation

The precise and automatic segmentation of prostate magnetic resonance imaging(MRI)images is vital for assisting doctors in diagnosing prostate diseases.In recent years,many advanced methods have been applied to prostate segmentation,but due to the variability caused by prostate diseases,automatic segmentation of the prostate presents significant challenges.In this paper,we propose an attention-guided multi-scale feature fusion network(AGMSF-Net)to segment prostate MRI images.We propose an attention mechanism for extracting multi-scale features,and introduce a 3D transformer module to enhance global feature representation by adding it during the transition phase from encoder to decoder.In the decoder stage,a feature fusion module is proposed to obtain global context information.We evaluate our model on MRI images of the prostate acquired from a local hospital.The relative volume difference(RVD)and dice similarity coefficient(DSC)between the results of automatic prostate segmentation and ground truth were 1.21%and 93.68%,respectively.To quantitatively evaluate prostate volume on MRI,which is of significant clinical significance,we propose a unique AGMSF-Net.The essential performance evaluation and validation experiments have demonstrated the effectiveness of our method in automatic prostate segmentation.

Ship recognition based on HRRP via multi-scale sparse preserving method

In order to extract the richer feature information of ship targets from sea clutter, and address the high dimensional data problem, a method termed as multi-scale fusion kernel sparse preserving projection(MSFKSPP) based on the maximum margin criterion(MMC) is proposed for recognizing the class of ship targets utilizing the high-resolution range profile(HRRP). Multi-scale fusion is introduced to capture the local and detailed information in small-scale features, and the global and contour information in large-scale features, offering help to extract the edge information from sea clutter and further improving the target recognition accuracy. The proposed method can maximally preserve the multi-scale fusion sparse of data and maximize the class separability in the reduced dimensionality by reproducing kernel Hilbert space. Experimental results on the measured radar data show that the proposed method can effectively extract the features of ship target from sea clutter, further reduce the feature dimensionality, and improve target recognition performance.

Feature Fusion-Based Deep Learning Network to Recognize Table Tennis Actions

A system for classifying four basic table tennis strokes using wearable devices and deep learning networks is proposed in this study.The wearable device consisted of a six-axis sensor,Raspberry Pi 3,and a power bank.Multiple kernel sizes were used in convolutional neural network(CNN)to evaluate their performance for extracting features.Moreover,a multiscale CNN with two kernel sizes was used to perform feature fusion at different scales in a concatenated manner.The CNN achieved recognition of the four table tennis strokes.Experimental data were obtained from20 research participants who wore sensors on the back of their hands while performing the four table tennis strokes in a laboratory environment.The data were collected to verify the performance of the proposed models for wearable devices.Finally,the sensor and multi-scale CNN designed in this study achieved accuracy and F1 scores of 99.58%and 99.16%,respectively,for the four strokes.The accuracy for five-fold cross validation was 99.87%.This result also shows that the multi-scale convolutional neural network has better robustness after fivefold cross validation.

Multi-Scale Feature Fusion Model for Bridge Appearance Defect Detection

Although the Faster Region-based Convolutional Neural Network(Faster R-CNN)model has obvious advantages in defect recognition,it still cannot overcome challenging problems,such as time-consuming,small targets,irregular shapes,and strong noise interference in bridge defect detection.To deal with these issues,this paper proposes a novel Multi-scale Feature Fusion(MFF)model for bridge appearance disease detection.First,the Faster R-CNN model adopts Region Of Interest(ROl)pooling,which omits the edge information of the target area,resulting in some missed detections and inaccuracies in both detecting and localizing bridge defects.Therefore,this paper proposes an MFF based on regional feature Aggregation(MFF-A),which reduces the missed detection rate of bridge defect detection and improves the positioning accuracy of the target area.Second,the Faster R-CNN model is insensitive to small targets,irregular shapes,and strong noises in bridge defect detection,which results in a long training time and low recognition accuracy.Accordingly,a novel Lightweight MFF(namely MFF-L)model for bridge appearance defect detection using a lightweight network EfficientNetV2 and a feature pyramid network is proposed,which fuses multi-scale features to shorten the training speed and improve recognition accuracy.Finally,the effectiveness of the proposed method is evaluated on the bridge disease dataset and public computational fluid dynamic dataset.

Grasp Detection with Hierarchical Multi-Scale Feature Fusion and Inverted Shuffle Residual

Grasp detection plays a critical role for robot manipulation.Mainstream pixel-wise grasp detection networks with encoder-decoder structure receive much attention due to good accuracy and efficiency.However,they usually transmit the high-level feature in the encoder to the decoder,and low-level features are neglected.It is noted that low-level features contain abundant detail information,and how to fully exploit low-level features remains unsolved.Meanwhile,the channel information in high-level feature is also not well mined.Inevitably,the performance of grasp detection is degraded.To solve these problems,we propose a grasp detection network with hierarchical multi-scale feature fusion and inverted shuffle residual.Both low-level and high-level features in the encoder are firstly fused by the designed skip connections with attention module,and the fused information is then propagated to corresponding layers of the decoder for in-depth feature fusion.Such a hierarchical fusion guarantees the quality of grasp prediction.Furthermore,an inverted shuffle residual module is created,where the high-level feature from encoder is split in channel and the resultant split features are processed in their respective branches.By such differentiation processing,more high-dimensional channel information is kept,which enhances the representation ability of the network.Besides,an information enhancement module is added before the encoder to reinforce input information.The proposed method attains 98.9%and 97.8%in image-wise and object-wise accuracy on the Cornell grasping dataset,respectively,and the experimental results verify the effectiveness of the method.

Neighborhood fusion-based hierarchical parallel feature pyramid network for object detection

In order to improve the detection accuracy of small objects,a neighborhood fusion-based hierarchical parallel feature pyramid network(NFPN)is proposed.Unlike the layer-by-layer structure adopted in the feature pyramid network(FPN)and deconvolutional single shot detector(DSSD),where the bottom layer of the feature pyramid network relies on the top layer,NFPN builds the feature pyramid network with no connections between the upper and lower layers.That is,it only fuses shallow features on similar scales.NFPN is highly portable and can be embedded in many models to further boost performance.Extensive experiments on PASCAL VOC 2007,2012,and COCO datasets demonstrate that the NFPN-based SSD without intricate tricks can exceed the DSSD model in terms of detection accuracy and inference speed,especially for small objects,e.g.,4%to 5%higher mAP(mean average precision)than SSD,and 2%to 3%higher mAP than DSSD.On VOC 2007 test set,the NFPN-based SSD with 300×300 input reaches 79.4%mAP at 34.6 frame/s,and the mAP can raise to 82.9%after using the multi-scale testing strategy.

Industrial Fusion Cascade Detection of Solder Joint

With the remarkable advancements in machine vision research and its ever-expanding applications,scholars have increasingly focused on harnessing various vision methodologies within the industrial realm.Specifically,detecting vehicle floor welding points poses unique challenges,including high operational costs and limited portability in practical settings.To address these challenges,this paper innovatively integrates template matching and the Faster RCNN algorithm,presenting an industrial fusion cascaded solder joint detection algorithm that seamlessly blends template matching with deep learning techniques.This algorithm meticulously weights and fuses the optimized features of both methodologies,enhancing the overall detection capabilities.Furthermore,it introduces an optimized multi-scale and multi-template matching approach,leveraging a diverse array of templates and image pyramid algorithms to bolster the accuracy and resilience of object detection.By integrating deep learning algorithms with this multi-scale and multi-template matching strategy,the cascaded target matching algorithm effectively accurately identifies solder joint types and positions.A comprehensive welding point dataset,labeled by experts specifically for vehicle detection,was constructed based on images from authentic industrial environments to validate the algorithm’s performance.Experiments demonstrate the algorithm’s compelling performance in industrial scenarios,outperforming the single-template matching algorithm by 21.3%,the multi-scale and multitemplate matching algorithm by 3.4%,the Faster RCNN algorithm by 19.7%,and the YOLOv9 algorithm by 17.3%in terms of solder joint detection accuracy.This optimized algorithm exhibits remarkable robustness and portability,ideally suited for detecting solder joints across diverse vehicle workpieces.Notably,this study’s dataset and feature fusion approach can be a valuable resource for other algorithms seeking to enhance their solder joint detection capabilities.This work thus not only presents a novel and effective solution for industrial solder joint detection but lays the groundwork for future advancements in this critical area.

Vehicle color recognition based on smooth modulation neural network with multi-scale feature fusion

Vehicle Color Recognition(VCR)plays a vital role in intelligent traffic management and criminal investigation assistance.However,the existing vehicle color datasets only cover 13 classes,which can not meet the current actual demand.Besides,although lots of efforts are devoted to VCR,they suffer from the problem of class imbalance in datasets.To address these challenges,in this paper,we propose a novel VCR method based on Smooth Modulation Neural Network with Multi-Scale Feature Fusion(SMNN-MSFF).Specifically,to construct the benchmark of model training and evaluation,we first present a new VCR dataset with 24 vehicle classes,Vehicle Color-24,consisting of 10091 vehicle images from a 100-hour urban road surveillance video.Then,to tackle the problem of long-tail distribution and improve the recognition performance,we propose the SMNN-MSFF model with multiscale feature fusion and smooth modulation.The former aims to extract feature information from local to global,and the latter could increase the loss of the images of tail class instances for training with class-imbalance.Finally,comprehensive experimental evaluation on Vehicle Color-24 and previously three representative datasets demonstrate that our proposed SMNN-MSFF outperformed state-of-the-art VCR methods.And extensive ablation studies also demonstrate that each module of our method is effective,especially,the smooth modulation efficiently help feature learning of the minority or tail classes.Vehicle Color-24 and the code of SMNN-MSFF are publicly available and can contact the author to obtain.

Bidirectional parallel multi-branch convolution feature pyramid network for target detection in aerial images of swarm UAVs

In this paper,based on a bidirectional parallel multi-branch feature pyramid network(BPMFPN),a novel one-stage object detector called BPMFPN Det is proposed for real-time detection of ground multi-scale targets by swarm unmanned aerial vehicles(UAVs).First,the bidirectional parallel multi-branch convolution modules are used to construct the feature pyramid to enhance the feature expression abilities of different scale feature layers.Next,the feature pyramid is integrated into the single-stage object detection framework to ensure real-time performance.In order to validate the effectiveness of the proposed algorithm,experiments are conducted on four datasets.For the PASCAL VOC dataset,the proposed algorithm achieves the mean average precision(mAP)of 85.4 on the VOC 2007 test set.With regard to the detection in optical remote sensing(DIOR)dataset,the proposed algorithm achieves 73.9 mAP.For vehicle detection in aerial imagery(VEDAI)dataset,the detection accuracy of small land vehicle(slv)targets reaches 97.4 mAP.For unmanned aerial vehicle detection and tracking(UAVDT)dataset,the proposed BPMFPN Det achieves the mAP of 48.75.Compared with the previous state-of-the-art methods,the results obtained by the proposed algorithm are more competitive.The experimental results demonstrate that the proposed algorithm can effectively solve the problem of real-time detection of ground multi-scale targets in aerial images of swarm UAVs.

Face anti-spoofing based on multi-modal and multi-scale features fusion

Face anti-spoofing is used to assist face recognition system to judge whether the detected face is real face or fake face. In the traditional face anti-spoofing methods, features extracted by hand are used to describe the difference between living face and fraudulent face. But these handmade features do not apply to different variations in an unconstrained environment. The convolutional neural network(CNN) for face deceptions achieves considerable results. However, most existing neural network-based methods simply use neural networks to extract single-scale features from single-modal data, while ignoring multi-scale and multi-modal information. To address this problem, a novel face anti-spoofing method based on multi-modal and multi-scale features fusion(MMFF) is proposed. Specifically, first residual network(Resnet)-34 is adopted to extract features of different scales from each modality, then these features of different scales are fused by feature pyramid network(FPN), finally squeeze-and-excitation fusion(SEF) module and self-attention network(SAN) are combined to fuse features from different modalities for classification. Experiments on the CASIA-SURF dataset show that the new method based on MMFF achieves better performance compared with most existing methods.

基于MFF与IWOA-LSSVM的电机轴承故障诊断研究

针对电机轴承故障诊断过程中,存在种种困难的问题,提出了一种基于多特征融合(MFF)与改进鲸鱼优化算法(IWOA)优化最小二乘支持向量机(LSSVM)的电机轴承状态诊断方法。首先,利用Sobol序列去初始化算法种群,在算法种群搜索过程中加入了莱维飞行策略,并在WOA算法位置更新公式中添加了惯性权重;然后,提取了电机轴承振动信号的小波包能量特征、平均值和峭度,并将以上电机轴承振动信号特征作为算法的输入;最后,为了验证基于MFF与IWOA-LSSVM的电机轴承诊断方法的有效性,分别以单独使用小波包能量特征作为算法输入,以及小波包能量特征和时域特征共同作为算法输入,进行了两组相关的电机轴承状态识别对比实验。研究结果表明:相比于单一小波包能量特征,采用多特征融合能更全面地反映电机轴承真实运行状态;相比于PSO、GA算法,基于WOA算法可以更有效地避免局部最优;相比于基本WOA算法,改进后的WOA算法可以更有效地避免局部最优;相比于其他电机轴承状态识别算法,IWOA-LSSVM算法的分类性能更优,对电机轴承状态识别率达到99.5%。

基于多特征融合的Chirp扩频通信调制样式分类识别方法

自动调制分类(AMC)在频谱监测和认知无线电中具有重要意义。近年来,Chirp扩频通信(CSS)由于其良好的抗干扰能力和稳健性得到了较大发展,但是对CSS信号的AMC方法却鲜有研究。针对这种情况,该文提出了一种基于多特征融合(MFF)的CSS信号调制分类方法,利用频谱和时频图特征融合学习并引入注意力模块来实现CSS信号调制识别。对11类CSS信号调制样式的仿真实验结果表明,该方法有优越的识别性能。

Multi-source Remote Sensing Image Registration Based on Contourlet Transform and Multiple Feature Fusion

Image registration is an indispensable component in multi-source remote sensing image processing. In this paper, we put forward a remote sensing image registration method by including an improved multi-scale and multi-direction Harris algorithm and a novel compound feature. Multi-scale circle Gaussian combined invariant moments and multi-direction gray level co-occurrence matrix are extracted as features for image matching. The proposed algorithm is evaluated on numerous multi-source remote sensor images with noise and illumination changes. Extensive experimental studies prove that our proposed method is capable of receiving stable and even distribution of key points as well as obtaining robust and accurate correspondence matches. It is a promising scheme in multi-source remote sensing image registration.

Modulation recognition network of multi-scale analysis with deep threshold noise elimination

To improve the accuracy of modulated signal recognition in variable environments and reduce the impact of factors such as lack of prior knowledge on recognition results,researchers have gradually adopted deep learning techniques to replace traditional modulated signal processing techniques.To address the problem of low recognition accuracy of the modulated signal at low signal-to-noise ratios,we have designed a novel modulation recognition network of multi-scale analysis with deep threshold noise elimination to recognize the actually collected modulated signals under a symmetric cross-entropy function of label smoothing.The network consists of a denoising encoder with deep adaptive threshold learning and a decoder with multi-scale feature fusion.The two modules are skip-connected to work together to improve the robustness of the overall network.Experimental results show that this method has better recognition accuracy at low signal-to-noise ratios than previous methods.The network demonstrates a flexible self-learning capability for different noise thresholds and the effectiveness of the designed feature fusion module in multi-scale feature acquisition for various modulation types.

复杂场景下基于改进YOLOv4的小型舰船目标检测

针对日益复杂的海洋环境对舰船目标检测更高识别率、实时性、智能化的需求,提出了一种基于改进YOLOv4的舰船目标检测算法。算法将新设计的多层特征融合(MFF)模块和多层接收域块(M-RFB)模块集成到YOLOv4的颈部,改进了网络特征提取的能力,解决了海洋环境中小型舰船的检测和分类问题,模型训练过程中引入迁移学习的策略防止模型过拟合并加速模型训练的参数。实验结果表明:该算法能有效解决小型舰船在复杂海洋环境下检测困难、识别率低的问题。与现有算法相比,该算法能够在复杂的海洋导航条件下获得更高的精度,特别是与YOLOv4相比,准确率提高了约11%。

结合深度学习与特征多尺度融合的微钙化簇检测

为了准确识别X线图像中的微钙化簇以进行乳腺癌的辅助诊断与早期预防,结合细粒度级联增强网络(FCE-Net)与多尺度特征融合算法(MFF),提出微钙化簇目标检测方法.首先构建FCE-Net累加卷积模块层级权重,并增强多分支结构,得到细粒度卷积特征图.然后构建MFF候选检测网络,通过二倍上采样融合多尺度特征,得到目标置信度和区域坐标.最后在感兴趣区域池化层分类目标并调整边界框.在MIAS数据集上实验表明,结合FCE-Net与MFF可以提升微小目标的深层特征提取能力,同时增强目标分类与定位的准确度.

一种金属类酒瓶盖瑕疵质检算法

针对酒瓶盖瑕疵会影响产品质量的问题,提出了一种酒瓶盖瑕疵YOLOv3-MRHA检测算法,基于YOLOv3算法,对其主干网络和特征提取层进行改进。为减少主干网络特征丢失现象,提出了多级特征融合(multilevel feature fusion,MFF)模块;为提高检测的准确率,增加了尺度为104×104的特征层,并构造了一种增强特征信息的残差特征增强(residual feature enhancement,RFE)模块;为提高深层特征层的检测能力,引入了空洞卷积,使浅层信息向下融合,在特征提取层使用通道注意力机制。结果表明,所提YOLOv3-MRHA算法的检测精度比YOLOv3算法提高近6%,可有效地提高瑕疵检测的准确率,满足工业质检的要求。

Research on pedestrian detection based on multi-level fine-grained YOLOX algorithm

Purpose-The purpose of the study is to address the problems of low accuracy and missed detection of occluded pedestrians and small target pedestrians when using the YOLOX general object detection algorithm for pedestrian detection.This study proposes a multi-level fine-grained YOLOX pedestrian detection algorithm.Design/methodology/approach-First,to address the problem of the original YOLOX algorithm in obtaining a single perceptual field for the feature map before feature fusion,this study improves the PAFPN structure by adding the ResCoT module to increase the diversity of the perceptual field of the feature map and divides the pedestrian multi-scale features into finer granularity.Second,for the CSPLayer of the PAFPN,a weight gain-based normalization-based attention module(NAM)is proposed to make the model pay more attention to the context information when extracting pedestrian features and highlight the salient features of pedestrians.Finally,the authors experimentally determined the optimal values for the confidence loss function.Findings-The experimental results show that,compared with the original YOLOX algorithm,the AP of the improved algorithm increased by 2.90%,the Recall increased by 3.57%,and F1 increased by 2%on the pedestrian dataset.Research limitations/implications-The multi-level fine-grained YOLOX pedestrian detection algorithm can effectively improve the detection of occluded pedestrians and small target pedestrians.Originality/value-The authors introduce a multi-level fine-grained ResCoT module and a weight gain-based NAM attention module.