A Lightweight Convolutional Neural Network with Hierarchical Multi-Scale Feature Fusion for Image Classification

Convolutional neural networks (CNNs) are widely used in image classification tasks, but their increasing model size and computation make them challenging to implement on embedded systems with constrained hardware resources. To address this issue, the MobileNetV1 network was developed, which employs depthwise convolution to reduce network complexity. MobileNetV1 employs a stride of 2 in several convolutional layers to decrease the spatial resolution of feature maps, thereby lowering computational costs. However, this stride setting can lead to a loss of spatial information, particularly affecting the detection and representation of smaller objects or finer details in images. To maintain the trade-off between complexity and model performance, a lightweight convolutional neural network with hierarchical multi-scale feature fusion based on the MobileNetV1 network is proposed. The network consists of two main subnetworks. The first subnetwork uses a depthwise dilated separable convolution (DDSC) layer to learn imaging features with fewer parameters, which results in a lightweight and computationally inexpensive network. Furthermore, depthwise dilated convolution in DDSC layer effectively expands the field of view of filters, allowing them to incorporate a larger context. The second subnetwork is a hierarchical multi-scale feature fusion (HMFF) module that uses parallel multi-resolution branches architecture to process the input feature map in order to extract the multi-scale feature information of the input image. Experimental results on the CIFAR-10, Malaria, and KvasirV1 datasets demonstrate that the proposed method is efficient, reducing the network parameters and computational cost by 65.02% and 39.78%, respectively, while maintaining the network performance compared to the MobileNetV1 baseline.

Attention Guided Multi Scale Feature Fusion Network for Automatic Prostate Segmentation

The precise and automatic segmentation of prostate magnetic resonance imaging(MRI)images is vital for assisting doctors in diagnosing prostate diseases.In recent years,many advanced methods have been applied to prostate segmentation,but due to the variability caused by prostate diseases,automatic segmentation of the prostate presents significant challenges.In this paper,we propose an attention-guided multi-scale feature fusion network(AGMSF-Net)to segment prostate MRI images.We propose an attention mechanism for extracting multi-scale features,and introduce a 3D transformer module to enhance global feature representation by adding it during the transition phase from encoder to decoder.In the decoder stage,a feature fusion module is proposed to obtain global context information.We evaluate our model on MRI images of the prostate acquired from a local hospital.The relative volume difference(RVD)and dice similarity coefficient(DSC)between the results of automatic prostate segmentation and ground truth were 1.21%and 93.68%,respectively.To quantitatively evaluate prostate volume on MRI,which is of significant clinical significance,we propose a unique AGMSF-Net.The essential performance evaluation and validation experiments have demonstrated the effectiveness of our method in automatic prostate segmentation.

Ship recognition based on HRRP via multi-scale sparse preserving method

In order to extract the richer feature information of ship targets from sea clutter, and address the high dimensional data problem, a method termed as multi-scale fusion kernel sparse preserving projection(MSFKSPP) based on the maximum margin criterion(MMC) is proposed for recognizing the class of ship targets utilizing the high-resolution range profile(HRRP). Multi-scale fusion is introduced to capture the local and detailed information in small-scale features, and the global and contour information in large-scale features, offering help to extract the edge information from sea clutter and further improving the target recognition accuracy. The proposed method can maximally preserve the multi-scale fusion sparse of data and maximize the class separability in the reduced dimensionality by reproducing kernel Hilbert space. Experimental results on the measured radar data show that the proposed method can effectively extract the features of ship target from sea clutter, further reduce the feature dimensionality, and improve target recognition performance.

Feature Fusion-Based Deep Learning Network to Recognize Table Tennis Actions

A system for classifying four basic table tennis strokes using wearable devices and deep learning networks is proposed in this study.The wearable device consisted of a six-axis sensor,Raspberry Pi 3,and a power bank.Multiple kernel sizes were used in convolutional neural network(CNN)to evaluate their performance for extracting features.Moreover,a multiscale CNN with two kernel sizes was used to perform feature fusion at different scales in a concatenated manner.The CNN achieved recognition of the four table tennis strokes.Experimental data were obtained from20 research participants who wore sensors on the back of their hands while performing the four table tennis strokes in a laboratory environment.The data were collected to verify the performance of the proposed models for wearable devices.Finally,the sensor and multi-scale CNN designed in this study achieved accuracy and F1 scores of 99.58%and 99.16%,respectively,for the four strokes.The accuracy for five-fold cross validation was 99.87%.This result also shows that the multi-scale convolutional neural network has better robustness after fivefold cross validation.

Grasp Detection with Hierarchical Multi-Scale Feature Fusion and Inverted Shuffle Residual

Grasp detection plays a critical role for robot manipulation.Mainstream pixel-wise grasp detection networks with encoder-decoder structure receive much attention due to good accuracy and efficiency.However,they usually transmit the high-level feature in the encoder to the decoder,and low-level features are neglected.It is noted that low-level features contain abundant detail information,and how to fully exploit low-level features remains unsolved.Meanwhile,the channel information in high-level feature is also not well mined.Inevitably,the performance of grasp detection is degraded.To solve these problems,we propose a grasp detection network with hierarchical multi-scale feature fusion and inverted shuffle residual.Both low-level and high-level features in the encoder are firstly fused by the designed skip connections with attention module,and the fused information is then propagated to corresponding layers of the decoder for in-depth feature fusion.Such a hierarchical fusion guarantees the quality of grasp prediction.Furthermore,an inverted shuffle residual module is created,where the high-level feature from encoder is split in channel and the resultant split features are processed in their respective branches.By such differentiation processing,more high-dimensional channel information is kept,which enhances the representation ability of the network.Besides,an information enhancement module is added before the encoder to reinforce input information.The proposed method attains 98.9%and 97.8%in image-wise and object-wise accuracy on the Cornell grasping dataset,respectively,and the experimental results verify the effectiveness of the method.

Multi-Scale Feature Fusion Model for Bridge Appearance Defect Detection

Although the Faster Region-based Convolutional Neural Network(Faster R-CNN)model has obvious advantages in defect recognition,it still cannot overcome challenging problems,such as time-consuming,small targets,irregular shapes,and strong noise interference in bridge defect detection.To deal with these issues,this paper proposes a novel Multi-scale Feature Fusion(MFF)model for bridge appearance disease detection.First,the Faster R-CNN model adopts Region Of Interest(ROl)pooling,which omits the edge information of the target area,resulting in some missed detections and inaccuracies in both detecting and localizing bridge defects.Therefore,this paper proposes an MFF based on regional feature Aggregation(MFF-A),which reduces the missed detection rate of bridge defect detection and improves the positioning accuracy of the target area.Second,the Faster R-CNN model is insensitive to small targets,irregular shapes,and strong noises in bridge defect detection,which results in a long training time and low recognition accuracy.Accordingly,a novel Lightweight MFF(namely MFF-L)model for bridge appearance defect detection using a lightweight network EfficientNetV2 and a feature pyramid network is proposed,which fuses multi-scale features to shorten the training speed and improve recognition accuracy.Finally,the effectiveness of the proposed method is evaluated on the bridge disease dataset and public computational fluid dynamic dataset.

Neighborhood fusion-based hierarchical parallel feature pyramid network for object detection

In order to improve the detection accuracy of small objects,a neighborhood fusion-based hierarchical parallel feature pyramid network(NFPN)is proposed.Unlike the layer-by-layer structure adopted in the feature pyramid network(FPN)and deconvolutional single shot detector(DSSD),where the bottom layer of the feature pyramid network relies on the top layer,NFPN builds the feature pyramid network with no connections between the upper and lower layers.That is,it only fuses shallow features on similar scales.NFPN is highly portable and can be embedded in many models to further boost performance.Extensive experiments on PASCAL VOC 2007,2012,and COCO datasets demonstrate that the NFPN-based SSD without intricate tricks can exceed the DSSD model in terms of detection accuracy and inference speed,especially for small objects,e.g.,4%to 5%higher mAP(mean average precision)than SSD,and 2%to 3%higher mAP than DSSD.On VOC 2007 test set,the NFPN-based SSD with 300×300 input reaches 79.4%mAP at 34.6 frame/s,and the mAP can raise to 82.9%after using the multi-scale testing strategy.

Industrial Fusion Cascade Detection of Solder Joint

With the remarkable advancements in machine vision research and its ever-expanding applications,scholars have increasingly focused on harnessing various vision methodologies within the industrial realm.Specifically,detecting vehicle floor welding points poses unique challenges,including high operational costs and limited portability in practical settings.To address these challenges,this paper innovatively integrates template matching and the Faster RCNN algorithm,presenting an industrial fusion cascaded solder joint detection algorithm that seamlessly blends template matching with deep learning techniques.This algorithm meticulously weights and fuses the optimized features of both methodologies,enhancing the overall detection capabilities.Furthermore,it introduces an optimized multi-scale and multi-template matching approach,leveraging a diverse array of templates and image pyramid algorithms to bolster the accuracy and resilience of object detection.By integrating deep learning algorithms with this multi-scale and multi-template matching strategy,the cascaded target matching algorithm effectively accurately identifies solder joint types and positions.A comprehensive welding point dataset,labeled by experts specifically for vehicle detection,was constructed based on images from authentic industrial environments to validate the algorithm’s performance.Experiments demonstrate the algorithm’s compelling performance in industrial scenarios,outperforming the single-template matching algorithm by 21.3%,the multi-scale and multitemplate matching algorithm by 3.4%,the Faster RCNN algorithm by 19.7%,and the YOLOv9 algorithm by 17.3%in terms of solder joint detection accuracy.This optimized algorithm exhibits remarkable robustness and portability,ideally suited for detecting solder joints across diverse vehicle workpieces.Notably,this study’s dataset and feature fusion approach can be a valuable resource for other algorithms seeking to enhance their solder joint detection capabilities.This work thus not only presents a novel and effective solution for industrial solder joint detection but lays the groundwork for future advancements in this critical area.

Vehicle color recognition based on smooth modulation neural network with multi-scale feature fusion

Vehicle Color Recognition(VCR)plays a vital role in intelligent traffic management and criminal investigation assistance.However,the existing vehicle color datasets only cover 13 classes,which can not meet the current actual demand.Besides,although lots of efforts are devoted to VCR,they suffer from the problem of class imbalance in datasets.To address these challenges,in this paper,we propose a novel VCR method based on Smooth Modulation Neural Network with Multi-Scale Feature Fusion(SMNN-MSFF).Specifically,to construct the benchmark of model training and evaluation,we first present a new VCR dataset with 24 vehicle classes,Vehicle Color-24,consisting of 10091 vehicle images from a 100-hour urban road surveillance video.Then,to tackle the problem of long-tail distribution and improve the recognition performance,we propose the SMNN-MSFF model with multiscale feature fusion and smooth modulation.The former aims to extract feature information from local to global,and the latter could increase the loss of the images of tail class instances for training with class-imbalance.Finally,comprehensive experimental evaluation on Vehicle Color-24 and previously three representative datasets demonstrate that our proposed SMNN-MSFF outperformed state-of-the-art VCR methods.And extensive ablation studies also demonstrate that each module of our method is effective,especially,the smooth modulation efficiently help feature learning of the minority or tail classes.Vehicle Color-24 and the code of SMNN-MSFF are publicly available and can contact the author to obtain.

Bidirectional parallel multi-branch convolution feature pyramid network for target detection in aerial images of swarm UAVs

In this paper,based on a bidirectional parallel multi-branch feature pyramid network(BPMFPN),a novel one-stage object detector called BPMFPN Det is proposed for real-time detection of ground multi-scale targets by swarm unmanned aerial vehicles(UAVs).First,the bidirectional parallel multi-branch convolution modules are used to construct the feature pyramid to enhance the feature expression abilities of different scale feature layers.Next,the feature pyramid is integrated into the single-stage object detection framework to ensure real-time performance.In order to validate the effectiveness of the proposed algorithm,experiments are conducted on four datasets.For the PASCAL VOC dataset,the proposed algorithm achieves the mean average precision(mAP)of 85.4 on the VOC 2007 test set.With regard to the detection in optical remote sensing(DIOR)dataset,the proposed algorithm achieves 73.9 mAP.For vehicle detection in aerial imagery(VEDAI)dataset,the detection accuracy of small land vehicle(slv)targets reaches 97.4 mAP.For unmanned aerial vehicle detection and tracking(UAVDT)dataset,the proposed BPMFPN Det achieves the mAP of 48.75.Compared with the previous state-of-the-art methods,the results obtained by the proposed algorithm are more competitive.The experimental results demonstrate that the proposed algorithm can effectively solve the problem of real-time detection of ground multi-scale targets in aerial images of swarm UAVs.

Face anti-spoofing based on multi-modal and multi-scale features fusion

Face anti-spoofing is used to assist face recognition system to judge whether the detected face is real face or fake face. In the traditional face anti-spoofing methods, features extracted by hand are used to describe the difference between living face and fraudulent face. But these handmade features do not apply to different variations in an unconstrained environment. The convolutional neural network(CNN) for face deceptions achieves considerable results. However, most existing neural network-based methods simply use neural networks to extract single-scale features from single-modal data, while ignoring multi-scale and multi-modal information. To address this problem, a novel face anti-spoofing method based on multi-modal and multi-scale features fusion(MMFF) is proposed. Specifically, first residual network(Resnet)-34 is adopted to extract features of different scales from each modality, then these features of different scales are fused by feature pyramid network(FPN), finally squeeze-and-excitation fusion(SEF) module and self-attention network(SAN) are combined to fuse features from different modalities for classification. Experiments on the CASIA-SURF dataset show that the new method based on MMFF achieves better performance compared with most existing methods.

基于MA-ConvNext网络和分步关系知识蒸馏的苹果叶片病害识别

针对复杂环境下苹果叶片病害图像背景杂乱、病斑大小不一,以及现有模型参数多、计算量大的问题,提出基于注意力和多尺度特征融合的苹果叶片病害识别网络(MA-ConvNext).通过引入多尺度空间通道重组块(MSCB)和融合三分支注意力机制的特征提取模块(TAFB),有效提取苹果叶片病害图像不同尺度的特征,增强模型对叶片病斑的关注.采用分步关系知识蒸馏方法,将“教师”网络(MA-ConvNext)和“中间”网络(DenseNet121)融合,指导“学生”网络(EfficientNet-B0)训练,实现模型轻量化.实验结果表明,MA-ConvNext网络识别准确率为99.38%,较ResNet50、MobileNet-V3和EfficientNet-V2网络分别提高了3.98个百分点、7.55个百分点和4.27个百分点.经过分步关系知识蒸馏后,识别准确率较蒸馏前提高了1.76个百分点,并且具有更小的网络规模和参数量,分别为1.56×10^(7)、5.29×10^(6).所提方法能为后续精准农业的病虫害检测提供新思路和技术支持.

基于特征优选和逐步回归的黄土滑坡监测数据融合改进方法

针对滑坡监测多源异构数据融合处理中存在的影响因子筛选难、结果差异大、数据处理复杂程度高等问题,提出一种基于最大互信息系数(MIC)、灰色关联分析(GRA)和逐步回归的黄土滑坡多源多点位异构监测数据融合方法。该方法首先将最大互信息系数和灰色关联分析结合起来,采用基于加权关联度的特征优选方法综合筛选滑坡变形影响因子,提取具有代表性的影响因子并剔除关联性差的影响因子;然后,通过逐步回归方法赋予各监测点位移和优选后的影响因子对应的重要性权重系数,获取多源异构数据融合序列;最后,采用甘肃黑方台党川滑坡监测设备所获取的全球卫星导航系统(GNSS)监测数据、裂缝位移计数据及气象数据进行实验验证。结果表明:在滑坡变形影响因子筛选性能方面,基于加权关联度的特征优选方法优于传统的Pearson相关系数法;基于特征优选和逐步回归的多源多点位异构数据融合模型的预测精度较传统的BP神经网络有所提升,其中均方根误差(RMSE)降低了51.8%,平均绝对百分比误差(MAPE)降低了2.26%,拟合优度达到了0.964。

Multi-source Remote Sensing Image Registration Based on Contourlet Transform and Multiple Feature Fusion

Image registration is an indispensable component in multi-source remote sensing image processing. In this paper, we put forward a remote sensing image registration method by including an improved multi-scale and multi-direction Harris algorithm and a novel compound feature. Multi-scale circle Gaussian combined invariant moments and multi-direction gray level co-occurrence matrix are extracted as features for image matching. The proposed algorithm is evaluated on numerous multi-source remote sensor images with noise and illumination changes. Extensive experimental studies prove that our proposed method is capable of receiving stable and even distribution of key points as well as obtaining robust and accurate correspondence matches. It is a promising scheme in multi-source remote sensing image registration.

Modulation recognition network of multi-scale analysis with deep threshold noise elimination

To improve the accuracy of modulated signal recognition in variable environments and reduce the impact of factors such as lack of prior knowledge on recognition results,researchers have gradually adopted deep learning techniques to replace traditional modulated signal processing techniques.To address the problem of low recognition accuracy of the modulated signal at low signal-to-noise ratios,we have designed a novel modulation recognition network of multi-scale analysis with deep threshold noise elimination to recognize the actually collected modulated signals under a symmetric cross-entropy function of label smoothing.The network consists of a denoising encoder with deep adaptive threshold learning and a decoder with multi-scale feature fusion.The two modules are skip-connected to work together to improve the robustness of the overall network.Experimental results show that this method has better recognition accuracy at low signal-to-noise ratios than previous methods.The network demonstrates a flexible self-learning capability for different noise thresholds and the effectiveness of the designed feature fusion module in multi-scale feature acquisition for various modulation types.

多传感器数据融合技术在电子舌中的应用研究

针对传统猪肉新鲜度检测中存在的操作复杂、时间久、易受人为因素影响等缺点,设计了基于多传感器数据融合的电子舌系统。该系统由电位传感器和伏安传感器两部分构成,通过分析计算传感器阵列对猪肉响应曲线的特征值来考量猪肉新鲜度的变化。采用电位传感器与伏安传感器特征融合的方法对猪肉进行了特征向量的提取,使用逐步判别法对所提取的特征向量进行了优化,使用Bayes判别法分析了猪肉的新鲜度变化情况。实验结果表明,该电子舌系统能够准确鉴别出5组测量间隔时间均为12h猪肉新鲜度的不同。该系统为猪肉新鲜度智能化检测提供了新的有效方法。

基于分步多尺度特征融合的SSD目标检测算法

为解决SSD(single shot multibox detector)目标检测算法由于浅层特征表征能力不强,导致对小目标检测精度较低的问题,提出了一种基于分步多尺度特征融合的SSD目标检测算法。为增加SSD模型浅层特征包含的细节信息和语义信息,在模型的低层特征部分引入了两个特征层;只对模型低层的两个特征图进行反卷积操作,并且分两步将低层三个不同尺度的特征图进行特征融合,不仅提高了模型浅层特征的表征能力,而且减少了算法运行过程中的计算量。实验结果表明,在PASCAL VOC2007数据集上,改进后的算法小目标类别的AP值得到了大幅度提高,mAP值比SSD算法提高了3.6%,算法的检测速度也满足实时性要求。

Research on pedestrian detection based on multi-level fine-grained YOLOX algorithm

Purpose-The purpose of the study is to address the problems of low accuracy and missed detection of occluded pedestrians and small target pedestrians when using the YOLOX general object detection algorithm for pedestrian detection.This study proposes a multi-level fine-grained YOLOX pedestrian detection algorithm.Design/methodology/approach-First,to address the problem of the original YOLOX algorithm in obtaining a single perceptual field for the feature map before feature fusion,this study improves the PAFPN structure by adding the ResCoT module to increase the diversity of the perceptual field of the feature map and divides the pedestrian multi-scale features into finer granularity.Second,for the CSPLayer of the PAFPN,a weight gain-based normalization-based attention module(NAM)is proposed to make the model pay more attention to the context information when extracting pedestrian features and highlight the salient features of pedestrians.Finally,the authors experimentally determined the optimal values for the confidence loss function.Findings-The experimental results show that,compared with the original YOLOX algorithm,the AP of the improved algorithm increased by 2.90%,the Recall increased by 3.57%,and F1 increased by 2%on the pedestrian dataset.Research limitations/implications-The multi-level fine-grained YOLOX pedestrian detection algorithm can effectively improve the detection of occluded pedestrians and small target pedestrians.Originality/value-The authors introduce a multi-level fine-grained ResCoT module and a weight gain-based NAM attention module.