A Lightweight Convolutional Neural Network with Hierarchical Multi-Scale Feature Fusion for Image Classification

Convolutional neural networks (CNNs) are widely used in image classification tasks, but their increasing model size and computation make them challenging to implement on embedded systems with constrained hardware resources. To address this issue, the MobileNetV1 network was developed, which employs depthwise convolution to reduce network complexity. MobileNetV1 employs a stride of 2 in several convolutional layers to decrease the spatial resolution of feature maps, thereby lowering computational costs. However, this stride setting can lead to a loss of spatial information, particularly affecting the detection and representation of smaller objects or finer details in images. To maintain the trade-off between complexity and model performance, a lightweight convolutional neural network with hierarchical multi-scale feature fusion based on the MobileNetV1 network is proposed. The network consists of two main subnetworks. The first subnetwork uses a depthwise dilated separable convolution (DDSC) layer to learn imaging features with fewer parameters, which results in a lightweight and computationally inexpensive network. Furthermore, depthwise dilated convolution in DDSC layer effectively expands the field of view of filters, allowing them to incorporate a larger context. The second subnetwork is a hierarchical multi-scale feature fusion (HMFF) module that uses parallel multi-resolution branches architecture to process the input feature map in order to extract the multi-scale feature information of the input image. Experimental results on the CIFAR-10, Malaria, and KvasirV1 datasets demonstrate that the proposed method is efficient, reducing the network parameters and computational cost by 65.02% and 39.78%, respectively, while maintaining the network performance compared to the MobileNetV1 baseline.

Attention Guided Multi Scale Feature Fusion Network for Automatic Prostate Segmentation

The precise and automatic segmentation of prostate magnetic resonance imaging(MRI)images is vital for assisting doctors in diagnosing prostate diseases.In recent years,many advanced methods have been applied to prostate segmentation,but due to the variability caused by prostate diseases,automatic segmentation of the prostate presents significant challenges.In this paper,we propose an attention-guided multi-scale feature fusion network(AGMSF-Net)to segment prostate MRI images.We propose an attention mechanism for extracting multi-scale features,and introduce a 3D transformer module to enhance global feature representation by adding it during the transition phase from encoder to decoder.In the decoder stage,a feature fusion module is proposed to obtain global context information.We evaluate our model on MRI images of the prostate acquired from a local hospital.The relative volume difference(RVD)and dice similarity coefficient(DSC)between the results of automatic prostate segmentation and ground truth were 1.21%and 93.68%,respectively.To quantitatively evaluate prostate volume on MRI,which is of significant clinical significance,we propose a unique AGMSF-Net.The essential performance evaluation and validation experiments have demonstrated the effectiveness of our method in automatic prostate segmentation.

Feature Fusion-Based Deep Learning Network to Recognize Table Tennis Actions

A system for classifying four basic table tennis strokes using wearable devices and deep learning networks is proposed in this study.The wearable device consisted of a six-axis sensor,Raspberry Pi 3,and a power bank.Multiple kernel sizes were used in convolutional neural network(CNN)to evaluate their performance for extracting features.Moreover,a multiscale CNN with two kernel sizes was used to perform feature fusion at different scales in a concatenated manner.The CNN achieved recognition of the four table tennis strokes.Experimental data were obtained from20 research participants who wore sensors on the back of their hands while performing the four table tennis strokes in a laboratory environment.The data were collected to verify the performance of the proposed models for wearable devices.Finally,the sensor and multi-scale CNN designed in this study achieved accuracy and F1 scores of 99.58%and 99.16%,respectively,for the four strokes.The accuracy for five-fold cross validation was 99.87%.This result also shows that the multi-scale convolutional neural network has better robustness after fivefold cross validation.

Grasp Detection with Hierarchical Multi-Scale Feature Fusion and Inverted Shuffle Residual

Grasp detection plays a critical role for robot manipulation.Mainstream pixel-wise grasp detection networks with encoder-decoder structure receive much attention due to good accuracy and efficiency.However,they usually transmit the high-level feature in the encoder to the decoder,and low-level features are neglected.It is noted that low-level features contain abundant detail information,and how to fully exploit low-level features remains unsolved.Meanwhile,the channel information in high-level feature is also not well mined.Inevitably,the performance of grasp detection is degraded.To solve these problems,we propose a grasp detection network with hierarchical multi-scale feature fusion and inverted shuffle residual.Both low-level and high-level features in the encoder are firstly fused by the designed skip connections with attention module,and the fused information is then propagated to corresponding layers of the decoder for in-depth feature fusion.Such a hierarchical fusion guarantees the quality of grasp prediction.Furthermore,an inverted shuffle residual module is created,where the high-level feature from encoder is split in channel and the resultant split features are processed in their respective branches.By such differentiation processing,more high-dimensional channel information is kept,which enhances the representation ability of the network.Besides,an information enhancement module is added before the encoder to reinforce input information.The proposed method attains 98.9%and 97.8%in image-wise and object-wise accuracy on the Cornell grasping dataset,respectively,and the experimental results verify the effectiveness of the method.

Multi-Scale Feature Fusion Model for Bridge Appearance Defect Detection

Although the Faster Region-based Convolutional Neural Network(Faster R-CNN)model has obvious advantages in defect recognition,it still cannot overcome challenging problems,such as time-consuming,small targets,irregular shapes,and strong noise interference in bridge defect detection.To deal with these issues,this paper proposes a novel Multi-scale Feature Fusion(MFF)model for bridge appearance disease detection.First,the Faster R-CNN model adopts Region Of Interest(ROl)pooling,which omits the edge information of the target area,resulting in some missed detections and inaccuracies in both detecting and localizing bridge defects.Therefore,this paper proposes an MFF based on regional feature Aggregation(MFF-A),which reduces the missed detection rate of bridge defect detection and improves the positioning accuracy of the target area.Second,the Faster R-CNN model is insensitive to small targets,irregular shapes,and strong noises in bridge defect detection,which results in a long training time and low recognition accuracy.Accordingly,a novel Lightweight MFF(namely MFF-L)model for bridge appearance defect detection using a lightweight network EfficientNetV2 and a feature pyramid network is proposed,which fuses multi-scale features to shorten the training speed and improve recognition accuracy.Finally,the effectiveness of the proposed method is evaluated on the bridge disease dataset and public computational fluid dynamic dataset.

Vehicle color recognition based on smooth modulation neural network with multi-scale feature fusion

Vehicle Color Recognition(VCR)plays a vital role in intelligent traffic management and criminal investigation assistance.However,the existing vehicle color datasets only cover 13 classes,which can not meet the current actual demand.Besides,although lots of efforts are devoted to VCR,they suffer from the problem of class imbalance in datasets.To address these challenges,in this paper,we propose a novel VCR method based on Smooth Modulation Neural Network with Multi-Scale Feature Fusion(SMNN-MSFF).Specifically,to construct the benchmark of model training and evaluation,we first present a new VCR dataset with 24 vehicle classes,Vehicle Color-24,consisting of 10091 vehicle images from a 100-hour urban road surveillance video.Then,to tackle the problem of long-tail distribution and improve the recognition performance,we propose the SMNN-MSFF model with multiscale feature fusion and smooth modulation.The former aims to extract feature information from local to global,and the latter could increase the loss of the images of tail class instances for training with class-imbalance.Finally,comprehensive experimental evaluation on Vehicle Color-24 and previously three representative datasets demonstrate that our proposed SMNN-MSFF outperformed state-of-the-art VCR methods.And extensive ablation studies also demonstrate that each module of our method is effective,especially,the smooth modulation efficiently help feature learning of the minority or tail classes.Vehicle Color-24 and the code of SMNN-MSFF are publicly available and can contact the author to obtain.

Bidirectional parallel multi-branch convolution feature pyramid network for target detection in aerial images of swarm UAVs

In this paper,based on a bidirectional parallel multi-branch feature pyramid network(BPMFPN),a novel one-stage object detector called BPMFPN Det is proposed for real-time detection of ground multi-scale targets by swarm unmanned aerial vehicles(UAVs).First,the bidirectional parallel multi-branch convolution modules are used to construct the feature pyramid to enhance the feature expression abilities of different scale feature layers.Next,the feature pyramid is integrated into the single-stage object detection framework to ensure real-time performance.In order to validate the effectiveness of the proposed algorithm,experiments are conducted on four datasets.For the PASCAL VOC dataset,the proposed algorithm achieves the mean average precision(mAP)of 85.4 on the VOC 2007 test set.With regard to the detection in optical remote sensing(DIOR)dataset,the proposed algorithm achieves 73.9 mAP.For vehicle detection in aerial imagery(VEDAI)dataset,the detection accuracy of small land vehicle(slv)targets reaches 97.4 mAP.For unmanned aerial vehicle detection and tracking(UAVDT)dataset,the proposed BPMFPN Det achieves the mAP of 48.75.Compared with the previous state-of-the-art methods,the results obtained by the proposed algorithm are more competitive.The experimental results demonstrate that the proposed algorithm can effectively solve the problem of real-time detection of ground multi-scale targets in aerial images of swarm UAVs.

基于改进DETR的机器人铆接缺陷检测方法研究

铆接作为铁道车辆结构件的主要连接方式,合格的铆接质量是车辆安全稳定运行的重要保证。针对现有铆接缺陷检测方法存在检测精度低、检测点位少、检测智能化水平不高等问题,提出一种基于改进DETR的机器人铆接缺陷检测方法。首先,搭建铆接缺陷检测系统,依次采集工件尺寸大、铆钉尺寸小工况下的铆接缺陷图像。其次,为了增强DETR模型在小目标中的图像特征提取能力和检测性能,以EfficientNet作为DETR中的主干特征提取网络,并将3-D权重注意力机制SimAM引入EfficientNet网络,从而有效保留图像特征层的镦头形态信息和铆点区域的空间信息。然后,在颈部网络中引入加权双向特征金字塔模块,以EfficientNet网络的输出作为特征融合模块的输入对各尺度特征信息进行聚合,增大不同铆接缺陷的类间差异。最后,利用Smooth L1和DIoU的线性组合改进原模型预测网络的回归损失函数,提高模型的检测精度和收敛速度。结果表明,改进模型表现出较高的检测性能,对于铆接缺陷的平均检测精度mAP为97.12%,检测速度FPS为25.4帧/s,与Faster RCNN、YOLOX等其他主流检测模型相比,在检测精度和检测速度方面均具有较大优势。研究结果能够满足实际工况中大型铆接件的小尺寸铆钉铆接缺陷实时在线检测的需求,为视觉检测技术在铆接工艺中的应用提供一定的参考价值。

基于改进的YOLOv5安全帽佩戴检测算法

针对安全帽佩戴检测中存在的误检和漏检的问题,提出一种基于YOLOv5模型改进的安全帽佩戴检测算法。改进模型引入多尺度加权特征融合网络,即在YOLOv5的网络结构中增加一个浅层检测尺度,并引入特征权重进行加权融合,构成新的四尺检测结构,有效地提升图像浅层特征的提取及融合能力;在YOLOv5的Neck网络的BottleneckCSP结构中加入SENet模块,使模型更多地关注目标信息忽略背景信息;针对大分辨率的图像,添加图像切割层,避免多倍下采样造成的小目标特征信息大量丢失。对YOLOv5模型进行改进之后,通过自制的安全帽数据集进行训练检测,mAP和召回率分别达到97.06%、92.54%,与YOLOv5相比较分别提升了4.74%和4.31%。实验结果表明:改进的YOLOv5算法可有效提升安全帽佩戴的检测性能,能够准确识别施工人员的安全帽佩戴情况,从而大大降低施工现场的安全风险。

改进YOLOv5的无人机航拍图像目标检测算法

针对无人机航拍图像目标检测中目标尺度多样、相似目标众多、目标聚集导致的目标漏检、误检问题,提出了改进YOLOv5的无人机航拍图像目标检测算法DA-YOLO。提出由特征图注意力生成器和动态权重学习模块组成的多尺度动态特征加权融合网络,特征图注意力生成器融合处理不同尺度目标更重要的特征,权重学习模块自适应地调节对不同尺度目标特征的学习,该网络可增强在目标尺度多样下的辨识度从而降低目标漏检。设计一种并行选择性注意力机制(PSAM)添加到特征提取网络中,该模块通过动态融合空间信息和通道信息,加强特征的表达获得更优质的特征图,提高网络对相似目标的区分能力以减少误检。使用Soft-NMS代替YOLOv5中采用的非极大值抑制(NMS)以改善目标聚集场景下的漏检、误检。实验结果表明,改进算法在VisDrone数据集上检测精度达到37.79%,相比于YOLOv5s算法精度提高了5.59个百分点,改进后的算法可以更好地应用于无人机航拍图像目标检测中。

Hydraulic directional valve fault diagnosis using a weighted adaptive fusion of multi-dimensional features of a multi-sensor

Because the hydraulic directional valve usually works in a bad working environment and is disturbed by multi-factor noise,the traditional single sensor monitoring technology is difficult to use for an accurate diagnosis of it.Therefore,a fault diagnosis method based on multi-sensor information fusion is proposed in this paper to reduce the inaccuracy and uncertainty of traditional single sensor information diagnosis technology and to realize accurate monitoring for the location or diagnosis of early faults in such valves in noisy environments.Firstly,the statistical features of signals collected by the multi-sensor are extracted and the depth features are obtained by a convolutional neural network(CNN)to form a complete and stable multi-dimensional feature set.Secondly,to obtain a weighted multi-dimensional feature set,the multi-dimensional feature sets of similar sensors are combined,and the entropy weight method is used to weight these features to reduce the interference of insensitive features.Finally,the attention mechanism is introduced to improve the dual-channel CNN,which is used to adaptively fuse the weighted multi-dimensional feature sets of heterogeneous sensors,to flexibly select heterogeneous sensor information so as to achieve an accurate diagnosis.Experimental results show that the weighted multi-dimensional feature set obtained by the proposed method has a high fault-representation ability and low information redundancy.It can diagnose simultaneously internal wear faults of the hydraulic directional valve and electromagnetic faults of actuators that are difficult to diagnose by traditional methods.This proposed method can achieve high fault-diagnosis accuracy under severe working conditions.

Face anti-spoofing based on multi-modal and multi-scale features fusion

Face anti-spoofing is used to assist face recognition system to judge whether the detected face is real face or fake face. In the traditional face anti-spoofing methods, features extracted by hand are used to describe the difference between living face and fraudulent face. But these handmade features do not apply to different variations in an unconstrained environment. The convolutional neural network(CNN) for face deceptions achieves considerable results. However, most existing neural network-based methods simply use neural networks to extract single-scale features from single-modal data, while ignoring multi-scale and multi-modal information. To address this problem, a novel face anti-spoofing method based on multi-modal and multi-scale features fusion(MMFF) is proposed. Specifically, first residual network(Resnet)-34 is adopted to extract features of different scales from each modality, then these features of different scales are fused by feature pyramid network(FPN), finally squeeze-and-excitation fusion(SEF) module and self-attention network(SAN) are combined to fuse features from different modalities for classification. Experiments on the CASIA-SURF dataset show that the new method based on MMFF achieves better performance compared with most existing methods.

多尺度增强特征融合的钢表面缺陷目标检测

针对轻量级目标检测算法在钢表面缺陷检测任务中识别精度低的问题,提出一种多尺度增强特征融合的钢表面缺陷目标检测算法。该算法采用提出的自适应加权融合模块为不同层级特征自适应计算融合权重,将深层语义与浅层细节进行加权融合,使得浅层特征在不丢失细节信息的同时获得丰富的深层语义。利用提出的空间特征增强模块从3个独立方向强化融合特征,通过引出残差旁路增强网络结构的稳定性,使卷积过程能够挖掘到更多的关键信息。根据先验框与真实框的整体交并程度为模型选择更为合适的训练样本。实验结果表明,该算法的检测精度达到80.47%,相比原始算法提升6.81%。该算法的参数量为2.36 M,计算量为952.67 MFLOPs,能快速且高精度检测钢材表面的缺陷信息,具有较高的应用价值。

基于改进YOLOv5s的车脸检测算法

针对车检站中车辆检测的实际需求,提出一种改进YOLOv5s的轻量级车脸检测算法。使用ShuffleNetV2网络作为Backbone,在保证模型检测精度的同时实现模型的轻量化与实时性;将通道-空间注意力(SA-Net)与跨通道注意力(Triplet)相结合,提出一种跨通道-空间注意力模块(SA-Triplet attention, STA),提高模型的检测精度;提出一种基于STA注意力模块的跨层特征融合模块(SA-Triplet attention feature fusion, STA-FF),进一步提高模型的检测精度。在自建车脸检测数据集Car-Data上进行实验,所提模型的平均检测精度达到了94.3%,检测速度达到了105.60 FPS,模型参数量为12.36 M。

融合摄食过程声像特征的鱼类摄食强度量化方法研究

基于鱼类摄食行为反馈的精准投喂是确保饲料高效利用与降低水体污染的有效手段,针对当前单一传感器难以精确测量鱼群摄食强度的难题,提出一种基于摄食过程声像特征融合的鱼类摄食强度量化方法。首先利用深度图包含的三维空间信息分析水体表层摄食鱼类数量,设计基于帧间差分运算的深度图能量变化测量系统表征鱼群摄食活跃程度;进而利用近红外光源因水面反射而导致的高亮度饱和点在近红外图中的位置变化测量水体流场的波动程度;同时利用音轨记录仪存储摄食音频。最后通过加权融合方式,综合摄食动态、水体流场变化、摄食音频三类具有不同物理属性的特征信息,精确量化了鱼群摄食强度,总体识别精确度达到97%。本文采用新的成像技术,取得分析速度与分析精度的最佳平衡,为精准投喂提供了一种鲁棒性强、分析速度快的实用方法。

基于UNet++卷积神经网络的断层识别

针对传统相干体属性和机器学习在复杂断裂识别能力差的问题,提出一种基于UNet++卷积神经网络的断层识别方法。该方法采用加权交叉熵损失函数做目标函数,使网络模型训练过程中避免了数据样本不平衡的问题,引入注意力机制和密集卷积块,以及更多的跳跃连接,更好地实现深层次断层语义信息和浅层次断层空间信息之间的特征融合,进而可以使UNet++网络模型更好地实现断层识别。实验结果表明,该网络模型将F1值提高到了92.38%, loss降低到0.012 0,可以更好地学习断层特征信息。将该模型应用于西南庄断层的识别中,结果表明,该方法可以更准确预测断层位置,在识别连续断层的准确率上有所提高,有效防止了地下噪音对于断层识别的不利影响,从而验证了UNet++网络模型在断层识别上具有一定的研究价值。

Boosting Multi-modal Ocular Recognition via Spatial Feature Reconstruction and Unsupervised Image Quality Estimation

In the daily application of an iris-recognition-at-a-distance(IAAD)system,many ocular images of low quality are acquired.As the iris part of these images is often not qualified for the recognition requirements,the more accessible periocular regions are a good complement for recognition.To further boost the performance of IAAD systems,a novel end-to-end framework for multi-modal ocular recognition is proposed.The proposed framework mainly consists of iris/periocular feature extraction and matching,unsupervised iris quality assessment,and a score-level adaptive weighted fusion strategy.First,ocular feature reconstruction(OFR)is proposed to sparsely reconstruct each probe image by high-quality gallery images based on proper feature maps.Next,a brand new unsupervised iris quality assessment method based on random multiscale embedding robustness is proposed.Different from the existing iris quality assess-ment methods,the quality of an iris image is measured by its robustness in the embedding space.At last,the fusion strategy exploits the iris quality score as the fusion weight to coalesce the complementary information from the iris and periocular regions.Extensive experi-mental results on ocular datasets prove that the proposed method is obviously better than unimodal biometrics,and the fusion strategy can significantly improve therecognition performance.

基于特征加权与融合的小样本遥感目标检测

基于深度卷积神经网络的目标检测器需要大量标注样本展开训练,针对训练样本数量不足导致目标检测器泛化能力较差的问题,基于元特征调制提出一种特征加权与融合的小样本遥感目标检测方法。首先,在元特征提取网络中嵌入瓶颈结构式特征学习模块C3,增加网络深度和感受野;其次,利用路径聚合网络(PAN)进行元特征融合,有效提升了网络对多尺度遥感目标的感知能力;最后,使用轻量级卷积神经网络学习原型向量以加权元特征,在轻量化模型的同时,利用模型已有知识快速微调模型,以适应对新类目标的检测。实验结果显示,在NWPU VHR-10和DIOR数据集上,该方法相比于FSODM方法,在新类对象上的平均检测精度分别提高了29.40%和11.78%。可视化结果表明,该方法在小样本遥感目标检测上效果更优。

基于加权多矩融合特征的矿物影像智能识别算法研究

随着数字识别技术在镜下影像分析的广泛应用,镜下物质类型的智能识别成为了一个微观分析的基础问题。镜下影像自动识别不仅能有效节约人工成本,还能提高识别效率。针对镜下矿物智能识别精度低的问题,以镜下影像的颜色矩、纹理矩以及形态RSTC矩3类指标为识别特征,以指标熵权和变异系数权为识别初始权,构建了一种多矩融合机器学习智能识别模型。选取磁铁矿、云母、方解石、黄铜等的影像集为第一类样本,以烧结矿中的玻璃相、铁酸钙等影像作为第二类样本,提取样本颜色矩、纹理矩和形状RSTC矩的特征,量化分析了特征在影像识别中的贡献率,开展了多矩融合机器学习智能识别试验。结果表明:不同类型特征指标对影像识别过程贡献率有明显差异,多矩融合机器学习智能识别模型具有较好的识别率和鲁棒性,能够明显提高影像识别精度,指标熵权和变异系数权为初始权能够明显促进算法快速收敛,减少识别时间,该研究为矿石镜下影像识别提供了新的方法。

基于特征点动态选择的三维人脸点云模型重建

针对典型的点云配准方法中伪特征点过多导致配准效率低和配准结果不精确的问题,提出一种基于特征点动态选择的三维人脸点云模型重建方法。该方法在粗配准阶段,采用动态特征矩阵求解法获取粗匹配特征变换矩阵以避免伪特征点的干扰。在精配准过程中,采用二次加权法向量垂直距离法在人脸流形表面选择更有效的特征点以减少伪特征点的数量,并采用基于特征融合与局部特征一致性的迭代最近点方法进行精配准。经过对比实验验证了算法的可行性,实验结果表明,该算法能够实现高精度且快速的三维人脸点云模型重建,且均方根误差达到1.8165 mm,相较其他算法,其在模型重建精度和效率方面都有所提升,具有良好的应用前景。