A Lightweight Convolutional Neural Network with Hierarchical Multi-Scale Feature Fusion for Image Classification

Convolutional neural networks (CNNs) are widely used in image classification tasks, but their increasing model size and computation make them challenging to implement on embedded systems with constrained hardware resources. To address this issue, the MobileNetV1 network was developed, which employs depthwise convolution to reduce network complexity. MobileNetV1 employs a stride of 2 in several convolutional layers to decrease the spatial resolution of feature maps, thereby lowering computational costs. However, this stride setting can lead to a loss of spatial information, particularly affecting the detection and representation of smaller objects or finer details in images. To maintain the trade-off between complexity and model performance, a lightweight convolutional neural network with hierarchical multi-scale feature fusion based on the MobileNetV1 network is proposed. The network consists of two main subnetworks. The first subnetwork uses a depthwise dilated separable convolution (DDSC) layer to learn imaging features with fewer parameters, which results in a lightweight and computationally inexpensive network. Furthermore, depthwise dilated convolution in DDSC layer effectively expands the field of view of filters, allowing them to incorporate a larger context. The second subnetwork is a hierarchical multi-scale feature fusion (HMFF) module that uses parallel multi-resolution branches architecture to process the input feature map in order to extract the multi-scale feature information of the input image. Experimental results on the CIFAR-10, Malaria, and KvasirV1 datasets demonstrate that the proposed method is efficient, reducing the network parameters and computational cost by 65.02% and 39.78%, respectively, while maintaining the network performance compared to the MobileNetV1 baseline.

Grid Side Distributed Energy Storage Cloud Group End Region Hierarchical Time-Sharing Configuration Algorithm Based onMulti-Scale and Multi Feature Convolution Neural Network

There is instability in the distributed energy storage cloud group end region on the power grid side.In order to avoid large-scale fluctuating charging and discharging in the power grid environment and make the capacitor components showa continuous and stable charging and discharging state,a hierarchical time-sharing configuration algorithm of distributed energy storage cloud group end region on the power grid side based on multi-scale and multi feature convolution neural network is proposed.Firstly,a voltage stability analysis model based onmulti-scale and multi feature convolution neural network is constructed,and the multi-scale and multi feature convolution neural network is optimized based on Self-OrganizingMaps(SOM)algorithm to analyze the voltage stability of the cloud group end region of distributed energy storage on the grid side under the framework of credibility.According to the optimal scheduling objectives and network size,the distributed robust optimal configuration control model is solved under the framework of coordinated optimal scheduling at multiple time scales;Finally,the time series characteristics of regional power grid load and distributed generation are analyzed.According to the regional hierarchical time-sharing configuration model of“cloud”,“group”and“end”layer,the grid side distributed energy storage cloud group end regional hierarchical time-sharing configuration algorithm is realized.The experimental results show that after applying this algorithm,the best grid side distributed energy storage configuration scheme can be determined,and the stability of grid side distributed energy storage cloud group end region layered timesharing configuration can be improved.

MSSTNet:Multi-scale facial videos pulse extraction network based on separable spatiotemporal convolution and dimension separable attention

Background The use of remote photoplethysmography(rPPG)to estimate blood volume pulse in a noncontact manner has been an active research topic in recent years.Existing methods are primarily based on a singlescale region of interest(ROI).However,some noise signals that are not easily separated in a single-scale space can be easily separated in a multi-scale space.Also,existing spatiotemporal networks mainly focus on local spatiotemporal information and do not emphasize temporal information,which is crucial in pulse extraction problems,resulting in insufficient spatiotemporal feature modelling.Methods Here,we propose a multi-scale facial video pulse extraction network based on separable spatiotemporal convolution(SSTC)and dimension separable attention(DSAT).First,to solve the problem of a single-scale ROI,we constructed a multi-scale feature space for initial signal separation.Second,SSTC and DSAT were designed for efficient spatiotemporal correlation modeling,which increased the information interaction between the long-span time and space dimensions;this placed more emphasis on temporal features.Results The signal-to-noise ratio(SNR)of the proposed network reached 9.58dB on the PURE dataset and 6.77dB on the UBFC-rPPG dataset,outperforming state-of-the-art algorithms.Conclusions The results showed that fusing multi-scale signals yielded better results than methods based on only single-scale signals.The proposed SSTC and dimension-separable attention mechanism will contribute to more accurate pulse signal extraction.

Multi-Scale Convolutional Gated Recurrent Unit Networks for Tool Wear Prediction in Smart Manufacturing

As an integrated application of modern information technologies and artificial intelligence,Prognostic and Health Management(PHM)is important for machine health monitoring.Prediction of tool wear is one of the symbolic applications of PHM technology in modern manufacturing systems and industry.In this paper,a multi-scale Convolutional Gated Recurrent Unit network(MCGRU)is proposed to address raw sensory data for tool wear prediction.At the bottom of MCGRU,six parallel and independent branches with different kernel sizes are designed to form a multi-scale convolutional neural network,which augments the adaptability to features of different time scales.These features of different scales extracted from raw data are then fed into a Deep Gated Recurrent Unit network to capture long-term dependencies and learn significant representations.At the top of the MCGRU,a fully connected layer and a regression layer are built for cutting tool wear prediction.Two case studies are performed to verify the capability and effectiveness of the proposed MCGRU network and results show that MCGRU outperforms several state-of-the-art baseline models.

Pedestrian attribute classification with multi-scale and multi-label convolutional neural networks

Pedestrian attribute classification from a pedestrian image captured in surveillance scenarios is challenging due to diverse clothing appearances,varied poses and different camera views. A multiscale and multi-label convolutional neural network( MSMLCNN) is proposed to predict multiple pedestrian attributes simultaneously. The pedestrian attribute classification problem is firstly transformed into a multi-label problem including multiple binary attributes needed to be classified. Then,the multi-label problem is solved by fully connecting all binary attributes to multi-scale features with logistic regression functions. Moreover,the multi-scale features are obtained by concatenating those featured maps produced from multiple pooling layers of the MSMLCNN at different scales. Extensive experiment results show that the proposed MSMLCNN outperforms state-of-the-art pedestrian attribute classification methods with a large margin.

Clothing Parsing Based on Multi-Scale Fusion and Improved Self-Attention Mechanism

Due to the lack of long-range association and spatial location information,fine details and accurate boundaries of complex clothing images cannot always be obtained by using the existing deep learning-based methods.This paper presents a convolutional structure with multi-scale fusion to optimize the step of clothing feature extraction and a self-attention module to capture long-range association information.The structure enables the self-attention mechanism to directly participate in the process of information exchange through the down-scaling projection operation of the multi-scale framework.In addition,the improved self-attention module introduces the extraction of 2-dimensional relative position information to make up for its lack of ability to extract spatial position features from clothing images.The experimental results based on the colorful fashion parsing dataset(CFPD)show that the proposed network structure achieves 53.68%mean intersection over union(mIoU)and has better performance on the clothing parsing task.

Lightweight Image Super-Resolution via Weighted Multi-Scale Residual Network

The tradeoff between efficiency and model size of the convolutional neural network(CNN)is an essential issue for applications of CNN-based algorithms to diverse real-world tasks.Although deep learning-based methods have achieved significant improvements in image super-resolution(SR),current CNNbased techniques mainly contain massive parameters and a high computational complexity,limiting their practical applications.In this paper,we present a fast and lightweight framework,named weighted multi-scale residual network(WMRN),for a better tradeoff between SR performance and computational efficiency.With the modified residual structure,depthwise separable convolutions(DS Convs)are employed to improve convolutional operations’efficiency.Furthermore,several weighted multi-scale residual blocks(WMRBs)are stacked to enhance the multi-scale representation capability.In the reconstruction subnetwork,a group of Conv layers are introduced to filter feature maps to reconstruct the final high-quality image.Extensive experiments were conducted to evaluate the proposed model,and the comparative results with several state-of-the-art algorithms demonstrate the effectiveness of WMRN.

Defect Detection Algorithm of Patterned Fabrics Based on Convolutional Neural Network

The background pattern of patterned fabrics is complex,which has a great interference in the extraction of defect features.Traditional machine vision algorithms rely on artificially designed features,which are greatly affected by background patterns and are difficult to effectively extract flaw features.Therefore,a convolutional neural network(CNN)with automatic feature extraction is proposed.On the basis of the two-stage detection model Faster R-CNN,Resnet-50 is used as the backbone network,and the problem of flaws with extreme aspect ratio is solved by improving the initialization algorithm of the prior frame aspect ratio,and the improved multi-scale model is designed to improve detection of small defects.The cascade R-CNN is introduced to improve the accuracy of defect detection,and the online hard example mining(OHEM)algorithm is used to strengthen the learning of hard samples to reduce the interference of complex backgrounds on the defect detection of patterned fabrics,and construct the focal loss as a loss function to reduce the impact of sample imbalance.In order to verify the effectiveness of the improved algorithm,a defect detection comparison experiment was set up.The experimental results show that the accuracy of the defect detection algorithm of patterned fabrics in this paper can reach 95.7%,and it can accurately locate the defect location and meet the actual needs of the factory.

A Multi-Scale Network with the Encoder-Decoder Structure for CMR Segmentation

Cardiomyopathy is one of the most serious public health threats.The precise structural and functional cardiac measurement is an essential step for clinical diagnosis and follow-up treatment planning.Cardiologists are often required to draw endocardial and epicardial contours of the left ventricle(LV)manually in routine clinical diagnosis or treatment planning period.This task is time-consuming and error-prone.Therefore,it is necessary to develop a fully automated end-to-end semantic segmentation method on cardiac magnetic resonance(CMR)imaging datasets.However,due to the low image quality and the deformation caused by heartbeat,there is no effective tool for fully automated end-to-end cardiac segmentation task.In this work,we propose a multi-scale segmentation network(MSSN)for left ventricle segmentation.It can effectively learn myocardium and blood pool structure representations from 2D short-axis CMR image slices in a multi-scale way.Specifically,our method employs both parallel and serial of dilated convolution layers with different dilation rates to capture multi-scale semantic features.Moreover,we design graduated up-sampling layers with subpixel layers as the decoder to reconstruct lost spatial information and produce accurate segmentation masks.We validated our method using 164 T1 Mapping CMR images and showed that it outperforms the advanced convolutional neural network(CNN)models.In validation metrics,we archived the Dice Similarity Coefficient(DSC)metric of 78.96%.

Identification of tomato leaf diseases using convolutional neural network with multi-scale and feature reuse

Various diseases seriously affect the quality and yield of tomatoes. Fast and accurate identification of disease types is of great significance for the development of smart agriculture. Many Convolution Neural Network (CNN) models have been applied to the identification of tomato leaf diseases and achieved good results. However, some of these are executed at the cost of large calculation time and huge storage space. This study proposed a lightweight CNN model named MFRCNN, which is established by the multi-scale and feature reuse structure rather than simply stacking convolution layer by layer. To examine the model performances, two types of tomato leaf disease datasets were collected. One is the laboratory-based dataset, including one healthy and nine diseases, and the other is the field-based dataset, including five kinds of diseases. Afterward, the proposed MFRCNN and some popular CNN models (AlexNet, SqueezeNet, VGG16, ResNet18, and GoogLeNet) were tested on the two datasets. The results showed that compared to traditional models, the MFRCNN achieved the optimal performance, with an accuracy of 99.01% and 98.75% in laboratory and field datasets, respectively. The MFRCNN not only had the highest accuracy but also had relatively less computing time and few training parameters. Especially in terms of storage space, the MFRCNN model only needs 2.7 MB of space. Therefore, this work provides a novel solution for plant disease diagnosis, which is of great importance for the development of plant disease diagnosis systems on low-performance terminals.

密度导向的点云动态图卷积网络

针对现有主流网络对于点云局部特征提取的能力不足,以及在特征提取过程中未考虑点云密度的问题,提出一种密度导向的点云动态图卷积网络.首先提出点云局部密度指数的概念,衡量点及其邻域点在相应的空间位置中的密集程度;然后利用局部密度指数动态赋予每个点一个膨胀因子,提出密度导向的动态点分组方法对点云构建局部图结构,对每个局部图结构构造动态边缘卷积模块进行特征的提取与聚合,既提取了点云的几何特征,又实现了置换不变性;最后采用残差网络的思想优化图神经网络的过平滑问题.实验结果表明,在分类数据集ModelNet40与ScanObjectNN上,所提网络的分类准确率分别为93.5%和82.2%;在分割数据集ShapeNet与S3DIS上,该网络的平均交并比分别为85.6%和60.4%,均高于DGCNN等主流网络;所提网络在多项任务中的精度都得到显著提升,且在处理密度不均的点云时有较好的鲁棒性,验证了所提算法的可行性与有效性.

基于卷积神经网络的“舌边白涎”舌象识别研究

目的通过机器学习分析“舌边白涎”舌象特性,对舌象进行局部特征识别研究,探讨卷积神经网络算法在舌象识别应用中的性能。方法使用Python进行图像预处理,搭建用于舌象识别的视觉几何组16层(visual geometry group 16,VGG16)卷积神经网络模型,分析其对“舌边白涎”舌象鉴别分析的效果,并结合热力图分析“舌边白涎”典型舌象表现。结果基于PyTorch框架,进行卷积神经网络的舌象鉴别研究,VGG16及残差网络50层(residual network 50,ResNet50)模型验证准确率均较高,达到80%以上,且ResNet50模型优于VGG16模型,可为舌象识别提供一定参考。基于加权梯度类激活映射(gradient-weighted class activation mapping,Grad-CAM)技术,通过舌苔舌色差异分布的网络可视化,有助于直观进行模型评估分析。结论基于卷积神经网络模型对舌象数据库进行分析,实现“舌边白涎”舌象识别,有助于临床诊疗的客观化辅助分析,为舌诊智能化发展提供一定借鉴。

基于伪点云特征增强的多模态三维目标检测方法

环境感知是自动驾驶汽车落地的关键技术之一,它对于提高自动驾驶汽车的安全性和可靠性至关重要.三维目标检测是其中的一项核心任务,旨在识别和定位三维空间中的物体,为后续决策提供重要的信息.点云和图像是该任务最常用的输入数据,点云由三维空间中不规则分布的点组成,而图像则是由二维空间上规则分布的像素组成.因此,点云和图像之间难以进行有效的融合.而伪点云作为一种点云表征的图像信息,近几年受到了该领域学者的广泛关注.现阶段基于伪点云的三维目标检测方法还存在伪点云特征提取粗糙和相应感兴趣区域(Region-of-Intersts,RoI)特征表征能力差的问题.本文针对上述问题开展研究,分别提出细粒度注意力卷积和多尺度分组稀疏卷积.细粒度注意力卷积将规则图像处理中常用的深度可分离卷积引入不规则点云的处理流程,并在此基础上嵌入通道和分组注意力机制,进行精细的特征提取,增强伪点云特征;多尺度分组稀疏卷积将格网池化后的Ro I特征分组,进行差异化特征学习,获取不同尺度的Ro I特征,增强伪点云Ro I格网特征的表征能力.基于此,本文在SFD(Sparse Fuse Dense)网络的伪点云特征提取流程中引入细粒度注意力卷积,同时在其伪点云Ro I特征学习流程中引入多尺度分组稀疏卷积,构建SFD++多模态三维目标检测网络.在权威KITTI自动驾驶数据集上的实验结果表明,SFD++每秒可以处理8.33帧数据,其精度在简单、中等和困难的三维汽车检测上达到95.74%、88.80%和86.04%,比次优SFD的精度高出0.15%、0.84%和0.58%.除此之外,一系列消融和补充实验结果验证了所提出卷积的有效性和相关参数设置的合理性.

融合自注意力和图卷积的多视图群组推荐

为了解决大多数现有的群组推荐仅仅从群组和用户的单一交互中学习群组表示,以及固定融合策略难以动态调整权重的问题。提出了一种融合自注意力和图卷积的多视图群组推荐模型(MVGR),设计了成员级、项目级和组级三个不同的视图,来捕捉群组、用户和项目三者之间的高阶交互信息,缓解数据稀疏问题,增强群组表示建模过程;对于项目级视图,利用基于二分图的图卷积神经网络来学习群组偏好向量以及项目嵌入;进一步提出了自适应融合组件来动态调整不同视图权重,得到最终的群组偏好向量。在两个真实数据集上的实验结果表明,与基线模型相比,MVGR模型的命中率(HR)和归一化折损累计增益(NDCG)在Mafengwo数据集上平均提高了8.89个百分点和1.56个百分点,在CAMRa2011数据集上平均提高了2.79个百分点和2.7个百分点。

基于RegNet-CSAM与ZOA-KELM模型的滚动轴承故障诊断

针对现有深度卷积神经网络对滚动轴承混合故障诊断效果不佳以及模型复杂度过高导致计算成本过大等问题,提出了一种基于RegNet-CSAM与ZOA-KELM模型的滚动轴承故障诊断方法。该模型由RegNet-CSAM网络和ZOA-KELM分类算法组成。首先,将融合了通道和空间特征的注意力机制CSAM与组卷积残差模块结合,提升该结构的表征能力,由此构建的RegNet-CSAM网络,模型复杂度为0.48GF;其次,在分类阶段将斑马优化核极限学习机(ZOA-KELM)替代原来网络中使用的Softmax函数完成最后的分类任务。滚动轴承故障诊断试验结果表明,RegNet网络对滚动轴承混合故障样本容易产生误判,CSAM的融入虽将RegNet网络的分类精度进一步提高,但是仍然存在一定程度的滚动轴承混合故障误判问题;而将ZOA-KELM替代Softmax函数后再对RegNet-CSAM网络输出特征进行分类,能够有效识别出滚动轴承的单一和混合故障,准确率达到了99.92%。所提方法对比其他网络,诊断精度最大提升5.02%,模型复杂度最大缩减32倍。

基于深度学习及改进模糊KMeans的寻常型银屑病智能诊断方法

为了解决寻常型银屑病在样本分布不平衡的数据中可能会导致的深度学习模型诊断效果下降等问题,通过结合改进模糊KMeans聚类算法对高聚类复杂度数据的处理能力以及Visual Geometry Group 13(VGG13)深度卷积神经网络模型的预测能力,提出一种基于改进模糊KMeans聚类算法的VGG13深度卷积神经网络(VGG13-KMeans)模型,并将其应用于寻常型银屑病的诊断任务中。实验结果表明,相较于VGG13以及ResNet18两种方法,本文方法更适用于对银屑病特征的识别。

基于空洞卷积自注意力机制的煤岩显微组分组识别模型

基于深度学习的识别模型是目前解决煤岩显微组分组识别问题的主要手段,但这些模型在计算过程中参数不断堆叠,导致模型的算力需求增加,影响模型的训练效率。针对上述问题,构建了一种基于空洞卷积自注意力(DCSA)机制的改进Swin-Transformer模型——DA-ViT。首先,为了在加强煤岩显微组分组图像的局部特征信息的同时保留其二维空间信息,提出了DCSA机制,通过对煤岩显微图像的大尺寸卷积核进行多尺度分解,加强了煤岩显微图像不同区域像素之间的联系,显著降低了图像注意力的参数量,降低率为81.18%。然后,为了加强煤岩显微组分组图像间的形态特征关联性,将DCSA和改进的Swin-Transformer框架相结合,提出了DA-ViT识别模型。实验验证结果表明,与现有的其他识别模型相比,DA-ViT模型在提高预测结果准确率的同时,可显著降低模型的算力需求,其像素准确率(PA)和平均交并比(mIoU)的最大值分别为92.14%和63.18%,模型参数总量(Params)和浮点运算次数(FLOPs)的最小值分别为4.95×106和8.99×109。

主动学习联合聚类分组网络的高光谱遥感图像分类

基于卷积神经网络的高光谱图像(Hyperspectral Image,HSI)分类面临网络参数量大,带类标样本少的现状,针对这些问题,提出了基于主动学习和聚类分组网络的高光谱图像分类方法(AL-CGNet)。AL-CGNet采用主动学习和聚类联合卷积神经网络进行HSI的特征提取与分类,设计了基于分组卷积的轻量化网络模型以降低网络参数量。对线性判别分析(Linear Discriminant Analysis,LDA)降维后的高光谱图像采用小批量K均值聚类算法划分成不同的簇,并用簇中心的光谱特征代表不同的簇,以利用无类标样本的信息。在分组网络中将生成的特征图沿光谱维划分成一系列小组,每组通过多个残差块依次提取空间-光谱特征,这种分组策略可以充分利用波段的冗余性和差异性,降低网络参数,并实现轻量化。最后,采用主动学习选取信息量大的样本作为训练样本集,以解决带类标样本少的问题。实验结果表明,AL-CGNet在使用相同比例的6%训练样本的情况下,在Indian Pines,Botswana,Houston 3个数据集下的分类结果明显高于ClusterCNN,SSRN和HybridSN等方法,其OA分别为99.57%,99.23%,98.82%,甚至在训练样本更少5%的小样本情况下也是有效的。该方法不仅大大提高了HSI的分类效率,在获得高精度的同时还能高效率地提取特征。

深度学习轻量化侦察图像压缩网络

为了满足小型侦察平台对低复杂度图像编码算法的应用需求,提出基于深度学习轻量化侦察图像压缩网络。轻量化侦察图像压缩网络编码端利用三个卷积模块直接将图像映射为服从均匀分布的二进制码流,得到压缩数据;在卷积模块中采用深度可分离卷积、分组卷积+通道重排等方式降低了编码端参数量和计算量。轻量化侦察图像压缩网络解码端采用转置卷积和残差连接等方式提高特征提取能力,进而提高解码图像质量。对分辨率为128×128实际采集图像的测试结果表明,与JPGE2000算法相比,基于深度学习轻量化侦察图像压缩网络PSNR提高了3.85 dB,编码时间降低了91%,实现了图像的轻量化编码压缩。

基于CNN和Group Normalization的校园垃圾图像分类

为解决大学校园的垃圾回收分类问题,提出了一种基于卷积神经网络和归一化技术的垃圾图像分类方法,不需要对输入的图像进行复杂的处理,网络模型即可根据算法提取图像特征,通过加入群组归一化(Group Normalization)和网络模型各层之间的协作,克服传统分类算法的缺点,实现对垃圾图像的分类。实验表明,该识别方法具有较高准确率,可以较好识别不可回收及可回收垃圾。