A Lightweight Convolutional Neural Network with Hierarchical Multi-Scale Feature Fusion for Image Classification

Convolutional neural networks (CNNs) are widely used in image classification tasks, but their increasing model size and computation make them challenging to implement on embedded systems with constrained hardware resources. To address this issue, the MobileNetV1 network was developed, which employs depthwise convolution to reduce network complexity. MobileNetV1 employs a stride of 2 in several convolutional layers to decrease the spatial resolution of feature maps, thereby lowering computational costs. However, this stride setting can lead to a loss of spatial information, particularly affecting the detection and representation of smaller objects or finer details in images. To maintain the trade-off between complexity and model performance, a lightweight convolutional neural network with hierarchical multi-scale feature fusion based on the MobileNetV1 network is proposed. The network consists of two main subnetworks. The first subnetwork uses a depthwise dilated separable convolution (DDSC) layer to learn imaging features with fewer parameters, which results in a lightweight and computationally inexpensive network. Furthermore, depthwise dilated convolution in DDSC layer effectively expands the field of view of filters, allowing them to incorporate a larger context. The second subnetwork is a hierarchical multi-scale feature fusion (HMFF) module that uses parallel multi-resolution branches architecture to process the input feature map in order to extract the multi-scale feature information of the input image. Experimental results on the CIFAR-10, Malaria, and KvasirV1 datasets demonstrate that the proposed method is efficient, reducing the network parameters and computational cost by 65.02% and 39.78%, respectively, while maintaining the network performance compared to the MobileNetV1 baseline.

A multi-scale convolutional auto-encoder and its application in fault diagnosis of rolling bearings

Aiming at the difficulty of fault identification caused by manual extraction of fault features of rotating machinery,a one-dimensional multi-scale convolutional auto-encoder fault diagnosis model is proposed,based on the standard convolutional auto-encoder.In this model,the parallel convolutional and deconvolutional kernels of different scales are used to extract the features from the input signal and reconstruct the input signal;then the feature map extracted by multi-scale convolutional kernels is used as the input of the classifier;and finally the parameters of the whole model are fine-tuned using labeled data.Experiments on one set of simulation fault data and two sets of rolling bearing fault data are conducted to validate the proposed method.The results show that the model can achieve 99.75%,99.3%and 100%diagnostic accuracy,respectively.In addition,the diagnostic accuracy and reconstruction error of the one-dimensional multi-scale convolutional auto-encoder are compared with traditional machine learning,convolutional neural networks and a traditional convolutional auto-encoder.The final results show that the proposed model has a better recognition effect for rolling bearing fault data.

Multi-Scale Convolutional Gated Recurrent Unit Networks for Tool Wear Prediction in Smart Manufacturing

As an integrated application of modern information technologies and artificial intelligence,Prognostic and Health Management(PHM)is important for machine health monitoring.Prediction of tool wear is one of the symbolic applications of PHM technology in modern manufacturing systems and industry.In this paper,a multi-scale Convolutional Gated Recurrent Unit network(MCGRU)is proposed to address raw sensory data for tool wear prediction.At the bottom of MCGRU,six parallel and independent branches with different kernel sizes are designed to form a multi-scale convolutional neural network,which augments the adaptability to features of different time scales.These features of different scales extracted from raw data are then fed into a Deep Gated Recurrent Unit network to capture long-term dependencies and learn significant representations.At the top of the MCGRU,a fully connected layer and a regression layer are built for cutting tool wear prediction.Two case studies are performed to verify the capability and effectiveness of the proposed MCGRU network and results show that MCGRU outperforms several state-of-the-art baseline models.

Pedestrian attribute classification with multi-scale and multi-label convolutional neural networks

Pedestrian attribute classification from a pedestrian image captured in surveillance scenarios is challenging due to diverse clothing appearances,varied poses and different camera views. A multiscale and multi-label convolutional neural network( MSMLCNN) is proposed to predict multiple pedestrian attributes simultaneously. The pedestrian attribute classification problem is firstly transformed into a multi-label problem including multiple binary attributes needed to be classified. Then,the multi-label problem is solved by fully connecting all binary attributes to multi-scale features with logistic regression functions. Moreover,the multi-scale features are obtained by concatenating those featured maps produced from multiple pooling layers of the MSMLCNN at different scales. Extensive experiment results show that the proposed MSMLCNN outperforms state-of-the-art pedestrian attribute classification methods with a large margin.

MSSTNet:Multi-scale facial videos pulse extraction network based on separable spatiotemporal convolution and dimension separable attention

Background The use of remote photoplethysmography(rPPG)to estimate blood volume pulse in a noncontact manner has been an active research topic in recent years.Existing methods are primarily based on a singlescale region of interest(ROI).However,some noise signals that are not easily separated in a single-scale space can be easily separated in a multi-scale space.Also,existing spatiotemporal networks mainly focus on local spatiotemporal information and do not emphasize temporal information,which is crucial in pulse extraction problems,resulting in insufficient spatiotemporal feature modelling.Methods Here,we propose a multi-scale facial video pulse extraction network based on separable spatiotemporal convolution(SSTC)and dimension separable attention(DSAT).First,to solve the problem of a single-scale ROI,we constructed a multi-scale feature space for initial signal separation.Second,SSTC and DSAT were designed for efficient spatiotemporal correlation modeling,which increased the information interaction between the long-span time and space dimensions;this placed more emphasis on temporal features.Results The signal-to-noise ratio(SNR)of the proposed network reached 9.58dB on the PURE dataset and 6.77dB on the UBFC-rPPG dataset,outperforming state-of-the-art algorithms.Conclusions The results showed that fusing multi-scale signals yielded better results than methods based on only single-scale signals.The proposed SSTC and dimension-separable attention mechanism will contribute to more accurate pulse signal extraction.

A non-invasive diagnostic method of cavity detuning based on a convolutional neural network

As modern accelerator technologies advance toward more compact sizes,conventional invasive diagnostic methods of cavity detuning introduce negligible interference in measurements and run the risk of harming structural surfaces.To overcome these difficulties,this study developed a non-invasive diagnostic method using knowledge of scattering parameters with a convolutional neural network and the interior point method.Meticulous construction and training of the neural network led to remarkable results on three typical acceleration structures:a 13-cell S-band standing-wave linac,a 12-cell X-band traveling-wave linac,and a 3-cell X-band RF gun.The trained networks significantly reduced the burden of the tuning process,freed researchers from tedious tuning tasks,and provided a new perspective for the tuning of side-coupling,semi-enclosed,and total-enclosed structures.

Two Stages Segmentation Algorithm of Breast Tumor in DCE-MRI Based on Multi-Scale Feature and Boundary Attention Mechanism

Nuclearmagnetic resonance imaging of breasts often presents complex backgrounds.Breast tumors exhibit varying sizes,uneven intensity,and indistinct boundaries.These characteristics can lead to challenges such as low accuracy and incorrect segmentation during tumor segmentation.Thus,we propose a two-stage breast tumor segmentation method leveraging multi-scale features and boundary attention mechanisms.Initially,the breast region of interest is extracted to isolate the breast area from surrounding tissues and organs.Subsequently,we devise a fusion network incorporatingmulti-scale features and boundary attentionmechanisms for breast tumor segmentation.We incorporate multi-scale parallel dilated convolution modules into the network,enhancing its capability to segment tumors of various sizes through multi-scale convolution and novel fusion techniques.Additionally,attention and boundary detection modules are included to augment the network’s capacity to locate tumors by capturing nonlocal dependencies in both spatial and channel domains.Furthermore,a hybrid loss function with boundary weight is employed to address sample class imbalance issues and enhance the network’s boundary maintenance capability through additional loss.Themethod was evaluated using breast data from 207 patients at RuijinHospital,resulting in a 6.64%increase in Dice similarity coefficient compared to the benchmarkU-Net.Experimental results demonstrate the superiority of the method over other segmentation techniques,with fewer model parameters.

Lightweight Image Super-Resolution via Weighted Multi-Scale Residual Network

The tradeoff between efficiency and model size of the convolutional neural network(CNN)is an essential issue for applications of CNN-based algorithms to diverse real-world tasks.Although deep learning-based methods have achieved significant improvements in image super-resolution(SR),current CNNbased techniques mainly contain massive parameters and a high computational complexity,limiting their practical applications.In this paper,we present a fast and lightweight framework,named weighted multi-scale residual network(WMRN),for a better tradeoff between SR performance and computational efficiency.With the modified residual structure,depthwise separable convolutions(DS Convs)are employed to improve convolutional operations’efficiency.Furthermore,several weighted multi-scale residual blocks(WMRBs)are stacked to enhance the multi-scale representation capability.In the reconstruction subnetwork,a group of Conv layers are introduced to filter feature maps to reconstruct the final high-quality image.Extensive experiments were conducted to evaluate the proposed model,and the comparative results with several state-of-the-art algorithms demonstrate the effectiveness of WMRN.

Defect Detection Algorithm of Patterned Fabrics Based on Convolutional Neural Network

The background pattern of patterned fabrics is complex,which has a great interference in the extraction of defect features.Traditional machine vision algorithms rely on artificially designed features,which are greatly affected by background patterns and are difficult to effectively extract flaw features.Therefore,a convolutional neural network(CNN)with automatic feature extraction is proposed.On the basis of the two-stage detection model Faster R-CNN,Resnet-50 is used as the backbone network,and the problem of flaws with extreme aspect ratio is solved by improving the initialization algorithm of the prior frame aspect ratio,and the improved multi-scale model is designed to improve detection of small defects.The cascade R-CNN is introduced to improve the accuracy of defect detection,and the online hard example mining(OHEM)algorithm is used to strengthen the learning of hard samples to reduce the interference of complex backgrounds on the defect detection of patterned fabrics,and construct the focal loss as a loss function to reduce the impact of sample imbalance.In order to verify the effectiveness of the improved algorithm,a defect detection comparison experiment was set up.The experimental results show that the accuracy of the defect detection algorithm of patterned fabrics in this paper can reach 95.7%,and it can accurately locate the defect location and meet the actual needs of the factory.

A Multi-Scale Network with the Encoder-Decoder Structure for CMR Segmentation

Cardiomyopathy is one of the most serious public health threats.The precise structural and functional cardiac measurement is an essential step for clinical diagnosis and follow-up treatment planning.Cardiologists are often required to draw endocardial and epicardial contours of the left ventricle(LV)manually in routine clinical diagnosis or treatment planning period.This task is time-consuming and error-prone.Therefore,it is necessary to develop a fully automated end-to-end semantic segmentation method on cardiac magnetic resonance(CMR)imaging datasets.However,due to the low image quality and the deformation caused by heartbeat,there is no effective tool for fully automated end-to-end cardiac segmentation task.In this work,we propose a multi-scale segmentation network(MSSN)for left ventricle segmentation.It can effectively learn myocardium and blood pool structure representations from 2D short-axis CMR image slices in a multi-scale way.Specifically,our method employs both parallel and serial of dilated convolution layers with different dilation rates to capture multi-scale semantic features.Moreover,we design graduated up-sampling layers with subpixel layers as the decoder to reconstruct lost spatial information and produce accurate segmentation masks.We validated our method using 164 T1 Mapping CMR images and showed that it outperforms the advanced convolutional neural network(CNN)models.In validation metrics,we archived the Dice Similarity Coefficient(DSC)metric of 78.96%.

Clothing Parsing Based on Multi-Scale Fusion and Improved Self-Attention Mechanism

Due to the lack of long-range association and spatial location information,fine details and accurate boundaries of complex clothing images cannot always be obtained by using the existing deep learning-based methods.This paper presents a convolutional structure with multi-scale fusion to optimize the step of clothing feature extraction and a self-attention module to capture long-range association information.The structure enables the self-attention mechanism to directly participate in the process of information exchange through the down-scaling projection operation of the multi-scale framework.In addition,the improved self-attention module introduces the extraction of 2-dimensional relative position information to make up for its lack of ability to extract spatial position features from clothing images.The experimental results based on the colorful fashion parsing dataset(CFPD)show that the proposed network structure achieves 53.68%mean intersection over union(mIoU)and has better performance on the clothing parsing task.

A multi-scale convolutional neural network for bearing compound fault diagnosis under various noise conditions

Recently,with the urgent demand for data-driven approaches in practical industrial scenarios,the deep learning diagnosis model in noise environments has attracted increasing attention.However,the existing research has two limitations:(1)the complex and changeable environmental noise,which cannot ensure the high-performance diagnosis of the model in different noise domains and(2)the possibility of multiple faults occurring simultaneously,which brings challenges to the model diagnosis.This paper presents a novel anti-noise multi-scale convolutional neural network(AM-CNN)for solving the issue of compound fault diagnosis under different intensity noises.First,we propose a residual pre-processing block according to the principle of noise superposition to process the input information and present the residual loss to construct a new loss function.Additionally,considering the strong coupling of input information,we design a multi-scale convolution block to realize multi-scale feature extraction for enhancing the proposed model’s robustness and effectiveness.Finally,a multi-label classifier is utilized to simultaneously distinguish multiple bearing faults.The proposed AM-CNN is verified under our collected compound fault dataset.On average,AM-CNN improves 39.93%accuracy and 25.84%F1-macro under the no-noise working condition and 45.67%accuracy and 27.72%F1-macro under different intensity noise working conditions compared with the existing methods.Furthermore,the experimental results show that AM-CNN can achieve good cross-domain performance with 100%accuracy and 100%F1-macro.Thus,AM-CNN has the potential to be an accurate and stable fault diagnosis tool.

Land cover classification from remote sensing images based on multi-scale fully convolutional network

Although the Convolutional Neural Network(CNN)has shown great potential for land cover classification,the frequently used single-scale convolution kernel limits the scope of informa-tion extraction.Therefore,we propose a Multi-Scale Fully Convolutional Network(MSFCN)with a multi-scale convolutional kernel as well as a Channel Attention Block(CAB)and a Global Pooling Module(GPM)in this paper to exploit discriminative representations from two-dimensional(2D)satellite images.Meanwhile,to explore the ability of the proposed MSFCN for spatio-temporal images,we expand our MSFCN to three-dimension using three-dimensional(3D)CNN,capable of harnessing each land cover category’s time series interac-tion from the reshaped spatio-temporal remote sensing images.To verify the effectiveness of the proposed MSFCN,we conduct experiments on two spatial datasets and two spatio-temporal datasets.The proposed MSFCN achieves 60.366%on the WHDLD dataset and 75.127%on the GID dataset in terms of mIoU index while the figures for two spatio-temporal datasets are 87.753%and 77.156%.Extensive comparative experiments and abla-tion studies demonstrate the effectiveness of the proposed MSFCN.

An efficient projection defocus algorithm based on multi-scale convolution kernel templates

The focal problems of projection include out-of-focus projection images from the projector caused by incomplete mechanical focus and screen-door effects produced by projection pixilation. To eliminate these defects and enhance the imaging quality and clarity of projectors, a novel adaptive projection defocus algorithm is proposed based on multi-scale convolution kernel templates. This algorithm applies the improved Sobel-Tenengrad focus evaluation function to calculate the sharpness degree of intensity equalization and then constructs multi-scale defocus convolution kernels to remap and render the defocus projection image. The resulting projection defocus corrected images can eliminate out-of-focus effects and improve the sharpness of uncorrected images. Experiments show that the algorithm works quickly and robustly and that it not only effectively eliminates visual artifacts and can run on a self-designed smart projection system in real time but also significantly improves the resolution and clarity of the observer's visual perception.

Identification of tomato leaf diseases using convolutional neural network with multi-scale and feature reuse

Various diseases seriously affect the quality and yield of tomatoes. Fast and accurate identification of disease types is of great significance for the development of smart agriculture. Many Convolution Neural Network (CNN) models have been applied to the identification of tomato leaf diseases and achieved good results. However, some of these are executed at the cost of large calculation time and huge storage space. This study proposed a lightweight CNN model named MFRCNN, which is established by the multi-scale and feature reuse structure rather than simply stacking convolution layer by layer. To examine the model performances, two types of tomato leaf disease datasets were collected. One is the laboratory-based dataset, including one healthy and nine diseases, and the other is the field-based dataset, including five kinds of diseases. Afterward, the proposed MFRCNN and some popular CNN models (AlexNet, SqueezeNet, VGG16, ResNet18, and GoogLeNet) were tested on the two datasets. The results showed that compared to traditional models, the MFRCNN achieved the optimal performance, with an accuracy of 99.01% and 98.75% in laboratory and field datasets, respectively. The MFRCNN not only had the highest accuracy but also had relatively less computing time and few training parameters. Especially in terms of storage space, the MFRCNN model only needs 2.7 MB of space. Therefore, this work provides a novel solution for plant disease diagnosis, which is of great importance for the development of plant disease diagnosis systems on low-performance terminals.

Image Denoising Using Dual Convolutional Neural Network with Skip Connection

In recent years, deep convolutional neural networks have shown superior performance in image denoising. However, deep network structures often come with a large number of model parameters, leading to high training costs and long inference times, limiting their practical application in denoising tasks. This paper proposes a new dual convolutional denoising network with skip connections(DECDNet), which achieves an ideal balance between denoising effect and network complexity. The proposed DECDNet consists of a noise estimation network, a multi-scale feature extraction network, a dual convolutional neural network, and dual attention mechanisms. The noise estimation network is used to estimate the noise level map, and the multi-scale feature extraction network is combined to improve the model's flexibility in obtaining image features. The dual convolutional neural network branch design includes convolution and dilated convolution interactive connections, with the lower branch consisting of dilated convolution layers, and both branches using skip connections. Experiments show that compared with other models, the proposed DECDNet achieves superior PSNR and SSIM values at all compared noise levels, especially at higher noise levels, showing robustness to images with higher noise levels. It also demonstrates better visual effects, maintaining a balance between denoising and detail preservation.

基于短源声波管道的鼻腔三维结构重塑方法

声波反射技术是一种非侵入性测量技术,并且可通过回传的反射波构造物体内部结构,所以在医学测量鼻腔结构方向有着天然优势。然而,该技术中的长源声波管不便于携带以及成本较高的问题制约了该方法在医学方向的应用。因此,本研究提出了基于短源声波管道的鼻腔三维结构重塑方法以解决这些问题。本研究通过分析短源声波管道中的脉冲声波传播特性,解决了短源声波管道分离入射波与反射波困难的问题;并且采用了校正直流偏移分量和引入约束因子抑制高频噪声的方法对鼻腔声波信号进行预处理;最后利用鼻腔三维结构分层重塑方法重塑出鼻腔结构二维曲线并进行了三维可视化。实验结果表明:约束因子对重塑鼻腔结果有较大的影响,本文确定了约束因子的最佳取值范围。

基于多尺度空洞卷积结构的路面裂缝分割方法

为了实现道路裂缝的自动化检测,改善现有裂缝分割模型存在分割不连续与嘈杂背景误分割等问题,提出了一种基于多尺度空洞卷积结构的裂缝分割模型MAC-UNet。以UNet作为基础网络,首先提出了多尺度空洞卷积结构,替换编码器与解码器中的双卷积结构,提升了网络对复杂拓扑结构的分割性能。然后,构建了交叉注意力机制,使用金字塔注意力模块代替编、解码器之间的跳跃连接,保留因池化丢失的空间特征。增加通道注意力引导多尺度信息,有效地融合到解码器特征中,使得恢复裂缝时,细节更加丰富,定位更准确。最后,在道路裂缝数据集CFD和GAPS384上与FCN、PSPNet等5种方法进行试验对比,相较于UNet,在CFD数据集上,MIOU和Kappa系数分别提升了8.4%和8.52%。在GAPS384数据集上,分别提升了6.84%和8.23%,对于道路裂缝的分割更加清晰与完整。结果表明:与主流的分割算法相比,所提出算法的识别精度方面具有较明显的优势,在光照不均匀、各种噪音干扰、背景灰度水平不同的情况下,所提模型仍然能够获取稳定的检测结果,能够应对复杂裂缝分割问题,并且可视化裂缝检测误差较小,符合实际工程需求,且模型体积较小,具有一定的工程应用价值。

基于多尺度双重自注意力的遥感影像变化检测

针对遥感影像地物目标尺度不一、上下文信息不足和边缘细节信息难以恢复等问题,提出一种基于多尺度双重自注意力的像素级变化检测网络(Pixel-based change detection Network,PixelNet)实现遥感影像变化检测任务。一方面,使用基于混合空洞卷积的多尺度特征金字塔提取卷积特征,并加入双重自注意力模块获取通道和空间注意力,兼顾细节和语义信息的同时增加特征感受野,进一步增加了全局上下文信息。另一方面,为了优化地物目标的边界圆滑模糊问题,通过边缘感知损失与加权对比损失的自动化联合训练,实现新的边缘修复模块。针对样本不均衡问题提出了带阈值的加权均衡采样的数据处理策略,以减轻变化像素数目远远小于未变化像素数目造成的网络训练倾斜问题。在遥感影像数据集CDD和LEVIR-CD上通过实验证明,所提像素级变化检测网络PixelNet在遥感变化检测任务上的主观视觉效果及客观评价指标优于SOTA的检测结果。在CDD数据集上检测精度达到98.0%,F1分数达到96.7%;在LEVIR-CD数据集上检测精度达到95.8%,F1分数为87.2%。该网络有效解决了遥感变化检测中样本不平衡、双时相特征上下文信息不足、边缘难例分类错误等问题。

A novel multi-resolution network for the open-circuit faults diagnosis of automatic ramming drive system

The open-circuit fault is one of the most common faults of the automatic ramming drive system(ARDS),and it can be categorized into the open-phase faults of Permanent Magnet Synchronous Motor(PMSM)and the open-circuit faults of Voltage Source Inverter(VSI). The stator current serves as a common indicator for detecting open-circuit faults. Due to the identical changes of the stator current between the open-phase faults in the PMSM and failures of double switches within the same leg of the VSI, this paper utilizes the zero-sequence voltage component as an additional diagnostic criterion to differentiate them.Considering the variable conditions and substantial noise of the ARDS, a novel Multi-resolution Network(Mr Net) is proposed, which can extract multi-resolution perceptual information and enhance robustness to the noise. Meanwhile, a feature weighted layer is introduced to allocate higher weights to characteristics situated near the feature frequency. Both simulation and experiment results validate that the proposed fault diagnosis method can diagnose 25 types of open-circuit faults and achieve more than98.28% diagnostic accuracy. In addition, the experiment results also demonstrate that Mr Net has the capability of diagnosing the fault types accurately under the interference of noise signals(Laplace noise and Gaussian noise).