Disparity estimation for multi-scale multi-sensor fusion

The perception module of advanced driver assistance systems plays a vital role.Perception schemes often use a single sensor for data processing and environmental perception or adopt the information processing results of various sensors for the fusion of the detection layer.This paper proposes a multi-scale and multi-sensor data fusion strategy in the front end of perception and accomplishes a multi-sensor function disparity map generation scheme.A binocular stereo vision sensor composed of two cameras and a light deterction and ranging(LiDAR)sensor is used to jointly perceive the environment,and a multi-scale fusion scheme is employed to improve the accuracy of the disparity map.This solution not only has the advantages of dense perception of binocular stereo vision sensors but also considers the perception accuracy of LiDAR sensors.Experiments demonstrate that the multi-scale multi-sensor scheme proposed in this paper significantly improves disparity map estimation.

A Lightweight Convolutional Neural Network with Hierarchical Multi-Scale Feature Fusion for Image Classification

Convolutional neural networks (CNNs) are widely used in image classification tasks, but their increasing model size and computation make them challenging to implement on embedded systems with constrained hardware resources. To address this issue, the MobileNetV1 network was developed, which employs depthwise convolution to reduce network complexity. MobileNetV1 employs a stride of 2 in several convolutional layers to decrease the spatial resolution of feature maps, thereby lowering computational costs. However, this stride setting can lead to a loss of spatial information, particularly affecting the detection and representation of smaller objects or finer details in images. To maintain the trade-off between complexity and model performance, a lightweight convolutional neural network with hierarchical multi-scale feature fusion based on the MobileNetV1 network is proposed. The network consists of two main subnetworks. The first subnetwork uses a depthwise dilated separable convolution (DDSC) layer to learn imaging features with fewer parameters, which results in a lightweight and computationally inexpensive network. Furthermore, depthwise dilated convolution in DDSC layer effectively expands the field of view of filters, allowing them to incorporate a larger context. The second subnetwork is a hierarchical multi-scale feature fusion (HMFF) module that uses parallel multi-resolution branches architecture to process the input feature map in order to extract the multi-scale feature information of the input image. Experimental results on the CIFAR-10, Malaria, and KvasirV1 datasets demonstrate that the proposed method is efficient, reducing the network parameters and computational cost by 65.02% and 39.78%, respectively, while maintaining the network performance compared to the MobileNetV1 baseline.

Ship recognition based on HRRP via multi-scale sparse preserving method

In order to extract the richer feature information of ship targets from sea clutter, and address the high dimensional data problem, a method termed as multi-scale fusion kernel sparse preserving projection(MSFKSPP) based on the maximum margin criterion(MMC) is proposed for recognizing the class of ship targets utilizing the high-resolution range profile(HRRP). Multi-scale fusion is introduced to capture the local and detailed information in small-scale features, and the global and contour information in large-scale features, offering help to extract the edge information from sea clutter and further improving the target recognition accuracy. The proposed method can maximally preserve the multi-scale fusion sparse of data and maximize the class separability in the reduced dimensionality by reproducing kernel Hilbert space. Experimental results on the measured radar data show that the proposed method can effectively extract the features of ship target from sea clutter, further reduce the feature dimensionality, and improve target recognition performance.

Clothing Parsing Based on Multi-Scale Fusion and Improved Self-Attention Mechanism

Due to the lack of long-range association and spatial location information,fine details and accurate boundaries of complex clothing images cannot always be obtained by using the existing deep learning-based methods.This paper presents a convolutional structure with multi-scale fusion to optimize the step of clothing feature extraction and a self-attention module to capture long-range association information.The structure enables the self-attention mechanism to directly participate in the process of information exchange through the down-scaling projection operation of the multi-scale framework.In addition,the improved self-attention module introduces the extraction of 2-dimensional relative position information to make up for its lack of ability to extract spatial position features from clothing images.The experimental results based on the colorful fashion parsing dataset(CFPD)show that the proposed network structure achieves 53.68%mean intersection over union(mIoU)and has better performance on the clothing parsing task.

基于改进SSD的工件表面缺陷检测

工件的表面缺陷不仅影响外观而且直接影响产品的质量、寿命和性能,因此对工件进行实时表面缺陷检测很有必要。针对当前SSD算法不利于小目标检测易导致误检的情况,提出了一种基于单阶段多层检测器的改进SSD自动检测方法。采用了以ResNet替换SSD中原始的VGGNet的方法,研究了小目标检测的问题;采用了对深层特征进行反卷积且将深层特征与浅层特征融合的方法,研究了语义信息不足易误检的问题。结果表明,该方法较原SSD模型在工件的表面缺陷检测上m AP值提高了约4.6%,从而认为本方法可用于工件表面缺陷的实时自动检测。

Sub-Regional Infrared-Visible Image Fusion Using Multi-Scale Transformation

Infrared-visible image fusion plays an important role in multi-source data fusion,which has the advantage of integrating useful information from multi-source sensors.However,there are still challenges in target enhancement and visual improvement.To deal with these problems,a sub-regional infrared-visible image fusion method(SRF)is proposed.First,morphology and threshold segmentation is applied to extract targets interested in infrared images.Second,the infrared back-ground is reconstructed based on extracted targets and the visible image.Finally,target and back-ground regions are fused using a multi-scale transform.Experimental results are obtained using public data for comparison and evaluation,which demonstrate that the proposed SRF has poten-tial benefits over other methods.

An infrared and visible image fusion method based upon multi-scale and top-hat transforms

The high-frequency components in the traditional multi-scale transform method are approximately sparse, which can represent different information of the details. But in the low-frequency component, the coefficients around the zero value are very few, so we cannot sparsely represent low-frequency image information. The low-frequency component contains the main energy of the image and depicts the profile of the image. Direct fusion of the low-frequency component will not be conducive to obtain highly accurate fusion result. Therefore, this paper presents an infrared and visible image fusion method combining the multi-scale and top-hat transforms. On one hand, the new top-hat-transform can effectively extract the salient features of the low-frequency component. On the other hand, the multi-scale transform can extract highfrequency detailed information in multiple scales and from diverse directions. The combination of the two methods is conducive to the acquisition of more characteristics and more accurate fusion results. Among them, for the low-frequency component, a new type of top-hat transform is used to extract low-frequency features, and then different fusion rules are applied to fuse the low-frequency features and low-frequency background; for high-frequency components, the product of characteristics method is used to integrate the detailed information in high-frequency. Experimental results show that the proposed algorithm can obtain more detailed information and clearer infrared target fusion results than the traditional multiscale transform methods. Compared with the state-of-the-art fusion methods based on sparse representation, the proposed algorithm is simple and efficacious, and the time consumption is significantly reduced.

基于RSSD的遥感图像目标检测算法

针对SSD算法检测遥感图像目标时存在容易漏检且检测精度低的问题,提出基于残差SSD网络的遥感图像目标检测算法。该算法在SSD网络结构的基础上,将基准网络模型VGG替换为残差网络模型ResNet-50,通过增加网络深度,充分提取遥感图像小目标数据集的底层特征,引入注意力模块,使感受野更关注目标特征,增强低层网络的信息表征能力,采用特征金字塔融合方法融合网络结构的高层语义特征和低层视觉特征,增强检测目标的定位能力。实验结果表明,该算法增强了复杂背景的干扰抑制性,提高了小目标的检测精度,比传统的SSD算法具有更强的检测性能。

Attention Guided Multi Scale Feature Fusion Network for Automatic Prostate Segmentation

The precise and automatic segmentation of prostate magnetic resonance imaging(MRI)images is vital for assisting doctors in diagnosing prostate diseases.In recent years,many advanced methods have been applied to prostate segmentation,but due to the variability caused by prostate diseases,automatic segmentation of the prostate presents significant challenges.In this paper,we propose an attention-guided multi-scale feature fusion network(AGMSF-Net)to segment prostate MRI images.We propose an attention mechanism for extracting multi-scale features,and introduce a 3D transformer module to enhance global feature representation by adding it during the transition phase from encoder to decoder.In the decoder stage,a feature fusion module is proposed to obtain global context information.We evaluate our model on MRI images of the prostate acquired from a local hospital.The relative volume difference(RVD)and dice similarity coefficient(DSC)between the results of automatic prostate segmentation and ground truth were 1.21%and 93.68%,respectively.To quantitatively evaluate prostate volume on MRI,which is of significant clinical significance,we propose a unique AGMSF-Net.The essential performance evaluation and validation experiments have demonstrated the effectiveness of our method in automatic prostate segmentation.

基于改进SSD的自然场景小交通标志检测

为提高在复杂的自然交通场景下对小交通标志检测的准确度,改进了SSD模型。在SSD多个检测层加入并行多尺度特征融合,通过结合深、浅特征层的检测优势,改善了SSD模型在小目标检测方面的不足;在SSD模型的多个检测头分别加入注意力机制模块,增强对小交通标志的特征提取效果;加入focal loss损失函数减小背景对整体损失的贡献,防止背景过拟合。实验结果表明,在复杂自然场景下,改进的方法相比原始模型对小交通标志检测的mAP提升了4.9百分点。

基于改进SSD的钢材表面缺陷检测

工业生产过程中,钢材表面缺陷的检测对于钢材的质量控制发挥着十分重要的作用,针对钢材表面缺陷检测中存在的检测精度低、检测速度慢等问题,提出了一种钢材表面缺陷检测的改进SSD算法。在所提算法中,采用Transformer多头注意力机制模块代替原SSD结构中的Conv5_1层,以提高小目标检测的能力;原SSD结构中的Conv7操作替换为Involution算子操作,以减少运算的参数量;对网络结构进行特征融合处理,以更全面地检测特征图中所包含的信息。利用NEU-DET数据集进行实验,实验结果表明改进后的SSD算法是有效的,可以高效检测到钢材表面的小目标缺陷,相比改进前平均检测精度提高了4.5%,检测速度提高了13.6%。

Grasp Detection with Hierarchical Multi-Scale Feature Fusion and Inverted Shuffle Residual

Grasp detection plays a critical role for robot manipulation.Mainstream pixel-wise grasp detection networks with encoder-decoder structure receive much attention due to good accuracy and efficiency.However,they usually transmit the high-level feature in the encoder to the decoder,and low-level features are neglected.It is noted that low-level features contain abundant detail information,and how to fully exploit low-level features remains unsolved.Meanwhile,the channel information in high-level feature is also not well mined.Inevitably,the performance of grasp detection is degraded.To solve these problems,we propose a grasp detection network with hierarchical multi-scale feature fusion and inverted shuffle residual.Both low-level and high-level features in the encoder are firstly fused by the designed skip connections with attention module,and the fused information is then propagated to corresponding layers of the decoder for in-depth feature fusion.Such a hierarchical fusion guarantees the quality of grasp prediction.Furthermore,an inverted shuffle residual module is created,where the high-level feature from encoder is split in channel and the resultant split features are processed in their respective branches.By such differentiation processing,more high-dimensional channel information is kept,which enhances the representation ability of the network.Besides,an information enhancement module is added before the encoder to reinforce input information.The proposed method attains 98.9%and 97.8%in image-wise and object-wise accuracy on the Cornell grasping dataset,respectively,and the experimental results verify the effectiveness of the method.

基于改进SSD的液晶玻璃基板缺陷检测

在液晶玻璃基板生产过程中容易产生夹杂、气泡、锡点、节瘤、裂痕等多种缺陷,这些缺陷严重降低了液晶玻璃的性能,本文提出了一种基于深度学习的液晶玻璃基板缺陷检测,在SSD目标检测网络的基础之上,引入了ResNet中的残差模块用于主干网络的特征提取,同时对提取到的特征进行了跨通道多尺度的融合,经实验表明该方法可以有效地改善SSD网络对缺陷检测的精度,特别是提高了对小目标的检测的检测精度。

Multi-Scale Feature Fusion Model for Bridge Appearance Defect Detection

Although the Faster Region-based Convolutional Neural Network(Faster R-CNN)model has obvious advantages in defect recognition,it still cannot overcome challenging problems,such as time-consuming,small targets,irregular shapes,and strong noise interference in bridge defect detection.To deal with these issues,this paper proposes a novel Multi-scale Feature Fusion(MFF)model for bridge appearance disease detection.First,the Faster R-CNN model adopts Region Of Interest(ROl)pooling,which omits the edge information of the target area,resulting in some missed detections and inaccuracies in both detecting and localizing bridge defects.Therefore,this paper proposes an MFF based on regional feature Aggregation(MFF-A),which reduces the missed detection rate of bridge defect detection and improves the positioning accuracy of the target area.Second,the Faster R-CNN model is insensitive to small targets,irregular shapes,and strong noises in bridge defect detection,which results in a long training time and low recognition accuracy.Accordingly,a novel Lightweight MFF(namely MFF-L)model for bridge appearance defect detection using a lightweight network EfficientNetV2 and a feature pyramid network is proposed,which fuses multi-scale features to shorten the training speed and improve recognition accuracy.Finally,the effectiveness of the proposed method is evaluated on the bridge disease dataset and public computational fluid dynamic dataset.

Multi-Scale Fusion Model Based on Gated Recurrent Unit for Enhancing Prediction Accuracy of State-of-Charge in Battery Energy Storage Systems

Accurate prediction of the state-of-charge(SOC)of battery energy storage system(BESS)is critical for its safety and lifespan in electric vehicles.To overcome the imbalance of existing methods between multi-scale feature fusion and global feature extraction,this paper introduces a novel multi-scale fusion(MSF)model based on gated recurrent unit(GRU),which is specifically designed for complex multi-step SOC prediction in practical BESSs.Pearson correlation analysis is first employed to identify SOC-related parameters.These parameters are then input into a multi-layer GRU for point-wise feature extraction.Concurrently,the parameters undergo patching before entering a dual-stage multi-layer GRU,thus enabling the model to capture nuanced information across varying time intervals.Ultimately,by means of adaptive weight fusion and a fully connected network,multi-step SOC predictions are rendered.Following extensive validation over multiple days,it is illustrated that the proposed model achieves an absolute error of less than 1.5%in real-time SOC prediction.

基于改进SSD卷积神经网络的苹果定位与分级方法

为实现苹果果径与果形快速准确自动化分级,提出了基于改进型SSD卷积神经网络的苹果定位与分级算法。深度图像与两通道图像融合提高苹果分级效率,即对从顶部获取的苹果RGB图像进行通道分离,并提取分离通道中影响苹果识别精度最大的两个通道与基于ZED双目立体相机从苹果顶部获取的苹果部分深度图像进行融合,在融合图像中计算苹果的纵径相关信息,实现了基于顶部融合图像的多个苹果果形分级和信息输出;使用深度可分离卷积模块替换原SSD网络主干特征提取网络中部分标准卷积,实现了网络的轻量化。经过训练的算法在验证集下的识别召回率、精确率、mAP和F1值分别为93.68%、94.89%、98.37%和94.25%。通过对比分析了4种输入层识别精确率的差异,实验结果表明输入层的图像通道组合为DGB时对苹果的识别与分级mAP最高。在使用相同输入层的情况下,比较原SSD、Faster R-CNN与YOLO v5算法在不同果实数目下对苹果的实际识别定位与分级效果,并以mAP为评估值,实验结果表明改进型SSD在密集苹果的mAP与原SSD相当,比Faster R-CNN高1.33个百分点,比YOLO v5高14.23个百分点。并且在不同硬件条件下验证了该算法定位分级效率的优势,单幅图像在GPU下的检测时间为5.71 ms,在CPU下的检测时间为15.96 ms,检测视频的帧率达到175.17 f/s和62.64 f/s。该研究可为自动化分级设备在高速环境下精准定位并分级苹果提供理论基础。

改进的SSD生活垃圾检测算法

针对目前垃圾资源化利用的问题,为提升垃圾分拣工作的速率,并减少人工成本,通过对目标检测算法SSD(Sin⁃gle Shot Multibox Detector)的研究与分析,提出了基于改进的SSD垃圾分类算法,对基础特征提取网络VGG16参数量大、检测性能低等问题,使用DenseNet的网络结构,加深网络层数,并使用通道叠加的方式加强信息传递,从特征复用的角度上加强网络性能;对原网络对于小目标检测能力弱的问题,利用FPN结构加强特征图中包含的语义信息,提高对小目标的检测能力;对原损失函数在模型评估时的不等价情况,引入GIoU损失提高定位精度。这里的算法在PASCAL VOC数据集与自己制作的生活垃圾检测数据集上测试,其中在PASCAL VOC数据集上的检测结果显示,这里的算法相比于SSD300和SSD512分别有1.7%和1.9%的提升;在生活垃圾检测数据集上,分别有2.1%和3%的提升。

基于改进SSD的行人检测算法

针对目前主流的目标检测算法在检测行人时无法兼顾精度与实时性的问题,提出一种改进单次多框检测器(single shot multibox detector,SSD)的行人检测算法。首先,将高效通道注意力机制引入浅层网络中并重新分配特征权重,引导网络更加关注小尺度行人的特征信息;其次,构造一种新的特征融合模块以改善浅层特征语义信息不足的问题;最后,通过优化原始先验框的参数来生成适用于检测行人的先验框。实验结果表明,改进后的算法在PASCAL VOC2007行人测试集上的平均精度达到82.96%,较SSD提高了3.83%,在小尺度行人测试集上提高了5.48%,同时检测速度达到了69.2FPS,满足实时性的要求。

跨层融合和感受野扩增的SSD目标检测算法

鉴于SSD(Single Shot Multibox Detector)不同层缺乏信息的交互以及模型感受野的限制,提出了一种改进的SSD目标检测算法——ESSD(Enhanced SSD),以提高目标检测的准确性。首先,使用SSD模型中原有的多尺度特征图,利用FPN(Feature Pyramid Networks)的思想,设计了一种跨层信息交互模块,在增强了不同层的语义信息能力的同时减小了不同层的信息差异。然后,为了提高模型的感受野和多尺度检测能力,设计了一种感受野扩增模块。最后,采用批处理归一化层缩短训练时间,以提高模型的收敛速度。为了评价ESSD的有效性,在PASCAL VOC2007测试集以及PASCAL VOC2012测试集上进行了实验。实验结果表明,在PASCAL VOC2007数据集上其mAP为82.1%且检测速度为15.7FPS,相比原有的SSD512,其mAP提升了2.3%;在PASCAL VOC2012测试集上其mAP达到了80.6%,也比SSD512高2.1%。实验证明了ESSD检测器在达到较高检测精度的情况下,仍然可以满足实时性。

改进SSD算法的小目标检测研究

针对SSD目标检测算法在实际的复杂的检测背景下存在着小目标错检和漏检率较高、小目标特征信息不足的缺点,提出一种改进的SSD算法。受特征金字塔网络(FPN)启发,设计特征融合模块(FFM),将SSD网络的高分辨率的低层特征图与高语义信息的高层特征图进行融合,由此保留了特征图的上下文信息,提高了多尺度小目标特征信息的提取能力;通过引入注意力机制构建特征注意力模块(FAM),将融合后的特征层通过FAM,学习重要特征并抑制不必要特征,提高了网络的表示能力。通过在公开的PASCAL VOC(2007+2012)数据集上进行实验,改进的模型平均精度均值(mAP)达到79.9%,与改进前的SSD算法相比,提高了4.24%,减少了错检和漏检的情况,使得改进后的SSD算法具有较强的鲁棒性,提升了目标检测性能。