Disparity estimation for multi-scale multi-sensor fusion

The perception module of advanced driver assistance systems plays a vital role.Perception schemes often use a single sensor for data processing and environmental perception or adopt the information processing results of various sensors for the fusion of the detection layer.This paper proposes a multi-scale and multi-sensor data fusion strategy in the front end of perception and accomplishes a multi-sensor function disparity map generation scheme.A binocular stereo vision sensor composed of two cameras and a light deterction and ranging(LiDAR)sensor is used to jointly perceive the environment,and a multi-scale fusion scheme is employed to improve the accuracy of the disparity map.This solution not only has the advantages of dense perception of binocular stereo vision sensors but also considers the perception accuracy of LiDAR sensors.Experiments demonstrate that the multi-scale multi-sensor scheme proposed in this paper significantly improves disparity map estimation.

A Lightweight Convolutional Neural Network with Hierarchical Multi-Scale Feature Fusion for Image Classification

Convolutional neural networks (CNNs) are widely used in image classification tasks, but their increasing model size and computation make them challenging to implement on embedded systems with constrained hardware resources. To address this issue, the MobileNetV1 network was developed, which employs depthwise convolution to reduce network complexity. MobileNetV1 employs a stride of 2 in several convolutional layers to decrease the spatial resolution of feature maps, thereby lowering computational costs. However, this stride setting can lead to a loss of spatial information, particularly affecting the detection and representation of smaller objects or finer details in images. To maintain the trade-off between complexity and model performance, a lightweight convolutional neural network with hierarchical multi-scale feature fusion based on the MobileNetV1 network is proposed. The network consists of two main subnetworks. The first subnetwork uses a depthwise dilated separable convolution (DDSC) layer to learn imaging features with fewer parameters, which results in a lightweight and computationally inexpensive network. Furthermore, depthwise dilated convolution in DDSC layer effectively expands the field of view of filters, allowing them to incorporate a larger context. The second subnetwork is a hierarchical multi-scale feature fusion (HMFF) module that uses parallel multi-resolution branches architecture to process the input feature map in order to extract the multi-scale feature information of the input image. Experimental results on the CIFAR-10, Malaria, and KvasirV1 datasets demonstrate that the proposed method is efficient, reducing the network parameters and computational cost by 65.02% and 39.78%, respectively, while maintaining the network performance compared to the MobileNetV1 baseline.

Adaptive Multi-Feature Fusion for Vehicle Micro-Motor Noise Recognition Considering Auditory Perception

The deployment of vehicle micro-motors has witnessed an expansion owing to the progression in electrification and intelligent technologies.However,some micro-motors may exhibit design deficiencies,component wear,assembly errors,and other imperfections that may arise during the design or manufacturing phases.Conse-quently,these micro-motors might generate anomalous noises during their operation,consequently exerting a substantial adverse influence on the overall comfort of drivers and passengers.Automobile micro-motors exhibit a diverse array of structural variations,consequently leading to the manifestation of a multitude of distinctive auditory irregularities.To address the identification of diverse forms of abnormal noise,this research presents a novel approach rooted in the utilization of vibro-acoustic fusion-convolutional neural network(VAF-CNN).This method entails the deployment of distinct network branches,each serving to capture disparate features from the multi-sensor data,all the while considering the auditory perception traits inherent in the human auditory sys-tem.The intermediary layer integrates the concept of adaptive weighting of multi-sensor features,thus affording a calibration mechanism for the features hailing from multiple sensors,thereby enabling a further refinement of features within the branch network.For optimal model efficacy,a feature fusion mechanism is implemented in the concluding layer.To substantiate the efficacy of the proposed approach,this paper initially employs an augmented data methodology inspired by modified SpecAugment,applied to the dataset of abnormal noise sam-ples,encompassing scenarios both with and without in-vehicle interior noise.This serves to mitigate the issue of limited sample availability.Subsequent comparative evaluations are executed,contrasting the performance of the model founded upon single-sensor data against other feature fusion models reliant on multi-sensor data.The experimental results substantiate that the suggested methodology yields heightened recognition accuracy and greater resilience against interference.Moreover,it holds notable practical significance in the engineering domain,as it furnishes valuable support for the targeted management of noise emanating from vehicle micro-motors.

Clothing Parsing Based on Multi-Scale Fusion and Improved Self-Attention Mechanism

Due to the lack of long-range association and spatial location information,fine details and accurate boundaries of complex clothing images cannot always be obtained by using the existing deep learning-based methods.This paper presents a convolutional structure with multi-scale fusion to optimize the step of clothing feature extraction and a self-attention module to capture long-range association information.The structure enables the self-attention mechanism to directly participate in the process of information exchange through the down-scaling projection operation of the multi-scale framework.In addition,the improved self-attention module introduces the extraction of 2-dimensional relative position information to make up for its lack of ability to extract spatial position features from clothing images.The experimental results based on the colorful fashion parsing dataset(CFPD)show that the proposed network structure achieves 53.68%mean intersection over union(mIoU)and has better performance on the clothing parsing task.

Method of Multi-Mode Sensor Data Fusion with an Adaptive Deep Coupling Convolutional Auto-Encoder

To address the difficulties in fusing multi-mode sensor data for complex industrial machinery, an adaptive deep coupling convolutional auto-encoder (ADCCAE) fusion method was proposed. First, the multi-mode features extracted synchronously by the CCAE were stacked and fed to the multi-channel convolution layers for fusion. Then, the fused data was passed to all connection layers for compression and fed to the Softmax module for classification. Finally, the coupling loss function coefficients and the network parameters were optimized through an adaptive approach using the gray wolf optimization (GWO) algorithm. Experimental comparisons showed that the proposed ADCCAE fusion model was superior to existing models for multi-mode data fusion.

Multimodal Medical Image Fusion Based on Parameter Adaptive PCNN and Latent Low-rank Representation

Medical image fusion has been developed as an efficient assistive technology in various clinical applications such as medical diagnosis and treatment planning.Aiming at the problem of insufficient protection of image contour and detail information by traditional image fusion methods,a new multimodal medical image fusion method is proposed.This method first uses non-subsampled shearlet transform to decompose the source image to obtain high and low frequency subband coefficients,then uses the latent low rank representation algorithm to fuse the low frequency subband coefficients,and applies the improved PAPCNN algorithm to fuse the high frequency subband coefficients.Finally,based on the automatic setting of parameters,the optimization method configuration of the time decay factorαe is carried out.The experimental results show that the proposed method solves the problems of difficult parameter setting and insufficient detail protection ability in traditional PCNN algorithm fusion images,and at the same time,it has achieved great improvement in visual quality and objective evaluation indicators.

Research on Data Fusion of Adaptive Weighted Multi-Source Sensor

Data fusion can effectively process multi-sensor information to obtain more accurate and reliable results than a single sensor.The data of water quality in the environment comes from different sensors,thus the data must be fused.In our research,self-adaptive weighted data fusion method is used to respectively integrate the data from the PH value,temperature,oxygen dissolved and NH3 concentration of water quality environment.Based on the fusion,the Grubbs method is used to detect the abnormal data so as to provide data support for estimation,prediction and early warning of the water quality.

Multi-Source Adaptive Selection and Fusion for Pedestrian Dead Reckoning

Accurate multi-source fusion is based on the reliability, quantity, and fusion mode of the sources. The problem of selecting the optimal set for participating in the fusion process is nondeterministic-polynomial-time-hard and is neither sub-modular nor super-modular. Furthermore, in the case of the Kalman filter(KF) fusion algorithm, accurate statistical characteristics of noise are difficult to obtain, and this leads to an unsatisfactory fusion result. To settle the referred cases, a distributed and adaptive weighted fusion algorithm based on KF has been proposed in this paper. In this method, on the basis of the pseudo prior probability of the estimated state of each source, the reliability of the sources is evaluated and the optimal set is selected on a certain threshold. Experiments were performed on multi-source pedestrian dead reckoning for verifying the proposed algorithm. The results obtained from these experiments indicate that the optimal set can be selected accurately with minimal computation, and the fusion error is reduced by 16.6% as compared to the corresponding value resulting from the algorithm without improvements.The proposed adaptive source reliability and fusion weight evaluation is effective against the varied-noise multi-source fusion system, and the fusion error caused by inaccurate statistical characteristics of the noise is reduced by the adaptive weight evaluation.The proposed algorithm exhibits good robustness, adaptability,and value on applications.

Sub-Regional Infrared-Visible Image Fusion Using Multi-Scale Transformation

Infrared-visible image fusion plays an important role in multi-source data fusion,which has the advantage of integrating useful information from multi-source sensors.However,there are still challenges in target enhancement and visual improvement.To deal with these problems,a sub-regional infrared-visible image fusion method(SRF)is proposed.First,morphology and threshold segmentation is applied to extract targets interested in infrared images.Second,the infrared back-ground is reconstructed based on extracted targets and the visible image.Finally,target and back-ground regions are fused using a multi-scale transform.Experimental results are obtained using public data for comparison and evaluation,which demonstrate that the proposed SRF has poten-tial benefits over other methods.

An infrared and visible image fusion method based upon multi-scale and top-hat transforms

The high-frequency components in the traditional multi-scale transform method are approximately sparse, which can represent different information of the details. But in the low-frequency component, the coefficients around the zero value are very few, so we cannot sparsely represent low-frequency image information. The low-frequency component contains the main energy of the image and depicts the profile of the image. Direct fusion of the low-frequency component will not be conducive to obtain highly accurate fusion result. Therefore, this paper presents an infrared and visible image fusion method combining the multi-scale and top-hat transforms. On one hand, the new top-hat-transform can effectively extract the salient features of the low-frequency component. On the other hand, the multi-scale transform can extract highfrequency detailed information in multiple scales and from diverse directions. The combination of the two methods is conducive to the acquisition of more characteristics and more accurate fusion results. Among them, for the low-frequency component, a new type of top-hat transform is used to extract low-frequency features, and then different fusion rules are applied to fuse the low-frequency features and low-frequency background; for high-frequency components, the product of characteristics method is used to integrate the detailed information in high-frequency. Experimental results show that the proposed algorithm can obtain more detailed information and clearer infrared target fusion results than the traditional multiscale transform methods. Compared with the state-of-the-art fusion methods based on sparse representation, the proposed algorithm is simple and efficacious, and the time consumption is significantly reduced.

An Adaptive and Image-guided Fusion for Stereo Satellite Image Derived Digital Surface Models

The accuracy of Digital Surface Models(DSMs)generated using stereo matching methods varies due to the varying acquisition conditions and configuration parameters of stereo images.It has been a good practice to fuse these DSMs generated from various stereo pairs to achieve enhanced,in which multiple DSMs are combined through computational approaches into a single,more accurate,and complete DSM.However,accurately characterizing detailed objects and their boundaries still present a challenge since most boundary-ware fusion methods still struggle to achieve sharpened depth discontinuities due to the averaging effects of different DSMs.Therefore,we propose a simple and efficient adaptive image-guided DSM fusion method that applies k-means clustering on small patches of the orthophoto to guide the pixel-level fusion adapted to the most consistent and relevant elevation points.The experiment results show that our proposed method has outperformed comparing methods in accuracy and the ability to preserve sharpened depth edges.

Multi-sensor Hybrid Fusion Algorithm Based on Adaptive Square-root Cubature Kalman Filter

处于正常操作条件，一个常规方形根的求容积法 Kalman 过滤器(SRCKF ) 给足够地好的评价结果。然而，如果大小不是可靠的， SRCKF 可以给不精密的结果并且到时间分叉。这研究与过滤器获得修正介绍一个适应 SRCKF 算法因为测量的盒子失灵。由建议一个切换的标准，一个最佳的过滤器根据测量质量从适应、常规的 SRCKF 被选择。一个分系统软差错察觉算法与过滤器剩余被造。利用一个清楚的分系统差错系数，有缺点的分系统由于系统重建被孤立。以便改进多传感器系统的性能，一个混合熔化算法基于适应 SRCKF 被介绍。状态和错误协变性矩阵被 priori 熔化估计也预言，并且被分系统的预言并且估计的信息更新。建议算法被用于容器动态放系统模拟。他们与正常 SRCKF 和本地评价相比是加权的熔化算法。模拟结果证明介绍适应 SRCKF 改进分系统过滤的坚韧性，并且混合熔化算法有更好的表演。模拟验证建议算法的有效性。

Attention Guided Multi Scale Feature Fusion Network for Automatic Prostate Segmentation

The precise and automatic segmentation of prostate magnetic resonance imaging(MRI)images is vital for assisting doctors in diagnosing prostate diseases.In recent years,many advanced methods have been applied to prostate segmentation,but due to the variability caused by prostate diseases,automatic segmentation of the prostate presents significant challenges.In this paper,we propose an attention-guided multi-scale feature fusion network(AGMSF-Net)to segment prostate MRI images.We propose an attention mechanism for extracting multi-scale features,and introduce a 3D transformer module to enhance global feature representation by adding it during the transition phase from encoder to decoder.In the decoder stage,a feature fusion module is proposed to obtain global context information.We evaluate our model on MRI images of the prostate acquired from a local hospital.The relative volume difference(RVD)and dice similarity coefficient(DSC)between the results of automatic prostate segmentation and ground truth were 1.21%and 93.68%,respectively.To quantitatively evaluate prostate volume on MRI,which is of significant clinical significance,we propose a unique AGMSF-Net.The essential performance evaluation and validation experiments have demonstrated the effectiveness of our method in automatic prostate segmentation.

Multi-rate sensor fusion-based adaptive discrete finite-time synergetic control for flexible-joint mechanical systems

This paper proposes an adaptive discrete finite-time synergetic control （ADFTSC） scheme based on a multi-rate sensor fusion estimator for flexible-joint mechanical systems in the presence of unmeasured states and dynamic uncertainties. Multi-rate sensors are employed to observe the system states which cannot be directly obtained by encoders due to the existence of joint flexibilities. By using an extended Kalman filter （EKF）, the finite-time synergetic controller is designed based on a sensor fusion estimator which estimates states and parameters of the mechanical system with multi-rate measurements. The proposed controller can guarantee the finite-time convergence of tracking errors by the theoretical derivation. Simulation and experimental studies are included to validate the effectiveness of the proposed approach.

Multi-Feature Fusion Based Relative Pose Adaptive Estimation for On-Orbit Servicing of Non-Cooperative Spacecraft

On-orbit servicing, such as spacecraft maintenance, on-orbit assembly, refueling, and de-orbiting, can reduce the cost of space missions, improve the performance of spacecraft, and extend its life span. The relative state between the servicing and target spacecraft is vital for on-orbit servicing missions, especially the final approaching stage. The major challenge of this stage is that the observed features of the target are incomplete or are constantly changing due to the short distance and limited Field of View (FOV) of camera. Different from cooperative spacecraft, non-cooperative target does not have artificial feature markers. Therefore, contour features, including triangle supports of solar array, docking ring, and corner points of the spacecraft body, are used as the measuring features. To overcome the drawback of FOV limitation and imaging ambiguity of the camera, a "selfie stick" structure and a self-calibration strategy were implemented, ensuring that part of the contour features could be observed precisely when the two spacecraft approached each other. The observed features were constantly changing as the relative distance shortened. It was difficult to build a unified measurement model for different types of features, including points, line segments, and circle. Therefore, dual quaternion was implemented to model the relative dynamics and measuring features. With the consideration of state uncertainty of the target, a fuzzy adaptive strong tracking filter( FASTF) combining fuzzy logic adaptive controller (FLAC) with strong tracking filter(STF) was designed to robustly estimate the relative states between the servicing spacecraft and the target. Finally, the effectiveness of the strategy was verified by mathematical simulation. The achievement of this research provides a theoretical and technical foundation for future on-orbit servicing missions.

An Adaptive Padding Correlation Filter With Group Feature Fusion for Robust Visual Tracking

In recent visual tracking research,correlation filter(CF)based trackers become popular because of their high speed and considerable accuracy.Previous methods mainly work on the extension of features and the solution of the boundary effect to learn a better correlation filter.However,the related studies are insufficient.By exploring the potential of trackers in these two aspects,a novel adaptive padding correlation filter(APCF)with feature group fusion is proposed for robust visual tracking in this paper based on the popular context-aware tracking framework.In the tracker,three feature groups are fused by use of the weighted sum of the normalized response maps,to alleviate the risk of drift caused by the extreme change of single feature.Moreover,to improve the adaptive ability of padding for the filter training of different object shapes,the best padding is selected from the preset pool according to tracking precision over the whole video,where tracking precision is predicted according to the prediction model trained by use of the sequence features of the first several frames.The sequence features include three traditional features and eight newly constructed features.Extensive experiments demonstrate that the proposed tracker is superior to most state-of-the-art correlation filter based trackers and has a stable improvement compared to the basic trackers.

A Robust Interacting Multisensor State Fusion Based on Adaptive Outlier Controlling Multirate Model

An adaptive outlier controlling multirate model based on Hong’s multirate kinetic model was represented in order to resist the outliers and utilize their useful information. Wavelet transform was introduced to detect and control the outliers. The multirate information extraction and the controlling of outliers were properly integrated to establish an adaptive outlier controlling multirate model. The proposed model was applied to multisensor state fusion with interacting multiple model (IMM), and a robust interacting multisensor state fusion algorithm was established based on adaptive outlier controlling multirate model. The Monte-Carlo simulation shows that it could improve the accuracy of fusion estimation by 70% compared to Hong’s algorithm and at least 14% to Xiao’s algorithm.

Grasp Detection with Hierarchical Multi-Scale Feature Fusion and Inverted Shuffle Residual

Grasp detection plays a critical role for robot manipulation.Mainstream pixel-wise grasp detection networks with encoder-decoder structure receive much attention due to good accuracy and efficiency.However,they usually transmit the high-level feature in the encoder to the decoder,and low-level features are neglected.It is noted that low-level features contain abundant detail information,and how to fully exploit low-level features remains unsolved.Meanwhile,the channel information in high-level feature is also not well mined.Inevitably,the performance of grasp detection is degraded.To solve these problems,we propose a grasp detection network with hierarchical multi-scale feature fusion and inverted shuffle residual.Both low-level and high-level features in the encoder are firstly fused by the designed skip connections with attention module,and the fused information is then propagated to corresponding layers of the decoder for in-depth feature fusion.Such a hierarchical fusion guarantees the quality of grasp prediction.Furthermore,an inverted shuffle residual module is created,where the high-level feature from encoder is split in channel and the resultant split features are processed in their respective branches.By such differentiation processing,more high-dimensional channel information is kept,which enhances the representation ability of the network.Besides,an information enhancement module is added before the encoder to reinforce input information.The proposed method attains 98.9%and 97.8%in image-wise and object-wise accuracy on the Cornell grasping dataset,respectively,and the experimental results verify the effectiveness of the method.

Multi-Scale Feature Fusion Model for Bridge Appearance Defect Detection

Although the Faster Region-based Convolutional Neural Network(Faster R-CNN)model has obvious advantages in defect recognition,it still cannot overcome challenging problems,such as time-consuming,small targets,irregular shapes,and strong noise interference in bridge defect detection.To deal with these issues,this paper proposes a novel Multi-scale Feature Fusion(MFF)model for bridge appearance disease detection.First,the Faster R-CNN model adopts Region Of Interest(ROl)pooling,which omits the edge information of the target area,resulting in some missed detections and inaccuracies in both detecting and localizing bridge defects.Therefore,this paper proposes an MFF based on regional feature Aggregation(MFF-A),which reduces the missed detection rate of bridge defect detection and improves the positioning accuracy of the target area.Second,the Faster R-CNN model is insensitive to small targets,irregular shapes,and strong noises in bridge defect detection,which results in a long training time and low recognition accuracy.Accordingly,a novel Lightweight MFF(namely MFF-L)model for bridge appearance defect detection using a lightweight network EfficientNetV2 and a feature pyramid network is proposed,which fuses multi-scale features to shorten the training speed and improve recognition accuracy.Finally,the effectiveness of the proposed method is evaluated on the bridge disease dataset and public computational fluid dynamic dataset.

Multi-Scale Fusion Model Based on Gated Recurrent Unit for Enhancing Prediction Accuracy of State-of-Charge in Battery Energy Storage Systems

Accurate prediction of the state-of-charge(SOC)of battery energy storage system(BESS)is critical for its safety and lifespan in electric vehicles.To overcome the imbalance of existing methods between multi-scale feature fusion and global feature extraction,this paper introduces a novel multi-scale fusion(MSF)model based on gated recurrent unit(GRU),which is specifically designed for complex multi-step SOC prediction in practical BESSs.Pearson correlation analysis is first employed to identify SOC-related parameters.These parameters are then input into a multi-layer GRU for point-wise feature extraction.Concurrently,the parameters undergo patching before entering a dual-stage multi-layer GRU,thus enabling the model to capture nuanced information across varying time intervals.Ultimately,by means of adaptive weight fusion and a fully connected network,multi-step SOC predictions are rendered.Following extensive validation over multiple days,it is illustrated that the proposed model achieves an absolute error of less than 1.5%in real-time SOC prediction.