Disparity estimation for multi-scale multi-sensor fusion

The perception module of advanced driver assistance systems plays a vital role.Perception schemes often use a single sensor for data processing and environmental perception or adopt the information processing results of various sensors for the fusion of the detection layer.This paper proposes a multi-scale and multi-sensor data fusion strategy in the front end of perception and accomplishes a multi-sensor function disparity map generation scheme.A binocular stereo vision sensor composed of two cameras and a light deterction and ranging(LiDAR)sensor is used to jointly perceive the environment,and a multi-scale fusion scheme is employed to improve the accuracy of the disparity map.This solution not only has the advantages of dense perception of binocular stereo vision sensors but also considers the perception accuracy of LiDAR sensors.Experiments demonstrate that the multi-scale multi-sensor scheme proposed in this paper significantly improves disparity map estimation.

Progress and Achievements of Multi-sensor Fusion Navigation in China during 2019—2023

Global Navigation Satellite System(GNSS)can provide all-weather,all-time,high-precision positioning,navigation and timing services,which plays an important role in national security,national economy,public life and other aspects.However,in environments with limited satellite signals such as urban canyons,tunnels,and indoor spaces,it is difficult to provide accurate and reliable positioning services only by satellite navigation.Multi-source sensor integrated navigation can effectively overcome the limitations of single-sensor navigation through the fusion of different types of sensor data such as Inertial Measurement Unit(IMU),vision sensor,and LiDAR,and provide more accurate,stable and robust navigation information in complex environments.We summarizes the research status of multi-source sensor integrated navigation technology,and focuses on the representative innovations and applications of integrated navigation and positioning technology by major domestic scientific research institutions in China during 2019—2023.

Research on Optimal Preload Method of Controllable Rolling Bearing Based on Multisensor Fusion

Angular contact ball bearings have been widely used in machine tool spindles,and the bearing preload plays an important role in the performance of the spindle.In order to solve the problems of the traditional optimal preload prediction method limited by actual conditions and uncertainties,a roller bearing preload test method based on the improved D-S evidence theorymulti-sensor fusion method was proposed.First,a novel controllable preload system is proposed and evaluated.Subsequently,multiple sensors are employed to collect data on the bearing parameters during preload application.Finally,a multisensor fusion algorithm is used to make predictions,and a neural network is used to optimize the fitting of the preload data.The limitations of conventional preload testing methods are identified,and the integration of complementary information frommultiple sensors is used to achieve accurate predictions,offering valuable insights into the optimal preload force.Experimental results demonstrate that the multi-sensor fusion approach outperforms traditional methods in accurately measuring the optimal preload for rolling bearings.

Weighted Multi-sensor Data Level Fusion Method of Vibration Signal Based on Correlation Function

As the differences of sensor＇s precision and some random factors are difficult to control,the actual measurement signals are far from the target signals that affect the reliability and precision of rotating machinery fault diagnosis.The traditional signal processing methods,such as classical inference and weighted averaging algorithm usually lack dynamic adaptability that is easy for trends to cause the faults to be misjudged or left out.To enhance the measuring veracity and precision of vibration signal in rotary machine multi-sensor vibration signal fault diagnosis,a novel data level fusion approach is presented on the basis of correlation function analysis to fast determine the weighted value of multi-sensor vibration signals.The approach doesn＇t require knowing the prior information about sensors,and the weighted value of sensors can be confirmed depending on the correlation measure of real-time data tested in the data level fusion process.It gives greater weighted value to the greater correlation measure of sensor signals,and vice versa.The approach can effectively suppress large errors and even can still fuse data in the case of sensor failures because it takes full advantage of sensor＇s own-information to determine the weighted value.Moreover,it has good performance of anti-jamming due to the correlation measures between noise and effective signals are usually small.Through the simulation of typical signal collected from multi-sensors,the comparative analysis of dynamic adaptability and fault tolerance between the proposed approach and traditional weighted averaging approach is taken.Finally,the rotor dynamics and integrated fault simulator is taken as an example to verify the feasibility and advantages of the proposed approach,it is shown that the multi-sensor data level fusion based on correlation function weighted approach is better than the traditional weighted average approach with respect to fusion precision and dynamic adaptability.Meantime,the approach is adaptable and easy to use,can be applied to other areas of vibration measurement.

STUDY ON THE COAL-ROCK INTERFACE RECOGNITION METHOD BASED ON MULTI-SENSOR DATA FUSION TECHNIQUE

The coal-rock interface recognition method based on multi-sensor data fusiontechnique is put forward because of the localization of single type sensor recognition method. Themeasuring theory based on multi-sensor data fusion technique is analyzed, and hereby the testplatform of recognition system is manufactured. The advantage of data fusion with the fuzzy neuralnetwork (FNN) technique has been probed. The two-level FNN is constructed and data fusion is carriedout. The experiments show that in various conditions the method can always acquire a much higherrecognition rate than normal ones.

A Novel Multi-sensor Data Fusion Algorithm and Its Application to Diagnostics

To Meet the requirements of multi-sensor data fusion in diagnosis for complex equipment systems,a novel, fuzzy similarity-based data fusion algorithm is given. Based on fuzzy set theory, it calculates the fuzzy similarity among a certain sensor's measurement values and the multiple sensor's objective prediction values to determine the importance weigh of each sensor,and realizes the multi-sensor diagnosis parameter data fusion.According to the principle, its application software is also designed. The applied example proves that the algorithm can give priority to the high-stability and high -reliability sensors and it is laconic ,feasible and efficient to real-time circumstance measure and data processing in engine diagnosis.

Multi-sensor measurement and data fusion technology for manufacturing process monitoring:a literature review

Due to the rapid development of precision manufacturing technology,much research has been conducted in the field of multisensor measurement and data fusion technology with a goal of enhancing monitoring capabilities in terms of measurement accuracy and information richness,thereby improving the efficiency and precision of manufacturing.In a multisensor system,each sensor independently measures certain parameters.Then,the system uses a relevant signalprocessing algorithm to combine all of the independent measurements into a comprehensive set of measurement results.The purpose of this paper is to describe multisensor measurement and data fusion technology and its applications in precision monitoring systems.The architecture of multisensor measurement systems is reviewed,and some implementations in manufacturing systems are presented.In addition to the multisensor measurement system,related data fusion methods and algorithms are summarized.Further perspectives on multisensor monitoring and data fusion technology are included at the end of this paper.

Acoustic Emission Studies on Weld Bead Defects in Nuclear Grade SS 316L Materials

This paper contributes about the behaviour of Acoustic Emission (AE) signatures of implanted weld defects of SS 316L materials. Lack of penetration and lack of side fusion defects were implanted in weld bead region of the materials. Tungsten Inert Gas Welding (TIG) is adopted to weld the Stainless Steel (SS316L) nuclear grade materials. The material is fabricated with dimensions of 140 × 16 × 10 mm and AE signatures are studied under preload conditions. Mechanical Jig is fabricated to maintain constant load in concentrated weld region. When external load is applied on the weld region, the deformed specimen experiences acoustic emission signals form the weld defect region which are potential source of releasing stress energy. Liner Location Technique (LLT) is adopted for AE singal studies and the generated signal is processed by 2-channel USB—AE node and AE-WIN software. The tests are conducted on two different samples having each defect. A conventional NDT method i.e. X-ray Radiography is conducted on the samples to know the defect ranging and correlated with AE signatures. This study will be helpful to standardize the AE signals for different implanted weld defects of SS 316L materials and it is found that, the parameter “counts vs. amplitude” has given the widest distinction with respect to the type of defects.

An Indoor Pedestrian Localization Algorithm Based on Multi-Sensor Information Fusion

For existing indoor localization algorithm has low accuracy, high cost in deployment and maintenance, lack of robustness, and low sensor utilization, this paper proposes a particle filter algorithm based on multi-sensor fusion. The pedestrian’s localization in indoor environment is described as dynamic system state estimation problem. The algorithm combines the smart mobile terminal with indoor localization, and filters the result of localization with the particle filter. In this paper, a dynamic interval particle filter algorithm based on pedestrian dead reckoning (PDR) information and RSSI localization information have been used to improve the filtering precision and the stability. Moreover, the localization results will be uploaded to the server in time, and the location fingerprint database will be built incrementally, which can adapt the dynamic changes of the indoor environment. Experimental results show that the algorithm based on multi-sensor improves the localization accuracy and robustness compared with the location algorithm based on Wi-Fi.

A Study of Multi-sensor Data Fusion System Based on MAS for Nutrient Solution Measurement

For complementarity and redundancy of multi-sensor data fusion (MSDF) system,it is an effective approach for multiple components measurement.In order to measure nutrient solution on-line,a dynamic and complex system under greenhouse environment,sensors should have intelligent properties including self-calibration and self-compensation. Meanwhile,it is necessary for multiple sensors to cooperate and interact for enhancing reliability of multi-sensor system. Because of the properties of multi-agent system (MAS),it is an appropriate tool to study MSDF system.This paper proposed an architecture of MSDF system based on MAS for the multiple components measurement of nutrient solution.The sensor agent's structure and function modules are analyzed and described in detail,the formal definitions are given,too.The relations of the sensors are modeled to implement reliability diagnosis of the multi-sensor system,so that the reliability of nutrient control system is enhanced.This study offers an effective approach for the study of MSDF.

Multi-sensor Data Fusion by Improved Hough Transformation

In this paper we present an evidence-gathering approach to slove the multi-sensor data fusion problem. It uses an improved Hough transformation method rather than the usual statistical or geometric approach to extract the directions and positions of the walls in a room and update the location (orientation and position)of a mobile robot. The simulation results show that the proposed method is of practical importance since it is very simple and easy to implement.

Adaptive Multi-Feature Fusion for Vehicle Micro-Motor Noise Recognition Considering Auditory Perception

The deployment of vehicle micro-motors has witnessed an expansion owing to the progression in electrification and intelligent technologies.However,some micro-motors may exhibit design deficiencies,component wear,assembly errors,and other imperfections that may arise during the design or manufacturing phases.Conse-quently,these micro-motors might generate anomalous noises during their operation,consequently exerting a substantial adverse influence on the overall comfort of drivers and passengers.Automobile micro-motors exhibit a diverse array of structural variations,consequently leading to the manifestation of a multitude of distinctive auditory irregularities.To address the identification of diverse forms of abnormal noise,this research presents a novel approach rooted in the utilization of vibro-acoustic fusion-convolutional neural network(VAF-CNN).This method entails the deployment of distinct network branches,each serving to capture disparate features from the multi-sensor data,all the while considering the auditory perception traits inherent in the human auditory sys-tem.The intermediary layer integrates the concept of adaptive weighting of multi-sensor features,thus affording a calibration mechanism for the features hailing from multiple sensors,thereby enabling a further refinement of features within the branch network.For optimal model efficacy,a feature fusion mechanism is implemented in the concluding layer.To substantiate the efficacy of the proposed approach,this paper initially employs an augmented data methodology inspired by modified SpecAugment,applied to the dataset of abnormal noise sam-ples,encompassing scenarios both with and without in-vehicle interior noise.This serves to mitigate the issue of limited sample availability.Subsequent comparative evaluations are executed,contrasting the performance of the model founded upon single-sensor data against other feature fusion models reliant on multi-sensor data.The experimental results substantiate that the suggested methodology yields heightened recognition accuracy and greater resilience against interference.Moreover,it holds notable practical significance in the engineering domain,as it furnishes valuable support for the targeted management of noise emanating from vehicle micro-motors.

Obstacle avoidance technology of bionic quadruped robot based on multi-sensor information fusion

In order to improve the ability of a bionic quadruped robot to percept the location of obstacles in a complex and dynamic environment, the information fusion between an ultrasonic sensor and a binocular sensor was studied under the condition that the robot moves in the Walk gait on a structured road. Firstly, the distance information of obstacles from these two sensors was separately processed by the Kalman filter algorithm, which largely reduced the noise interference. After that, we obtained two groups of estimated distance values from the robot to the obstacle and a variance of the estimation value. Additionally, a fusion of the estimation values and the variances was achieved based on the STF fusion algorithm. Finally, a simulation was performed to show that the curve of a real value was tracked well by that of the estimation value, which attributes to the effectiveness of the Kalman filter algorithm. In contrast to statistics before fusion, the fusion variance of the estimation value was sharply decreased. The precision of the position information is 4. 6 cm, which meets the application requirements of the robot.

Multi-Sensor Image Fusion: A Survey of the State of the Art

Image fusion has been developing into an important area of research. In remote sensing, the use of the same image sensor in different working modes, or different image sensors, can provide reinforcing or complementary information. Therefore, it is highly valuable to fuse outputs from multiple sensors (or the same sensor in different working modes) to improve the overall performance of the remote images, which are very useful for human visual perception and image processing task. Accordingly, in this paper, we first provide a comprehensive survey of the state of the art of multi-sensor image fusion methods in terms of three aspects: pixel-level fusion, feature-level fusion and decision-level fusion. An overview of existing fusion strategies is then introduced, after which the existing fusion quality measures are summarized. Finally, this review analyzes the development trends in fusion algorithms that may attract researchers to further explore the research in this field.

A Robust Approach of Multi-sensor Fusion for Fault Diagnosis Using Convolution Neural Network

Multi-sensor measurement iswidely employed in rotatingmachinery to ensure the safety ofmachines.The information provided by the single sensor is not comprehensive.Multi-sensor signals can provide complementary information in characterizing the health condition of machines.This paper proposed a multi-sensor fusion convolution neural network(MF-CNN)model.The proposed model adds a 2-D convolution layer before the classical 1-D CNN to automatically extract complementary features of multi-sensor signals and minimize the loss of information.A series of experiments are carried out on a rolling bearing test rig to verify the model.Vibration and sound signals are fused to achieve higher classification accuracy than typical machine learning model.In addition,the model is further applied to gas turbine abnormal detection,and shows great robustness and generalization.

柔性浅埋物的声-振智能探测

提出一种基于目标检测算法的柔性浅埋物的声-振智能探测方法,将声波激励、激光散斑干涉测振和目标检测算法有机结合,用以柔性浅埋物的大范围快速探测。在论述YOLO系列目标检测算法原理的基础上,选择并优化柔性浅埋物的智能探测网络模型;然后,搭建声-光融合智能探测系统,构建不同柔性浅埋物的激光散斑干涉条纹图数据集;最后,对数据集进行训练和测试,验证该算法用于干涉条纹图识别的可行性。实验结果表明:在给定实验条件下,柔性浅埋物智能探测网络模型的精确率为98.39%,召回率为84.72%,平均识别精度为99.66%。该声-振智能探测方法可以在给定实验环境下对多种柔性浅埋物的激光散斑干涉条纹图进行智能识别,适用于浅层地下柔性掩埋物的大面积快速探测。

基于声阵列时空关联特征融合的不平衡局部放电类型识别方法

麦克风阵列能非接触且灵活地对电力设备局部放电现象进行检测,但现有方法对麦克风阵列的数据特点考虑不足,对局放类型识别的研究不足。针对麦克风阵列数据的关联性特征和不平衡分布特点,首先对麦克风阵列数据的时间关联性和空间关联性特征进行深入分析。然后,以1维卷积神经网络和压缩-激活关联性挖掘方法为基础,提出基于时空关联特征融合的声阵列数据局部放电类型识别模型。最后,针对麦克风阵列数据类别间分布不平衡问题,使用损失函数调整法和数据分布调整法进行应对。仿真结果表明:相对不考虑关联性的方法,该文所提方法的精确率、召回率提升均大于12%;相对不考虑样本不均衡性方法,该文所用方法在精确率和召回率均提高大于60%,验证了基于声阵列数据的局放类型识别中考虑数据关联性和不平衡性的必要性。

Camera,LiDAR,and IMU Based Multi-Sensor Fusion SLAM:A Survey

In recent years,Simultaneous Localization And Mapping(SLAM)technology has prevailed in a wide range of applications,such as autonomous driving,intelligent robots,Augmented Reality(AR),and Virtual Reality(VR).Multi-sensor fusion using the most popular three types of sensors(e.g.,visual sensor,LiDAR sensor,and IMU)is becoming ubiquitous in SLAM,in part because of the complementary sensing capabilities and the inevitable shortages(e.g.,low precision and long-term drift)of the stand-alone sensor in challenging environments.In this article,we survey thoroughly the research efforts taken in this field and strive to provide a concise but complete review of the related work.Firstly,a brief introduction of the state estimator formation in SLAM is presented.Secondly,the state-of-the-art algorithms of different multi-sensor fusion algorithms are given.Then we analyze the deficiencies associated with the reviewed approaches and formulate some future research considerations.This paper can be considered as a brief guide to newcomers and a comprehensive reference for experienced researchers and engineers to explore new interesting orientations.

基于声振信号融合的设备智能诊断

单一传感器检测易受外界干扰或自身故障等多种因素限制导致滚动轴承故障诊断结果欠佳一直是设备智能诊断领域难点问题。针对上述问题,提出了一种基于声振信号融合的智能诊断方法。首先,通过传感器配置采集滚动轴承的振动信号和声音信号;然后,利用变分模态分解(variational mode decomposition,VMD)对振动信号和声音信号进行分解与重构;随后,将重构后的声振信号输入双通道卷积神经网络(dual-channel convolutional neural network,DCNN)实现故障特征提取与特征融合;最后,将提取和融合的故障特征输入至DCNN网络SoftMax层进行故障分类建模。结果表明,与基于单一振动信号的CNN故障诊断模型相比,提出的基于声振信号融合的故障诊断方法准确率可以达到99.3%,融合后的特征更能有效区分设备不同的故障状态。

基于深度融合模型的气膜密封端面状态识别方法

气膜密封装置是工业领域应用广泛的一种密封技术,其可靠的密封性能对于设备正常运行至关重要。气膜密封装置的动静密封环接触端面相对运动会产生摩擦,摩擦过程会产生复杂的声发射信号,这些信号往往隐含密封端面运行状况的重要信息。采用传统的方法往往难以准确识别和分类这些微弱的特征信号,因此需要开发更高精度的故障诊断方法。针对机械密封动、静环端面摩擦状态难以识别这一问题,以气膜密封装置为研究对象,提出了一种基于深度融合模型的气膜密封端面状态识别方法。首先,采用声发射传感器及采集设备,对密封端面的声发射信号进行了采集;其次,利用小波包变换方法对采集到的信号进行了滤波处理,并提取了时域和频域的微弱特征;然后,将深度随机森林(DRF)作为分类层融入卷积神经网络(CNN)形成了融合模型,对预先处理过的密封装置运行状态的特征信息进行了识别和分类;最后,根据实验的泄漏量,使用混淆矩阵和受试者工作曲线分析了两种模型的特征提取能力。研究结果表明:CNN-DRF融合模型对于密封端面声发射信号的两种特征识别精度分别为96%和98%,与传统的CNN模型相比,其可以充分提取信号特征信息,具有更出色的故障诊断能力。