Disparity estimation for multi-scale multi-sensor fusion

The perception module of advanced driver assistance systems plays a vital role.Perception schemes often use a single sensor for data processing and environmental perception or adopt the information processing results of various sensors for the fusion of the detection layer.This paper proposes a multi-scale and multi-sensor data fusion strategy in the front end of perception and accomplishes a multi-sensor function disparity map generation scheme.A binocular stereo vision sensor composed of two cameras and a light deterction and ranging(LiDAR)sensor is used to jointly perceive the environment,and a multi-scale fusion scheme is employed to improve the accuracy of the disparity map.This solution not only has the advantages of dense perception of binocular stereo vision sensors but also considers the perception accuracy of LiDAR sensors.Experiments demonstrate that the multi-scale multi-sensor scheme proposed in this paper significantly improves disparity map estimation.

State Estimation Method for GNSS/INS/Visual Multi-sensor Fusion Based on Factor Graph Optimization for Unmanned System

With the development of unmanned driving technology,intelligent robots and drones,high-precision localization,navigation and state estimation technologies have also made great progress.Traditional global navigation satellite system/inertial navigation system(GNSS/INS)integrated navigation systems can provide high-precision navigation information continuously.However,when this system is applied to indoor or GNSS-denied environments,such as outdoor substations with strong electromagnetic interference and complex dense spaces,it is often unable to obtain high-precision GNSS positioning data.The positioning and orientation errors will diverge and accumulate rapidly,which cannot meet the high-precision localization requirements in large-scale and long-distance navigation scenarios.This paper proposes a method of high-precision state estimation with fusion of GNSS/INS/Vision using a nonlinear optimizer factor graph optimization as the basis for multi-source optimization.Through the collected experimental data and simulation results,this system shows good performance in the indoor environment and the environment with partial GNSS signal loss.

Progress and Achievements of Multi-sensor Fusion Navigation in China during 2019—2023

Global Navigation Satellite System(GNSS)can provide all-weather,all-time,high-precision positioning,navigation and timing services,which plays an important role in national security,national economy,public life and other aspects.However,in environments with limited satellite signals such as urban canyons,tunnels,and indoor spaces,it is difficult to provide accurate and reliable positioning services only by satellite navigation.Multi-source sensor integrated navigation can effectively overcome the limitations of single-sensor navigation through the fusion of different types of sensor data such as Inertial Measurement Unit(IMU),vision sensor,and LiDAR,and provide more accurate,stable and robust navigation information in complex environments.We summarizes the research status of multi-source sensor integrated navigation technology,and focuses on the representative innovations and applications of integrated navigation and positioning technology by major domestic scientific research institutions in China during 2019—2023.

A Robust Approach of Multi-sensor Fusion for Fault Diagnosis Using Convolution Neural Network

Multi-sensor measurement iswidely employed in rotatingmachinery to ensure the safety ofmachines.The information provided by the single sensor is not comprehensive.Multi-sensor signals can provide complementary information in characterizing the health condition of machines.This paper proposed a multi-sensor fusion convolution neural network(MF-CNN)model.The proposed model adds a 2-D convolution layer before the classical 1-D CNN to automatically extract complementary features of multi-sensor signals and minimize the loss of information.A series of experiments are carried out on a rolling bearing test rig to verify the model.Vibration and sound signals are fused to achieve higher classification accuracy than typical machine learning model.In addition,the model is further applied to gas turbine abnormal detection,and shows great robustness and generalization.

Obstacle detection using multi-sensor fusion

This paper presents an obstacle detection approach for blind pedestrians by fusing data from camera and laser sensor.For purely vision-based blind guidance system,it is difficult to discriminate low-level obstacles with cluttered road surface,while for purely laser-based system,it usually requires to scan the forward environment,which turns out to be very inconvenient.To overcome these inherent problems when using camera and laser sensor independently,a sensor-fusion model is proposed to associate range data from laser domain with edges from image domain.Based on this fusion model,obstacle's position,size and shape can be estimated.The proposed method is tested in several indoor scenes,and its efficiency is confirmed.

Camera,LiDAR,and IMU Based Multi-Sensor Fusion SLAM:A Survey

In recent years,Simultaneous Localization And Mapping(SLAM)technology has prevailed in a wide range of applications,such as autonomous driving,intelligent robots,Augmented Reality(AR),and Virtual Reality(VR).Multi-sensor fusion using the most popular three types of sensors(e.g.,visual sensor,LiDAR sensor,and IMU)is becoming ubiquitous in SLAM,in part because of the complementary sensing capabilities and the inevitable shortages(e.g.,low precision and long-term drift)of the stand-alone sensor in challenging environments.In this article,we survey thoroughly the research efforts taken in this field and strive to provide a concise but complete review of the related work.Firstly,a brief introduction of the state estimator formation in SLAM is presented.Secondly,the state-of-the-art algorithms of different multi-sensor fusion algorithms are given.Then we analyze the deficiencies associated with the reviewed approaches and formulate some future research considerations.This paper can be considered as a brief guide to newcomers and a comprehensive reference for experienced researchers and engineers to explore new interesting orientations.

Deep Weighted Fusion Learning(DWFL)-based multi-sensor fusion model for accurate building occupancy detection

With the advancement of artificial intelligence,the dominance of deep learning(DL)models over ordinary machine learning(ML)algorithms has become a reality in recent years due to its capability of handling complex pattern recognition without manual feature pre-definition.With the growing demands for power savings,building energy loss reduction could benefit from DL techniques.For buildings/rooms with the varying number of occupants,heating,ventilation,and air conditioning(HVAC)systems are often found in operations without much necessity.To reduce the building’s energy loss,accurate occupancy detection/prediction(ODP)results could be used to control the proper operations of HVACs.However,ODP is a challenging issue due to multiple reasons,such as improper selection/deployment of sensors,inefficient learning algorithms for pattern recognition,varying room conditions,etc.To overcome the above challenges,we propose a DL-based framework,i.e.,Deep Weighted Fusion Learning(DWFL),to detect and predict occupancy counts with optimal multi-sensor fusion structure.DWFL fuses the extracted features from multiple types of sensors with the priority/weight assignment to each sensor.Such weight assignment considers different room conditions and the pros/cons of each type of sensor.To evaluate DWFL model in terms of occupancy prediction accuracy,we have set up an experimental testbed with low-cost cameras,carbon dioxide(CO_(2)),and passive infrared(PIR)sensors.Among the recently proposed occupancy detection models,DeepFusion utilized deep learning model on heterogeneous sensor data and achieved 88%accuracy in occupancy count estimation(Xue et al.,2019).Another deep learning-based model MI-PIR achieved 91%accuracy on raw analog data from PIR sensors(Andrews et al.,2020).Our research outcome is 94%.Therefore,the experiment results show that our DWFL scheme outperforms the state-of-the-art ODP methods by 3%.

Maneuvering Vehicle Tracking Based on Multi-sensor Fusion

Maneuvering targets tracking is a fundamental task in intelligent vehicle research. Thispaper focuses on the problem of fusion between radar and image sensors in targets tracking. Inorder to improve positioning accuracy and narrow down the image working area, a novel methodthat integrates radar filter with image intensity is proposed to establish an adaptive vision window.A weighted Hausdor? distance is introduced to define the functional relationship between image andmodel projection, and a modified simulated annealing algorithm is used to find optimum orientationparameter. Furthermore, the global state is estimated, which refers to the distributed data fusionalgorithm. Experiment results show that our method is accurate.

Research on the Mechanism of Multi-Sensor Fusion Configuration Based on the Optimal Principle of the Vehicle

In order to address the issue of sensor configuration redundancy in intelligent driving,this paper constructs a multi-objective optimization model that considers cost,coverage ability,and perception performance.And then,combining a specific set of parameters,the NSGA-II algorithm is used to solve the multi-objective model established in this paper,and a Pareto front containing 24 typical configuration schemes is extracted after considering empirical constraints.Finally,using the decision preference method proposed in this paper that combines subjective and objective factors,decision scores are calculated and ranked for various configuration schemes from both cost and performance preferences.The research results indicate that the multi-objective optimization model established in this paper can screen and optimize various configuration schemes from the optimal principle of the vehicle,and the optimized configuration schemes can be quantitatively ranked to obtain the decision results for the vehicle under different preference tendencies.

Development and testing of a wireless smart toolholder with multi-sensor fusion

The smart toolholder is the core component in the development of intelligent and precise manufacturing.It enables in situ monitoring of cutting data and machining accuracy evolution and has become a focal point in academic research and industrial applications.However,current table and rotational dynamometers for milling force,vibration,and temperature testing suffer from cumbersome installation and provide only a single acquisition signal,which limits their use in laboratory settings.In this study,we propose a wireless smart toolholder with multi-sensor fusion for simultaneous sensing of milling force,vibration,and temperature signals.We select force,vibration,and temperature sensors suitable for smart toolholder fusion to adapt to the cutting environment.Thereafter,structural design,circular runout,dynamic balancing,static stiffness,and dynamic inherent frequency tests are conducted to assess its dynamic and static performance.Finally,the smart toolholder is tested for accuracy and repeatability in terms of force,vibration,and temperature.Experimental results demonstrate that the smart toolholder accurately captures machining data with a relative deviation of less than 1.5%compared with existing force gauges and provides high repeatability of milling temperature and vibration signals.Therefore,it is a smart solution for machining condition monitoring.

基于改进Centerfusion的自动驾驶3D目标检测模型

针对自动驾驶路面上目标漏检和错检的问题,提出一种基于改进Centerfusion的自动驾驶3D目标检测模型。该模型通过将相机信息和雷达特征融合,构成多通道特征数据输入,从而增强目标检测网络的鲁棒性,减少漏检问题;为了能够得到更加准确丰富的3D目标检测信息,引入了改进的注意力机制,用于增强视锥网格中的雷达点云和视觉信息融合;使用改进的损失函数优化边框预测的准确度。在Nuscenes数据集上进行模型验证和对比,实验结果表明,相较于传统的Centerfusion模型,提出的模型平均检测精度均值(mean Average Precision,mAP)提高了1.3%,Nuscenes检测分数(Nuscenes Detection Scores,NDS)提高了1.2%。

Hilbert空间中的fusion-Besselian框架与拟fusion-Riesz基

fusion框架作为Hilbert空间中g-框架的特例,与g-框架有许多类似的性质.该文在已有文献的基础上,借助算子理论知识,举反例说明去掉有限维空间的条件下结论不成立,进一步给出fusion-Besselian框架的算子刻画.结合fusion-Besselian框架的算子刻画和反例1,阐明在探讨该类框架性质时,应关注其适用条件和范围.随后讨论拟fusion-Riesz基与拟Riesz基、fusion-Besselian框架之间的关系.最后讨论fusion-Besselian框架和拟fusion-Riesz基的算子扰动,所得结论补充了算子扰动方面的研究.

Influence of heat treatment on microstructure,mechanical and corrosion behavior of WE43 alloy fabricated by laser-beam powder bed fusion

Magnesium(Mg)alloys are considered to be a new generation of revolutionary medical metals.Laser-beam powder bed fusion(PBF-LB)is suitable for fabricating metal implants withpersonalized and complicated structures.However,the as-built part usually exhibits undesirable microstructure and unsatisfactory performance.In this work,WE43 parts were firstly fabricated by PBF-LB and then subjected to heat treatment.Although a high densification rate of 99.91%was achieved using suitable processes,the as-built parts exhibited anisotropic and layeredmicrostructure with heterogeneously precipitated Nd-rich intermetallic.After heat treatment,fine and nano-scaled Mg24Y5particles were precipitated.Meanwhile,theα-Mg grainsunderwent recrystallization and turned coarsened slightly,which effectively weakened thetexture intensity and reduced the anisotropy.As a consequence,the yield strength and ultimate tensile strength were significantly improved to(250.2±3.5)MPa and(312±3.7)MPa,respectively,while the elongation was still maintained at a high level of 15.2%.Furthermore,the homogenized microstructure reduced the tendency of localized corrosion and favoredthe development of uniform passivation film.Thus,the degradation rate of WE43 parts was decreased by an order of magnitude.Besides,in-vitro cell experiments proved their favorable biocompatibility.

Influence of layer thickness on formation quality,microstructure,mechanical properties,and corrosion resistance of WE43 magnesium alloy fabricated by laser powder bed fusion

Laser powder bed fusion(L-PBF)of Mg alloys has provided tremendous opportunities for customized production of aeronautical and medical parts.Layer thickness(LT)is of great significance to the L-PBF process but has not been studied for Mg alloys.In this study,WE43 Mg alloy bulk cubes,porous scaffolds,and thin walls with layer thicknesses of 10,20,30,and 40μm were fabricated.The required laser energy input increased with increasing layer thickness and was different for the bulk cubes and porous scaffolds.Porosity tended to occur at the connection joints in porous scaffolds for LT40 and could be eliminated by reducing the laser energy input.For thin wall parts,a large overhang angle or a small wall thickness resulted in porosity when a large layer thicknesses was used,and the porosity disappeared by reducing the layer thickness or laser energy input.A deeper keyhole penetration was found in all occasions with porosity,explaining the influence of layer thickness,geometrical structure,and laser energy input on the porosity.All the samples achieved a high fusion quality with a relative density of over 99.5%using the optimized laser energy input.The increased layer thickness resulted to more precipitation phases,finer grain sizes and decreased grain texture.With the similar high fusion quality,the tensile strength and elongation of bulk samples were significantly improved from 257 MPa and 1.41%with the 10μm layer to 287 MPa and 15.12%with the 40μm layer,in accordance with the microstructural change.The effect of layer thickness on the compressive properties of porous scaffolds was limited.However,the corrosion rate of bulk samples accelerated with increasing the layer thickness,mainly attributed to the increased number of precipitation phases.

Pyrimethamine upregulates BNIP3 to interfere SNARE-mediated autophagosome-lysosomal fusion in hepatocellular carcinoma

Hepatocellular carcinoma (HCC) is one of the most common tumor types and remains a major clinical challenge. Increasing evidence has revealed that mitophagy inhibitors can enhance the effect of chemotherapy on HCC. However, few mitophagy inhibitors have been approved for clinical use in humans. Pyrimethamine (Pyr) is used to treat infections caused by protozoan parasites. Recent studies have reported that Pyr may be beneficial in the treatment of various tumors. However, its mechanism of action is still not clearly defined. Here, we found that blocking mitophagy sensitized cells to Pyr-induced apoptosis. Mechanistically, Pyr potently induced the accumulation of autophagosomes by inhibiting autophagosome-lysosome fusion in human HCC cells. In vitro and in vivo studies revealed that Pyr blocked autophagosome-lysosome fusion by upregulating BNIP3 to inhibit synaptosomal-associated protein 29 (SNAP29)-vesicle-associated membrane protein 8 (VAMP8) interaction. Moreover, Pyr acted synergistically with sorafenib (Sora) to induce apoptosis and inhibit HCC proliferation in vitro and in vivo. Pyr enhances the sensitivity of HCC cells to Sora, a common chemotherapeutic, by inhibiting mitophagy. Thus, these results provide new insights into the mechanism of action of Pyr and imply that Pyr could potentially be further developed as a novel mitophagy inhibitor. Notably, Pyr and Sora combination therapy could be a promising treatment for malignant HCC.

Alloy design for laser powder bed fusion additive manufacturing:a critical review

Metal additive manufacturing(AM)has been extensively studied in recent decades.Despite the significant progress achieved in manufacturing complex shapes and structures,challenges such as severe cracking when using existing alloys for laser powder bed fusion(L-PBF)AM have persisted.These challenges arise because commercial alloys are primarily designed for conventional casting or forging processes,overlooking the fast cooling rates,steep temperature gradients and multiple thermal cycles of L-PBF.To address this,there is an urgent need to develop novel alloys specifically tailored for L-PBF technologies.This review provides a comprehensive summary of the strategies employed in alloy design for L-PBF.It aims to guide future research on designing novel alloys dedicated to L-PBF instead of adapting existing alloys.The review begins by discussing the features of the L-PBF processes,focusing on rapid solidification and intrinsic heat treatment.Next,the printability of the four main existing alloys(Fe-,Ni-,Al-and Ti-based alloys)is critically assessed,with a comparison of their conventional weldability.It was found that the weldability criteria are not always applicable in estimating printability.Furthermore,the review presents recent advances in alloy development and associated strategies,categorizing them into crack mitigation-oriented,microstructure manipulation-oriented and machine learning-assisted approaches.Lastly,an outlook and suggestions are given to highlight the issues that need to be addressed in future work.

Infrared and Visible Image Fusion Based on Res2Net-Transformer Automatic Encoding and Decoding

A novel image fusion network framework with an autonomous encoder and decoder is suggested to increase thevisual impression of fused images by improving the quality of infrared and visible light picture fusion. The networkcomprises an encoder module, fusion layer, decoder module, and edge improvementmodule. The encoder moduleutilizes an enhanced Inception module for shallow feature extraction, then combines Res2Net and Transformerto achieve deep-level co-extraction of local and global features from the original picture. An edge enhancementmodule (EEM) is created to extract significant edge features. A modal maximum difference fusion strategy isintroduced to enhance the adaptive representation of information in various regions of the source image, therebyenhancing the contrast of the fused image. The encoder and the EEM module extract features, which are thencombined in the fusion layer to create a fused picture using the decoder. Three datasets were chosen to test thealgorithmproposed in this paper. The results of the experiments demonstrate that the network effectively preservesbackground and detail information in both infrared and visible images, yielding superior outcomes in subjectiveand objective evaluations.

Fusion SST from Infrared and Microwave Measurement of FY-3D Meteorological Satellite

Sea surface temperature(SST)is one of the important parameters of global ocean and climate research,which can be retrieved by satellite infrared and passive microwave remote sensing instruments.While satellite infrared SST offers high spatial resolution,it is limited by cloud cover.On the other hand,passive microwave SST provides all-weather observation but suffers from poor spatial resolution and susceptibility to environmental factors such as rainfall,coastal effects,and high wind speeds.To achieve high-precision,comprehensive,and high-resolution SST data,it is essential to fuse infrared and microwave SST measurements.In this study,data from the Fengyun-3D(FY-3D)medium resolution spectral imager II(MERSI-II)SST and microwave imager(MWRI)SST were fused.Firstly,the accuracy of both MERSIII SST and MWRI SST was verified,and the latter was bilinearly interpolated to match the 5km resolution grid of MERSI SST.After pretreatment and quality control of MERSI SST and MWRI SST,a Piece-Wise Regression method was employed to correct biases in MWRI SST.Subsequently,SST data were selected based on spatial resolution and accuracy within a 3-day window of the analysis date.Finally,an optimal interpolation method was applied to fuse the FY-3D MERSI-II SST and MWRI SST.The results demonstrated a significant improvement in spatial coverage compared to MERSI-II SST and MWRI SST.Furthermore,the fusion SST retained true spatial distribution details and exhibited an accuracy of–0.12±0.74℃compared to OSTIA SST.This study has improved the accuracy of FY satellite fusion SST products in China.

Characterization, preparation, and reuse of metallic powders for laser powder bed fusion: a review

Laser powder bed fusion(L-PBF) has attracted significant attention in both the industry and academic fields since its inception, providing unprecedented advantages to fabricate complex-shaped metallic components. The printing quality and performance of L-PBF alloys are infuenced by numerous variables consisting of feedstock powders, manufacturing process,and post-treatment. As the starting materials, metallic powders play a critical role in infuencing the fabrication cost, printing consistency, and properties. Given their deterministic roles, the present review aims to retrospect the recent progress on metallic powders for L-PBF including characterization, preparation, and reuse. The powder characterization mainly serves for printing consistency while powder preparation and reuse are introduced to reduce the fabrication costs.Various powder characterization and preparation methods are presented in the beginning by analyzing the measurement principles, advantages, and limitations. Subsequently, the effect of powder reuse on the powder characteristics and mechanical performance of L-PBF parts is analyzed, focusing on steels, nickel-based superalloys, titanium and titanium alloys, and aluminum alloys. The evolution trends of powders and L-PBF parts vary depending on specific alloy systems, which makes the proposal of a unified reuse protocol infeasible. Finally,perspectives are presented to cater to the increased applications of L-PBF technologies for future investigations. The present state-of-the-art work can pave the way for the broad industrial applications of L-PBF by enhancing printing consistency and reducing the total costs from the perspective of powders.

CAEFusion: A New Convolutional Autoencoder-Based Infrared and Visible Light Image Fusion Algorithm

To address the issues of incomplete information,blurred details,loss of details,and insufficient contrast in infrared and visible image fusion,an image fusion algorithm based on a convolutional autoencoder is proposed.The region attention module is meant to extract the background feature map based on the distinct properties of the background feature map and the detail feature map.A multi-scale convolution attention module is suggested to enhance the communication of feature information.At the same time,the feature transformation module is introduced to learn more robust feature representations,aiming to preserve the integrity of image information.This study uses three available datasets from TNO,FLIR,and NIR to perform thorough quantitative and qualitative trials with five additional algorithms.The methods are assessed based on four indicators:information entropy(EN),standard deviation(SD),spatial frequency(SF),and average gradient(AG).Object detection experiments were done on the M3FD dataset to further verify the algorithm’s performance in comparison with five other algorithms.The algorithm’s accuracy was evaluated using the mean average precision at a threshold of 0.5(mAP@0.5)index.Comprehensive experimental findings show that CAEFusion performs well in subjective visual and objective evaluation criteria and has promising potential in downstream object detection tasks.