CAEFusion: A New Convolutional Autoencoder-Based Infrared and Visible Light Image Fusion Algorithm

To address the issues of incomplete information,blurred details,loss of details,and insufficient contrast in infrared and visible image fusion,an image fusion algorithm based on a convolutional autoencoder is proposed.The region attention module is meant to extract the background feature map based on the distinct properties of the background feature map and the detail feature map.A multi-scale convolution attention module is suggested to enhance the communication of feature information.At the same time,the feature transformation module is introduced to learn more robust feature representations,aiming to preserve the integrity of image information.This study uses three available datasets from TNO,FLIR,and NIR to perform thorough quantitative and qualitative trials with five additional algorithms.The methods are assessed based on four indicators:information entropy(EN),standard deviation(SD),spatial frequency(SF),and average gradient(AG).Object detection experiments were done on the M3FD dataset to further verify the algorithm’s performance in comparison with five other algorithms.The algorithm’s accuracy was evaluated using the mean average precision at a threshold of 0.5(mAP@0.5)index.Comprehensive experimental findings show that CAEFusion performs well in subjective visual and objective evaluation criteria and has promising potential in downstream object detection tasks.

Infrared and Visible Image Fusion Based on Res2Net-Transformer Automatic Encoding and Decoding

A novel image fusion network framework with an autonomous encoder and decoder is suggested to increase thevisual impression of fused images by improving the quality of infrared and visible light picture fusion. The networkcomprises an encoder module, fusion layer, decoder module, and edge improvementmodule. The encoder moduleutilizes an enhanced Inception module for shallow feature extraction, then combines Res2Net and Transformerto achieve deep-level co-extraction of local and global features from the original picture. An edge enhancementmodule (EEM) is created to extract significant edge features. A modal maximum difference fusion strategy isintroduced to enhance the adaptive representation of information in various regions of the source image, therebyenhancing the contrast of the fused image. The encoder and the EEM module extract features, which are thencombined in the fusion layer to create a fused picture using the decoder. Three datasets were chosen to test thealgorithmproposed in this paper. The results of the experiments demonstrate that the network effectively preservesbackground and detail information in both infrared and visible images, yielding superior outcomes in subjectiveand objective evaluations.

Fusion SST from Infrared and Microwave Measurement of FY-3D Meteorological Satellite

Sea surface temperature(SST)is one of the important parameters of global ocean and climate research,which can be retrieved by satellite infrared and passive microwave remote sensing instruments.While satellite infrared SST offers high spatial resolution,it is limited by cloud cover.On the other hand,passive microwave SST provides all-weather observation but suffers from poor spatial resolution and susceptibility to environmental factors such as rainfall,coastal effects,and high wind speeds.To achieve high-precision,comprehensive,and high-resolution SST data,it is essential to fuse infrared and microwave SST measurements.In this study,data from the Fengyun-3D(FY-3D)medium resolution spectral imager II(MERSI-II)SST and microwave imager(MWRI)SST were fused.Firstly,the accuracy of both MERSIII SST and MWRI SST was verified,and the latter was bilinearly interpolated to match the 5km resolution grid of MERSI SST.After pretreatment and quality control of MERSI SST and MWRI SST,a Piece-Wise Regression method was employed to correct biases in MWRI SST.Subsequently,SST data were selected based on spatial resolution and accuracy within a 3-day window of the analysis date.Finally,an optimal interpolation method was applied to fuse the FY-3D MERSI-II SST and MWRI SST.The results demonstrated a significant improvement in spatial coverage compared to MERSI-II SST and MWRI SST.Furthermore,the fusion SST retained true spatial distribution details and exhibited an accuracy of–0.12±0.74℃compared to OSTIA SST.This study has improved the accuracy of FY satellite fusion SST products in China.

Deep near infrared light-excited stable synergistic photodynamic and photothermal therapies based on P-IR890 nano-photosensitizer constructed via a non-cyanine dye

The cyanine dyes represented by IR780 can achieve synergistic photodynamic therapy(PDT)and photothermal therapy(PTT)under the stimulation of near-infrared(NIR)light(commonly 808 nm).Unfortunately,the stability of NIR-excited cyanine dyes is not satisfactory.These cyanine dyes can be attacked by self-generated reactive oxygen species(ROS)during PDT processes,resulting in structural damage and rapid degradation,which is fatal for phototherapy.To address this issue,a novel non-cyanine dye(IR890)was elaborately designed and synthesized by our team.The maximum absorption wavelength of IR890 was located in the deep NIR region(ca.890 nm),which was beneficial for further improving tissue penetration depth.Importantly,IR890 exhibited good stability when continuously illuminated by deep NIR light.To improve the hydrophilicity and biocompatibility,the hydrophobic IR890 dye was grafted onto the side chain of hydrophilic polymer(POEGMA-b-PGMA-g-C≡CH)via click chemistry.Then,the synthesized POEGMA-b-PGMA-g-IR890 amphiphilic polymerwas utilized to prepare P-IR890 nano-photosensitizer via self-assembly method.Under irradiation with deep NIR light(850 nm,0.5 W/cm^(2),10 min),the dye degradation rate of P-IR890 was less than 5%.However,IR780 was almost completely degraded with the same light output power density and irradiation duration.In addition,P-IR890 could stably generate a large number of ROS and heat at the same time.It was rarely reported that the stable synergistic combination therapy of PDT and PTT could be efficiently performed by a single photosensitizer via irradiation with deep NIR light.P-IR890 exhibited favorable anti-tumor outcomes through apoptosis pathway.Therefore,the P-IR890 could provide a new insight into the design of photosensitizers and new opportunities for synergistic combination therapy of PDT and PTT.

Fault diagnosis method of AC motor rolling bearing based on heterogeneous data fusion of current and infrared image

In order to improve the accuracy of rolling bearing fault diagnosis when the motor is running under non-stationary conditions,an AC motor rolling bearing fault diagnosis method was proposed based on heterogeneous data fusion of current and infrared images.Firstly,VMD was used to decompose the motor current signal and extract the low-frequency component of the bearing fault signal.On this basis,the current signal was transformed into a two-dimensional graph suitable for convolutional neural network,and the data set was classified by convolutional neural network and softmax classifier.Secondly,the infrared image was segmented and the fault features were extracted,so as to calculate the similarity with the infrared image of the fault bearing in the library,and further the sigmod classifier was used to classify the data.Finally,a decision-level fusion method was introduced to fuse the current signal with the infrared image signal diagnosis result according to the weight,and the motor bearing fault diagnosis result was obtained.Through experimental verification,the proposed fault diagnosis method could be used for the fault diagnosis of motor bearing outer ring under the condition of load variation,and the accuracy of fault diagnosis can reach 98.85%.

Infrared polarization image fusion based on combination of NSST and improved PCA

In view of the problem that current mainstream fusion method of infrared polarization image—Multiscale Geometry Analysis method only focuses on a certain characteristic to image representation.And spatial domain fusion method,Principal Component Analysis(PCA)method has the shortcoming of losing small target,this paper presents a new fusion method of infrared polarization images based on combination of Nonsubsampled Shearlet Transformation(NSST)and improved PCA.This method can make full use of the effectiveness to image details expressed by NSST and the characteristics that PCA can highlight the main features of images.The combination of the two methods can integrate the complementary features of themselves to retain features of targets and image details fully.Firstly,intensity and polarization images are decomposed into low frequency and high frequency components with different directions by NSST.Secondly,the low frequency components are fused with improved PCA,while the high frequency components are fused by joint decision making rule with local energy and local variance.Finally,the fused image is reconstructed with the inverse NSST to obtain the final fused image of infrared polarization.The experiment results show that the method proposed has higher advantages than other methods in terms of detail preservation and visual effect.

An infrared target intrusion detection method based on feature fusion and enhancement

Infrared target intrusion detection has significant applications in the fields of military defence and intelligent warning.In view of the characteristics of intrusion targets as well as inspection difficulties,an infrared target intrusion detection algorithm based on feature fusion and enhancement was proposed.This algorithm combines static target mode analysis and dynamic multi-frame correlation detection to extract infrared target features at different levels.Among them,LBP texture analysis can be used to effectively identify the posterior feature patterns which have been contained in the target library,while motion frame difference method can detect the moving regions of the image,improve the integrity of target regions such as camouflage,sheltering and deformation.In order to integrate the advantages of the two methods,the enhanced convolutional neural network was designed and the feature images obtained by the two methods were fused and enhanced.The enhancement module of the network strengthened and screened the targets,and realized the background suppression of infrared images.Based on the experiments,the effect of the proposed method and the comparison method on the background suppression and detection performance was evaluated,and the results showed that the SCRG and BSF values of the method in this paper had a better performance in multiple data sets,and it’s detection performance was far better than the comparison algorithm.The experiment results indicated that,compared with traditional infrared target detection methods,the proposed method could detect the infrared invasion target more accurately,and suppress the background noise more effectively.

Extraction of the key infrared radiation temperature features concerning stress and crack evolution of loaded rocks

The infrared radiation temperature(IRT)variation concerning stress and crack evolution of rocks is a critical focus in rock mechanics domain and engineering disaster warning.In this paper,a methodology to extract the key IRT features related to stress and crack evolution of loaded rocks is proposed.Specifically,the wavelet denoising and reconstruction in thermal image sequence(WDRTIS)method is employed to eliminate temporal noise in thermal image sequences.Subsequently,the adaptive partition temperature drift correction(APTDC)method is introduced to alleviate temperature drift.On this basis,the spatial noise correction method based on threshold segmentation and adaptive median filtering(OTSU-AMF)is proposed to extract the key IRT features associated with microcracks of loaded rocks.Following temperature drift correction,IRT provides an estimation of the thermoelastic factor in rocks,typically around 5.29×10^(-5) MPa^(-1) for sandstones.Results reveal that the high-temperature concentrated region in cumulative thermal images of crack evolution(TICE)can elucidate the spatiotemporal evolution of localized damage.Additionally,heat dissipation of crack evolution(HDCE)acquired from TICE quantifies the progressive failure process of rocks.The proposed methodology enhances the reliability of IRT monitoring results and provides an innovative approach for conducting research in rock mechanics and monitoring engineering disasters.

Feature-Based Fusion of Dual Band Infrared Image Using Multiple Pulse Coupled Neural Network

To improve the quality of the infrared image and enhance the information of the object,a dual band infrared image fusion method based on feature extraction and a novel multiple pulse coupled neural network(multi-PCNN)is proposed.In this multi-PCNN fusion scheme,the auxiliary PCNN which captures the characteristics of feature image extracting from the infrared image is used to modulate the main PCNN,whose input could be original infrared image.Meanwhile,to make the PCNN fusion effect consistent with the human vision system,Laplacian energy is adopted to obtain the value of adaptive linking strength in PCNN.After that,the original dual band infrared images are reconstructed by using a weight fusion rule with the fire mapping images generated by the main PCNNs to obtain the fused image.Compared to wavelet transforms,Laplacian pyramids and traditional multi-PCNNs,fusion images based on our method have more information,rich details and clear edges.

Intelligent Fusion of Infrared and Visible Image Data Based on Convolutional Sparse Representation and Improved Pulse-Coupled Neural Network

Multi-source information can be obtained through the fusion of infrared images and visible light images,which have the characteristics of complementary information.However,the existing acquisition methods of fusion images have disadvantages such as blurred edges,low contrast,and loss of details.Based on convolution sparse representation and improved pulse-coupled neural network this paper proposes an image fusion algorithm that decompose the source images into high-frequency and low-frequency subbands by non-subsampled Shearlet Transform(NSST).Furthermore,the low-frequency subbands were fused by convolutional sparse representation(CSR),and the high-frequency subbands were fused by an improved pulse coupled neural network(IPCNN)algorithm,which can effectively solve the problem of difficulty in setting parameters of the traditional PCNN algorithm,improving the performance of sparse representation with details injection.The result reveals that the proposed method in this paper has more advantages than the existing mainstream fusion algorithms in terms of visual effects and objective indicators.

An infrared and visible image fusion method based upon multi-scale and top-hat transforms

The high-frequency components in the traditional multi-scale transform method are approximately sparse, which can represent different information of the details. But in the low-frequency component, the coefficients around the zero value are very few, so we cannot sparsely represent low-frequency image information. The low-frequency component contains the main energy of the image and depicts the profile of the image. Direct fusion of the low-frequency component will not be conducive to obtain highly accurate fusion result. Therefore, this paper presents an infrared and visible image fusion method combining the multi-scale and top-hat transforms. On one hand, the new top-hat-transform can effectively extract the salient features of the low-frequency component. On the other hand, the multi-scale transform can extract highfrequency detailed information in multiple scales and from diverse directions. The combination of the two methods is conducive to the acquisition of more characteristics and more accurate fusion results. Among them, for the low-frequency component, a new type of top-hat transform is used to extract low-frequency features, and then different fusion rules are applied to fuse the low-frequency features and low-frequency background; for high-frequency components, the product of characteristics method is used to integrate the detailed information in high-frequency. Experimental results show that the proposed algorithm can obtain more detailed information and clearer infrared target fusion results than the traditional multiscale transform methods. Compared with the state-of-the-art fusion methods based on sparse representation, the proposed algorithm is simple and efficacious, and the time consumption is significantly reduced.

Fusion of Infrared and Visible Images Using Fuzzy Based Siamese Convolutional Network

Traditional techniques based on image fusion are arduous in integrating complementary or heterogeneous infrared(IR)/visible(VS)images.Dissimilarities in various kind of features in these images are vital to preserve in the single fused image.Hence,simultaneous preservation of both the aspects at the same time is a challenging task.However,most of the existing methods utilize the manual extraction of features;and manual complicated designing of fusion rules resulted in a blurry artifact in the fused image.Therefore,this study has proposed a hybrid algorithm for the integration of multi-features among two heterogeneous images.Firstly,fuzzification of two IR/VS images has been done by feeding it to the fuzzy sets to remove the uncertainty present in the background and object of interest of the image.Secondly,images have been learned by two parallel branches of the siamese convolutional neural network(CNN)to extract prominent features from the images as well as high-frequency information to produce focus maps containing source image information.Finally,the obtained focused maps which contained the detailed integrated information are directly mapped with the source image via pixelwise strategy to result in fused image.Different parameters have been used to evaluate the performance of the proposed image fusion by achieving 1.008 for mutual information(MI),0.841 for entropy(EG),0.655 for edge information(EI),0.652 for human perception(HP),and 0.980 for image structural similarity(ISS).Experimental results have shown that the proposed technique has attained the best qualitative and quantitative results using 78 publically available images in comparison to the existing discrete cosine transform(DCT),anisotropic diffusion&karhunen-loeve(ADKL),guided filter(GF),random walk(RW),principal component analysis(PCA),and convolutional neural network(CNN)methods.

Sub-Regional Infrared-Visible Image Fusion Using Multi-Scale Transformation

Infrared-visible image fusion plays an important role in multi-source data fusion,which has the advantage of integrating useful information from multi-source sensors.However,there are still challenges in target enhancement and visual improvement.To deal with these problems,a sub-regional infrared-visible image fusion method(SRF)is proposed.First,morphology and threshold segmentation is applied to extract targets interested in infrared images.Second,the infrared back-ground is reconstructed based on extracted targets and the visible image.Finally,target and back-ground regions are fused using a multi-scale transform.Experimental results are obtained using public data for comparison and evaluation,which demonstrate that the proposed SRF has poten-tial benefits over other methods.

Driver Fatigue Detection System Based on Colored and Infrared Eye Features Fusion

Real-time detection of driver fatigue status is of great significance for road traffic safety.In this paper,a proposed novel driver fatigue detection method is able to detect the driver’s fatigue status around the clock.The driver’s face images were captured by a camera with a colored lens and an infrared lens mounted above the dashboard.The landmarks of the driver’s face were labeled and the eye-area was segmented.By calculating the aspect ratios of the eyes,the duration of eye closure,frequency of blinks and PERCLOS of both colored and infrared,fatigue can be detected.Based on the change of light intensity detected by a photosensitive device,the weight matrix of the colored features and the infrared features was adjusted adaptively to reduce the impact of lighting on fatigue detection.Video samples of the driver’s face were recorded in the test vehicle.After training the classification model,the results showed that our method has high accuracy on driver fatigue detection in both daytime and nighttime.

Study on the Authenticity of Identification Methods in White Spirit by Infrared Spectrometer

This experiment studies on the used infrared spectroscopy to establish technology methods for liquor identification methods, as well as offers the science data for establishment of the fingerprint in white spirit. The results have shown that using near-infrared spectroscopy analysis of liquor has the obvious features such as strong specificity, good reproducibility, simple operation, and finally confirmed that it is an authentic and ideal method for identification in white spirit.

Research on Infrared Image Fusion Technology Based on Road Crack Detection

This study aimed to propose road crack detection method based on infrared image fusion technology.By analyzing the characteristics of road crack images,this method uses a variety of infrared image fusion methods to process different types of images.The use of this method allows the detection of road cracks,which not only reduces the professional requirements for inspectors,but also improves the accuracy of road crack detection.Based on infrared image processing technology,on the basis of in-depth analysis of infrared image features,a road crack detection method is proposed,which can accurately identify the road crack location,direction,length,and other characteristic information.Experiments showed that this method has a good effect,and can meet the requirement of road crack detection.

Automated Registration for Infrared Image Based on Wavelet Analysis

To develop a quick, accurate and antinoise automated image registration technique for infrared images, the wavelet analysis technique was used to extract the feature points in two images followed by the compensation for input image with angle difference between them. A hi erarchical feature matching algorithm was adopted to get the final transform parameters between the two images. The simulation results for two infrared images show that the method can effectively, quickly and accurately register images and be antinoise to some extent.

Infrared modeling and imaging simulation of midcourse ballistic targets based on strap-down platform

An infrared （IR） imaging simulation framework based on the strap-down platform is proposed for midcourse ballistic targets. It overcomes the shortcoming of the existing algorithms, which cannot simulate IR imaging from the entire midcourse process. The proposed framework includes three steps, target characteristic modeling, motion modeling, and imaging modeling. In imaging modeling, the staring focal plane is taken into account due to its wide employment. In order to obtain IR images of high fidelity, especially that the fluctuation of the target signal-to-noise ratio （SNR） is reasonably similar to the actual one, this paper proposes an improved IR imaging simulation method. The proposed method considers two critical factors of the pixel plane, occupy-empty ratio and defect elements, which affect the imaging of targets markedly but are neglected in previous work. Finally, the IR image sequence of high fidelity is obtained. And the correlative parameters of simulation can be set according to the given scene. Thus the generated images can satisfy the needs of algorithms validation for tracking and recognition.

A pixel-level local contrast measure for infrared small target detection

Infrared(IR) small target detection is one of the key technologies of infrared search and track(IRST)systems. Existing methods have some limitations in detection performance, especially when the target size is irregular or the background is complex. In this paper, we propose a pixel-level local contrast measure(PLLCM), which can subdivide small targets and backgrounds at pixel level simultaneously.With pixel-level segmentation, the difference between the target and the background becomes more obvious, which helps to improve the detection performance. First, we design a multiscale sliding window to quickly extract candidate target pixels. Then, a local window based on random walker(RW) is designed for pixel-level target segmentation. After that, PLLCM incorporating probability weights and scale constraints is proposed to accurately measure local contrast and suppress various types of background interference. Finally, an adaptive threshold operation is applied to separate the target from the PLLCM enhanced map. Experimental results show that the proposed method has a higher detection rate and a lower false alarm rate than the baseline algorithms, while achieving a high speed.

Micro-motion dynamics analysis of ballistic targets based on infrared detection

The dynamic characteristics related to micro-motions, such as mechanical vibration or rotation, play an essential role in classifying and recognizing ballistic targets in the midcourse, and recent researches explore ways of extracting the micro-motion features from radar signals of ballistic targets. In this paper, we focus on how to investigate the micro-motion dynamic characteristics of the ballistic targets from the signals based on infrared (IR) detection, which is mainly achieved by analyzing the periodic fluctuation characteristics of the target IR irradiance intensity signatures. Simulation experiments demonstrate that the periodic characteristics of IR signatures can be used to distinguish different micro motion types and estimate related parameters. Consequently, this is possible to determine the micro-motion dynamics of ballistic targets based on IR detection.