Color Estimation for Thermal Infrared Imagery Based on Kernel PCA and Sparse Representation

Adding colors to monochrome thermal infrared images can help observers understand the scenery better. A nonlinear color estimation method for single-band thermal infrared imagery based on kernel principal component analysis (KPCA) and sparse representation was proposed. Nonlinear features of infrared image were extracted using KPCA. The relationship between image features and chromatic values was learned using sparse representation and a color estimation model was obtained. The thermal infrared images can be rendered automatically using the color estimation model. The experimental results show that the proposed method can render infrared image with an accurate color appearance. The proposed idea can also be used in other color estimation problem.

Double-ended passivator enables dark-current-suppressed colloidal quantum dot photodiodes for CMOS-integrated infrared imagers

Lead sulfide(PbS)colloidal quantum dot(CQD)photodiodes integrated with silicon-based readout integrated circuits(ROICs)offer a promising solution for the next-generation short-wave infrared(SWIR)imaging technology.Despite their potential,large-size CQD photodiodes pose a challenge due to high dark currents resulting from surface states on nonpassivated(100)facets and trap states generated by CQD fusion.In this work,we present a novel approach to address this issue by introducing double-ended ligands that supplementally passivate(100)facets of halidecapped large-size CQDs,leading to suppressed bandtail states and reduced defect concentration.Our results demonstrate that the dark current density is highly suppressed by about an order of magnitude to 9.6 nA cm^(2) at -10 mV,which is among the lowest reported for PbS CQD photodiodes.Furthermore,the performance of the photodiodes is exemplary,yielding an external quantum efficiency of 50.8%(which corresponds to a responsivity of 0.532 A W^(-1))and a specific detectivity of 2.5×10^(12) Jones at 1300 nm.By integrating CQD photodiodes with CMOS ROICs,the CQD imager provides high-resolution(640×512)SWIR imaging for infrared penetration and material discrimination.

Infrared and Visible Image Fusion Based on Res2Net-Transformer Automatic Encoding and Decoding

A novel image fusion network framework with an autonomous encoder and decoder is suggested to increase thevisual impression of fused images by improving the quality of infrared and visible light picture fusion. The networkcomprises an encoder module, fusion layer, decoder module, and edge improvementmodule. The encoder moduleutilizes an enhanced Inception module for shallow feature extraction, then combines Res2Net and Transformerto achieve deep-level co-extraction of local and global features from the original picture. An edge enhancementmodule (EEM) is created to extract significant edge features. A modal maximum difference fusion strategy isintroduced to enhance the adaptive representation of information in various regions of the source image, therebyenhancing the contrast of the fused image. The encoder and the EEM module extract features, which are thencombined in the fusion layer to create a fused picture using the decoder. Three datasets were chosen to test thealgorithmproposed in this paper. The results of the experiments demonstrate that the network effectively preservesbackground and detail information in both infrared and visible images, yielding superior outcomes in subjectiveand objective evaluations.

Research on Defect Detection of Wind Turbine Blades Based on Morphology and Improved Otsu Algorithm Using Infrared Images

To address the issues of low accuracy and high false positive rate in traditional Otsu algorithm for defect detection on infrared images of wind turbine blades(WTB),this paper proposes a technique that combines morphological image enhancement with an improved Otsu algorithm.First,mathematical morphology’s differential multi-scale white and black top-hat operations are applied to enhance the image.The algorithm employs entropy as the objective function to guide the iteration process of image enhancement,selecting appropriate structural element scales to execute differential multi-scale white and black top-hat transformations,effectively enhancing the detail features of defect regions and improving the contrast between defects and background.Afterwards,grayscale inversion is performed on the enhanced infrared defect image to better adapt to the improved Otsu algorithm.Finally,by introducing a parameter K to adjust the calculation of inter-class variance in the Otsu method,the weight of the target pixels is increased.Combined with the adaptive iterative threshold algorithm,the threshold selection process is further fine-tuned.Experimental results show that compared to traditional Otsu algorithms and other improvements,the proposed method has significant advantages in terms of defect detection accuracy and reducing false positive rates.The average defect detection rate approaches 1,and the average Hausdorff distance decreases to 0.825,indicating strong robustness and accuracy of the method.

IR-YOLO: Real-Time Infrared Vehicle and Pedestrian Detection

Road traffic safety can decrease when drivers drive in a low-visibility environment.The application of visual perception technology to detect vehicles and pedestrians in infrared images proves to be an effective means of reducing the risk of accidents.To tackle the challenges posed by the low recognition accuracy and the substan-tial computational burden associated with current infrared pedestrian-vehicle detection methods,an infrared pedestrian-vehicle detection method A proposal is presented,based on an enhanced version of You Only Look Once version 5(YOLOv5).First,A head specifically designed for detecting small targets has been integrated into the model to make full use of shallow feature information to enhance the accuracy in detecting small targets.Second,the Focal Generalized Intersection over Union(GIoU)is employed as an alternative to the original loss function to address issues related to target overlap and category imbalance.Third,the distribution shift convolution optimization feature extraction operator is used to alleviate the computational burden of the model without significantly compromising detection accuracy.The test results of the improved algorithm show that its average accuracy(mAP)reaches 90.1%.Specifically,the Giga Floating Point Operations Per second(GFLOPs)of the improved algorithm is only 9.1.In contrast,the improved algorithms outperformed the other algorithms on similar GFLOPs,such as YOLOv6n(11.9),YOLOv8n(8.7),YOLOv7t(13.2)and YOLOv5s(16.0).The mAPs that are 4.4%,3%,3.5%,and 1.7%greater than those of these algorithms show that the improved algorithm achieves higher accuracy in target detection tasks under similar computational resource overhead.On the other hand,compared with other algorithms such as YOLOv8l(91.1%),YOLOv6l(89.5%),YOLOv7(90.8%),and YOLOv3(90.1%),the improved algorithm needs only 5.5%,2.3%,8.6%,and 2.3%,respectively,of the GFLOPs.The improved algorithm has shown significant advancements in balancing accuracy and computational efficiency,making it promising for practical use in resource-limited scenarios.

Heat transfer and temperature evolution in underground mininginduced overburden fracture and ground fissures: Optimal time window of UAV infrared monitoring

Heat transfer and temperature evolution in overburden fracture and ground fissures are one of the essential topics for the identification of ground fissures via unmanned aerial vehicle(UAV) infrared imager. In this study, discrete element software UDEC was employed to investigate the overburden fracture field under different mining conditions. Multiphysics software COMSOL were employed to investigate heat transfer and temperature evolution of overburden fracture and ground fissures under the influence of mining condition, fissure depth, fissure width, and month alternation. The UAV infrared field measurements also provided a calibration for numerical simulation. The results showed that for ground fissures connected to underground goaf(Fissure Ⅰ), the temperature difference increased with larger mining height and shallow buried depth. In addition, Fissure Ⅰ located in the boundary of the goaf have a greater temperature difference and is easier to be identified than fissures located above the mining goaf. For ground fissures having no connection to underground goaf(Fissure Ⅱ), the heat transfer is affected by the internal resistance of the overlying strata fracture when the depth of Fissure Ⅱ is greater than10 m, the temperature of Fissure Ⅱ gradually equals to the ground temperature as the fissures’ depth increases, and the fissures are difficult to be identified. The identification effect is most obvious for fissures larger than 16 cm under the same depth. In spring and summer, UAV infrared identification of mining fissures should be carried out during nighttime. This study provides the basis for the optimal time and season for the UAV infrared identification of different types of mining ground fissures.

Advancing automated pupillometry:a practical deep learning model utilizing infrared pupil images

AIM:To establish pupil diameter measurement algorithms based on infrared images that can be used in real-world clinical settings.METHODS:A total of 188 patients from outpatient clinic at He Eye Specialist Shenyang Hospital from Spetember to December 2022 were included,and 13470 infrared pupil images were collected for the study.All infrared images for pupil segmentation were labeled using the Labelme software.The computation of pupil diameter is divided into four steps:image pre-processing,pupil identification and localization,pupil segmentation,and diameter calculation.Two major models are used in the computation process:the modified YoloV3 and Deeplabv 3+models,which must be trained beforehand.RESULTS:The test dataset included 1348 infrared pupil images.On the test dataset,the modified YoloV3 model had a detection rate of 99.98% and an average precision(AP)of 0.80 for pupils.The DeeplabV3+model achieved a background intersection over union(IOU)of 99.23%,a pupil IOU of 93.81%,and a mean IOU of 96.52%.The pupil diameters in the test dataset ranged from 20 to 56 pixels,with a mean of 36.06±6.85 pixels.The absolute error in pupil diameters between predicted and actual values ranged from 0 to 7 pixels,with a mean absolute error(MAE)of 1.06±0.96 pixels.CONCLUSION:This study successfully demonstrates a robust infrared image-based pupil diameter measurement algorithm,proven to be highly accurate and reliable for clinical application.

Research on fast detection method of infrared small targets under resourceconstrained conditions

Infrared small target detection is a common task in infrared image processing.Under limited computa⁃tional resources.Traditional methods for infrared small target detection face a trade-off between the detection rate and the accuracy.A fast infrared small target detection method tailored for resource-constrained conditions is pro⁃posed for the YOLOv5s model.This method introduces an additional small target detection head and replaces the original Intersection over Union(IoU)metric with Normalized Wasserstein Distance(NWD),while considering both the detection accuracy and the detection speed of infrared small targets.Experimental results demonstrate that the proposed algorithm achieves a maximum effective detection speed of 95 FPS on a 15 W TPU,while reach⁃ing a maximum effective detection accuracy of 91.9 AP@0.5,effectively improving the efficiency of infrared small target detection under resource-constrained conditions.

Meibomian glands segmentation in infrared images with limited annotation

●AIM:To investigate a pioneering framework for the segmentation of meibomian glands(MGs),using limited annotations to reduce the workload on ophthalmologists and enhance the efficiency of clinical diagnosis.●METHODS:Totally 203 infrared meibomian images from 138 patients with dry eye disease,accompanied by corresponding annotations,were gathered for the study.A rectified scribble-supervised gland segmentation(RSSGS)model,incorporating temporal ensemble prediction,uncertainty estimation,and a transformation equivariance constraint,was introduced to address constraints imposed by limited supervision information inherent in scribble annotations.The viability and efficacy of the proposed model were assessed based on accuracy,intersection over union(IoU),and dice coefficient.●RESULTS:Using manual labels as the gold standard,RSSGS demonstrated outcomes with an accuracy of 93.54%,a dice coefficient of 78.02%,and an IoU of 64.18%.Notably,these performance metrics exceed the current weakly supervised state-of-the-art methods by 0.76%,2.06%,and 2.69%,respectively.Furthermore,despite achieving a substantial 80%reduction in annotation costs,it only lags behind fully annotated methods by 0.72%,1.51%,and 2.04%.●CONCLUSION:An innovative automatic segmentation model is developed for MGs in infrared eyelid images,using scribble annotation for training.This model maintains an exceptionally high level of segmentation accuracy while substantially reducing training costs.It holds substantial utility for calculating clinical parameters,thereby greatly enhancing the diagnostic efficiency of ophthalmologists in evaluating meibomian gland dysfunction.

Fusion of visible and infrared imagery for face recognition

In recent years face recognition has received substantial attention, but still remained very challenging in real applications. Despite the variety of approaches and tools studied, face recognition is not accurate or robust enough to be used in uncontrolled environments. Infrared (IR) imagery of human faces offers a promising alternative to visible imagery, however, IR has its own limitations. In this paper, a scheme to fuse information from the two modalities is proposed. The scheme is based on eigenfaces and probabilistic neural network (PNN), using fuzzy integral to fuse the objective evidence supplied by each modality. Recognition rate is used to evaluate the fusion scheme. Experimental results show that the scheme improves recognition performance substantially.

Model-based deep learning for fiber bundle infrared image restoration

As the representative of flexibility in optical imaging media,in recent years,fiber bundles have emerged as a promising architecture in the development of compact visual systems.Dedicated to tackling the problems of universal honeycomb artifacts and low signal-to-noise ratio(SNR)imaging in fiber bundles,the iterative super-resolution reconstruction network based on a physical model is proposed.Under the constraint of solving the two subproblems of data fidelity and prior regularization term alternately,the network can efficiently“regenerate”the lost spatial resolution with deep learning.By building and calibrating a dual-path imaging system,the real-world dataset where paired low-resolution(LR)-high-resolution(HR)images on the same scene can be generated simultaneously.Numerical results on both the United States Air Force(USAF)resolution target and complex target objects demonstrate that the algorithm can restore high-contrast images without pixilated noise.On the basis of super-resolution reconstruction,compound eye image composition based on fiber bundle is also embedded in this paper for the actual imaging requirements.The proposed work is the first to apply a physical model-based deep learning network to fiber bundle imaging in the infrared band,effectively promoting the engineering application of thermal radiation detection.

Target tracking in infrared imagery using a novel particle filter

To address two challenging problems in infrared target tracking, target appearance changes and unpre- dictable abrupt motions, a novel particle filtering based tracking algorithm is introduced. In this method, a novel saliency model is proposed to distinguish the salient target from background, and the eigenspace model is invoked to adapt target appearance changes. To account for the abrupt motions efficiently, a two- step sampling method is proposed to combine the two observation models. The proposed tracking method is demonstrated through two real infrared image sequences, which include the changes of luminance and size, and the drastic abrupt motions of the target.

Real-time and high-transmission middle-infrared optical imaging system based on a pixel-wise metasurface micro-polarization array

Real-time polarization medium-wave infrared(MIR)optical imaging systems enable the acquisition of infrared and polarization information for a target.At present,real-time polarization MIR devices face the following problems:poor real-time performance,low transmission and high requirements for fabrication and integration.Herein,we aim to improve the performance of real-time polarization imaging systems in the MIR waveband and solve the above-mentioned defects.Therefore,we propose a MIR polarization imaging system to achieve real-time polarization-modulated imaging with high transmission as well as improved performance based on a pixel-wise metasurface micro-polarization array(PMMPA).The PMMPA element comprises several linear polarization(LP)filters with different polarization angles.The optimization results demonstrate that the transmittance of the center field of view for the LP filters is up to 77%at a wavelength of4.0μm and an extinction ratio of 88 d B.In addition,a near-diffraction-limited real-time MIR imaging optical system is designed with a field of view of 5°and an F-number of 2.The simulation results show that an MIR polarization imaging system with excellent real-time performance and high transmission is achieved by using the optimized PMMPA element.Therefore,the method is compatible with the available optical system design technologies and provides a way to realize real-time polarization imaging in MIR wavebands.

Evaluation of facial temperature distribution changes during meditation using infrared thermal imaging:An experimental,cross-over study

Objective:To investigate the differences between meditation and resting states using infrared thermal imaging(IRTI)to determine facial temperature distribution features during meditation and annotate the patterns of facial temperature changes during meditation from the perspective of traditional Chinese medicine facial diagnosis.Methods:Each participant performed 10 min meditation and 10 min resting but in different sequences.A concentration test was set as the task load,followed by a meditation/resting or resting/meditation session,during which the participants'facial temperatures were observed using IRTI.Participants were scored on the Big Five Inventory(BFI)and Mindful Attention Awareness Scale(MAAS).Results:Forehead temperatures decreased more during meditation than during the resting state.The chin temperature increased only during meditation(P<.0001).For the subjects with meditation experience,there were significant differences in the temperatures of the left forehead(P<.01),right forehead(P<.01)and chin(P<.05)between the meditation and resting state at the 10~(th)min.In the nontask state,the BFI-Extraversion showed a negative correlation with the temperature of the left forehead(R=-0.41,P=.03).In the post-task state,the temperature of the left forehead was negatively correlated with scores on the MAAS(R=-0.42,P=.02).Conclusion:Using IRTI to study meditation offers a practical solution to the challenges in meditation research.The results indicate that an increase in chin temperature may be a representative feature of a meditation state,and forehead temperature is also a potential indicator.

Feasibility study of assessing cotton fiber maturity from near infrared hyperspectral imaging technique

Background Fiber maturity is a key cotton quality property,and its variability in a sample impacts fiber processing and dyeing performance.Currently,the maturity is determined by using established protocols in laboratories under a controlled environment.There is an increasing need to measure fiber maturity using low-cost(in general less than $20000)and small portable systems.In this study,a laboratory feasibility was performed to assess the ability of the shortwave infrared hyperspectral imaging(SWIR HSI)technique for determining the conditioned fiber maturity,and as a comparison,a bench-top commercial and expensive(in general greater than $60000)near infrared(NIR)instrument was used.Results Although SWIR HSI and NIR represent different measurement technologies,consistent spectral characteristics were observed between the two instruments when they were used to measure the maturity of the locule fiber samples in seed cotton and of the well-defined fiber samples,respectively.Partial least squares(PLS)models were established using different spectral preprocessing parameters to predict fiber maturity.The high prediction precision was observed by a lower root mean square error of prediction(RMSEP)(<0.046),higher R_(p)^(2)(>0.518),and greater percentage(97.0%)of samples within the 95% agreement range in the entire NIR region(1000-2500 nm)without the moisture band at 1940 nm.Conclusion SWIR HSI has a good potential for assessing cotton fiber maturity in a laboratory environment.

Research on Infrared Image Fusion Technology Based on Road Crack Detection

This study aimed to propose road crack detection method based on infrared image fusion technology.By analyzing the characteristics of road crack images,this method uses a variety of infrared image fusion methods to process different types of images.The use of this method allows the detection of road cracks,which not only reduces the professional requirements for inspectors,but also improves the accuracy of road crack detection.Based on infrared image processing technology,on the basis of in-depth analysis of infrared image features,a road crack detection method is proposed,which can accurately identify the road crack location,direction,length,and other characteristic information.Experiments showed that this method has a good effect,and can meet the requirement of road crack detection.

9μm Cutoff 128×128 AlGaAs/GaAs Quantum Well Infrared Photodetector Focal Plane Arrays

We design and fabricate a 128 × 128 AlGaAs/GaAs quantum well infrared photodetector focal plane array （FPA）. The device is achieved by metal organic chemical vapor deposition and GaAs integrated circuit processing technology. A test structure of the photodetector with a mesa size of 300μm × 300μm is also made in order to obtain the device parameters. The measured dark current density at 77K is 1.5 × 10^-3A/cm^2 with a bias voltage of 2V. The peak of the responsivity spectrum is at 8.4μm,with a cutoff wavelength of 9μm. The blackbody detectivity is shown to be 3.95 × 10^8 （cm · Hz^1/2）/W. The final FPA is flip-chip bonded on a CMOS read-out integrated circuit. The infrared thermal images of some targets at room temperature background are successfully demonstrated at 80K operating temperature with a ratio of dead pixels of less than 1%.

Infrared polarization image fusion based on combination of NSST and improved PCA

In view of the problem that current mainstream fusion method of infrared polarization image—Multiscale Geometry Analysis method only focuses on a certain characteristic to image representation.And spatial domain fusion method,Principal Component Analysis(PCA)method has the shortcoming of losing small target,this paper presents a new fusion method of infrared polarization images based on combination of Nonsubsampled Shearlet Transformation(NSST)and improved PCA.This method can make full use of the effectiveness to image details expressed by NSST and the characteristics that PCA can highlight the main features of images.The combination of the two methods can integrate the complementary features of themselves to retain features of targets and image details fully.Firstly,intensity and polarization images are decomposed into low frequency and high frequency components with different directions by NSST.Secondly,the low frequency components are fused with improved PCA,while the high frequency components are fused by joint decision making rule with local energy and local variance.Finally,the fused image is reconstructed with the inverse NSST to obtain the final fused image of infrared polarization.The experiment results show that the method proposed has higher advantages than other methods in terms of detail preservation and visual effect.

Automated Registration for Infrared Image Based on Wavelet Analysis

To develop a quick, accurate and antinoise automated image registration technique for infrared images, the wavelet analysis technique was used to extract the feature points in two images followed by the compensation for input image with angle difference between them. A hi erarchical feature matching algorithm was adopted to get the final transform parameters between the two images. The simulation results for two infrared images show that the method can effectively, quickly and accurately register images and be antinoise to some extent.

Application value of indocyanine green fluorescence imaging in guiding sentinel lymph node biopsy diagnosis of gastric cancer: Meta-analysis

BACKGROUND Gastric cancer is a common malignant tumor of the digestive system worldwide,and its early diagnosis is crucial to improve the survival rate of patients.Indocyanine green fluorescence imaging(ICG-FI),as a new imaging technology,has shown potential application prospects in oncology surgery.The meta-analysis to study the application value of ICG-FI in the diagnosis of gastric cancer sentinel lymph node biopsy is helpful to comprehensively evaluate the clinical effect of this technology and provide more reliable guidance for clinical practice.AIM To assess the diagnostic efficacy of optical imaging in conjunction with indocya-nine green(ICG)-guided sentinel lymph node(SLN)biopsy for gastric cancer.METHODS Electronic databases such as PubMed,Embase,Medline,Web of Science,and the Cochrane Library were searched for prospective diagnostic tests of optical imaging combined with ICG-guided SLN biopsy.Stata 12.0 software was used for analysis by combining the"bivariable mixed effect model"with the"midas"command.The true positive value,false positive value,false negative value,true negative value,and other information from the included literature were extracted.A literature quality assessment map was drawn to describe the overall quality of the included literature.A forest plot was used for heterogeneity analysis,and P<0.01 was considered to indicate statistical significance.A funnel plot was used to assess publication bias,and P<0.1 was considered to indicate statistical significance.The summary receiver operating characteristic(SROC)curve was used to calculate the area under the curve(AUC)to determine the diagnostic accuracy.If there was interstudy heterogeneity(I2>50%),meta-regression analysis and subgroup analysis were performed.analysis were performed.RESULTS Optical imaging involves two methods:Near-infrared(NIR)imaging and fluorescence imaging.A combination of optical imaging and ICG-guided SLN biopsy was useful for diagnosis.The positive likelihood ratio was 30.39(95%CI:0.92-1.00),the sensitivity was 0.95(95%CI:0.82-0.99),and the specificity was 1.00(95%CI:0.92-1.00).The negative likelihood ratio was 0.05(95%CI:0.01-0.20),the diagnostic odds ratio was 225.54(95%CI:88.81-572.77),and the SROC AUC was 1.00(95%CI:The crucial values were sensitivity=0.95(95%CI:0.82-0.99)and specificity=1.00(95%CI:0.92-1.00).The Deeks method revealed that the"diagnostic odds ratio"funnel plot of SLN biopsy for gastric cancer was significantly asymmetrical(P=0.01),suggesting significant publication bias.Further meta-subgroup analysis revealed that,compared with fluorescence imaging,NIR imaging had greater sensitivity(0.98 vs 0.73).Compared with optical imaging immediately after ICG injection,optical imaging after 20 minutes obtained greater sensitivity(0.98 vs 0.70).Compared with that of patients with an average SLN detection number<4,the sensitivity of patients with a SLN detection number≥4 was greater(0.96 vs 0.68).Compared with hematoxylin-eosin(HE)staining,immunohistochemical(+HE)staining showed greater sensitivity(0.99 vs 0.84).Compared with subserous injection of ICG,submucosal injection achieved greater sensitivity(0.98 vs 0.40).Compared with 5 g/L ICG,0.5 and 0.05 g/L ICG had greater sensitivity(0.98 vs 0.83),and cT1 stage had greater sensitivity(0.96 vs 0.72)than cT2 to cT3 clinical stage.Compared with that of patients≤26,the sensitivity of patients>26 was greater(0.96 vs 0.65).Compared with the literature published before 2010,the sensitivity of the literature published after 2010 was greater(0.97 vs 0.81),and the differences were statistically significant(all P<0.05).CONCLUSION For the diagnosis of stomach cancer,optical imaging in conjunction with ICG-guided SLN biopsy is a therapeut-ically viable approach,especially for early gastric cancer.The concentration of ICG used in the SLN biopsy of gastric cancer may be too high.Moreover,NIR imaging is better than fluorescence imaging and may obtain higher sensitivity.