Advancing automated pupillometry:a practical deep learning model utilizing infrared pupil images

AIM:To establish pupil diameter measurement algorithms based on infrared images that can be used in real-world clinical settings.METHODS:A total of 188 patients from outpatient clinic at He Eye Specialist Shenyang Hospital from Spetember to December 2022 were included,and 13470 infrared pupil images were collected for the study.All infrared images for pupil segmentation were labeled using the Labelme software.The computation of pupil diameter is divided into four steps:image pre-processing,pupil identification and localization,pupil segmentation,and diameter calculation.Two major models are used in the computation process:the modified YoloV3 and Deeplabv 3+models,which must be trained beforehand.RESULTS:The test dataset included 1348 infrared pupil images.On the test dataset,the modified YoloV3 model had a detection rate of 99.98% and an average precision(AP)of 0.80 for pupils.The DeeplabV3+model achieved a background intersection over union(IOU)of 99.23%,a pupil IOU of 93.81%,and a mean IOU of 96.52%.The pupil diameters in the test dataset ranged from 20 to 56 pixels,with a mean of 36.06±6.85 pixels.The absolute error in pupil diameters between predicted and actual values ranged from 0 to 7 pixels,with a mean absolute error(MAE)of 1.06±0.96 pixels.CONCLUSION:This study successfully demonstrates a robust infrared image-based pupil diameter measurement algorithm,proven to be highly accurate and reliable for clinical application.

Research on Defect Detection of Wind Turbine Blades Based on Morphology and Improved Otsu Algorithm Using Infrared Images

To address the issues of low accuracy and high false positive rate in traditional Otsu algorithm for defect detection on infrared images of wind turbine blades(WTB),this paper proposes a technique that combines morphological image enhancement with an improved Otsu algorithm.First,mathematical morphology’s differential multi-scale white and black top-hat operations are applied to enhance the image.The algorithm employs entropy as the objective function to guide the iteration process of image enhancement,selecting appropriate structural element scales to execute differential multi-scale white and black top-hat transformations,effectively enhancing the detail features of defect regions and improving the contrast between defects and background.Afterwards,grayscale inversion is performed on the enhanced infrared defect image to better adapt to the improved Otsu algorithm.Finally,by introducing a parameter K to adjust the calculation of inter-class variance in the Otsu method,the weight of the target pixels is increased.Combined with the adaptive iterative threshold algorithm,the threshold selection process is further fine-tuned.Experimental results show that compared to traditional Otsu algorithms and other improvements,the proposed method has significant advantages in terms of defect detection accuracy and reducing false positive rates.The average defect detection rate approaches 1,and the average Hausdorff distance decreases to 0.825,indicating strong robustness and accuracy of the method.

Meibomian glands segmentation in infrared images with limited annotation

●AIM:To investigate a pioneering framework for the segmentation of meibomian glands(MGs),using limited annotations to reduce the workload on ophthalmologists and enhance the efficiency of clinical diagnosis.●METHODS:Totally 203 infrared meibomian images from 138 patients with dry eye disease,accompanied by corresponding annotations,were gathered for the study.A rectified scribble-supervised gland segmentation(RSSGS)model,incorporating temporal ensemble prediction,uncertainty estimation,and a transformation equivariance constraint,was introduced to address constraints imposed by limited supervision information inherent in scribble annotations.The viability and efficacy of the proposed model were assessed based on accuracy,intersection over union(IoU),and dice coefficient.●RESULTS:Using manual labels as the gold standard,RSSGS demonstrated outcomes with an accuracy of 93.54%,a dice coefficient of 78.02%,and an IoU of 64.18%.Notably,these performance metrics exceed the current weakly supervised state-of-the-art methods by 0.76%,2.06%,and 2.69%,respectively.Furthermore,despite achieving a substantial 80%reduction in annotation costs,it only lags behind fully annotated methods by 0.72%,1.51%,and 2.04%.●CONCLUSION:An innovative automatic segmentation model is developed for MGs in infrared eyelid images,using scribble annotation for training.This model maintains an exceptionally high level of segmentation accuracy while substantially reducing training costs.It holds substantial utility for calculating clinical parameters,thereby greatly enhancing the diagnostic efficiency of ophthalmologists in evaluating meibomian gland dysfunction.

Infrared and Visible Image Fusion Based on Res2Net-Transformer Automatic Encoding and Decoding

A novel image fusion network framework with an autonomous encoder and decoder is suggested to increase thevisual impression of fused images by improving the quality of infrared and visible light picture fusion. The networkcomprises an encoder module, fusion layer, decoder module, and edge improvementmodule. The encoder moduleutilizes an enhanced Inception module for shallow feature extraction, then combines Res2Net and Transformerto achieve deep-level co-extraction of local and global features from the original picture. An edge enhancementmodule (EEM) is created to extract significant edge features. A modal maximum difference fusion strategy isintroduced to enhance the adaptive representation of information in various regions of the source image, therebyenhancing the contrast of the fused image. The encoder and the EEM module extract features, which are thencombined in the fusion layer to create a fused picture using the decoder. Three datasets were chosen to test thealgorithmproposed in this paper. The results of the experiments demonstrate that the network effectively preservesbackground and detail information in both infrared and visible images, yielding superior outcomes in subjectiveand objective evaluations.

CAEFusion: A New Convolutional Autoencoder-Based Infrared and Visible Light Image Fusion Algorithm

To address the issues of incomplete information,blurred details,loss of details,and insufficient contrast in infrared and visible image fusion,an image fusion algorithm based on a convolutional autoencoder is proposed.The region attention module is meant to extract the background feature map based on the distinct properties of the background feature map and the detail feature map.A multi-scale convolution attention module is suggested to enhance the communication of feature information.At the same time,the feature transformation module is introduced to learn more robust feature representations,aiming to preserve the integrity of image information.This study uses three available datasets from TNO,FLIR,and NIR to perform thorough quantitative and qualitative trials with five additional algorithms.The methods are assessed based on four indicators:information entropy(EN),standard deviation(SD),spatial frequency(SF),and average gradient(AG).Object detection experiments were done on the M3FD dataset to further verify the algorithm’s performance in comparison with five other algorithms.The algorithm’s accuracy was evaluated using the mean average precision at a threshold of 0.5(mAP@0.5)index.Comprehensive experimental findings show that CAEFusion performs well in subjective visual and objective evaluation criteria and has promising potential in downstream object detection tasks.

Optical design of a novel near-infrared phase contrast imaging(NI-PCI)diagnostic on the HL-2A tokamak

The optical design of near-infrared phase contrast imaging(NI-PCI)diagnosis on HL-2A is introduced in this paper.This scheme benefits from the great progress of near-infrared laser technology and is a broadening of traditional phase contrast technology.This diagnostic can work as a keen tool to measure plasma wavenumber spectra by inferring string-integrated plasma density fluctuations.Design of both the front optical path which is the path before the laser transmitting into the tokamak plasma and the rear optics which is the path after the laser passing through the plasma is detailed.The 1550 nm laser is chosen as the probe beam and highprecision optical components are designed to fit the laser beam,in which a phase plate with a 194-nm-deep silver groove is the key.Compared with the conventional 10.6μm laser-based PCI system on HL-2A,NI-PCI significantly overcomes the unwanted phase scintillation effect and promotes the measurement capability of high-wavenumber turbulence with an increased maximal measurable wavenumber from 15 cm^(-1)to 32.6 cm^(-1).

Research on fast detection method of infrared small targets under resourceconstrained conditions

Infrared small target detection is a common task in infrared image processing.Under limited computa⁃tional resources.Traditional methods for infrared small target detection face a trade-off between the detection rate and the accuracy.A fast infrared small target detection method tailored for resource-constrained conditions is pro⁃posed for the YOLOv5s model.This method introduces an additional small target detection head and replaces the original Intersection over Union(IoU)metric with Normalized Wasserstein Distance(NWD),while considering both the detection accuracy and the detection speed of infrared small targets.Experimental results demonstrate that the proposed algorithm achieves a maximum effective detection speed of 95 FPS on a 15 W TPU,while reach⁃ing a maximum effective detection accuracy of 91.9 AP@0.5,effectively improving the efficiency of infrared small target detection under resource-constrained conditions.

IR-YOLO: Real-Time Infrared Vehicle and Pedestrian Detection

Road traffic safety can decrease when drivers drive in a low-visibility environment.The application of visual perception technology to detect vehicles and pedestrians in infrared images proves to be an effective means of reducing the risk of accidents.To tackle the challenges posed by the low recognition accuracy and the substan-tial computational burden associated with current infrared pedestrian-vehicle detection methods,an infrared pedestrian-vehicle detection method A proposal is presented,based on an enhanced version of You Only Look Once version 5(YOLOv5).First,A head specifically designed for detecting small targets has been integrated into the model to make full use of shallow feature information to enhance the accuracy in detecting small targets.Second,the Focal Generalized Intersection over Union(GIoU)is employed as an alternative to the original loss function to address issues related to target overlap and category imbalance.Third,the distribution shift convolution optimization feature extraction operator is used to alleviate the computational burden of the model without significantly compromising detection accuracy.The test results of the improved algorithm show that its average accuracy(mAP)reaches 90.1%.Specifically,the Giga Floating Point Operations Per second(GFLOPs)of the improved algorithm is only 9.1.In contrast,the improved algorithms outperformed the other algorithms on similar GFLOPs,such as YOLOv6n(11.9),YOLOv8n(8.7),YOLOv7t(13.2)and YOLOv5s(16.0).The mAPs that are 4.4%,3%,3.5%,and 1.7%greater than those of these algorithms show that the improved algorithm achieves higher accuracy in target detection tasks under similar computational resource overhead.On the other hand,compared with other algorithms such as YOLOv8l(91.1%),YOLOv6l(89.5%),YOLOv7(90.8%),and YOLOv3(90.1%),the improved algorithm needs only 5.5%,2.3%,8.6%,and 2.3%,respectively,of the GFLOPs.The improved algorithm has shown significant advancements in balancing accuracy and computational efficiency,making it promising for practical use in resource-limited scenarios.

Heat transfer and temperature evolution in underground mininginduced overburden fracture and ground fissures: Optimal time window of UAV infrared monitoring

Heat transfer and temperature evolution in overburden fracture and ground fissures are one of the essential topics for the identification of ground fissures via unmanned aerial vehicle(UAV) infrared imager. In this study, discrete element software UDEC was employed to investigate the overburden fracture field under different mining conditions. Multiphysics software COMSOL were employed to investigate heat transfer and temperature evolution of overburden fracture and ground fissures under the influence of mining condition, fissure depth, fissure width, and month alternation. The UAV infrared field measurements also provided a calibration for numerical simulation. The results showed that for ground fissures connected to underground goaf(Fissure Ⅰ), the temperature difference increased with larger mining height and shallow buried depth. In addition, Fissure Ⅰ located in the boundary of the goaf have a greater temperature difference and is easier to be identified than fissures located above the mining goaf. For ground fissures having no connection to underground goaf(Fissure Ⅱ), the heat transfer is affected by the internal resistance of the overlying strata fracture when the depth of Fissure Ⅱ is greater than10 m, the temperature of Fissure Ⅱ gradually equals to the ground temperature as the fissures’ depth increases, and the fissures are difficult to be identified. The identification effect is most obvious for fissures larger than 16 cm under the same depth. In spring and summer, UAV infrared identification of mining fissures should be carried out during nighttime. This study provides the basis for the optimal time and season for the UAV infrared identification of different types of mining ground fissures.

Model-based deep learning for fiber bundle infrared image restoration

As the representative of flexibility in optical imaging media,in recent years,fiber bundles have emerged as a promising architecture in the development of compact visual systems.Dedicated to tackling the problems of universal honeycomb artifacts and low signal-to-noise ratio(SNR)imaging in fiber bundles,the iterative super-resolution reconstruction network based on a physical model is proposed.Under the constraint of solving the two subproblems of data fidelity and prior regularization term alternately,the network can efficiently“regenerate”the lost spatial resolution with deep learning.By building and calibrating a dual-path imaging system,the real-world dataset where paired low-resolution(LR)-high-resolution(HR)images on the same scene can be generated simultaneously.Numerical results on both the United States Air Force(USAF)resolution target and complex target objects demonstrate that the algorithm can restore high-contrast images without pixilated noise.On the basis of super-resolution reconstruction,compound eye image composition based on fiber bundle is also embedded in this paper for the actual imaging requirements.The proposed work is the first to apply a physical model-based deep learning network to fiber bundle imaging in the infrared band,effectively promoting the engineering application of thermal radiation detection.

Real-time and high-transmission middle-infrared optical imaging system based on a pixel-wise metasurface micro-polarization array

Real-time polarization medium-wave infrared(MIR)optical imaging systems enable the acquisition of infrared and polarization information for a target.At present,real-time polarization MIR devices face the following problems:poor real-time performance,low transmission and high requirements for fabrication and integration.Herein,we aim to improve the performance of real-time polarization imaging systems in the MIR waveband and solve the above-mentioned defects.Therefore,we propose a MIR polarization imaging system to achieve real-time polarization-modulated imaging with high transmission as well as improved performance based on a pixel-wise metasurface micro-polarization array(PMMPA).The PMMPA element comprises several linear polarization(LP)filters with different polarization angles.The optimization results demonstrate that the transmittance of the center field of view for the LP filters is up to 77%at a wavelength of4.0μm and an extinction ratio of 88 d B.In addition,a near-diffraction-limited real-time MIR imaging optical system is designed with a field of view of 5°and an F-number of 2.The simulation results show that an MIR polarization imaging system with excellent real-time performance and high transmission is achieved by using the optimized PMMPA element.Therefore,the method is compatible with the available optical system design technologies and provides a way to realize real-time polarization imaging in MIR wavebands.

Evaluation of facial temperature distribution changes during meditation using infrared thermal imaging:An experimental,cross-over study

Objective:To investigate the differences between meditation and resting states using infrared thermal imaging(IRTI)to determine facial temperature distribution features during meditation and annotate the patterns of facial temperature changes during meditation from the perspective of traditional Chinese medicine facial diagnosis.Methods:Each participant performed 10 min meditation and 10 min resting but in different sequences.A concentration test was set as the task load,followed by a meditation/resting or resting/meditation session,during which the participants'facial temperatures were observed using IRTI.Participants were scored on the Big Five Inventory(BFI)and Mindful Attention Awareness Scale(MAAS).Results:Forehead temperatures decreased more during meditation than during the resting state.The chin temperature increased only during meditation(P<.0001).For the subjects with meditation experience,there were significant differences in the temperatures of the left forehead(P<.01),right forehead(P<.01)and chin(P<.05)between the meditation and resting state at the 10~(th)min.In the nontask state,the BFI-Extraversion showed a negative correlation with the temperature of the left forehead(R=-0.41,P=.03).In the post-task state,the temperature of the left forehead was negatively correlated with scores on the MAAS(R=-0.42,P=.02).Conclusion:Using IRTI to study meditation offers a practical solution to the challenges in meditation research.The results indicate that an increase in chin temperature may be a representative feature of a meditation state,and forehead temperature is also a potential indicator.

Feasibility study of assessing cotton fiber maturity from near infrared hyperspectral imaging technique

Background Fiber maturity is a key cotton quality property,and its variability in a sample impacts fiber processing and dyeing performance.Currently,the maturity is determined by using established protocols in laboratories under a controlled environment.There is an increasing need to measure fiber maturity using low-cost(in general less than $20000)and small portable systems.In this study,a laboratory feasibility was performed to assess the ability of the shortwave infrared hyperspectral imaging(SWIR HSI)technique for determining the conditioned fiber maturity,and as a comparison,a bench-top commercial and expensive(in general greater than $60000)near infrared(NIR)instrument was used.Results Although SWIR HSI and NIR represent different measurement technologies,consistent spectral characteristics were observed between the two instruments when they were used to measure the maturity of the locule fiber samples in seed cotton and of the well-defined fiber samples,respectively.Partial least squares(PLS)models were established using different spectral preprocessing parameters to predict fiber maturity.The high prediction precision was observed by a lower root mean square error of prediction(RMSEP)(<0.046),higher R_(p)^(2)(>0.518),and greater percentage(97.0%)of samples within the 95% agreement range in the entire NIR region(1000-2500 nm)without the moisture band at 1940 nm.Conclusion SWIR HSI has a good potential for assessing cotton fiber maturity in a laboratory environment.

Research on Infrared Image Fusion Technology Based on Road Crack Detection

This study aimed to propose road crack detection method based on infrared image fusion technology.By analyzing the characteristics of road crack images,this method uses a variety of infrared image fusion methods to process different types of images.The use of this method allows the detection of road cracks,which not only reduces the professional requirements for inspectors,but also improves the accuracy of road crack detection.Based on infrared image processing technology,on the basis of in-depth analysis of infrared image features,a road crack detection method is proposed,which can accurately identify the road crack location,direction,length,and other characteristic information.Experiments showed that this method has a good effect,and can meet the requirement of road crack detection.

Fusion of Infrared and Visible Images Using Fuzzy Based Siamese Convolutional Network

Traditional techniques based on image fusion are arduous in integrating complementary or heterogeneous infrared(IR)/visible(VS)images.Dissimilarities in various kind of features in these images are vital to preserve in the single fused image.Hence,simultaneous preservation of both the aspects at the same time is a challenging task.However,most of the existing methods utilize the manual extraction of features;and manual complicated designing of fusion rules resulted in a blurry artifact in the fused image.Therefore,this study has proposed a hybrid algorithm for the integration of multi-features among two heterogeneous images.Firstly,fuzzification of two IR/VS images has been done by feeding it to the fuzzy sets to remove the uncertainty present in the background and object of interest of the image.Secondly,images have been learned by two parallel branches of the siamese convolutional neural network(CNN)to extract prominent features from the images as well as high-frequency information to produce focus maps containing source image information.Finally,the obtained focused maps which contained the detailed integrated information are directly mapped with the source image via pixelwise strategy to result in fused image.Different parameters have been used to evaluate the performance of the proposed image fusion by achieving 1.008 for mutual information(MI),0.841 for entropy(EG),0.655 for edge information(EI),0.652 for human perception(HP),and 0.980 for image structural similarity(ISS).Experimental results have shown that the proposed technique has attained the best qualitative and quantitative results using 78 publically available images in comparison to the existing discrete cosine transform(DCT),anisotropic diffusion&karhunen-loeve(ADKL),guided filter(GF),random walk(RW),principal component analysis(PCA),and convolutional neural network(CNN)methods.

Automated Registration for Infrared Image Based on Wavelet Analysis

To develop a quick, accurate and antinoise automated image registration technique for infrared images, the wavelet analysis technique was used to extract the feature points in two images followed by the compensation for input image with angle difference between them. A hi erarchical feature matching algorithm was adopted to get the final transform parameters between the two images. The simulation results for two infrared images show that the method can effectively, quickly and accurately register images and be antinoise to some extent.

Infrared polarization image fusion based on combination of NSST and improved PCA

In view of the problem that current mainstream fusion method of infrared polarization image—Multiscale Geometry Analysis method only focuses on a certain characteristic to image representation.And spatial domain fusion method,Principal Component Analysis(PCA)method has the shortcoming of losing small target,this paper presents a new fusion method of infrared polarization images based on combination of Nonsubsampled Shearlet Transformation(NSST)and improved PCA.This method can make full use of the effectiveness to image details expressed by NSST and the characteristics that PCA can highlight the main features of images.The combination of the two methods can integrate the complementary features of themselves to retain features of targets and image details fully.Firstly,intensity and polarization images are decomposed into low frequency and high frequency components with different directions by NSST.Secondly,the low frequency components are fused with improved PCA,while the high frequency components are fused by joint decision making rule with local energy and local variance.Finally,the fused image is reconstructed with the inverse NSST to obtain the final fused image of infrared polarization.The experiment results show that the method proposed has higher advantages than other methods in terms of detail preservation and visual effect.

High Performance of Imaging Extraction for Infrared Satellite Cloud Image

The isotherm is an important feature of infrared satellite cloud images (ISCI), which can directly reveal substantial information of cloud systems. The isotherm extraction of ISCI can remove the redundant information and therefore helps to compress the information of ISCI. In this paper, an isotherm extraction method is presented. The main aggregate of clouds can be segmented based on mathematical morphology. T algorithm and IP algorithm are then applied to extract the isotherms from the main aggregate of clouds. A concrete example for the extraction of isotherm based on IBM SP2 is described. The result shows that this is a high efficient algorithm. It can be used in feature extractions of infrared images for weather forecasts.

Design and implementation of GM- APD array readout circuit for infrared imaging

Based on an avalanche photodiode（ APD） detecting array working in Geiger mode（ GM-APD）, a high-performance infrared sensor readout integrated circuit（ ROIC） used for infrared 3D（ three-dimensional） imaging is proposed. The system mainly consists of three functional modules, including active quenching circuit（ AQC）, time-to-digital converter（ TDC） circuit and other timing controller circuit. Each AQC and TDC circuit together constitutes the pixel circuit. Under the cooperation with other modules, the current signal generated by the GM-APD sensor is detected by the AQC, and the photon time-of-flight（ TOF） is measured and converted to a digital signal output to achieve a better noise suppression and a higher detection sensitivity by the TDC. The ROIC circuit is fabricated by the CSMC 0. 5 μm standard CMOS technology. The array size is 8 × 8, and the center distance of two adjacent cells is 100μm. The measurement results of the chip showthat the performance of the circuit is good, and the chip can achieve 1 ns time resolution with a 250 MHz reference clock, and the circuit can be used in the array structure of the infrared detection system or focal plane array（ FPA）.

Infrared-PV:面向监控应用的红外目标检测数据集

红外摄像机虽然能够全天候24 h工作,但是相比于可见光摄像机,其获得的红外图像分辨率和信杂比低,目标纹理信息缺乏,因此足够的标记图像和进行模型优化设计对于提高基于深度学习的红外目标检测性能具有重要意义。为解决面向监控应用场景的红外目标检测数据集缺乏的问题,首先采用红外摄像机采集了不同极性的红外图像,基于自研图像标注软件实现了VOC格式的图像标注任务,构建了一个包含行人和车辆两类目标的红外图像数据集(Infrared-PV),并对数据集中的目标特性进行了统计分析。然后采用主流的基于深度学习的目标检测模型进行了模型训练与测试,定性和定量分析了YOLO系列和Faster R-CNN系列等模型对于该数据集的目标检测性能。构建的红外目标数据集共包含图像2138张,场景中目标包含白热、黑热和热力图3种模式。当采用各模型进行目标检测性能测试时,Cascade R-CNN模型性能最优,mAP0.5值达到了82.3%,YOLO v5系列模型能够兼顾实时性和检测精度的平衡,推理速度达到175.4帧/s的同时mAP0.5值仅降低2.7%。构建的红外目标检测数据集能够为基于深度学习的红外图像目标检测模型优化研究提供一定的数据支撑,同时也可以用于目标的红外特性分析。