Emotion recognition from thermal infrared images using deep Boltzmann machine

Facial expression and emotion recognition from thermal infrared images has attracted more and more attentions in recent years. However, the features adopted in current work are either temperature statistical parameters extracted from the facial regions of interest or several hand-crafted features that are commonly used in visible spectrum. Till now there are no image features specially designed for thermal infrared images. In this paper, we propose using the deep Boltzmann machine to learn thermal features for emotion recognition from thermal infrared facial images. First, the face is located and normalized from the thermal infrared im- ages. Then, a deep Boltzmann machine model composed of two layers is trained. The parameters of the deep Boltzmann machine model are further fine-tuned for emotion recognition after pre-tralning of feature learning. Comparative experimental results on the NVIE database demonstrate that our approach outperforms other approaches using temperature statistic features or hand-crafted features borrowed from visible domain. The learned features from the forehead, eye, and mouth are more effective for discriminating valence dimension of emotion than other facial areas. In addition, our study shows that adding unlabeled data from other database during training can also improve feature learning performance.

Evaluation of facial temperature distribution changes during meditation using infrared thermal imaging:An experimental,cross-over study

Objective:To investigate the differences between meditation and resting states using infrared thermal imaging(IRTI)to determine facial temperature distribution features during meditation and annotate the patterns of facial temperature changes during meditation from the perspective of traditional Chinese medicine facial diagnosis.Methods:Each participant performed 10 min meditation and 10 min resting but in different sequences.A concentration test was set as the task load,followed by a meditation/resting or resting/meditation session,during which the participants'facial temperatures were observed using IRTI.Participants were scored on the Big Five Inventory(BFI)and Mindful Attention Awareness Scale(MAAS).Results:Forehead temperatures decreased more during meditation than during the resting state.The chin temperature increased only during meditation(P<.0001).For the subjects with meditation experience,there were significant differences in the temperatures of the left forehead(P<.01),right forehead(P<.01)and chin(P<.05)between the meditation and resting state at the 10~(th)min.In the nontask state,the BFI-Extraversion showed a negative correlation with the temperature of the left forehead(R=-0.41,P=.03).In the post-task state,the temperature of the left forehead was negatively correlated with scores on the MAAS(R=-0.42,P=.02).Conclusion:Using IRTI to study meditation offers a practical solution to the challenges in meditation research.The results indicate that an increase in chin temperature may be a representative feature of a meditation state,and forehead temperature is also a potential indicator.

Heat transfer and temperature evolution in underground mininginduced overburden fracture and ground fissures: Optimal time window of UAV infrared monitoring

Heat transfer and temperature evolution in overburden fracture and ground fissures are one of the essential topics for the identification of ground fissures via unmanned aerial vehicle(UAV) infrared imager. In this study, discrete element software UDEC was employed to investigate the overburden fracture field under different mining conditions. Multiphysics software COMSOL were employed to investigate heat transfer and temperature evolution of overburden fracture and ground fissures under the influence of mining condition, fissure depth, fissure width, and month alternation. The UAV infrared field measurements also provided a calibration for numerical simulation. The results showed that for ground fissures connected to underground goaf(Fissure Ⅰ), the temperature difference increased with larger mining height and shallow buried depth. In addition, Fissure Ⅰ located in the boundary of the goaf have a greater temperature difference and is easier to be identified than fissures located above the mining goaf. For ground fissures having no connection to underground goaf(Fissure Ⅱ), the heat transfer is affected by the internal resistance of the overlying strata fracture when the depth of Fissure Ⅱ is greater than10 m, the temperature of Fissure Ⅱ gradually equals to the ground temperature as the fissures’ depth increases, and the fissures are difficult to be identified. The identification effect is most obvious for fissures larger than 16 cm under the same depth. In spring and summer, UAV infrared identification of mining fissures should be carried out during nighttime. This study provides the basis for the optimal time and season for the UAV infrared identification of different types of mining ground fissures.

Utilization of Thermal Infrared Image for Inversion of Winter Wheat Yield and Biomass

The present paper utilizes thermal infrared image for inversion of winter wheat yield and biomass with different technology of irrigation(drip irrigation,sprinkler irrigation,flood irrigation).It is the first time that thermal infrared image is used for predicting the winter wheat yield and biomass.The temperature of crop and background was measured by thermal infrared image.It is necessary to get the crop background separation index(CBSIL,CBSIH),which can be used for distinguishing the crop value from the image.CBSIL and CBSIH(the temperature when the leaves are wet adequately;the temperature when the stomata of leaf is closed completely) are the threshold values.The temperature of crop ranged from CBSIL to CBSIH.Then the ICWSI was calculated based on relevant theoretical method.The value of stomata leaf has strong negative correlation with ICWSI proving the reliable value of ICWSI.In order to construct the high accuracy simulation model,the samples were divided into two parts.One was used for constructing the simulation model,the other for checking the accuracy of the model.Such result of the model was concluded as:(1) As for the simulation model of soil moisture,the correlation coefficient(R2) is larger than 0.887 6,the average of relative error(Er) ranges from 13.33% to 16.88%;(2) As for the simulation model of winter wheat yield,drip irrigation(0.887 6,16.89%,-0.12),sprinkler irrigation(0.970 0,14.85%,-0.12),flood irrigation(0.969 0,18.87%,0.18),with the values of R2,Er and CRM listed in the parentheses followed by the individual term.(3) As for winter wheat biomass,drip irrigation(0.980 0,13.70%,0.13),sprinkler irrigation(0.95,13.15%,-0.14),flood irrigation(0.970 0,14.48%,-0.13),and the values in the parentheses are demonstrated the same as above.Both the CRM and Er are shown to be very low values,which points to the accuracy and reliability of the model investigated.The accuracy of model is high and reliable.The results indicated that thermal infrared image can be used potentially for inversion of winter wheat yield and biomass.

INFRARED THERMAL IMAGE STUDY ON THE FOREWARNING OF COAL AND SANDSTONE FAILURE UNDER LOAD

In the experimental study, AGE-782 thermal instrument was used to detect the infrared radiation variation of coal and sandstone (wave-length range 3.6～5.5 μm was used). It's discovered that coal and sandstone failure under load have three kinds of infrared thermal features as well as infrared forewarning messages. That are: (1) temperature rises gradually but drops before failure ; (2) temperature rises gradually but quickly rises before failure; (3) first rises,then drops and lower temperature emerges before failure. The further researches and the prospect of micro-wave remote sensing detection .on ground pressure is also discussed.

Improving wavelet reconstruction algorithm to achieve comprehensive application of thermal infrared remote sensing data from TM and MODIS

According to the data characteristics of Landsat thematic mapper （TM） and MODIS, a new fu sion algorithm about thermal infrared data has been proposed in the article based on improving wave let reconstruction. Under the domain of neighborhood wavelet reconstruction, data of TM and MO DIS are divided into three layers using wavelet decomposition. The texture information of TM data is retained by fusing highfrequency information. The neighborhood correction coefficient method （NC CM） is set up based on the search neighborhood of a certain size to fuse lowfrequency information. Thermal infrared value of MODIS data is reduced to the space value of TM data by applying NCCM. The data with high spectrum, high spatial and high temporal resolution, are obtained through the al gorithm in the paper. Verification results show that the texture information of TM data and high spec tral information of MODIS data could be preserved well by the fusion algorithm. This article could provide technical support for high precision and fast extraction of the surface environment parame ters.

Thermal Infrared Salient Human Detection Model Combined with Thermal Features in Airport Terminal

Target detection in low light background is one of the main tasks of night patrol robots for airport terminal.However,if some algorithms can run on a robot platform with limited computing resources,it is difficult for these algorithms to ensure the detection accuracy of human body in the airport terminal. A novel thermal infrared salient human detection model combined with thermal features called TFSHD is proposed. The TFSHD model is still based on U-Net,but the decoder module structure and model lightweight have been redesigned. In order to improve the detection accuracy of the algorithm in complex scenes,a fusion module composed of thermal branch and saliency branch is added to the decoder of the TFSHD model. Furthermore,a predictive loss function that is more sensitive to high temperature regions of the image is designed. Additionally,for the sake of reducing the computing resource requirements of the algorithm,a model lightweight scheme that includes simplifying the encoder network structure and controlling the number of decoder channels is adopted. The experimental results on four data sets show that the proposed method can not only ensure high detection accuracy and robustness of the algorithm,but also meet the needs of real-time detection of patrol robots with detection speed above 40 f/s.

Gait Recognition System in Thermal Infrared Night Imaging by Using Deep Convolutional Neural Networks

Gait is an essential biomedical feature that distinguishes individuals through walking.This feature automatically stimulates the need for remote human recognition in security-sensitive visual monitoring applications.However,there is still a lack of sufficient accuracy of gait recognition at night,in addition to taking some critical factors that affect the performances of the recognition algorithm.Therefore,a novel approach is proposed to automatically identify individuals from thermal infrared(TIR)images according to their gaits captured at night.This approach uses a new night gait network(NGaitNet)based on similarity deep convolutional neural networks(CNNs)method to enhance gait recognition at night.First,the TIR image is represented via personal movements and enhanced body skeleton segments.Then,the state-space method with a Hough transform is used to extract gait features to obtain skeletal joints′angles.These features are trained to identify the most discriminating gait patterns that indicate a change in human identity.To verify the proposed method,the experimental results are performed by using learning and validation curves via being connected by the Visdom website.The proposed thermal infrared imaging night gait recognition(TIRNGaitNet)approach has achieved the highest gait recognition accuracy rates(99.5%,97.0%),especially under normal walking conditions on the Chinese Academy of Sciences Institute of Automation infrared night gait dataset(CASIA C)and Donghua University thermal infrared night gait database(DHU night gait dataset).On the same dataset,the results of the TIRNGaitNet approach provide the record scores of(98.0%,87.0%)under the slow walking condition and(94.0%,86.0%)for the quick walking condition.

9μm Cutoff 128×128 AlGaAs/GaAs Quantum Well Infrared Photodetector Focal Plane Arrays

We design and fabricate a 128 × 128 AlGaAs/GaAs quantum well infrared photodetector focal plane array （FPA）. The device is achieved by metal organic chemical vapor deposition and GaAs integrated circuit processing technology. A test structure of the photodetector with a mesa size of 300μm × 300μm is also made in order to obtain the device parameters. The measured dark current density at 77K is 1.5 × 10^-3A/cm^2 with a bias voltage of 2V. The peak of the responsivity spectrum is at 8.4μm,with a cutoff wavelength of 9μm. The blackbody detectivity is shown to be 3.95 × 10^8 （cm · Hz^1/2）/W. The final FPA is flip-chip bonded on a CMOS read-out integrated circuit. The infrared thermal images of some targets at room temperature background are successfully demonstrated at 80K operating temperature with a ratio of dead pixels of less than 1%.

Experimental study on the infrared precursor characteristics of gas-bearing coal failure under loading

The stress and gas pressure in deep coal seams are very high,and instability and failure rapidly and intensely occur.It is important to study the infrared precursor characteristics of gas-bearing coal instability and failure.In this paper,a self-developed stress-gas coupling failure infrared experimental system was used to analyse the infrared radiation temperature(IRT)and infrared thermal image precursor characteristics of gas-free coal and gas-bearing coal.The changes in the areas of the infrared temperature anomalous precursor regions and the effect of the gas on the infrared precursors were examined.The results show that high-temperature anomalous precursors arise mainly when the gas-free coal fails under loading,whereas the gas-bearing coal has high-temperature and low-temperature anomalous precursors.The area of the high-temperature anomalous precursor is approximately 30%–40%under gasbearing coal unstable failure,which is lower than the 60%–70%of the gas-free coal.The area of the low-temperature abnormal precursor is approximately 3%–6%,which is higher than the 1%–2%of the gas-free coal.With increasing gas pressure,the area of the high-temperature anomalous precursor gradually decreases,and the area of the low-temperature anomalous precursor gradually increases.The highand low-temperature anomalous precursors of gas-bearing coal are mainly caused by gas desorption,volume expansion,and thermal friction.The presence of gas inhibits the increase in IRT on the coal surface and increases the difficulty of infrared radiation(IR)monitoring and early warning for gas-bearing coal.

Device topological thermal management of β-Ga_(2)O_(3) Schottky barrier diodes

The ultra-wide bandgap semiconductor β gallium oxide(β-Ga_(2) O_(3)) gives promise to low conduction loss and high power for electronic devices. However, due to the natural poor thermal conductivity of β-Ga_(2) O_(3), their power devices suffer from serious self-heating effect. To overcome this problem, we emphasize on the effect of device structure on peak temperature in β-Ga_(2) O_(3) Schottky barrier diodes(SBDs) using TCAD simulation and experiment. The SBD topologies including crystal orientation of β-Ga_(2) O_(3), work function of Schottky metal, anode area, and thickness, were simulated in TCAD, showing that the thickness of β-Ga_(2) O_(3) plays a key role in reducing the peak temperature of diodes. Hence, we fabricated β-Ga_(2) O_(3) SBDs with three different thickness epitaxial layers and five different thickness substrates. The surface temperature of the diodes was measured using an infrared thermal imaging camera. The experimental results are consistent with the simulation results. Thus, our results provide a new thermal management strategy for high power β-Ga_(2) O_(3) diode.

Monitoring coal fires in Datong coalfield using multi-source remote sensing data

Numerous coal fires burn underneath the Datong coalfield because of indiscriminate mining.Landsat TM/ETM,unmanned aerial vehicle（UAV）,and infrared thermal imager were employed to monitor underground coal fires in the Majiliang mining area.The thermal field distributions of this area in 2000,2002,2006,2007,and 2009 were obtained using Landsat TM/ETM.The changes in the distribution were then analyzed to approximate the locations of the coal fires.Through UAV imagery employed at a very high resolution（0.2 m）,the texture information,linear features,and brightness of the ground fissures in the coal fire area were determined.All these data were combined to build a knowledge model of determining fissures and were used to support underground coal fire detection.An infrared thermal imager was used to map the thermal field distribution of areas where coal fire is serious.Results were analyzed to identify the hot spot trend and the depth of the burning point.

Fusion of visible and thermal images for facial expression recognition

Most present research into facial expression recognition focuses on the visible spectrum, which is sen- sitive to illumination change. In this paper, we focus on in- tegrating thermal infrared data with visible spectrum images for spontaneous facial expression recognition. First, the ac- tive appearance model AAM parameters and three defined head motion features are extracted from visible spectrum im- ages, and several thermal statistical features are extracted from infrared （IR） images. Second, feature selection is per- formed using the F-test statistic. Third, Bayesian networks BNs and support vector machines SVMs are proposed for both decision-level and feature-level fusion. Experiments on the natural visible and infrared facial expression （NVIE） spontaneous database show the effectiveness of the proposed methods, and demonstrate thermal 1R images＇ supplementary role for visible facial expression recognition.

Influence of UAV flight speed on droplet deposition characteristics with the application of infrared thermal imaging

A plant protection unmanned aerial vehicle(UAV)applied for spraying pesticide has the advantages of low cost,high efficiency and environmental protection.However,the complex and changeable farmland environment is not conductive to perform spray test effectively.It is therefore necessary to carry out spray test under controlled conditions.The current study aimed to illuminate the variation law of droplet deposition characteristics under different UAV flight speeds,and to verify the feasibility for applying infrared thermal imaging in detection of droplet deposition effects.A UAV simulation platform with an airborne spray system was established and an analysis program Droplet Analysis for dealing with water-sensitive paper was developed.The results showed that,when the flight speed was set at 0.3 m/s,0.5 m/s,0.7 m/s,0.9 m/s and 1 m/s,respectively,the droplet deposition density,droplet deposition coverage and arithmetic mean of droplet size D0 decreased as the UAV flight speed increased.On the contrary,the droplet diameter variation coefficient CV increased with the increase of UAV flight speed,resulting in the worse uniformity of sprayed droplet distribution as well.The results can provide a theoretical support for optimizing the spraying parameters of plant protection UAV,and demonstrate the practicability of infrared thermal imaging in evaluating the droplet deposition in the field of aerial spraying.

Study on improving blood flow with korean red ginseng substances using digital infrared thermal imaging and doppler sonography:randomized,double blind,placebo-controlled clinical trial with parallel design

OBJECTIVE： To examine ginseng for improving people. the efficacy of Korean red blood flow in healthy METHODS： Participants were randomized and treated with 1500 mg of Korean red ginseng extract or placebo for 8 weeks. The effect of Korean red ginseng was evaluated by digital infrared thermal images, and Doppler sonography, and blood test. RESULTS： Imbalance Forty subjects completed the protoco n local thermal distribution was significantly decreased in the Korean red ginseng group confirmed by digital infrared thermal images. Doppler sonography showed no significant change in maximum and average rates of blood circulation in single or complex areas. Blood analyses for coagulation and lipid metabolism factors revealed no significant changes. No abnormal reactions to the Korean red ginseng were observed. CONCLUSION： Digital infrared thermal imaging showed that the temperature deviation in the whole body decreased safely in the Korean red ginseng group, which mitigated the body- temperature imbalance. This result suggests that the Korean red ginseng improves blood circulation in the human body.

Infrared Thermal Imaging-Based Research on the Intermediate Structures of the Lung and Large Intestine Exterior-Interior Relationship in Asthma Patients

Objective: By observing body surface temperature variation of the intermediate structures of the Lung(Fei) and Large Intestine(Dachang) exterior-interior relationship in asthmatic patients, to investigate the pathological response on the pathway of channels and to substantiate the objective existence of the intermediary structures. Methods: The study included 60 subjects meeting the bronchial asthma inclusion criteria(experimental group) and 60 healthy subjects(normal control group). ATIR-M301 infrared thermal imaging device was used for detecting body surface temperature of the subjects and collecting the infrared thermal images. The temperature values of the intermediate structures of Lung and Large Intestine exterior-interior relationship [throat, Quepen, elbow, nose, Lieque(LU 7), Pianli(LI 6)], control areas(0.2 cm lateral to the above structures) and Yintang(EX-HN 3) were measured on the infrared thermal image by infrared imaging system. Then, the above temperature values were compared and analyzed within and between two groups. Results: There were insignificant differences between the temperature on the left and right sides of the intermediate structures(Quepen, elbow, LU 7, LI 6) in normal control group(P>0.05). Except for that of Quepen, there were insignificant differences between the temperature of the intermediate structures and their corresponding control areas in normal control group(P>0.05). In the experimental group, the temperature on the left and right sides of the intermediate structures(Quepen, elbow, LU 7, LI 6) showed statistically significant differences(P<0.05 or P<0.01); the temperature difference between intermediate structure(throat, Quepen, elbow, nose, LI 6) and their respective control areas were also significant(P<0.05 or P<0.01). The temperature of the intermediate structures(throat, Quepen, elbow, LU7, LI 6) between the experimental group and normal control group showed significant differences(P<0.05 or P<0.01). Conclusions: This study is an initial step to validate the objective existence of Lung and Large Intestine exterior-interior relationship intermediate structures, as described in the Chinese classical medical literatures, through the functional imaging angle. The intermediate structures are the pathological reaction areas of the bronchial asthmatic patients.

Detection of Thermophysical Properties for High Strength Concrete after Exposure to High Temperature

Using the detection principle of infrared thermal imaging technique and the detection principle of DRH thermal conductivity tester laboratory,we investigated the infrared thermal image inspection,coefficient of thermal conductivity,apparent density,and compressive strength test on C80 high-strength concrete（HSC） in the presence and absence of polypropylene fibers under completely heated conditions.Only slight damages were detected below 400 ℃,whereas more and more severe deterioration events were expected when the temperature was above 500 ℃.The results show that the elevated temperature through infrared images generally exhibits an upward trend with increasing temperature,while the coefficient of thermal conductivity and apparent density decrease gradually.Additionally,the addition of polypropylene fibers with appropriate length,diameter,and quantity contributes to the improvement of the high-temperature resistance of HSC.

Study on the Temperature of the Bridge Wire in the Initiator Used in Nuclear Explosion Valve

This paper established the mathematical model of bridge wire temperature rise under direct current condition and gave the solution. It computed bridge wire temperature by using the thermal-electric coupling method provided by ANSYS-Workbench finite element analysis software. In the end, the temperature bridge wire applied to different electric current was measured by the infrared thermal imaging temperature measurement method. The result shows that the ANSYS simulation results are in agreement with the theoretical calculation and the experimental results. It is feasible to compute bridge wire temperature of initiator by using ANSYS-Workbench software, and it is an important method to analyze complex structure of pyrotechnics.

An empirical method for improving accuracy of human eye temperature measured by uncooled infrared thermal imager

In order to reduce the temperature measurement error with the uncooled infrared thermal imager, experiments were conducted to evaluate the effects of environment temperature and measurement distance on the measurement error of human eye temperature. First, the forehead temperature was used as an intermediate variable to obtain the actual temperature of human eyes. Then, the effects of environment temperature and measurement distance on the temperature measurement were separately analyzed. Finally, an empirical model was established to correlate actual eye temperature with the measured temperature, environment temperature, and measurement distance. To verify the formula, three different environment temperatures were tested at different distances. The measurement errors were substantially reduced using the empirical model for temperature correction. The results show that this method can effectively improve the accuracy of temperature measurement using the infrared thermal imager.

Basic Study of Air Leakage Position Based on Thermography Mosaic Technology

When locating the leakage of thermal field of a certain container, it is hard for us to evaluate the leakage with one or several separate thermal infrared images. Instead, panoramic imagery with a wide visual angle and high distinguishability is always needed. However, we can only get images for part of the scene because of the limited visual field of thermography and the large size of the measured object in practical. What is more, the hardware that can be used to get panoramic images is always expensive and cannot be used in large scale. Therefore, this paper introduces image mosaic technology, which can be used to get the intact thermal infrared image of the measured and increase the efficiency of detection. The experiment results demonstrate its effectiveness.