Affective State Recognition Using Thermal-Based Imaging: A Survey

The thermal-based imaging technique has recently attracted the attention of researchers who are interested in the recognition of human affects dueto its ability to measure the facial transient temperature, which is correlated withhuman affects and robustness against illumination changes. Therefore, studieshave increasingly used the thermal imaging as a potential and supplemental solution to overcome the challenges of visual (RGB) imaging, such as the variation oflight conditions and revealing original human affect. Moreover, the thermal-basedimaging has shown promising results in the detection of psychophysiological signals, such as pulse rate and respiration rate in a contactless and noninvasive way.This paper presents a brief review on human affects and focuses on the advantages and challenges of the thermal imaging technique. In addition, this paper discusses the stages of thermal-based human affective state recognition, such asdataset type, preprocessing stage, region of interest (ROI), feature descriptors,and classification approaches with a brief performance analysis based on a number of works in the literature. This analysis could help beginners in the thermalimaging and affective recognition domain to explore numerous approaches usedby researchers to construct an affective state system based on thermal imaging.

Multimodal spontaneous affect recognition using neural networks learned with hints

A multimodal fusion classifier is presented based on neural networks （NNs） learned with hints for automatic spontaneous affect recognition. In case that different channels can provide com- plementary information, features are utilized from four behavioral cues： frontal-view facial expres- sion, profile-view facial expression, shoulder movement, and vocalization （audio）. NNs are used in both single cue processing and multimodal fusion. Coarse categories and quadrants in the activation- evaluation dimensional space are utilized respectively as the heuristic information （hints） of NNs during training, aiming at recognition of basic emotions. With the aid of hints, the weights in NNs could learn optimal feature groupings and the subtlety and complexity of spontaneous affective states could be better modeled. The proposed method requires low computation effort and reaches high recognition accuracy, even if the training data is insufficient. Experiment results on the Semaine nat- uralistic dataset demonstrate that our method is effective and promising.

Social skills and psychopathology are associated with autonomic function in children:a cross-sectional observational study

In recent years,the increase of psychopathological disorders in the population has become a health emergency,leading to a great effort to understand psychological vulnerability mechanisms.In this scenario,the role of the autonomic nervous system(ANS)has become increasingly important.This study investigated the association between ANS,social skills,and psychopathological functioning in children.As an ANS status proxy,we measured heart rate variability(HRV).Infants admitted to the neonatal intensive care unit of the University Hospital of Padova because of preterm birth or neonatal hypoxic-ischemic encephalopathy were sequentially recruited from January 2011 to June 2013 and followed long-term up to school age in this cross-sectional observational study.We recorded 5 minutes of HRV immediately before measuring performance in social abilities tasks(affect recognition and theory of mind,NEPSY-II)in 50 children(mean age 7.4±1.4 years)with and without risk factors for developing neuropsychiatric disorders due to pre-/perinatal insults without major sequelae.Children also completed extensive cognitive,neuropsychological,and psychosocial assessment.Parents were assessed with psychopathological interviews and a questionnaire(CBCL 6-18).Analysis in a robust Bayesian framework was used to unearth dependencies between HRV,social skills,and psychopathological functioning.Social task scores were associated with HRV components,with high frequency the most consistent.HRV bands were also associated with the psychopathological questionnaire.Only normalized HRV high frequency was able to distinguish impaired children in the affect recognition task.Our data suggest that ANS may be implicated in social cognition both in typical and atypical developmental conditions and that HRV has cross-disease sensitivity.We suggest that HRV parameters may reflect a neurobiological vulnerability to psychopathology.The study was approved by the Ethics Committee of the University Hospital of Padova(Comitato Etico per la Sperimentazione,Azienda Opedaliera di Padova,approval No.1693 P).