Research Article: Emotion analysis in children through facial emissivity of infrared thermal imaging

Date Published: March 20, 2019

Publisher: Public Library of Science

Author(s): Christiane Goulart, Carlos Valadão, Denis Delisle-Rodriguez, Eliete Caldeira, Teodiano Bastos, Christos Papadelis.


Physiological signals may be used as objective markers to identify emotions, which play relevant roles in social and daily life. To measure these signals, the use of contact-free techniques, such as Infrared Thermal Imaging (IRTI), is indispensable to individuals who have sensory sensitivity. The goal of this study is to propose an experimental design to analyze five emotions (disgust, fear, happiness, sadness and surprise) from facial thermal images of typically developing (TD) children aged 7–11 years using emissivity variation, as recorded by IRTI. For the emotion analysis, a dataset considered emotional dimensions (valence and arousal), facial bilateral sides and emotion classification accuracy. The results evidence the efficiency of the experimental design with interesting findings, such as the correlation between the valence and the thermal decrement in nose; disgust and happiness as potent triggers of facial emissivity variations; and significant emissivity variations in nose, cheeks and periorbital regions associated with different emotions. Moreover, facial thermal asymmetry was revealed with a distinct thermal tendency in the cheeks, and classification accuracy reached a mean value greater than 85%. From the results, the emissivity variations were an efficient marker to analyze emotions in facial thermal images, and IRTI was confirmed to be an outstanding technique to study emotions. This study contributes a robust dataset to analyze the emotions of 7-11-year-old TD children, an age range for which there is a gap in the literature.

Partial Text

Studies focused on emotions has increased worldwide, mainly due to their importance for the interpersonal relationship field but also as they are considered potent facilitators of cognitive processes and contributors to many illnesses [1]. In many aspects of daily life, emotions frequently mold social relationships, contributing to communication between humans [2] and enabling the identification of a person’s intention to adopt appropriate responses [3].

The focus of this work was the emotion analysis by IRTI elicited in typical children of developmental ages, for which there is a gap in the literature. Our method presents some limitations, and one is related to the absence of an automated ROI tracking and positioning method. In our work, manual ROI positioning was performed in the first frame of each thermal video, which was used as a reference for the next ROI marking, being automatically propagated to the rest of the video. This was made to ensure the correct positioning of the ROIs. The advantage of using a manual check method is to discard ROIs that are not positioned correctly, assuring an appropriate selection of frames for our analyses. Automatic positioning of ROIs and their tracking methods are subjects for future works, as real-time face tracking could reduce the frame discarding due to ROI positional error or head movements. It could also ensure a correct ROI positioning even during such movements. An example of a facial tracking method could be the combination of particle filtering with a probabilistic template algorithm (spatiotemporal smoothing template) proposed in [50], which enables a fast, flexible and accurate tracker in spite of head movements and physiological changes.

The proposed experimental design was carried out in a careful way with typically developing children aged between 7 and 11 years, in which five emotions (disgust, fear, happiness, sadness and surprise) were evoked by audio-visual stimuli, triggering significant facial emissivity variations in 11 facial ROIs, as recorded using IRTI. An analysis set of emotions considered emotional dimensions, facial bilateral ROIs and emotion classification. To the best of our knowledge, to date, there is no similar experimental design applied in children of this age group in literature.




Leave a Reply

Your email address will not be published.