Machine Learning Techniques for Extracting Implicit Feedback from Natural Facial Reactions

Facial Emotion Recognition (FER) aims to identify a person’s emotions by analyzing her face [3]. Most FER research focuses on the analysis of static images. Moreover, in the most well-known datasets available in the research literature, data were collected from the wild, that is, using predefined queries (e.g., happy, sad) on image search engines or by asking participants for particular poses (in this case, the dataset is called posed [1]). With our research activities, we intend to fill this gap by creating a dataset of the natural reactions of participants recorded while viewing artworks. So far, we have collected the reactions of more than 50 participants, and we are carrying out a com- parative analysis between different state-of-the-art Machine Learning techniques (e.g., Convolutional Neural Networks [4] and autoencoders [2]) to identify those capable of providing the best performance in this challenging task. We hope that our research can provide valuable insights to researchers and prac- titioners regarding the accuracy of ML models in this task and can open up new perspectives in collecting and generating natural video datasets, given the lack of such resources in the literature.