Towards Empathetic Social Robots: Investigating the Interplay between Facial Expressions and Brain Activity

The pursuit of creating empathetic social robots that can understand and respond to human emotions is a critical challenge in Robotics and Artificial Intelligence. Social robots, designed to interact with humans in various settings, from healthcare to customer service, require a sophisticated understanding of human emotional states to resonate and effectively assist truly. Our research contributes to this ambitious goal by exploring the relationship between natural facial expressions and brain activity in these human-robot interactions, as captured by electroencephalogram (EEG) signals. This paper presents our initial steps towards this attempt. We want to find which areas in the participant user’s brain are most activated and how these activations correlate with facial expressions. Understanding these correlations is essential for developing social robots that recognize and empathize with various human emotions. Our approach combines neuroscience and computer science, offering a novel perspective in the quest to enhance the emotional intelligence of social robots. We share some preliminary results on a new multimodal dataset that we are developing, providing valuable insights into the potential of our work to improve the personalization and emotional depth of social robot interactions.