Affective Computing


We present novel algorithms for identifying emotion, dominance, and friendliness characteristics of pedestrians based on their motion behaviors. We also propose models for conveying emotions, friendliness, and dominance traits in virtual agents. We present applications of our algorithms to simulate interpersonal relationships between virtual characters, facilitate socially-aware robot navigation, identify perceived emotions from videos of walking individuals, and increase the sense of presence in scenarios involving multiple virtual agents. We also present a dataset of videos of walking individuals with gaits and labeled emotions.
Currently, our efforts are focused on predicting perceived emotions from multiple modalities such as faces, gaits, speech, and text, by investigating the correlation between these modalities. This direction will also lead us to be able to infer or generate missing modalities.

Project Conference/Journal Year
M3ER: Multiplicative Multimodal Emotion Recognition Using Facial, Textual, and Speech Cues
STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits
FVA: Modeling Perceived Friendliness of Virtual Agents Using Movement Characteristics ISMAR 2019
Modeling Data-Driven Dominance Traits for Virtual Characters using Gait Analysis
Identifying Emotions from Walking using Affective and Deep Features arXiv 2019
Pedestrian Dominance Modeling for Socially-Aware Robot Navigation ICRA 2019
The Emotionally Intelligent Robot: Improving Social Navigation in Crowded Environments IROS 2019
Data-Driven Modeling of Group Entitativity in Virtual Environments VRST 2018
Classifying Group Emotions for Socially-Aware Autonomous Vehicle Navigation CVPR Workshop 2018
Aggressive, Tense or Shy? Identifying Personality Traits from Crowd Videos IJCAI 2017
Sociosense: Robot navigation amongst pedestrians with social and psychological constraints IROS 2017
F2FCrowds: Planning Agent Movements to Enable Face-to-Face Interactions PRESENCE 2017
Generating Virtual Avatars with Personalized Walking Gaits using commodity hardware ACM Multimedia 2017
PedVR: Simulating Gaze-Based Interactions between a Real User and Virtual Crowds VRST 2016