Capturing drivers’ affective responses given driving context and driver-pedestrian interactions remains a challenge for designing in-vehicle, empathic interfaces. To address this, we conducted two lab-based studies using camera and physiological sensors. Our first study collected participants’ (N = 21) emotion self-reports and physiological signals (including facial temperatures) toward non-verbal, pedestrian crossing videos from the Joint Attention for Autonomous Driving dataset. Our second study increased realism by employing a hybrid driving simulator setup to capture participants’ affective responses (N = 24) toward enacted, non-verbal pedestrian crossing actions. Key findings showed: (a) non-positive actions in videos elicited higher arousal ratings, whereas different in-video pedestrian crossing actions significantly influenced participants’ physiological signals. (b) Non-verbal pedestrian interactions in the hybrid simulator setup significantly influenced participants’ facial expressions, but not their physiological signals. We contribute to the development of in-vehicle empathic interfaces that draw on behavioral and physiological sensing to in-situ infer driver affective responses during non-verbal pedestrian interactions.

, , , , ,
International Journal of Human-Computer Interaction
Centrum Wiskunde & Informatica, Amsterdam (CWI), The Netherlands

Rao, S., Ghosh, S., Pons Rodriguez, G., Röggla, T., César Garcia, P. S., & El Ali, A. (2023). From video to hybrid simulator: Exploring affective responses toward non-verbal pedestrian crossing actions using camera and physiological sensors. International Journal of Human-Computer Interaction. doi:10.1080/10447318.2023.2224955