Voice is a rich modality for conveying emotions, however emotional prosody production can be situationally or medically impaired. Since thermal displays have been shown to evoke emotions, we explore how thermal stimulation can augment perception of neutrally-spoken voice messages with affect. We designed ThermalWear, a wearable on-chest thermal display, then tested in a controlled study (N=12) the effects of fabric, thermal intensity, and direction of change. Thereafter, we synthesized 12 neutrally-spoken voice messages, validated (N=7) them, then tested (N=12) if thermal stimuli can augment their perception with affect. We found warm and cool stimuli (a) can be perceived on the chest, and quickly without fabric (4.7-5s) (b) do not incur discomfort (c) generally increase arousal of voice messages and (d) increase / decrease message valence, respectively. We discuss how thermal displays can augment voice perception, which can enhance voice assistants and support individuals with emotional prosody impairments.

thermal, affect, emotion, voice, prosody, wearable, display, chest
ACM CHI Conference on Human Factors in Computing Systems
Centrum Wiskunde & Informatica, Amsterdam, The Netherlands

El Ali, A, Yang, X, Ananthanarayan, S, Röggla, T, Jansen, A.J, Hartcher-O'Brien, J, … César Garcia, P.S. (2020). ThermalWear: Exploring wearable on-chest thermal displays to augment voice messages with affect. In Conference on Human Factors in Computing Systems - Proceedings. doi:10.1145/3313831.3376682