C67: Detection of Challenging Dialogue Stages Using Acoustic Signals and Biosignals

Egorow, O., Wendemuth, A.

Abstract:
Emotions play an important role in human-human interaction. But they are also expressed during human-computer interaction, and thus should be recognised and responded to. Therefore, emotion recognition is an important feature for human-computer interaction. The task of emotion recognition is not an easy one - in "in the wild" scenarios, the occurring emotions are rarely expressive and clear, but often more subtle. One example for such an affect crucial for human-computer interaction is trouble experienced by the human in case of unexpected dialogue course. But emotions are not only expressed via voice, the same information is contained in the physiological reactions. These reactions are are much harder to conceal, therefore representing the "true signal". These physiological signals should not be left unattended. In this paper we use data from naturalistic human-computer dialogues containing challenging dialogue stages to show that it is possible to differentiate between troubled and untroubled dialogue in acoustic as well as in physiological signals. We achieve an UAR of 64% using the acoustic signal, and an UAR of 88% using the biosignals.