Manipulating Stress and Cognitive Load in Conversational Interactions with a Multimodal System for Crisis Management Support

  • Andreea Niculescu
  • Yujia Cao
  • Anton Nijholt
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5967)


The quality assessment of multimodal conversational interfaces is influenced by many factors. Stress and cognitive load are two of most important. In the literature, these two factors are considered as being related and accordingly summarized under the single concept of ‘cognitive demand’. However, our assumption is that even if they are related, these two factors can still occur independently. Therefore, it is essential to control their levels during the interaction in order to determine the impact that each factor has on the perceived conversational quality. In this paper we present preliminary experiments in which we tried to achieve a factor separation by inducing alternating low/high levels of both stress and cognitive load. The stress/cognitive load levels were manipulated by varying task difficulty, information presentation and time pressure. Physiological measurements, performance metrics, as well as subjective reports were deployed to validate the induced stress and cognitive load levels. Results showed that our manipulations were successful for the cognitive load and partly for the stress. The levels of both factors were better indicated by subjective reports and performance metrics than by physiological measurements.


Multimodal interfaces verbal communication stress cognitive load physiological measurements qualitative evaluations 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Moeller, S.: Quality of telephone-based spoken dialogue systems. Springer, New York (2005)Google Scholar
  2. 2.
    Oviatt, S.: Human-centered design meets cognitive load theory: designing interfaces that help people think. In: Proc. of the 14th annual ACM international conference on Multimedia, Santa Barbara (2006)Google Scholar
  3. 3.
    Love, S., Dutton, R.T., Foster, J.C., Jack, M.A., Stentiford, F.W.: Identifying salient usability attributes for automated telephone services. In: Proc. of the 3rd Conf. on Spoken Language (ICSLP), Yokohama, Japan (1994)Google Scholar
  4. 4.
    Jack, M.A., Foster, J.C., Stentiford, F.W.M.: Intelligent dialogues in automated telephone services. In: Proc. 2nd Int. Conf. on Spoken Language Processing (ICSLP), Banff, Canada (1992)Google Scholar
  5. 5.
    Patil, S.A., Hansen, J.H.L.: Speech Under Stress: Analysis, Modeling and Recognition. In: Müller, C. (ed.) Speaker Classification I: Fundamentals, Features, and Methods, pp. 108–137. Springer, Heidelberg (2007)Google Scholar
  6. 6.
    Hone, K.S., Graham, R.: Towards a tool for the subjective assessment of speech and System Interfaces (SASSI). Natural Language Engineering 6(3-4), 287–303 (2000)CrossRefGoogle Scholar
  7. 7.
    Lazarus, R.S.: Theory based stress measurement. Psychological Inquiry 1, 3–13 (1990)CrossRefGoogle Scholar
  8. 8.
    Kryter, K.D.: The handbook of hearing and the effects of noise: Physiology, psychology, and public health. Academic Press, New York (1994)Google Scholar
  9. 9.
    Chandler, P., Sweller, J.: Cognitive Load Theory and the Format of Instruction. Cognition and Instruction 8, 293–332 (1991)CrossRefGoogle Scholar
  10. 10.
    Schell, K.L., Grasha, A.F.: State anxiety, performance accuracy, and work pace in a simulated pharmacy dispensing task. In: Perceptual and Motor Skills, vol. 90, pp. 547–561 (2000)Google Scholar
  11. 11.
    Scerbo, M.W., Freeman, F.G., Mikulka, P.J., Parasuraman, R., Di Nocero, F.: The efficacy of psychophysiological measures for implementing adaptive technology. TP-2001-211018 (2001)Google Scholar
  12. 12.
    Wilson, G.F., Eggemeier, F.T.: Psychophysiological assessment of workload in multi-task environments. In: Damos, D.L. (ed.) Multiple-task performance. CRC Press, Boca Raton (1991)Google Scholar
  13. 13.
    Verwey, W.B., Veltman, H.: Detecting short periods of elevated workload. A comparison of nine workload assessment techniques. Applied Experimental Psychology 2, 270–285 (1996)Google Scholar
  14. 14.
    Boucsein, W., Haarmann, A., Schaefer, F.: Combining skin conductance and heart rate variability for adaptive automation during simulated IFR flight. In: Harris, D. (ed.) HCII 2007 and EPCE 2007. LNCS (LNAI), vol. 4562, pp. 639–647. Springer, Heidelberg (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Andreea Niculescu
    • 1
  • Yujia Cao
    • 1
  • Anton Nijholt
    • 1
  1. 1.Human Media InteractionUniversity of TwenteEnschedeThe Netherlands

Personalised recommendations