Skip to main content
SpringerLink
Log in

Are Behavioral Measures Useful for Detecting Cognitive Workload During Human-Computer Interaction?

  • Conference paper
  • First Online:
Advances in The Human Side of Service Engineering

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 494))

Abstract

Commonly used techniques for measuring cognitive workload during human-computer interactions can be cumbersome or intrusive to task performance. In the current work, we examine the utility of heuristic behavior analysis, including keystroke dynamics, mouse tracking, and body positioning for measuring cognitive workload during direct interactions between humans and computers. We present a method for modeling behavioral measures as well as physiological and neurophysiological data using probabilistic, statistical, and machine learning algorithms for real-time estimation of human states. We believe this discussion will inform the capability to provide estimates of cognitive workload in real-world scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Veltman, J.A., Gaillard, A.W.K.: Physiological indices of workload in a simulated flight task. Biol. Psychol. 42, 323–342 (1996)

    Article  Google Scholar 

  2. Pomplun, M., Sunkara, S.: Pupil dilation as an indicator of cognitive workload in human-computer interaction. In: Proceedings of the International Conference on HCI (2003)

    Google Scholar 

  3. Tsai, Y.-F., Viirre, E., Strychacz, C., Chase, B., Jung, T.-P.: Task performance and eye activity: predicting behavior relating to cognitive workload. Aviat. Space Environ. Med. 78, B176–B185 (2007)

    Google Scholar 

  4. Ahlstrom, U., Friedman-Berg, F.J.: Using eye movement activity as a correlate of cognitive workload. Int. J. Ind. Ergon. 36, 623–636 (2006)

    Article  Google Scholar 

  5. Hirshfield, L.M., Chauncey, K., Gulotta, R., Girouard, A., Solovey, E.T., Jacob, R.J., Sassaroli, A., Fantini, S.: Combining electroencephalograph and functional near infrared spectroscopy to explore users’ mental workload. Presented at the (2009)

    Google Scholar 

  6. Ayaz, H., Shewokis, P.A., Bunce, S., Izzetoglu, K., Willems, B., Onaral, B.: Optical brain monitoring for operator training and mental workload assessment. Neuroimage 59, 36–47 (2012)

    Article  Google Scholar 

  7. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Adv. Psychol. 52, 139–183 (1988)

    Article  Google Scholar 

  8. Spector, P.E., Jex, S.M.: Development of four self-report measures of job stressors and strain: interpersonal conflict at work scale, organizational constraints scale, quantitative workload inventory, and physical symptoms inventory. J. Occup. Health Psychol. 3, 356 (1998)

    Article  Google Scholar 

  9. Gawron, V.J.: Human Performance, Workload, and Situational Awareness Measures Handbook. CRC Press (2008)

    Google Scholar 

  10. Mota, S., Picard, R.W.: Automated posture analysis for detecting learner’s interest level. In: Anonymous (ed.) CVPRW’03. Conference on Computer Vision and Pattern Recognition Workshop, 2003. pp. 49–49. IEEE (2003)

    Google Scholar 

  11. Beatty, J., Wagoner, B.L.: Pupillometric signs of brain activation vary with level of cognitive processing. Science 199, 1216–1218 (1978)

    Article  Google Scholar 

  12. Knoll, A., Wang, Y., Chen, F., Xu, J., Ruiz, N., Epps, J., Zarjam, P.: Measuring cognitive workload with low-cost electroencephalograph. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds.) Human-Computer Interaction—INTERACT 2011, pp. 568–571. Springer, Berlin (2011)

    Chapter  Google Scholar 

  13. Berka, C., Levendowski, D.J., Lumicao, M.N., Yau, A., Davis, G., Zivkovic, V.T., Olmstead, R.E., Tremoulet, P.D., Craven, P.L.: EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviat. Space Environ. Med. 78, B231–B244 (2007)

    Google Scholar 

  14. Aaslid, R. (ed.) Transcranial Doppler Sonography. Springer, Wien (1986)

    Google Scholar 

  15. Duschek, S., Schandry, R.: Functional transcranial Doppler sonography as a tool in psychophysiological research. Psychophysiology 40, 436–454 (2003)

    Article  Google Scholar 

  16. Parasuraman, R., Warm, J.S., See, J.E.: Brain systems of vigilance. In: The Attentive Brain. pp. 221–256. The MIT Press, Cambridge, MA (1998)

    Google Scholar 

  17. Shaw, T.H., Warm, J.S., Finomore, V., Tripp, L., Matthews, G., Weiler, E., Parasuraman, R.: Effects of sensory modality on cerebral blood flow velocity during vigilance. Neurosci. Lett. 461, 207–211 (2009)

    Article  Google Scholar 

  18. Shaw, T., Finomore, V., Warm, J., Matthews, G.: Effects of regular or irregular event schedules on cerebral hemovelocity during a sustained attention task. J. Clin. Exp. Neuropsychol. 34, 57–66 (2012)

    Article  Google Scholar 

  19. Warm, J.S., Matthews, G., Parasuraman, R.: Cerebral hemodynamics and vigilance performance. Mil. Psychol. 21, S75–S100 (2009)

    Article  Google Scholar 

  20. Hitchcock, E.M., Warm, J.S., Matthews, G., Dember, W.N., Shear, P.K., Tripp, L.D., Mayleben, D.W., Parasuraman, R.: Automation cueing modulates cerebral blood flow and vigilance in a simulated air traffic control task. Theor. Issues Ergon. Sci. 4, 89–112 (2003)

    Article  Google Scholar 

  21. Helton, W.S., Warm, J.S., Tripp, L.D., Matthews, G., Parasuraman, R., Hancock, P.A.: Cerebral lateralization of vigilance: a function of task difficulty. Neuropsychologia 48, 1683–1688 (2010)

    Article  Google Scholar 

  22. Nygren, T.E.: Psychometric properties of subjective workload measurement techniques: implications for their use in the assessment of perceived mental workload. Hum. Factors J. Hum. Factors Ergon. Soc. 33, 17–33 (1991)

    Google Scholar 

  23. Noyes, J.M., Bruneau, D.P.: A self-analysis of the NASA-TLX workload measure. Ergonomics 50, 514–519 (2007)

    Article  Google Scholar 

  24. Yeh, Y.Y., Wickens, C.D.: Dissociation of performance and subjective measures of workload. Hum. Factors J. Hum. Factors Ergon. Soc. 30, 111–120 (1988)

    Google Scholar 

  25. Vizer, L.M., Zhou, L., Sears, A.: Automated stress detection using keystroke and linguistic features: An exploratory study. Int. J. Hum.-Comput. Stud. 67, 870–886 (2009)

    Article  Google Scholar 

  26. Qi, Y., Reynolds, C., Picard, R.W.: The Bayes Point Machine for computer-user frustration detection via pressuremouse. In: Anonymous (ed.) Proceedings of the 2001 Workshop on Perceptive User Interfaces. pp. 1–5. ACM (2001)

    Google Scholar 

  27. D’Mello, S., Picard, R., Graesser, A.: Toward an affect-sensitive autotutor. IEEE Intell. Syst. Special issue on Int (2007)

    Google Scholar 

  28. Frank, G.R.: Monitoring seated postural responses to assess cognitive state (2007)

    Google Scholar 

  29. Pearl, J., Russell, S.: Bayesian Networks (2000)

    Google Scholar 

  30. Pfautz, J., Cox, Z., Koelle, D., Catto, G., Campolongo, J., Roth, E.: User-centered methods for rapid creation and validation of bayesian networks. In: Anonymous (ed.) 5th Bayesian Applications Workshop at Uncertainty in Artificial Intelligence (UAI ’07) (2007)

    Google Scholar 

  31. Hart, S.G.: NASA-task load index (NASA-TLX); 20 years later. In: Anonymous (ed.) Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp. 904–908. Sage Publications (2006)

    Google Scholar 

  32. Cao, A., Chintamani, K.K., Pandya, A.K., Ellis, R.D.: NASA TLX: software for assessing subjective mental workload. Behav. Res. Methods. 41, 113–117 (2009)

    Article  Google Scholar 

  33. Pfautz, J., Cox, Z., Catto, G., Koelle, D., Campolongo, J., Roth, E.: User-centered methods for rapid creation and validation of bayesian belief networks. In: Anonymous (ed.) 23nd Annual Conference on Uncertainty in Artificial Intelligence: UAI ’07 (2006)

    Google Scholar 

  34. Cox, Z., Pfautz, J.: Causal influence models: a method for simplifying construction of bayesian networks. Charles River Analytics Inc. (2007)

    Google Scholar 

  35. Pfautz, J., Koelle, D., Carlson, E., Roth, E.: Complexities and challenges in the use of bayesian belief networks: informing the design of causal influence models. Presented at the (2009)

    Google Scholar 

  36. Cao, D., Guarrera, T.K., Jenkins, M., Pennathur, P.R., Bisantz, A.M., Stone, R., Farry, M., Pfautz, J., Roth, E.: Evaluating the creation and interpretation of causal influence models. In: Anonymous (ed.) Proceedings of the Human Factors and Ergonomics Society Annual Meeting. pp. 222–226. Sage Publications (2009)

    Google Scholar 

  37. Yerkes, R.M., Dodson, J.D.: The relation of strength of stimulus to rapidity of habit-formation. J. Comp. Neurol. Psychol. 18, 459–482 (1908)

    Article  Google Scholar 

Download references

Acknowledgments

This material is based on work supported by the United States Air Force under Contract No. FA8650-15-C-6628. The views, opinions, and/or findings expressed are those of the author(s) and should not be interpreted as representing the official views or policies of the Department of Defense or the US Government.

Author information

Authors and Affiliations

  1. Charles River Analytics, 625 Mount Auburn Street, Cambridge, MA, 02138, USA

    Seth Elkin-Frankston, Bethany K. Bracken, Scott Irvin & Michael Jenkins

Authors
  1. Seth Elkin-Frankston
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bethany K. Bracken
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Scott Irvin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Jenkins
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seth Elkin-Frankston .

Editor information

Editors and Affiliations

  1. University of Central Florida , Orlando, Florida, USA

    Tareq Z. Ahram

  2. University of Central Florida , Orlando, USA

    Waldemar Karwowski

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this paper

Cite this paper

Elkin-Frankston, S., Bracken, B.K., Irvin, S., Jenkins, M. (2017). Are Behavioral Measures Useful for Detecting Cognitive Workload During Human-Computer Interaction?. In: Ahram, T., Karwowski, W. (eds) Advances in The Human Side of Service Engineering. Advances in Intelligent Systems and Computing, vol 494. Springer, Cham. https://doi.org/10.1007/978-3-319-41947-3_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-41947-3_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-41946-6

  • Online ISBN: 978-3-319-41947-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation