Considerations in Physiological Metric Selection for Online Detection of Operator State: A Case Study

  • Ryan W. WohleberEmail author
  • Gerald Matthews
  • Gregory J. Funke
  • Jinchao Lin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9743)


The development of closed-loop systems is fraught with many challenges. One of the many important decisions to be made in this development is the selection of suitable metrics to detect operator state. Successful metrics can inform adaptations in an interface’s design, features, or task elements allocated to automated systems. This paper will discuss various challenges and considerations involved in the selection of metrics for detecting fatigue in operators of unmanned aerial vehicles (UAVs). Using Eggemeier and colleague’s guidelines for workload metric selection as a basis, we review several criteria for metric selection and how they are applied to selection of metrics designed to assess operator fatigue in an applied closed-loop system.


Metric selection Fatigue Automated decision making aid Human factors Supervisory control 



This research was sponsored by AFOSR A9550-13-1-0016 and 13RH05COR. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of AFOSR or the US Government.


  1. 1.
    Gertler, J.: U.S. Unmanned Aerial Systems (No. ADA566235). Library of Congress, Congressional Research Service, Washington (2012)Google Scholar
  2. 2.
    Cummings, M.L., Clare, A., Hart, C.: The role of human-automation consensus in multiple unmanned vehicle scheduling. Hum. Factors 52(1), 17–27 (2010)CrossRefGoogle Scholar
  3. 3.
    Neubauer, C., Matthews, G., Langheim, L., Saxby, D.: Fatigue and voluntary utilization of automation in simulated driving. Hum. Factors 54(5), 734–746 (2012)CrossRefGoogle Scholar
  4. 4.
    Calhoun, G.L., Draper, M.H., Miller, C., Ruff, H.A., Breeden, C., Hamell, J.: Adaptable automation interface for multi-unmanned aerial systems control: preliminary usability evaluation. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 57(1), 26–30 (2013)CrossRefGoogle Scholar
  5. 5.
    Wohleber, R.W., Calhoun, G.L., Funke, G.J., Ruff, H.A., Chiu, C.-Y.P., Lin, J., Matthews, G.: The impact of automation reliability on performance and reliance changes with operator fatigue (in preparation)Google Scholar
  6. 6.
    Matthews, G., Desmond, P.A., Hitchcock, E.M.: Dimensional models of fatigue. In: Matthews, G., Desmond, P.A., Neubauer, C., Hancock, P.A. (eds.) The Handbook of Operator Fatigue, pp. 139–154. Ashgate Publishing Company, Burlington (2012)Google Scholar
  7. 7.
    Michielsen, H.J., De Vries, J., Van Heck, G.L., Van de Vijver, F.J., Sijtsma, K.: Examination of the dimensionality of fatigue: the construction of the fatigue assessment scale (FAS). Eur. J. Psychol. Assess. 20(1), 39–48 (2004)CrossRefGoogle Scholar
  8. 8.
    Matthews, G., Campbell, S.E., Falconer, S., Joyner, L.A., Huggins, J., Gilliland, K., Grier, R., Warm, J.S.: Fundamental dimensions of subjective state in performance settings: task engagement, distress, and worry. Emotion 2(4), 315–340 (2002)CrossRefGoogle Scholar
  9. 9.
    Saxby, D.J., Matthews, G., Warm, J.S., Hitchcock, E.M., Neubauer, C.: Active and passive fatigue in simulated driving: discriminating styles of workload regulation and their safety impacts. J. Exp. Psychol. Appl. 19(4), 287–300 (2013)CrossRefGoogle Scholar
  10. 10.
    Åkerstedt, T., Gillberg, M.: Subjective and objective sleepiness in the active individual. Int. J. Neurosci. 52(1–2), 29–37 (1990)CrossRefGoogle Scholar
  11. 11.
    Philip, P., Sagaspe, P., Taillard, J., Moore, N., Guilleminault, C., Sanchez-Ortuno, M., Akerstedt, T., Bioulac, B.: Fatigue, sleep restriction, and performance in automobile drivers: a controlled study in a natural environment. Sleep 26(3), 277–284 (2003)Google Scholar
  12. 12.
    Abich, J., Reinerman-Jones, L., Taylor, G.S.: Investigating workload measures for adaptive training systems. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 57(1), 2091–2095 (2013)CrossRefGoogle Scholar
  13. 13.
    Warm, J.S., Matthews, G., Finomore, V.S.: Vigilance, workload, and stress. In: Hancock, P.A., Szalma, J.L. (eds.) Performance Under Stress, pp. 115–141. Ashgate Publishing Company, Burlington (2008)Google Scholar
  14. 14.
    Warm, J.S., Parasuraman, R., Matthews, G.: Vigilance requires hard mental work and is stressful. Hum. Factors 50(3), 433–441 (2008)CrossRefGoogle Scholar
  15. 15.
    Matthews, G., Desmond, P.A.: Task-induced fatigue states and simulated driving performance. Q. J. Exp. Psychol. Sect. A 55(2), 659–686 (2002)CrossRefGoogle Scholar
  16. 16.
    Desmond, P.A., Hancock, P.A.: Active and passive fatigue states. In: Hancock, P.A., Desmond, P.A. (eds.) Stress, Workload, and Fatigue, pp. 455–465. Lawrence Erlbaum Associates, Mahwah (2001)Google Scholar
  17. 17.
    Hockey, G.R.J.: Compensatory control in the regulation of human performance under stress and high workload: a cognitive-energetical framework. Biol. Psychol. 45(1–3), 73–93 (1997)CrossRefGoogle Scholar
  18. 18.
    Mascord, D.J., Heath, R.A.: Behavioral and physiological indices of fatigue in a visual tracking task. J. Saf. Res. 23(1), 19–25 (1992)CrossRefGoogle Scholar
  19. 19.
    Körber, M., Cingel, A., Zimmermann, M., Bengler, K.: Vigilance decrement and passive fatigue caused by monotony in automated driving. Procedia Manufact. 3, 2403–2409 (2015)CrossRefGoogle Scholar
  20. 20.
    Saxby, D.J., Matthews, G., Hitchcock, E.M., Warm, J.S., Funke, G.J., Gantzer, T.: Effect of active and passive fatigue on performance using a driving simulator. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 52, 1751–1755 (2008)CrossRefGoogle Scholar
  21. 21.
    Lee, I.-S., Bardwell, W.A., Ancoli-Israel, S., Dimsdale, J.E.: Number of lapses during the psychomotor vigilance task as an objective measure of fatigue. J. Clin. Sleep Med. 6(2), 163–168 (2010)Google Scholar
  22. 22.
    Matthews, G., Warm, J.S., Reinerman-Jones, L.E., Langheim, L.K., Washburn, D.A., Tripp, L.: Task engagement, cerebral blood flow velocity, and diagnostic monitoring for sustained attention. J. Exp. Psychol. Appl. 16(2), 187–203 (2010)CrossRefGoogle Scholar
  23. 23.
    Craig, A., Tran, Y.: The influence of fatigue on brain activity. In: Matthews, G., Desmond, P.A., Neubauer, C., Hancock, P.A. (eds.) The Handbook of Operator Fatigue, pp. 185–196. Ashgate Publishing Company, Burlington (2012)Google Scholar
  24. 24.
    Verwey, W.B., Zaidel, D.M.: Predicting drowsiness accidents from personal attributes, eye blinks and ongoing driving behaviour. Pers. Individ. Differ. 28(1), 123–142 (2000)CrossRefGoogle Scholar
  25. 25.
    Smith, E.R., DeCoster, J.: Dual-process models in social and cognitive psychology: conceptual integration and links to underlying memory systems. Pers. Soc. Psychol. Rev. 4(2), 108–131 (2000)CrossRefGoogle Scholar
  26. 26.
    Prinzel, L.J., Freeman, F.G., Scerbo, M.W., Mikulka, P.J., Pope, A.T.: Effects of a psychophysiological system for adaptive automation on performance, workload, and the event-related potential P300 component. Hum. Factors 45(4), 601–614 (2003)CrossRefGoogle Scholar
  27. 27.
    Gevins, A.S., Bressler, S.L., Cutillo, B.A., Illes, J., Miller, J.C., Stern, J., Jex, H.R.: Effects of prolonged mental work on functional brain topography. Electroencephalogr. Clin. Neurophysiol. 76(4), 339–350 (1990)CrossRefGoogle Scholar
  28. 28.
    Segerstrom, S.C., Nes, L.S.: Heart rate variability reflects self-regulatory strength, effort, and fatigue. Psychol. Sci. 18(3), 275–281 (2007)CrossRefGoogle Scholar
  29. 29.
    Warm, J.S., Tripp, L.D., Matthews, G., Helton, W.S.: Cerebral hemodynamic indices of operator fatigue in vigilance. In: Matthews, G., Desmond, P.A., Neubauer, C., Hancock, P.A. (eds.) The Handbook of Operator Fatigue. Ashgate Publishing Company, Burlington (2012)Google Scholar
  30. 30.
    Li, Z., Zhang, M., Zhang, X., Dai, S., Yu, X., Wang, Y.: Assessment of cerebral oxygenation during prolonged simulated driving using near infrared spectroscopy: its implications for fatigue development. Eur. J. Appl. Physiol. 107(3), 281–287 (2009)CrossRefGoogle Scholar
  31. 31.
    DeGreef, T., Lafeber, H., van Oostendorp, H., Lindenberg, J.: Eye movement as indicators of mental workload to trigger adaptive automation. In: Schmorrow, D.D., Estabrooke, I.V., Grootjen, M. (eds.) Foundations of Augmented Cognition. Neuroergonomics and Operational Neuroscience, pp. 219–228. Springer, New York (2009)CrossRefGoogle Scholar
  32. 32.
    Stern, R.M., Ray, W.J., Quigley, K.S.: Psychophysiological Recording, 2nd edn. Oxford University Press, New York (2001)Google Scholar
  33. 33.
    Wierwille, W.W., Wreggit, S.S., Kirn, C.L., Ellsworth, L.A., Fairbanks, R.J.: Research on vehicle-based driver status/performance monitoring; development, validation, and refinement of algorithms for detection of driver drowsiness (No. HS-808 247 VPISU ISE 94-04) (1994)Google Scholar
  34. 34.
    Kozak, K., Curry, R., Greenberg, J., Artz, B., Blommer, M., Cathey, L.: Leading indicators of drowsiness in simulated driving. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 49, 1917–1921 (2005). SAGE PublicationsCrossRefGoogle Scholar
  35. 35.
    Schleicher, R., Galley, N., Briest, S., Galley, L.: Blinks and saccades as indicators of fatigue in sleepiness warnings: looking tired? Ergonomics 51(7), 982–1010 (2008)CrossRefGoogle Scholar
  36. 36.
    Palmer, S.E.: Vision Science: Photons to Phenomenology. MIT Press, Cambridge (1999)Google Scholar
  37. 37.
    Eggemeier, F.T., Wilson, G.F., Kramer, A.F., Damos, D.L.: Workload assessment in multi-task environments. In: Damos, D.L. (ed.) Multiple-Task Performance, pp. 207–216. Taylor & Francis, London (1991)Google Scholar
  38. 38.
    Freeman, F.G., Mikulka, P.J., Prinzel, L.J., Scerbo, M.W.: Evaluation of an adaptive automation system using three EEG indices with a visual tracking task. Biol. Psychol. 50(1), 61–76 (1999)CrossRefGoogle Scholar
  39. 39.
    Briest, S., Karrer, K., Schleicher, R.: Driving without awareness: examination of the phenomenon. In: Gale, A. (ed.) Vision in Vehicles XI, pp. 89–141. Elsevier, Amsterdam (2006)Google Scholar
  40. 40.
    Matthews, G.: Towards a transactional ergonomics for driver stress and fatigue. Theor. Issues Ergon. Sci. 3(2), 195–211 (2002)CrossRefGoogle Scholar
  41. 41.
    Hitchcock, E.M., Warm, J.S., Matthews, G., Dember, W.N., Shear, P.K., Tripp, L.D., Mayleben, D.W., Parasuraman, R.: Automation cueing modulates cerebral blood flow and vigilance in a simulated air traffic control task. Theor. Issues Ergon. Sci. 4(1–2), 89–112 (2003)CrossRefGoogle Scholar
  42. 42.
    Tattersall, A.J., Foord, P.S.: An experimental evaluation of instantaneous self-assessment as a measure of workload. Ergonomics 39(5), 740–748 (1996)CrossRefGoogle Scholar
  43. 43.
    Metzger, U., Parasuraman, R.: The role of the air traffic controller in future air traffic management: an empirical study of active control versus passive monitoring. Hum. Factors 43(4), 519–528 (2001)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Ryan W. Wohleber
    • 1
    Email author
  • Gerald Matthews
    • 1
  • Gregory J. Funke
    • 2
  • Jinchao Lin
    • 1
  1. 1.Institute for Simulation and TrainingUniversity of Central FloridaOrlandoUSA
  2. 2.Air Force Research LaboratoryWright-Patterson AFBDaytonUSA

Personalised recommendations