Using Context to Optimize a Functional State Estimation Engine in Unmanned Aircraft System Operations

  • Kevin Durkee
  • Scott Pappada
  • Andres Ortiz
  • John Feeney
  • Scott Galster
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9183)


As UAS operations continue to expand, the ability to monitor real-time cognitive states of human operators would be a valuable asset. Although great strides have been made toward this capability using physiological signals, the inherent noisiness of these data hinders its readiness for operational deployment. We theorize the addition of contextual data alongside physiological signals could improve the accuracy of cognitive state classifiers. In this paper, we review a cognitive workload model development effort conducted in a simulated UAS task environment at the Air Force Research Laboratory (AFRL). Real-time workload model classifiers were trained using three levels of physiological data inputs both with and without context added. Following the evaluation of each classifier using four model evaluation metrics, we conclude that by adding contextual data to physiological-based models, we improved the ability to reliably measure real-time cognitive workload in our UAS operations test case.


Context Human performance Modeling and simulation Physiological measurement Workload UAS 



Distribution A: Approved for public release; distribution unlimited. This material is based on work supported by AFRL under Contract FA8650-11-C-6236. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of AFRL.


  1. 1.
    Blackhurst, J., Gresham, J., Stone, M.: The quantified warrior: how DoD should lead human performance augmentation. Armed Forces Journal (2012)Google Scholar
  2. 2.
    Gevins, A., Smith, M.E., Leong, H., McEvoy, L., Whitfield, S., Du, R., Rush, G.: Monitoring working memory load during computer-based tasks with EEG pattern recognition methods. Hum. Factors 40, 79–91 (1998)CrossRefGoogle Scholar
  3. 3.
    Hoover, A., Singh, A., Fishel-Brown, S., Muth, E.: Real-time detection of workload changes using heart rate variability. Biomed. Signal Process. Control 7, 333–341 (2012)CrossRefMATHGoogle Scholar
  4. 4.
    Just, M.A., Carpenter, P.A.: The intensity dimension of thought: pupillometric indices of sentence processing. Can. J. Exp. Psychol. 47(2), 310–339 (1993)CrossRefGoogle Scholar
  5. 5.
    Setz, C., Arnrich, B., Schumm, J., La Marca, R., Troster, G.: Discriminating stress from cognitive load using a wearable EDA device. Technology 14(2), 410–417 (2010)Google Scholar
  6. 6.
    John, M., Kobus, D.A., Morrison, J.G., Schmorrow, D.: Overview of the DARPA augmented cognition technical integration experiment. Int. J. Hum. Comput. Interact. 17(2), 131–149 (2004)CrossRefGoogle Scholar
  7. 7.
    Wilson, G.F., Russell, C.A.: Real-time assessment of mental workload using psychophysiological measures and artificial neural networks. Hum. Factors 45(4), 635–644 (2003)CrossRefGoogle Scholar
  8. 8.
    Wang, Z., Hope, R.M., Wang, Z., Ji, Q., Gray, W.: Cross-subject workload classification with a hierarchical bayes model. NeuroImage 59(1), 64–69 (2012)CrossRefGoogle Scholar
  9. 9.
    Durkee, K., Geyer, A., Pappada, S., Ortiz, A., Galster, S.: Real-time workload assessment as a foundation for human performance augmentation. In: Schmorrow, D.D., Fidopiastis, C.M. (eds.) AC 2013. LNCS, vol. 8027, pp. 279–288. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  10. 10.
    Rowe, A.J., Liggett, K.K., Davis, J.E.: Vigilant spirit control station: a research testbed for multi-UAS supervisory control interfaces. In: Proceedings of the 15th International Symposium on Aviation Psychology, Dayton, OH (2009)Google Scholar
  11. 11.
    Dey, A.: Understanding and using context. Pers. Ubiquitous Comput. 5(1), 4–7 (2001)CrossRefGoogle Scholar
  12. 12.
    Dey, A., Abowd, G., Salber, D.: A conceptual framework and a toolkit for supporting the rapid prototyping of context-aware applications. Hum. Comput. Interact. 16(2), 97–166 (2001)CrossRefGoogle Scholar
  13. 13.
    Finomore, V., Popik, D., Dallman, R., Stewart, J., Satterfield, K., Castle, C.: Demonstration of a network-centric communication management suite: multi-modal communication. In: Proceedings of the 55th Human Factors and Ergonomics Society Annual Meeting. HFES, Las Vegas (2011)Google Scholar
  14. 14.
    Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In: Peter, A.H., Najmedin, M. (eds.) Advances in Psychology, vol. 52, pp. 139–183. Elseiver, North-Holland (2006)Google Scholar
  15. 15.
    Xu, J., Bakardjian, H., Cichocki, A., Principe, J.: A new nonlinear similarity measure for multichannel signals. Neural Netw. 21, 222–231 (2008)CrossRefMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Kevin Durkee
    • 1
  • Scott Pappada
    • 1
  • Andres Ortiz
    • 1
  • John Feeney
    • 1
  • Scott Galster
    • 2
  1. 1.Aptima, IncFairbornUSA
  2. 2.Air Force Research LaboratoryDaytonUSA

Personalised recommendations