Advertisement

Enhancing HMD-Based F-35 Training through Integration of Eye Tracking and Electroencephalography Technology

  • Meredith Carroll
  • Glenn Surpris
  • Shayna Strally
  • Matthew Archer
  • Frank Hannigan
  • Kelly Hale
  • Wink Bennett
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8027)

Abstract

The ever increasing complexity of knowledge, skills and abilities (KSAs) demanded of Department of Defense (DoD) personnel has created the need to develop tools to increase the efficiency and effectiveness of training. This is especially true for the F-35, the first 5th-generation aircraft to use an HMD as the primary instrument display. Additionally, the F-35 can perform operations previously performed by multiple operators, which potentially places incredible strain on the pilot’s cognitive resources by exposing him to large amounts of data from disparate sources. It is critical to ensure training results in pilots learning optimal strategies for operating in this information rich environment. This paper discusses current efforts to develop and evaluate a performance monitoring and assessment system which integrates eye tracking and Electroencephalography (EEG) technology into an HMD enabled F-35 training environment to extend traditional behavioral metrics and better understand how a pilot interacts with data presented in the HMD.

Keywords

Training Performance Assessment Eye tracking EEG Helmetmounted display Heads-up display F-35 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    United States Air Force. The Edge Air Force Transformation 2010 (2008), http://permanent.access.gpo.gov/lps40477/edgeweb.pdf (retrieved June 1, 2011)
  2. 2.
    Endsley, M.R.: Designing for Situation Awareness in Complex Systems. In: Proceedings of the 2nd International Workshop on Symbiosis of Humans, Artifacts and Environment, Kyoto, Japan (2001)Google Scholar
  3. 3.
    Raab, M., Johnson, J.G.: Expertise-based differences in search and option-generation strategies. Journal of Experimental Psychology: Applied 13(3), 158–170 (2007)CrossRefGoogle Scholar
  4. 4.
    Jarodzka, H., Scheiter, K., Gerjets, P., van Gog, T.: In the eyes of the beholder: How experts and novices interpret dynamic stimuli. Learning and Instruction 20(2), 146–154 (2009)CrossRefGoogle Scholar
  5. 5.
    Mello-Thoms, C., Ganott, M., Sumkin, J., Hakim, C., Britton, C., Wallace, L., Hardesty, L.: Different search patterns and similar decision outcomes: How can experts agree in the decisions they make when reading digital mammograms? In: Krupinski, E.A. (ed.) IWDM 2008. LNCS, vol. 5116, pp. 212–219. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  6. 6.
    White Jr., K.P., Hutson, T.L., Hutchinson, T.E.: Modeling human eye behavior during mammographic scanning: Preliminary results. IEEE Transactions on Systems, Man, & Cybernetics Part A: Systems & Humans 27(4), 494–505 (1997)CrossRefGoogle Scholar
  7. 7.
    Weibel, N., Fouse, A., Emmenegger, C., Kimmich, S., Hutchins, E.: Let’s look at the cockpit: exploring mobile eye-tracking for observational research on the flight deck. In: Proceedings of the Symposium on Eye Tracking Research and Applications, pp. 107–114. ACM (March 2012)Google Scholar
  8. 8.
    Wetzel, P.A., Anderson, G.M., Barelka, B.A.: Instructor use of eye position based feedback for pilot training. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 42(20), pp. 1388–1392. Sage Publications (October 1998)Google Scholar
  9. 9.
    Carroll, M., Fuchs, S., Hale, K., Dargue, B., Buck, B.: Advanced Training Evaluation System: Leveraging Neuro-physiological Measurement to Individualize Training. In: The Interservice/Industry Training, Simulation & Education Conference (I/ITSEC), vol. 2010(1), National Training Systems Association (January 2010)Google Scholar
  10. 10.
    Schnell, T., Hamel, N., Postnikov, A., Hoke, J., McLean III, A.L.: Physiological Based Simulator Fidelity Design Guidance (2012)Google Scholar
  11. 11.
    Carroll, M.B., Kokini, C., Moss, J.: Multi-Axis Performance Interpretation Tool (MAPIT) Training Effectiveness Evaluation Report (Program Interim Rep., Contract No. N00014-10-C-0091). Office of Naval Research, Arlington (2010a)Google Scholar
  12. 12.
    Hale, K.S., Fuchs, S., Axelsson, P., Baskin, A., Jones, D.: Determining gaze parameters to guide EEG/ERP evaluation of imagery analysis. In: Schmorrow, D.D., Nicholson, D.M., Drexler, J.M., Reeves, L.M. (eds.) Foundations of Augmented Cognition, 4th edn., pp. 33–40. Strategic Analysis Inc., Arlington (2007)Google Scholar
  13. 13.
    Hale, K.S., Fuchs, S., Axelsson, P., Berka, C., Cowell, A.J.: Using physiological measures to discriminate signal detection outcome during imagery analysis. Human Factors and Ergonomics Society Annual Meeting Proceedings 52(3), 182–186 (2008a)CrossRefGoogle Scholar
  14. 14.
    Hale, K.S., Fuchs, S., Berka, C.: Driving EEG cognitive assessment using eye fixations. In: Applied Human Factors and Ergonomics 2nd International Conference, Las Vegas, NV, July 14-17 (2008b)Google Scholar
  15. 15.
    Carpenter, A., Fuchs, S., Whitlow, S., Fiore, S., Hale, K.: Auto-Diagnostic Adaptive Precision Training for Baggage Screeners (Screen-ADAPT). Phase I Final Report. Department of Homeland Security Contract No. N10PC20028 (2010); Carroll, M., Fuchs, S., Hale, K., Dargue, B., Buck, B.: Advanced training evaluation system: Leveraging Neuro-physiological measurement to individualize training. In: Proceedings of the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) Annual Meeting, Orlando, FL (2010b)Google Scholar
  16. 16.
    Vision Systems International - JHMCS, Vsi-hmcs.com (retrieved September 15, 2010)

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Meredith Carroll
    • 1
  • Glenn Surpris
    • 1
  • Shayna Strally
    • 1
  • Matthew Archer
    • 1
  • Frank Hannigan
    • 1
  • Kelly Hale
    • 1
  • Wink Bennett
    • 2
  1. 1.Design InteractiveOviedoUSA
  2. 2.Air Force Research LaboratoryWright- Patterson Air Force BaseOhioUSA

Personalised recommendations