Enhancing HMD-Based F-35 Training through Integration of Eye Tracking and Electroencephalography Technology
The ever increasing complexity of knowledge, skills and abilities (KSAs) demanded of Department of Defense (DoD) personnel has created the need to develop tools to increase the efficiency and effectiveness of training. This is especially true for the F-35, the first 5th-generation aircraft to use an HMD as the primary instrument display. Additionally, the F-35 can perform operations previously performed by multiple operators, which potentially places incredible strain on the pilot’s cognitive resources by exposing him to large amounts of data from disparate sources. It is critical to ensure training results in pilots learning optimal strategies for operating in this information rich environment. This paper discusses current efforts to develop and evaluate a performance monitoring and assessment system which integrates eye tracking and Electroencephalography (EEG) technology into an HMD enabled F-35 training environment to extend traditional behavioral metrics and better understand how a pilot interacts with data presented in the HMD.
KeywordsTraining Performance Assessment Eye tracking EEG Helmetmounted display Heads-up display F-35
Unable to display preview. Download preview PDF.
- 1.United States Air Force. The Edge Air Force Transformation 2010 (2008), http://permanent.access.gpo.gov/lps40477/edgeweb.pdf (retrieved June 1, 2011)
- 2.Endsley, M.R.: Designing for Situation Awareness in Complex Systems. In: Proceedings of the 2nd International Workshop on Symbiosis of Humans, Artifacts and Environment, Kyoto, Japan (2001)Google Scholar
- 5.Mello-Thoms, C., Ganott, M., Sumkin, J., Hakim, C., Britton, C., Wallace, L., Hardesty, L.: Different search patterns and similar decision outcomes: How can experts agree in the decisions they make when reading digital mammograms? In: Krupinski, E.A. (ed.) IWDM 2008. LNCS, vol. 5116, pp. 212–219. Springer, Heidelberg (2008)CrossRefGoogle Scholar
- 7.Weibel, N., Fouse, A., Emmenegger, C., Kimmich, S., Hutchins, E.: Let’s look at the cockpit: exploring mobile eye-tracking for observational research on the flight deck. In: Proceedings of the Symposium on Eye Tracking Research and Applications, pp. 107–114. ACM (March 2012)Google Scholar
- 8.Wetzel, P.A., Anderson, G.M., Barelka, B.A.: Instructor use of eye position based feedback for pilot training. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 42(20), pp. 1388–1392. Sage Publications (October 1998)Google Scholar
- 9.Carroll, M., Fuchs, S., Hale, K., Dargue, B., Buck, B.: Advanced Training Evaluation System: Leveraging Neuro-physiological Measurement to Individualize Training. In: The Interservice/Industry Training, Simulation & Education Conference (I/ITSEC), vol. 2010(1), National Training Systems Association (January 2010)Google Scholar
- 10.Schnell, T., Hamel, N., Postnikov, A., Hoke, J., McLean III, A.L.: Physiological Based Simulator Fidelity Design Guidance (2012)Google Scholar
- 11.Carroll, M.B., Kokini, C., Moss, J.: Multi-Axis Performance Interpretation Tool (MAPIT) Training Effectiveness Evaluation Report (Program Interim Rep., Contract No. N00014-10-C-0091). Office of Naval Research, Arlington (2010a)Google Scholar
- 12.Hale, K.S., Fuchs, S., Axelsson, P., Baskin, A., Jones, D.: Determining gaze parameters to guide EEG/ERP evaluation of imagery analysis. In: Schmorrow, D.D., Nicholson, D.M., Drexler, J.M., Reeves, L.M. (eds.) Foundations of Augmented Cognition, 4th edn., pp. 33–40. Strategic Analysis Inc., Arlington (2007)Google Scholar
- 14.Hale, K.S., Fuchs, S., Berka, C.: Driving EEG cognitive assessment using eye fixations. In: Applied Human Factors and Ergonomics 2nd International Conference, Las Vegas, NV, July 14-17 (2008b)Google Scholar
- 15.Carpenter, A., Fuchs, S., Whitlow, S., Fiore, S., Hale, K.: Auto-Diagnostic Adaptive Precision Training for Baggage Screeners (Screen-ADAPT). Phase I Final Report. Department of Homeland Security Contract No. N10PC20028 (2010); Carroll, M., Fuchs, S., Hale, K., Dargue, B., Buck, B.: Advanced training evaluation system: Leveraging Neuro-physiological measurement to individualize training. In: Proceedings of the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) Annual Meeting, Orlando, FL (2010b)Google Scholar
- 16.Vision Systems International - JHMCS, Vsi-hmcs.com (retrieved September 15, 2010)