Abstract
In an attempt to predict and prevent accident situations in complex socio-technical systems, one needs to be able to model and simulate concepts such as situation awareness (SA) and processes responsible for maintaining it. This is particularly true in the case of online support systems and adaptive displays which cannot rely on SA measurement techniques based on freeze probe techniques. This work investigates the state of the art in computational models of situation awareness and proposes a method to calibrate and evaluate such models using virtual reality human-in-the-loop experiments. This work introduces a new methodology to evaluate and calibrate online SA assessment systems taking advantage of the flexibility and reconfigurable power of virtual reality environments. This technology provides the experimenter with full control on the scenarios, cockpit types and interfaces. It also allows testing of off-nominal situations such as the loss of an instrument and more severe failures. Moreover, eye tracking capabilities provide an accurate way of registering monitoring events and feed SA assessment models with realistic data.
Chapter PDF
Similar content being viewed by others
References
Aslandere, T.I.: Interaction Methods in a Generic Virtual Reality Flight Simulator. Master’s thesis, Technische Universitat Munchen (2013)
Dehais, F., Tessier, C., Christophe, L., Reuzeau, F.: The perseveration syndrome in the pilots activity: Guidelines and cognitive countermeasures. In: Palanque, P., Vanderdonckt, J., Winckler, M. (eds.) HESSD 2009. LNCS, vol. 5962, pp. 68–80. Springer, Heidelberg (2010)
Dekker, S., Hollnagel, E.: Human factors and folk models. Cognition, Technology & Work 6(2), 79–86 (2004)
Dreyer, D., Bandow, D., Oberhauser, M.: Hud symbology evaluation in a virtual reality flight simulation. Paper submitted to HCI Aero 2014, Silicon Valley, USA (2014)
Ehrlich, J., Knerr, B., Lampton, D., McDonald, D.: Team situational awareness training in virtual environments: Potential capabilities and research issues. Tech. rep., U.S. Army Research Institute for the Behavioral and Social Sciences (1997)
Endsley, M.R.: Measurement of situation awareness in dynamic systems. Human Factors: The Journal of the Human Factors and Ergonomics Society 37(1), 65–84 (1995)
Endsley, M.R.: Toward a theory of situation awareness in dynamic systems. Human Factors: The Journal of the Human Factors and Ergonomics Society 37(1), 32–64 (1995)
Endsley, M.R.: Designing for situation awareness: An approach to user-centered design. Taylor & Francis, US (2003)
Endsley, M.: Situation awareness global assessment technique (sagat). In: Proceedings of the IEEE 1988 National Aerospace and Electronics Conference, NAECON 1988, pp. 789–795. IEEE (1988)
Feigh, K.M., Gelman, G., Mamessier, S., Pritchett, A.R.: Simulating first-principles models of situated human performance. IEEE Transactions on System Man and Cybernetics: Part A (submitted June 2012)
Hanson, M.L., Sullivan, O., Harper, K.A.: On-line situation assessment for unmanned air vehicles. In: FLAIRS Conference, pp. 44–48 (2001)
Hooey, B.L., Gore, B.F., Wickens, C.D., Scott-Nash, S., Socash, C., Salud, E., Foyle, D.C.: Modeling pilot situation awareness. In: Human Modelling in Assisted Transportation, pp. 207–213. Springer, Heidelberg (2011)
Horsch, C., Smets, N., Neerincx, M., Cuijpers, R.: Comparing performance and situation awareness in usar unit tasks in a virtual and real environment. In: Proceedings of the 10th International ISCRAM Conference, Baden-Baden, Germany, pp. 556–560 (2013)
Jones, D.G., Endsley, M.R.: Sources of situation awareness errors in aviation. Aviation, Space, and Environmental Medicine (1996)
Kalawsky, R.: New methodologies and techniques for evaluating user performance in advanced 3d virtual interfaces. IEEE Colloquium Digest 98/43 (1998)
Kalawsky, R.: Vruse - a computerised diagnostic tool for usability evaluation of virtual/synthetic environment systems. Applied Ergonomics 30, 11–25 (1999)
Kalawsky, R., Bee, S., Nee, S.: Human factors evaluation techniques to aid understanding of virtual interfaces. BT Technology Journal 17(1), 128–141 (1999)
Kleinman, D., Baron, S., Levison, W.: An optimal control model of human response part i: Theory and validation. Automatica 6(3), 357–369 (1970)
Laptaned, U.: Situation awareness in virtual environments: A theoretical model and investigation with different interface designs. In: Proceedings of the 9th IASTED International Conference Computers and Advanced Technology in Education, Lima, Peru, pp. 277–283 (2006)
Lee, H.C., Seong, P.H.: A computational model for evaluating the effects of attention, memory, and mental models on situation assessment of nuclear power plant operators. Reliability Engineering & System Safety 94(11), 1796–1805 (2009)
Letsu-Dake, E., Ntuen, C.A.: A case study of experimental evaluation of adaptive interfaces. International Journal of Industrial Ergonomics 40(1), 34–40 (2010)
Liesecke, S.: Eye-Tracking in Virtual Reality. Master’s thesis, Universitt der Bundeswehr Munchen (2013)
Mamessier, S.: A computational approach to situation awareness and mental models in aviation. Master’s thesis, Georgia Institute of Technology (2013)
Mamessier, S., Feigh, K.: Simulating the impact of mental models on human automation interaction in aviation. In: Duffy, V.G. (ed.) DHM/HCII 2013, Part I. LNCS, vol. 8025, pp. 61–69. Springer, Heidelberg (2013)
Mamessier, S., Feigh, K.: A computational approach to situation awareness and mental models for continuous dynamics in aviation. IEEE Transactions on Human-Machine Systems, (2014) (Under review)
Matthews, M., Beal, S., Pleban, R.: Situation awareness in a virtual environment: Description of a subjective assessment scale. Tech. rep. U.S. Army Research Institute for the Behavioral and Social Sciences (2002)
McCarley, J.S., Wickens, C.D., Goh, J., Horrey, W.J.: A computational model of attention/situation awareness. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 46, pp. 1669–1673. SAGE Publications (2002)
Ratwani, R.M., McCurry, J.M., Trafton, J.G.: Single operator, multiple robots: An eye movement based theoretic model of operator situation awareness. In: 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 235–242. IEEE (2010)
Salmon, P., Stanton, N., Walker, G., Green, D.: Situation awareness measurement: A review of applicability for c4i environments. Applied Ergonomics 37(2), 225–238 (2006)
Sarter, N.: Coping with complexity through adaptive interface design. In: Jacko, J.A. (ed.) Human-Computer Interaction, Part III, HCII 2007. LNCS, vol. 4552, pp. 493–498. Springer, Heidelberg (2007)
Sarter, N.B., Woods, D.D.: Situation awareness: A critical but ill-defined phenomenon. The International Journal of Aviation Psychology 1(1), 45–57 (1991)
Stanton, N.A., Chambers, P., Piggott, J.: Situational awareness and safety. Safety Science 39(3), 189–204 (2001)
Tien, G., Atkins, M.S., Zheng, B., Swindells, C.: Measuring situation awareness of surgeons in laparoscopic training. In: Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications, pp. 149–152. ACM (2010)
Vicente, K., Pawlak, W.: Cognitive work analysis for the duress ii system. Cognitive Engineering Laboratory, Department of Industrial Engineering, Toronto, Canada CEL, pp. 94–93. University of Toronto (1994)
Vicente, K.: Cognitive work analysis: Toward safe, productive, and healthy computer-based work. Lawrence Erlbaum (1999)
Zacharias, G.L., Miao, A.X., Illgen, C., Yara, J.M., Siouris, G.M.: Sample: Situation awareness model for pilot-in-the-loop evaluation. Final Report R 95192 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Mamessier, S., Dreyer, D., Oberhauser, M. (2014). Calibration of Online Situation Awareness Assessment Systems Using Virtual Reality. In: Duffy, V.G. (eds) Digital Human Modeling. Applications in Health, Safety, Ergonomics and Risk Management. DHM 2014. Lecture Notes in Computer Science, vol 8529. Springer, Cham. https://doi.org/10.1007/978-3-319-07725-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-07725-3_12
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07724-6
Online ISBN: 978-3-319-07725-3
eBook Packages: Computer ScienceComputer Science (R0)