Skip to main content
SpringerLink
Log in

The Evaluations of the Impact of the Pilot’s Visual Behaviours on the Landing Performance by Using Eye Tracking Technology

  • Conference paper
  • First Online:
Engineering Psychology and Cognitive Ergonomics (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14017))

Included in the following conference series:

  • 699 Accesses

Abstract

Introduction. Eye tracking technology can be used to characterise a pilot's visual behaviour as well as to further analyse the workload and status of the pilot, which is crucial for tracking and predicting pilot performance and enhancing flight safety. Research questions. This research aims to investigate and identify the visual-related factors that could affect the pilot's landing operation performance (depending on whether the landing was successful or not). Method. There are 23 participants who performed the task of landing in the Future system simulator (FSS) while wearing eye trackers. Their eye tracking parameters including proportion of fixation count on primary flight display (PFC on PFD), proportion of fixation count on out the window (PFC on OTW), percentage change in pupil diameter (PCPD) and blink count were trained for classification using XGBoost according to whether they landed successfully or not. Results & Discussion. The results demonstrated that eye-movement features can be used to classify and predict a pilot's landing performance with an accuracy of 77.02%. PCPD and PFC on PFD are more crucial for performance classification out of the four features. Conclusion. It is practical to classify and predict pilot performance using eye-tracking technologies. The high importance of PCPD and PFC on PFD indicates that there is a correlation between pilots’ workload and attention distribution and performance, which has important implications for future predictive and analytical research on performance. The prediction of performance using eye tracking suggests that pilot status monitoring has a useful application in flight deck design.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Nguyen, T., Lim, C.P., Nguyen, N.D., Gordon-Brown, L., Nahavandi, S.: A review of situation awareness assessment approaches in aviation environments. IEEE Syst. J. 13(3), 3590–3603 (2019). https://doi.org/10.1109/JSYST.2019.2918283

    Article  Google Scholar 

  2. Novak, A., Mrazova, M.: Research of physiological factors affecting pilot performance in flight simulation training device. Commun.-Sci. Lett. Univ. Zilina 17(3), 103–107 (2015). https://doi.org/10.26552/com.C.2015.3.103-107

    Article  Google Scholar 

  3. Lee, K.: Effects of flight factors on pilot performance, workload, and stress at final approach to landing phase of flight. University of Central Florida (2010). Accessed 20 Feb 2023. https://stars.library.ucf.edu/etd/1628

  4. Carroll, M., Dahlstrom, N.: Human computer interaction on the modern flight deck. Int. J. Hum.-Comput. Interact. 37(7), 585–587 (2021). https://doi.org/10.1080/10447318.2021.1890495

  5. Li, W.C., Zhang, J., Minh, T.L., Cao, J., Wang, L.: Visual scan patterns reflect to human-computer interactions on processing different types of messages in the flight deck. Int. J. Ind. Ergon. 72, 54–60 (2019). https://doi.org/10.1016/j.ergon.2019.04.003

  6. Stanton, N.A., Plant, K.L., Roberts, A.P., Allison, C.K.: Use of highways in the sky and a virtual pad for landing head up display symbology to enable improved helicopter pilots situation awareness and workload in degraded visual conditions. Ergonomics 62(2), 255–267 (2017). https://doi.org/10.1080/00140139.2017.1414301

    Article  Google Scholar 

  7. Hebbar, P.A., Pashilkar, A.A., Biswas, P.: Using eye tracking system for aircraft design – a flight simulator study. Aviation 26(1), 11–21 (2022). https://doi.org/10.3846/AVIATION.2022.16398

  8. Ryffel, C.P., Muehlethaler, C.M., Huber, S.M., Elfering, A.: Eye tracking as a debriefing tool in upset prevention and recovery training (UPRT) for general aviation pilots. Ergonomics 62(2), 319–329 (2019). https://doi.org/10.1080/00140139.2018.1501093

    Article  Google Scholar 

  9. Chen, S., Epps, J.: Using task-induced pupil diameter and blink rate to infer cognitive load. Hum. Comput. Interact. 29, 390–413 (2014). https://doi.org/10.1080/07370024.2014.892428

    Article  Google Scholar 

  10. Yang, L., Yu, R., Lin, X., Xie, Y., Ma, L.: Visual search tasks: measurement of dynamic visual lobe and relationship with display movement velocity. Ergonomics 61(2), 273–283 (2017). https://doi.org/10.1080/00140139.2017.1353138

  11. Yahoodik, S., Tahami, H., Unverricht, J., Yamani, Y., Handley, H., Thompson, D.: Blink rate as a measure of driver workload during simulated driving. In: Proceedings of the 2020 HFES 64th International Annual Meeting, vol. 64, no. 1, pp. 1825–1828 (2021). https://doi.org/10.1177/1071181320641439

  12. Li, W.C., Braithwaite, G., Wang, T., Yung, M., Kearney, P.: The benefits of integrated eye tracking with airborne image recorders in the flight deck: a rejected landing case study. Int. J. Ind. Ergon. 78, 102982 (2020). https://doi.org/10.1016/J.ERGON.2020.102982

    Article  Google Scholar 

  13. Neboshynsky, C.M.: Expertise on cognitive workloads and performance during navigation and target detection (2012). Accessed 19 Feb 2023. https://apps.dtic.mil/sti/citations/ADA561981

  14. Biswas, P., Jeevithashree, D.V.: Eye gaze controlled MFD for military aviation. In: International Conference on Intelligent User Interfaces, Proceedings IUI, pp. 79–89 (2018).https://doi.org/10.1145/3172944.3172973

  15. Causse, M., Lancelot, F., Maillant, J., Behrend, J., Cousy, M., Schneider, N.: Encoding decisions and expertise in the operator’s eyes: using eye-tracking as input for system adaptation. Int. J. Hum. Comput. Stud. 125, 55–65 (2019). https://doi.org/10.1016/J.IJHCS.2018.12.010

    Article  Google Scholar 

  16. Babu, M.D., JeevithaShree, D.V., Prabhakar, G., Saluja, K.P.S., Pashilkar, A., Biswas, P.: Estimating pilots’ cognitive load from ocular parameters through simulation and in-flight studies. J. Eye Mov. Res. 12(3), 1–16 (2019). https://doi.org/10.16910/JEMR.12.3.3

  17. Callaway, F., Rangel, A., Griffiths, T.L.: Fixation patterns in simple choice reflect optimal information sampling. PLoS Comput. Biol. 17(3), e1008863 (2021). https://doi.org/10.1371/JOURNAL.PCBI.1008863

    Article  Google Scholar 

  18. Li, W.C., Horn, A., Sun, Z., Zhang, J., Braithwaite, G.: Augmented visualization cues on primary flight display facilitating pilot’s monitoring performance. Int. J. Hum. Comput. Stud. 135, 102377 (2020). https://doi.org/10.1016/j.ijhcs.2019.102377

  19. Chen, S., Epps, J.: Automatic classification of eye activity for cognitive load measurement with emotion interference. Comput. Methods Programs Biomed. 110(2), 111–124 (2013). https://doi.org/10.1016/J.CMPB.2012.10.021

    Article  Google Scholar 

  20. Chen, S., Epps, J., Ruiz, N., Chen, F.: Eye activity as a measure of human mental effort in HCI. In: International Conference on Intelligent User Interfaces, Proceedings IUI, pp. 315–318 (2011).https://doi.org/10.1145/1943403.1943454

  21. Krejtz, K., Duchowski, A.T., Niedzielska, A., Biele, C., Krejtz, I.: Eye tracking cognitive load using pupil diameter and microsaccades with fixed gaze. PLoS ONE 13(9), e0203629 (2018). https://doi.org/10.1371/JOURNAL.PONE.0203629

    Article  Google Scholar 

  22. Kruger, J.L., Hefer, E., Matthew, G.: Measuring the impact of subtitles on cognitive load: eye tracking and dynamic audiovisual texts. In: ACM International Conference Proceeding Series, pp. 62–66 (2013).https://doi.org/10.1145/2509315.2509331

  23. Coyne, J.T., Foroughi, C., Sibley, C.: Pupil diameter and performance in a supervisory control task: a measure of effort or individual differences? In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp. 865–869 (2017). https://doi.org/10.1177/1541931213601689

  24. Alhanbali, S., Munro, K.J., Dawes, P., Carolan, P.J., Millman, R.E.: Dimensions of self-reported listening effort and fatigue on a digits-in-noise task, and association with baseline pupil size and performance accuracy. Int. J. Audiol. 60(10), 762–772 (2020). https://doi.org/10.1080/14992027.2020.1853262

    Article  Google Scholar 

  25. van den Brink, R.L., Murphy, P.R., Nieuwenhuis, S.: Pupil diameter tracks lapses of attention. PLoS ONE 11(10), e0165274 (2016). https://doi.org/10.1371/JOURNAL.PONE.0165274

    Article  Google Scholar 

  26. Appel, T., Scharinger, C., Gerjets, P., Kasneci, E.: Cross-subject workload classification using pupil-related measures. In: Eye Tracking Research and Applications Symposium (ETRA), vol. 18 (2018). https://doi.org/10.1145/3204493.3204531

  27. iF Design. Future Systems Simulator (FSS) (2021). https://ifdesign.com/en/winner-ranking/project/future-systems-simulator-fss/314432. Accessed 21 Nov 2022

  28. Korek, W.T., Li, W.C., Lu, L., Lone, M.: Investigating pilots’ operational behaviours while interacting with different types of inceptors. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13307 LNAI, pp. 314–325 (2022). https://doi.org/10.1007/978-3-031-06086-1_24/COVER

  29. Li, W.C., Wang, Y., Korek, W.T.: To be or not to be? assessment on using touchscreen as inceptor in flight operation. Transport. Res. Procedia 66(C), 117–124 (2022). https://doi.org/10.1016/J.TRPRO.2022.12.013

  30. Li, W.-C., Moore, P., Zhang, J., Lin, J., Kearney, P.: The impact of out-the-window size on air traffic controllers’ visual behaviours and response time on digital tower operations. Int. J. Hum. Comput. Stud. 166, 102880 (2022). https://doi.org/10.1016/J.IJHCS.2022.102880

    Article  Google Scholar 

  31. Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 13–17 August 2016, pp. 785–794 (2016). https://doi.org/10.1145/2939672.2939785

  32. Ziv, G.: Gaze behavior and visual attention: a review of eye tracking studies in aviation. Int. J. Aviat. Psychol. 26(3–4), 75–104 (2017). https://doi.org/10.1080/10508414.2017.1313096

    Article  Google Scholar 

  33. Pfleging, B., Fekety, D.K., Schmidt, A., Kun, A.L.: A model relating pupil diameter to mental workload and lighting conditions. In: Conference on Human Factors in Computing Systems - Proceedings, pp. 5776–5788 (2016). https://doi.org/10.1145/2858036.2858117

Download references

Acknowledgements

This research is co-financed by the European Union through the European Social Fund (grant number POWR.03.02.00-00-I029). The authors would like to thank the FSS Team in Cranfield, especially Mudassir Lone, for his generous support during the project’s development, and Rolls-Royce, particularly Peter Beecroft, for approving the research to be carried out in the Future Systems Simulator.

Author information

Authors and Affiliations

  1. Safety and Accident Investigation Centre, Cranfield University, Cranfield, UK

    Yifan Wang & Wen-Chin Li

  2. Centre for Life-Cycle Engineering and Management, Cranfield University, Cranfield, UK

    Lichao Yang & Yifan Zhao

  3. Dynamics, Simulation and Control Group, Cranfield University, Cranfield, UK

    Wojciech Tomasz Korek

  4. Faculty of Automatic Control, Electronics and Computer Science, Silesian University of Technology, Gliwice, Poland

    Wojciech Tomasz Korek

Authors
  1. Yifan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lichao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wojciech Tomasz Korek
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yifan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wen-Chin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Coventry University, Coventry, UK

    Don Harris

  2. Cranfield University, Cranfield, UK

    Wen-Chin Li

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, Y., Yang, L., Korek, W.T., Zhao, Y., Li, WC. (2023). The Evaluations of the Impact of the Pilot’s Visual Behaviours on the Landing Performance by Using Eye Tracking Technology. In: Harris, D., Li, WC. (eds) Engineering Psychology and Cognitive Ergonomics. HCII 2023. Lecture Notes in Computer Science(), vol 14017. Springer, Cham. https://doi.org/10.1007/978-3-031-35392-5_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35392-5_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35391-8

  • Online ISBN: 978-3-031-35392-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation