Improving the scalability of formal human–automation interaction verification analyses that use task-analytic models

  • Matthew L. Bolton
  • Xi Zheng
  • Kylie Molinaro
  • Adam Houser
  • Meng Li
Original Paper

Abstract

The enhanced operator function model with communications (EOFMCs) is a task-analytic modeling formalism used for including human behavior in formal models of larger systems. This allows the contribution of human behavior to the safety of the system to be evaluated with model checking. The previous method for translating the EOFMCs into model checker input language was conceptually straightforward, but extremely statespace inefficient. This limited the applications that could be formally verified using EOFMC. In this paper, we present an alternative approach for formally representing EOFMCs that substantially decreases the model’s statespace size and verification time. This paper motivates this effort, describes how the improvement was achieved, presents benchmarks demonstrating the improvements in statespace size and verification time, discusses the implications of these results, and outlines directions for future improvement.

Keywords

Model checking Task analytic models Formal methods Scalability 

References

  1. 1.
    Aït-Ameur Y, Baron M (2006) Formal and experimental validation approaches in HCI systems design based on a shared event B model. Int J Softw Tools Technol Transfer 8(6):547–563CrossRefGoogle Scholar
  2. 2.
    Aït-Ameur Y, Baron M, Girard P (2003) Formal validation of HCI user tasks. In: Proceedings of the international conference on software engineering research and practice. CSREA Press, Las Vegas, pp 732–738Google Scholar
  3. 3.
    Angluin D (1987) Learning regular sets from queries and counterexamples. Inf Comput 75(2):87–106MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Basnyat S, Palanque P, Schupp B, Wright P (2007) Formal socio-technical barrier modelling for safety-critical interactive systems design. Saf Sci 45(5):545–565CrossRefGoogle Scholar
  5. 5.
    Basnyat S, Palanque PA, Bernhaupt R, Poupart E (2008) Formal modelling of incidents and accidents as a means for enriching training material for satellite control operations. In: Proceedings of the Joint ESREL 2008 and 17th SRA-Europe Conference, Taylor and Francis Group, London, pp CD–ROMGoogle Scholar
  6. 6.
    Bass EJ, Bolton ML, Feigh K, Griffith D, Gunter E, Mansky W, Rushby J (2011) Toward a multi-method approach to formalizing human–automation interaction and human–human communications. In: Proceedings of the IEEE international conference on systems, man, and cybernetics. IEEE, Piscataway, pp 1817–1824Google Scholar
  7. 7.
    Bolton ML (2010) Using task analytic behavior modeling, erroneous human behavior generation, and formal methods to evaluate the role of human–automation interaction in system failure. PhD thesis, University of Virginia, CharlottesvilleGoogle Scholar
  8. 8.
    Bolton ML (2013) Automatic validation and failure diagnosis of human-device interfaces using task analytic models and model checking. Comput Math Organ Theory 19:288–312CrossRefGoogle Scholar
  9. 9.
    Bolton ML (2015) Model checking human–human communication protocols using task models and miscommunication generation. J Aerosp Inf Syst. doi:10.2514/1.I010276
  10. 10.
    Bolton ML, Bass EJ (2009a) Building a formal model of a human-interactive system: insights into the integration of formal methods and human factors engineering. In: Proceedings of the 1st NASA formal methods symposium. NASA Ames Research Center, Moffett Field, pp 6–15Google Scholar
  11. 11.
    Bolton ML, Bass EJ (2009b) A method for the formal verification of human interactive systems. In: Proceedings of the 53rd annual meeting of the human factors and ergonomics society. HFES, Santa Monica, pp 764–768Google Scholar
  12. 12.
    Bolton ML, Bass EJ (2010a) Formally verifying human–automation interaction as part of a system model: limitations and tradeoffs. Innov Syst Softw Eng NASA J 6(3):219–231CrossRefGoogle Scholar
  13. 13.
    Bolton ML, Bass EJ (2010) Using task analytic models to visualize model checker counterexamples. In: Proceedings of the 2010 IEEE international conference on systems, man, and cybernetics. IEEE, Piscataway, pp 2069–2074Google Scholar
  14. 14.
    Bolton ML, Bass EJ (2012) Using model checking to explore checklist-guided pilot behavior. Int J Aviat Psychol 22(4):343–366CrossRefGoogle Scholar
  15. 15.
    Bolton ML, Bass EJ (2013) Generating erroneous human behavior from strategic knowledge in task models and evaluating its impact on system safety with model checking. IEEE Trans Syst Man Cybern Syst 43(6):1314–1327Google Scholar
  16. 16.
    Bolton ML, Siminiceanu RI, Bass EJ (2011) A systematic approach to model checking human–automation interaction using task-analytic models. IEEE Trans Syst Man Cybern Part A 41(5):961–976CrossRefGoogle Scholar
  17. 17.
    Bolton ML, Bass EJ, Siminiceanu RI (2012) Using phenotypical erroneous human behavior generation to evaluate human–automation interaction using model checking. Int J Hum Comput Stud 70(11):888–906CrossRefGoogle Scholar
  18. 18.
    Bolton ML, Bass EJ, Siminiceanu RI (2013) Using formal verification to evaluate human–automation interaction in safety critical systems, a review. IEEE Trans Syst Man Cybern Syst 43(3):488–503CrossRefGoogle Scholar
  19. 19.
    Bolton ML, Jimenez N, van Paassen MM, Trujillo M (2014) Automatically generating specification properties from task models for the formal verification of human–automation interaction. IEEE Trans Hum Mach Syst 44(5):561–575CrossRefGoogle Scholar
  20. 20.
    Campos JC (2003) Using task knowledge to guide interactor specifications analysis. In: Proceedings of the 10th international workshop on interactive systems. Design, specification, and verification. Springer, Berlin, pp 171–186Google Scholar
  21. 21.
    Clarke EM, Grumberg O, Peled DA (1999) Model checking. MIT Press, CambridgeGoogle Scholar
  22. 22.
    De Moura L, Owre S, Shankar N (2003) The SAL language manual. Technical report CSL-01-01, Computer Science Laboratory, SRI International, Menlo ParkGoogle Scholar
  23. 23.
    Degani A (2004) Taming HAL: designing interfaces beyond 2001. Macmillan, New YorkGoogle Scholar
  24. 24.
    Degani A, Heymann M (2002) Formal verification of human–automation interaction. Hum Factors 44(1):28–43CrossRefMATHGoogle Scholar
  25. 25.
    Degani A, Kirlik A (1995) Modes in human–automation interaction: initial observations about a modeling approach. In: Proceedings of the IEEE international conference on systems, man and cybernetics, vol 4. IEEE, Piscataway, pp 3443–3450Google Scholar
  26. 26.
    Degani A, Heymann M, Shafto M (1999a) Formal aspects of procedures: the problem of sequential correctness. In: Proceedings of the 43rd annual meeting of the human factors and ergonomics society. HFES, Santa Monica, pp 1113–1117Google Scholar
  27. 27.
    Degani A, Shafto M, Kirlik A (1999b) Modes in human–machine systems: review, classification, and application. Int J Aviat Psychol 9(2):125–138CrossRefGoogle Scholar
  28. 28.
    Degani A, Gellatly A, Heymann M (2011) HMI aspects of automotive climate control systems. In: Proceeding of the IEEE international conference on systems, man, and cybernetics. IEEE, Piscataway, pp 1795–1800Google Scholar
  29. 29.
    Emerson EA (1990) Temporal and modal logic. In: van Leeuwen J, Meyer AR, Nivat M, Paterson M, Perrin D (eds) Handbook of theoretical computer science, chapter 16. MIT Press, Cambridge, pp 995–1072Google Scholar
  30. 30.
    Fields RE (2001) Analysis of erroneous actions in the design of critical systems. PhD thesis, University of York, YorkGoogle Scholar
  31. 31.
    Gunter EL, Yasmeen A, Gunter CA, Nguyen A (2009) Specifying and analyzing workflows for automated identification and data capture. In: Proceedings of the 42nd Hawaii international conference on system sciences. IEEE Computer Society, Los Alatimos, pp 1–11Google Scholar
  32. 32.
    Harel D (1987) Statecharts: a visual formalism for complex systems. Sci Comput Program 8(3):231–274MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    Hartson HR, Siochi AC, Hix D (1990) The UAN: a user-oriented representation for direct manipulation interface designs. ACM Trans Inf Syst 8(3):181–203CrossRefGoogle Scholar
  34. 34.
    Heymann M, Degani A (2007) Formal analysis and automatic generation of user interfaces: approach, methodology, and an algorithm. Hum Factors 49(2):311–330CrossRefGoogle Scholar
  35. 35.
    Heymann M, Degani A, Barshi I (2007) Generating procedures and recovery sequences: a formal approach. In: Proceedings of the 14th international symposium on aviation psychology. Wright State University, DaytonGoogle Scholar
  36. 36.
    John BE (2009) CogTool user guide. Carnegie Mellon University, PittsburghGoogle Scholar
  37. 37.
    Kirwan B, Ainsworth LK (1992) A guide to task analysis. Taylor and Francis, LondonCrossRefGoogle Scholar
  38. 38.
    Leveson NG, Turner CS (1993) An investigation of the therac-25 accidents. Computer 26(7):18–41CrossRefGoogle Scholar
  39. 39.
    Li M, Molinaro K, Bolton ML (2015) Learning formal human–machine interface designs from task analytic models. In: Proceedings of the HFES annual meeting. HFES, Santa Monica (in press)Google Scholar
  40. 40.
    Mitchell CM, Miller RA (1986) A discrete control model of operator function: a methodology for information display design. IEEE Trans Syst Man Cybern Part A Syst Hum 16(3):343–357CrossRefGoogle Scholar
  41. 41.
    Palanque PA, Bastide R, Senges V (1996) Validating interactive system design through the verification of formal task and system models. In: Proceedings of the IFIP TC2/WG2.7 working conference on engineering for human–computer interaction. Chapman and Hall, London, pp 189–212Google Scholar
  42. 42.
    Paternò F, Santoro C (2001) Integrating model checking and HCI tools to help designers verify user interface properties. In: Proceedings of the 7th international workshop on the design, specification, and verification of interactive systems. Springer, Berlin, pp 135–150Google Scholar
  43. 43.
    Paternò F, Mancini C, Meniconi S (1997) Concurtasktrees: a diagrammatic notation for specifying task models. In: Proceedings of the IFIP TC13 international conference on human–computer interaction. Chapman and Hall, London, pp 362–369Google Scholar
  44. 44.
    Paternò F, Santoro C, Tahmassebi S (1998) Formal model for cooperative tasks: concepts and an application for en-route air traffic control. In: Proceedings of the 5th international conference on the design, specification, and verification of interactive systems. Springer, Vienna, pp 71–86Google Scholar
  45. 45.
    Rushby J (2014) The versatile synchronous observer. In: Iida S, Meseguer J, Ogata K (eds) Specification, algebra, and software: essays dedicated to Kokichi Futatsugi. Springer, Berlin, pp 110–128Google Scholar
  46. 46.
    Wing JM (1990) A specifier’s introduction to formal methods. Computer 23(9):8, 10–22, 24CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2016

Authors and Affiliations

  • Matthew L. Bolton
    • 1
  • Xi Zheng
    • 1
  • Kylie Molinaro
    • 1
  • Adam Houser
    • 1
  • Meng Li
    • 1
  1. 1.Department of Industrial and Systems EngineeringState University of New York at BuffaloAmherstUSA

Personalised recommendations