Computational and Mathematical Organization Theory

, Volume 19, Issue 3, pp 288–312

Automatic validation and failure diagnosis of human-device interfaces using task analytic models and model checking

SI: BRIMS 2011


When evaluating designs of human-device interfaces for safety critical systems, it is very important that they support the goal-directed tasks they were designed to facilitate. This paper describes a novel method that generates task-related temporal logic properties from task analytic models created early in the system design process. This allows analysts to use model checking (a means of performing exhaustive mathematical proofs) to automatically validate that formal models of human-device interfaces will let human operators successfully perform the necessary tasks with the system. This paper also presents an algorithm that uses the method to diagnose why a particular task is not valid for a given design. The application of both the method and algorithm are illustrated with a patient-controlled analgesia pump programming example. The method and algorithm are discussed and avenues for future work are described.


Formal methods Model checking Task analysis Temporal logic Validation Human-automation interaction 


  1. Abowd GD, Wang H, Monk AF (1995) A formal technique for automated dialogue development. In: Proceedings of the 1st conference on designing interactive systems. ACM Press, New York, pp 219–226 Google Scholar
  2. Aït-Ameur Y, Baron M (2006) Formal and experimental validation approaches in HCI systems design based on a shared event B model. Int J Softw Tools Technol Transf 8(6):547–563 CrossRefGoogle Scholar
  3. Aït-Ameur Y, Baron M, Girard P (2003) Formal validation of HCI user tasks. In: Proceedings of the international conference on software engineering research and practice. CSREA Press, Las Vegas, pp 732–738 Google Scholar
  4. Amant R, Freed A, Ritter F (2005) Specifying act-r models of user interaction with a goms language. Cogn Syst Res 6(1):71–88 CrossRefGoogle Scholar
  5. Basnyat S, Palanque P, Schupp B, Wright P (2007) Formal socio-technical barrier modelling for safety-critical interactive systems design. Saf Sci 45(5):545–565 CrossRefGoogle Scholar
  6. Basnyat S, Palanque PA, Bernhaupt R, Poupart E (2008) Formal modelling of incidents and accidents as a means for enriching training material for satellite control operations. In: Proceedings of the joint ESREL 2008 and 17th SRA-Europe conference. Taylor and Francis, London, CD–ROM Google Scholar
  7. Bass EJ, Bolton ML, Feigh K, Griffith D, Gunter E, Mansky W, Rushby J (2011) Toward a multi-method approach to formalizing human-automation interaction and human-human communications. In: Proceedings of the IEEE international conference on systems, man, and cybernetics. IEEE, Piscataway, pp 1817–1824 Google Scholar
  8. Basuki TA, Cerone A, Griesmayer A, Schlatte R (2009) Model-checking user behaviour using interacting components. Form Asp Comput 21(6):571–588 CrossRefGoogle Scholar
  9. Bolton ML (2010) Using task analytic behavior modeling, erroneous human behavior generation, and formal methods to evaluate the role of human-automation interaction in system failure. PhD thesis, University of Virginia, Charlottesville Google Scholar
  10. Bolton ML, Bass EJ (2009) A method for the formal verification of human interactive systems. In: Proceedings of the 53rd annual meeting of the human factors and ergonomics society. HFES, Santa Monica, pp 764–768 Google Scholar
  11. Bolton ML, Bass EJ (2010a) Formally verifying human-automation interaction as part of a system model: limitations and tradeoffs. Innov Syst Softw Eng 6(3):219–231 CrossRefGoogle Scholar
  12. Bolton ML, Bass EJ (2010b) Using task analytic models to visualize model checker counterexamples. In: Proceedings of the 2010 IEEE international conference on systems, man, and cybernetics. IEEE, Piscataway, pp 2069–2074 CrossRefGoogle Scholar
  13. Bolton ML, Bass EJ, Siminiceanu RI (2012) Using formal verification to evaluate human-automation interaction in safety critical systems, a review. IEEE Trans Syst Man Cybern, Part A, Syst Hum (accepted) Google Scholar
  14. Bolton ML, Siminiceanu RI, Bass EJ (2011) A systematic approach to model checking human-automation interaction using task-analytic models. IEEE Trans Syst Man Cybern, Part A, Syst Hum 41(5):961–976 CrossRefGoogle Scholar
  15. Booher H, Minninger J (2003) Human systems integration in army systems acquisition. In: Booher HR (ed) Handbook of human systems integration. Wiley, Hoboken, pp 663–698 CrossRefGoogle Scholar
  16. Bredereke J, Lankenau A (2005) Safety-relevant mode confusions–modelling and reducing them. Reliab Eng Syst Saf 88(3):229–245 CrossRefGoogle Scholar
  17. Burch JR, Clarke EM, Dill DL, Hwang J, McMillan KL (1992) Symbolic model checking: 1020 states and beyond. Inf Comput 98(2):142–171 CrossRefGoogle Scholar
  18. Byrne M, Kirlik A (2005) Using computational cognitive modeling to diagnose possible sources of aviation error. Int J Aviat Psychol 15(2):135–155 CrossRefGoogle Scholar
  19. Campos JC, Harrison M (1997) Formally verifying interactive systems: a review. In: Proceedings of the fourth international Eurographics workshop on the design, specification, and verification of interactive systems. Springer, Berlin, pp 109–124 Google Scholar
  20. Campos JC, Harrison MD (2008) Systematic analysis of control panel interfaces using formal tools. In: Proceedings of the 15th international workshop on the design, verification and specification of interactive systems. Springer, Berlin, pp 72–85 CrossRefGoogle Scholar
  21. Campos JC, Harrison MD (2009) Interaction engineering using the ivy tool. In: Proceedings of the 1st ACM SIGCHI symposium on engineering interactive computing systems. ACM Press, New York, pp 35–44 CrossRefGoogle Scholar
  22. Cerone A, PA Lindsay, Connelly S (2005) Formal analysis of human-computer interaction using model-checking. In: Proceedings of the 3rd IEEE international conference on software engineering and formal methods. IEEE Computer Society, Los Alamitos, pp 352–362 CrossRefGoogle Scholar
  23. Chu RW, Mitchell CM, Jones PM (1995) Using the operator function model and OFMspert as the basis for an intelligent tutoring system: towards a tutor/aid paradigm for operators of supervisory control systems. IEEE Trans Syst Man Cybern, Part A, Syst Hum 25(7):1054–1075 CrossRefGoogle Scholar
  24. Clarke E, Grumberg O, Jha S, Lu Y, Veith H (2003) Counterexample-guided abstraction refinement for symbolic model checking. J ACM 50(5):752–794 CrossRefGoogle Scholar
  25. Clarke EM, Enders R, Filkorn T, Jha S (1996) Exploiting symmetry in temporal logic model checking. Form Methods Syst Des 9(1):77–104 CrossRefGoogle Scholar
  26. Clarke EM, Grumberg O, Peled DA (1999) Model checking. MIT Press, Cambridge Google Scholar
  27. Cobleigh J, Giannakopoulou D, Păsăreanu C (2003) In: Proceedings of the 9th international conference on tools and algorithms for the construction and analysis of systems. Springer, Berlin, pp. 331–346 CrossRefGoogle Scholar
  28. De Moura L, Owre S, Shankar N (2003) The SAL language manual. Tech. Rep. CSL-01-01, Computer Science Laboratory, SRI International, Menlo Park Google Scholar
  29. Dwyer MB, Carr V, Hines L (1997) Model checking graphical user interfaces using abstractions. In: Proceedings of the sixth European software engineering conference. Springer, New York, pp 244–261 Google Scholar
  30. Dwyer MB, Tkachuk O, Robby, Visser W (2004) Analyzing interaction orderings with model checking. In: Proceedings of the 19th IEEE international conference on automated software engineering. IEEE Computer Society, Los Alamitos, pp 154–163 Google Scholar
  31. Emerson EA (1990) Temporal and modal logic. In: van Leeuwen J, Meyer AR, Nivat M, Paterson M, Perrin D (eds) Handbook of theoretical computer science. MIT Press, Cambridge, Chap 16, pp 995–1072 Google Scholar
  32. Feary M (2007) Automatic detection of interaction vulnerabilities in an executable specification. In: Proceedings of the 7th international conference on engineering psychology and cognitive ergonomics. Springer, Berlin, pp 487–496 CrossRefGoogle Scholar
  33. Fields RE (2001) Analysis of erroneous actions in the design of critical systems. PhD thesis, University of York, York Google Scholar
  34. Göknur S, Bolton ML, Bass EJ (2004) Adding a motor control component to the operator function model expert system to investigate air traffic management concepts using simulation. In: Proceedings of the IEEE international conference and systems, man, and cybernetics. IEEE, Piscataway, pp 886–892 Google Scholar
  35. Hamon G, De Moura L, Rushby J (2005) Automated test generation with SAL. Tech. rep., Menlo Park.
  36. Hartson HR, Siochi AC, Hix D (1990) The UAN: a user-oriented representation for direct manipulation interface designs. ACM Trans Inf Syst 8(3):181–203 CrossRefGoogle Scholar
  37. Holzmann G, Peled D (1994) An improvement in formal verification. In: Proceedings of the 7th international conference on formal description techniques. Chapman and Hall, London, pp 197–211 Google Scholar
  38. John BE, Kieras DE (1996) Using GOMS for user interface design and evaluation: which technique? ACM Trans Comput-Hum Interact 3(4):287–319 CrossRefGoogle Scholar
  39. Jonker CM, Schut MC, Treur J, Yolum P (2007) Analysis of meeting protocols by formalisation, simulation, and verification. Comput Math Organ Theory 13(3):283–314 CrossRefGoogle Scholar
  40. Joshi A, Miller SP, Heimdahl MP (2003) Mode confusion analysis of a flight guidance system using formal methods. In: Proceedings of the 22nd digital avionics systems conference. IEEE, Piscataway, pp 2.D.1-1–2.D.1-12 Google Scholar
  41. Kieras D (2003) Goms models for task analysis. Lawrence Erlbaum Associates, Mahwah, pp 83–116 Google Scholar
  42. Kirwan B, Ainsworth LK (1992) A guide to task analysis. Taylor and Francis, London Google Scholar
  43. Lecerof A, Paternò F (1998) Automatic support for usability evaluation. IEEE Trans Softw Eng 24(10):863–888 CrossRefGoogle Scholar
  44. Limbourg Q, Vanderdonckt J (2003) Comparing task models for user interface design. In: Diaper D, Stanton N (eds) The handbook of task analysis for human-computer interaction. Lawrence Erlbaum Associates, Mahwah, pp 135–154 Google Scholar
  45. Loer K, Harrison MD (2006) An integrated framework for the analysis of dependable interactive systems (IFADIS): its tool support and evaluation. Autom Softw Eng 13(4):469–496 CrossRefGoogle Scholar
  46. Mansouri-Samani M, Pasareanu CS, Penix JJ, Mehlitz PC, O’Malley O, Visser WC, Brat GP, Markosian LZ, Pressburger TT (2007) Program model checking: a practitioner’s guide. Tech. rep., Intelligent Systems Division, NASA Ames Research Center, Moffett Field Google Scholar
  47. Mitchell CM, Miller RA (1986) A discrete control model of operator function: a methodology for information display design. IEEE Trans Syst Man Cybern, Part A, Syst Hum 16(3):343–357 CrossRefGoogle Scholar
  48. Mueller S, Simpkins B, Anno G, Fallon C, Price O, McClellan G (2011) Adapting the task-taxon-task methodology to model the impact of chemical protective gear. Comput Math Organ Theory 17:251–271 CrossRefGoogle Scholar
  49. Palanque PA, Bastide R, Senges V (1996) Validating interactive system design through the verification of formal task and system models. In: Proceedings of the IFIP TC2/WG2.7 working conference on engineering for human-computer interaction. Chapman and Hall, London, pp 189–212 Google Scholar
  50. Parnas DL (1969) On the use of transition diagrams in the design of a user interface for an interactive computer system. In: Proceedings of the 24th national ACM conference. ACM Press, New York, pp 379–385 Google Scholar
  51. Paternò F (1997) Formal reasoning about dialogue properties with automatic support. Interact Comput 9(2):173–196 CrossRefGoogle Scholar
  52. Paternò F, Santoro C (2001) Integrating model checking and HCI tools to help designers verify user interface properties. In: Proceedings of the 7th international workshop on the design, specification, and verification of interactive systems. Springer, Berlin, pp 135–150 CrossRefGoogle Scholar
  53. Paternò F, Mancini C, Meniconi S (1997) Concurtasktrees: a diagrammatic notation for specifying task models. In: Proceedings of the IFIP TC13 international conference on human-computer interaction. Chapman and Hall, London, pp 362–369 Google Scholar
  54. Paternò F, Santoro C, Tahmassebi S (1998) Formal model for cooperative tasks: concepts and an application for en-route air traffic control. In: Proceedings of the 5th international conference on the design, specification, and verification of interactive systems. Springer, Vienna, pp 71–86 Google Scholar
  55. Pew R, Mavor A (2007) Human-system integration in the system development process: a new look. National Academies Press, Washington Google Scholar
  56. Ritter F, Kukreja U, Amant R (2007) Including a model of visual processing with a cognitive architecture to model a simple teleoperation task. J Cogn Eng Decis Mak 1(2):121 CrossRefGoogle Scholar
  57. Ritter FE, Van Rooy D, Amant RS, Simpson K (2006) Providing user models direct access to interfaces: an exploratory study of a simple interface with implications for HRI and HCI. IEEE Trans Syst Man Cybern, Part A, Syst Hum 36(3):592–601 CrossRefGoogle Scholar
  58. Rukšenas R, Back J, Curzon P, Blandford A (2009) Verification-guided modelling of salience and cognitive load. Form Asp Comput 21(6):541–569 CrossRefGoogle Scholar
  59. Rushby J (2002) Using model checking to help discover mode confusions and other automation surprises. Reliab Eng Syst Saf 75(2):167–177 CrossRefGoogle Scholar
  60. Shankar N (2000) Symbolic analysis of transition systems. In: Proceedings of the international workshop on abstract state machines, theory and applications. Springer, London, pp 287–302 CrossRefGoogle Scholar
  61. Wing JM (1990) A specifier’s introduction to formal methods. Computer 23(9):8, 10–22, 24 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.Department of Mechanical and Industrial EngineeringUniversity of Illinois at ChicagoChicagoUSA

Personalised recommendations