Abstract
The design of a human–computer interactive system can be unacceptable for a range of reasons. User performance concerns, for example the likelihood of user errors and time needed for a user to complete tasks, are important areas of consideration. For safety-critical systems it is vital that tools are available to support the analysis of such properties before expensive design commitment has been made. In this work, we give a unified formal verification framework for integrating two kinds of analysis: (1) predicting bounds for task-completion times via exhaustive state-space exploration, and (2) detecting user-error related design issues. The framework is based on a generic model of cognitively plausible behaviour that captures assumptions about cognitive behaviour decided through a process of interdisciplinary negotiation. Assumptions made in an analysis, including those relating to the performance consequences of users recovering from likely errors, are also investigated in this framework. We further present a novel way of exploring the consequences of cognitive mismatches, on both correctness and performance grounds. We illustrate our analysis approach with a realistic medical device scenario: programming an infusion pump. We explore an initial pump design and then two variations based on features found in real designs, illustrating how the approach identifies both timing and human error issues.
Similar content being viewed by others
References
Anderson JR, Lebiere C (1998) The atomic components of thought. Lawrence Erlbaum Associates, Mahwah
Altmann EM, Trafton JG (2002) Memory for goals: an activation-based model. Cogn Sci 26(1): 39–83
Bartlett F (1958) Thinking: an experimental and social study. Basic Books, New York
Byrne MD, Bovair S (1997) A working memory model of a common procedural error. Cogn Sci 21(1): 31–61
Beckert B, Beuster G (2006) A method for formalizing, analyzing, and verifying secure user interfaces. In: Liu Z, He J (eds) Formal methods and software engineering, vol 4260. Lecture notes in computer science. Springer, Berlin, pp 55–73
Bolton ML, Bass EJ (2010) Formally verifying human-automation interaction as part of a system model: limitations and tradeoffs. Innov Syst Softw Eng 6: 219–231
Bolton ML, Bass EJ, Siminiceanu RI (2012) Generating phenotypical erroneous human behavior to evaluate human–automation interaction using model checking. Int J Hum Comput Stud 70(11): 888–906
Butterworth RJ, Blandford AE, Duke DJ (2000) Demonstrating the cognitive plausibility of interactive systems. Form Asp Comput 12: 237–259
Bowman H, Faconti G (1999) Analysing cognitive behaviour using LOTOS and Mexitl. Form Asp Comput 11: 132–159
Barnard PJ, May J (1995) Interactions with advanced graphical interfaces and the deployment of latent human knowledge. In: Interactive systems: design, specification, and verification (DSV-IS’95). Springer, Berlin pp 15–49
Curzon P, Blandford AE (2001) Detecting multiple classes of user errors. In: Little R, Nigay L (eds) Proceedings of the 8th IFIP working conference on engineering for human–computer interaction (EHCI’01), vol 2254. Lecture notes in computer science. Springer, Berlin, pp 57–71
Chung PH, Byrne MD (2008) Cue effectiveness in mitigating postcompletion errors in a routine procedural task. Int J Hum Comput Stud 66(4): 217–232
Campos JC, Harrison MD (2011) Modelling and analysing the interactive behaviour of an infusion pump. In: Proceedings of the fourth international workshop on formal methods for interactive systems: FMIS 2011, vol 45. Electronic communications of the EASST
Card SK, Moran TP, Newell A (1980) The keystroke-level model for user performance time with interactive systems. Commun. ACM 23: 396–410
Card SK, Moran TP, Newell A (1983) The psychology of human–computer interaction. Lawrence Erlbaum Associates, London
Curzon P, Rukšėnas R, Blandford A (2007) An approach to formal verification of human–computer interaction. Form Asp Comput 19: 513–550
de Moura L, Owre S, Ruess H, Rushby J, Shankar N, Sorea M, Tiwari A (2004) SAL 2. In: Alur R, Peled DA (eds) Computer aided verification: CAV 2004, vol 3114. Lecture notes in computer science. Springer, Berlin, pp 496–500
Fields RE (2001) Analysis of erroneous actions in the design of critical systems. Technical Report YCST 20001/09, University of York, Department of Computer Science. D.Phil Thesis
Fields B, Wright P, Harrison M (1996) Time, tasks and errors. SIGCHI Bull 28: 53–56
Hudson SE, John BE, Knudsen K, Byrne MD (1999) A tool for creating predictive performance models from user interface demonstrations. In: UIST ’99: proceedings of the 12th annual ACM symposium on user interface software and technology. ACM Press, New York, pp 93–102
Hollnagel E (1993) Human reliability analysis: context and control. Academic Press, London
Hollnagel E (1993) The phenotype of erroneous actions. Int J Man Mach Stud 39(1): 1–32
Huang H, Rukšėnas R, Ament MGA, Curzon P, Cox AL, Blandford A, Brumby D (2011) Capturing the distinction between task and device errors in a formal model of user behaviour. In: Proceedings of the fourth international workshop on formal methods for interactive systems: FMIS 2011, vol 45. Electronic communications of the EASST
John BE, Kieras DE (1996) The GOMS family of user interface analysis techniques: comparison and contrast. ACM Trans Comput Hum Interact 3(4): 320–351
John BE, Kieras DE (1996) Using GOMS for user interface design and evaluation: which technique. ACM Trans Comput Hum Interact 3: 287–319
John BE, Prevas K, Salvucci DD, Koedinger K (2004) Predictive human performance modeling made easy. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’04, New York, NY, USA. ACM, New York, pp 455–462
Kim BG, Ayoub A, Sokolsky O, Lee I, Jones P, Zhang Y, Jetley R (2011) Safety-assured development of the GPCA infusion pump software. In: Proceedings of the ninth ACM international conference on Embedded software, EMSOFT ’11, New York, NY, USA. ACM, New York, pp 155–164
Kieras D, Polson PG (1999) An approach to the formal analysis of user complexity. Int J Hum Comput Stud 51(2): 405–434
Kieras DE, Wood SD, Meyer DE (1997) Predictive engineering models based on the EPIC architecture for a multimodal high-performance human–computer interaction task. ACM Trans Comput Hum Interact 4(3): 230–275
Lacaze X, Palanque P, Navarre D, Bastide R (2002) Performance evaluation as a tool for quantitative assessment of complexity of interactive systems. In: Forbrig P, Limbourg Q, Vanderdonckt J, Urban B (eds) Interactive systems: design, specification, and verification, vol 2545. Lecture notes in computer science. Springer, Berlin, pp 208–222
Newell A (1990) Unified theories of cognition. Harvard University Press, Cambridge
Osman A, Kornblum S, Meyer DE (1986) The point of no return in choice reaction time: controlled and ballistic stages of response preparation. J Exp Psychol Hum Percept Perform 12(3): 243–258
Rasmussen J (1983) Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE Trans Syst Man Cybern SMC- 13(3): 257–266
Rukšėnas R, Back J, Curzon P, Blandford A (2009) Verification-guided modelling of salience and cognitive load. Form Asp Comput 21: 541–569
Rukšėnas R, Curzon P, Back J, Blandford A (2007) Formal modelling of cognitive interpretation. In: Doherty G, Blandford A (eds) Interactive systems. Design, specification, and verification, vol 4323. Lecture notes in computer science. Springer, Berlin, pp 123–136
Rushby J (2001) Analyzing cockpit interfaces using formal methods. Electron Notes Theor Comput Sci 43: 1–14
Sankaranarayanan S, Homaei H, Lewis C (2011) Model-based dependability analysis of programmable drug infusion pumps. In: Fahrenberg U, Tripakis S (eds) Formal modeling and analysis of timed systems, vol 6919. Lecture notes in computer science. Springer, Berlin, pp 317–334
Salvucci DD, Lee FJ (2003) Simple cognitive modeling in a complex cognitive architecture. In: Proceedings of the SIGCHI conference on Human factors in computing systems, CHI ’03, New York, NY, USA. ACM, New York, pp 265–272
Thimbleby H (2002) Analysis and simulation of user interfaces. In: Waern Y, McDonald S, Cockton G (eds) Human computer interaction 2000, vol XIV. BCS conference on human–computer interaction. Springer, Berlin, pp 221–237
Author information
Authors and Affiliations
Corresponding author
Additional information
by D.A. Duce
Rights and permissions
About this article
Cite this article
Rukšėnas, R., Curzon, P., Blandford, A. et al. Combining human error verification and timing analysis: a case study on an infusion pump. Form Asp Comp 26, 1033–1076 (2014). https://doi.org/10.1007/s00165-013-0288-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00165-013-0288-1