Cognition, Technology & Work

, Volume 20, Issue 1, pp 23–47 | Cite as

User and design requirements and production of evidence: using incident analysis data to (1) inform user scenarios and bow ties, and (2) generate user and design requirements

  • Joan CahillEmail author
  • Una Geary
  • Ewan Douglas
  • Simon Wilson
  • Michael Ferreira
  • Brian Gilbert
Original Article


This paper reports on an innovative human–machine interaction methodology adopted to assess the case, role and requirements for a new ground collision awareness technology. Specifically, this paper reports on the analysis of ground collision incident data and the subsequent advancement of user scenarios and bow-ties based on this data analysis, for the purpose of generating preliminary user and design requirements for this technology. In so doing, the requirements elicitation and validation methods used in this research are framed from an epistemological perspective. Accordingly, the particular methods adopted are presented and discussed in terms of concepts of evidence, bearing witness and the distinction between facts and values. As such, this paper promotes thinking about evidence-based design practices. Overall, this evidence-based approach aims to improve the development of scenarios and associated problem solving around technology cases, user requirements and user interface design features. The proposed method is useful in terms of bridging the gap from data analysis to design, and validating design decisions. In this regard, it is argued that the generation of user scenarios based on the analysis of incident data (i.e. data coding and statistical analysis), and the reframing of such scenarios in terms of bow-ties for the purpose of requirements/design envisionment, extends existing scenario-based design approaches. Although the use of bow-ties is not new, the advancement of bow-ties from data-driven scenarios is. Specifically, the bow-tie method was applied in a design context, to support problem solving around design decisions, as opposed to formal risk analysis.


Incident analysis Scenario-based design Bow-ties Flight safety Evidence-based design Data analysis Ground collisions 



The study was funded by Boeing.


  1. Baranzini, Christou MD (2010) Human factors data traceability and analysis in the European Community’s Major Accident Reporting System. Cognit Technol Work 12(1):1–12. CrossRefGoogle Scholar
  2. Bødker S, Burr J (2002) The design collaboratorium. A place for usability design. ACM Trans Comput Hum Interact 9(2):152–169CrossRefGoogle Scholar
  3. Cacciabue CP (2004) Guide to applying human factors methods: human error and accident management in safety critical systems. Springer, LondonCrossRefGoogle Scholar
  4. Cahill J (2014) Human factors research in the wing watch project. In: Proceedings of the preventing aircraft damage conference, DublinGoogle Scholar
  5. Cahill J, Butler W, Redmond P (2008) Identifying the human factors requirements and associated human computer interaction design concept, for a proposed collision avoidance system, for use by commercial pilots operating on the airport ramp and in taxiway areas. In: Proceedings of the IHCI conference, National University of Ireland, CorkGoogle Scholar
  6. Cahill J, Redmond P, Yous S, Butler W (2010) Human–computer action design (HCI) methods supporting the envisionment, design and evaluation of a collision avoidance system. Poster presented at IHCI 2010 conference, Dublin City University (DCU), DublinGoogle Scholar
  7. Cahill J, Redmond P, Yous S, Lacey G, Butler W (2012) The design of a collision avoidance system for use by pilots operating on the airport ramp and in taxiway areas. Cognit Technol Work 15(2):219–238CrossRefGoogle Scholar
  8. Carpignano A, Piccini M (1999) Cognitive theories and engineering approaches for safety assessment and design of automated systems: a case study of a power plant. Cognit Technol Work 1(1):47–61. CrossRefGoogle Scholar
  9. Carroll JM (ed) (1995) Scenario-based design: envisioning work and technology in system development. Wiley, New YorkGoogle Scholar
  10. Carroll JM (2000) Making use: scenario-based design of human–computer interactions. MIT Press, CambridgeCrossRefGoogle Scholar
  11. Civil Aviation Authority (2017) Where did bowtie come from? Downloaded from: 25 Oct 2017
  12. Constantine LL, Lockwood LAD (1999) Software for use: a practical guide to the models and methods of usage-centered design. Addison-Wesley, BostonGoogle Scholar
  13. Cooper A (2007) The inmates are running the asylum, 7th edn. SAMS Publishing, IndianapolisGoogle Scholar
  14. Cousins JB, Whitmore E, Shulha L (2013) Arguments for a common set of principles for collaborative inquiry in evaluation. Am J Eval 34(1):7–22CrossRefGoogle Scholar
  15. Dekker S (2002) The field guide to human error investigations. Cranfield University Press, BedfordGoogle Scholar
  16. Dodd J, Stern-Gillet S (1995) The is/ought gap, the fact/value distinction and the naturalistic fallacy. Dialogue 34(4):727–746CrossRefGoogle Scholar
  17. Dowell J, Long J (1998) Conception of the cognitive engineering design problem. Ergonomics 41(2):126–139CrossRefGoogle Scholar
  18. Endsley MR (1995) Toward a theory of situation awareness in dynamic systems. Hum Factors J Hum Factors Ergon Soc 37:32–64. CrossRefGoogle Scholar
  19. Evidence (2017) In Retrieved from Accessed 7 July 2017
  20. Fact (2017) In Retrieved from Accessed 7 July 2017
  21. Goodwin K (2011) Developing effective scenarios. Retrieved from Accessed 7 July 2017
  22. Hackos JA, Redish JC (1998) User and task analysis for interface design. Wiley, New YorkGoogle Scholar
  23. Hollnagel E (2004) Barrier analysis and accident prevention. Ashgate, AldershotGoogle Scholar
  24. Hollnagel E (2012) Functional resonance analysis method: modelling complex sociotechnical systems. Ashgate, AldershotGoogle Scholar
  25. Hollnagel E (2014) Safety-I and safety-II: the past and future of safety management. Ashgate, FarnhamGoogle Scholar
  26. Hollnagel E, Woods DD (1983) Cognitive systems engineering: new wine in new bottles. Int J Man Mach Stud 18:583–600CrossRefGoogle Scholar
  27. Hollnagel E, Wears RL, Braithwaite J (2015) From safety-I to safety-II: a white paper. The resilient health care net: published simultaneously by the University of Southern Denmark, University of Florida, USA, and Macquarie University, AustraliaGoogle Scholar
  28. Idoughi D, Seffah A, Kolski C (2012) Adding user experience into the interactive service design loop: a persona-based approach. Behav Inform Technol 31(3):287–303CrossRefGoogle Scholar
  29. International Civil Aviation Organisation (2010) Annex 13: aircraft accident and incident investigationGoogle Scholar
  30. John BE (2005) Evidence-based practice in human–computer interaction and evidence maps. ACM SIGSOFT Softw Eng Notes 30:1–5Google Scholar
  31. Johnston N (2003) Expertise, skills, habits and repertoires. Outline presented at a meeting of the aerospace research psychology group, DublinGoogle Scholar
  32. Kontogiannis T, Malakis S (2017) Cognitive engineering and safety organization in air traffic management. CRC Press, Boca RatonCrossRefGoogle Scholar
  33. Mayhew DJ (1999) The usability engineering lifecycle. Morgan Kaufmann Publishers, BurlingtonCrossRefGoogle Scholar
  34. McDonald N (2000) SCARF: safety courses for airport ramp functions, Trinity College, Dublin. 14 Sept 2001
  35. McDonald N (2002) The development and evaluation of SCARF—and what happened after. In: Proceedings of the managing risk: the flight-ground safety interface, human factors training and tools conference, MelbourneGoogle Scholar
  36. Niwa Y (2009) A proposal for a new accident analysis method and its application to a catastrophic railway accident in Japan. Cognit Technol Work 11(3):187–204. CrossRefGoogle Scholar
  37. Niwa Y, Takahashi M, Kitamura M (2001) The design of human–machine interface for accident support in nuclear power plants. Cognit Technol Work 3(3):161–176. CrossRefGoogle Scholar
  38. Norros L, Salo L (2009) Design of joint systems: a theoretical challenge for cognitive systems engineering. Cognit Technol Work 11(1):43–56. CrossRefGoogle Scholar
  39. O’Hare D (2006) Cognitive functions and performance shaping factors in aviation accidents and incidents. Int J Aviat Psychol 16(2):145–156CrossRefGoogle Scholar
  40. O’Sullivan RG (2012) Collaborative evaluation within a framework of stakeholder-oriented evaluation approaches. Eval Program Plan 35:518–522CrossRefGoogle Scholar
  41. Pawson R (2006) Evidence-based policy: a realist perspective. Sage, Newcastle upon TyneCrossRefGoogle Scholar
  42. Preece J, Rogers Y, Sharp H (2007) Interaction design: beyond human–computer interaction, 2nd edn. Wiley, West SussexGoogle Scholar
  43. Pruitt J, Grudin J (2003) Personas: practice and theory. In: Proceedings of the 2003 conference on designing for user experiencesGoogle Scholar
  44. Reason J (1990) Human error. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  45. Reason J (1995) A systems approach to organizational error. Ergonomics 38(8):1708–1721CrossRefGoogle Scholar
  46. Reason J (2000) Human error: models and management. BMJ 320:768–770CrossRefGoogle Scholar
  47. Rheinboldt P (2017) The bowtie techniques. Accessed 7 July 2017
  48. Rosson MB, Carroll JM (2001) Usability engineering: scenario-based development of human-computer interaction (interactive technologies) 1st edition. Morgan Kaufmann, Burlington, Massachusetts, United StatesGoogle Scholar
  49. Rosson MB, Carroll JM (2002) Scenario based design. In: Jacko J, Sears A (eds) The human–computer interaction handbook: fundamentals, evolving technologies and emerging applications. Lawrence Erlbaum Associates, Mahwah, pp 1032–1050Google Scholar
  50. Shappell SA, Wiegmann DA (1997) A human error approach to accident investigation: the taxonomy of unsafe operations. Int J Aviat Psychol 7(4):269–291. CrossRefGoogle Scholar
  51. Shappell S, Wiegmann D (2001) Applying reason: the human factors analysis and classification system. Hum Factors Aerospace Saf 1:59–86Google Scholar
  52. SKYbrary Aviation Safety (2017) Retrieved from Accessed 7 July 2017
  53. Ulrich R, Zimring C, Xuemei Z, DuBose J, Seo H, Choi Y, Quan Xiaobo, Joseph Anjali (2008) A review of the research literature on evidence-based healthcare design. HERD 1(3):61–125CrossRefGoogle Scholar
  54. Valent P (2013) Bearing witness. Retrieved from Accessed 7 July 2017
  55. Values (2017) In Retrieved from
  56. Weaver R, Despotou G, Kelly T, McDermid J (2005) Combining software evidence: arguments and assurance. In: Proceedings of the 2005 workshop on realising evidence-based software engineering (REBSE ‘05). ACM, New York, pp 1–7Google Scholar
  57. Webster L, Steinke C (2009) Evidence-based design: a new direction for health care. Design Quarterly, Winter 2009Google Scholar
  58. Weener E (2007) Ground accident prevention—the foundations answer. In: Proceedings of the EASS conference, AmsterdamGoogle Scholar
  59. Wiegmann DA, Shappell SA (2003) A human error approach to aviation accident analysis: the human factors analysis and classification system. Ashgate Publishing Ltd, BurlingtonGoogle Scholar
  60. Wilson PF, Dell LD, Anderson GF (1993) Root cause analysis: a tool for total quality management. ASQC Quality Press, MilwaukeeGoogle Scholar
  61. Yoon YS, Ham DH, Yoon WC (2017) A new approach to analysing human-related accidents by combined use of HFACS and activity theory-based method. Cognit Technol Work 19(4):759–783. CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of PsychologyTCDDublinIreland
  2. 2.School of Computer Science and StatisticsTCDDublinIreland
  3. 3.The Boeing CompanyChicagoUSA

Personalised recommendations