An Experimental Evaluation of Bow-Tie Analysis for Cybersecurity Requirements

  • Per Håkon MelandEmail author
  • Karin Bernsmed
  • Christian Frøystad
  • Jingyue Li
  • Guttorm Sindre
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11387)


Bow-tie analysis includes a graphical representation for depicting threats and consequences related to unwanted events, and shows how preventive and reactive barriers can provide control over such situations. This kind of analysis has traditionally been used to elicit requirements for safety and reliability engineering, but as a consequence of the ever-increasing coupling between the cyber and physical world, security has become an additional concern. Through a controlled experiment, we provide evidence that the expressiveness of the bow-tie notation is suitable for this purpose as well. Our results show that a sample population of graduate students, inexperienced in security modelling, perform similarly as security experts when we have a well-defined scope and familiar target system/situation. We also demonstrate that misuse case diagrams should be regarded as more of a complementary than competing modelling technique.


Bow-tie analysis Requirements elicitation Controlled experiment Digital exams 



The research leading to these results has partially been performed by the Cyber Security in Merchant Shipping (CySiMS) project, which received funding from the Research Council of Norway under Grant No. 256508. We would like to thank all participants in the experiment, as well as the group of NTNU students developing the bow-tie modelling tool that has supported our work greatly.

Supplementary material


  1. 1.
    ISO/IEC 27005 Information technology - Security techniques - Information security risk management. Technical report (2008).
  2. 2.
    Banerjee, A., Venkatasubramanian, K.K., Mukherjee, T., Gupta, S.K.S.: Ensuring safety, security, and sustainability of mission-critical cyber-physical systems. Proc. IEEE 100(1), 283–299 (2012)CrossRefGoogle Scholar
  3. 3.
    Bau, J., Mitchell, J.C.: Security modeling and analysis. IEEE Secur. Priv. 9(3), 18–25 (2011)CrossRefGoogle Scholar
  4. 4.
    Bernsmed, K., Frøystad, C., Meland, P.H., Nesheim, D.A., Rødseth, Ø.J.: Visualizing cyber security risks with bow-tie diagrams. In: Liu, P., Mauw, S., Stølen, K. (eds.) GraMSec 2017. LNCS, vol. 10744, pp. 38–56. Springer, Cham (2018). Scholar
  5. 5.
    Carver, J., Jaccheri, L., Morasca, S., Shull, F.: Issues in using students in empirical studies in software engineering education. In: 2003 Proceedings of the Ninth International Software Metrics Symposium, pp. 239–249. IEEE (2004)Google Scholar
  6. 6.
    Chen, Y., He, W.: Security risks and protection in online learning: a survey. Int. Rev. Res. Open Distrib. Learn. 14(5), 108–127 (2013)Google Scholar
  7. 7.
    Chockalingam, S., Hadziosmanovic, D., Pieters, W., Teixeira, A., van Gelder, P.: Integrated safety and security risk assessment methods: a survey of key characteristics and applications. arXiv preprint arXiv:1707.02140 (2017)
  8. 8.
    Falessi, D., et al.: Empirical software engineering experts on the use of students and professionals in experiments. Empirical Softw. Eng. 23(1), 452–489 (2018)CrossRefGoogle Scholar
  9. 9.
    Höst, M., Wohlin, C., Thelin, T.: Experimental context classification: incentives and experience of subjects. In: Proceedings of the 27th International Conference on Software Engineering, pp. 470–478. ACM (2005)Google Scholar
  10. 10.
    Jacobson, I.: Object-Oriented Software Engineering: A Use Case Driven Approach. Pearson Education India, Delhi (1993)Google Scholar
  11. 11.
    Johnson, C.: Using assurance cases and Boolean logic driven Markov processes to formalise cyber security concerns for safety-critical interaction with global navigation satellite systems. Electron. Commun. EASST 45, 1–18 (2011)Google Scholar
  12. 12.
    Khakzad, N., Khan, F., Amyotte, P.: Quantitative risk analysis of offshore drilling operations: a Bayesian approach. Saf. Sci. 57, 108–117 (2013)CrossRefGoogle Scholar
  13. 13.
    Kitchenham, B.A., et al.: Preliminary guidelines for empirical research in software engineering. IEEE Trans. Softw. Eng. 28(8), 721–734 (2002)CrossRefGoogle Scholar
  14. 14.
    Kordy, B., Mauw, S., Radomirović, S., Schweitzer, P.: Attack-defense trees. J. Log. Comput. 24(1), 55–87 (2014)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Kriaa, S., Pietre-Cambacedes, L., Bouissou, M., Halgand, Y.: A survey of approaches combining safety and security for industrial control systems. Reliab. Eng. Syst. Saf. 139, 156–178 (2015)CrossRefGoogle Scholar
  16. 16.
    Kumar, R., Stoelinga, M.: Quantitative security and safety analysis with attack-fault trees. In: 2017 IEEE 18th International Symposium on High Assurance Systems Engineering (HASE), pp. 25–32. IEEE (2017)Google Scholar
  17. 17.
    Lewis, S., Smith, K.: Lessons learned from real world application of the bow-tie method. In: 6th Global Congress on Process Safety. American Institute of Chemical Engineers (2010)Google Scholar
  18. 18.
  19. 19.
    Lu, L., Liang, W., Zhang, L., Zhang, H., Lu, Z., Shan, J.: A comprehensive risk evaluation method for natural gas pipelines by combining a risk matrix with a bow-tie model. J. Nat. Gas Sci. Eng. 25, 124–133 (2015)CrossRefGoogle Scholar
  20. 20.
    Maggi, F., Quarta, D., Pogliani, M., Polino, M., Zanchettin, A.M., Zanero, S.: Rogue robots: testing the limits of an industrial robot’s security. Technical report, Trend Micro, Politecnico di Milano (2017)Google Scholar
  21. 21.
    Marsh, S.: More university students are using tech to cheat in exams, April 2017.
  22. 22.
    Matulevicius, R., Mayer, N., Heymans, P.: Alignment of misuse cases with security risk management. In: 2008 Third International Conference on Availability, Reliability and Security, ARES 2008, pp. 1397–1404. IEEE (2008)Google Scholar
  23. 23.
    Mokhtari, K., Ren, J., Roberts, C., Wang, J.: Application of a generic bow-tie based risk analysis framework on risk management of sea ports and offshore terminals. J. Hazard. Mater. 192(2), 465–475 (2011)CrossRefGoogle Scholar
  24. 24.
    Moody, D.L.: The method evaluation model: a theoretical model for validating information systems design methods. In: ECIS 2003 Proceedings, p. 79 (2003)Google Scholar
  25. 25.
    Nolan, D.P.: Safety and Security Review for the Process Industries: Application of HAZOP, PHA, What-IF and SVA Reviews. Elsevier, Amsterdam (2014)Google Scholar
  26. 26.
    Pfleeger, S.L.: Design and analysis in software engineering: the language of case studies and formal experiments. SIGSOFT Softw. Eng. Notes 19(4), 16–20 (1994)CrossRefGoogle Scholar
  27. 27.
    Piètre-Cambacédès, L., Bouissou, M.: Cross-fertilization between safety and security engineering. Reliab. Eng. Syst. Saf. 110, 110–126 (2013)CrossRefGoogle Scholar
  28. 28.
    Raspotnig, C., Karpati, P., Katta, V.: A combined process for elicitation and analysis of safety and security requirements. In: Bider, I., et al. (eds.) BPMDS/EMMSAD -2012. LNBIP, vol. 113, pp. 347–361. Springer, Heidelberg (2012). Scholar
  29. 29.
    Røstad, L.: An extended misuse case notation: including vulnerabilities and the insider threat. Ph.D. thesis, Access Control in Healthcare Information Systems, pp. 66–77 (2008)Google Scholar
  30. 30.
    Runeson, P.: Using students as experiment subjects-an analysis on graduate and freshmen student data. In: Proceedings of the 7th International Conference on Empirical Assessment in Software Engineering, pp. 95–102. Citeseer (2003)Google Scholar
  31. 31.
    Salman, I., Misirli, A.T., Juristo, N.: Are students representatives of professionals in software engineering experiments? In: Proceedings of the 37th International Conference on Software Engineering, vol. 1, pp. 666–676. IEEE Press (2015)Google Scholar
  32. 32.
    Schmittner, C., Ma, Z., Smith, P.: FMVEA for safety and security analysis of intelligent and cooperative vehicles. In: Bondavalli, A., Ceccarelli, A., Ortmeier, F. (eds.) SAFECOMP 2014. LNCS, vol. 8696, pp. 282–288. Springer, Cham (2014). Scholar
  33. 33.
    Schneier, B.: Dr. Dobb’s J. Attack trees 24(12), 21–29 (1999)Google Scholar
  34. 34.
    Shostack, A.: Experiences threat modeling at microsoft. In: Modeling Security Workshop. Department of Computing, Lancaster University, UK (2008)Google Scholar
  35. 35.
    Shostack, A.: Threat Modeling: Designing for Security. Wiley, Hoboken (2014)Google Scholar
  36. 36.
    Sindre, G., Opdahl, A.L.: Eliciting security requirements with misuse cases. Requirements Eng. 10(1), 34–44 (2005)CrossRefGoogle Scholar
  37. 37.
    Sjoeberg, D.I.K., Hannay, J.E., Hansen, O., Kampenes, V.B., Karahasanovic, A., Liborg, N.K., Rekdal, A.C.: A survey of controlled experiments in software engineering. IEEE Trans. Softw. Eng. 31(9), 733–753 (2005)CrossRefGoogle Scholar
  38. 38.
    Svahnberg, M., Aurum, A., Wohlin, C.: Using students as subjects-an empirical evaluation. In: Proceedings of the Second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, pp. 288–290. ACM (2008)Google Scholar
  39. 39.
    Tichy, W.F.: Should computer scientists experiment more? Computer 31(5), 32–40 (1998)MathSciNetCrossRefGoogle Scholar
  40. 40.
    Trbojevic, V.M., Carr, B.J.: Risk based methodology for safety improvements in ports. J. Hazard. Mater. 71(1–3), 467–480 (2000)CrossRefGoogle Scholar
  41. 41.
    Winther, R., Johnsen, O.-A., Gran, B.A.: Security assessments of safety critical systems using HAZOPs. In: Voges, U. (ed.) SAFECOMP 2001. LNCS, vol. 2187, pp. 14–24. Springer, Heidelberg (2001). Scholar
  42. 42.
    World Maritime News: IMB: Shipping Next Playground for Hackers (2014).
  43. 43.
    Zalewski, J., Drager, S., McKeever, W., Kornecki, A.J.: Towards experimental assessment of security threats in protecting the critical infrastructure. In: Proceedings of the 7th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2012, Wroclaw, Poland (2012)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Per Håkon Meland
    • 1
    • 2
    Email author
  • Karin Bernsmed
    • 1
  • Christian Frøystad
    • 1
  • Jingyue Li
    • 2
  • Guttorm Sindre
    • 2
  1. 1.SINTEF DigitalTrondheimNorway
  2. 2.Norwegian University of Science and TechnologyTrondheimNorway

Personalised recommendations