Skip to main content

A descriptive study of Microsoft’s threat modeling technique

Requirements Engineering volume 20, pages 163–180 (2015)Cite this article

Abstract

Microsoft’s STRIDE is a popular threat modeling technique commonly used to discover the security weaknesses of a software system. In turn, discovered weaknesses are a major driver for incepting security requirements. Despite its successful adoption, to date no empirical study has been carried out to quantify the cost and effectiveness of STRIDE. The contribution of this paper is the evaluation of STRIDE via a descriptive study that involved 57 students in their last master year in computer science. The study addresses three research questions. First, it assesses how many valid threats per hour are produced on average. Second, it evaluates the correctness of the analysis results by looking at the average number of false positives, i.e., the incorrect threats. Finally, it determines the completeness of the analysis results by looking at the average number of false negatives, i.e., the overlooked threats.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Notes

  1. Of course, the extra assumptions made by the teams have been considered.

References

  1. Avgeriou P, Grundy J, Hall J, Lago, P, Mistrik I (eds) (2011) Relating software requirements and architectures. Springer, Berlin

  2. Berander P (2004) Using students as subjects in requirements prioritization. In: International symposium on empirical software engineering (ISESE)

  3. Carver J, Jaccheri L, Morasca S (2010) A checklist for integrating student empirical studies with research and teaching goals. Empir Softw Eng 15(1):35–59

    Google Scholar 

  4. Chandra P, Wohleber T, Feragamo J, Williams J (2007) CLASP v1.2: comprehensive, lightweight application security process. Tech. rep., OWASP

  5. Clements P, Kazman R, Klein M (2001) Evaluating software architectures: methods and case studies. Addison-Wesley, Reading

    Google Scholar 

  6. Deng M, Wuyts K, Scandariato R, Preneel B, Joosen W (2011) A privacy threat analysis framework. Requir Eng 16(1):3–32

    Article  Google Scholar 

  7. Dhillon D (2011) Developer-driven threat modeling: lessons learned in the trenches. IEEE Secur Priv 9(4):41–47

    Google Scholar 

  8. Diallo M, Romero-Mariona J, Sim SE, Alspaugh T, Richardson D (2006) A comparative evaluation of three approaches to specifying security requirements. In: Working conference on requirements engineering: foundation for software quality (REFSQ)

  9. Fabian B, Gürses S, Heisel M, Santen T, Schmidt H (2010) A comparison of security requirements engineering methods. Requir Eng 15(1):7–40

    Article  Google Scholar 

  10. Grimes D, Schulz K (2002) Descriptive studies: what they can and cannot do. Lancet 359:145–149

    Article  Google Scholar 

  11. Haley C, Laney R, Moffett J, Nuseibeh B (2008) Security requirements engineering: a framework for representation and analysis. IEEE Trans Softw Eng 34(1):133–153

    Article  Google Scholar 

  12. Hatebur D, Heisel M, Schmidt H (2007) A pattern system for security requirements engineering. In: International conference on availability, reliability and security (ARES)

  13. Hernan S, Lambert S, Ostwald T, Shostack A (2006) Uncover security design flaws using the STRIDE approach. MSDN Mag. http://msdn.microsoft.com/en-us/magazine/cc163519.aspx

  14. Heyman T, Yskout K, Scandariato R, Schmidt H, Yu Y (2011) The security twin peaks. In: International symposium on engineering secure software and systems (ESSoS)

  15. Hogganvik I, Stølen K (2005) On the comprehension of security risk scenarios. In: International workshop on program comprehension (IWPC)

  16. Hogganvik I, Stølen K (2006) A graphical approach to risk identification motivated by empirical investigations. In: International conference on model driven engineering languages and systems (MoDELS)

  17. Hogganvik I, Lund M, Stølen K (2009) Reducing the effort to comprehend risk models: textlabels are often preferred over graphical means. Risk Anal 51(5):916–932

    Google Scholar 

  18. Höst M, Regnell B, Wohlin C (2000) Using students as subjects—a comparative study of students and professionals in lead-time impact assessment. J Empir Softw Eng 5(3):201–214

    Google Scholar 

  19. Howard M, Lipner S (2006) The security development lifecycle. Microsoft Press, Redmond

    Google Scholar 

  20. Ingalsbe J, Kunimatsu L, Baeten T, Mead N (2008) Threat modeling: diving into the deep end. IEEE Softw 25(1):28–34

    Article  Google Scholar 

  21. Johnstone M (2010) Threat modelling with STRIDE and UML. In: Australian information security management conference

  22. Karpati P, Opdahl A, Sindre G (2011) Experimental comparison of misuse case maps with misuse cases and system architecture diagrams for eliciting security vulnerabilities and mitigations. In: International conference on availability, reliability and security (ARES)

  23. Karpati P, Sindre G, Matulevicius R (2012) Comparing misuse case and mal-activity diagrams for modelling social engineering attacks. Int J Secur Softw Eng 3(2):54–73

    Google Scholar 

  24. KU Leuven DigiNews project. http://goo.gl/M6xkF

  25. Massacci F, Mylopoulos J, Zannone N (2010) Security requirements engineering: the SI* modeling language and the secure tropos methodology. In: Ras ZW, Tsay LS (eds) Advances in intelligent information systems. Springer, New York, pp 147–174

  26. McGraw G (2006) Software security: building security in. Addison-Wesley, Reading

    Google Scholar 

  27. McGraw G, Migues S, West J (2013) Building security in maturity model (BSIMM-V). Tech. rep., Cigital

  28. Meland P, Tøndel I, Jensen J (2010) Idea: reusability of threat models—two approaches with an experimental evaluation. In: Engineering secure software and systems (ESSoS)

  29. Mellado D, Blanco C, Sanchez LE, Fernandez-Medina E (2010) A systematic review of security requirements engineering. Comput Stand Interface 32(4):153–165

    Article  Google Scholar 

  30. Mouratidis H, Giorgini P (2007) Secure Tropos: a security-oriented extension of the tropos methodology. Int J Softw Eng Knowl Eng 17(2):285–309

    Article  Google Scholar 

  31. Myagmar S, Lee A, Yurcik W (2005) Threat modeling as a basis for security requirements. In: Symposium on requirements engineering for information security (SREIS)

  32. Nuseibeh B (2001) Weaving together requirements and architectures. IEEE Comput 34(3):115–119

    Google Scholar 

  33. Opdahl AL, Sindre G (2009) Experimental comparison of attack trees and misuse cases for security threat identification. Inf Softw Technol 51(5):916–932

    Article  Google Scholar 

  34. OWASP Mobile security project: mobile threat model. https://www.owasp.org/index.php/OWASP_Mobile_Security_Project

  35. Paja E, Dalpiaz F, Poggianella M, Roberti P, Giorgini P (2012) STS-Tool: socio-technical security requirements through social commitments. In: International conference on requirements engineering (RE)

  36. Runeson P (2003) Using students as experiment subjects—an analysis on graduate and freshmen student data. In: International conference on empirical assessment in software engineering (EASE)

  37. Scandariato R, Wuyts K, Joosen W Experimental material. https://sites.google.com/site/descriptivestudy/

  38. Schaad A, Borozdin M (2012) TAM2: automated threat analysis. In: Annual ACM symposium on applied computing (SAC)

  39. Schneier B (1999) Attack trees. Dr. Dobb’s J 24(12):21–29

  40. Shostack A (2008) Experiences threat modeling at Microsoft. In: Workshop on modeling security (ModSec)

  41. Shostack A (2009) Getting started with the SDL threat modeling tool. MSDN Mag. http://msdn.microsoft.com/en-us/magazine/dd347831.aspx

  42. Sindre G, Opdahl AL (2005) Eliciting security requirements with misuse cases. Requir Eng 10(1):34–44

    Article  Google Scholar 

  43. Sindre G, Opdahl, AL (2002) Templates for misuse case description. In: Workshop on requirements engineering: foundations for software quality (REFSQ)

  44. Svahnberg M, Aurum A, Wohlin C (2008) Using students as subjects—an empirical evaluation. In: International symposium on empirical software engineering and measurement (ESEM)

  45. Tichy W (2000) Hints for reviewing empirical work in software engineering. Empir Softw Eng 5(4):309–312

    Google Scholar 

  46. Torr P (2005) Demystifying the threat-modeling process. IEEE Secur Priv 3(5):66–70

    Google Scholar 

  47. Van Lamsweerde A (2004) Elaborating security requirements by construction of intentional anti-models. In: International conference on software engineering (ICSE)

  48. Van Landuyt D, Gregoire J, Michiels S, Truyen E, Joosen W (2006) Architectural design of a digital publishing system. Tech. rep., KU Leuven

Download references

Acknowledgments

This research is partially funded by the Research Fund KU Leuven, and by the EU FP7 project NESSoS, with financial support from the Prevention of and Fight against Crime Programme of the European Union (B-CCENTRE).

Author information

Authors and Affiliations

  1. iMinds-DistriNet, KU Leuven, 3001, Leuven, Belgium

    Riccardo Scandariato, Kim Wuyts & Wouter Joosen

Authors
  1. Riccardo Scandariato
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kim Wuyts
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wouter Joosen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Riccardo Scandariato.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Scandariato, R., Wuyts, K. & Joosen, W. A descriptive study of Microsoft’s threat modeling technique. Requirements Eng 20, 163–180 (2015). https://doi.org/10.1007/s00766-013-0195-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00766-013-0195-2

Keywords

  • Secure software
  • Empirical study
  • Threat modeling
  • STRIDE
  • Anti-requirements