Assessing ICT risk through a Monte Carlo method

Environment Systems and Decisions volume 33, pages486–499(2013)Cite this article

Abstract

To assess and manage the risk due to an information and communication system before its deployment, data of interest can be produced by a Monte Carlo method. This paper presents Haruspex, a software tool that applies a Monte Carlo method to simulate intelligent and adaptive threat agents that reach predefined goals through plan with several attacks. The samples that Haruspex collects are used to compute statistics on the agent’s impacts and their plans as well as to select cost-effective countermeasures. We describe the rationale and the implementation of Haruspex, the inputs it requires and the simulation of how the agents select and implement their plans. After discussing the validation and the performance of the first version of Haruspex, we present a case study and the first set of experimental results.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

US$ 39.95

Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. 1.

    An ancient forecaster of Tuscany

  2. 2.

    The non-disclosure agreement we have signed does not allow us to provide further information on the names and the versions of these modules.

References

  1. Alberts C, Allen J, Stoddard R (2012) Risk-based measurement and analysis: application to software security. Tech. rep., Software Engineering Inst., CMU

  2. Ammann P, Wijesekera D, Kaushik S (2002) Scalable, graph-based network vulnerability analysis. In: proceedings of the 9th ACM conference on computer and communications security, CCS ’02. ACM, pp 217–224

  3. Arora A, Hall D, Piato C, Ramsey D, Telang R (2004) Measuring the risk-based value of it security solutions. IT Prof 6(6):35–42

    Article  Google Scholar 

  4. Baiardi F,F, T, F., C., L, G.: Gvscan: Scanning networks for global vulnerabilities. In: first international workshop on emerging cyberthreats and countermeasures (2013)

  5. Barnum S (2008) Common attack pattern enumeration and classification (capec) schema description. Cigital Inc. http://capec.mitre.org/documents/documentation/CAPEC_Schema_Description_v13

  6. Barreto AB, H., H., E., Y.: Developing a complex simulation environment for evaluating cyber attacks. In: the interservice/industry training, simulation and education conference (I/ITSEC) (2012)

  7. Bier VM, Oliveros S, Samuelson L (2007) Choosing What to protect: strategic defensive allocation against an unknown attacker. J Public Econ Theory 9:563–587

    Article  Google Scholar 

  8. Boddy M, Gohde J, Haigh T, Harp S (2005) Course of action generation for cyber security using classical planning. In: proceedings ICAPS 2005. AAAI Press, pp 12–21

  9. Bouissou M, Bon J (2003) A new formalism that combines advantages of fault-trees and Markov models: Boolean logic driven Markov processes. Reliab Eng Syst Saf 82(2):149–163

    Article  Google Scholar 

  10. Braynov S, Jadliwala M (2003) Representation and analysis of coordinated attacks. In: proceedings of the 2003 ACM workshop on formal methods in security engineering, FMSE ’03. ACM, New York, pp 43–51

  11. Brown T, Beyeler W, Barton D (2004) Assessing infrastructure interdependencies: the challenge of risk analysis for complex adaptive systems. Int. Journal of Critical Infrastructures 1(1):108–117

    Article  Google Scholar 

  12. Buede DM, Mahoney S, Ezell B, Lathrop J (2012) Using plural modeling for predicting decisions made by adaptive adversaries. Reliab Eng Syst Saf 108(0):77–89

    Article  Google Scholar 

  13. Casalicchio E, Galli E, Tucci S (2007) Federated agent-based modeling and simulation approach to study interdependencies in IT critical infrastructures. In: proceedings of the 11th IEEE International symposium on distributed simulation and real-time applications, DS-RT ’07. IEEE Computer Society, Washington, pp 182–189

  14. Chen Y, Cárdenas AA, Greenstadt R, Rubinstein BIP (eds.) (2011) 4th ACM workshop on security and artificial intelligence, Chicago, IL, USA, October 2011. ACM

  15. Cheung S, Lindqvist U, Fong M (2003) Modeling multistep cyber attacks for scenario recognition. In: DARPA information survivability conference and exposition, 2003. In: proceedings, vol.1. pp 284–292

  16. Clark K, Tyree S, Dawkins J, Hale J (2004) Qualitative and quantitative analytical techniques for network security assessment. In: information assurance workshop IEEE, 2004. pp 321–328

  17. Colbaugh R, Glass K (2012) Proactive defense for evolving cyber threats. Tech. rep., Sandia National Labs

  18. Conrad SH, LeClaire RJ, O’Reilly GP, Uzunalioglu H (2006) Critical national infrastructure reliability modeling and analysis. Bell Labs Tech J 11(3):57–71

    Article  Google Scholar 

  19. Cuppens F, Autrel F, Miege A, Benferhat S (2002) Correlation in an intrusion detection process. In: internet security communication workshop (SECI’02). pp 153–172

  20. Deb K (2005) Multi-objective optimization. In: Burke E, Kendall G (eds) Search methodologies. Springer, US, pp 273–316

    Google Scholar 

  21. Dong G, Li J (1999) Efficient mining of emerging patterns: discovering trends and differences. In: proceedings of the fifth ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’99. ACM, New York, pp 43–52

  22. Epstein S, Rauzy A (2005) Can we trust pra? Reliab Eng Syst Saf 88(3):195-205

    Article  Google Scholar 

  23. Florencio D, Herley C (2011) Sex, lies and cyber-crime survey. In: the tenth workshop on economics of information security

  24. Florencio D, Herley C (2011) Where do all the attacks go?. In: the tenth workshop on economics of information security

  25. Ghorbani A, Bagheri E, Onut, Zafarani R, Baghi H, Noye G (2006) Agent-based interdependencies modeling and simulation (AIMS). Tech. rep., technical rep. no. IAS-TR01-06, Intelligent and Adaptive Systems Research Group, Faculty of Computer Science, UNB

  26. Ghosh N, Ghosh S (2010) A planner-based approach to generate and analyze minimal attack graph. Appl Intell: pp 1–22

  27. Gordon LA, Loeb MP (2002) The economics of information security investment. ACM Trans. ACM Trans Inf Syst Secur. 5:438–457

    Article  Google Scholar 

  28. Gorodetski V, Kotenko I (2002) Attacks against computer network: formal grammar-based framework and simulation tool. In: recent advances in intrusion detection, lecture notes in computer science, vol. 2516. Springer, pp 219–238

  29. Gorodetski V, Kotenko I, Karsaev O (2003) Multiagent technologies for computer network security: attack simulation, intrusion detection and intrusion detection learning. Comput Syst Sci Eng 18(4):191–200

    Google Scholar 

  30. Haimes YY (2006) On the definition of vulnerabilities in measuring risks to infrastructures. Risk Anal 26(2):293–296

    Article  Google Scholar 

  31. Han J, Pei J, Yin Y (2000) Mining frequent patterns without candidate generation. In: proceedings of the 2000 ACM SIGMOD international conference on management of data, SIGMOD ’00. ACM, New York, pp 1–12

  32. Hausken K (2010) Defense and attack of complex and dependent systems. Reliab Eng Syst Saf 95(1):29–42

    Article  Google Scholar 

  33. Hausken K, Bier VM (2011) Defending against multiple different attackers. Eur J Oper Res 211:370–384

    Article  Google Scholar 

  34. Helbing D, Balietti S (2011) How to do agent based simulations in the future

  35. Herrmann A (2012) The quantitative estimation of it-related risk probabilities. Risk Anal

  36. Holm H, Sommestad T, Almroth J, Persson M (2006) A quantitative evaluation of vulnerability scanning. Inf Manag Comput Secur 19(4):231–247

    Google Scholar 

  37. Ingols K, Chu M, Lippmann R, Webster S, Boyer S (2009) Modeling modern network attacks and countermeasures using attack graphs. In: proceedings of the annual computer security applications conference. IEEE Computer Society, Washington, pp 117–126

  38. Jagatic TN, Johnson NA, Jakobsson M, Menczer F (2007) Social phishing. Commun ACM 50(10):94–100

    Article  Google Scholar 

  39. Jajodia S, Noel S (2010) Topological vulnerability analysis. In: Jajodia S, Liu P,Swarup V,Wang C (eds) Cyber situational awareness. Advances in information security, vol. 46. Springer, US, pp 139–154

  40. Jha S, Sheyner O, Wing J (2002) Two formal analyses of attack graphs. In: proceedings of the 15th computer security foundation workshop, pp 49–63

  41. Konak A, Coit DW, Smith AE (2006) Multi-objective optimization using genetic algorithms: a tutorial. Reliab Eng Syst Saf 91(9):992–1007

    Article  Google Scholar 

  42. Kotenko I (2003) Active vulnerability assessment of computer networks by simulation of complex remote attacks. In: proceedings of international conference on computer networks and mobile computing. p 40

  43. Lee W, Grosh D, Tillman F (1985) Fault tree analysis, methods, and applications- a review. IEEE transactions on reliability

  44. LeMay E, Unkenholz W, Parks D, Muehrcke C, Keefe K, Sanders W (2010) Adversary-driven state-based system security evaluation. In: proceedings of the 6th international workshop on security measurements and metrics, metriSec ’10. ACM, New York, pp 5:1–5:9

  45. LeMay E, Unkenholz W, Parks D, Muehrcke C, Keefe K, Sanders W (2011) Model-based security metrics using adversary view security evaluation (ADVISE). In: proceedings of the 8th international conference on quantitative evaluation of systems (QEST 2011)

  46. Levitin G, Ben-Haim H (2008) Importance of protections against intentional attacks. Reliab Eng Syst Saf 93(4):639–646

    Article  Google Scholar 

  47. Lippmann R, Ingols K, Scott C, Piwowarski K, Kratkiewicz K, Artz M, Cunningham R (2005) Evaluating and strengthening enterprise network security using attack graphs. Project report. Lincoln Laboratory, MIT IA-2

  48. Lippmann R, Ingols K, Scott C, Piwowarski K, Kratkiewicz K, Artz M, Cunningham R (2006) Validating and restoring defense in depth using attack graphs. In: proceedings of the 2006 IEEE conference on military communications. Piscataway, NJ, pp 981–990

  49. Macal CM, North M (2010) Tutorial on agent-based modelling and simulation. JJ Simul 4(3):151–162

    Article  Google Scholar 

  50. Marler R, Arora J (2004) Survey of multi-objective optimization methods for engineering. Struct Multidiscip Optim 26:369–395

    Article  Google Scholar 

  51. Mehta V, Bartzis C, Zhu H, Clarke E, Wing J (2006) Ranking attack graphs. In: Zamboni D, Kruegel C (eds) Recent advances in intrusion detection. Lecture notes in computer science, vol. 4219. Springer, Berlin, pp 127–144

  52. MITRE CWE—common weakness enumeration. Tech. rep. http://cww.mitre.org/

  53. NIST national vulnerability database. Tech. rep. http://nvd.nist.gov/

  54. Noel S, Jajodia S, Wang L, Singhal A (2010) Measuring security risk of networks using attack graphs. Int J Next-Gener Comput 1(1):135–147

    Google Scholar 

  55. Noel S, Robertson E, Jajodia S (2004) Correlating intrusion events and building attack scenarios through attack graph distances. In: proceedings of the 20th annual computer security applications conference. IEEE Computer Society, Washington, pp 350–359

  56. One A (1996) Smashing the stack for fun and profit. Phrack magazine 7(49):14–16

    Google Scholar 

  57. Ou X, Boyer WF, McQueen MA (2006) A scalable approach to attack graph generation. In: proceedings of the 13th ACM conference on computer and communications security, CCS ’06. ACM, New York, pp 336–345

  58. Poolsappasit N, Dewri R, Ray I (2012) Dynamic security risk management using Bayesian attack graphs. Dependable Secur Comput IEEE Trans 9(1):61–74. doi:10.1109/TDSC.2011.34

    Article  Google Scholar 

  59. Rios Insua D, Rios J, Banks D (2009) Adversarial risk analysis. J Am Stat Assoc 104(486):841–854. doi:10.1198/jasa.2009.0155

    Article  Google Scholar 

  60. Rob A (2010) A survey of agent based modelling and simulation tools. Technical report DL-TR-2010-07, science and technology facilities council

  61. Ryan J, Jefferson T (2003) The use, misuse and abuse of statistics in information security research. In: proceedings of the 2003 ASEM national conference, St Louis

  62. Sarraute C (2011) On exploit quality metrics—and how to use them for automated pentesting. In: proceedings of 8.8 computer security conference

  63. Sarraute C, Richarte G, Lucángeli Obes J (2011) An algorithm to find optimal attack paths in nondeterministic scenarios. In: proceedings of the 4th ACM workshop on security and artificial intelligence, AISec ’11. ACM, New York, pp 71–80

  64. Scarfone K, Mell P (2009) An analysis of cvss version 2 vulnerability scoring. In: empirical software eng and measurement, 2009, pp 516–525

  65. Sheyner O, Haines J, Jha S, Lippmann R, Wing JM (2002) Automated generation and analysis of attack graphs. In: proceedings of the 2002 IEEE symposium on security and privacy. Washington, pp 273

  66. Noel S, Wang L, Singhal A, Jajodia S (2010) Measuring security risk of networks using attack graphs. Int J Next Gener Comput 1(1)

  67. Sommestad T, Ekstedt M, Johnson P (2009) Cyber security risks assessment with Bayesian defense graphs and architectural models. In: system sciences, 2009. HICSS ’09. 42nd Hawaii international conference on, pp 1 –10

  68. Sood A, Enbody R (2012) Targeted cyber attacks—a superset of advanced persistent threats. Secur Priv IEEE (99):1

    Article  Google Scholar 

  69. Swiler L, Phillips C, Ellis D, Chakerian S (2001) Computer-attack graph generation tool. In: DARPA information survivability conference exposition II, 2001, vol. 2, pp 307–321

  70. Thonnard O, Bilge L, Gorman G, Kiernan S, Lee M (2012) Industrial espionage and targeted attacks: Understanding the characteristics of an escalating threat. Springer, Berlin, pp 64–85

    Google Scholar 

  71. Verizon (2012) data breach investigation report. Tech. rep. http://www.verizonbusiness.com/resources/reports/rp_data-breach-investigations-report-2012_en_xg.pdf

  72. Wang S, Zhang Z, Kadobayashi Y (2013) Exploring attack graph for cost-benefit security hardening: a probabilistic approach. Comput Secur 32(0):158–169

    Article  Google Scholar 

  73. Wang W, Daniels TE (2008) A graph based approach toward network forensics analysis. ACM Trans Inf Syst Secur 12 4:1–4:33

    Google Scholar 

  74. Wang Y, Yun X, Zhang Y, Jin S, Qiao Y (2012) Research of network vulnerability analysis based on attack capability transfer. In: computer and IT, 2012 IEEE 12th international conference on, pp 38 –44

  75. Zhang S, Song S (2011) A novel attack graph posterior inference model based on Bayesian network. J Inf Secur 2:8–27

    Article  Google Scholar 

Download references

Acknowledgments

We thank the referees for their suggestions that noticeably improved the paper. The design of Haruspex has been discussed in a long and fruitful cooperation with C. Telmon who also has been involved in the design of the first prototype. The first prototype has been developed by G. Piga in his graduation thesis. The assessment of the Università di Pisa ICT network has been implemented by R. Bertolotti with the support of the Centro Serra, Università di Pisa. This works has been supported by an IBM Shared University Research Grant.

Author information

Affiliations

  1. Dipartimento di Informatica, Università di Pisa, Pisa, Italy

    Fabrizio Baiardi

  2. Istituto di Informatica e Telematica, CNR, Pisa, Italy

    Daniele Sgandurra

Authors
  1. Fabrizio Baiardi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniele Sgandurra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniele Sgandurra.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Baiardi, F., Sgandurra, D. Assessing ICT risk through a Monte Carlo method. Environ Syst Decis 33, 486–499 (2013). https://doi.org/10.1007/s10669-013-9463-4

Download citation

Keywords

  • Risk assessment
  • ICT system
  • Monte Carlo simulation
  • Attack plans
  • Countermeasures