Environment Systems and Decisions

, Volume 33, Issue 4, pp 486–499 | Cite as

Assessing ICT risk through a Monte Carlo method

  • Fabrizio Baiardi
  • Daniele SgandurraEmail author


To assess and manage the risk due to an information and communication system before its deployment, data of interest can be produced by a Monte Carlo method. This paper presents Haruspex, a software tool that applies a Monte Carlo method to simulate intelligent and adaptive threat agents that reach predefined goals through plan with several attacks. The samples that Haruspex collects are used to compute statistics on the agent’s impacts and their plans as well as to select cost-effective countermeasures. We describe the rationale and the implementation of Haruspex, the inputs it requires and the simulation of how the agents select and implement their plans. After discussing the validation and the performance of the first version of Haruspex, we present a case study and the first set of experimental results.


Risk assessment ICT system Monte Carlo simulation Attack plans Countermeasures 



We thank the referees for their suggestions that noticeably improved the paper. The design of Haruspex has been discussed in a long and fruitful cooperation with C. Telmon who also has been involved in the design of the first prototype. The first prototype has been developed by G. Piga in his graduation thesis. The assessment of the Università di Pisa ICT network has been implemented by R. Bertolotti with the support of the Centro Serra, Università di Pisa. This works has been supported by an IBM Shared University Research Grant.


  1. Alberts C, Allen J, Stoddard R (2012) Risk-based measurement and analysis: application to software security. Tech. rep., Software Engineering Inst., CMUGoogle Scholar
  2. Ammann P, Wijesekera D, Kaushik S (2002) Scalable, graph-based network vulnerability analysis. In: proceedings of the 9th ACM conference on computer and communications security, CCS ’02. ACM, pp 217–224Google Scholar
  3. Arora A, Hall D, Piato C, Ramsey D, Telang R (2004) Measuring the risk-based value of it security solutions. IT Prof 6(6):35–42CrossRefGoogle Scholar
  4. Baiardi F,F, T, F., C., L, G.: Gvscan: Scanning networks for global vulnerabilities. In: first international workshop on emerging cyberthreats and countermeasures (2013)Google Scholar
  5. Barnum S (2008) Common attack pattern enumeration and classification (capec) schema description. Cigital Inc.
  6. Barreto AB, H., H., E., Y.: Developing a complex simulation environment for evaluating cyber attacks. In: the interservice/industry training, simulation and education conference (I/ITSEC) (2012)Google Scholar
  7. Bier VM, Oliveros S, Samuelson L (2007) Choosing What to protect: strategic defensive allocation against an unknown attacker. J Public Econ Theory 9:563–587CrossRefGoogle Scholar
  8. Boddy M, Gohde J, Haigh T, Harp S (2005) Course of action generation for cyber security using classical planning. In: proceedings ICAPS 2005. AAAI Press, pp 12–21Google Scholar
  9. Bouissou M, Bon J (2003) A new formalism that combines advantages of fault-trees and Markov models: Boolean logic driven Markov processes. Reliab Eng Syst Saf 82(2):149–163CrossRefGoogle Scholar
  10. Braynov S, Jadliwala M (2003) Representation and analysis of coordinated attacks. In: proceedings of the 2003 ACM workshop on formal methods in security engineering, FMSE ’03. ACM, New York, pp 43–51Google Scholar
  11. Brown T, Beyeler W, Barton D (2004) Assessing infrastructure interdependencies: the challenge of risk analysis for complex adaptive systems. Int. Journal of Critical Infrastructures 1(1):108–117CrossRefGoogle Scholar
  12. Buede DM, Mahoney S, Ezell B, Lathrop J (2012) Using plural modeling for predicting decisions made by adaptive adversaries. Reliab Eng Syst Saf 108(0):77–89CrossRefGoogle Scholar
  13. Casalicchio E, Galli E, Tucci S (2007) Federated agent-based modeling and simulation approach to study interdependencies in IT critical infrastructures. In: proceedings of the 11th IEEE International symposium on distributed simulation and real-time applications, DS-RT ’07. IEEE Computer Society, Washington, pp 182–189Google Scholar
  14. Chen Y, Cárdenas AA, Greenstadt R, Rubinstein BIP (eds.) (2011) 4th ACM workshop on security and artificial intelligence, Chicago, IL, USA, October 2011. ACMGoogle Scholar
  15. Cheung S, Lindqvist U, Fong M (2003) Modeling multistep cyber attacks for scenario recognition. In: DARPA information survivability conference and exposition, 2003. In: proceedings, vol.1. pp 284–292Google Scholar
  16. Clark K, Tyree S, Dawkins J, Hale J (2004) Qualitative and quantitative analytical techniques for network security assessment. In: information assurance workshop IEEE, 2004. pp 321–328Google Scholar
  17. Colbaugh R, Glass K (2012) Proactive defense for evolving cyber threats. Tech. rep., Sandia National LabsGoogle Scholar
  18. Conrad SH, LeClaire RJ, O’Reilly GP, Uzunalioglu H (2006) Critical national infrastructure reliability modeling and analysis. Bell Labs Tech J 11(3):57–71CrossRefGoogle Scholar
  19. Cuppens F, Autrel F, Miege A, Benferhat S (2002) Correlation in an intrusion detection process. In: internet security communication workshop (SECI’02). pp 153–172Google Scholar
  20. Deb K (2005) Multi-objective optimization. In: Burke E, Kendall G (eds) Search methodologies. Springer, US, pp 273–316CrossRefGoogle Scholar
  21. Dong G, Li J (1999) Efficient mining of emerging patterns: discovering trends and differences. In: proceedings of the fifth ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’99. ACM, New York, pp 43–52Google Scholar
  22. Epstein S, Rauzy A (2005) Can we trust pra? Reliab Eng Syst Saf 88(3):195-205CrossRefGoogle Scholar
  23. Florencio D, Herley C (2011) Sex, lies and cyber-crime survey. In: the tenth workshop on economics of information securityGoogle Scholar
  24. Florencio D, Herley C (2011) Where do all the attacks go?. In: the tenth workshop on economics of information securityGoogle Scholar
  25. Ghorbani A, Bagheri E, Onut, Zafarani R, Baghi H, Noye G (2006) Agent-based interdependencies modeling and simulation (AIMS). Tech. rep., technical rep. no. IAS-TR01-06, Intelligent and Adaptive Systems Research Group, Faculty of Computer Science, UNBGoogle Scholar
  26. Ghosh N, Ghosh S (2010) A planner-based approach to generate and analyze minimal attack graph. Appl Intell: pp 1–22Google Scholar
  27. Gordon LA, Loeb MP (2002) The economics of information security investment. ACM Trans. ACM Trans Inf Syst Secur. 5:438–457CrossRefGoogle Scholar
  28. Gorodetski V, Kotenko I (2002) Attacks against computer network: formal grammar-based framework and simulation tool. In: recent advances in intrusion detection, lecture notes in computer science, vol. 2516. Springer, pp 219–238Google Scholar
  29. Gorodetski V, Kotenko I, Karsaev O (2003) Multiagent technologies for computer network security: attack simulation, intrusion detection and intrusion detection learning. Comput Syst Sci Eng 18(4):191–200Google Scholar
  30. Haimes YY (2006) On the definition of vulnerabilities in measuring risks to infrastructures. Risk Anal 26(2):293–296CrossRefGoogle Scholar
  31. Han J, Pei J, Yin Y (2000) Mining frequent patterns without candidate generation. In: proceedings of the 2000 ACM SIGMOD international conference on management of data, SIGMOD ’00. ACM, New York, pp 1–12Google Scholar
  32. Hausken K (2010) Defense and attack of complex and dependent systems. Reliab Eng Syst Saf 95(1):29–42CrossRefGoogle Scholar
  33. Hausken K, Bier VM (2011) Defending against multiple different attackers. Eur J Oper Res 211:370–384CrossRefGoogle Scholar
  34. Helbing D, Balietti S (2011) How to do agent based simulations in the futureGoogle Scholar
  35. Herrmann A (2012) The quantitative estimation of it-related risk probabilities. Risk AnalGoogle Scholar
  36. Holm H, Sommestad T, Almroth J, Persson M (2006) A quantitative evaluation of vulnerability scanning. Inf Manag Comput Secur 19(4):231–247Google Scholar
  37. Ingols K, Chu M, Lippmann R, Webster S, Boyer S (2009) Modeling modern network attacks and countermeasures using attack graphs. In: proceedings of the annual computer security applications conference. IEEE Computer Society, Washington, pp 117–126Google Scholar
  38. Jagatic TN, Johnson NA, Jakobsson M, Menczer F (2007) Social phishing. Commun ACM 50(10):94–100CrossRefGoogle Scholar
  39. Jajodia S, Noel S (2010) Topological vulnerability analysis. In: Jajodia S, Liu P,Swarup V,Wang C (eds) Cyber situational awareness. Advances in information security, vol. 46. Springer, US, pp 139–154Google Scholar
  40. Jha S, Sheyner O, Wing J (2002) Two formal analyses of attack graphs. In: proceedings of the 15th computer security foundation workshop, pp 49–63Google Scholar
  41. Konak A, Coit DW, Smith AE (2006) Multi-objective optimization using genetic algorithms: a tutorial. Reliab Eng Syst Saf 91(9):992–1007CrossRefGoogle Scholar
  42. Kotenko I (2003) Active vulnerability assessment of computer networks by simulation of complex remote attacks. In: proceedings of international conference on computer networks and mobile computing. p 40Google Scholar
  43. Lee W, Grosh D, Tillman F (1985) Fault tree analysis, methods, and applications- a review. IEEE transactions on reliabilityGoogle Scholar
  44. LeMay E, Unkenholz W, Parks D, Muehrcke C, Keefe K, Sanders W (2010) Adversary-driven state-based system security evaluation. In: proceedings of the 6th international workshop on security measurements and metrics, metriSec ’10. ACM, New York, pp 5:1–5:9Google Scholar
  45. LeMay E, Unkenholz W, Parks D, Muehrcke C, Keefe K, Sanders W (2011) Model-based security metrics using adversary view security evaluation (ADVISE). In: proceedings of the 8th international conference on quantitative evaluation of systems (QEST 2011)Google Scholar
  46. Levitin G, Ben-Haim H (2008) Importance of protections against intentional attacks. Reliab Eng Syst Saf 93(4):639–646CrossRefGoogle Scholar
  47. Lippmann R, Ingols K, Scott C, Piwowarski K, Kratkiewicz K, Artz M, Cunningham R (2005) Evaluating and strengthening enterprise network security using attack graphs. Project report. Lincoln Laboratory, MIT IA-2Google Scholar
  48. Lippmann R, Ingols K, Scott C, Piwowarski K, Kratkiewicz K, Artz M, Cunningham R (2006) Validating and restoring defense in depth using attack graphs. In: proceedings of the 2006 IEEE conference on military communications. Piscataway, NJ, pp 981–990Google Scholar
  49. Macal CM, North M (2010) Tutorial on agent-based modelling and simulation. JJ Simul 4(3):151–162CrossRefGoogle Scholar
  50. Marler R, Arora J (2004) Survey of multi-objective optimization methods for engineering. Struct Multidiscip Optim 26:369–395CrossRefGoogle Scholar
  51. Mehta V, Bartzis C, Zhu H, Clarke E, Wing J (2006) Ranking attack graphs. In: Zamboni D, Kruegel C (eds) Recent advances in intrusion detection. Lecture notes in computer science, vol. 4219. Springer, Berlin, pp 127–144Google Scholar
  52. MITRE CWE—common weakness enumeration. Tech. rep.
  53. NIST national vulnerability database. Tech. rep.
  54. Noel S, Jajodia S, Wang L, Singhal A (2010) Measuring security risk of networks using attack graphs. Int J Next-Gener Comput 1(1):135–147Google Scholar
  55. Noel S, Robertson E, Jajodia S (2004) Correlating intrusion events and building attack scenarios through attack graph distances. In: proceedings of the 20th annual computer security applications conference. IEEE Computer Society, Washington, pp 350–359Google Scholar
  56. One A (1996) Smashing the stack for fun and profit. Phrack magazine 7(49):14–16Google Scholar
  57. Ou X, Boyer WF, McQueen MA (2006) A scalable approach to attack graph generation. In: proceedings of the 13th ACM conference on computer and communications security, CCS ’06. ACM, New York, pp 336–345Google Scholar
  58. Poolsappasit N, Dewri R, Ray I (2012) Dynamic security risk management using Bayesian attack graphs. Dependable Secur Comput IEEE Trans 9(1):61–74. doi: 10.1109/TDSC.2011.34 CrossRefGoogle Scholar
  59. Rios Insua D, Rios J, Banks D (2009) Adversarial risk analysis. J Am Stat Assoc 104(486):841–854. doi: 10.1198/jasa.2009.0155 CrossRefGoogle Scholar
  60. Rob A (2010) A survey of agent based modelling and simulation tools. Technical report DL-TR-2010-07, science and technology facilities councilGoogle Scholar
  61. Ryan J, Jefferson T (2003) The use, misuse and abuse of statistics in information security research. In: proceedings of the 2003 ASEM national conference, St LouisGoogle Scholar
  62. Sarraute C (2011) On exploit quality metrics—and how to use them for automated pentesting. In: proceedings of 8.8 computer security conferenceGoogle Scholar
  63. Sarraute C, Richarte G, Lucángeli Obes J (2011) An algorithm to find optimal attack paths in nondeterministic scenarios. In: proceedings of the 4th ACM workshop on security and artificial intelligence, AISec ’11. ACM, New York, pp 71–80Google Scholar
  64. Scarfone K, Mell P (2009) An analysis of cvss version 2 vulnerability scoring. In: empirical software eng and measurement, 2009, pp 516–525Google Scholar
  65. Sheyner O, Haines J, Jha S, Lippmann R, Wing JM (2002) Automated generation and analysis of attack graphs. In: proceedings of the 2002 IEEE symposium on security and privacy. Washington, pp 273Google Scholar
  66. Noel S, Wang L, Singhal A, Jajodia S (2010) Measuring security risk of networks using attack graphs. Int J Next Gener Comput 1(1)Google Scholar
  67. Sommestad T, Ekstedt M, Johnson P (2009) Cyber security risks assessment with Bayesian defense graphs and architectural models. In: system sciences, 2009. HICSS ’09. 42nd Hawaii international conference on, pp 1 –10Google Scholar
  68. Sood A, Enbody R (2012) Targeted cyber attacks—a superset of advanced persistent threats. Secur Priv IEEE (99):1CrossRefGoogle Scholar
  69. Swiler L, Phillips C, Ellis D, Chakerian S (2001) Computer-attack graph generation tool. In: DARPA information survivability conference exposition II, 2001, vol. 2, pp 307–321Google Scholar
  70. Thonnard O, Bilge L, Gorman G, Kiernan S, Lee M (2012) Industrial espionage and targeted attacks: Understanding the characteristics of an escalating threat. Springer, Berlin, pp 64–85Google Scholar
  71. Wang S, Zhang Z, Kadobayashi Y (2013) Exploring attack graph for cost-benefit security hardening: a probabilistic approach. Comput Secur 32(0):158–169CrossRefGoogle Scholar
  72. Wang W, Daniels TE (2008) A graph based approach toward network forensics analysis. ACM Trans Inf Syst Secur 12 4:1–4:33Google Scholar
  73. Wang Y, Yun X, Zhang Y, Jin S, Qiao Y (2012) Research of network vulnerability analysis based on attack capability transfer. In: computer and IT, 2012 IEEE 12th international conference on, pp 38 –44Google Scholar
  74. Zhang S, Song S (2011) A novel attack graph posterior inference model based on Bayesian network. J Inf Secur 2:8–27CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Dipartimento di InformaticaUniversità di PisaPisaItaly
  2. 2.Istituto di Informatica e TelematicaCNRPisaItaly

Personalised recommendations