Cyber Defense and Situational Awareness pp 263-295

Part of the Advances in Information Security book series (ADIS, volume 62) | Cite as

Metrics of Security

  • Yi Cheng
  • Julia Deng
  • Jason Li
  • Scott A. DeLoach
  • Anoop Singhal
  • Xinming Ou
Chapter

Abstract

Discussion of challenges and ways of improving Cyber Situational Awareness dominated our previous chapters. However, we have not yet touched on how to quantify any improvement we might achieve. Indeed, to get an accurate assessment of network security and provide sufficient Cyber Situational Awareness (CSA), simple but meaningful metrics—the focus of the Metrics of Security chapter—are necessary. The adage, “what can’t be measured can’t be effectively managed,” applies here. Without good metrics and the corresponding evaluation methods, security analysts and network operators cannot accurately evaluate and measure the security status of their networks and the success of their operations. In particular, this chapter explores two distinct issues: (i) how to define and use metrics as quantitative characteristics to represent the security state of a network, and (ii) how to define and use metrics to measure CSA from a defender’s point of view.

References

  1. Alberts C., et al. (2005). Mission Assurance Analysis Protocol (MAAP): Assessing Risk in Complex Environments. CMU/SEI-2005-TN-032. Pittsburgh, PA: Carnegie Mellon University.Google Scholar
  2. Ammann P., et al. (2002). Scalable, Graph-based Network Vulnerability Analysis. the 9th ACM Conference on Computer and Communications Security. Google Scholar
  3. Bolstad C. and Cuevas H. (2010). Integrating Situation Awareness Assessment into Test and Evaluation. The International Test and Evaluation Association (ITEA), 31: 240–246.Google Scholar
  4. Cheung S., et al. (2003). Modeling Multi-Step Cyber Attacks for Scenario Recognition. the 3rd DARPA Information Survivability Conference and Exhibition. Washington D. C.Google Scholar
  5. Dahl, O. (2005). Using colored petri nets in penetration testing. Master’s thesis. Gjøvik, Norway: Gjøvik University College.Google Scholar
  6. Durso F., et al. (1995). Expertise and chess: A pilot study comparing situation awareness methodologies. In experimental analysis and measurement of situation awareness, (pp. 295–303).Google Scholar
  7. Endsley, M. R. (1988). Situation awareness global assessment technique (SAGAT). the National Aerospace and Electronics Conference (NAECON). Google Scholar
  8. Endsley, M. R. (1990). Predictive utility of an objective measure of situation awareness. the Human Factors Society 34th Annual Meeting, (pp. 41–45).Google Scholar
  9. Endsley, M. R. (1995). Measurement of situation awareness in dynamic systems. Human Factors, 37(1), 65–84.CrossRefGoogle Scholar
  10. Endsley, M. R., et al. (1998). A comparative evaluation of SAGAT and SART for evaluations of situation awareness. the Human Factors and Ergonomics Society Annual Meeting, (pp. 82–86).Google Scholar
  11. Fracker, M. (1991a). Measures of situation awareness: Review and future directions (Report No. AL-TR-1991-0128). Wright-Patterson Air Force Base, OH: Armstrong Laboratories.Google Scholar
  12. Fracker, M. (1991b). Measures of situation awareness: An experimental evaluation (Report No. AL-TR-1991-0127). Wright-Patterson Air Force Base, OH: Armstrong Laboratories.Google Scholar
  13. Gomez M., et al. (2008). An Ontology-Centric Approach to Sensor-Mission Assignment. Springer.Google Scholar
  14. Goodall J., et al. (2009). Camus: Automatically Mapping Cyber Assets to Missions and Users. IEEE Military Communications Conference. Boston MA.Google Scholar
  15. Grimaila M., et al. (2008). Improving the Cyber Incident Mission Impact Assessment Processes. the 4th Annual Workshop on Cyber Security and Information Intelligence Research. Google Scholar
  16. Grimaila M., et al. (2009). Design Considerations for a Cyber Incident Mission Impact Assessment (CIMIA) Process. the 2009 International Conference on Security and Management (SAM09). Las Vegas, Nevada.Google Scholar
  17. Harwood K., et al. (1988). Situational awareness: A conceptual and methodological framework. the 11th Biennial Psychology in the Department of Defense Symposium, (pp. pp. 23–27).Google Scholar
  18. Hecker, A. (2008). On System Security Metrics and the Definition Approaches. the 2nd International Conference on Emerging Security Information, Systems and Technologies. Google Scholar
  19. Heyman T., et al. (2008). Using security patterns to combine security metrics. the 3rd International Conference on Availability, Reliability and Security. Google Scholar
  20. Holsopple J., et al. (2008). FuSIA: Future Situation and Impact Awareness. Information Fusion.Google Scholar
  21. Jakobson G. (2011). Mission Cyber Security Situation Assessment Using Impact Dependency Graphs. the 14th International Conference on Information Fusion (FUSION) (pp. 1–8). Chicago, IL: IEEE.Google Scholar
  22. Jansen, W. (2009). Directions in Security Metrics Research. National Institute of Standards and Technology, Computer Security Division.Google Scholar
  23. Jones D. and Endsley M. R. (2000). Examining the validity of real-time probes as a metric of situation awareness. the 14th Triennial Congress of the International Ergonomics Association. Google Scholar
  24. Kotenko I., et al. (2006). Attack graph based evaluation of network security. the 10th IFIP TC-6 TC-11 international conference on Communications and Multimedia Security, (pp. 216–227).Google Scholar
  25. Lewis L., et al. (2008). Enabling Cyber Situation Awareness, Impact Assessment, and Situation Projection. Situation Management (SIMA).Google Scholar
  26. Lindstrom, P. (2005). Security: Measuring Up. Retrieved from http://searchsecurity.techtarget.com/tip/Security-Measuring-Up
  27. Manadhata P. and Wing J. (2011). An Attack Surface Metric. Software Engineering, IEEE Transactions on, vol. 37, no. 3, pp. 371–386.CrossRefGoogle Scholar
  28. Matthews M., et al. (2000). Measures of infantry situation awareness for a virtual MOUT environment. the Human Performance, Situation Awareness and Automation: User-Centered Design for the New Millennium. Google Scholar
  29. McDermott, J. (2000). Attack net penetration testing. Workshop on New Security Paradigms. Google Scholar
  30. Meland P. and Jensen J. (2008). Secure Software Design in Practice. the 3rd International Conference on Availability, Reliability and Security. Google Scholar
  31. Musman S., et al. (2010). Evaluating the Impact of Cyber Attacks on Missions. MITRE Technical Paper #09-4577.Google Scholar
  32. Natarajan A., et al. (2012). NSDMiner: Automated discovery of network service dependencies. INFOCOM (pp. 2507–2515). IEEE.Google Scholar
  33. Nebel B., et al. (1995). Reasoning about temporal relations: a maximal tractable subclass of Allen's interval algebra. Journal of the ACM (JACM), vol. 42, no. 1, pp. 43–66.CrossRefMATHMathSciNetGoogle Scholar
  34. Noel S., et al. (2004). Correlating Intrusion Events and Building Attack Scenarios through Attack Graph Distance. the 20th Annual Computer Security Conference. Tucson, Arizona.Google Scholar
  35. Ou X., et al. (2006). A Scalable Approach to Attack Graph Generation. the 13th ACM Conference on Computer and Communication Security (CCS), (pp. 336–345).Google Scholar
  36. Qin X. and Lee W. (2004). Attack Plan Recognition and prediction Using Causal Networks. the 20th Annual Computer Security Applications Conference. Google Scholar
  37. Salerno J., et al. (2005). A Situation Awareness Model Applied to Multiple Domains. Multisensor, Multisource Information Fusion.Google Scholar
  38. Salerno, J. (2008). Measuring situation assessment performance through the activities of interest score. the 11th International Conference on Information Fusion. Google Scholar
  39. Sheyner O., et al. (2002). Automated Generation and Analysis of Attack Graphs. the 2002 IEEE Symposium on Security and Privacy, (pp. 254–265).Google Scholar
  40. Singhal A., et al. (2010). Ontologies for modeling enterprise level security metrics. the 6th Annual Workshop on Cyber Security and Information Intelligence Research. ACM.Google Scholar
  41. Strater L., et al. (2001). Measures of platoon leader situation awareness in virtual decision making exercises (No. Research Report 1770). Army Research Institute.Google Scholar
  42. Tadda G., et al. (2006). Realizing Situation Awareness within a Cyber Environment. Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications (p. 1–8). Orlando: SPIE Vol.6242.Google Scholar
  43. Taylor, R. (1989). Situational awareness rating technique (SART): The development of a tool for aircrew systems design. the AGARD AMP Symposium on Situational Awareness in Aerospace Operations, CP478. Google Scholar
  44. Tu W., et. al. (2009). Automated Service Discovery for Enterprise Network Management. Stony Brook University. Retrieved May 8, 2014, from http://www.cs.sunysb.edu/~live3/research/asd_ppt.pdf
  45. Vidulich M. (2000). Testing the sensitivity of situation awareness metrics in interface evaluations. Situation awareness analysis and measurement, 227–246.Google Scholar
  46. Wang J., et al. (2009). Security Metrics for Software Systems. the 47th Annual Southeast Regional Conference. Google Scholar
  47. Watters J., et al. (2009). The Risk-to-Mission Assessment Process (RiskMAP): A Sensitivity Analysis and an Extension to Treat Confidentiality Issues. Google Scholar
  48. Zhou S., et al. (2003). Colored petri net based attack modeling. Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing: the 9th International Conference (pp. vol. 2639, pp. 715–718). Chongqing, China: Springer.Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Yi Cheng
    • 1
  • Julia Deng
    • 1
  • Jason Li
    • 1
  • Scott A. DeLoach
    • 2
  • Anoop Singhal
    • 3
  • Xinming Ou
    • 2
  1. 1.Intelligent Automation, IncRockvilleUSA
  2. 2.Kansas State UniversityManhattanUSA
  3. 3.National Institute of Standards and TechnologyGaithersburgUSA

Personalised recommendations