Skip to main content
SpringerLink
Log in

SARR: A Cybersecurity Metrics and Quantification Framework (Keynote)

  • Conference paper
  • First Online:
Science of Cyber Security (SciSec 2021)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 13005))

Included in the following conference series:

Abstract

Cybersecurity Metrics and Quantification is a fundamental but notoriously hard problem and is undoubtedly one of the pillars underlying the emerging Science of Cybersecurity. In this paper, we present an novel approach to addressing this problem by unifying Security, Agility, Resilience and Risk (SARR) metrics into a single framework. The SARR approach and the resulting framework are unique because: (i) it is driven by the assumptions that are made when modeling, designing, implementing, operating, and defending systems, which are broadly defined to include infrastructures and enterprise networks; and (ii) it embraces the uncertainty inherent to the cybersecurity domain. We will review the status quo by looking into existing metrics and quantification research through the SARR lens and discuss a range of open problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Charlton, J., Du, P., Cho, J., Xu, S.: Measuring relative accuracy of malware detectors in the absence of ground truth. In: Proceedings of IEEE MILCOM, pp. 450–455 (2018)

    Google Scholar 

  2. Charlton, J., Du, P., Xu, S.: A new method for inferring ground-truth labels. In: Proceedings of SciSec (2021)

    Google Scholar 

  3. Chen, H., Cho, J., Xu, S.: Quantifying the security effectiveness of firewalls and DMZs. In: Proceedings of HoTSoS 2018, pp. 9:1–9:11 (2018)

    Google Scholar 

  4. Chen, H., Cho, J., Xu, S.: Quantifying the security effectiveness of network diversity. In: Proceedings of HoTSoS 2018, p. 24:1 (2018)

    Google Scholar 

  5. Chen, Y., Huang, Z., Xu, S., Lai, Y.: Spatiotemporal patterns and predictability of cyberattacks. PLoS ONE 10(5), e0124472 (2015)

    Article  Google Scholar 

  6. Cheng, Y., Deng, J., Li, J., DeLoach, S., Singhal, A., Ou, X.: Metrics of security. In: Cyber Defense and Situational Awareness, pp. 263–295 (2014)

    Google Scholar 

  7. Cho, J., Hurley, P., Xu, S.: Metrics and measurement of trustworthy systems. In: Proceedings IEEE MILCOM (2016)

    Google Scholar 

  8. Cho, J., Xu, S., Hurley, P., Mackay, M., Benjamin, T., Beaumont, M.: STRAM: measuring the trustworthiness of computer-based systems. ACM Comput. Surv. 51(6), 128:1–128:47 (2019)

    Google Scholar 

  9. National Research Council: Review of the Department of Homeland Security’s Approach to Risk Analysis. The National Academies Press (2010)

    Google Scholar 

  10. INFOSEC Research Council. Hard problem list. http://www.infosec-research.org/docs_public/20051130-IRC-HPL-FINAL.pdf (2007)

  11. Da, G., Xu, M., Xu, S.: A new approach to modeling and analyzing security of networked systems. In: Proceedings HotSoS 2014, pp. 6:1–6:12 (2014)

    Google Scholar 

  12. Dai, W., Parker, P., Jin, H., Xu, S.: Enhancing data trustworthiness via assured digital signing. IEEE TDSC 9(6), 838–851 (2012)

    Google Scholar 

  13. Du, P., Sun, Z., Chen, H., Cho, J.H., Xu, S.: Statistical estimation of malware detection metrics in the absence of ground truth. IEEE T-IFS 13(12), 2965–2980 (2018)

    Google Scholar 

  14. Durumeric, Z., et al.: The matter of heartbleed. In: Proceedings IMC (2014)

    Google Scholar 

  15. Fang, Z., Xu, M., Xu, S., Hu, T.: A framework for predicting data breach risk: leveraging dependence to cope with sparsity. IEEE T-IFS 16, 2186–2201 (2021)

    Google Scholar 

  16. Goldreich, O.: The Foundations of Cryptography, vol. 1. Cambridge University Press (2001)

    Google Scholar 

  17. Haimes, Y.Y.: On the definition of resilience in systems. Risk Anal. 29(4), 498–501 (2009)

    Article  Google Scholar 

  18. Han, Y., Lu, W., Xu, S.: Characterizing the power of moving target defense via cyber epidemic dynamics. In: HotSoS, pp. 1–12 (2014)

    Google Scholar 

  19. Han, Y., Lu, W., Xu, S.: Preventive and reactive cyber defense dynamics with ergodic time-dependent parameters is globally attractive. IEEE TNSE, accepted for publication (2021)

    Google Scholar 

  20. Harrison, K., Xu, S.: Protecting cryptographic keys from memory disclosures. In: IEEE/IFIP DSN 2007, pp. 137–143 (2007)

    Google Scholar 

  21. Homer, J., et al.: Aggregating vulnerability metrics in enterprise networks using attack graphs. J. Comput. Secur. 21(4), 561–597 (2013)

    Article  MathSciNet  Google Scholar 

  22. Jensen, U.: Probabilistic risk analysis: foundations and methods. J. Am. Stat. Assoc. 97(459), 925 (2002)

    Article  Google Scholar 

  23. Kantchelian, A., et al.: Better malware ground truth: techniques for weighting anti-virus vendor labels. In: Proceedings AISec, pp. 45–56 (2015)

    Google Scholar 

  24. Li, D., Li, Q., Ye, Y., Xu, S.: SoK: arms race in adversarial malware detection. CoRR, abs/2005.11671 (2020)

    Google Scholar 

  25. Li, D., Li, Q., Ye, Y., Xu, S.: A framework for enhancing deep neural networks against adversarial malware. IEEE TNSE 8(1), 736–750 (2021)

    MathSciNet  Google Scholar 

  26. Li, X., Parker, P., Xu, S.: A stochastic model for quantitative security analyses of networked systems. IEEE TDSC 8(1), 28–43 (2011)

    Google Scholar 

  27. Lin, Z., Lu, W., Xu, S.: Unified preventive and reactive cyber defense dynamics is still globally convergent. IEEE/ACM ToN 27(3), 1098–1111 (2019)

    Article  Google Scholar 

  28. Lu, W., Xu, S., Yi, X.: Optimizing active cyber defense dynamics. In: Proceedings GameSec 2013, pp. 206–225 (2013)

    Google Scholar 

  29. Lynch, N.: Distributed Algorithms. Morgan Kaufmann (1996)

    Google Scholar 

  30. Mireles, J., Ficke, E., Cho, J., Hurley, P., Xu, S.: Metrics towards measuring cyber agility. IEEE T-IFS 14(12), 3217–3232 (2019)

    Google Scholar 

  31. Morales, J., Xu, S., Sandhu, R.: Analyzing malware detection efficiency with multiple anti-malware programs. In: Proceedings CyberSecurity (2012)

    Google Scholar 

  32. Nicol, D., et al.: The science of security 5 hard problems, August 2015. http://cps-vo.org/node/21590

  33. Noel, S., Jajodia, S.: A suite of metrics for network attack graph analytics. In: Network Security Metrics, pp. 141–176. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66505-4_7

    Chapter  Google Scholar 

  34. Park, J., Seager, T.P., Rao, P.S.C., Convertino, M., Linkov, I.: Integrating risk and resilience approaches to catastrophe management in engineering systems. Risk Anal. 33(3), 356–367 (2013)

    Article  Google Scholar 

  35. Pendleton, M., Garcia-Lebron, R., Cho, J., Xu, S.: A survey on systems security metrics. ACM Comput. Surv. 49(4), 62:1–62:35 (2016)

    Google Scholar 

  36. Pfleeger, S.L., Cunningham, R.K.: Why measuring security is hard. IEEE Secur. Priv. 8(4), 46–54 (2010)

    Article  Google Scholar 

  37. Ramos, A., Lazar, M., Filho, R.H., Rodrigues, J.J.P.C.: Model-based quantitative network security metrics: a survey. IEEE Commun. Surv. Tutor. 19(4), 2704–2734 (2017)

    Article  Google Scholar 

  38. National Science and Technology Council: Trustworthy cyberspace: strategic plan for the federal cybersecurity research and development program (2011). https://www.nitrd.gov/SUBCOMMITTEE/csia/Fed_Cybersecurity_RD_Strategic_Plan_2011.pdf

  39. Wang, L., Jajodia, S., Singhal, A.: Network Security Metrics. Network Security Metrics, Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66505-4

    Book  Google Scholar 

  40. Wang, L., Jajodia, S., Singhal, A., Cheng, P., Noel, S.: k-zero day safety: a network security metric for measuring the risk of unknown vulnerabilities. IEEE TDSC 11(1), 30–44 (2014)

    Google Scholar 

  41. Xu, L., et al.: KCRS: a blockchain-based key compromise resilient signature system. In: Proceedings BlockSys, pp. 226–239 (2019)

    Google Scholar 

  42. Xu, M., Da, G., Xu, S.: Cyber epidemic models with dependences. Internet Math. 11(1), 62–92 (2015)

    Article  MathSciNet  Google Scholar 

  43. Xu, M., Hua, L., Xu, S.: A vine copula model for predicting the effectiveness of cyber defense early-warning. Technometrics 59(4), 508–520 (2017)

    Article  MathSciNet  Google Scholar 

  44. Xu, M., Schweitzer, K.M., Bateman, R.M., Xu, S.: Modeling and predicting cyber hacking breaches. IEEE T-IFS 13(11), 2856–2871 (2018)

    Google Scholar 

  45. Xu, M., Xu, S.: An extended stochastic model for quantitative security analysis of networked systems. Internet Math. 8(3), 288–320 (2012)

    Article  MathSciNet  Google Scholar 

  46. Xu, S.: Emergent behavior in cybersecurity. In: Proceedings HotSoS, pp. 13:1–13:2 (2014)

    Google Scholar 

  47. Xu, S.: Cybersecurity dynamics: a foundation for the science of cybersecurity. In: Proactive and Dynamic Network Defense, pp. 1–31 (2019)

    Google Scholar 

  48. Xu, S.: The cybersecurity dynamics way of thinking and landscape (invited paper). In: ACM Workshop on Moving Target Defense (2020)

    Google Scholar 

  49. Xu, S., Lu, W., Xu, L.: Push- and pull-based epidemic spreading in networks: thresholds and deeper insights. ACM TAAS 7(3), 1–26 (2012)

    Article  Google Scholar 

  50. Xu, S., Lu, W., Xu, L., Zhan, Z.: Adaptive epidemic dynamics in networks: thresholds and control. ACM TAAS 8(4), 1–19 (2014)

    Article  Google Scholar 

  51. Xu, S., Lu, W., Zhan, Z.: A stochastic model of multivirus dynamics. IEEE Trans. Dependable Secure Comput. 9(1), 30–45 (2012)

    Article  Google Scholar 

  52. Xu, S., Yung, M.: Expecting the unexpected: towards robust credential infrastructure. In: Financial Crypto, pp. 201–221 (2009)

    Google Scholar 

  53. Xu, S.: Cybersecurity dynamics. In: Proceedings HotSoS 2014, pp. 14:1–14:2 (2014)

    Google Scholar 

  54. Shouhuai, X., Wenlian, L., Li, H.: A stochastic model of active cyber defense dynamics. Internet Math. 11(1), 23–61 (2015)

    Article  MathSciNet  Google Scholar 

  55. Xu, S., Trivedi, K.: Report of the 2019 SATC pi meeting break-out session on “cybersecurity metrics: Why is it so hard?” (2019)

    Google Scholar 

  56. Shouhuai, X., Yung, M., Wang, J.: Seeking foundations for the science of cyber security. Inf. Syst. Front. 23, 263–267 (2021)

    Article  Google Scholar 

  57. Zhan, Z., Xu, M., Xu, S.: Characterizing honeypot-captured cyber attacks: statistical framework and case study. IEEE T-IFS 8(11), 1775–1789 (2013)

    Google Scholar 

  58. Zhan, Z., Maochao, X., Shouhuai, X.: Predicting cyber attack rates with extreme values. IEEE T-IFS 10(8), 1666–1677 (2015)

    Google Scholar 

  59. Zhang, M., Wang, L., Jajodia, S., Singhal, A., Albanese, M.: Network diversity: a security metric for evaluating the resilience of networks against zero-day attacks. IEEE Trans. Inf. Forensics Secur. 11(5), 1071–1086 (2016)

    Article  Google Scholar 

  60. Zheng, R., Lu, W., Xu, S.: Active cyber defense dynamics exhibiting rich phenomena. In: Proceedings HotSoS (2015)

    Google Scholar 

  61. Zheng, R., Lu, W., Xu, S.: Preventive and reactive cyber defense dynamics is globally stable. IEEE TNSE 5(2), 156–170 (2018)

    MathSciNet  Google Scholar 

Download references

Acknowledgement

We thank Moti Yung for illuminating discussions and Eric Ficke for proofreading the paper. This work was supported in part by ARO Grant #W911NF-17-1-0566, NSF Grants #2115134 and #2122631 (#1814825), and by a Grant from the State of Colorado.

Author information

Authors and Affiliations

  1. Laboratory for Cybersecurity Dynamics Department of Computer Science, University of Colorado Colorado Springs, Colorado Springs, USA

    Shouhuai Xu

Authors
  1. Shouhuai Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shouhuai Xu .

Editor information

Editors and Affiliations

  1. Fudan University, Shanghai, China

    Wenlian Lu

  2. George Mason University, Fairfax, VA, USA

    Kun Sun

  3. Computer Science Department, Columbia University, New York, NY, USA

    Moti Yung

  4. Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China

    Feng Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xu, S. (2021). SARR: A Cybersecurity Metrics and Quantification Framework (Keynote). In: Lu, W., Sun, K., Yung, M., Liu, F. (eds) Science of Cyber Security. SciSec 2021. Lecture Notes in Computer Science(), vol 13005. Springer, Cham. https://doi.org/10.1007/978-3-030-89137-4_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-89137-4_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-89136-7

  • Online ISBN: 978-3-030-89137-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation