Skip to main content
SpringerLink
Log in

Maintaining the level of operational effectiveness of a CSOC under adverse conditions

  • regular contribution
  • Published:
International Journal of Information Security Aims and scope Submit manuscript

Abstract

The level of operational effectiveness (LOE) is a color-coded performance metric that is monitored by the cybersecurity operations center (CSOC). It is determined using the average time to analyze alerts (AvgTTA) in every hour of shift operation, where the time to analyze an alert (TTA) is the sum of waiting time in the queue and investigation time by the analysts. Ideally, the CSOC managers would set a predetermined baseline target for AvgTTA to be maintained for every hour of shift operation. However, due to adverse events, an imbalance may exist if the alert arrival rate far exceeds the service rate, resulting in high AvgTTA or low LOE. Upon exhausting all the analyst resources, the only option available to a CSOC manager is to discard alerts for restoring the LOE of the CSOC. The paper proposes two strategies: the value-based strategy is developed using a static optimization model while the reinforcement learning-based strategy is developed using a dynamic optimization model. The paper compares various strategies for discarding alerts and measures the following desiderata for comparing them: (1) minimize the number of alerts discarded, (2) ensure highest utilization of analysts, (3) determine the optimal time at which the alerts must be discarded in a shift, and (4) maintain the best possible LOE closest to the baseline target LOE. Results indicate that, overall, the RL strategy is the best performer among all strategies that guarantees the AvgTTA below the threshold value in every hour of shift operation while discarding the fewest number of alerts under adverse events.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. This observation is based on our numerous conversations with cybersecurity analysts and the CSOC managers.

References

  1. Aydin, M.E., Oztemel, E.: Dynamic job-shop scheduling using reinforcement learning agents. Robot. Auton. Syst. 33(2), 169–178 (2000)

    Article  Google Scholar 

  2. Bejtlich, R.: The Tao of Network Security Monitoring: Beyond Intrusion Detection. Pearson Education Inc., London (2005)

    Google Scholar 

  3. Bhatt, S., Manadhata, P.K., Zomlot, L.: The operational role of security information and event management systems. IEEE Secur. Privacy 12(5), 35–41 (2014)

    Article  Google Scholar 

  4. CIO: DON cyber crime handbook. Dept. of Navy, Washington, DC (2008)

  5. Crothers, T.: Implementing Intrusion Detection Systems. Wiley Publishing Inc., New York (2002)

    Google Scholar 

  6. D’Amico, A., Whitley, K.: The real work of computer network defense analysts. In: VizSEC 2007: Proceedings of the Workshop on Visualization for Computer Security (2008)

  7. D’Amico, A., Whitley, K.: The real work of computer network defense analysts: The analysis roles and processes that transform network data into security situation awareness. In: Proceedings of the Workshop on Visualization for Computer Security, pp. 19–37 (2008)

  8. Edwards, B., Hofmeyr, S., Forrest, S.: Hype and heavy tails: a closer look at data breaches. J. Cybersecur. 2(1), 3–14 (2016)

    Article  Google Scholar 

  9. Farris, K.A., Shah, A., Cybenko, G., Ganesan, R., Jajodia, S.: VULCON–a system for vulnerability prioritization, mitigation, and management. ACM Trans. Privacy Secur. 21(4), 16:2-16:28 (2018)

    Google Scholar 

  10. Ganesan, R., Jajodia, S., Cam, H.: Optimal scheduling of cybersecurity analyst for minimizing risk. ACM Trans. Intell. Syst. Technol. 8(4), 1–32 (2017)

    Article  Google Scholar 

  11. Ganesan, R., Jajodia, S., Shah, A., Cam, H.: Dynamic scheduling of cybersecurity analysts for minimizing risk using reinforcement learning. ACM Trans. Intell. Syst. Technol. 8(1), 1–21 (2016). https://doi.org/10.1145/2882969

    Article  Google Scholar 

  12. Gosavi, A.: Simulation Based Optimization: Parametric Optimization Techniques and Reinforcement Learning. Kluwer Academic, Norwell, MA (2003)

    Book  Google Scholar 

  13. Killcrece, G., Kossakowski, K.P., Ruefle, R., Zajicek, M.: State of the practice of computer security incident response teams (csirts). Tech. Rep. CMU/SEI-2003-TR-001, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA (2003)

  14. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)

  15. Paternina-Arboleda, C.D., Das, T.K.: A multi-agent reinforcement learning approach to obtaining dynamic control policies for stochastic lot scheduling problem. Simul. Modell. Pract. Theory 13(5), 389–406 (2005)

    Article  Google Scholar 

  16. Powell, W.B.: Approximate Dynamic Programming: Solving the Curses of Dimensionality. Wiley-Interscience, New York (2007)

    Book  Google Scholar 

  17. Rasoulifard, A., Bafghi, A.G., Kahani, M.: Incremental hybrid intrusion detection using ensemble of weak classifiers. In: Advances in Computer Science and Engineering, pp. 577–584. Springer, Cham (2008)

    Chapter  Google Scholar 

  18. Scarfone, K., Mell, P.: Guide to intrusion detection and prevention systems (IDPS). Special Publication 800-94, NIST (2007)

  19. Shah, A., Ganesan, R., Jajodia, S.: A methodology for ensuring fair allocation of CSOC effort for alert investigation. Int. J. Inf. Secur. (2018). https://doi.org/10.1007/s10207-018-0407-3

    Article  Google Scholar 

  20. Shah, A., Ganesan, R., Jajodia, S., Cam, H.: Adaptive reallocation of cybersecurity analysts to sensors for balancing risk between sensors. Serv. Orient. Comput. Appl. (2018). https://doi.org/10.1007/s11761-018-0235-3

    Article  Google Scholar 

  21. Shah, A., Ganesan, R., Jajodia, S., Cam, H.: Dynamic optimization of the level of operational effectiveness of a CSOC under adverse conditions. ACM Trans. Intell. Syst. Technol. 9(5), 51:1-51:20 (2018). https://doi.org/10.1145/3173457

    Article  Google Scholar 

  22. Shah, A., Ganesan, R., Jajodia, S., Cam, H.: A methodology to measure and monitor level of operational effectiveness of a CSOC. Int. J. Inf. Secur. 17(2), 121–134 (2018). https://doi.org/10.1007/s10207-017-0365-1

    Article  Google Scholar 

  23. Shah, A., Ganesan, R., Jajodia, S., Cam, H.: Optimal assignment of sensors to analysts in a cybersecurity operations center. IEEE Syst. J. (2018). https://doi.org/10.1109/JSYST.2018.2809506

    Article  Google Scholar 

  24. Shah, A., Ganesan, R., Jajodia, S., Cam, H.: Understanding tradeoffs between throughput, quality, and cost of alert analysis in a CSOC. IEEE Trans. Inf. Forens. Secur. 14(5), 1155–1170 (2019)

    Article  Google Scholar 

  25. Smith, M., Paté-Cornell, E.: Cyber risk analysis for a smart grid: How smart is smart enough? a multi-armed bandit approach. In: Proceedings of the 2nd Singapore Cyber-Security R&D Conference (SG-CRC 2017), pp. 37–56 (2017)

  26. Sundaramurthy, S.C., Bardas, A.G., Case, J., Ou, X., Wesch, M., McHugh, J., Rajagopalan, S.R.: A human capital model for mitigating security analyst burnout. In: Eleventh Symposium on Usable Privacy and Security (SOUPS 2015), pp. 347–359. USENIX Association (2015)

  27. Sundaramurthy, S.C., McHugh, J., Ou, X., Wesch, M., Bardas, A.G., Rajagopalan, S.R.: Turning contradictions into innovations or: How we learned to stop whining and improve security operations. In: Twelfth Symposium on Usable Privacy and Security (SOUPS 2016), pp. 237–250. USENIX Association (2016)

  28. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT press, Cambridge (2018)

    MATH  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Dr. Cliff Wang of the Army Research Office for the many discussions which served as the inspiration for this research.

Funding

This study was funded by the Army Research Office (grant number W911NF-13-1-0421), by the Office of Naval Research (grant numbers N00014-18-1-2670 and N00014-20-1-2407), and by the National Science Foundation under (grant number CNS-1822094).

Author information

Authors and Affiliations

  1. University of South Florida, Mail Stop ENG-030, Tampa, FL, 33620, USA

    Ankit Shah

  2. Center for Secure Information Systems, George Mason University, Mail Stop 5B5, Fairfax, VA, 22030-4422, USA

    Rajesh Ganesan & Sushil Jajodia

  3. Army Research Laboratory, 2800 Powder Mill Road, Adelphi, Maryland, 20783-1138, USA

    Hasan Cam

Authors
  1. Ankit Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rajesh Ganesan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sushil Jajodia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hasan Cam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sushil Jajodia.

Ethics declarations

Conflicts of interest

Authors Shah, Ganesan, Jajodia, and Cam declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shah, A., Ganesan, R., Jajodia, S. et al. Maintaining the level of operational effectiveness of a CSOC under adverse conditions. Int. J. Inf. Secur. 21, 637–651 (2022). https://doi.org/10.1007/s10207-021-00573-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10207-021-00573-4

Keywords

Search

Navigation