Skip to main content
SpringerLink
Log in

A methodology to measure and monitor level of operational effectiveness of a CSOC

  • Regular Contribution
  • Published:
International Journal of Information Security Aims and scope Submit manuscript

Abstract

In a cybersecurity operations center (CSOC), under normal operating conditions in a day, sufficient numbers of analysts are available to analyze the amount of alert workload generated by intrusion detection systems (IDSs). For the purpose of this paper, this means that the cybersecurity analysts can fully investigate each and every alert that is generated by the IDSs in a reasonable amount of time. However, there are a number of disruptive factors that can adversely impact the normal operating conditions such as (1) higher alert generation rates from a few IDSs, (2) new alert patterns that decreases the throughput of the alert analysis process, and (3) analyst absenteeism. The impact of all the above factors is that the alerts wait for a long duration before being analyzed, which impacts the readiness of the CSOC. It is imperative that the readiness of the CSOC be quantified, which in this paper is defined as the level of operational effectiveness (LOE) of a CSOC. LOE can be quantified and monitored by knowing the exact deviation of the CSOC conditions from normal and how long it takes for the condition to return to normal. In this paper, we quantify LOE by defining a new metric called total time for alert investigation (TTA), which is the sum of the waiting time in the queue and the analyst investigation time of an alert after its arrival in the CSOC database. A dynamic TTA monitoring framework is developed in which a nominal average TTA per hour (avgTTA/hr) is established as the baseline for normal operating condition using individual TTA of alerts that were investigated in that hour. At the baseline value of avgTTA/hr, LOE is considered to be ideal. Also, an upper-bound (threshold) value for avgTTA/hr is established, below which the LOE is considered to be optimal. Several case studies illustrate the impact of the above disruptive factors on the dynamic behavior of avgTTA/hr, which provide useful insights about the current LOE of the system. Also, the effect of actions taken to return the CSOC to its normal operating condition is studied by varying both the amount and the time of action, which in turn impacts the dynamic behavior of avgTTA/hr. Results indicate that by using the insights learnt from measuring, monitoring, and controlling the dynamic behavior of avgTTA/hr, a manager can quantify and color-code the LOE of the CSOC. Furthermore, the above insights allow for a deeper understanding of acceptable downtime for the IDS, acceptable levels for absenteeism, and the recovery time and effort needed to return the CSOC to its ideal LOE.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. CIO, DON Cyber Crime Handbook, Dept. of Navy, Washington, DC (2008)

  2. Zimmerman, C.: The Strategies of a World-Class Cybersecurity Operations Center. The MITRE Corporation, McLean (2014)

    Google Scholar 

  3. Anderson, J.P.: Computer security threat monitoring and surveillance. Tech. rep., James P. Anderson Co., Fort Washington, PA (1980)

  4. Denning, D.E.: An intrusion-detection model. In: Proceedings of IEEE Symposium on Security and Privacy, pp. 118–131. Oakland, CA (1986)

  5. Denning, D.E.: An intrusion-detection model. IEEE Trans. Softw. Eng. 13(2), 222 (1987)

  6. Northcutt, S., Novak, J.: Network Intrusion Detection, 3rd edn. New Riders Publishing, Thousand Oaks (2002)

    Google Scholar 

  7. Di Pietro, R., Mancini, L.V. (eds.): Intrusion Detection Systems, Advances in Information Security, vol. 38. Springer, Berlin (2008)

    Google Scholar 

  8. Subrahmanian, V.S., Ovelgonne, M., Dumitras, T., Prakash, B.A.: The Global Cyber-Vulnerability Report. Springer, Switzerland (2015)

  9. Sommer, R., Paxson, V.: Outside the closed world: on using machine learning for network intrusion detection. In: Proceedings of IEEE Symposium on Security and Privacy, pp. 305–316 (2010)

  10. Barbara, D., Jajodia, S. (eds.): Application of Data Mining in Computer Security, Advances in Information Security, vol. 6. Springer, Berlin (2002)

    MATH  Google Scholar 

  11. Helm, J.E., AhmadBeygi, S., Van Oyen, M.P.: Design and analysis of hospital admission control for operational effectiveness. Prod. Oper. Manag. 20(3), 359 (2011)

    Article  Google Scholar 

  12. Chen, Z., King, W., Pearcey, R., Kerba, M., Mackillop, W.J.: The relationship between waiting time for radiotherapy and clinical outcomes: a systematic review of the literature. Radiother. Oncol. 87(1), 3 (2008)

    Article  Google Scholar 

  13. Guerriero, F., Guido, R.: Operational research in the management of the operating theatre: a survey. Health Care Manag. Sci. 14(1), 89 (2011)

    Article  Google Scholar 

  14. Vansteenwegen, P., Van Oudheusden, D.: Decreasing the passenger waiting time for an intercity rail network. Transp. Res. Part B: Methodol. 41(4), 478 (2007)

    Article  Google Scholar 

  15. Kelly, C.: A framework for improving operational effectiveness and cost efficiency in emergency planning and response. Disaster Prev. Manag. Int. J. 4(3), 25 (1995)

    Article  Google Scholar 

  16. Robbins, T.R., Medeiros, D.J., Dum, P.: Evaluating arrival rate uncertainty in call centers. In: Proceedings of the 2006 Winter Simulation Conference, pp. 2180–2187. IEEE (2006)

  17. Jack, E.P., Bedics, T.A., McCary, C.E.: Operational challenges in the call center industry: a case study and resource-based framework. Manag. Serv. Qual. Int. J. 16(5), 477 (2006)

    Article  Google Scholar 

  18. Tijms, H.: New and old results for the M/D/c queue. AEU Int. J. Electron. Commun. 60(2), 125 (2006)

    Article  Google Scholar 

  19. Marianov, V., Serra, D.: Location models for airline hubs behaving as M/D/c queues. Comput. Oper. Res. 30(7), 983 (2003)

    Article  MATH  Google Scholar 

  20. Julisch, K., Dacier, M.: Mining intrusion detection alarms for actionable knowledge. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 366–375 (2002)

  21. Goodall, J.R., Lutters, W.G., Komlodi, A.: I know my network: collaboration and expertise in intrusion detection. In: Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work, pp. 342–345 (2004)

  22. Ganesan, R., Jajodia, S., Cam, H.: Optimal scheduling of cybersecurity analyst for minimizing risk. ACM Trans. Intell. Syst. Technol. 8(4), 52:1–52:33 (2017). doi:10.1145/2914795

  23. Ganesan, R., Jajodia, S., Shah, A., Cam, H.: Dynamic scheduling of cybersecurity analysts for minimizing risk using reinforcement learning. ACM Trans. Intell. Syst. Technol. 8(1), 4:1 (2016). doi:10.1145/2882969

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Dr. Cliff Wang of the Army Research Office for the many discussions which served as the inspiration for this research.

Author information

Authors and Affiliations

  1. Center for Secure Information Systems, George Mason University, Mail Stop 5B5, Fairfax, VA, 22030-4422, USA

    Ankit Shah, Rajesh Ganesan & Sushil Jajodia

  2. Army Research Laboratory, 2800 Powder Mill Road, Adelphi, MD, 20783-1138, USA

    Hasan Cam

Authors
  1. Ankit Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rajesh Ganesan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sushil Jajodia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hasan Cam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sushil Jajodia.

Additional information

Shah, Ganesan, and Jajodia were partially supported by the Army Research Office under grants W911NF-13-1-0421 and W911NF-15-1-0576 and by the Office of Naval Research under grant N00014-15-1-2007.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shah, A., Ganesan, R., Jajodia, S. et al. A methodology to measure and monitor level of operational effectiveness of a CSOC. Int. J. Inf. Secur. 17, 121–134 (2018). https://doi.org/10.1007/s10207-017-0365-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10207-017-0365-1

Keywords

Search

Navigation