Studying Analysts’ Data Triage Operations in Cyber Defense Situational Analysis

  • Chen ZhongEmail author
  • John Yen
  • Peng Liu
  • Rob F. Erbacher
  • Christopher Garneau
  • Bo Chen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10030)


Cyber defense analysts are playing a critical role in Security Operations Centers (SOCs) to make sense of the immense amount of network monitoring data for detecting and responding to cyber attacks, including large-scale cyber attack campaigns involving advanced persistent threats. The network data continuously generated by multiple cyber defense systems, which may contain many false alerts, are overwhelming to the analysts. Analysts often need to make quick decisions/responses in a very short time based on their awareness of the situation at that moment. Data triage is the first and the most fundamental step performed routinely by the analysts — it filters a massive network monitoring data to identify known malicious events. Due to the high noise-to-signal ratio of network monitoring data, this steps accounts for a very significant portion of the time and attention of intrusion detection analysts. Therefore, a smart human-machine system that improves the performance of data triage operation in SOC is highly desirable. In this chapter, we describe a human-centered smart data triage system that leverages the cognitive trace of intrusion detection analysts. Our approach is based on a dynamic cyber-human system that integrates three dimensions: cyber defense analysts, network monitoring data, and attack activities. The approach leverages recorded analytic processes of intrusion detection analysts, which we refer to as “cognitive traces”. These traces of the analysts capture the examples of malicious events detected from the network monitoring data. Such traces from senior analysts provide a powerful opportunity for training junior analysts in performing data triage operations. To realize this potential, we also developed a smart retrieval framework that automatically retrieves traces of other senior analysts based on their similarity to the events already identified by a junior analyst. The traces from analysts, as demonstrated by a case study, also enable us to better understand their analytic processes in a systematic, yet minimum-reactive way. We summarize this chapter by discussing limitations of the proposed framework and the directions of future research regarding improving the data triage operations of cyber defense analysts.



This work was supported by ARO W911NF-09-1-0525 (MURI), ARO W911NF-15-1-0576, NSF CNS-1422594, and NIETP CAE Cybersecurity Grant (BAA-003-15).


  1. 1.
    Security Operations: Building a Successful SOC, Hewlett-Packard Development Company, (2013)
  2. 2.
    D’Amico, A., Whitley, K.: The real work of computer network defense analysts. In: Goodall, J.R., Conti, G., Ma, K.-L. (eds.) VizSEC 2007, pp. 19–37. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  3. 3.
    D’Amico, A., Whitley, K., Tesone, D., O’Brien, B., Roth, E.: Achieving cyber defense situational awareness: a cognitive task analysis of information assurance analysts. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 49, no. 3, pp. 229–233. SAGE Publications (2005)Google Scholar
  4. 4.
    Erbacher, R.F., Frincke, D.A., Wong, P.C., Moody, S., Fink, G.: A multi-phase network situational awareness cognitive task analysis. Inf. Vis. 9(3), 204–219 (2010)CrossRefGoogle Scholar
  5. 5.
    Granåsen, M., Dennis, A.: Measuring team effectiveness in cyber-defense exercises: a cross-disciplinary case study. Cogn. Technol. Work 18(1), 1–23 (2015)Google Scholar
  6. 6.
    Yen, J., Erbacher, R.F., Zhong, C., Liu, P.: Cognitive process. In: Kott, A., Wang, C., Erbacher, R.F. (eds.) Cyber Defense and Situational Awareness. AIS, vol. 62, pp. 119–144. Springer, Cham (2014). doi: 10.1007/978-3-319-11391-3_7 Google Scholar
  7. 7.
    Etoty, R.E., Erbacher, R.F.: A survey of visualization tools assessed for anomaly-based intrusion detection analysis. No. ARL-TR-6891. Army Research Lab Adelphi MD Computational and Information Sciences Directorate (2014)Google Scholar
  8. 8.
    Barford, P., et al.: Cyber SA: situational awareness for cyber defense. In: Jajodia, S., Liu, P., Swarup, V., Wang, C. (eds.) Cyber Situational Awareness, vol. 46, pp. 3–13. Springer, US (2010)CrossRefGoogle Scholar
  9. 9.
    Dutt, V., Ahn, Y.-S., Gonzalez, C.: Cyber situation awareness: modeling the security analyst in a cyber-attack scenario through instance-based learning. In: Li, Y. (ed.) DBSec 2011. LNCS, vol. 6818, pp. 280–292. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-22348-8_24 CrossRefGoogle Scholar
  10. 10.
    Endsley, M.R.: Toward a theory of situation awareness in dynamic systems. Hum. Factors J. Hum. Factors Ergon. Soc. 37(1), 32–64 (1995)CrossRefGoogle Scholar
  11. 11.
    Boyd, J.R.: The Essence of Winning and Losing (1996). Unpublished lecture notesGoogle Scholar
  12. 12.
    Pirolli, P., Card, S.: The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In: Proceedings of International Conference on Intelligence Analysis, vol. 5, pp. 2–4 (2005)Google Scholar
  13. 13.
    Bass, T.: Intrusion detection systems and multisensor data fusion. Commun. ACM 43(4), 99–105 (2000)CrossRefGoogle Scholar
  14. 14.
    Mahmood, T., Afzal, U.: Security analytics: Big Data analytics for cybersecurity: a review of trends, techniques and tools. In: 2nd National Conference on Information Assurance (NCIA), pp. 129–134. IEEE (2013)Google Scholar
  15. 15.
    Zuech, R., Khoshgoftaar, T.M., Wald, R.: Intrusion detection and big heterogeneous data: a survey. J. Big Data 2(1), 1–41 (2015)CrossRefGoogle Scholar
  16. 16.
    Biros, D.P., Eppich, T.: THEME: security-human element key to intrusion detection. Signal-Fairfax 55(12), 31–34 (2001)Google Scholar
  17. 17.
    Ericsson, K.A., Lehmann, A.C.: Expert and exceptional performance: evidence of maximal adaptation to task constraints. Annu. Rev. Psychol. 47(1), 273–305 (1996)CrossRefGoogle Scholar
  18. 18.
    Chen, P.C., Liu, P., Yen, J., Mullen, T.: Experience-based cyber situation recognition using relaxable logic patterns. In: IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), pp. 243–250. IEEE (2012)Google Scholar
  19. 19.
    Grance, T., Kent, K., Kim, B.: Computer security incident handling guide. NIST Spec. Publ. 800, 61 (2004)Google Scholar
  20. 20.
    Information Security: Agencies Need to Improve Cyber Incident Response Practices. GAO-14-354, 30 April 2014. Publicly Released: May 30, 2014Google Scholar
  21. 21.
    Freiling, F.C., Schwittay, B.: A common process model for incident response and computer forensics. IMF 7, 19–40 (2007)Google Scholar
  22. 22.
    Prosise, C., Mandia, K., Pepe, M.: Incident Response & Computer Forensics. McGraw-Hill/Osborne, New York (2003)Google Scholar
  23. 23.
    Dawkins, J., Hale, J.: A systematic approach to multi-stage network attack analysis. In: Second IEEE International Information Assurance Workshop, Proceedings, pp. 48–56. IEEE (2004)Google Scholar
  24. 24.
    Jha, S., Sheyner, O., Jeannette, M.W.: Minimization and reliability analyses of attack graphs. No. CMU-CS-02-109. Carnegie-Mellon Univ. Pittsburgh PA School of Computer Science (2002)Google Scholar
  25. 25.
    Thomas, J.J., Cook, K.A.: The science of analytical reasoning. In: Illuminating the Path: The Research and Development Agenda for Visual Analytics, pp. 32–68 (2005)Google Scholar
  26. 26.
    Mancuso, V.F., Minotra, D., Giacobe, N., McNeese, M., Tyworth, M.: idsNETS: an experimental platform to study situation awareness for intrusion detection analysts. In: IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), pp. 73–79. IEEE (2012)Google Scholar
  27. 27.
    Giacobe, N.A.: Measuring the effectiveness of visual analytics and data fusion techniques on situation awareness in cyber-security. PhD diss., The Pennsylvania State University (2013)Google Scholar
  28. 28.
    Poling, A., Methot, L.L., LeSage, M.G.: Fundamentals of Behavior Analytic Research. Springer Science & Business Media, US (2013)Google Scholar
  29. 29.
    Lee, F.J., Anderson, J.R.: Does learning a complex task have to be complex? A study in learning decomposition. Cogn. Psychol. 42(3), 267–316 (2001)CrossRefGoogle Scholar
  30. 30.
    Kukreja, U., Stevenson, W.E., Ritter, F.E.: RUI: recording user input from interfaces under Windows and Mac OS X. Behav. Res. Methods 38(4), 656–659 (2006)CrossRefGoogle Scholar
  31. 31.
    Allopenna, P.D., Magnuson, J.S., Tanenhaus, M.K.: Tracking the time course of spoken word recognition using eye movements: evidence for continuous mapping models. J. Mem. Lang. 38(4), 419–439 (1998)CrossRefGoogle Scholar
  32. 32.
    Rabinovich, M.I., Huerta, R., Varona, P., Afraimovich, V.S.: Transient cognitive dynamics, metastability, and decision making. PLoS Comput. Biol. 4(5), e1000072 (2008)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Tom, P., Santtila, P., Bosco, D.: The ability of human judges to link crimes using behavioral information: current knowledge and unresolved issues. In: Crime Linkage: Theory, Research, and Practice. CRC Press, p. 268 (2014)Google Scholar
  34. 34.
    Zhong, C., Samuel, D., Yen, J., Liu, P., Erbacher, R., Hutchinson, S., Etoty, R., Cam, H., Glodek, W.: RankAOH: context-driven similarity-based retrieval of experiences in cyber analysis. In: IEEE International Inter-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), pp. 230–236. IEEE (2014)Google Scholar
  35. 35.
    Zhong, C., Yen, J., Liu, P., Erbacher, R., Etoty, R., Garneau, C.: An integrated computer-aided cognitive task analysis method for tracing cyber-attack analysis processes. In: Proceedings of the 2015 Symposium and Bootcamp on the Science of Security, p. 9. ACM (2015)Google Scholar
  36. 36.
    Pirolli, P.: Information Foraging Theory: Adaptive Interaction with Information. Oxford University Press (2007)Google Scholar
  37. 37.
    Pirolli, P., Card, S.: Information foraging. Psychol. Rev. 106(4), 643 (1999)CrossRefGoogle Scholar
  38. 38.
    Zhong, C., Yen, J., Liu, P., Erbacher, R., Etoty, R., Garneau, C.: ARSCA: a computer tool for tracing the cognitive processes of cyber-attack analysis. In: IEEE International Inter-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), pp. 165–171. IEEE (2015)Google Scholar
  39. 39.
    “VAST Challenge 2012 Mini-Challenge 2”, Visual Analytics Community (2012)Google Scholar
  40. 40.
    Scholtz, J., Whiting, M.A., Plaisant, C., Grinstein, G.: A reflection on seven years of the VAST challenge. In: Proceedings of the 2012 BELIV Workshop: Beyond Time and Errors-Novel Evaluation Methods for Visualization, p. 13. ACM (2012)Google Scholar
  41. 41.
    Bass, T.: Multisensor data fusion for next generation distributed intrusion detection systems, pp. 24–27 (1999)Google Scholar
  42. 42.
    Lan, F., Chunlei, W., Guoqing, M.: A framework for network security situation awareness based on knowledge discovery. In: 2nd international conference on Computer Engineering and Technology (ICCET), vol. 1, pp. V1–226. IEEE (2010)Google Scholar
  43. 43.
    Fink, G.A., North, C.L., Endert, A., Rose, S.: Visualizing cyber security: usable workspaces. In: 6th International Workshop on Visualization for Cyber Security, VizSec 2009, pp. 45–56. IEEE (2009)Google Scholar
  44. 44.
    McClain, J., Silva, A., Emmanuel, G., Anderson, B., Nauer, K., Abbott, R., Forsythe, C.: Human Performance Factors in Cyber Security Forensic Analysis (2015)Google Scholar
  45. 45.
    Zhong, C., Kirubakaran, D.S., Yen, J., Liu, P., Hutchinson, S., Cam, H.: How to use experience in cyber analysis: an analytical reasoning support system. In: IEEE International Conference on Intelligence and Security Informatics (ISI), pp. 263–265. IEEE (2013)Google Scholar
  46. 46.
    Giacobe, N.A.: Application of the JDL data fusion process model for cyber security. In: SPIE Defense, Security, and Sensing, p. 77100R. International Society for Optics and Photonics (2010)Google Scholar
  47. 47.
    Yang, S.J., Stotz, A., Holsopple, J., Sudit, M., Kuhl, M.: High level information fusion for tracking and projection of multistage cyber attacks. Inf. Fusion 10(1), 107–121 (2009)CrossRefGoogle Scholar
  48. 48.
    Vandenberghe, G.: Visually assessing possible courses of action for a computer network incursion. In: SANS Institute, InfoSec Reading Room (2007)Google Scholar
  49. 49.
    Aamodt, A., Plaza, E.: Case-based reasoning: foundational issues, methodological variations, and system approaches. AI Commun. 7(1), 39–59 (1994)Google Scholar
  50. 50.
    Cockburn, A., Karlson, A., Bederson, B.B.: A review of overview+detail, zooming, and focus+context interfaces. ACM Comput. Surv. (CSUR) 41(1), 2 (2009)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Chen Zhong
    • 3
    Email author
  • John Yen
    • 1
  • Peng Liu
    • 1
  • Rob F. Erbacher
    • 2
  • Christopher Garneau
    • 2
  • Bo Chen
    • 4
  1. 1.College of Information Sciences and TechnologyPennsylvania State UniversityState CollegeUSA
  2. 2.Army Research LabAdelphiUSA
  3. 3.Indiana University KokomoKokomoUSA
  4. 4.University of MemphisMemphisUSA

Personalised recommendations