Near-miss situation based visual analysis of SIEM rules for real time network security monitoring

Abstract

Security information and event management (SIEM) systems are generally used to monitor the network for malicious activities. These systems are capable of detecting a wide range of malicious activities in the network using built-in rules to generate alerts on malicious activities. Although SIEM systems provide comprehensive reports about each alert including relevant details such as, severity score, events, and events counts. However, a key limitation of SIEM systems is not presenting the rule’s status in real time before an alert is raised. This paper presents a novel visual tool that enables security analyst to grasp visually, and in real time a complete overview of SIEM rules execution, and alert circumstances that may happen in advance based on near-miss situation. Apart from the real time rules analysis, it also enables security analysts to explore the reasoning behind the alerts in an organized and efficient manner via security questions. The essence of the approach is to evaluate and visualize the current status of each rule execution according to pre-compiled conditions in real time. We demonstrate the utility of our approach using IBM QRadar events data to support the informative analysis of different rules in real time, and security questions based insight about the rules via story page.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

References

  1. Ab Rahman NH, Cahyani NDW, Choo KKR (2017) Cloud incident handling and forensic‐by‐design: cloud storage as a case study. Concurr Comp Pract Exp 29(14):e3868

    Article  Google Scholar 

  2. Ab Rahman NH, Choo K-KR (2015) A survey of information security incident handling in the cloud. Comput Secur 49:45–69

    Article  Google Scholar 

  3. Aguirre I, Alonso S (2012) Improving the automation of security information management: a collaborative approach. IEEE Secur Priv 10(1):55–59

    Article  Google Scholar 

  4. Alsaleh M, Barrera D, van Oorschot PC (2008) Improving security visualization with exposure map filtering. In: Computer security applications conference, 2008. ACSAC 2008. Annual, IEEE, pp 205–214

  5. Anuar NB, Papadaki M, Furnell S, Clarke N (2010) An investigation and survey of response options for intrusion response systems (irss). In: Information security for South Africa (ISSA), 2010, IEEE, pp 1–8

  6. Attipoe AE, Yan J, Turner C, Richards D (2016) Visualization tools for network security. Electron Imaging 1:1–8

    Article  Google Scholar 

  7. Azodi A, Cheng F, Meinel C (2016) Towards better attack path visualizations based on deep normalization of host/network ids alerts. In: 2016 IEEE 30th international conference on advanced information networking and applications (AINA), IEEE, pp 1064–1071

  8. Azodi A, Jaeger D, Cheng F, Meinel C (2013) A new approach to building a multi-tier direct access knowledgebase for IDS/SIEM systems. In: 2013 IEEE 11th international conference on dependable, autonomic and secure computing (DASC), IEEE, pp 118–123

  9. Balabine I, Velednitsky A (2018) Streaming method and system for processing network metadata. U.S. Patent No. 9,860,154. U.S. Patent and Trademark Office, Washington, DC

  10. Barnum S (2008) Common attack pattern enumeration and classification (capec) schema description. Cigital Inc, http://capec.mitre.org/documents/documentation/CAPEC_Schema_Description_v1 (3)

  11. Bauernschmidt B, Schuck J, Webb N (2004) Method and system for visualizing data from multiple, cached data sources with user defined treemap reports. U.S. Patent Application No. 10/371,638

  12. Brewer R (2015) Cyber threats: reducing the time to detection and response. Netw Secur 5:5–8

    Article  Google Scholar 

  13. Briesemeister L, Cheung S, Lindqvist U, Valdes A (2010) Detection, correlation, and visualization of attacks against critical infrastructure systems. In: 2010 eighth annual international conference on privacy security and trust (PST), IEEE, pp 15–22

  14. Cheswick WR, Bellovin SM, Rubin AD (2003) Firewalls and Internet security: repelling the wily hacker. Addison-Wesley Longman Publishing Co., Inc, Boston

    Google Scholar 

  15. Cheung S, Lindqvist U, Fong MW (2003) Modeling multistep cyber attacks for scenario recognition, In: DARPA information survivability conference and exposition, 2003. Proceedings, vol 1, IEEE, pp 284–292

  16. Choo K-KR, Dehghantanha A (2018) Introduction to the minitrack on cyber threat intelligence and analytics. In: Proceedings of the 51st Hawaii international conference on system sciences

  17. Choo K-KR, Esposito C, Castiglione A (2017) Evidence and forensics in the cloud: challenges and future research directions. IEEE Cloud Comput 4(3):14–19

    Article  Google Scholar 

  18. Chuvakin A (2010) Siem: moving beyond compliance. White Paper for RSA

  19. Clayton J (2017) Statement on cybersecurity. Last Accessed 5 Feb 2018

  20. Conroy D (2016) Forensic data analysis challenges in large scale systems, In: Intelligent distributed computing IX, Springer, Berlin, pp 451–457

  21. Constantinescu Z, Vlădoiu M, Moise G (2016) Viznetdynamic visualization of networks and internet of things. In: RoEduNet conference: networking in education and research, 2016 15th, IEEE, pp 1–6

  22. Coppolino L, DAntonio S, Formicola V, Romano L (2011) Integration of a system for critical infrastructure protection with the ossim siem platform: a dam case study. In: International conference on computer safety, reliability, and security. Springer, Berlin, pp 199–212

  23. Coudriau M, Lahmadi A, François J (2016) Topological analysis and visualisation of network monitoring data: Darknet case study. In: 2016 IEEE international workshop on information forensics and security (WIFS), IEEE, pp 1–6

  24. DAmico A, Buchanan L, Kirkpatrick D, Walczak P (2016) Cyber operator perspectives on security visualization, In: Advances in Human Factors in Cybersecurity, Springer, pp 69–81

  25. Flynn L, Huth C, Trzeciak R, Buttles P (2012) Best practices against insider threats for all nations. In: Cybersecurity Summit (WCS), 2012 Third Worldwide, IEEE, pp 1–8

  26. Franklin L, Pirrung M, Blaha L, Dowling M, Feng M (2017) Toward a visualization-supported workflow for cyber alert management using threat models and human-centered design. In: 2017 IEEE symposium on visualization for cyber security (VizSec), IEEE, pp 1–8

  27. Fratto M (2004) Sneak previews-anomaly detection gets better-q1 labs qradar 3.0 provides comprehensive network behavior anomaly detection with its graphical views of all network traffic. Netw Comput Niles 15(11):32–33

    Google Scholar 

  28. Gray CC, Ritsos PD, Roberts JC (2015) Contextual network navigation to provide situational awareness for network administrators. In: 2015 IEEE symposium on visualization for cyber security (VizSec), IEEE, pp 1–8

  29. Guimarães VT, Rendon OMC, dos Santos GL, da Cunha Rodrigues G, Freitas CMDS., Tarouco LMR, Granville LZ (2017) A reuse-based approach to promote the adoption of visualizations for network management tasks. In: 2017 IEEE 31st international conference on advanced information networking and applications (AINA), IEEE, pp 712–719

  30. Gupta A (2017) Review on big data promises for information security. J Data Min Manage 1(2):1–8

    Google Scholar 

  31. Hao L, Healey CG, Hutchinson SE (2013) Flexible web visualization for alert-based network security analytics. In: Proceedings of the tenth workshop on visualization for cyber security, ACM, pp 33–40

  32. Hideshima, Y, Koike H (2006) Starmine: A visualization system for cyber attacks. In: Proceedings of the 2006 Asia-Pacific symposium on information visualisation, vol 60. Australian Computer Society, Inc., pp 131–138

  33. Hinze SR, Rapp DN, Williamson VM, Shultz MJ, Deslongchamps G, Williamson KC (2013) Beyond ball-and-stick: Students’ processing of novel stem visualizations. Learn Instr 26:12–21

    Article  Google Scholar 

  34. Holik F, Horalek J, Neradova S, Zitta S, Marik O (2015) The deployment of security information and event management in cloud infrastructure. In: 2015 25th international conference on Radioelektronika (RADIOELEKTRONIKA), IEEE, pp 399–404

  35. Huang Z, Shen C-C, Doshi S, Thomas N, Duong H (2015) Cognitive task analysis based training for cyber situation awareness. In: IFIP World conference on information security education. Springer, Berlin, pp 27–40

  36. Kabiri P, Ghorbani AA (2005) Research on intrusion detection and response: a survey. IJ Netw Secur 1(2):84–102

    Google Scholar 

  37. Kotenko I, Polubelova O, Saenko I, Doynikova E (2013) The ontology of metrics for security evaluation and decision support in SIEM systems. In: 2013 eighth international conference on availability, reliability and security (ARES), IEEE, pp 638–645

  38. Lakkaraju K, Yurcik W, Lee AJ (2004) Nvisionip: netflow visualizations of system state for security situational awareness. In: Proceedings of the 2004 ACM workshop on visualization and data mining for computer security, ACM, pp 65–72

  39. Lakkaraju K, Bearavolu R, Slagell A, Yurcik W, North S (2005) Closing-the-loop in nvisionip: Integrating discovery and search in security visualizations. In: IEEE workshop on visualization for computer security, 2005 (VizSEC 05), IEEE, pp 75–82

  40. Langton JT, Baker A (2013) Information visualization metrics and methods for cyber security evaluation. In: 2013 IEEE international conference on intelligence and security informatics (ISI), IEEE, pp 292–294

  41. Lee D-G, Kim HK, Kim E (2015) Study on security log visualization and security threat detection using rgb palette. J Korea Inst Inf Secur Cryptol 25(1):61–73

    Article  Google Scholar 

  42. Levine J, LaBella R, Owen H, Contis D, Culver B (2003) The use of honeynets to detect exploited systems across large enterprise networks, In: Information assurance workshop, 2003. IEEE systems, man and cybernetics society, IEEE, pp 92–99

  43. Li T, Yan L (2017) Siem based on big data analysis. In: International conference on cloud computing and security. Springer, Berlin, pp 167–175

  44. Lu M, Chen S, Lai C, Lin L, Yuan X (2017) Frontier of information visualization and visual analytics in 2016. J Vis 20(4):667–686

    Article  Google Scholar 

  45. Mahmood T, Afzal U (2013) Security analytics: Big data analytics for cybersecurity: A review of trends, techniques and tools. In: 2013 2nd national conference on information assurance (NCIA), IEEE, pp 129–134

  46. Mantere M, Sailio M, Noponen S (2013) Network traffic features for anomaly detection in specific industrial control system network. Future Internet 5(4):460–473

    Article  Google Scholar 

  47. Marty R (2009) Applied security visualization. Addison-Wesley Upper Saddle River, Boston

    Google Scholar 

  48. McKenna S, Staheli D, Meyer M (2015) Unlocking user-centered design methods for building cyber security visualizations. In: 2015 IEEE symposium on visualization for cyber security (VizSec), IEEE, pp 1–8

  49. Montesino R, Fenz S, Baluja W (2012) Siem-based framework for security controls automation. Inf Manag Comput Secur 20(4):248–263

    Article  Google Scholar 

  50. Nguyen HT, Tran AVT, Nguyen TAT, Vo LT, Tran PV (2016) Multivariate cube for representing multivariable data in visual analytics. In: International conference on context-aware systems and applications. Springer, Berlin, pp 91–100

  51. Nicolett M, Kavanagh KM (2011) Magic quadrant for security information and event management. Gartner RAS Core Reasearch Note (May 2009)

  52. Novikova ES, Bekeneva YA, Shorov AV (2017) Towards visual analytics tasks for the security information and event management. In: 2017 international conference quality management, transport and information security, information technologies (IT&QM&IS), IEEE, pp 90–93

  53. Oseku-Afful T (2016) The use of big data analytics to protect critical information infrastructures from cyber-attacks

  54. Parmelee MC (2010) Toward the semantic interoperability of the security information and event management lifecycle. In: Working Notes for the 2010 AAAI workshop on intelligent security (SecArt), Citeseer, pp 18

  55. Patil S, Meshram BB (2012) Intrusion prevention system. Int J Emerg Trends Eng Dev 4(2)

  56. Pavlik J, Komarek A, Sobeslav V (2014) Security information and event management in the cloud computing infrastructure. In: 2014 IEEE 15th International Symposium on Computational Intelligence and Informatics (CINTI), IEEE, pp 209–214

  57. Product Brief (2008) ArcSight Logger, Simplifying Log Collection, Storage and Analysis, ArcSight

  58. Pronoza AA, Chechulin AA, Kotenko IV (2016) Mathematical models of visualization in siem systems. Trudy SPIIRAN 46:90–107

    Google Scholar 

  59. Qamar S, Anwar Z, Rahman MA, Al-Shaer E, Chu B-T (2017) Data-driven analytics for cyber-threat intelligence and information sharing. Comput Secur 67:35–58

    Article  Google Scholar 

  60. Ring T (2014) Threat intelligence: why people don’t share. Comput Fraud Secur 3:5–9

    Article  Google Scholar 

  61. Roberts C (2013) Discovering security events of interest using splunk. SANS Institute

  62. Rohs M, Essl G (2006) Which one is better? Information navigation techniques for spatially aware handheld displays. In: Proceedings of the 8th international conference on multimodal interfaces. ACM, pp 100–107

  63. Rowland CH (2002) Intrusion detection system. U.S. Patent No. 6,405,318. U.S. Patent and Trademark Office, Washington, DC

  64. Sethi A, Paci F, Wills G (2016) Eevi-framework for evaluating the effectiveness of visualization in cyber-security. In: 2016 11th international conference for internet technology and secured transactions (ICITST), IEEE, pp 340–345

  65. Shabtai A, Klimov D, Shahar Y, Elovici Y (2006) An intelligent, interactive tool for exploration and visualization of time-oriented security data. In: Proceedings of the 3rd international workshop on visualization for computer security. ACM, pp 15–22

  66. Shah A, Abualhaol I, Gad M, Weiss M (2017) Combining exploratory analysis and automated analysis for anomaly detection in real-time data streams. Technol Innov Manag Rev 7(4):25–31

    Article  Google Scholar 

  67. Ohnof K, Koikef H, Koizumi K (2005) Ipmatrix: an effective visualization framework for cyber threat monitoring, pp 678–685

  68. Stein G, Chen B, Wu AS, Hua KA (2005) Decision tree classifier for network intrusion detection with ga-based feature selection. In: Proceedings of the 43rd annual Southeast regional conference, vol 2. ACM, pp 136–141

  69. Suarez-Tangil G, Palomar E, Ribagorda A, Zhang Y (2014) Towards an intelligent security event information management system. http://www.seg.inf.uc3m.es/papers/2013nova-AIS-SIEM.pdf

  70. Sun K, Jajodia S, Li J, Cheng Y, Tang W, Singhal A (2011) Automatic security analysis using security metrics. In: Military communications conference, 2011-Milcom 2011, IEEE, pp 1207–1212

  71. Sun Y, Overbye TJ (2004) Visualizations for power system contingency analysis data. IEEE Trans Power Syst 19(4):1859–1866

    Article  Google Scholar 

  72. Tassone CF, Martini B, Choo K-KR (2017) Visualizing digital forensic datasets: a proof of concept. J Forensic Sci 62(5):1197–1204

    Article  Google Scholar 

  73. Villella P, Petersen C (2011) Log collection, structuring and processing. U.S. Patent No. 8,032,489. U.S. Patent and Trademark Office, Washington, DC

  74. Wenge O, Lampe U, Rensing C, Steinmetz R (2014) Security information and event monitoring as a service: a survey on current concerns and solutions. PIK-Praxis der Informationsverarbeitung und Kommunikation 37(2):163–170

    Article  Google Scholar 

  75. Yin X, Yurcik W, Treaster M, Li Y, Lakkaraju K (2004) Visflowconnect: netflow visualizations of link relationships for security situational awareness, In: Proceedings of the 2004 ACM workshop on Visualization and data mining for computer security. ACM, pp 26–34

  76. Zander S, Nguyen T, Armitage G (2005) Automated traffic classification and application identification using machine learning, In: The IEEE conference on local computer networks, 2005. 30th Anniversary, IEEE, pp 250–257

  77. Zhang T, Liao Q, Shi L, Dong W (2014) Analyzing spatiotemporal anomalies through interactive visualization. In: Informatics, vol 1. Multidisciplinary Digital Publishing Institute, pp 100–125

  78. Zuech R, Khoshgoftaar TM, Wald R (2015) Intrusion detection and big heterogeneous data: a survey. J Big Data 2(1):3

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Masoom Alam.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Majeed, A., ur Rasool, R., Ahmad, F. et al. Near-miss situation based visual analysis of SIEM rules for real time network security monitoring. J Ambient Intell Human Comput 10, 1509–1526 (2019). https://doi.org/10.1007/s12652-018-0936-7

Download citation

Keywords

  • Malicious
  • SIEM systems
  • Near-miss situation
  • SIEM rules
  • Alerts
  • Informative analysis