Abstract
This chapter describes data mining and data warehousing techniques that can improve the performance and usability of Intrusion Detection Systems (IDS). Current IDS do not provide support for historical data analysis and data summarization. This chapter presents techniques to model network traffic and alerts using a multi-dimensional data model and star schemas. This data model was used to perform network security analysis and detect denial of service attacks. Our data model can also be used to handle heterogeneous data sources (e.g. firewall logs, system calls, net-flow data) and enable up to two orders of magnitude faster query response times for analysts as compared to the current state of the art. We have used our techniques to implement a prototype system that is being successfully used at Army Research Labs. Our system has helped the security analyst in detecting intrusions and in historical data analysis for generating reports on trend analysis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Singhal A. and Jajodia S., “Data Mining for Intrusion Detection”, Published as a chapter in Data Mining Handbook, Kluwer, December 2004.
Lee W., Stolfo, S. J., and Kwok K. W. Mining audit data to build intrusion detection models. In Proc. Fourth International Conference on Knowledge Discovery and Data Mining, New York, 1998.
Lee W. and Stolfo S. J. Data Mining approaches for intrusion detection, In Proc. Seventh USENIX Security Symposium, San Antonio, TX, 1998.
Barbara D., Wu N., and Jajodia S., Detecting novel network intrusions using bayes estimators. In Proc. First SIAM Conference on Data Mining, Chicago, IL, April 2001.
Barbara D., Couto J., Jajodia S., and Wu N., Adam: Detecting Intrusions by Data Mining, In Proc. 2nd Annual IEEE Information Assurance Workshop, West Point, NY, June 2001.
Ertoz L., Eilertson E., Lazarevic A., Tan P., Dokes P., Kumar V., Srivastava J., Detection of Novel Attacks using Data Mining, Proc. IEEE Workshop on Data Mining and Computer Security, November 2003.
Kumar V., Lazarevic A., Ertoz L., Ozgur A., Srivastava J., A Comparative Study of Anomaly Detection Schemes in Network intrusion Detection, Proc. Third SIAM International Conference on Data Mining, San Francisco, May 2003.
Portnoy L., Eskin E., Stolfo S. J., Intrusion Detection with unlabeled data using clustering. In Proceedings of ACM Workshop on Data Mining Applied to Security, 2001.
Abraham T. (2001) IDDM: intrusion Detection Using Data Mining Techniques. Technical Report DSTO-GD-0286, DSTO Electronics and Surveillance Research Laboratory
Valdes A. and Skinner K. (2000) Adaptive, model based monitoring for cyber attack detection. In Recent Advances on intrusion Detection, pp 80–93, France, Springer Verlag
Ning P., Cui Y., Reeves D. S., Constructing Attack Scenarios through Correlation of Intrusion Alerts, Proc. ACM Computer and Communications Security Conf., 2002.
Ning P., Xu D., Learning Attack Strategies from Intrusion Alerts, Proc. ACM Computer and Communications Security Conf., 2003.
Cuppens F. and Miege A., Alert Correlation in a Cooperative Intrusion Detection Framework, Proc. IEEE Symposium on Security and Privacy, May 2002.
A. Singhal, “ANSWER: Network Monitoring using Object Oriented Rules” (with G. Weiss and J. Ros), Proceedings of the Tenth Conference on Innovative Application of Artificial Intelligence, Madison, Wisconsin, July 1998
Singhal A., “Design of GEMS Data Warehouse for AT&T Business Services”, Proceedings of AT&T Software Architecture Symposium, Somerset, NJ, March 2000
Singhal A., “Design of Data Warehouse for Network/Web Services”, Proceedings of Conference on Information and Knowledge Management (CIKM), November 2004..
DARPA, DARPA 1998 Intrusion Detection Evaluation, http://ideval.ll.mit.edu/1998_index.hfml
SNORT, SNORT Intrusion Detection System, http://www.snort.org
RealSecure IDS, http://www.iss.net
KDD Cup 1999, http://www.kdd.ics.uci.edu/databases/kddcup99/task.html
GraphViz, Graph layout and drawing software, http://www.research.att.com/sw/tools/graphviz
X. Qin and W. Lee, “Statistical causality analysis of INFOSEC alert data”, In Proceedings of 6th International Symposium on Recent Advances in Intrusion Detection (RAID 2003), September 2003.
X. Qin and W. Lee, “Discovering novel attack strategies from INFOSEC alerts”, in Proceedings of the 9th European Symposium on Research in Computer Security (ESORICS 2004), September 2004.
Steven Noel, Eric Robertson, Sushil Jajodia, “Correlating Intrusion Events and Building Attack Scenarios through Attack Graph Distances,” in Proceedings of the 20 th Annual Computer Security Applications Conference, Tucson, Arizona, December 2004.
Rights and permissions
Copyright information
© 2007 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Singhal, A. (2007). Data Modeling and Data Warehousing Techniques to Improve Intrusion Detection. In: Data Warehousing and Data Mining Techniques for Cyber Security. Advances in Information Security, vol 31. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-47653-7_5
Download citation
DOI: https://doi.org/10.1007/978-0-387-47653-7_5
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-26409-7
Online ISBN: 978-0-387-47653-7
eBook Packages: Computer ScienceComputer Science (R0)