Abstract
Data collection is one of the most important steps when designing an Intrusion Detection System (IDS) and it influences the whole design and implementation process, and also the final detection result. Usually, the attacks target not only one individual computer but also aim for a group of hosts. As a result, some intrusions might show an anomalous behavior at the network layer, while others could exhibit anomawe lous behaviors at the application layer. In order to cover various network intrusions we need to monitor each layer on networks. Although ideally it is possible to design and implement an IDS that can inspect a wide range of data extracted from both network and application layer, it is infeasible in practical due to two main reasons: one is the diversity of the data, and the other one is the time and space resources that the system has to consume for collecting and interpreting the data. Intrusion detection systems collect data from many different sources, such as system log files, network packets or flows, system calls and a running code itself. The place where the data are collected decides the detection capability and scope of IDSs, i.e. a network based IDS can not detect a User-to-Root attack, while an application based IDS is not able to find a port scanning attack. In this chapter, we discuss the data collection in terms of the different locus including host-based, network-based and application-based.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
TCPDUMP, Available on: http://www.tcpdump.org/, September 2008.
FlowScan,a network analysis and reporting tool, Available on: http://net.doit.wisc.edu/ plonka/FlowScan/, October 2008.
KDD Cup 1999. Available on: http://kdd.ics.uci.edu/databases/kddcup 99/kddcup99.html, October 2007.
M. Almgren and U. Lindqvist, Application-integrated data collection for security monitoring, Lecture Notes in Computer Science (2001), 22–36.
V. Berk, G. Bakos, and R. Morris, Designing a framework for active worm detection on global networks, Proceedings of the IEEE International Workshop on Information Assurance (Darmstadt, Germany), 2003.
Joachim Biskup and Ulrich Flegel, Transaction-based pseudonyms in audit data for privacy respecting intrusion detection, Proceedings of Recent Advances in Intrusion Detection (RAID), Lecture Notes in Computer Science, Springer-Verlag Heidelberg, October 2000, pp. 28–48.
P. Dokas, L. Ertoz, V. Kumar, A. Lazarevic, J. Srivastava, and P.N. Tan, Data mining for network intrusion detection, Proceedings of the NSF Workshop on Next Generation Data Mining, 2002, pp. 21–30.
L. Ertoz, E. Eilertson, A. Lazarevic, P.N. Tan, P. Dokas, V. Kumar, and J. Srivastava, Detection of novel network attacks using data mining, Proceedings of the Workshop on Data Mining for Computer Security (DMSEC), 2003.
Chapman Flack and Mikhail J. Atallah, Better logging through formality applying formal specification techniques to improve audit logs and log consumers, Proceedings of Recent Advances in Intrusion Detection (RAID), Lecture Notes in Computer Science, Springer-Verlag Heidelberg, October 2000, pp. 1–16.
S. Forrest, S. Hofmeyr, A. Somayaji, and T. Longstaff, A sense of self for unix processes, Proceedings of the 1996 IEEE Symposium on Security and Privacy (Los Alamitos, CA), IEEE Computer Society Press, 1996, p. 120128.
Anup K. Ghosh, Christoph Michael, and Michael Schatz, A real-time intrusion detection system based on learning program behavior, Proceedings of Recent Advances in Intrusion Detection (RAID), Lecture Notes in Computer Science, Springer-Verlag Heidelberg, October 2000, pp. 93–109.
G. Helmer, J.S.K. Wong, V. Honavar, L. Miller, and Y. Wang, Lightweight agents for intrusion detection, The Journal of Systems & Software 67 (2003), no. 2, 109–122.
C. Ko, System health and intrusion monitoring (shim): project summary, Proceedings of the DARPA Information Survivability Conference and Exposition II, DISCEX'03, vol. 2, April 2003, pp. 202–207.
C. Kruegel, D. Mutz, F. Valeur, and G. Vigna, On the detection of anomalous system call arguments, Proceedings of the 8th European Symposium on Research in Computer Security (ESORICS) (Gjovik, Norway), LNCS, Springer-Verlag, October 2003, pp. 326–343.
Christopher Kruegel and Giovanni Vigna, Anomaly detection of web-based attacks, Proceedings of the 10th ACM conference on Computer and communication security (Washington D.C., USA), ACM Press, October 2003, pp. 251–261.
Josu Kuri, Gonzalo Navarro, Ludovic M, and Laurent Heye, A pattern matching based filter for audit reduction and fast detection of potential intrusions, Proceedings of Recent Advances in Intrusion Detection (RAID), Lecture Notes in Computer Science, Springer-Verlag Heidelberg, October 2000, pp. 17–27.
W. Lee, S. J. Stolfo, and K. W. Mok, A data mining framework for building intrusion detection models, Proceedings of the 1999 IEEE Symposium on Security and Privacy, May 1999, pp. 120–132.
V.A. Mahadik, X. Wu, and D.S. Reeves, Detection of Denial-of-QoS Attacks Based On χ 2 Statistic And EWMA Control Charts, (2002).
J. McHugh, Intrusion and intrusion detection, International Journal of Information Security 1 (2001), no. 1, 14–35.
CC Michael and A. Ghosh, Simple, state-based approaches to program-based anomaly detection, ACM Transactions on Information and System Security (TISSEC) 5 (2002), no. 3, 203–237.
Computer Security Center (NCSC), Audit in trusted systems, July 1987, Library no. S-228 470.
S. Noh, C. Lee, K. Choi, and G. Jung, Detecting distributed denial of service (ddos) attacks through inductive learning, Lecture Notes in Computer Science (2003), 286–295.
OKENA, Stormsystem, August 2002, Cisco acquired Okena in 2003.
TH Ong, CP Tan, YT Tan, and C. Ting, SNMSShadow Network Management System, Proceedings of the Second International Workshop on Recent Advances in Intrusion Detection, 1999.
I.V. Onut, A Fuzzy Feature Evaluation Framework for Network Intrusion Detection, PhD Thesis, Faculty of Computer Science, University of New Brunswick (2008).
I.V. Onut and A. A. Ghorbani, A Feature Classification Scheme For Network Intrusion Detection, International Journal of Network Security 5 (2007).
I.V. Onut and A.A. Ghorbani, Toward a feature classification scheme for network intrusion detection, Proceedings of The Fourth Annual Conference on Communication Networks and Services Research, 2006.
T. Peng, C. Leckie, and R. Kotagiri, Proactively detecting ddos attack using source ip address monitoring, Proceedings of the Networking 2004 (Athens, Greece), 2004.
X. Qin, W. Lee, L. Lewis, and JBD Cabrera, Integrating intrusion detection and network management, IEEE/IFIP Network Operations and Management Symposium (NOMS), 2002, pp. 329–344.
B. Schneier and J. Kelsey, Secure audit logs to support computer forensics, ACM Transactions on Information and System Security (TISSEC) 2 (1999), no. 2, 159–176.
Christos Siaterlis and Basil Maglaris, Towards multisensor data fusion for dos detection, Proceedings of the 2004 ACM symposium on Applied computing (Nicosia, Cyprus), 2004, pp. 439–446.
S. Soman, C. Krintz, and G. Vigna, Detecting malicious Java code using virtual machine auditing, Proceedings of the Twelfth USENIX Security Symposium, 2003, pp. 153–167.
S. Staniford, J. Hoagland, and J. McAlerney, Practical automated detection of stealthy portscans, Journal of Computer Security 10 (2002), no. 1 and 2, 105–126.
W.R. Stevens, TCP/IP illustrated (vol. 1): the protocols, Addison-Wesley Longman Publishing Co., Inc. Boston, MA, USA, 1993.
Marina Thottan and Chuanyi Ji, Anomaly detection in ip networks, IEEE Transactions on Signal Processing 51 (2003), no. 8, 148–166.
T. Toth and C. Kruegel, Connection-history based anomaly detection, Proceedings of IEEE Workshop on Information Assurance and Security (West Point, NY), 2002.
G. Vigna and A. Mitchell, Mnemosyne: Designing and implementing network short-term memory, Proceedings of the 8th IEEE International Conference on Engineering of Complex Computer Systems (ICECCS) (Greenbelt, MD), IEEE Press, December 2002, pp. 91–100.
H. Wang, D. Zhang, and K.G. Shin, Detecting SYN flooding attacks, Proceedings of the Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM), vol. 3, 2002.
M. Welz and A. Hutchison, Interfacing trusted applications with intrusion detection systems, Lecture notes in computer science (2001), 37–53.
Zonghua Zhang and Hong Shen, Online training of svms for real-time intrusion detection, Proceedings of the 18th International Conference on Advanced Information Networking and Applications (AINA), vol. 1, March 2004, pp. 568–573.
L. Zhuowei, A. Das, and S. Nandi, Utilizing statistical characteristics of N-grams for intrusion detection, Proceedings of the International Conference on Cyberworlds, 2003, pp. 486–493.
Cliff Changchun Zou, Lixin Gao, Weibo Gong, and Don Towsley, Monitoring and early warning for internet worms, Proceedings of the 10th ACM conference on Computer and communication security (Washington D.C., USA), ACM Press, October 2003, pp. 190–199.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2010 Springer-Verlag US
About this chapter
Cite this chapter
Ghorbani, A.A., Lu, W., Tavallaee, M. (2010). Data Collection. In: Network Intrusion Detection and Prevention. Advances in Information Security, vol 47. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-88771-5_3
Download citation
DOI: https://doi.org/10.1007/978-0-387-88771-5_3
Published:
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-88770-8
Online ISBN: 978-0-387-88771-5
eBook Packages: Computer ScienceComputer Science (R0)