Abstract
Driven by the permanent search for reliable anomaly-based intrusion detection mechanisms, we investigated different statistical methodologies to deal with the detection of polymorphic shellcode. The paper intends to give an overview on existing approaches in the literature as well as a synopsis of our efforts to evaluate the applicability of data mining techniques such as Neural Networks, Self Organizing Maps, Markov Models or Genetic Algorithms in the area of polymorphic code detection. We will then present our achieved results and conclusions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
AlephOne: Smashing the stack for fun and profit. Phrack Magazine 49(14) (1996)
Biles, S.: Detecting the Unknown with Snort and the Statistical Packet Anomaly Detection Engine (SPADE). retrieved on (2005), http://www.computersecurityonline.com/spade/SPADE.pdf
Bishop, C.M.: Neural networks for pattern recognition. The Clarendon Press Oxford University Press, New York (1995) With a foreword by Geoffrey Hinton
CLET team: Polymorphic shellcode engine. Phrack Magazine 61(9) (2003)
Duda, R., Hart, P., Stork, D.: Pattern classification, 2nd edn. Wiley-Interscience, New York (2001)
Helsinki University of Technology. Som toolbox for matlab (2005), http://www.cis.hut.fi/projects/somtoolbox/
K2. Admutate 0.8.4. (2004), Retrieved http://www.ktwo.ca
Kohonen, T.: Self-Organizing Maps. Springer, Heidelberg (2001)
Kraxberger, S., Payer, U.: Markov Model for Polymorphic Shellcode Detection. In: INC 2005 (2005) (accepted)
Mathworks. Neural network toolbox (2004), http://www.mathworks.com/products/neuralnet/
NASM SourceForge Project (2005), http://nasm.sourceforge.net
Pasupulati, A.C., Levitt, J., Wu, K., Li, S.F., Kuo, S.H., Fan, J.C.: Buttercup: on network-based detection of polymorphic buffer overflow vulnerabilities. In: Network Operations and Management Symposium, NOMS 2004. IEEE/IFIP, vol. 1, pp. 235–248 (2004)
Payer, U., Teufl, P., Lamberger, M.: Hybrid Engine for Polymorphic Shellcode Detection. In: accepted at DIMVA (2005)
Payer, U., Teufl, P., Lamberger, M.: Traffic classification using Self-Organizing Maps. In: accepted at INC 2005 (2005)
Roweis, S.: Levenberg-marquardt optimization (2005), http://www.cs.toronto.edu/~roweis/notes/lm.pdf
Ruiu, D.: Snort preprocessor - Multi-architecture mutated NOP sled detector (2005), http://cansecwest.com/spp_fnord.c
Sedalo, M.: Polymorphic Shellcode Engine (2004), http://www.shellcode.com.ar
Snort. Open Source Network Intrusion Detection System (2005), http://www.snort.org
Toth, T., Kruegel, C.: Accurate buffer overflow detection via abstract payload execution. In: Wespi, A., Vigna, G., Deri, L. (eds.) RAID 2002. LNCS, vol. 2516, pp. 274–291. Springer, Heidelberg (2002)
Weisstein, E.W.: Markov Chain. From MathWorld–A Wolfram Web Resource, http://mathworld.wolfram.com/MarkovChain.html
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Payer, U., Teufl, P., Kraxberger, S., Lamberger, M. (2005). Massive Data Mining for Polymorphic Code Detection. In: Gorodetsky, V., Kotenko, I., Skormin, V. (eds) Computer Network Security. MMM-ACNS 2005. Lecture Notes in Computer Science, vol 3685. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11560326_38
Download citation
DOI: https://doi.org/10.1007/11560326_38
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29113-8
Online ISBN: 978-3-540-31998-6
eBook Packages: Computer ScienceComputer Science (R0)