Natural Computing

, Volume 8, Issue 1, pp 3–27

Intrusion detection with evolutionary learning classifier systems

  • Kamran Shafi
  • Tim Kovacs
  • Hussein A. Abbass
  • Weiping Zhu
Article

Abstract

Evolutionary Learning Classifier Systems (LCSs) combine reinforcement learning or supervised learning with effective genetics-based search techniques. Together these two mechanisms enable LCSs to evolve solutions to decision problems in the form of easy to interpret rules called classifiers. Although LCSs have shown excellent performance on some data mining tasks, many enhancements are still needed to tackle features like high dimensionality, huge data sizes, non-uniform distribution of classes, etc. Intrusion detection is a real world problem where such challenges exist and to which LCSs have not previously been applied. An intrusion detection problem is characterised by huge network traffic volumes, difficult to realize decision boundaries between attacks and normal activities and highly imbalanced attack class distribution. Moreover, it demands high accuracy, fast processing times and adaptability to a changing environment. We present the results and analysis of two classifier systems (XCS and UCS) on a subset of a publicly available benchmark intrusion detection dataset which features serious class imbalances and two very rare classes. We introduce a better approach for handling the situation when no rules match an input on the test set and recommend this be adopted as a standard part of XCS and UCS. We detect little sign of overfitting in XCS but somewhat more in UCS. However, both systems tend to reach near-best performance in very few passes over the training data. We improve the accuracy of these systems with several modifications and point out aspects that can further enhance their performance. We also compare their performance with other machine learning algorithms and conclude that LCSs are a competitive approach to intrusion detection.

Keywords

Learning classifier systems LCS XCS UCS Intrusion 

References

  1. Bacardit J, Butz MV (2004) Data mining in learning classifier systems: comparing XCS with GAssist. Illinois Genetic Algorithms Laboratory, University of Illinois at Urbana-Champaign. IlliGAL Report No. 2004030Google Scholar
  2. Bernadó E (2002) Contributions to Genetic Based Classifier Systems. PhD thesis, Enginyeria i Arquitectura La Salle, Ramon Llull University, Barcelona, SpainGoogle Scholar
  3. Bernadó E, Garrell JM (2003) Accuracy-based learning classifier systems: models, analysis and applications to classification tasks. Evol Comput 11(3):209–238CrossRefGoogle Scholar
  4. Bernadó E, Llorà X, Garrell JM (2002) XCS and GALE: a comparative study of two learning classifier systems on data mining In: IWLCS ‘01: Revised papers from the 4th International Workshop on Advances in Learning Classifier Systems, Springer-Verlag, London, UK, pp 115–132Google Scholar
  5. Booker L (1985) Improving the performance of genetic algorithms in classifier systems. In: Grefenstette JJ (ed) Proceedings of the 1st Int Conf on genetic algorithms and their applications (ICGA). Lawrence Erlbaum Associates, pp 80–92Google Scholar
  6. Brown G, Kovacs T, Marshall JAR (2007) UCSpv: Principled Voting in UCS Rule Populations. In: Lipson H et al (eds) To appear in the proceedings of the 2007 Genetic and Evolutionary Computation Conference (GECCO’07). ACMGoogle Scholar
  7. Butz MV (2004) Rule-based evolutionary online learning systems: learning bounds, classification, and prediction. PhD thesis, University of Illinois at Urbana-ChampaignGoogle Scholar
  8. Butz MV, Pelikan M (2001) Analyzing the evolutionary pressures in XCS. In: Spector L et al (eds) Proceedings of the genetic and evolutionary computation conference (GECCO-2001). Morgan Kaufmann, pp 935–942Google Scholar
  9. Butz MV, Kovacs T, Lanzi PL, Wilson SW (2001) How XCS evolves accurate classifiers. In: Spector L et al (eds) Proceedings of the Genetic and evolutionary computation conference (GECCO-2001). Morgan Kaufmann, pp 927–934Google Scholar
  10. Butz M, Kovacs T, Lanzi PL, Wilson SW (2004) Toward a theory of generalization and learning in XCS. IEEE Trans Evol Comput 8(1):28–46CrossRefGoogle Scholar
  11. Dam HH, Abbass HA, Lokan C (2005) Be real! XCS with continuous valued inputs. In: Proceedings of eighth international workshop on learning classifier systems. Washington DCGoogle Scholar
  12. Dawson D (2003) Improving performance in size-constrained extended classifier systems. In: Cant’u-Paz E, Foster JA, Deb K, Davis D, Roy R, O’Reilly U-M, Beyer H-G, Standish R, Kendall G, Wilson S, Harman M, Wegener J, Dasgupta D, Potter MA, Schultz AC, Dowsland K, Jonoska N, Miller J (eds) Genetic and evolutionary computation—GECCO-2003. Springer-Verlag, Berlin, pp 1870–1881CrossRefGoogle Scholar
  13. Elkan C (2000) Results of the KDD’99 classifier learning. SIGKDD Explorat Newslett 1(2):63–64CrossRefGoogle Scholar
  14. Ertoz L, Eilertson E, Lazarevic A, Tan PN, Kumar V, Srivastava J, Dokas P (2004) Minds-minnesota intrusion detection system. Next Generation Data MiningGoogle Scholar
  15. Greenyer A (2000) Coil challenge 2000. The use of a learning classifier system JXCS. Technical Report LIACS Technical Report 2000-2009, Sentient Machine Research, Amsterdam and Leiden Institute of Advanced Computer Science, LeidenGoogle Scholar
  16. Hettich S, Bay SD (1999) The UCI KDD Archive. http://www.kdd.ics.uci.edu/databases/kddcup99/kddcup99.html
  17. Holland JH (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor, Republished by the MIT press, 1992Google Scholar
  18. Hurst J, Bull L (2003) Self-adaptation in classifier system controllers. Artif Life Robotics 5(2):109–119CrossRefGoogle Scholar
  19. Kovacs T (1997) XCS classifier system reliably evolves accurate, complete, and minimal representations for Boolean functions. In: Roy, Chawdhry, Pant (eds) Soft computing in engineering design and manufacturing. Springer-Verlag, London, pp 59–68Google Scholar
  20. Kovacs T (2000) Strength or accuracy? Fitness calculation in learning classifier systems. In: Lanzi PL, Stolzmann W, Wilson SW (eds) Learning classifier systems, from foundations to applications. Springer, pp 143–160Google Scholar
  21. Kovacs T (2006) A study of structural and parametric learning in XCS. Evol Comput 14(1):1–19CrossRefGoogle Scholar
  22. Lee W, Stolfo SJ (2001) A framework for constructing features and models for intrusion detection systems. ACM Trans Inf Syst Secur 3(4):227–261CrossRefGoogle Scholar
  23. Lee W, Stolfo S, Mok K (1999) A data mining framework for building intrusion detection models. In: Proceedings of the 1999 IEEE Symposium on Security and Privacy, May 1999, Oakland, CA, pp 120–132Google Scholar
  24. Lippmann RP, Zissman MA (1998) 1998 DARPA/AFRL off-line intrusion detection evaluation. Dataset available at http://www.ll.mit.edu/IST/ideval/data/data_index.html
  25. Mahoney MV, Chan PK (2003) Learning rules for anomaly detection of hostile network traffic. In: Third IEEE international conference on data mining, ICDM 2003, 19–22 Nov 2003, Melborne, Florida, USA, pp 601–604Google Scholar
  26. Ning P, Xu D, Healey C, Amant R (2004) Building attack scenarios through integration of complementary alert correlation methods. In: Proceedings of the 11th annual network and distributed system security symposium (NDSS’04), 5–6 February 2004, San Diego, California, USA, pp 97–111Google Scholar
  27. Noel S, Robertson E, Jajodia S (2004) Correlating intrusion events and building attack scenarios through attack graph distances. Computer Security Applications Conference, 2004. 20th Annual, 6–10 December 2004, Tucson, Arizona, USA, pp 350–359Google Scholar
  28. Orriols A, Bernado-Mansilla E (2006) Class imbalance problem in UCS classifier system: fitness adaptation. The 2005 IEEE Congress on Evolutionary Computation, 8–12 July 2006, Seatle, WA, USA, pp 604–611Google Scholar
  29. Orriols-Puig A, Bernadó-Mansilla E (2006) Bounding XCS’s parameters for unbalanced datasets. In: Proceedings of the 8th annual conference on Genetic and evolutionary computation, pp 1561–1568Google Scholar
  30. Ramesh A, Mahesh JV (2001) PNrule: a new framework for learning classifier models in data mining (a case-study in network intrusion detection). In: Grossman R, Kumar V (eds) 1st SIAM Conference on Data Mining, 5–7 April 2001, Chicago, IL, USAGoogle Scholar
  31. Sabhnani M, Serpen G (2003) Application of machine learning algorithms to KDD intrusion detection dataset within misuse detection context. In: MLMTA, 23–26 June 2006, Las Vegas, Nevada, USA, pp 209–215Google Scholar
  32. Sampson P (1982) Fitting conic sections to “very scattered” data: an iterative refinement of the Bookstein algorithm. Comp Graphics Image Proce 18:97–108CrossRefGoogle Scholar
  33. Shafi K, Abbass HA, Zhu W (2006) An adaptive rule-based intrusion detection architecture. In: Proceedings of the 2006 RNSA security technology conference. Canberra, Australia, pp 307–319Google Scholar
  34. Shafi K, Abbass HA, Zhu W (2006) The role of early stopping and population size in XCS for intrusion detection. In: Proceedings of the 6th international conference on simulated evolution and learning. Lecture Notes in Computer Science, pp 50–57Google Scholar
  35. Stone C, Bull L (2003) For real! XCS with continuous-valued inputs. Evol Comput 11(3):299–336CrossRefGoogle Scholar
  36. Wada A, Takadama K, Shimohara K, Katai O (2007) Analyzing parameter sensitivity and classifier representations for real-valued XCS. In: Kovacs T et al (eds) Learning classifier systems. International Workshops, IWLCS 2003–2005. Revised selected papers, LNAI 4399, Springer, pp 1–16Google Scholar
  37. Wilson SW (2000) Get real! XCS with continuous-valued inputs. In: Lanzi P, Stolzmann W, Wilson S (eds) Learning classifier systems, from foundations to applications, LNAI-1813. Berlin, pp 209–219Google Scholar
  38. Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149–175CrossRefGoogle Scholar
  39. Wilson SW (2001) Mining oblique data with XCS. In: Lanzi PL, Stolzmann W, Wilson SW (eds) Proceedings of the third international workshop (IWLCS-2000). Lecture Notes in Artificial Intelligence, pp 158–174Google Scholar
  40. Witten IH, Frank E (2000) Data mining: practical machine learning tools and techniques with java implementations. Morgan KaufmannGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2007

Authors and Affiliations

  • Kamran Shafi
    • 1
  • Tim Kovacs
    • 2
  • Hussein A. Abbass
    • 1
  • Weiping Zhu
    • 1
  1. 1.School of Information Technology and Electrical EngineeringUNSW@ADFACanberraAustralia
  2. 2.Department of Computer ScienceUniversity of BristolBristolUK

Personalised recommendations