A Novel Anomaly Detection System Based on HFR-MLR Method

  • Eunhye KimEmail author
  • Sehun Kim
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 274)


Reducing the data space and then classifying anomalies based on the reduced feature space is vital to real-time intrusion detection. In this study, a novel framework is developed for logistic regression-based anomaly detection and hierarchical feature reduction (HFR) to preprocess network traffic data before detection model training. The proposed dimensionality reduction algorithm optimally excludes the redundancy of features by considering the similarity of feature responses through a clustering analysis based on the feature space reduced by factor analysis, thus helping to rank the importance of input features (essential, secondary and insignificant) with low time complexity. Classification of anomalies over the reduced feature space is based on a multinomial logistic regression (MLR) model to detect multi-category attacks as an outcome with the goal of reinforcing detection efficiency. The proposed system not only achieves a significant detection performance, but also enables fast detection of multi-category attacks.


Anomaly detection Dimensionality reduction Hierarchical clustering Multinomial logistic regression 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kabiri, P., Ghorbani, A.A.: Research on Intrusion Detection and Response: a Survey. Int. J. Netw. Sec. 1, 84–102 (2005)Google Scholar
  2. 2.
    Lazarevic, A., Ozgur, A., Ertoz, L., Srivastava, J., Kumar, V.: A Comparative Study of Anomaly Detection Schemes in Network Intrusion Detection. In: SIAM International Conference (2003)Google Scholar
  3. 3.
    Leung, K., Leckie, C.: Unsupervised Anomaly Detection in Network Intrusion Detection. In: Australasian Computer Science Conference (2005)Google Scholar
  4. 4.
    Chan, P.K., Mahoney, M.V., Arshad, M.H.: Learning Rules and Clusters for Anomaly Detection in Network Traffic. In: Managing Cyber Threats: Issues, Approaches and Challenges, pp. 81–99. Springer (2005)Google Scholar
  5. 5.
    Valdes, A., Skinner, K.: Adaptive Model-based Monitoring for Cyber Attack Detection. In: Recent Advances in Intrusion Detection Toulouse, pp. 80–92 (2000)Google Scholar
  6. 6.
    Xu, J., Shelton, C.R.: Intrusion Detection using Continuous Time Bayesian Networks. J. Art. Int. Res. 39, 745–774 (2010)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Huang, L., Nguyen, X., Garofalakis, M., Jordan, M.I., Joseph, A., Taft, N.: In-Network PCA and Anomaly Detection. In: Neural Information Processing Systems, pp. 617–624 (2006)Google Scholar
  8. 8.
    Toosi, A.N., Kahani, M.: A New Approach to Intrusion Detection based on an Evolutionary Soft Computing Model using Neuro-Fuzzy Classifiers. Com. Comm. 30, 2201–2212 (2007)CrossRefGoogle Scholar
  9. 9.
    McFadden, D.: Conditional LogitAnalysis of Qualitative Choice Behavior. Frontiers in Econometrics, 105–142 (1974)Google Scholar
  10. 10.
    Tavallaee, M., Bagheri, E., Lu, W., Ghorbani, A.: A Detailed Analysis of the KDD CUP 99 Data Set. In: IEEE Symposium on Computational Intelligence for Security and Defense Applications (2009)Google Scholar
  11. 11.
    Horng, S.J., Su, M.Y., Chen, Y.H., Kao, T.W., Chen, R.J., Lai, J.L., Perkasa, C.D.: A Novel Intrusion Detection System based on Hierarchical Clustering and Support Vector Machines. Exp. Sys. W. Appl. 38, 306–313 (2011)CrossRefGoogle Scholar
  12. 12.
    Xuren, W., Famei, H., Rongsheng, X.: Modeling Intrusion Detection Systemby Discovering Association Rule in Rough Set Theory Framework. In: International Conference on Computational Intelligence for Modelling Controland Automation, and International Conference on Intelligent Agents, WebTechnologies and Internet Commerce (2006)Google Scholar
  13. 13.
    Sabhnani, M.R., Serpen, G.: Application of Machine Learning Algorithms to KDD Intrusion Detection Dataset within Misuse Detection Context. In: International Conference on Machine Learning: Models, Technologies, and Applications, pp. 209–215 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.IT Convergence Technology Research DivisionETRIDaejeonSouth Korea
  2. 2.Internet Security Lab.KAISTDaejeonSouth Korea

Personalised recommendations