A Data-Driven Network Intrusion Detection Model Based on Host Clustering and Integrated Learning: A Case Study on Botnet Detection

  • Lena Ara
  • Xiao LuoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11611)


The traditional machine learning based network intrusion detection system (NIDS) is based on training a model using known network traffic for selected attacks, and testing it on the unknown network traffic of the same attacks. Evaluating machine learning based IDS with new attack traffic that is different from that training set is rare. With a large amount of network traffic generated every second, it is tough to gather all the traffic then train a model. In this research, we designed and developed an intrusion detection model by treating a network as a community. Based on the traffic behaviors, we first developed a host clustering algorithm to group the hosts into clusters and unique hosts. Then, we developed an integrated learning algorithm to integrate model-based learning derived from host clusters, and instance-based learning obtained from individual hosts. We evaluated the intrusion detection model on the CTU-13 data set which is a botnet attack data set. The results show that our model is more robust and effective for network intrusion detection and gains an average 100% detection rate, with a 0% false positive rate on detecting known attack traffic, and 98.2% detection rate on identifying new Botnet attack traffic.


Network intrusion detection Network flow Integrated machine learning Host clustering Botnet detection 


  1. 1.
    Akramifard, H., Khanli, L.M., ABalafar, M., Davtalab, R.: Intrusion detection in the cloud environment using multi-level fuzzy neural networks. In: Proceedings of International Conference on Security and Management, pp. 152–159 (2015).
  2. 2.
    Altman, N.S.: An introduction to kernel and nearest-neighbor nonparametric regression. Am. Stat. 46(3), 175–185 (1992)MathSciNetGoogle Scholar
  3. 3.
    Amor, N.B., Benferhat, S., Elouedi, Z.: Naive bayes vs. decision trees in intrusion detection systems. In: Proceedings of the 2004 ACM Symposium on Applied Computing, pp. 420–424. ACM (2004)Google Scholar
  4. 4.
    Andrzejak, A., Langner, F., Zabala, S.: Interpretable models from distributed data via merging of decision trees. In: 2013 IEEE Symposium on Computational Intelligence and Data Mining (CIDM), pp. 1–9. IEEE (2013)Google Scholar
  5. 5.
    Chong, M., Abraham, A., Paprzycki, M.: Traffic accident analysis using machine learning paradigms. Informatica 29(1) (2005)Google Scholar
  6. 6.
    Clements, J., Yang, Y., Sharma, A., Hu, H., Lao, Y.: Rallying adversarial techniques against deep learning for network security. arXiv preprint arXiv:1903.11688 (2019)
  7. 7.
    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)zbMATHGoogle Scholar
  8. 8.
    García, S., Grill, M., Stiborek, J., Zunino, A.: An empirical comparison of botnet detection methods. Comput. Secur. 45, 100–123 (2014)CrossRefGoogle Scholar
  9. 9.
    Hasan, M.A.M., Nasser, M., Pal, B., Ahmad, S.: Support vector machine and random forest modeling for intrusion detection system (IDS). J. Intell. Learn. Syst. Appl. 6(01), 45 (2014)Google Scholar
  10. 10.
    Huang, H., Al-Azzawi, H., Brani, H.: Network traffic anomaly detection. arXiv preprint arXiv:1402.0856 (2014)
  11. 11.
    Huster, T.P., Chiang, C.Y.J., Chadha, R., Swami, A.: Towards the development of robust deep neural networks in adversarial settings. In: MILCOM 2018–2018 IEEE Military Communications Conference (MILCOM), pp. 419–424. IEEE (2018)Google Scholar
  12. 12.
    Kim, G., Lee, S., Kim, S.: A novel hybrid intrusion detection method integrating anomaly detection with misuse detection. Exp. Syst. Appl. 41(4), 1690–1700 (2014)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Le, D.C., Zincir-Heywood, A.N., Heywood, M.I.: Data analytics on network traffic flows for botnet behaviour detection. In: 2016 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–7. IEEE (2016)Google Scholar
  14. 14.
    Li, B., Gunes, M.H., Bebis, G., Springer, J.: A supervised machine learning approach to classify host roles on line using sflow. In: Proceedings of the First Edition Workshop on High Performance and Programmable Networking, pp. 53–60. ACM (2013)Google Scholar
  15. 15.
    Lin, W.Y., Hu, Y.H., Tsai, C.F.: Machine learning in financial crisis prediction: a survey. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 42(4), 421–436 (2012)Google Scholar
  16. 16.
    Sheikhan, M., Jadidi, Z., Farrokhi, A.: Intrusion detection using reduced-size rnn based on feature grouping, neural computing and applications. Neural Comput. Appl. 21(6), 1185–1190 (2012)CrossRefGoogle Scholar
  17. 17.
    Martinez, E.E.B., Oh, B., Li, F., Luo, X.: Evading deep neural network and random forest classifiers by generating adversarial samples. In: Zincir-Heywood, N., Bonfante, G., Debbabi, M., Garcia-Alfaro, J. (eds.) FPS 2018. LNCS, vol. 11358, pp. 143–155. Springer, Cham (2019). Scholar
  18. 18.
    Moh’d A Mesleh, A.: Chi square feature extraction based Svms Arabic language text categorization system. J. Comput. Sci. 3(6), 430–435 (2007)Google Scholar
  19. 19.
    Östergård, P.R.: A fast algorithm for the maximum clique problem. Disc. Appl. Math. 120(1–3), 197–207 (2002)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z.B., Swami, A.: Practical black-box attacks against machine learning. In: Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, pp. 506–519. ACM (2017)Google Scholar
  21. 21.
    Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z.B., Swami, A.: The limitations of deep learning in adversarial settings. In: 2016 IEEE European Symposium on Security and Privacy (EuroS&P), pp. 372–387. IEEE (2016)Google Scholar
  22. 22.
    Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986)Google Scholar
  23. 23.
    Reddy, R.R., Ramadevi, Y., Sunitha, K.: Real time anomaly detection using ensembles. In: 2014 International Conference on Information Science and Applications (ICISA), pp. 1–4. IEEE (2014)Google Scholar
  24. 24.
    Shone, N., Ngoc, T.N., Phai, V.D., Shi, Q.: A deep learning approach to network intrusion detection. IEEE Trans. Emerg. Top. Computat. Intell. 2(1), 41–50 (2018)CrossRefGoogle Scholar
  25. 25.
    Tsai, C.F., Hsu, Y.F., Lin, C.Y., Lin, W.Y.: Intrusion detection by machine learning: A review. Exp. Syst. Appl. 36(10), 11994–12000 (2009)CrossRefGoogle Scholar
  26. 26.
    Vanerio, J., Casas, P.: Ensemble-learning approaches for network security and anomaly detection. In: Proceedings of the Workshop on Big Data Analytics and Machine Learning for Data Communication Networks, pp. 1–6. ACM (2017)Google Scholar
  27. 27.
    Vinayakumar, R., Soman, K., Poornachandran, P.: Applying convolutional neural network for network intrusion detection. In: 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 1222–1228. IEEE (2017)Google Scholar
  28. 28.
    Wang, Y.: A multinomial logistic regression modeling approach for anomaly intrusion detection. Comput. Secur. 24(8), 662–674 (2005)CrossRefGoogle Scholar
  29. 29.
    Wei, S., Mirkovic, J., Kissel, E.: Profiling and clustering internet hosts. DMIN 6, 269–75 (2006)Google Scholar
  30. 30.
    Xu, K., Wang, F., Gu, L.: Network-aware behavior clustering of internet end hosts. In: 2011 Proceedings of the IEEE INFOCOM, pp. 2078–2086. IEEE (2011)Google Scholar
  31. 31.
    Zhang, J., Zulkernine, M., Haque, A.: Random-forests-based network intrusion detection systems. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 38(5), 649–659 (2008)Google Scholar
  32. 32.
    Zhang, S., Yang, L.T., Kuang, L., Feng, J., Chen, J., Piuri, V.: A tensor-based forensics framework for virtualized network functions in the internet of things: Utilizing tensor algebra in facilitating more efficient network forensic investigations. IEEE Consum. Electron. Mag. 8(3), 23–27 (2019)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Engineering and TechnologyIndiana University-Purdue University IndianapolisIndianapolisUSA

Personalised recommendations