Journal of Network and Systems Management

, Volume 26, Issue 3, pp 616–639 | Cite as

Cluster Ensemble with Link-Based Approach for Botnet Detection

  • Long Mai
  • Dong Kun NohEmail author


Botnet detection is one of the most imminent tasks for cyber security. Among popular botnet countermeasures, an intrusion detection system is the prominent mechanism. In the past, packet-based intrusion detection systems were popular. However, flow-based intrusion detection systems have been preferred in recent years due to their ability to adapt to modern high-speed networks. A collection of flows from an enterprise network usually contains both botnet traffic and normal traffic. To classify this traffic, supervised machine learning algorithms, i.e., classifications, have been applied and achieved a high accuracy. In an effort to improve the ability of intrusion detection systems against botnets, some studies have suggested partitioning flows into clusters before applying the classifications and this step could significantly reduce the complexity of a flow set. However, the instability of individual clustering algorithms is still a constraint for botnet detection.To overcome this bottleneck, we propose a novel method that combines individual partitions to become a strong learner through the use of a link-based algorithm. Our experiments show that our cluster ensemble model outperforms existing botnet detection mechanisms with a high reliability. We also determine the balance between accuracy and computer resources for botnet detection, and thereby propose a range for the maximum duration time of flows in botnet research.


Cyber crime Intrusion detection system Network flow Machine learning Classification Command and control 



This research was supported by the Ministry of Science, ICT and Future Planning (MSIP), Korea, under the Information Technology Research Center (ITRC) support Program (IITP-2017-2012-0-00646) supervised by the Institute for Information and Communications Technology Promotion (IITP).


  1. 1.
  2. 2.
  3. 3.
    Gu, G., Porras, P.A., Yegneswaran, V., Fong, M.W., Lee, W.: Bothunter: detecting malware infection through ids-driven dialog correlation. In: Usenix Security, vol. 7, pp. 1–16 (2007)Google Scholar
  4. 4.
    Roesch, M., et al.: Snort: lightweight intrusion detection for networks. LISA 99, 229–238 (1999)Google Scholar
  5. 5.
    Sperotto, A., Schaffrath, G., Sadre, R., Morariu, C., Pras, A., Stiller, B.: An overview of ip flow-based intrusion detection. IEEE Commun. Surv. Tutor. 12(3), 343–356 (2010)CrossRefGoogle Scholar
  6. 6.
    Stevanovic, M., Pedersen, J.M.: An efficient flow-based botnet detection using supervised machine learning. In: 2014 International Conference on Computing, Networking and Communications (ICNC), pp. 797–801. IEEE (2014)Google Scholar
  7. 7.
    Haddadi, F., Zincir-Heywood, A.N.: Botnet detection system analysis on the effect of botnet evolution and feature representation. In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 893–900. ACM (2015)Google Scholar
  8. 8.
    Al-Jarrah, O.Y., Alhussein, O., Yoo, P.D., Muhaidat, S., Taha, K., Kim, K.: Data randomization and cluster-based partitioning for botnet intrusion detection. IEEE Trans. Cybern. 46(8), 1796–1806 (2016)CrossRefGoogle Scholar
  9. 9.
    Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, Los Altos (2005)zbMATHGoogle Scholar
  10. 10.
    Iam-On, N., Boongoen, T., Garrett, S., Price, C.: A link-based approach to the cluster ensemble problem. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2396–2409 (2011)CrossRefGoogle Scholar
  11. 11.
    Zhao, D., Traore, I., Sayed, B., Lu, W., Saad, S., Ghorbani, A., Garant, D.: Botnet detection based on traffic behavior analysis and flow intervals. Comput. Secur. 39, 2–16 (2013)CrossRefGoogle Scholar
  12. 12.
    Silva, S.S.C., Silva, R.M.P., Pinto, R.C.G., Salles, R.M.: Botnets: a survey. Comput. Netw. 57(2), 378–403 (2013)CrossRefGoogle Scholar
  13. 13.
    Saad, S., Traore, I., Ghorbani, A., Sayed, B., Zhao, D., Lu, W., Felix, J., Hakimian, P.: Detecting p2p botnets through network behavior analysis and machine learning. In: 2011 Ninth Annual International Conference on Privacy, Security and Trust (PST), pp. 174–180. IEEE (2011)Google Scholar
  14. 14.
    Shiravi, A., Shiravi, H., Tavallaee, M., Ghorbani, A.A.: Toward developing a systematic approach to generate benchmark datasets for intrusion detection. Comput. Secur. 31(3), 357–374 (2012)CrossRefGoogle Scholar
  15. 15.
    Garcia, S., Grill, M., Stiborek, J., Zunino, A.: An empirical comparison of botnet detection methods. Comput. Secur. 45, 100–123 (2014)CrossRefGoogle Scholar
  16. 16.
    Haddadi, F., Zincir-Heywood, A.N.: Benchmarking the effect of flow exporters and protocol filters on botnet traffic classification. IEEE Sys. J. 10(4), 1390–1401 (2016)CrossRefGoogle Scholar
  17. 17.
    Jain, A.K.: Data clustering: 50 years beyond k-means. Pattern Recogn. Lett. 31(8), 651–666 (2010)CrossRefGoogle Scholar
  18. 18.
    Kuncheva, L.I., Hadjitodorov, S.T.: Using diversity in cluster ensembles. In: 2004 IEEE International Conference on Systems, Man and Cybernetics, vol. 2, pp. 1214–1219. IEEE (2004)Google Scholar
  19. 19.
    Strehl, A., Ghosh, J.: Cluster ensembles—a knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3(Dec), 583–617 (2002)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Topchy, A., Jain, A.K., Punch, W.: Clustering ensembles: models of consensus and weak partitions. IEEE Trans. Pattern Anal. Mach. Intell. 27(12), 1866–1881 (2005)CrossRefGoogle Scholar
  21. 21.
    Kuncheva, L.I., Vetrov, D.P.: Evaluation of stability of k-means cluster ensembles with respect to random initialization. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1798–1808 (2006)CrossRefGoogle Scholar
  22. 22.
    Fern, X.Z., Lin, W.: Cluster ensemble selection. Stat. Anal. Data Min. 1(3), 128–141 (2008)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Zhou, Z.-H.: Ensemble Methods: Foundations and Algorithms. CRC press, Boca Raton (2012)Google Scholar
  24. 24.
    Beigi, E.B., Jazi, H.H., Stakhanova, N., Ghorbani, A.A.: Towards effective feature selection in machine learning-based botnet detection approaches. In: 2014 IEEE Conference on Communications and Network Security (CNS), pp. 247–255. IEEE (2014)Google Scholar
  25. 25.
    The honeynet project. French chapter. (2011)
  26. 26.
    Szabó, G., Orincsay, D., Malomsoky, S., Szabó, I.: On the validation of traffic classification algorithms. In: International Conference on Passive and Active Network Measurement, pp. 72–81. Springer (2008)Google Scholar
  27. 27.
    Lawrence Berkeley National Laboratory and ICSI: LBNL/ICSI enterprise tracing project. LBNL enterprise trace repository. (2005)
  28. 28.
    Claise, B.: Specification of the IP flow information export (IPFIX) protocol for the exchange of IP traffic flow information. Technical report (2008)Google Scholar
  29. 29.
    Sadasivan, G., Brownlee, N., Claise, B., Quittek, J.: Architecture for IP flow information export. RFC 5470 (2009)Google Scholar
  30. 30.
  31. 31.
    Soysal, M., Schmidt, E.G.: Machine learning algorithms for accurate flow-based network traffic classification: evaluation and comparison. Perform. Eval. 67(6), 451–467 (2010)CrossRefGoogle Scholar
  32. 32.
    Loh, W.-Y.: Classification and regression trees. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 1(1), 14–23 (2011)CrossRefGoogle Scholar
  33. 33.
    Quinlan, J.R.: C4. 5: Programs for Machine Learning. Elsevier, New York (2014)Google Scholar
  34. 34.
    Rokach, L., Maimon, O.: Data Mining with Decision Trees: Theory and Applications. World Scientific, Singapore (2014)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Department of Information Communication, Materials, and Chemistry Convergence, TechnologySoongsil UniversitySeoulKorea
  2. 2.Department of Software ConvergenceSoongsil UniversitySeoulKorea

Personalised recommendations