BoTShark: A Deep Learning Approach for Botnet Traffic Detection

  • Sajad Homayoun
  • Marzieh Ahmadzadeh
  • Sattar Hashemi
  • Ali DehghantanhaEmail author
  • Raouf Khayami
Part of the Advances in Information Security book series (ADIS, volume 70)


While botnets have been extensively studied, bot malware is constantly advancing and seeking to exploit new attack vectors and circumvent existing measures. Existing intrusion detection systems are unlikely to be effective countering advanced techniques deployed in recent botnets. This chapter proposes a deep learning-based botnet traffic analyser called Botnet Traffic Shark (BoTShark). BoTShark uses only network transactions and is independent of deep packet inspection technique; thus, avoiding inherent limitations such as the inability to deal with encrypted payloads. This also allows us to identify correlations between original features and extract new features in every layer of an Autoencoder or a Convolutional Neural Networks (CNNs) in a cascading manner. Moreover, we utilise a Softmax classifier as the predictor to detect malicious traffics efficiently.


Botnet Intrusion detection Network flows Deep learning Autoencoder CNNs 


  1. 1.
    Nikola Milosevic, Ali Dehghantanha, and Kim-Kwang Raymond Choo. Machine learning aided android malware classification. Computers & Electrical Engineering, feb 2017.
  2. 2.
    Malware statistics & trends report, feb 2017.
  3. 3.
    Mohsen Damshenas, Ali Dehghantanha, and Ramlan Mahmoud. A survey on malware propagation, analysis, and detection. International Journal of Cyber-Security and Digital Forensics (IJCSDF), 2 (4): 10–29, 2013.Google Scholar
  4. 4.
    Mohsen Damshenas, Ali Dehghantanha, Kim-Kwang Raymond Choo, and Ramlan Mahmud. M0droid: An android behavioral-based malware detection model. Journal of Information Privacy and Security, 11 (3): 141–157, jul 2015.
  5. 5.
    Hamed Haddad Pajouh, Reza Javidan, Raouf Khayami, Dehghantanha Ali, and Kim-Kwang Raymond Choo. A two-layer dimension reduction and two-tier classification model for anomaly-based intrusion detection in IoT backbone networks. IEEE Transactions on Emerging Topics in Computing, pages 1–1, 2016.
  6. 6.
    Gunter Ollmann. Botnet communication topologies understanding the intricacies of botnet command-and-control, 2009.
  7. 7.
    Anoop Chowdary Atluri and Vinh Tran. Botnets threat analysis and detection. In Information Security Practices, pages 7–28. Springer International Publishing, 2017.
  8. 8.
    Hossein Rouhani Zeidanloo, Mohammad Jorjor Zadeh Shooshtari, Payam Vahdani Amoli, M. Safari, and Mazdak Zamani. A taxonomy of botnet detection techniques. In 2010 3rd International Conference on Computer Science and Information Technology. IEEE, jul 2010.
  9. 9.
    C.C. Zou and R. Cunningham. Honeypot-aware advanced botnet construction and maintenance. In International Conference on Dependable Systems and Networks (DSN06). IEEE, 2006.
  10. 10.
    Junjie Zhang, Roberto Perdisci, Wenke Lee, Xiapu Luo, and Unum Sarfraz. Building a scalable system for stealthy p2p-botnet detection. IEEE Transactions on Information Forensics and Security, 9 (1): 27–38, jan 2014. Scholar
  11. 11.
    Yee-Yang Teing, Ali Dehghantanha, Kim-Kwang Raymond Choo, and Laurence T Yang. Forensic investigation of p2p cloud storage services and backbone for IoT networks: BitTorrent sync as a case study. Computers & Electrical Engineering, 58: 350–363, feb 2017.
  12. 12.
    Opeyemi Osanaiye, Haibin Cai, Kim-Kwang Raymond Choo, Ali Dehghantanha, Zheng Xu, and Mqhele Dlodlo. Ensemble-based multi-filter feature selection method for DDoS detection in cloud computing. EURASIP Journal on Wireless Communications and Networking, 2016 (1), may 2016.
  13. 13.
    Guofei Gu, Roberto Perdisci, Junjie Zhang, and Wenke Lee. Botminer: Clustering analysis of network traffic for protocol- and structure-independent botnet detection. In Proceedings of the 17th Conference on Security Symposium, SS’08, pages 139–154, Berkeley, CA, USA, 2008. USENIX Association.
  14. 14.
    David Zhao, Issa Traore, Bassam Sayed, Wei Lu, Sherif Saad, Ali Ghorbani, and Dan Garant. Botnet detection based on traffic behavior analysis and flow intervals. Computers & Security, 39: 2–16, nov 2013. Scholar
  15. 15.
    Basil AsSadhan, Jose M. F. Moura, and David Lapsley. Periodic behavior in botnet command and control channels traffic. In GLOBECOM 2009 - 2009 IEEE Global Telecommunications Conference. IEEE, nov 2009.
  16. 16.
  17. 17.
    Sérgio S.C. Silva, Rodrigo M.P. Silva, Raquel C.G. Pinto, and Ronaldo M. Salles. Botnets: A survey. Computer Networks, 57 (2): 378–403, feb 2013.
  18. 18.
    Igal Zeifman. 2015 bot traffic report: Humans take back the web, bad bots not giving any ground. Report, Incapsula, 9 Dec. 2015 2015.
  19. 19.
    Brett Stone-Gross, Marco Cova, Lorenzo Cavallaro, Bob Gilbert, Martin Szydlowski, Richard Kemmerer, Christopher Kruegel, and Giovanni Vigna. Your botnet is my botnet: Analysis of a botnet takeover. In Proceedings of the 16th ACM conference on Computer and communications security - CCS. ACM Press, 2009.
  20. 20.
    Carl Livadas, Robert Walsh, David Lapsley, and W. Strayer. Usilng machine learning technliques to identify botnet traffic. In Proceedings. 2006 31st IEEE Conference on Local Computer Networks. IEEE, nov 2006.
  21. 21.
    Chia Yuan Cho, Domagoj Babic, Eui Chul Richard Shin, and Dawn Song. Inference and analysis of formal models of botnet command and control protocols. In Proceedings of the 17th ACM conference on Computer and communications security - CCS. ACM Press, 2010.
  22. 22.
    Rouhani S. Zeidanloo HR. Botnet Detection by Monitoring Common Network Behaviors: Botnet detection by monitoring common network behaviors. Lambert Academic Publishing, 2012. ISBN 9783848404759.Google Scholar
  23. 23.
    Jing Wang and Ioannis Ch. Paschalidis. Botnet detection based on anomaly and community detection. IEEE Transactions on Control of Network Systems, pages 1–1, 2016.
  24. 24.
    Guofei Gu, Phillip Porras, Vinod Yegneswaran, and Martin Fong. Bothunter: Detecting malware infection through ids-driven dialog correlation. In 16th USENIX Security Symposium (USENIX Security 07), Boston, MA, 2007. USENIX Association.
  25. 25.
    Snort - network intrusion detection & prevention system, jan 2017.
  26. 26.
    Hyunsang Choi, Hanwoo Lee, Heejo Lee, and Hyogon Kim. Botnet detection by monitoring group activities in DNS traffic. In 7th IEEE International Conference on Computer and Information Technology (CIT 2007). IEEE, oct 2007.
  27. 27.
    G. Kirubavathi and R. Anitha. Botnet detection via mining of traffic flow characteristics. Computers & Electrical Engineering, 50: 91–101, feb 2016. Scholar
  28. 28.
    Elaheh Biglar Beigi, Hossein Hadian Jazi, Natalia Stakhanova, and Ali A. Ghorbani. Towards effective feature selection in machine learning-based botnet detection approaches. In 2014 IEEE Conference on Communications and Network Security. IEEE, oct 2014.
  29. 29.
    Kuan-Cheng Lin, Wei-Chiang Li, and Jason C. Hung. Detection for different type botnets using feature subset selection. In Lecture Notes in Electrical Engineering, pages 523–529. Springer Singapore, 2016.
  30. 30.
  31. 31.
    Adam J. Aviv and Andreas Haeberlen. Challenges in experimenting with botnet detection systems. In Proceedings of the 4th Conference on Cyber Security Experimentation and Test, CSET’11, pages 6–6, Berkeley, CA, USA, 2011. USENIX Association.
  32. 32.
    Jeff Heaton. Artificial Intelligence for Humans, volume 3. CreateSpace Independent Publishing Platform, 2015. ISBN 1505714346.Google Scholar
  33. 33.
    Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 (11): 2278–2324, 1998. Scholar
  34. 34.
    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 1097–1105. Curran Associates, Inc., 2012.Google Scholar
  35. 35.
    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521 (7553): 436–444, may 2015.
  36. 36.
    Michael Blot, Matthieu Cord, and Nicolas Thome. Max-min convolutional neural networks for image classification. In 2016 IEEE International Conference on Image Processing (ICIP). IEEE, sep 2016.
  37. 37.
    Argus- auditing network activity, jan 2017.
  38. 38.
    Huy Hang, Xuetao Wei, M. Faloutsos, and T. Eliassi-Rad. Entelecheia: Detecting p2p botnets in their waiting stage. In 2013 IFIP Networking Conference, pages 1–9, May 2013.Google Scholar
  39. 39.
    Bishop Christopher. Pattern Recognition and Machine Learning. Springer-Verlag New York, 1 edition, 2006.Google Scholar
  40. 40.
    Sajad Homayoun, Ali Dehghantanha, Marzieh Ahmadzadeh, Sattar Hashemi, and Raouf Khayami. Know abnormal, find evil: Frequent pattern mining for ransomware threat hunting and intelligence. IEEE Transactions on Emerging Topics in Computing, pages 1–1, 2017 - In Press.
  41. 41.
    Mina Sohrabi, Mohammad M. Javidi, and Sattar Hashemi. Detecting intrusion transactions in database systems: a novel approach. Journal of Intelligent Information Systems, 42 (3): 619–644, dec 2013.
  42. 42.
    R. Mohammadi, R. Javidan, and M. Conti. Slicots: An sdn-based lightweight countermeasure for tcp syn flooding attacks. IEEE Transactions on Network and Service Management, 14 (2): 487–497, June 2017. ISSN 1932-4537. Scholar
  43. 43.
    Yet another flowmeter, jan 2017.
  44. 44.
    Gerard Drapper Gil, Arash Habibi Lashkari, Mohammad Mamun, and Ali A. Ghorbani. Characterization of encrypted and vpn traffic using time-related features. In Proceedings of the 2nd International Conference on Information Systems Security and Privacy ICISSP 2016, pages 407–414, 2016.Google Scholar
  45. 45.
    Yee-Yang Teing, Ali Dehghantanha, Kim-Kwang Raymond Choo, Tooska Dargahi, and Mauro Conti. Forensic investigation of cooperative storage cloud service: Symform as a case study. Journal of Forensic Sciences, 62 (3): 641–654, nov 2016.
  46. 46.
    Steve Watson and Ali Dehghantanha. Digital forensics: the missing piece of the internet of things promise. Computer Fraud & Security, 2016 (6): 5–8, jun 2016.

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Sajad Homayoun
    • 1
  • Marzieh Ahmadzadeh
    • 1
  • Sattar Hashemi
    • 2
  • Ali Dehghantanha
    • 3
    Email author
  • Raouf Khayami
    • 1
  1. 1.Department of Computer Engineering and Information TechnologyShiraz University of TechnologyShirazIran
  2. 2.Department of Computer EngineeringShiraz UniversityShirazIran
  3. 3.Department of Computer ScienceUniversity of SheffieldSheffieldUK

Personalised recommendations