BotSpot: fast graph based identification of structured P2P bots

  • Bharath Venkatesh
  • Sudip Hazra Choudhury
  • Shishir Nagaraja
  • N. Balakrishnan
Original Paper

Abstract

An essential component of a botnet is the Command and Control (C2) channel (a network). The mechanics of C2 establishment often involve the use of structured overlay techniques which create a scaffolding for sophisticated coordinated activities. However, it can also be used as a point of detection because of their distinct communication patterns. Achieving this is a needle-in-a-haystack search problem across distributed vantage points. The search technique must be efficient given the high traffic throughput of modern core routers. In this paper, we focus on efficient algorithms for C2 channel detection. Experimental results on real Internet traffic traces from an ISP’s backbone network indicate that our techniques, (i) have time complexity linear in the volume of traffic, (ii) have high F-measure, and (iii) are robust to the partial visibility arising from partial deployment of monitoring systems, and measurement inaccuracies arising from partial visibility and dynamics of background traffic.

References

  1. 1.
    Abu Rajab, M., Zarfoss, J., Monrose, F., Terzis, A.: A multifaceted approach to understanding the botnet phenomenon. In: MC ’06 Proceedings of the 6th ACM SIGCOMM on Internet Measurement, pp. 41–52. New York, New York, USA, Oct. 2006. ACM Press. ISBN 1595935614. doi:10.1145/1177080.1177086. http://dl.acm.org/citation.cfm?id=1177080.1177086. http://portal.acm.org/citation.cfm?doid=1177080.1177086
  2. 2.
    Antonakakis, M., Perdisci, R., Nadji, Y., Vasiloglou II N., Abu-Nimeh, S., Lee, W., Dagon, D.: From throw-away traffic to bots: detecting the rise of dga-based malware. In: USENIX Security Symposium, pp. 491–506. (2012)Google Scholar
  3. 3.
    Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. Science 286(5439), 509–512 (1999)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Binkley, J.R., Singh, S.: An algorithm for anomaly-based botnet detection. In: Proceedings of USENIX Steps to Reducing Unwanted Traffic on the Internet Workshop (SRUTI), pp. 43–48 (2006)Google Scholar
  5. 5.
    Biryukov, A., Pustogarov, I., Weinmann, R.: Trawling for tor hidden services: Detection, measurement, deanonymization. In: 2013 IEEE Symposium on Security and Privacy (SP), pp. 80–94. IEEE (2013)Google Scholar
  6. 6.
    Blondel, V.D., Guillaume, J.-L., Lambiotte, R., Lefebvre, E.: Fast unfolding of communities in large networks. J. Stat. Mech. Theory Exp. 2008(10):6 (2008). ISSN 1742-5468. doi:10.1088/1742-5468/2008/10/P10008. http://stacks.iop.org/1742-5468/2008/i=10/a=P10008?key=crossref.46968f6ec61eb8f907a760be1c5ace52.arXiv:0803.0476
  7. 7.
    Browet, A., Absil, P.-A., Van Dooren, P.: Fast community detection using local neighbourhood search (2013). arXiv preprint. arXiv:1308.6276
  8. 8.
    Choi, H., Lee, H., Kim, H.: Botgad: detecting botnets by capturing group activities in network traffic. In: Proceedings of the Fourth International ICST Conference on COMmunication System softWAre and middlewaRE, pp. 2–10. ACM (2009)Google Scholar
  9. 9.
    Collins, M.P., Reiter, M.K.: Hit-list worm detection and bot identification in large networks using protocol graphs. In: RAID’07 Proceedings of the 10th International Conference on Recent Advances in Intrusion Detection, pp. 276–295. (2007). ISBN 3-540-74319-7, 978-3-540-74319-4. http://dl.acm.org/citation.cfm?id=1776434.1776456
  10. 10.
    Cooke, E., Jahanian, F., McPherson, D.: The Zombie roundup: understanding, detecting, and disrupting botnets. In: SRUTI’05 Proceedings of the Steps to Reducing Unwanted Traffic on the Internet on Steps to Reducing Unwanted Traffic on the Internet Workshop, pp. 6–12 (2005). http://dl.acm.org/citation.cfm?id=1251282.1251288
  11. 11.
    Coscia, M., Giannotti, F., Pedreschi, D.: A classification for community discovery methods in complex networks. Stat. Anal. Data Min. ASA Data Sci. J. 4(5), 512–546 (2011)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Coskun, B., Dietrich, S., Memon, N.: Friends of an enemy. In: ACSAC ’10 Proceedings of the 26th Annual Computer Security Applications Annual Conference, pp. 131–140, New York, New York, USA, Dec. 2010. ACM Press. ISBN 9781450301336. doi:10.1145/1920261.1920283. http://dl.acm.org/citation.cfm?id=1920261.1920283
  13. 13.
    Dagon, D., Gu, G., Lee, C.P., Lee, W.: A taxonomy of botnet structures. In: ACSAC ’07 Proceedings of the 23rd Annual Computer Security Applications Annual Conference, pp. 325–339. IEEE, Dec. 2007. ISBN 0-7695-3060-5. doi:10.1109/ACSAC.2007.44. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4413000. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4413000
  14. 14.
    Davis, C.R., Neville, S., Fernandez, J.M., Robert, J.-M., Mchugh, J.: Structured peer-to-peer overlay networks: ideal botnets command and control infrastructures? In: Jajodia, S., Lopez, J. (eds.) ESORICS ’08 Proceedings of the 13th European Symposium on Research in Computer Security: Computer Security. Lecture Notes in Computer Science, vol. 5283, pp. 461–480, Berlin, Heidelberg, Oct. 2008. Springer, Berlin. ISBN 978-3-540-88312-8. doi:10.1007/978-3-540-88313-5. http://dl.acm.org/citation.cfm?id=1462455.1462495. http://www.springerlink.com/index/10.1007/978-3-540-88313-5
  15. 15.
    Erdos, P., Rényi, A.: On the evolution of random graphs. Publ. Math. Inst. Hung. Acad. Sci. 5, 17–61 (1960)MathSciNetMATHGoogle Scholar
  16. 16.
    Fortunato, S.: Community detection in graphs. Phys. Rep. 486(3–5), 75–174, (2010). ISSN 03701573. doi:10.1016/j.physrep.2009.11.002. http://linkinghub.elsevier.com/retrieve/pii/S0370157309002841
  17. 17.
    François, J., Wang, S., State, R., Engel, T.: BotTrack: tracking botnets using NetFlow and PageRank. In: NETWORKING ’11 Proceedings of the 10th International IFIP TC 6 Conference on Networking, pp. 1–14, May 2011. ISBN 978-3-642-20756-3. doi:10.1007/978-3-642-20757-0_1. http://dl.acm.org/citation.cfm?id=2008780.2008782. http://hal.inria.fr/docs/00/61/35/97/PDF/networking11CR.pdf
  18. 18.
    Freiling, F.C., Holz, T., Wicherski, G.: Botnet tracking: exploring a root-cause methodology to prevent distributed denial-of-service attacks. In: Vimercati, S.D.C., Syverson, P., Gollmann, D. (eds.) ESORICS’05 Proceedings of the 10th European Conference on Research in Computer Security. Lecture Notes in Computer Science, vol. 3679, pp. 319–335. Berlin, Heidelberg, Sept. 2005. Springer, Berlin. ISBN 978-3-540-28963-0. doi:10.1007/11555827. http://dl.acm.org/citation.cfm?id=2156732.2156751
  19. 19.
    Gardiner, J., Cova, M., Nagaraja, S.: Command & control: understanding, denying and detecting (2014). arXiv preprint. arXiv:1408.1136
  20. 20.
    Golovanov, S., Soumenkov, I.: TDL4-Top Bot (2011). http://securelist.com/analysis/36152/tdl4-top-bot/
  21. 21.
    Gu, G., Porras, P., Yegneswaran, V., Fong, M., Lee, W.: BotHunter: detecting malware infection through IDS-driven dialog correlation. In: USENIX Security ’07 Proceedings of the 16th USENIX Security Symposium, pp. 1–16. Aug. 2007. ISBN 111-333-5555-77-9. http://dl.acm.org/citation.cfm?id=1362903.1362915
  22. 22.
    Gu, G., Perdisci, R., Zhang, J., Lee, W.: BotMiner: clustering analysis of network traffic for protocol- and structure-independent botnet detection. In: USENIX Security ’08 Proceedings of the 17th USENIX Security Symposium, pp. 139–154, July 2008. http://dl.acm.org/citation.cfm?id=1496711.1496721
  23. 23.
    Gu, G., Zhang, J., Lee, W.: BotSniffer: Detecting botnet command and control channels in network traffic. In: NDSS ’08 Proceedings of the 15th Annual Network and Distributed System Security Symposium, pp. 1–18. Citeseer (2008). http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.110.8092&rep=rep1&type=pdf. http://users.csc.tntech.edu/~weberle/Fall2008/CSC6910/Papers/17_botsniffer_detecting_botnet.pdf
  24. 24.
    Hang, H., Wei, X., Faloutsos, M., Eliassi-Rad, T.: Entelecheia: detecting p2p botnets in their waiting stage. In: IFIP Networking Conference, 2013, pp. 1–9. IEEE (2013)Google Scholar
  25. 25.
    Iliofotou, M., Kim, H.-C., Faloutsos, M., Mitzenmacher, M., Pappu, P., Varghese, G.: Graption: a graph-based P2P traffic classification framework for the internet backbone. Comput. Netw. 55(8):1909–1920 (2011). ISSN 13891286. doi:10.1016/j.comnet.2011.01.020. http://dl.acm.org/citation.cfm?id=1982705.1983058
  26. 26.
    Jaikumar, P., Kak, A.C.: A graph-theoretic framework for isolating botnets in a network. Secur. Commun. Netw. (2012). ISSN 19390114. doi:10.1002/sec.500. http://onlinelibrary.wiley.com/doi/10.1002/sec.500/full. http://doi.wiley.com/10.1002/sec.500
  27. 27.
    Jiang, H., Shao, X.: Detecting P2P botnets by discovering flow dependency in C&C traffic. Peer-to-Peer Netw. Appl. (2012). ISSN 1936-6442. doi:10.1007/s12083-012-0150-x. http://www.springerlink.com/index/10.1007/s12083-012-0150-x
  28. 28.
    Kaashoek, M.F., Karger, D.R.: Koorde: a simple degree-optimal distributed hash table. In: Peer-to-Peer Systems II, pp. 98–107. Springer, Berlin (2003)Google Scholar
  29. 29.
    Li, L., Mathur, S., Coskun, B.: Gangs of the internet: towards automatic discovery of peer-to-peer communities. In: 2013 IEEE Conference on Communications and Network Security (CNS), pp. 64–72. IEEE (2013)Google Scholar
  30. 30.
    Lu, W., Tavallaee, M., Ghorbani, A.A.: Automatic discovery of botnet communities on large-scale communication networks. In: ASIACCS ’09 Proceedings of the 4th International Symposium on Information, Computer, and Communications Security, pp. 1–10, New York, New York, USA, Mar. 2009. ACM Press. ISBN 9781605583945. doi:10.1145/1533057.1533062. http://dl.acm.org/citation.cfm?id=1533057.1533062
  31. 31.
    Maymounkov, P., Mazieres, D.: Kademlia: a peer-to-peer information system based on the xor metric. In: Peer-to-Peer Systems, pp. 53–65. Springer, Berlin (2002)Google Scholar
  32. 32.
    Nagaraja, S., Mittal, P., Hong, C.-Y., Caesar, M., Borisov, N.: BotGrep: finding P2P bots with structured graph analysis. In: USENIX Security’10 Proceedings of the 19th USENIX Security Symposium, p. 7, Aug. 2010. ISBN 888-7-6666-5555-4. doi:10.1.1.172.8756. http://static.usenix.org/event/sec10/tech/full_papers/Nagaraja.pdf
  33. 33.
    Nelms, T., Perdisci, R., Ahamad, M.: Execscent: mining for new c&c domains in live networks with adaptive control protocol templates. In: USENIX Security, pp. 589–604 (2013)Google Scholar
  34. 34.
    Newman, M.E.: Modularity and community structure in networks. Proc. Natl. Acad. Sci. 103(23), 8577–8582 (2006)CrossRefGoogle Scholar
  35. 35.
    Rahbarinia, B., Perdisci, R., Lanzi, A., Li, K.: Peerrush: mining for unwanted p2p traffic. J. Inf. Secur. Appl. 19(3), 194–208 (2014)Google Scholar
  36. 36.
    Rossow, C., Andriesse, D., Werner, T., Stone-Gross, B., Plohmann, D., Dietrich, C.J., Bos, H.: Sok: P2pwned-modeling and evaluating the resilience of peer-to-peer botnets. In: 2013 IEEE Symposium on Security and Privacy (SP), pp. 97–111. IEEE (2013)Google Scholar
  37. 37.
    Schaeffer, S.E.: Graph clustering. Comput. Sci. Rev. 1(1), 27–64 (2007). ISSN 15740137. doi:10.1016/j.cosrev.2007.05.001. http://linkinghub.elsevier.com/retrieve/pii/S1574013707000020
  38. 38.
    Schiavoni, S., Maggi, F., Cavallaro, L., Zanero, S.: Phoenix: Dga-based botnet tracking and intelligence. In: Detection of Intrusions and Malware, and Vulnerability Assessment, pp. 192–211. Springer, Berlin (2014)Google Scholar
  39. 39.
    Schonewille, A., van Helmond, D.-J.: The domain name service as an IDS. Research Project for the Master System-and Network Engineering at the University of Amsterdam (2006)Google Scholar
  40. 40.
    Silva, S.S., Silva, R.M., Pinto, R.C., Salles, R.M.: Botnets: a survey. Comput. Netw. (2012). ISSN 13891286. doi:10.1016/j.comnet.2012.07.021. http://linkinghub.elsevier.com/retrieve/pii/S1389128612003568
  41. 41.
    Stinson, E., Mitchell, J.C.: Towards systematic evaluation of the evadability of bot/botnet detection methods. In: WOOT’08 Proceedings of the 2nd Conference on USENIX Workshop on Offensive Technologies, pp. 1–9, July 2008. http://dl.acm.org/citation.cfm?id=1496702.1496707
  42. 42.
    Stoica, I., Morris, R., Karger, D., Kaashoek, M.F., Balakrishnan, H.: Chord: a scalable peer-to-peer lookup service for internet applications. In: ACM SIGCOMM Computer Communication Review, vol. 31, pp. 149–160. ACM (2001)Google Scholar
  43. 43.
    Strayer, W., Walsh, R., Livadas, C., Lapsley, D.: Detecting Botnets with Tight Command and Control. In: Proceedings. 2006 31st IEEE Conference on Local Computer Networks, pp. 195–202. IEEE, Nov. 2006. ISBN 1-4244-0418-5. doi:10.1109/LCN.2006.322100. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4116547
  44. 44.
    van Laarhoven, T., Marchiori, E.: Graph clustering with local search optimization: the resolution bias of the objective function matters most. Phys. Rev. E Stat. Nonlinear Soft Matter Phys. 87(1), 012812 (2013). ISSN 1550-2376. http://www.ncbi.nlm.nih.gov/pubmed/23410393
  45. 45.
    Walsworth, C., Aben, E., Claffy, K., Andersen, D.: The CAIDA UCSD anonymized internet traces (2011). http://www.caida.org/data/passive/passive_2011_dataset.xml. Accessed 8 Mar 2015
  46. 46.
    Yen, T.-F., Reiter, M.K.: Are your hosts trading or plotting? Telling P2P file-sharing and bots apart. In: ICDCS ’10 IEEE 30th International Conference on Distributed Computing Systems, pp. 241–252. IEEE, June 2010. ISBN 978-1-4244-7261-1. doi:10.1109/ICDCS.2010.76. http://dl.acm.org/citation.cfm?id=1845878.1846291. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5541681
  47. 47.
    Yen, T.-F., Reiter, M.K.: Revisiting botnet models and their implications for takedown strategies. In: Degano, P., Guttman, J.D. (eds.) POST’12 Proceedings of the First International Conference on Principles of Security and Trust. Lecture Notes in Computer Science, vol. 7215, pp. 249–268. Berlin, Heidelberg, Mar. 2012. Springer, Berlin. ISBN 978-3-642-28640-7. doi:10.1007/978-3-642-28641-4. http://dl.acm.org/citation.cfm?id=2260577.2260591. http://www.springerlink.com/index/10.1007/978-3-642-28641-4
  48. 48.
    Zhang, J., Perdisci, R., Lee, W., Sarfraz, U., Luo, X.: Detecting stealthy P2P botnets using statistical traffic fingerprints. In: 2011 IEEE/IFIP 41st International Conference on Dependable Systems and Networks (DSN), pp. 121–132. IEEE, June 2011. ISBN 978-1-4244-9232-9. doi:10.1109/DSN.2011.5958212. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5958212. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5958212
  49. 49.
    Zhang, J., Perdisci, R., Lee, W., Luo, X., Sarfraz, U.: Building a scalable system for stealthy p2p-botnet detection. IEEE Trans. Inf. Forensics Secur. 9(1), 27–38 (2014)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag France 2015

Authors and Affiliations

  • Bharath Venkatesh
    • 1
  • Sudip Hazra Choudhury
    • 1
  • Shishir Nagaraja
    • 2
  • N. Balakrishnan
    • 1
  1. 1.Supercomputer Education Research CentreIndian Institute of ScienceBangaloreIndia
  2. 2.School of Computing and CommunicationsLancaster UniversityLancasterUK

Personalised recommendations