Fast Detection of Heavy Hitters in Software Defined Networking Using an Adaptive and Learning Method

  • Zhiliang WangEmail author
  • Changping Zhou
  • Yang Yu
  • Xingang Shi
  • Xia Yin
  • Jiangyuan Yao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11065)


Heavy Hitters refer to the set of flows that represent a significantly large proportion of the link capacity or of the active traffic. Identifying Heavy Hitters is of particular importance in both network management and security applications. Traditional methods are focusing on sampling in the middle box and analyzing those packets using streaming algorithms. The paradigm of Software Defined Network (SDN) simplifies the work of flow counting. However, continuously monitoring the network will introduce overhead, which needs to be considered as a tradeoff between accurate measurement in real-time. In this paper, We propose a novel method that stamps each suspicious flow with a weight based on an online learning algorithm. The granularity of measurement is dynamically changed according to the importance of each flow. We take advantage of history flows to make the procedure of finding a heavy hitter faster so that applications can make decisions instantly. Using real-world data, we show that our online learning method can detect heavy hitters faster with less overhead and the same accuracy.


Software-defined-network Heavy hitter Online learning 



This work is supported by Hainan Provincial Natural Science Foundation of China (618QN219) and the National High Technology Research and Development Program of China (863 Program) No. 2015AA016105.


  1. 1.
    The CAIDA UCSD anonymized internet traces 2013.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
    Software defined networks: the new norm of networks.
  7. 7.
  8. 8.
    Argyropoulos, C., Kalogeras, D., Androulidakis, G., Maglaris, V.: PaFloMon-a slice aware passive flow monitoring framework for openflow enabled experimental facilities. In: 2012 European Workshop on Software Defined Networking (EWSDN), pp. 97–102. IEEE (2012)Google Scholar
  9. 9.
    Bandi, N., Metwally, A., Agrawal, D., El Abbadi, A.: Fast data stream algorithms using associative memories. In: Proceedings of the 2007 ACM SIGMOD international conference on Management of data, pp. 247–256. ACM (2007)Google Scholar
  10. 10.
    Cho, K.: Recursive lattice search: hierarchical heavy hitters revisited. In: Proceedings of the 2017 Internet Measurement Conference, pp. 283–289. ACM (2017)Google Scholar
  11. 11.
    Chowdhury, S.R., Bari, M.F., Ahmed, R., Boutaba, R.: Payless: a low cost network monitoring framework for software defined networks. In: Network Operations and Management Symposium (NOMS), 2014 IEEE, pp. 1–9. IEEE (2014)Google Scholar
  12. 12.
    Claise, B.: Cisco systems netflow services export version 9 (2004)Google Scholar
  13. 13.
    Cormode, G., Hadjieleftheriou, M.: Methods for finding frequent items in data streams. VLDB J. 19(1), 3–20 (2010)CrossRefGoogle Scholar
  14. 14.
    Cormode, G., Johnson, T., Korn, F., Muthukrishnan, S., Spatscheck, O., Srivastava, D.: Holistic UDAFs at streaming speeds. In: Proceedings of the 2004 ACM SIGMOD International Conference on Management of Data, pp. 35–46. ACM (2004)Google Scholar
  15. 15.
    Cormode, G., Korn, F., Muthukrishnan, S., Srivastava, D.: Finding hierarchical heavy hitters in data streams. In: Proceedings 2003 VLDB Conference, pp. 464–475. Elsevier (2003)Google Scholar
  16. 16.
    Da Cruz, M.A., e Silva, L.C., Correa, S., Cardoso, K.V.: Accurate online detection of bidimensional hierarchical heavy hitters in software-defined networks. In: 2013 IEEE Latin-America Conference on Communications (LATINCOM), pp. 1–6. IEEE (2013)Google Scholar
  17. 17.
    Curtis, A.R., Mogul, J.C., Tourrilhes, J., Yalagandula, P., Sharma, P., Banerjee, S.: DevoFlow: scaling flow management for high-performance networks. In: ACM SIGCOMM Computer Communication Review, vol. 41, pp. 254–265. ACM (2011)Google Scholar
  18. 18.
    Handigol, N., Heller, B., Jeyakumar, V., Lantz, B., McKeown, N.: Reproducible network experiments using container-based emulation. In: Proceedings of the 8th International Conference on Emerging Networking Experiments and Technologies, pp. 253–264. ACM (2012)Google Scholar
  19. 19.
    Huici, F., Di Pietro, A., Trammell, B., Gomez Hidalgo, J.M., Martinez Ruiz, D., d’Heureuse, N.: Blockmon: a high-performance composable network traffic measurement system. ACM SIGCOMM Comput. Commun. Rev. 42(4), 79–80 (2012)CrossRefGoogle Scholar
  20. 20.
    Locher, T.: Finding heavy distinct hitters in data streams. In: Proceedings of the Twenty-third Annual ACM Symposium on Parallelism in Algorithms and Architectures, pp. 299–308. ACM (2011)Google Scholar
  21. 21.
    Malboubi, M., Wang, L., Chuah, C.N., Sharma, P.: Intelligent SDN based traffic (de) aggregation and measurement paradigm (iSTAMP). In: INFOCOM, 2014 Proceedings IEEE, pp. 934–942. IEEE (2014)Google Scholar
  22. 22.
    Moshref, M., Yu, M., Govindan, R.: Resource/accuracy tradeoffs in software-defined measurement. In: Proceedings of the Second ACM SIGCOMM Workshop on Hot topics in Software Defined Networking, pp. 73–78. ACM (2013)Google Scholar
  23. 23.
    Shirali-Shahreza, S., Ganjali, Y.: Flexam: flexible sampling extension for monitoring and security applications in openflow. In: Proceedings of the Second ACM SIGCOMM Workshop on Hot topics in Software Defined Networking, pp. 167–168. ACM (2013)Google Scholar
  24. 24.
    Su, Z., Wang, T., Xia, Y., Hamdi, M.: Flowcover: Low-cost flow monitoring scheme in software defined networks. In: Global Communications Conference (GLOBECOM), 2014 IEEE, pp. 1956–1961. IEEE (2014)Google Scholar
  25. 25.
    Thottan, M., Liu, G., Ji, C.: Anomaly detection approaches for communication networks. In: Cormode, G., Thottan, M. (eds.) Algorithms for Next Generation Networks. Computer Communications and Networks, pp. 239–261. Springer, London (2010). Scholar
  26. 26.
    Tootoonchian, A., Ghobadi, M., Ganjali, Y.: OpenTM: Traffic matrix estimator for openflow networks. In: Krishnamurthy, A., Plattner, B. (eds.) PAM 2010. LNCS, vol. 6032, pp. 201–210. Springer, Heidelberg (2010). Scholar
  27. 27.
    Van Adrichem, N.L., Doerr, C., Kuipers, F.A.: Opennetmon: network monitoring in openflow software-defined networks. In: Network Operations and Management Symposium (NOMS), 2014 IEEE, pp. 1–8. IEEE (2014)Google Scholar
  28. 28.
    Yang, L., Ng, B., Seah, W.K.: Heavy hitter detection and identification in software defined networking. In: 2016 25th International Conference on Computer Communication and Networks (ICCCN), pp. 1–10. IEEE (2016)Google Scholar
  29. 29.
    Yu, C., Lumezanu, C., Zhang, Y., Singh, V., Jiang, G., Madhyastha, H.V.: FlowSense: monitoring network utilization with zero measurement cost. In: Roughan, M., Chang, R. (eds.) PAM 2013. LNCS, vol. 7799, pp. 31–41. Springer, Heidelberg (2013). Scholar
  30. 30.
    Yu, M., Jose, L., Miao, R.: Software defined traffic measurement with opensketch. In: NSDI, vol. 13, pp. 29–42 (2013)Google Scholar
  31. 31.
    Yuan, L., Chuah, C.N., Mohapatra, P.: ProgME: towards programmable network measurement. IEEE/ACM Trans. Netw. (TON) 19(1), 115–128 (2011)CrossRefGoogle Scholar
  32. 32.
    Zhang, Y.: An adaptive flow counting method for anomaly detection in SDN. In: Proceedings of the Ninth ACM Conference on Emerging Networking Experiments And Technologies, pp. 25–30. ACM (2013)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Zhiliang Wang
    • 1
    Email author
  • Changping Zhou
    • 1
  • Yang Yu
    • 2
  • Xingang Shi
    • 1
  • Xia Yin
    • 2
  • Jiangyuan Yao
    • 3
  1. 1.Institute for the Network Sciences and CyberspaceTsinghua UniversityBeijingChina
  2. 2.Department of Computer Science and TechnologyTsinghua UniversityBeijingChina
  3. 3.College of Information Science and TechnologyHainan UniversityHaikouChina

Personalised recommendations