Abstract
Heavy Hitters refer to the set of flows that represent a significantly large proportion of the link capacity or of the active traffic. Identifying Heavy Hitters is of particular importance in both network management and security applications. Traditional methods are focusing on sampling in the middle box and analyzing those packets using streaming algorithms. The paradigm of Software Defined Network (SDN) simplifies the work of flow counting. However, continuously monitoring the network will introduce overhead, which needs to be considered as a tradeoff between accurate measurement in real-time. In this paper, We propose a novel method that stamps each suspicious flow with a weight based on an online learning algorithm. The granularity of measurement is dynamically changed according to the importance of each flow. We take advantage of history flows to make the procedure of finding a heavy hitter faster so that applications can make decisions instantly. Using real-world data, we show that our online learning method can detect heavy hitters faster with less overhead and the same accuracy.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
The CAIDA UCSD anonymized internet traces 2013. http://www.caida.org/data/passive/passive_2013_dataset.xml
Django. https://www.djangoproject.com/
Openflow. https://www.opennetworking.org
sflow. www.sow.org
Software defined networks: the new norm of networks. https://www.opennetworking.org/sdn-definition/
Tcpreply. http://tcpreplay.synfn.net/
Argyropoulos, C., Kalogeras, D., Androulidakis, G., Maglaris, V.: PaFloMon-a slice aware passive flow monitoring framework for openflow enabled experimental facilities. In: 2012 European Workshop on Software Defined Networking (EWSDN), pp. 97–102. IEEE (2012)
Bandi, N., Metwally, A., Agrawal, D., El Abbadi, A.: Fast data stream algorithms using associative memories. In: Proceedings of the 2007 ACM SIGMOD international conference on Management of data, pp. 247–256. ACM (2007)
Cho, K.: Recursive lattice search: hierarchical heavy hitters revisited. In: Proceedings of the 2017 Internet Measurement Conference, pp. 283–289. ACM (2017)
Chowdhury, S.R., Bari, M.F., Ahmed, R., Boutaba, R.: Payless: a low cost network monitoring framework for software defined networks. In: Network Operations and Management Symposium (NOMS), 2014 IEEE, pp. 1–9. IEEE (2014)
Claise, B.: Cisco systems netflow services export version 9 (2004)
Cormode, G., Hadjieleftheriou, M.: Methods for finding frequent items in data streams. VLDB J. 19(1), 3–20 (2010)
Cormode, G., Johnson, T., Korn, F., Muthukrishnan, S., Spatscheck, O., Srivastava, D.: Holistic UDAFs at streaming speeds. In: Proceedings of the 2004 ACM SIGMOD International Conference on Management of Data, pp. 35–46. ACM (2004)
Cormode, G., Korn, F., Muthukrishnan, S., Srivastava, D.: Finding hierarchical heavy hitters in data streams. In: Proceedings 2003 VLDB Conference, pp. 464–475. Elsevier (2003)
Da Cruz, M.A., e Silva, L.C., Correa, S., Cardoso, K.V.: Accurate online detection of bidimensional hierarchical heavy hitters in software-defined networks. In: 2013 IEEE Latin-America Conference on Communications (LATINCOM), pp. 1–6. IEEE (2013)
Curtis, A.R., Mogul, J.C., Tourrilhes, J., Yalagandula, P., Sharma, P., Banerjee, S.: DevoFlow: scaling flow management for high-performance networks. In: ACM SIGCOMM Computer Communication Review, vol. 41, pp. 254–265. ACM (2011)
Handigol, N., Heller, B., Jeyakumar, V., Lantz, B., McKeown, N.: Reproducible network experiments using container-based emulation. In: Proceedings of the 8th International Conference on Emerging Networking Experiments and Technologies, pp. 253–264. ACM (2012)
Huici, F., Di Pietro, A., Trammell, B., Gomez Hidalgo, J.M., Martinez Ruiz, D., d’Heureuse, N.: Blockmon: a high-performance composable network traffic measurement system. ACM SIGCOMM Comput. Commun. Rev. 42(4), 79–80 (2012)
Locher, T.: Finding heavy distinct hitters in data streams. In: Proceedings of the Twenty-third Annual ACM Symposium on Parallelism in Algorithms and Architectures, pp. 299–308. ACM (2011)
Malboubi, M., Wang, L., Chuah, C.N., Sharma, P.: Intelligent SDN based traffic (de) aggregation and measurement paradigm (iSTAMP). In: INFOCOM, 2014 Proceedings IEEE, pp. 934–942. IEEE (2014)
Moshref, M., Yu, M., Govindan, R.: Resource/accuracy tradeoffs in software-defined measurement. In: Proceedings of the Second ACM SIGCOMM Workshop on Hot topics in Software Defined Networking, pp. 73–78. ACM (2013)
Shirali-Shahreza, S., Ganjali, Y.: Flexam: flexible sampling extension for monitoring and security applications in openflow. In: Proceedings of the Second ACM SIGCOMM Workshop on Hot topics in Software Defined Networking, pp. 167–168. ACM (2013)
Su, Z., Wang, T., Xia, Y., Hamdi, M.: Flowcover: Low-cost flow monitoring scheme in software defined networks. In: Global Communications Conference (GLOBECOM), 2014 IEEE, pp. 1956–1961. IEEE (2014)
Thottan, M., Liu, G., Ji, C.: Anomaly detection approaches for communication networks. In: Cormode, G., Thottan, M. (eds.) Algorithms for Next Generation Networks. Computer Communications and Networks, pp. 239–261. Springer, London (2010). https://doi.org/10.1007/978-1-84882-765-3_11
Tootoonchian, A., Ghobadi, M., Ganjali, Y.: OpenTM: Traffic matrix estimator for openflow networks. In: Krishnamurthy, A., Plattner, B. (eds.) PAM 2010. LNCS, vol. 6032, pp. 201–210. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12334-4_21
Van Adrichem, N.L., Doerr, C., Kuipers, F.A.: Opennetmon: network monitoring in openflow software-defined networks. In: Network Operations and Management Symposium (NOMS), 2014 IEEE, pp. 1–8. IEEE (2014)
Yang, L., Ng, B., Seah, W.K.: Heavy hitter detection and identification in software defined networking. In: 2016 25th International Conference on Computer Communication and Networks (ICCCN), pp. 1–10. IEEE (2016)
Yu, C., Lumezanu, C., Zhang, Y., Singh, V., Jiang, G., Madhyastha, H.V.: FlowSense: monitoring network utilization with zero measurement cost. In: Roughan, M., Chang, R. (eds.) PAM 2013. LNCS, vol. 7799, pp. 31–41. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36516-4_4
Yu, M., Jose, L., Miao, R.: Software defined traffic measurement with opensketch. In: NSDI, vol. 13, pp. 29–42 (2013)
Yuan, L., Chuah, C.N., Mohapatra, P.: ProgME: towards programmable network measurement. IEEE/ACM Trans. Netw. (TON) 19(1), 115–128 (2011)
Zhang, Y.: An adaptive flow counting method for anomaly detection in SDN. In: Proceedings of the Ninth ACM Conference on Emerging Networking Experiments And Technologies, pp. 25–30. ACM (2013)
Acknowledgments
This work is supported by Hainan Provincial Natural Science Foundation of China (618QN219) and the National High Technology Research and Development Program of China (863 Program) No. 2015AA016105.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, Z., Zhou, C., Yu, Y., Shi, X., Yin, X., Yao, J. (2018). Fast Detection of Heavy Hitters in Software Defined Networking Using an Adaptive and Learning Method. In: Sun, X., Pan, Z., Bertino, E. (eds) Cloud Computing and Security. ICCCS 2018. Lecture Notes in Computer Science(), vol 11065. Springer, Cham. https://doi.org/10.1007/978-3-030-00012-7_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-00012-7_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00011-0
Online ISBN: 978-3-030-00012-7
eBook Packages: Computer ScienceComputer Science (R0)