Abstract
The detection of network flows that send excessive amounts of traffic is of increasing importance to enforce QoS and to counter DDoS attacks. Large-flow detection has been previously explored, but the proposed approaches can be used on high-capacity core routers only at the cost of significantly reduced accuracy, due to their otherwise too high memory and processing overhead. We propose CLEF, a new large-flow detection scheme with low memory requirements, which maintains high accuracy under the strict conditions of high-capacity core routers. We compare our scheme with previous proposals through extensive theoretical analysis, and with an evaluation based on worst-case-scenario attack traffic. We show that CLEF outperforms previously proposed systems in settings with limited memory.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
The IP metadata consists of source and destination addresses, protocol number, and ports. Thus, it requires about 16 bytes and 40 bytes per counter for IPv4 and IPv6, respectively.
- 3.
The terms “counter tree” and “virtual counter” are also used by Chen et al. [7], but our technique differs in both approach and goal. Chen et al. efficiently manage a sufficient number of counters for per-flow accounting, while RLFD manages an insufficient number of counters to detect consistent overuse.
- 4.
If \(T_{\ell } \ll \beta /\gamma \), it is hard for a large flow to reach the burst threshold \(\beta \) in such a short time; if \(T_{\ell } \gg \beta /\gamma \), the detection delay is too long, resulting in excessive damage.
References
Andersen, D.G., Balakrishnan, H., Feamster, N., Koponen, T., Moon, D., Shenker, S.: Accountable internet protocol (AIP). In: Proceedings of ACM SIGCOMM (2008). https://doi.org/10.1145/1402958.1402997
Anderson, T., et al.: The NEBULA future internet architecture. In: Galis, A., Gavras, A. (eds.) FIA 2013. LNCS, vol. 7858, pp. 16–26. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38082-2_2
Antonakakis, M., et al.: Understanding the Mirai botnet. In: USENIX Security Symposium (2017)
Basescu, C., et al.: SIBRA: scalable internet bandwidth reservation architecture. In: Proceedings of Network and Distributed System Security Symposium (NDSS), February 2016
Braden, R., Clark, D., Shenker, S.: Integrated services in the internet architecture: an overview. RFC 1633 (Informational), June 1994. http://www.ietf.org/rfc/rfc1633.txt
CAIDA: CAIDA Anonymized Internet Traces 2016 (2016). https://data.caida.org/datasets/passive-2016/
Chen, M., Chen, S., Cai, Z.: Counter tree: a scalable counter architecture for per-flow traffic measurement. IEEE/ACM Trans. Netw. (TON) 25(2), 1249–1262 (2017)
Claise, B.: Cisco Systems NetFlow Services Export Version 9, RFC 3954 (Informational), October 2004. http://www.ietf.org/rfc/rfc3954.txt
Cormode, G., Muthukrishnan, S.: An improved data stream summary: the count-min sketch and its applications. J. Algorithms 55(1), 58–75 (2005). https://doi.org/10.1016/j.jalgor.2003.12.001
Demaine, E.D., López-Ortiz, A., Munro, J.I.: Frequency estimation of internet packet streams with limited space. In: Möhring, R., Raman, R. (eds.) ESA 2002. LNCS, vol. 2461, pp. 348–360. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45749-6_33
Estan, C.: Internet traffic measurement: what’s going on in my network? Ph.D. thesis (2003)
Estan, C., Varghese, G.: New directions in traffic measurement and accounting: focusing on the elephants, ignoring the mice. ACM Trans. Comput. Syst. (TOCS) 21(3), 270–313 (2003). http://dl.acm.org/citation.cfm?id=859719
Fang, M., Shivakumar, N.: Computing iceberg queries efficiently. In: Proceedings of VLDB (1999). http://ilpubs.stanford.edu:8090/423/
Han, D., et al.: XIA: efficient support for evolvable internetworking. In: Proceedings of the 9th USENIX NSDI, San Jose, CA, April 2012
Intel: Intel Xeon Processor E7 v4 Family (2016). https://ark.intel.com/products/series/93797/Intel-Xeon-Processor-E7-v4-Family
Karp, R.M., Shenker, S., Papadimitriou, C.H.: A simple algorithm for finding frequent elements in streams and bags. ACM Trans. Database Syst. 28(1), 51–55 (2003). https://doi.org/10.1145/762471.762473
Kim, T.H.J., Basescu, C., Jia, L., Lee, S.B., Hu, Y.C., Perrig, A.: Lightweight source authentication and path validation. In: ACM SIGCOMM Computer Communication Review, vol. 44, pp. 271–282. ACM (2014)
Kumar, A., Xu, J., Wang, J.: Space-code bloom filter for efficient per-flow traffic measurement. IEEE J. Sel. Areas Commun. 24(12), 2327–2339 (2006)
Lee, S.B., Kang, M.S., Gligor, V.D.: CoDef: collaborative defense against large-scale link-flooding attacks. In: Proceedings of CoNext (2013)
Li, A., Liu, X., Yang, X.: Bootstrapping accountability in the internet we have. In: Proceedings of USENIX/ACM NSDI, March 2011
Liu, X., Li, A., Yang, X., Wetherall, D.: Passport: secure and adoptable source authentication. In: Proceedings of USENIX/ACM NSDI (2008). http://www.usenix.org/event/nsdi08/tech/full_papers/liu_xin/liu_xin_html/
Liu, Z., Manousis, A., Vorsanger, G., Sekar, V., Braverman, V.: One sketch to rule them all: rethinking network flow monitoring with UnivMon. In: ACM SIGCOMM (2016). https://doi.org/10.1145/2934872.2934906
Liu, Z., Jin, H., Hu, Y.C., Bailey, M.: MiddlePolice: toward enforcing destination-defined policies in the middle of the internet. In: Proceedings of ACM CCS, October 2016
Manku, G., Motwani, R.: Approximate frequency counts over data streams. In: Proceedings of VLDB (2002). http://dl.acm.org/citation.cfm?id=1287400
Metwally, A., Agrawal, D., El Abbadi, A.: Efficient computation of frequent and top-k elements in data streams. In: Eiter, T., Libkin, L. (eds.) ICDT 2005. LNCS, vol. 3363, pp. 398–412. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30570-5_27
Misra, J., Gries, D.: Finding repeated elements. Sci. Comput. Program. 2(2), 143–152 (1982)
Naous, J., Walfish, M., Nicolosi, A., Mazières, D., Miller, M., Seehra, A.: Verifying and enforcing network paths with ICING. In: Proceedings of ACM CoNEXT (2011). https://doi.org/10.1145/2079296.2079326
Pagh, R., Rodler, F.F.: Cuckoo hashing. In: auf der Heide, F.M. (ed.) ESA 2001. LNCS, vol. 2161, pp. 121–133. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-44676-1_10
Shenker, S., Partridge, C., Guerin, R.: Specification of guaranteed quality of service, RFC 2212 (Proposed Standard), September 1997. http://www.ietf.org/rfc/rfc2212.txt
Sivaraman, V., Narayana, S., Rottenstreich, O., Muthukrishnan, S., Rexford, J.: Heavy-hitter detection entirely in the data plane. In: Proceedings of the Symposium on SDN Research, pp. 164–176. ACM (2017)
Tong, D., Prasanna, V.: High throughput sketch based online heavy hitter detection on FPGA. ACM SIGARCH Comput. Arch. News 43(4), 70–75 (2016)
Trybulec, W.A.: Pigeon hole principle. J. Formaliz. Math. 2, 4 (1990)
Wu, H., Hsiao, H.C., Asoni, D.E., Scherrer, S., Perrig, A., Hu, Y.C.: CLEF: limiting the damage caused by large flows in the internet core. Technical report, arXiv:1807.05652 [cs.NI], arXiv (2018). https://arxiv.org/abs/1807.05652
Wu, H., Hsiao, H.C., Hu, Y.C.: Efficient large flow detection over arbitrary windows: an algorithm exact outside an ambiguity region. In: Proceedings of the 2014 Conference on Internet Measurement Conference, pp. 209–222. ACM (2014)
Xiao, Q., Chen, S., Chen, M., Ling, Y.: Hyper-compact virtual estimators for big network data based on register sharing. In: ACM SIGMETRICS Performance Evaluation Review, vol. 43, pp. 417–428. ACM (2015)
Zhang, X., Hsiao, H.C., Hasker, G., Chan, H., Perrig, A., Andersen, D.G.: SCION: scalability, control, and isolation on next-generation networks. In: IEEE Symposium on Security and Privacy, pp. 212–227 (2011)
Acknowledgments
We thank Pratyaksh Sharma and Prateesh Goyal for early work on this project as part of their summer internship at ETH in Summer 2015. We also thank the anonymous reviewers, whose feedback helped to improve the paper.
The research leading to these results has received funding from the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007-2013), ERC grant agreement 617605, the Ministry of Science and Technology of Taiwan under grant number MOST 107-2636-E-002-005, and the US National Science Foundation under grant numbers CNS-1717313 and CNS-0953600. We also gratefully acknowledge support from ETH Zurich and from the Zurich Information Security and Privacy Center (ZISC).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Wu, H., Hsiao, HC., Asoni, D.E., Scherrer, S., Perrig, A., Hu, YC. (2018). CLEF: Limiting the Damage Caused by Large Flows in the Internet Core. In: Camenisch, J., Papadimitratos, P. (eds) Cryptology and Network Security. CANS 2018. Lecture Notes in Computer Science(), vol 11124. Springer, Cham. https://doi.org/10.1007/978-3-030-00434-7_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-00434-7_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00433-0
Online ISBN: 978-3-030-00434-7
eBook Packages: Computer ScienceComputer Science (R0)