Caching the Internet: A View from a Global Multi-tenant CDN

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11419)


Commercial Content Delivery Networks (CDNs) employ a variety of caching policies to achieve fast and reliable delivery in multi-tenant environments with highly variable workloads. In this paper, we explore the efficacy of popular caching policies in a large-scale, global, multi-tenant CDN. We examine the client behaviors observed in a network of over 125 high-capacity Points of Presence (PoPs). Using production data from the Edgecast CDN, we show that for such a large-scale and diverse use case, simpler caching policies dominate. We find that LRU offers the best compromise between hit-rate and disk I/O, providing \(60\%\) fewer writes than FIFO, while maintaining high hit-rates. We further observe that at disk sizes used in a large-scale CDN, LRU performs on par with complex polices like S4LRU. We further examine deterministic and probabilistic cache admission policies and quantify their trade-offs between hit-rate and origin traffic. Moreover, we explore the behavior of caches at multiple layers of the CDN and provide recommendations to reduce connections passing through the system’s load balancers by approximately \(50\%\).


  1. 1.
    Nginx http server.
  2. 2.
    Redis key-value store.
  3. 3.
    Varnish http cache.
  4. 4.
    Albrecht, C., et al.: Janus: optimal flash provisioning for cloud storage workloads. In: Proceedings of the USENIX ATC 2013, pp. 91–102 (2013)Google Scholar
  5. 5.
    Atikoglu, B., Xu, Y., Frachtenberg, E., Jiang, S., Paleczny, M.: Workload analysis of a large-scale key-value store. In: Proceedings of the SIGMETRICS 2012, pp. 53–64 (2012)Google Scholar
  6. 6.
    Berger, D.S., Sitaraman, R.K., Harchol-Balter, M.: AdaptSize: orchestrating the hot object memory cache in a content delivery network. In: Proceedings of the USENIX NSDI 2017, pp. 483–498 (2017)Google Scholar
  7. 7.
    Blankstein, A., Sen, S., Freedman, M.J.: Hyperbolic caching: flexible caching for web applications. In: Proceedings of the (USENIX ATC 2017), pp. 499–511 (2017)Google Scholar
  8. 8.
    Breslau, L., Cao, P., Fan, L., Phillips, G., Shenker, S.: Web caching and Zipf-like distributions: evidence and implications. In: Proceedings of the INFOCOM 1999, vol. 1, pp. 126–134, March 1999Google Scholar
  9. 9.
    Cáceres, R., Douglis, F., Feldmann, A., Glass, G., Rabinovich, M.: Web proxy caching: the devil is in the details. In: Proceedings of the WISP 1998, pp. 11–15 (1998)Google Scholar
  10. 10.
    Calder, M., Fan, X., Hu, Z., Katz-Bassett, E., Heidemann, J., Govindan, R.: Mapping the expansion of Google’s serving infrastructure. In: Proceedings of the IMC 2013, pp. 313–326 (2013)Google Scholar
  11. 11.
    Cao, P., Irani, S.: Cost-aware WWW proxy caching algorithms. In: Proceedings of the USITS 1997, p. 18 (1997)Google Scholar
  12. 12.
    Cao, P., Zhang, J., Beach, K.: Active cache: caching dynamic contents on the web. In: Proceedings of the Middleware 1998, pp. 373–388 (1998)Google Scholar
  13. 13.
    Chankhunthod, A., Danzig, P.B., Neerdaels, C., Schwartz, M.F., Worrell, K.J.: A hierarchical internet object cache. In: Proceedings of the USENIX ATC 1996, p. 13 (1996)Google Scholar
  14. 14.
    Chen, F., Sitaraman, R.K., Torres, M.: End-user mapping: next generation request routing for content delivery. In: Proceedings of the SIGCOMM 2015, pp. 167–181 (2015)Google Scholar
  15. 15.
    Cidon, A., Eisenman, A., Alizadeh, M., Katti, S.: Cliffhanger: scaling performance cliffs in web memory caches. In: Proceedings of the NSDI 2016, pp. 379–392 (2016)Google Scholar
  16. 16.
    Dilley, J., Maggs, B., Parikh, J., Prokop, H., Sitaraman, R., Weihl, B.: Globally distributed content delivery. IEEE Internet Comput. 6, 50–58 (2002)CrossRefGoogle Scholar
  17. 17.
    Fitzpatrick, B.: Distributed caching with memcached (2004)Google Scholar
  18. 18.
    Freedman, M.J.: Experiences with CoralCDN: a five-year operational view. In: Proceedings of the NSDI (2010)Google Scholar
  19. 19.
    Gummadi, K.P., Dunn, R.J., Saroiu, S., Gribble, S.D., Levy, H.M., Zahorjan, J.: Measurement, modeling, and analysis of a peer-to-peer file-sharing workload. In: Proceedings of the SOSP 2003, pp. 314–329 (2003)Google Scholar
  20. 20.
    Guo, L., Tan, E., Chen, S., Xiao, Z., Zhang, X.: The stretched exponential distribution of internet media access patterns. In: Proceedings of the PODC 2008, pp. 283–294 (2008)Google Scholar
  21. 21.
    Hasslinger, G., Ntougias, K., Hasslinger, F., Hohlfeld, O.: Performance evaluation for new web caching strategies combining LRU with score based object selection. In: Proceedings of the ITC 2016, pp. 322–330 (2016)Google Scholar
  22. 22.
    Huang, Q., Birman, K., van Renesse, R., Lloyd, W., Kumar, S., Li, H.C.: An analysis of Facebook photo caching. In: Proceedings of the SOSP 2013, pp. 167–181 (2013)Google Scholar
  23. 23.
    Ihm, S., Pai, V.S.: Towards understanding modern web traffic. In: Proceedings of the IMC 2011, pp. 295–312 (2011)Google Scholar
  24. 24.
    Jiang, S., Zhang, X.: LIRS: an efficient low inter-reference recency set replacement policy to improve buffer cache performance. In: Proceedings of the SIGMETRICS 2002, pp. 31–42 (2002)Google Scholar
  25. 25.
    Johnson, K., Carr, J., Day, M., Kaashoek, M.: The measured performance of content distribution networks. Comput. Commun. 24, 202–206 (2001)CrossRefGoogle Scholar
  26. 26.
    Johnson, T., Shasha, D.: 2Q: a low overhead high performance buffer management replacement algorithm. In: Proceedings of the VLDB 1994, pp. 439–450 (1994)Google Scholar
  27. 27.
    Jung, J., Krishnamurthy, B., Rabinovich, M.: Flash crowds and denial of service attacks: characterization and implications for CDNs and web sites. In: Proceedings of the WWW 2002, pp. 293–304 (2002)Google Scholar
  28. 28.
    Khakpour, A., Peters, R.J.: Optimizing multi-hit caching for long tail content. Patent No. US8639780 B2, January 2014Google Scholar
  29. 29.
    Krishnamurthy, B., Wills, C., Zhang, Y.: On the use and performance of content distribution networks. In: Proceedings of the IMW 2001, pp. 169–182 (2001)Google Scholar
  30. 30.
    Krishnan, R., et al.: Moving beyond end-to-end path information to optimize CDN performance. In: Proceedings of the IMC 2009, pp. 190–201 (2009)Google Scholar
  31. 31.
    Maggs, B.M., Sitaraman, R.K.: Algorithmic nuggets in content delivery. SIGCOMM Comput. Commun. Rev. 45, 52–66 (2015)CrossRefGoogle Scholar
  32. 32.
    Megiddo, N., Modha, D.S.: ARC: a self-tuning, low overhead replacement cache. In: Proceedings of the FAST 2003, pp. 115–130 (2003)Google Scholar
  33. 33.
    O’Neil, E.J., O’Neil, P.E., Weikum, G.: The LRU-K page replacement algorithm for database disk buffering. In: Proceedings of the SIGMOD 1993, pp. 297–306 (1993)Google Scholar
  34. 34.
    Saroiu, S., Gummadi, K.P., Dunn, R.J., Gribble, S.D., Levy, H.M.: An analysis of internet content delivery systems. In: Proceedings of the OSDI (2002)Google Scholar
  35. 35.
    Scellato, S., Mascolo, C., Musolesi, M., Crowcroft, J.: Track globally, deliver locally: improving content delivery networks by tracking geographic social cascades. In: Proceedings of the WWW 2011, pp. 457–466 (2011)Google Scholar
  36. 36.
    Shafiq, M.Z., Khakpour, A.R., Liu, A.X.: Characterizing caching workload of a large commercial content delivery network. In: Proceedings of INFOCOM 2016, pp. 1–9, April 2016Google Scholar
  37. 37.
    Shim, J., Scheuermann, P., Vingralek, R.: Proxy cache algorithms: design, implementation, and performance. IEEE Trans. Knowl. Data Eng. 11, 549–562 (1999)CrossRefGoogle Scholar
  38. 38.
    Tang, L., Huang, Q., Lloyd, W., Kumar, S., Li, K.: RIPQ: advanced photo caching on flash for Facebook. In: Proceedings of the FAST 2015, pp. 373–386 (2015)Google Scholar
  39. 39.
    Wang, J.: A survey of web caching schemes for the internet. SIGCOMM Comput. Commun. Rev. 29, 36–46 (1999)CrossRefGoogle Scholar
  40. 40.
    Wendell, P., Freedman, M.J.: Going viral: flash crowds in an open CDN. In: Proceedings of the IMC 2011, pp. 549–558 (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Verizon Digital Media ServicesLos AngelesUSA

Personalised recommendations