Lightweight and Accurate Memory Allocation in Key-Value Cache

  • Cheng PanEmail author
  • Lan Zhou
  • Yingwei Luo
  • Xiaolin Wang
  • Zhenlin Wang


The use of key-value caches in modern web servers is becoming more and more ubiquitous. Representatively, Memcached as a widely used key-value cache system, originally intended for speeding up dynamic web applications by alleviating database load. One of the key factors affecting the performance of Memcached is the memory allocation among different item classes. How to obtain the most efficient partitioning scheme with low time and space consumption is a focus of attention. In this paper, we propose a lightweight and accurate memory allocation scheme in Memcached, by sampling access patterns, analyzing data locality, and reassigning the memory space. One early study on optimizing memory allocation is LAMA, which uses footprint-based MRC to optimize memory allocation in Memcached. However, LAMA does not model deletion operations in Memcached and its spatial overhead is quite large. We propose a method that consumes only 3% of LAMA space and can handle read, write and deletion operations. Moreover, evaluation results show that the average stable-state miss ratio is reduced by 15.0% and the average stable-state response time is reduced by 12.3% when comparing our method to LAMA.


Key-value cache Lightweight and accurate memory allocation Performance prediction and optimization 



We would like to thank anonymous reviewers for their constructive comments and suggestions. The research is supported in part by the National Science Foundation of China (Nos. 61472008, 61672053 and U1611461), Shenzhen Key Research Project No. JCYJ20170412150946024, National Science Foundation CSR1618384 and the National Key R&D Program of China under Grant No. 2018YFB1003505.


  1. 1.
    Redis website. (2018). Accessed: 2018-07-10
  2. 2.
    Memcached website. (2018). Accessed: 2018-07-10
  3. 3.
    Atikoglu, B., Xu, Y., Frachtenberg, E., Jiang, S., Paleczny, M.: Workload analysis of a large-scale key-value store. In: ACM SIGMETRICS Performance Evaluation Review, vol. 40, pp. 53–64. ACM (2012)CrossRefGoogle Scholar
  4. 4.
    Cidon, A., Rushton, D., Rumble, S.M., Stutsman, R.: Memshare: a dynamic multi-tenant key-value cache. In: 2017 USENIX Annual Technical Conference (USENIX ATC 17), pp. 321–334. USENIX Association, Santa Clara (2017).
  5. 5.
    Caching with twemcache. (2018). Accessed: 2018-07-10
  6. 6.
    Hart, S., Frachtenberg, E., Berezecki, M.: Predicting Memcached throughput using simulation and modeling. TMS/DEVS’12, pp. 40:1–40:8.
  7. 7.
    Saemundsson, T., Bjornsson, H., Chockler, G., Vigfusson, Y.: Dynamic performance profiling of cloud caches. In: SOCC’14, pp. 28:1–28:14.
  8. 8.
    Jose, J., Subramoni, H., Kandalla, K., Wasi-ur Rahman, M., Wang, H., Narravula, S., Panda, D.K.: Scalable memcached design for InfiniBand clusters using hybrid transports. In: CCGRID’12, pp. 236–243.
  9. 9.
    Fan, B., Andersen, D.G., Kaminsky, M.: Memc3: Compact and concurrent Memcache with dumber caching and smarter hashing. In: NSDI’13, pp. 371–384.
  10. 10.
    Lim, K., Meisner, D., Saidi, A.G., Ranganathan, P., Wenisch, T.F.: Thin servers with smart pipes: designing soc accelerators for Memcached. ISCA’13.
  11. 11.
    Hwang, J., Wood, T.: Adaptive performance-aware distributed memory caching. In: ICAC’13, pp. 33–43.
  12. 12.
    Zhang, W., Hwang, J., Wood, T., Ramakrishnan, K., Huang, H.: Load balancing of heterogeneous workloads in memcached clusters. In: Feedback Computing’14.
  13. 13.
    Twemcache. (2018). Accessed: 2018-07-10
  14. 14.
    Nishtala, R., Fugal, H., Grimm, S., Kwiatkowski, M., Lee, H., Li, H.C., McElroy, R., Paleczny, M., Peek, D., Saab, P., Stafford, D., Tung, T., Venkataramani, V.: Scaling Memcache at facebook. In: NSDI’13, pp. 385–398.
  15. 15.
    Hu, X., Wang, X., Li, Y., Zhou, L., Luo, Y., Ding, C., Jiang, S., Wang, Z.: LAMA: Optimized locality-aware memory allocation for key-value cache. In: USENIX ATC’15, pp. 57–69.
  16. 16.
    Hu, X., Wang, X., Zhou, L., Luo, Y., Ding, C., Jiang, S., Wang, Z.: Optimizing locality-aware memory management of key-value caches. IEEE Trans. Comput. 66(5), 862–875 (2017). MathSciNetCrossRefGoogle Scholar
  17. 17.
    Xiang, X., Bao, B., Ding, C., Gao, Y.: Linear-time modeling of program working set in shared cache. In: PACT’11, pp. 350–360Google Scholar
  18. 18.
    Hu, X., Wang, X., Zhou, L., Luo, Y., Ding, C., Wang, Z.: Kinetic modeling of data eviction in cache. In: USENIX ATC’16Google Scholar
  19. 19.
    Byrne, D., Onder, N., Wang, Z.: mPart: miss-ratio curve guided partitioning in key-value stores. In: Proceedings of the 2018 ACM SIGPLAN International Symposium on Memory Management, ISMM 2018, pp. 84–95. ACM, New York (2018).
  20. 20.
    Hu, X., Wang, X., Zhou, L., Luo, Y., Wang, Z., Ding, C., Ye, C.: Fast miss ratio curve modeling for storage cache. ACM Trans. Storage 14(2), 12:1–12:34 (2018). CrossRefGoogle Scholar
  21. 21.
    Pan, C., Hu, X., Zhou, L., Luo, Y., Wang, X., Wang, Z.: PACE: penalty aware cache modeling with enhanced AET. In: APSys’18. ACM (2018).
  22. 22.
    Mutilate. (2018). Accessed: 2018-07-10
  23. 23.
    Mattson, R.L., Gecsei, J., Slutz, D.R., Traiger, I.L.: Evaluation techniques for storage hierarchies. IBM Syst. J. 9(2), 78–117 (1970). CrossRefzbMATHGoogle Scholar
  24. 24.
    Niu, Q., Dinan, J., Lu, Q., Sadayappan, P.: Parda: a fast parallel reuse distance analysis algorithm. IPDPS’12, pp. 1284–1294.
  25. 25.
    Wires, J., Ingram, S., Drudi, Z., Harvey, N.J.A., Warfield, A.: Characterizing storage workloads with counter stacks. In: OSDI’14, pp. 335–349.
  26. 26.
    Waldspurger, C.A., Park, N., Garthwaite, A., Ahmad, I.: Efficient MRC construction with SHARDS. In: FAST’15, pp. 95–110Google Scholar
  27. 27.
    Suh, G.E., Devadas, S., Rudolph, L.: Analytical cache models with applications to cache partitioning. In: ICS’01, pp. 1–12Google Scholar
  28. 28.
    Zhang, X., Dwarkadas, S., Shen, K.: Towards practical page coloring-based multicore cache management. In: EuroSys’09, pp. 89–102Google Scholar
  29. 29.
    Cascaval, C., Duesterwald, E., Sweeney, P.F., Wisniewski, R.W.: Multiple page size modeling and optimization. In: PSCT’05, pp. 339–349Google Scholar
  30. 30.
    Zhou, P., Pandey, V., Sundaresan, J., Raghuraman, A., Zhou, Y., Kumar, S.: Dynamic tracking of page miss ratio curve for memory management. ACM SIGOPS Oper. Syst. Rev. 38, 177–188 (2004)CrossRefGoogle Scholar
  31. 31.
    Kim, Y.H., Hill, M.D., Wood, D.A.: Implementing stack simulation for highly-associative memories. In: SIGMETRICS’91, pp. 212–213CrossRefGoogle Scholar
  32. 32.
    Bjornsson, H., Chockler, G., Saemundsson, T., Vigfusson, Y.: Dynamic performance profiling of cloud caches. In: SOCC’13Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer Science, Institute of Network Computing and Information SystemsPeking UniversityBeijingChina
  2. 2.Shenzhen Key Lab for Information Centric Networking and Blockchain Technology (ICNLAB), SECEPeking UniversityShenzhenChina
  3. 3.Department of Computer ScienceMichigan Technological UniversityHoughtonUSA

Personalised recommendations