Advertisement

Lightweight and Accurate Memory Allocation in Key-Value Cache

  • Cheng PanEmail author
  • Lan Zhou
  • Yingwei Luo
  • Xiaolin Wang
  • Zhenlin Wang
Article
  • 44 Downloads

Abstract

The use of key-value caches in modern web servers is becoming more and more ubiquitous. Representatively, Memcached as a widely used key-value cache system, originally intended for speeding up dynamic web applications by alleviating database load. One of the key factors affecting the performance of Memcached is the memory allocation among different item classes. How to obtain the most efficient partitioning scheme with low time and space consumption is a focus of attention. In this paper, we propose a lightweight and accurate memory allocation scheme in Memcached, by sampling access patterns, analyzing data locality, and reassigning the memory space. One early study on optimizing memory allocation is LAMA, which uses footprint-based MRC to optimize memory allocation in Memcached. However, LAMA does not model deletion operations in Memcached and its spatial overhead is quite large. We propose a method that consumes only 3% of LAMA space and can handle read, write and deletion operations. Moreover, evaluation results show that the average stable-state miss ratio is reduced by 15.0% and the average stable-state response time is reduced by 12.3% when comparing our method to LAMA.

Keywords

Key-value cache Lightweight and accurate memory allocation Performance prediction and optimization 

Notes

Acknowledgements

We would like to thank anonymous reviewers for their constructive comments and suggestions. The research is supported in part by the National Science Foundation of China (Nos. 61472008, 61672053 and U1611461), Shenzhen Key Research Project No. JCYJ20170412150946024, National Science Foundation CSR1618384 and the National Key R&D Program of China under Grant No. 2018YFB1003505.

References

  1. 1.
    Redis website. https://redis.io/ (2018). Accessed: 2018-07-10
  2. 2.
    Memcached website. http://memcached.org/ (2018). Accessed: 2018-07-10
  3. 3.
    Atikoglu, B., Xu, Y., Frachtenberg, E., Jiang, S., Paleczny, M.: Workload analysis of a large-scale key-value store. In: ACM SIGMETRICS Performance Evaluation Review, vol. 40, pp. 53–64. ACM (2012)CrossRefGoogle Scholar
  4. 4.
    Cidon, A., Rushton, D., Rumble, S.M., Stutsman, R.: Memshare: a dynamic multi-tenant key-value cache. In: 2017 USENIX Annual Technical Conference (USENIX ATC 17), pp. 321–334. USENIX Association, Santa Clara (2017). https://www.usenix.org/conference/atc17/technical-sessions/presentation/cidon
  5. 5.
    Caching with twemcache. https://blog.twitter.com/engineering/en_us/a/2012/caching-with-twemcache.html (2018). Accessed: 2018-07-10
  6. 6.
    Hart, S., Frachtenberg, E., Berezecki, M.: Predicting Memcached throughput using simulation and modeling. TMS/DEVS’12, pp. 40:1–40:8. http://dl.acm.org/citation.cfm?id=2346616.2346656
  7. 7.
    Saemundsson, T., Bjornsson, H., Chockler, G., Vigfusson, Y.: Dynamic performance profiling of cloud caches. In: SOCC’14, pp. 28:1–28:14.  https://doi.org/10.1145/2670979.2671007
  8. 8.
    Jose, J., Subramoni, H., Kandalla, K., Wasi-ur Rahman, M., Wang, H., Narravula, S., Panda, D.K.: Scalable memcached design for InfiniBand clusters using hybrid transports. In: CCGRID’12, pp. 236–243.  https://doi.org/10.1109/CCGrid.2012.141
  9. 9.
    Fan, B., Andersen, D.G., Kaminsky, M.: Memc3: Compact and concurrent Memcache with dumber caching and smarter hashing. In: NSDI’13, pp. 371–384. https://www.usenix.org/conference/nsdi13/technical-sessions/presentation/fan
  10. 10.
    Lim, K., Meisner, D., Saidi, A.G., Ranganathan, P., Wenisch, T.F.: Thin servers with smart pipes: designing soc accelerators for Memcached. ISCA’13.  https://doi.org/10.1145/2485922.2485926
  11. 11.
    Hwang, J., Wood, T.: Adaptive performance-aware distributed memory caching. In: ICAC’13, pp. 33–43. https://www.usenix.org/conference/icac13/technical-sessions/presentation/hwang
  12. 12.
    Zhang, W., Hwang, J., Wood, T., Ramakrishnan, K., Huang, H.: Load balancing of heterogeneous workloads in memcached clusters. In: Feedback Computing’14. https://www.usenix.org/conference/feedbackcomputing14/workshop-program/presentation/zhang
  13. 13.
    Twemcache. https://twitter.com/twemcache (2018). Accessed: 2018-07-10
  14. 14.
    Nishtala, R., Fugal, H., Grimm, S., Kwiatkowski, M., Lee, H., Li, H.C., McElroy, R., Paleczny, M., Peek, D., Saab, P., Stafford, D., Tung, T., Venkataramani, V.: Scaling Memcache at facebook. In: NSDI’13, pp. 385–398. https://www.usenix.org/conference/nsdi13/technical-sessions/presentation/nishtala
  15. 15.
    Hu, X., Wang, X., Li, Y., Zhou, L., Luo, Y., Ding, C., Jiang, S., Wang, Z.: LAMA: Optimized locality-aware memory allocation for key-value cache. In: USENIX ATC’15, pp. 57–69. https://www.usenix.org/conference/atc15/technical-session/presentation/hu
  16. 16.
    Hu, X., Wang, X., Zhou, L., Luo, Y., Ding, C., Jiang, S., Wang, Z.: Optimizing locality-aware memory management of key-value caches. IEEE Trans. Comput. 66(5), 862–875 (2017).  https://doi.org/10.1109/TC.2016.2618920 MathSciNetCrossRefGoogle Scholar
  17. 17.
    Xiang, X., Bao, B., Ding, C., Gao, Y.: Linear-time modeling of program working set in shared cache. In: PACT’11, pp. 350–360Google Scholar
  18. 18.
    Hu, X., Wang, X., Zhou, L., Luo, Y., Ding, C., Wang, Z.: Kinetic modeling of data eviction in cache. In: USENIX ATC’16Google Scholar
  19. 19.
    Byrne, D., Onder, N., Wang, Z.: mPart: miss-ratio curve guided partitioning in key-value stores. In: Proceedings of the 2018 ACM SIGPLAN International Symposium on Memory Management, ISMM 2018, pp. 84–95. ACM, New York (2018).  https://doi.org/10.1145/3210563.3210571
  20. 20.
    Hu, X., Wang, X., Zhou, L., Luo, Y., Wang, Z., Ding, C., Ye, C.: Fast miss ratio curve modeling for storage cache. ACM Trans. Storage 14(2), 12:1–12:34 (2018).  https://doi.org/10.1145/3185751 CrossRefGoogle Scholar
  21. 21.
    Pan, C., Hu, X., Zhou, L., Luo, Y., Wang, X., Wang, Z.: PACE: penalty aware cache modeling with enhanced AET. In: APSys’18. ACM (2018).  https://doi.org/10.1145/3265723.3265736
  22. 22.
    Mutilate. https://github.com/leverich/mutilate (2018). Accessed: 2018-07-10
  23. 23.
    Mattson, R.L., Gecsei, J., Slutz, D.R., Traiger, I.L.: Evaluation techniques for storage hierarchies. IBM Syst. J. 9(2), 78–117 (1970).  https://doi.org/10.1147/sj.92.0078 CrossRefzbMATHGoogle Scholar
  24. 24.
    Niu, Q., Dinan, J., Lu, Q., Sadayappan, P.: Parda: a fast parallel reuse distance analysis algorithm. IPDPS’12, pp. 1284–1294.  https://doi.org/10.1109/IPDPS.2012.117
  25. 25.
    Wires, J., Ingram, S., Drudi, Z., Harvey, N.J.A., Warfield, A.: Characterizing storage workloads with counter stacks. In: OSDI’14, pp. 335–349. https://www.usenix.org/conference/osdi14/technical-sessions/presentation/wires
  26. 26.
    Waldspurger, C.A., Park, N., Garthwaite, A., Ahmad, I.: Efficient MRC construction with SHARDS. In: FAST’15, pp. 95–110Google Scholar
  27. 27.
    Suh, G.E., Devadas, S., Rudolph, L.: Analytical cache models with applications to cache partitioning. In: ICS’01, pp. 1–12Google Scholar
  28. 28.
    Zhang, X., Dwarkadas, S., Shen, K.: Towards practical page coloring-based multicore cache management. In: EuroSys’09, pp. 89–102Google Scholar
  29. 29.
    Cascaval, C., Duesterwald, E., Sweeney, P.F., Wisniewski, R.W.: Multiple page size modeling and optimization. In: PSCT’05, pp. 339–349Google Scholar
  30. 30.
    Zhou, P., Pandey, V., Sundaresan, J., Raghuraman, A., Zhou, Y., Kumar, S.: Dynamic tracking of page miss ratio curve for memory management. ACM SIGOPS Oper. Syst. Rev. 38, 177–188 (2004)CrossRefGoogle Scholar
  31. 31.
    Kim, Y.H., Hill, M.D., Wood, D.A.: Implementing stack simulation for highly-associative memories. In: SIGMETRICS’91, pp. 212–213CrossRefGoogle Scholar
  32. 32.
    Bjornsson, H., Chockler, G., Saemundsson, T., Vigfusson, Y.: Dynamic performance profiling of cloud caches. In: SOCC’13Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer Science, Institute of Network Computing and Information SystemsPeking UniversityBeijingChina
  2. 2.Shenzhen Key Lab for Information Centric Networking and Blockchain Technology (ICNLAB), SECEPeking UniversityShenzhenChina
  3. 3.Department of Computer ScienceMichigan Technological UniversityHoughtonUSA

Personalised recommendations