Cluster Computing

, Volume 21, Issue 2, pp 1275–1287 | Cite as

HALO: a fast and durable disk write cache using phase change memory

  • Zhuo Liu
  • Bin Wang
  • Weikuan YuEmail author


To close the increasing performance gap between disk storage and processors, flash based solid-state devices (FSSDs) have been adopted within the memory hierarchy to improve the performance of hard disk drive (HDD) based storage system. However, FSSDs still suffer from erase-before-write restriction, coarse access granularity and limited write endurance. Recently, the cutting-edge non-volatile memory technologies are merging, e.g., phase-change memory and resistive memory, which offer us new storage alternatives with faster access, byte-addressability and better endurance. In order to address the imperative data intensive computing issues, we propose to leverage PCM as disk write cache for constructing a hybrid PCM+HDD storage architecture in this paper. First, we develop a novel hash-based write caching scheme called HALO to improve both spatial and temporal locality on hard disks, thus addressing the limitations of traditional LRU caching algorithms and rendering better I/O performance. To deal with the limited durability of PCM devices and reclaim the degraded spatial locality in previous wear-leveling techniques, we further propose novel PCM wear leveling algorithms that provide effectively uniform writes while maximizing access parallelism. We have evaluated this PCM-based hybrid storage architecture using applications with a diverse set of I/O access patterns. Our experimental results demonstrate that the HALO caching scheme leads to an average reduction of 36.8% in execution time compared to the LRU caching scheme, and that the space filling curve based wear leveling extends the lifetime of PCM by a factor of 21.6.


PCM Cache Wear leveling Hybrid storage Cuckoo hashing 



We are thankful to Mr. Michael Pritchard for his comments on the paper. This work is supported in part by the NSF awards ACI-1561041, CNS-1320016 and CNS-1059376 and by a UT-Battelle Grant to Auburn University.


  1. 1.
    Akel, A., Caulfield, A., Mollov, T., Gupta, R., Swanson, S.: Onyx: A prototype phase change memory storage array. In: HotStorage’11Google Scholar
  2. 2.
    Are hybrid drives finally coming of age?—single-user.
  3. 3.
    Baker, M., Asami, S., Deprit, E., Ouseterhout, J., Seltzer, M.: Non-volatile memory for fast, reliable file systems. ACM SIGPLAN Not. 27(9), 10–22 (1992)CrossRefGoogle Scholar
  4. 4.
    Brunelle, A.D.: Blktrace user guide (2007)Google Scholar
  5. 5.
    Bucy, J.S., Ganger, G.R.: Tech. rep.
  6. 6.
    Chen, S., Gibbons, P., Nath, S.: Rethinking database algorithms for phase change memory. CIDR11 pp. 21–31 (2011)Google Scholar
  7. 7.
    Ding, X., Jiang, S., Chen, F., Davis, K., Zhang, X.: Diskseen: exploiting disk layout and access history to enhance i/o prefetch. In: USENIX ATC (2007)Google Scholar
  8. 8.
    Doh, I., Choi, J., Lee, D., Noh, S.: Exploiting non-volatile ram to enhance flash file system performance. In: ACM EMSOFT’07Google Scholar
  9. 9.
    Doh, I., Lee, H., Moon, Y., Kim, E., Choi, J., Lee, D., Noh, S.: Impact of nvram write cache for file system metadata on i/o performance in embedded systems. In: ACM SAC’09Google Scholar
  10. 10.
    Fotakis, D., Pagh, R., Sanders, P., Spirakis, P.: Space efficient hash tables with worst case constant access time. STACS 2003, 271–282 (2003)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Gill, B., Modha, D.: Wow: wise ordering for writes-combining spatial and temporal locality in non-volatile caches. In: USENIX FAST’05Google Scholar
  12. 12.
    Ipek, E., Condit, J., Nightingale, E.B., Burger, D., Moscibroda, T.: Dynamically replicated memory: building reliable systems from nanoscale resistive memories. In: ASPLOS’10Google Scholar
  13. 13.
    Jiang, S., Ding, X., Chen, F., Tan, E., Zhang, X.: Dulo: an effective buffer cache management scheme to exploit both temporal and spatial locality. In: USENIX FAST’05Google Scholar
  14. 14.
    Jiang, S., Zhang, X.: LIRS: an efficient low inter-reference recency set replacement policy to improve buffer cache performance. ACM SIGMETRICS Perform. Eval. Rev. 30, 31–42 (2002)CrossRefGoogle Scholar
  15. 15.
    Kim, C.: LRFU: A spectrum of policies that subsumes the least recently used and least frequently used policies. IEEE Trans. Comput. 50(12), 1352–1361 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Lee, E., Bahn, H., Noh, S.H.: Unioning of the buffer cache and journaling layers with non-volatile memory. In: FAST, pp. 73–80. USENIX (2013)Google Scholar
  17. 17.
    Lee, K., Doh, I., Choi, J., Lee, D., Noh, S.: Write-aware buffer cache management scheme for nonvolatile ram. In: Proceedings of Advances in Computer Science and Technology. ACTA Press (2007)Google Scholar
  18. 18.
    Li, D., Vetter, J.S., Marin, G., McCurdy, C., Cira, C., Liu, Z., Yu, W.: Identifying opportunities for byte-addressable non-volatile memory in extreme-scale scientific applications. In: Parallel & Distributed Processing Symposium (IPDPS), 2012 IEEE 26th International, pp. 945–956. IEEE (2012)Google Scholar
  19. 19.
    Liu, X., Schrack, G.: An algorithm for encoding and decoding the 3-d hilbert order. IEEE Trans. Image Process. 6(9), 1333–1337 (1997)CrossRefGoogle Scholar
  20. 20.
    Liu, Z., Wang, B., Carpenter, P., Li, D., Vetter, J.S., Yu, W.: PCM-based durable write cache for fast disk I/O. In: Modeling, Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS), 2012 IEEE 20th International Symposium on, pp. 451–458. IEEE (2012)Google Scholar
  21. 21.
    Liu, Z., Wang, B., Wang, T., Tian, Y., Xu, C., Wang, Y., Yu, W., Cruz, C.A., Zhou, S., Clune, T., et al.: Profiling and improving i/o performance of a large-scale climate scientific application. In: Computer Communications and Networks (ICCCN), 2013 22nd International Conference on, pp. 1–7. IEEE (2013)Google Scholar
  22. 22.
  23. 23.
    Nightingale, T., Hu, Y., Yang, Q.: The design and implementation of a dcd device driver for unix. In: USENIX ATC’99Google Scholar
  24. 24.
    O’neil, E., O’neil, P., Weikum, G.: The LRU-K page replacement algorithm for database disk buffering. In: ACM SIGMOD’93Google Scholar
  25. 25.
    OSDL: Iometer project. (2004)
  26. 26.
    Pagh, R., Rodler, F.: Cuckoo hashing. AlgorithmsESA 2001, 121–133 (2001)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Park, Y., Lim, S., Lee, C., Park, K.: Pffs: a scalable flash memory file system for the hybrid architecture of phase-change ram and nand flash. In: ACM SAC’08Google Scholar
  28. 28.
    Qureshi, M., Karidis, J., Franceschini, M., Srinivasan, V., Lastras, L., Abali, B.: Enhancing lifetime and security of pcm-based main memory with start-gap wear leveling. In: IEEE/ACM Micro’09Google Scholar
  29. 29.
    Qureshi, M.K., Srinivasan, V., Rivers, J.A.: Scalable high performance main memory system using phase-change memory technology. In: Proc. of the Internationl Symposium on Computer Architecture (2009)Google Scholar
  30. 30.
    Ramos, L., Gorbatov, E., Bianchini, R.: Page placement in hybrid memory systems. In: Proc. of the International Conference on Supercomputing (2011)Google Scholar
  31. 31.
    Rosenblum, M., Ousterhout, J.: The design and implementation of a log-structured file system. ACM SIGOPS Oper. Syst. Rev. 25(5), 1–15 (1991)CrossRefGoogle Scholar
  32. 32.
    Seong, N.H., Woo, D.H., Lee, H.H.S.: Security refresh: prevent malicious wear-out and increase durability for phase-change memory with dynamically randomized address mapping. In: ISCA’10Google Scholar
  33. 33.
    Shi, L., Li, J., Jason Xue, C., Zhou, X.: Hybrid nonvolatile disk cache for energy-efficient and high-performance systems. ACM Trans. Des. Autom. Electron. Syst. 18(1), 8 (2013)Google Scholar
  34. 34.
    SNIA BlockIO Traces. (2006)
  35. 35.
    Soundararajan, G., Prabhakaran, V., Balakrishnan, M., Wobber, T.: Extending ssd lifetimes with disk-based write caches. In: USENIX FAST’10Google Scholar
  36. 36.
    Sun, G., Joo, Y., Chen, Y., Niu, D., Xie, Y., Chen, Y., Li, H.: A hybrid solid-state storage architecture for the performance, energy consumption, and lifetime improvement. In: IEEE HPCA’10Google Scholar
  37. 37.
    Umass trace repository.
  38. 38.
    Wang, A., Reiher, P., Popek, G., Kuenning, G.: Conquest: Better performance through a disk/persistent-ram hybrid file system. In: USENIX ATC’02Google Scholar
  39. 39.
    Wang, B., Liu, Z., Wang, X., Yu, W.: Eliminating intra-warp conflict misses in gpu. In: Proceedings of the Conference on Design, Automation and Test in Europe (DATE). IEEE (2015)Google Scholar
  40. 40.
    Wang, B., Wu, B., Li, D., Shen, X., Yu, W., Jiao, Y., Vetter, J.S.: Exploring hybrid memory for gpu energy efficiency through software-hardware co-design. In: Proceedings of the 22nd International Conference on Parallel Architectures and Compilation Techniques, PACT ’13, pp. 93–102. IEEE Press, Piscataway, NJ, USA (2013)Google Scholar
  41. 41.
    Wang, J., Dong, X., Xie, Y., Jouppi, N.P.: i2wap: Improving non-volatile cache lifetime by reducing inter-and intra-set write variations. In: High Performance Computer Architecture (HPCA2013), 2013 IEEE 19th International Symposium on, pp. 234–245. IEEE (2013)Google Scholar
  42. 42.
    Woodhouse, D.: Jffs: The journalling flash file system. In: Ottawa Linux Symposium, vol. 2001 (2001)Google Scholar
  43. 43.
    Zhang, W., Li, T.: Exploring phase change memory and 3D die-stacking for power/thermal friendly, fast and durable memory architecture. In: International Conference on Parallel Architecture and Compilation Techniques (2009)Google Scholar
  44. 44.
    Zhou, P., Zhao, B., Yang, J., Zhang, Y.: A durable and energy efficient main memory using phase change memory technology. In: ISCA’09Google Scholar
  45. 45.
    Zhou, Y., Philbin, J., Li, K.: The multi-queue replacement algorithm for second level buffer caches. In: USENIX ATC’02Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Auburn UniversityAuburnUSA
  2. 2.Florida State UniversityTallahasseeUSA

Personalised recommendations