Skip to main content
SpringerLink
Log in

A Lookahead Read Cache: Improving Read Performance for Deduplication Backup Storage

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Data deduplication (dedupe for short) is a special data compression technique. It has been widely adopted to save backup time as well as storage space, particularly in backup storage systems. Therefore, most dedupe research has primarily focused on improving dedupe write performance. However, backup storage dedupe read performance is also a crucial problem for storage recovery. This paper designs a new dedupe storage read cache for backup applications that improves read performance by exploiting a special characteristic: the read sequence is the same as the write sequence. Consequently, for better cache utilization, by looking ahead for future references within a moving window, it evicts victims from the cache having the smallest future access. Moreover, to further improve read cache performance, it maintains a small log buffer to judiciously cache future access data chunks. Extensive experiments with real-world backup workloads demonstrate that the proposed read cache scheme improves read performance by up to 64.3%

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Srinivasan K, Bisson T, Goodson G, Voruganti K. iDedup: Latency-aware, inline data deduplication for primary storage. In Proc. the 10th USENIX Conference on File and Storage Technologies, February 2012, pp.299-312.

  2. Nam Y, Park D, Du D H C. Virtual USB drive: A key component for smart home storage architecture. In Proc. IEEE International Conference on Consumer Electronics, January 2012, pp.23-24.

  3. Fu M, Feng D, Hua Y, He X B, Chen Z N, Xia W, Zhang Y C, Tan Y J. Design tradeoffs for data deduplication performance in backup workloads. In Proc. the 13th USENIX Conference on File and Storage Technologies, February 2015, pp.331-344.

  4. Fu Y J, Xiao N, Liao X K, Liu F. Application-aware client-side data reduction and encryption of personal data in cloud backup services. J. Comput. Sci. Technol., 2013, 28(6): 1012-1024.

  5. Dong W, Douglis F, Li K, Pattersonet H, Reddy S, Shilane P. Tradeoffs in scalable data routing for deduplication clusters. In Proc. the 9th USENIX Conference on File and Storage Technologies, February 2011, pp.15-29.

  6. Dubnicki C, Gryz L, Heldt L, Kaczmarczyk M, Kilian W, Strzelczak P, Szczepkowski J, Ungureanu C, Welnicki M. HYDRAstor: A scalable secondary storage. In Proc. the 7th Conference on File and Storage Technologies, February 2009, pp.197-210.

  7. Debnath B, Sengupta S, Li J. ChunkStash: Speeding up inline storage deduplication using flash memory. In Proc. USENIX Conference on USENIX Annual Technical Conference, June 2010.

  8. Zhu B, Li K, Patterson H. Avoiding the disk bottleneck in the data domain deduplication file system. In Proc. the 6th USENIX Conference on File and Storage Technologies, February 2008, Article No. 18.

  9. Meyer D T, Bolosky W J. A study of practical deduplication. In Proc. the 9th USENIX Conference on File and Storage Technologies, February 2011.

  10. Seo M K, Lim S H. Deduplication flash file system with PRAM for non-linear editing. IEEE Transactions on Consumer Electronics, 2010, 56(3): 1502-1510.

  11. Min J, Yoon D, Won Y. Efficient deduplication techniques for modern backup operation. IEEE Transactions on Computers, 2011, 60(6): 824-840.

  12. Fineberg S, Rabinovici-Cohen S. Long term information retention. In Proc. Storage Developer Conference, September 2011. http://www.snia.org/events/storage-developer2011/agenda2011/abstracts#long_term_ret, Dec. 2016.

  13. Kay D, Maybee M. Aspects of deduplication. Storage Networking Industry Association (SNIA) Tutorial, January 2010. http://www.snia.org/sites/default/education/tutorials/2010/spring/file/MarkMaybee-DominicKay_Aspects_of_Deduplication.pdf, Dec. 2016.

  14. Kim Y. Consolidate more: High-performance primary deduplication in the age of abundant capacity. Hitachi Data Systems, March 2013. http://www.slideshare.net/hd-scorp/consolidate-more-high-performance-primary-deduplication-in-the-age-of-abundant-capacity, Dec. 2016.

  15. Nam Y, Park D, Du D H C. Assuring demanded read performance of data deduplication storage with backup datasets. In Proc. the 20th IEEE International Symposium on Modeling, Analysis and Simulations of Computer and Telecommunication Systems, August 2012, pp.201-208.

  16. Lillibridge M, Eshghi K, Bhagwat D. Improving restore speed for backup systems that use inline chunk-based deduplication. In Proc. the 11th USENIX Conference on File and Storage Technologies, February 2013, pp.183-198.

  17. Welnicki M, Szczepkowski J, Dubnicki C. System and method for deduplication of distributed data. US Patent 9, 256, 368, February 2016. http://patft.uspto.gov/netac-gi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=9256368.PN.&OS=PN/9256368&RS=PN/9256368, Dec. 2016.

  18. Efstathopoulos P, Guo F L. Rethinking deduplication scalability. In Proc. the 2nd USENIX Conference on Hot Topics in Storage and File Systems, June 2010.

  19. Nam Y, Lu G L, Park N, Xiao W J, Du D H C. Chunk fragmentation level: An effective indicator for read performance degradation in deduplication storage. In Proc. the 13th IEEE International Conference on High Performance Computing and Communications, September 2011, pp.581-586.

  20. Rabin M O. Fingerprinting by random polynomials. Technical Report TR-1581, 1981. http://www.citeulike.org/user/dmeister/article/2706665, Nov. 2016.

  21. Liu C Y, Gu Y, Sun L C, Yan B, Wang D S. RADMAD: High reliability provision for large-scale de-duplication archival storage systems. In Proc. the 23rd International Conference on Supercomputing, June 2009, pp.370-379.

  22. Jin K R, Miller E L. The effectiveness of deduplication on virtual machine disk images. In Proc. SYSTOR 2009: The Israeli Experimental Systems Conference, May 2009, Article No. 7.

  23. Lim H, Fan B, Andersen D G, Kaminsky M. SILT: A memory-efficient, high-performance key-value store. In Proc. the 23rd ACM Symposium on Operating Systems Principles, October 2011, pp.1-13.

  24. Park N, Lilja D J. Characterizing datasets for data deduplication in backup applications. In Proc. IEEE International Symposium on Workload Characterization, December 2010.

  25. Bhagwat D, Eshghi K, Long D D E, Lillibridge M. Extreme Binning: Scalable, parallel deduplication for chunk-based file backup. In Proc. the 17th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, September 2009.

  26. Lillibridge M, Eshghi K, Bhagwat D, Deolalikar V, Trezise G, Camble P. Sparse indexing: Large scale, inline deduplication using sampling and locality. In Proc. the 7th Conference on File and Storage Technologies, February 2009, pp.111-123.

  27. Lu G L, Jin Y, Du D H C. Frequency based chunking for data de-duplication. In Proc. IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, August 2010, pp.287-296.

  28. Tsuchiya Y, Watanabe T. DBLK: Deduplication for primary block storage. In Proc. the 27th IEEE Symposium on Massive Storage Systems and Technologies, May 2011.

  29. Mao B, Jiang H, Wu S Z, Fu Y J, Tian L. Read-performance optimization for deduplication-based storage systems in the cloud. ACM Transactions on Storage, 2014, 10(2): Article No. 6.

  30. Wallace G, Douglis F, Qian H W, Shilane P, Smaldone S, Chamness M, Hsu W. Characteristics of backup workloads in production systems. In Proc. the 10th USENIX Conference on File and Storage Technologies, February 2012.

  31. Park D, Du D H C. Hot data identification for flash-based storage systems using multiple bloom filters. In Proc. the 27th IEEE Symposium on Mass Storage Systems and Technologies, May 2011.

  32. Megiddo N, Modha D. ARC: A self-tuning, low overhead replacement cache. In Proc. the 2nd USENIX Conference on File and Storage Technologies, March 2003, pp.115-130.

  33. Bucy J S, Schindler J, Schlosser S W, Ganger G R. The DiskSim simulation environment version 4.0 reference manual. Technical Report CMU-PDL-08-101, 2008. http://www.pdl.cmu.edu/PDL-FTP/DriveChar/CMU-PDL-08-101.pdf, Nov. 2016.

  34. Fu M, Feng D, Hua Y, He X B, Chen Z N, Xia W, Huang F T, Liu Q. Accelerating restore and garbage collection in deduplication-based backup systems via exploiting historical information. In Proc. USENIX Conference on USENIX Annual Technical Conference, June 2014, pp.181-192.

  35. Li W J, Jean-Baptise G, Riveros J, Narasimhan G, Zhang T, Zhao M. CacheDedup: In-line deduplication for flash caching. In Proc. the 14th USENIX Conference on File and Storage Technologies, February 2016, pp.301-314.

  36. Koller R, Rangaswami R. I/O deduplication: Utilizing content similarity to improve I/O performance. In Proc. the 8th USENIX Conference on File and Storage Technologies, February 2010.

  37. Li Y K, Xu M, Ng C H, Lee P P C. Efficient hybrid inline and out-of-line deduplication for backup storage. ACM Transactions on Storage, 2015, 11(1): Article No. 2.

  38. Tan Y J, Yan Z C, Feng D, He X B, Zou Q, Yang L. De-Frag: An efficient scheme to improve deduplication performance via reducing data placement de-linearization. Cluster Computing, 2015, 18(1): 79-92.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Minnesota-Twin Cities, Minneapolis, MN, 55455, USA

    Dongchul Park, Ziqi Fan, Young Jin Nam & David H. C. Du

Authors
  1. Dongchul Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ziqi Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Young Jin Nam
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David H. C. Du
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dongchul Park.

Additional information

This work is partially supported by the National Science Foundation Awards of USA under Grant Nos. 121756, 1305237, 142191 and 1439622.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Park, D., Fan, Z., Nam, Y.J. et al. A Lookahead Read Cache: Improving Read Performance for Deduplication Backup Storage. J. Comput. Sci. Technol. 32, 26–40 (2017). https://doi.org/10.1007/s11390-017-1680-8

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-017-1680-8

Keywords

Search

Navigation