Advertisement

Concurrent Regeneration Code with Local Reconstruction in Distributed Storage Systems

Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 354)

Abstract

Reed-Solomon (RS) codes are a standard erasure code choice and their repair cost is so high that it is a penalty for storage efficiency and high reliability. In this paper, we propose a novel class of Concurrent Regeneration codes with Local reconstruction (CRL), that enjoy three advantages: (1) to minimize the network bandwidth for node repair, (2) to minimize the number of accessed nodes, and (3) faster reconstruction in distributed storage systems. We show how they overcome the limitation of RS codes, and we demonstrate that they are optimal on a trade-off between minimum distance and locality. By conducting performance evaluation in both memory and JBOD environments, experimental results demonstrate the performance of the CRL codes.

Keywords

Regeneration codes Local construction codes Storage system 

Notes

Acknowledgments

This work is supported by A*STAR under Grant No. R15GAP-0004.

References

  1. 1.
    Xu, Q., Arumugam, R.V., Yong, K.L., Mahadevan, S.: DROP: facilitating distributed metadata management in EB-scale storage systems. In: MSST, pp. 1–10 (2013)Google Scholar
  2. 2.
    Huang, C., Simitci, H., Xu, Y., Ogus, A., Calder, B., Gopalan, P., Li, J., Yekhanin, S.: Erasure coding in windows azure storage. In: 2012 USENIX Annual Technical Conference, Boston, MA, USA, June 13–15, 2012, pp. 15–26 (2012)Google Scholar
  3. 3.
    Sathiamoorthy, M., Asteris, M., Papailiopoulos, D.S., Dimakis, A.G., Vadali, R., Chen, S., Borthakur, D.: Xoring elephants: novel erasure codes for big data. PVLDB 6(5), 325–336 (2013)Google Scholar
  4. 4.
    Rashmi, K.V., Shah, N.B., Gu, D., Kuang, H., Borthakur, D., Ramchandran, K.: A “hitchhiker’s” guide to fast and efficient data reconstruction in erasure-coded data centers. In: ACM SIGCOMM’14, pp. 331–342 (2014)Google Scholar
  5. 5.
    Xu, Q., Arumugam, R.V., Yong, K.L., Mahadevan, S.: Efficient and scalable metadata management in EB-scale file systems. IEEE Trans. Parallel Distrib. Syst. 25(11), 2840–2850 (2014)CrossRefGoogle Scholar
  6. 6.
    Dimakis, A., Godfrey, P., Wu, Y., Wainwright, M., Ramchandran, K.: Network coding for distributed storage systems. IEEE Trans. Inf. Theory 56(9), 4539–4551 (2010)CrossRefGoogle Scholar
  7. 7.
    Xu, Q., Ng, H. W., Xi, Q., Jin C.: Regenerating-Local Reconstruction Codes for Distributed Storage Systems. To appear in the 7th ICFCC (2015)Google Scholar
  8. 8.
    Wu, Y., Dimakis, A.G.: Reducing repair traffic for erasure coding-based storage via interference alignment. ISIT 2009, 2276–2280 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Quanqing Xu
    • 1
  • Weiya Xi
    • 1
  • Khai Leong Yong
    • 1
  • Chao Jin
    • 1
  1. 1.Data Storage InstituteA*STARSingaporeSingapore

Personalised recommendations