Advertisement

Designs, Codes and Cryptography

, Volume 87, Issue 12, pp 2753–2770 | Cite as

A new piggybacking design for systematic MDS storage codes

  • Chong Shangguan
  • Gennian GeEmail author
Article
  • 51 Downloads

Abstract

Distributed storage codes have important applications in the design of modern storage systems. In a distributed storage system, every storage node has a probability to fail and once an individual storage node fails, it must be reconstructed using the data stored in the surviving nodes. Computation load and network bandwidth are two important issues we need to concern when repairing a failed node. Generally speaking, the naive maximum distance separable (MDS) storage codes have low repair complexity but high repair bandwidth. On the contrary, minimum storage regenerating codes have low repair bandwidth but high repair complexity. Fortunately, the newly introduced piggybacked codes combine the advantages of both ones. The main result of this paper is a novel piggybacking design framework for \((k+r,k)\) systematic MDS storage codes, where kr denote the number of systematic nodes and the number of parity nodes, respectively. In the new code, the average repair bandwidth rate for the systematic nodes, i.e., the ratio of the average repair bandwidth of a single failed systematic node and the amount of the original data, can be as low as \(\sqrt{\frac{2}{r}}+\frac{1}{2r}+\frac{3}{k}+\frac{\sqrt{2r}}{k^2}\), which is roughly \(\sqrt{\frac{2}{r}}+\frac{1}{2r}\) when the code has high rate \(k\gg r\). For relatively large r (e.g., \(r\ge 6\)), this result significantly improves the previously known one which has average repair bandwidth rate roughly \(\frac{r-1}{2r-1}\). In the meanwhile, every failed systematic node of the new code can be reconstructed quickly using the decoding algorithm of a classical MDS code, only with some additional additions over the underlying finite field.

Keywords

Distributed storage system Systematic MDS code Piggybacked code 

Mathematics Subject Classification

68P20 68P30 

Notes

Acknowledgements

The authors express their great gratitude to the two anonymous reviewers for their detailed and constructive comments which are very helpful to the improvement of the presentation of this paper.

References

  1. 1.
    Bhagwan R., Tati K., Cheng Y., Savage S., Voelker G.M.: Total recall: system support for automated availability management. Nsdi 1, 337–350 (2004).Google Scholar
  2. 2.
  3. 3.
    Dimakis A.G., Godfrey P.B., Wu Y., Wainwright M.J.: Network coding for distributed storage systems. IEEE Trans. Inf. Theory 56(9), 4539–4551 (2010).CrossRefGoogle Scholar
  4. 4.
    Gopalan P., Huang C., Simitci H., Yekhanin S.: On the locality of codeword symbols. IEEE Trans. Inf. Theory 58(11), 6925–6934 (2012).MathSciNetCrossRefGoogle Scholar
  5. 5.
    Goparaju S., Tamo I., Calderbank R.: An improved sub-packetization bound for minimum storage regenerating codes. IEEE Trans. Inf. Theory 60(5), 2770–2779 (2014).MathSciNetCrossRefGoogle Scholar
  6. 6.
    Goparaju S., Fazeli A., Vardy A.: Minimum storage regenerating codes for all parameters. IEEE Trans. Inf. Theory 63(10), 6318–6328 (2017).MathSciNetCrossRefGoogle Scholar
  7. 7.
  8. 8.
    Huang C., Simitci H., Xu Y., Ogus A., Calder B., Gopalan P., Li J., Yekhanin S.: Erasure coding in windows azure storage. In: Usenix Conference on Technical Conference, p. 2 (2012).Google Scholar
  9. 9.
    Kubiatowicz J., Bindel D., Chen Y., Czerwinski S., Eaton P., Geels D., Gummadi R., Rhea S., Weatherspoon H., Wells C.: OceanStore: an architecture for global-scale persistent storage. ACM SIGPLAN Not. 35(11), 190–201 (2002).CrossRefGoogle Scholar
  10. 10.
    Kumar S., i Amat A.G., Andriyanova I., Brännström F.: A family of erasure correcting codes with low repair bandwidth and low repair complexity. In: IEEE Global Communication Conference, pp. 1–6 (2015).Google Scholar
  11. 11.
    Rashmi K.V., Shah N.B., Gu D., Kuang H., Borthakur D., Ramchandran K.: A solution to the network challenges of data recovery in erasure-coded distributed storage systems: a study on the facebook warehouse cluster. Usenix Hotstorage (2013)Google Scholar
  12. 12.
    Rashmi K.V., Shah N.B., Gu D., Kuang D., Borthakur D., Ramchandran K.: A Hitchhiker’s guide to fast and efficient data reconstruction in erasure-coded data centers. ACM SIFCOMM Comput. Commun. Rev. 44, 331–342 (2014).CrossRefGoogle Scholar
  13. 13.
    Rashmi K.V., Shah N.B., Ramchandran K.: A piggybacking design framework for read-and download-efficient distributed storage codes. IEEE Trans. Inf. Theory 63(9), 5802–5820 (2017).MathSciNetzbMATHGoogle Scholar
  14. 14.
    Rawat A.S., Koyluoglu O.O., Vishwanath S.: Progress on high-rate MSR codes: enabling arbitrary number of helper nodes. In: Information Theory and Applications Workshop (ITA), IEEE, pp. 1–6 (2016).Google Scholar
  15. 15.
    Sasidharan B., Vajha M., Kumar P.V.: An explicit, coupled-layer construction of a high-rate MSR code with low sub-packetization level, small field size and all-node repair. arXiv preprint arXiv:1607.07335 (2016).Google Scholar
  16. 16.
    Sasidharan B., Vajha M., Kumar P.V.: An explicit, coupled-layer construction of a high-rate MSR code with low sub-packetization level, small field size and $d<(n- 1)$. In: Proceedings of IEEE International Symposium on Information Theory, pp. 2048–2052 (2017)Google Scholar
  17. 17.
    Singleton R.C.: Maximum distance $q$-nary codes. IEEE Trans. Inf. Theory IT–10, 116–118 (1964).MathSciNetCrossRefGoogle Scholar
  18. 18.
    Tamo I., Barg A.: A family of optimal locally recoverable codes. IEEE Trans. Inf. Theory 60(8), 4661–4676 (2013).MathSciNetCrossRefGoogle Scholar
  19. 19.
    Tamo I., Wang Z., Bruck J.: Zigzag codes: MDS array codes with optimal rebuilding. IEEE Trans. Inf. Theory 59(3), 1597–1616 (2015).MathSciNetCrossRefGoogle Scholar
  20. 20.
    Wang Z., Tamo I., Bruck J.: Long MDS codes for optimal repair bandwidth. In: Proceedings of IEEE International Symposium on Information Theory, pp. 1182–1186 (2012)Google Scholar
  21. 21.
    Yang B., Tang X., Li J.: A systematic piggybacking design for minimum storage regenerating codes. IEEE Trans. Inf. Theory 61(11), 5779–5786 (2015).MathSciNetCrossRefGoogle Scholar
  22. 22.
    Ye M., Barg A.: Explicit constructions of high-rate MDS array codes with optimal repair bandwidth. IEEE Trans. Inf. Theory 63(4), 2001–2014 (2017).MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Mathematical SciencesCapital Normal UniversityBeijingChina
  2. 2.School of Mathematical SciencesZhejiang UniversityHangzhouChina

Personalised recommendations