Skip to main content
SpringerLink
Log in

Storage and repair bandwidth tradeoff for distributed storage systems with clusters and separate nodes

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

The optimal tradeoff between node storage and repair bandwidth is an important issue for distributed storage systems (DSSs). For realistic DSSs with clusters, while repairing a failed node, downloading more data from intra-cluster nodes than from cross-cluster nodes is effective. Therefore, differentiating the repair bandwidth from intra-cluster and cross-cluster is useful. For cluster DSSs, the tradeoff is considered with special repair assumptions where all alive nodes are used for repairing a failed node. In this paper, we investigate the optimal tradeoff for the cluster DSSs under more general storage/repair parameters. Furthermore, we propose a regenerating code construction strategy that achieves the points in the optimal tradeoff curve for the cluster DSSs with specific parameters as a numerical example. Moreover, we consider the influence of separate nodes for the tradeoff for the DSSs with clusters and separated nodes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Hu C, Li X L, Zhang M, et al. Optimal repair layering for erasure-coded data centers. ACM Trans Storage, 2017, 13: 1–24

    Article  Google Scholar 

  2. Zhang Y, Li H, Li R S. Repair tree: fast repair for single failure in erasure-coded distributed storage systems. IEEE Trans Parallel Distrib Syst, 2017, 28: 1728–1739

    Article  Google Scholar 

  3. Huang C, Chen H, Li J. Pyramid codes: flexible schemes to trade space for access efficiency in reliable data storage systems. In: Proceedings of the 6th IEEE International Symposium on Network Computing and Applications, Cambridge, 2007

    Google Scholar 

  4. Dimakis G, Godfrey P B, Wu Y N, et al. Network coding for distributed storage systems. IEEE Trans Inf Theory, 2010, 56: 4539–4551

    Article  Google Scholar 

  5. Ernvall T, Rouayheb E, Hollanti C, et al. Capacity and security of heterogeneous distributed storage systems. IEEE J Sel Areas Commun, 2013, 31: 2701–2709

    Article  Google Scholar 

  6. Yu Q, Shum W, Sung C W. Tradeoff between storage cost and repair cost in heterogeneous distributed storage systems. Trans Emerg Telecommun Technol, 2015, 26: 1201–1211

    Article  Google Scholar 

  7. Akhlaghi S, Kiani A, Ghanavati R. Cost-bandwidth tradeoff in distributed storage systems. Comput Commun, 2010, 33: 2105–2115

    Article  Google Scholar 

  8. Ford D, Labelle F, Popovici I, et al. Availability in globally distributed storage systems. In: Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation, Vancouver, 2010

    Google Scholar 

  9. Hu C, Lee P P C, Zhang X Y. Double regenerating codes for hierarchical data centers. In: Proceedings of IEEE International Symposium on Information Theory (ISIT), Barcelona, 2016. 245–249

    Google Scholar 

  10. Hou H X, Lee P P C, Shum K W, et al. Rack-aware regenerating codes for data centers. 2018. ArXiv:1802.04031

    Google Scholar 

  11. Prakash N, Abdrashitov V, Médard M. The storage vs repair-bandwidth trade-off for clustered storage systems. IEEE Trans Inf Theory, 2017. ArXiv:1701.04909

    Google Scholar 

  12. Pernas J, Yuen C, Gastón B, et al. Non-homogeneous two-rack model for distributed storage systems. In: Proceedings of IEEE International Symposium on Information Theory (ISIT), Istanbul, 2013. 1237–1241

    Google Scholar 

  13. Sohn Y, Choi B, Yoon S W, et al. Capacity of clustered distributed storage. In: Proceedings of IEEE International Conference on Communications (ICC), Paris, 2017

    Google Scholar 

  14. Kubiatowicz J, Weimer W, Wells C, et al. OceanStore: an architecture for global-scale persistent storage. SIGPLAN Notice, 2000, 35: 190–201

    Article  Google Scholar 

  15. Dimakis G, Ramchandran K, Yunnan Wu K, et al. A survey on network codes for distributed storage. Proc IEEE, 2011, 99: 476–489

    Article  Google Scholar 

  16. Rashmi V, Shah N B, Kumar P V. Optimal exact-regenerating codes for distributed storage at the MSR and MBR points via a product-matrix construction. IEEE Trans Inf Theory, 2011, 57: 5227–5239

    Article  MathSciNet  MATH  Google Scholar 

  17. Li Y R, Yeung R W, Ning Cai R W. Linear network coding. IEEE Trans Inf Theory, 2003, 49: 371–381

    Article  MathSciNet  MATH  Google Scholar 

  18. Ahlswede R, Cai N, Li S, et al. Network information flow. IEEE Trans Inf Theory, 2000, 46: 1204–1216

    Article  MathSciNet  MATH  Google Scholar 

  19. Smith K, Ahuja R K, Magnanti T L, et al. Network flows: theory, algorithms, and applications. J Oper Res Soc, 1994, 45: 1340

    Article  Google Scholar 

  20. Goparaju S, Fazeli A, Vardy A. Minimum storage regenerating codes for all parameters. IEEE Trans Inf Theory, 2017, 63: 6318–6328

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was partially supported by National Natural Science Foundation of China (Grant No. 61571293), China Program of International S&T Cooperation (Grant No. 2016YFE0100300), and SJTU-CUHK Joint Research Collaboration Fund 2018. The authors would like to thank Prof. Kenneth W. Shum for the help in improving this paper.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China

    Jingzhao Wang, Tinghan Wang & Yuan Luo

Authors
  1. Jingzhao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tinghan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuan Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuan Luo.

Additional information

Invited paper

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, J., Wang, T. & Luo, Y. Storage and repair bandwidth tradeoff for distributed storage systems with clusters and separate nodes. Sci. China Inf. Sci. 61, 100303 (2018). https://doi.org/10.1007/s11432-018-9499-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-018-9499-0

Keywords

Search

Navigation