Advertisement

On the Optimal Provider Selection for Repair in Distributed Storage System with Network Coding

  • Chengjin Jia
  • Jin WangEmail author
  • Yanqin Zhu
  • Xin Wang
  • Kejie Lu
  • Xiumin Wang
  • Zhengqing Wen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9531)

Abstract

In large-scale distributed storage systems (DSS), reliability is provided by redundancy spread over storage servers across the Internet. Network coding (NC) has been widely studied in DSS because it can improve the reliability with low repair time. To maintain reliability, an unavailable storage server should be firstly replaced by a new server, named new comer. Then, multiple storage servers, called providers, should be selected from surviving servers and send their coded data through the Internet to the new comer for regenerating the lost data. Therefore, in a large-scale DSS, provider selection and data routing during the regeneration phase have great impact on the performance of regeneration time. In this paper, we investigate a problem of optimal provider selection and data routing for minimizing the regeneration time in the DSS with NC. Specifically, we first define the problem in the DSS with NC. For the case that the providers are given, we model the problem as a mathematical programming. Based on the mathematical programming, we then formulate the optimal provider selection and data routing problem as an integer linear programming problem and develop an efficient near-optimal algorithm based on linear programming relaxation (BLP). Finally, extensive simulation experiments have been conducted, and the results show the effectiveness of the proposed algorithm.

Keywords

Network coding Distributed storage system Provider selection Routing Linear programming LP relaxation 

Notes

Acknowledgments

This work was supported in part by Natural Science Foundation of China No. 61202378, 61373164, China Postdoctoral Science Foundation No. 2013M531402, 2014T70544, Research Project of Jiangsu Province No. BY2013030-06 and Application Foundation Research of Suzhou No. SYG201401.

References

  1. 1.
  2. 2.
    Ghemawat, S., Gobioff, H., Leung, S.-T.: The google file system. ACM SIGOPS Oper. Syst. Rev. 37(5), 29–43 (2003)CrossRefGoogle Scholar
  3. 3.
    Calder, B.: Windows azure storage: a highly available cloud storage service with strong consistency. In: Proceedings of the 23rd ACM Sympossium on Operating Systems Principles (2011)Google Scholar
  4. 4.
    Reed, I., Solomon, G.: Polynomial codes over certain finite fields. J. Soc. Ind. Appl. Math. 8(2), 300–304 (1960)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Ahlswede, R., Cai, N., Li, S.-Y.R., Yeung, R.W.: Network information flowem. In: Proceedings of the 8th International Workshop on Quality of Service (IWQOS), vol. 46(4), pp. 1204–1216 (2000)Google Scholar
  6. 6.
    Hu, Y., Chen, H., Lee, P., Tang, Y.: NCCloud: applying network coding for the storage repair in a cloud-of-clouds. In: Proceedings of the 10th USENIX Annual Technical Conference (2012)Google Scholar
  7. 7.
    Rashmi, K.V., Shah, N.B., Kumar, P.V.: Optimal exact-regenerating codes for distributed storage at the MSR and MBR points via a productmatrix construction. In: Proceedings of 19th International Workshop on Quality of Service (IWQOS), vol. 57(8), pp. 5227–5239 (2011)Google Scholar
  8. 8.
    Cadambe, V.R., Jafar, S.A., Maleki, H.: Distributed data storage with minimum storage regenerating codes - exact and functional repair are asymptotically equally efficient. arXiv: 1004.4299 [cs.IT] (2010)
  9. 9.
    Wu, Y., Dimakis, A.G., Ramchandran, K.: Deterministic regenerating codes for distributed storage. In: Proceedings of 45th Annual Allerton Conference on Communication, Control, and Computing (2007)Google Scholar
  10. 10.
    Li, J., Yang, S., Wang, X., Li, B.: Tree-structured data regeneration in distributed storage systems with regenerating codes. In: Proceedings of the 29th Internationnal Conference on Computer Communications (2010)Google Scholar
  11. 11.
    Li, J., Yang, S., Wang, X., Xue, X., Li, B.: Tree-structured data regeneration with network coding in distributed storage systems. In: Proceedings of 17th International Workshop on Quality of Service(IWQoS) (2009)Google Scholar
  12. 12.
    Cullina, D., Dimakis, A.G., Ho, T.: Searching for minimum storage regenerating codes. In: Proceedings of 47th Annual Allerton Conference on Communication, Control, and Computing (2009)Google Scholar
  13. 13.
    Hu, Y., Lee, Patrick P.C., Shum, Kenneth W.: Analysis and construction of functionalregenerating codes with uncoded repair for distributed storage systems. In: Proceedings of the 32nd IEEE International Conference on Computer Communications, vol. 63(01), pp. 31–44 (2013)Google Scholar
  14. 14.
    Sun, W., Wang, Y., Pei, X.: Tree-structured parallel regeneration for multiple data losses in distributed storage systems based on erasure codes. Proc. Commun. 10(4), 113–125 (2013)Google Scholar
  15. 15.
    Wang, Y., Wei, D., Yin, X., Wang, X.: Heterogeneity-aware data regeneration in distributed storage systems. In: Proceedings of the 33rd IEEE International Conference on Computer Communications (2014)Google Scholar
  16. 16.
    Al-khedhairi, A.S.: Simulated annealing metaheuristic for solving p-median problem. Int. J. Contemp. Math. Sci. 3(28), 1357–1365 (2008)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Waxman, B.M.: Routing of multipoint connections. Int. J. Sel. Areas Commun. 6(9), 1617–1622 (1988)CrossRefGoogle Scholar
  18. 18.
  19. 19.
    Gong, Q., Wang, J., Wei, D., Wang, J., Wang, X.: Optimal node selection for data regeneration in heterogeneous distributed storage systems. In: Proceedings of the 44th International Conference on Parallel Processing (ICPP) (2015)Google Scholar
  20. 20.
    Gong, Q., Wang, J., Wang, Y., Wei, D., Wang, J., Wang, X.: Topology-aware node selection for data regeneration in heterogeneous distributed storage systems. preprint arXiv: 1506.05579 (2015)

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Chengjin Jia
    • 1
  • Jin Wang
    • 1
    Email author
  • Yanqin Zhu
    • 1
  • Xin Wang
    • 2
  • Kejie Lu
    • 3
    • 4
  • Xiumin Wang
    • 5
  • Zhengqing Wen
    • 1
  1. 1.Department of Computer Science and TechnologySoochow UniversitySuzhouChina
  2. 2.School of Computer ScienceFudan UniversityShanghaiChina
  3. 3.College of Computer Science and TechnologyShanghai University of Electronic PowerShanghaiChina
  4. 4.Department of Electrical and Computer EngineeringUniversity of Puerto Rico at MayagüezMayagüezUSA
  5. 5.School of Computer and InformationHefei University of TechnologyHefeiChina

Personalised recommendations