Skip to main content
SpringerLink
Log in

Erasure-Coded Hybrid Writes Based on Data Delta

  • Published:
International Journal of Parallel Programming Aims and scope Submit manuscript

Abstract

Erasure coding is extensively deployed in today’s data centers to tackle prevalent failures, because it can offer higher reliability at lower storage overhead than data replication. However, for each small write, erasure-coded storage systems have to perform a partial write to an entire erasure coding group, resulting in a time-consuming write-after-read. This paper presents DABRI, an erasure-coded hybrid write approach based on data delta for fast partial writes. DABRI uses data deltas that are the differences between latest data values and original data values, instead of parity deltas to recover the failed data. The data node sends the latest data instead of the parity delta to parity nodes for each partial write. The original data stored on the data node is read and sent to the parity nodes, only when the data stored on the parity nodes is insufficient to maintain the data reliability. This can bypass the computation of parity deltas and reduce the number of data reads. For a series of n partial writes to the same data, DABRI performs log-based updates for data and parity in the first write, performs in-place data updates and log-based parity updates for the last n-1 writes. In addition, the I/O between data nodes and parity nodes is scheduled for parallel I/O in each partial write. We implement an erasure-coded prototype storage system based on DABRI to perform performance evaluation. Experimental results running the real-world traces show that DABRI can significantly improve the I/O throughput, compared with the state-of-the-arts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Wang, Y., Pei, X., Ma, X., et al.: TA-update: an adaptive update scheme with tree-structured transmission in erasure-coded storage systems. IEEE Trans. Parallel Distrib. Syst. 29(8), 1893–1906 (2017)

    Article  Google Scholar 

  2. Huang, J., Xia, J., Qin, X., et al.: Optimization of small updates for erasure-coded in-memory stores. Comput. J. 62(6), 869–883 (2019)

    Article  Google Scholar 

  3. Silberstein, M., Ganesh, L., Wang, Y., et al.: Lazy means smart: Reducing repair bandwidth costs in erasure-coded distributed storage. In: International Conference on Systems and Storage, pp. 1–7 (2014)

  4. Ye, L., Feng, D., Hu, Y., et al.: Hybrid codes: flexible erasure codes with optimized recovery performance. ACM Trans. Storage 16(4), 1–26 (2020)

    Article  Google Scholar 

  5. Huang, C., Simitci, H., Xu, Y., et al.: Erasure coding in windows azure storage. In: USENIX Annual Technical Conference, pp. 15–26 (2012)

  6. Chen, Y., Mu, S., Li, J., et al.: Giza: erasure coding objects across global data centers. In: USENIX Annual Technical Conference, pp. 539–551 (2017)

  7. Subedi, P., Huang, P., Young, B., et al.: FINGER: a novel erasure coding scheme using fine granularity blocks to improve Hadoop write and update performance. In: IEEE International Conference on Networking, Architecture and Storage, pp. 255–264 (2015)

  8. Wei, B., Xiao, L., Song, Y., et al.: A self-tuning client-side metadata prefetching scheme for wide area network file systems. Sci. China Inform. Sci. 65(3), 1–17 (2022)

    Article  Google Scholar 

  9. Chan, J., Ding, Q., Lee, P., et al.: Parity logging with reserved space: towards efficient updates and recovery in erasure-coded clustered storage. In: USENIX Annual Technical Conference, pp. 163–176 (2014)

  10. Wei, B., Xiao, L., Zhou, B., et al.: Fine-grained management of I/O optimizations based on workload characteristics. Front. Comput. Sci. 15(3), 1–14 (2021)

    Article  Google Scholar 

  11. Shvachko, K., Kuang, H., Radia, S., et al.: The Hadoop distributed file system. In: IEEE Symposium on Massive Storage Systems and Technologies, pp. 1–10 (2010)

  12. Shen, J., Zhang, K., Gu, J., et al.: Efficient scheduling for multi-block updates in erasure coding based storage systems. IEEE Trans. Comput. 67(4), 573–581 (2017)

    Article  MathSciNet  Google Scholar 

  13. Jin, C., Feng, D., Jiang, H., et al.: RAID6L: a log-assisted RAID6 storage architecture with improved write performance. In: IEEE Symposium on Massive Storage Systems and Technologies, pp. 1–6 (2011)

  14. Plank, J., Greenan, K., Miller, E.: Screaming fast Galois field arithmetic using intel SIMD instructions. In: USENIX Conference on File and Storage Technologies, pp. 299–306 (2013)

  15. Zhang, Y., Li, H., Liu, S., et al.: PBS: an efficient erasure-coded block storage system based on speculative partial writes. ACM Trans. Storage 16(1), 1–25 (2020)

    Google Scholar 

  16. Ghemawat, S., Gobioff, H., Leung, S.: The Google file system. In: ACM Symposium on Operating Systems Principles, pp. 29–43 (2003)

  17. Hu, Y., Cheng, L., Yao, Q., et al.: Exploiting combined locality for wide-stripe erasure coding in distributed storage. In: USENIX Conference on File and Storage Technologies, pp. 233–248 (2021)

  18. Pamies-Juarez, L., Blagojevic, F., Mateescu, R., et al.: Opening the chrysalis: On the real repair performance of MSR codes. In: USENIX Conference on File and Storage Technologies, pp. 81–94 (2016)

  19. Rashmi, K., Nakkiran, P., Wang, J., et al.: Having your cake and eating it too: jointly optimal erasure codes for I/O, storage, and network-bandwidth. In: USENIX Conference on File and Storage Technologies, pp. 81–94 (2015)

  20. Vajha, M., Ramkumar, V., Puranik, B., et al.: Clay codes: moulding MDS codes to yield an MSR code. In: USENIX Conference on File and Storage Technologies, pp. 139–153 (2018)

  21. Li, R., Li, X., Lee, P.P.C., et al.: Repair pipelining for erasure-coded storage. In: USENIX Annual Technical Conference, pp. 567–579 (2017)

  22. Mitra, S., Panta, R., Ra, M.R., et al.: Partialparallel-repair (PPR): a distributed technique for repairing erasure coded storage. In: European Conference on Computer Systems, pp. 1–16 (2016)

  23. Lu, X., Wang, H., Wang, J., et al.: Internet-based virtual computing environment: beyond the data center as a computer. Future Gener. Comput. Syst. 29(1), 309–322 (2013)

    Article  Google Scholar 

  24. Xia, M., Saxena, M., Blaum, M., et al.: A tale of two erasure codes in HDFS. In: USENIX Conference on File and Storage Technologies, pp. 213–226 (2015)

  25. Liu, Y., Wei, B., Wu, J., et al.: Erasure-coded multi-block updates based on hybrid writes and common XORs first. In: IEEE International Conference on Computer Design, pp. 472–479 (2021)

  26. Shen, Z., Lee, P.P., Shu, J., et al.: Correlation-aware stripe organization for efficient writes in erasure-coded storage: algorithms and evaluation. IEEE Trans. Parallel Distrib. Syst. 30(7), 1552–1564 (2015)

    Article  Google Scholar 

  27. Shen, Z., Lee, P.P.: Cross-rack-aware updates in erasure-coded data centers: design and evaluation. IEEE Trans. Parallel Distrib. Syst. 31(10), 2315–2328 (2020)

    Article  Google Scholar 

  28. Gong, G., Shen, Z., Wu, S., et al.: Optimal rack-coordinated updates in erasure-coded data centers. In: IEEE Conference on Computer Communications, pp. 1–10 (2021)

Download references

Acknowledgements

This work was supported in part by the China Postdoctoral Science Foundation under Grant No. 2021M690733, the National Natural Science Foundation of China under Grant No. 62362019, and the Natural Science Foundation of Hainan Province under Grant No. 624RC482.

Author information

Authors and Affiliations

  1. School of Cyberspace Security, Hainan University, Haikou, China

    Bing Wei

  2. School of Computer Science and Engineering, Beihang University, Beijing, China

    Bing Wei, Chenhao Zhang & Limin Xiao

  3. School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China

    Qiang Huang & Hui Chen

Authors
  1. Bing Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiang Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hui Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chenhao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Limin Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bing Wei.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, B., Huang, Q., Chen, H. et al. Erasure-Coded Hybrid Writes Based on Data Delta. Int J Parallel Prog (2024). https://doi.org/10.1007/s10766-024-00773-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10766-024-00773-0

Keywords

Search

Navigation