A Performance Evaluation of Erasure Coding Libraries for Cloud-Based Data Stores
Erasure codes have been widely used over the last decade to implement reliable data stores. They offer interesting trade-offs between efficiency, reliability, and storage overhead. Indeed, a distributed data store holding encoded data blocks can tolerate the failure of multiple nodes while requiring only a fraction of the space necessary for plain replication, albeit at an increased encoding and decoding cost. There exists nowadays a number of libraries implementing several variations of erasure codes, which notably differ in terms of complexity and implementation-specific optimizations.
Seven years ago, Plank et al.  have conducted a comprehensive performance evaluation of open-source erasure coding libraries available at the time to compare their raw performance and measure the impact of different parameter configurations. In the present experimental study, we take a fresh perspective at the state of the art of erasure coding libraries. Not only do we cover a wider set of libraries running on modern hardware, but we also consider their efficiency when used in realistic settings for cloud-based storage, namely when deployed across several nodes in a data centre. Our measurements therefore account for the end-to-end costs of data accesses over several distributed nodes, including the encoding and decoding costs, and shed light on the performance one can expect from the various libraries when deployed in a real system. Our results reveal important differences in the efficiency of the different libraries, notably due to the type of coding algorithm and the use of hardware-specific optimizations.
- 1.Calder, B., Wang, J., Ogus, A., Nilakantan, N., Skjolsvold, A., McKelvie, S., Xu, Y., Srivastav, S., Wu, J., Simitci, H., Haridas, J., Uddaraju, C., Khatri, H., Edwards, A., Bedekar, V., Mainali, S., Abbasi, R., Agarwal, A., Fahim ul Haq, M., Ikram ul Haq, M., Bhardwaj, D., Dayanand, S., Adusumilli, A., McNett, M., Sankaran, S., Manivannan, K., Rigas, L.: Windows Azure Storage: a highly available cloud storage service with strong consistency. In: Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, SOSP 2011, pp. 143–157. ACM, New York (2011)Google Scholar
- 4.Greenan, K.M., Li, X., Wylie, J.J.: Flat XOR-based erasure codes in storage systems: constructions, efficient recovery, and tradeoffs. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–14. IEEE (2010)Google Scholar
- 6.Huang, C., Simitci, H., Xu, Y., Ogus, A., Calder, B., Gopalan, P., Li, J., Yekhanin, S.: Erasure coding in Windows Azure Storage. In: Proceedings of the USENIX Annual Technical Conference, USENIX ATC, pp. 15–26 (2012)Google Scholar
- 7.Khan, O., Burns, R.C., Plank, J.S., Pierce, W., Huang, C.: Rethinking erasure codes for cloud file systems: minimizing I/O for recovery and degraded reads. In: Proceedings of the 7th Conference on File and Storage Technologies, FAST 2012, p. 20. USENIX Association (2012)Google Scholar
- 9.Luby, M., Zuckermank, D.: An XOR-based erasure-resilient coding scheme. Technical report (1995)Google Scholar
- 10.Luby, M.G., Mitzenmacher, M., Shokrollahi, M.A., Spielman, D.A., Stemann, V.: Practical loss-resilient codes. In: Proceedings of the Twenty-Ninth Annual ACM Symposium on Theory of Computing, pp. 150–159. ACM (1997)Google Scholar
- 13.Plank, J.S., Greenan, K.M., Miller, E.L.: Screaming fast Galois field arithmetic using Intel SIMD instructions. In: FAST, pp. 299–306 (2013)Google Scholar
- 14.Plank, J.S., Luo, J., Schuman, C.D., Xu, L., Wilcox-O’Hearn, Z.: A performance evaluation and examination of open-source erasure coding libraries for storage. In: Proceedings of the 7th Conference on File and Storage Technologies, FAST 2009, pp. 253–265. USENIX Association, Berkeley (2009)Google Scholar
- 15.Plank, J.S., Simmerman, S., Schuman, C.D.: Jerasure: A library in C/C++ facilitating erasure coding for storage applications-version 1.2. Technical report, Technical Report CS-08-627, University of Tennessee (2008)Google Scholar
- 16.Plank, J.S., Xu, L.: Optimizing Cauchy Reed-Solomon codes for fault-tolerant network storage applications. In: 2006 Fifth IEEE International Symposium on Network Computing and Applications, NAC 2006, pp. 173–180. IEEE (2006)Google Scholar
- 18.Sankaran, J.: Reed Solomon decoder: TMS320C64x implementation. Application Report SPRA686, Texas Instruments (2000)Google Scholar