Efficient Random Network Coding for Distributed Storage Systems
Making distributed storage systems reliable is an important challenge. Simple replication may cause severe storage overhead when individual components of the system are very unreliable. Using erasure codes is a promising solution for this problem, but it presents its own challenges; it makes the design of such a system very complex, and it presents the problem of reparation. Network coding has been suggested to be used in the communication in these networks to help reduce overhead.
However, using random network coding as—not besides—erasure coding would be an even more promising field to investigate; such a system would have a simple design, need little or no centralization, and reparation of the system could be much simpler than it is in other erasure coding schemes.
The first step on this path is to investigate whether network coding can achieve such a performance that it is a feasible alternative to other erasure codes. This paper presents our experiences about the realization of random network coding based on the discrete logarithm of the finite field. We discuss possible performance optimizations for such a system, and provide performance measurement results focusing on data storage scenarios.
Unable to display preview. Download preview PDF.
- 1.Random Network Coding Library, https://github.com/avisegradi/rnc-lib (accessed: October 09, 2013 )
- 2.Acedanski, S., Deb, S., Médard, M., Koetter, R.: How good is random linear coding based distributed networked storage. In: Workshop on Network Coding, Theory and Applications (2005)Google Scholar
- 4.Anderson, D.: BOINC: a system for public-resource computing and storage. In: Proceedings of the Fifth IEEE/ACM International Workshop on Grid Computing, pp. 4–10 (November 2004)Google Scholar
- 5.Borthakur, D.: The hadoop distributed file system: Architecture and design. Hadoop Project Website 11, 21 (2007)Google Scholar
- 12.Rodrigues, R., Liskov, B.: High availability in DHTs: erasure coding vs. replication. In: Peer-to-Peer Systems IV, pp. 226–239 (2005)Google Scholar
- 13.Wang, M., Li, B.: How practical is network coding? In: 14th IEEE International Workshop on Quality of Service, IWQoS 2006, pp. 274–278 (2006)Google Scholar