Efficient Reliability in Volunteer Storage Systems with Random Linear Coding
Volunteer systems pose difficult challenges for data storage. Because of the extremely low reliability of volunteer nodes, these systems require so high redundancy that replication is infeasible. Erasure coding has been proposed to cope with this problem as it needs much less redundancy to achieve the same reliability. Its downside is that the reparation of the system creates high overhead, as fully decoding the original data is required to generate new coded data.
Random linear coding has been proposed to be used as a data storage method, as it provides a better redundancy/reliability ratio, and less control overhead. We propose that it also helps in the reparation of the system, as decoding is not required; instead, coded data can be generated from already existing coded data. However, it may be possible that this iterative reparation leads to degradation of data over time; even more so, if sparse coding is used to increase compute efficiency.
This paper examines the effects of random linear coding and the iterative reparation of the system. It shows the reliability that can be achieved with random linear coding in a highly volatile distributed system. We conclude that random linear coding can achieve high reliability even in highly volatile systems.
KeywordsNetwork Code Sparse Code Erasure Code Target Redundancy Original Block
Unable to display preview. Download preview PDF.
- 1.Random Network Coding Library, https://github.com/avisegradi/rnc-lib (accessed: September 10, 2013)
- 2.Efficient Random Network Coding for Distributed Storage Systems, Ádám Visegrádi and Péter Kacsuk. Euro-Par 2013, MHPC Workshop (2013) (in press)Google Scholar
- 3.Acedanski, S., Deb, S., Médard, M., Koetter, R.: How good is random linear coding based distributed networked storage. In: Workshop on Network Coding, Theory and Applications (2005)Google Scholar
- 5.Anderson, D.: BOINC volunteer data archival, https://boinc.berkeley.edu/trac/wiki/VolunteerDataArchival (accessed January 2014)
- 6.Anderson, D.: BOINC volunteer storage, https://boinc.berkeley.edu/trac/wiki/VolunteerStorage (accessed January 2014)
- 7.Anderson, D.P.: BOINC: a system for public-resource computing and storage. In: Proceedings of the Fifth IEEE/ACM International Workshop on Grid Computing, pp. 4–10 (November 2004)Google Scholar
- 8.Anderson, D.P., Fedak, G.: The computational and storage potential of volunteer computing. In: Sixth IEEE International Symposium on Cluster Computing and the Grid, CCGRID 2006, vol. 1, pp. 73–80. IEEE (2006)Google Scholar
- 9.Borthakur, D.: The hadoop distributed file system: Architecture and design. Hadoop Project Website 11(21) (2007)Google Scholar
- 16.Ma, G., Xu, Y., Lin, M., Xuan, Y.: A content distribution system based on sparse linear network coding. In: NetCod 2007 (2007)Google Scholar
- 20.Wang, M., Li, B.: How practical is network coding? In: 14th IEEE International Workshop on Quality of Service, IWQoS 2006, pp. 274–278 (2006)Google Scholar