Skip to main content
SpringerLink
Log in

When should I make preservation copies of myself?

And after I do, how will I send messages to my copies?

  • Published:
International Journal on Digital Libraries Aims and scope Submit manuscript

Abstract

We investigate how different replication policies ranging from least aggressive to most aggressive affect the level of preservation achieved by autonomic processes used by web objects (WOs). Based on simulations of small-world graphs of WOs created by the Unsupervised Small-World algorithm, we report quantitative and qualitative results for graphs ranging in order from 10 to 5000 WOs. Our results show that a moderately aggressive replication policy makes the best use of distributed host resources by not causing spikes in CPU resources nor spikes in network activity while meeting preservation goals. We examine different approaches that WOs can communicate with each other and determine the how long it would take for a message from one WO to reach a specific WO, or all WOs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Alam, S.: HTTP mailbox-asynchronous RESTful communication. Master’s thesis, Old Dominion University, Norfolk, VA (2013)

  2. Alam, S., Cartledge, C.L., Nelson, M.L.: HTTP mailbox-asynchronous RESTful communication. Technical report. arXiv:1305.1992 (2013)

  3. Albert, R., Jeong, H., Barabási, A.-L.: Error and attack tolerance of complex networks. Nature 406(6794), 378–382 (2000)

    Article  Google Scholar 

  4. Barabási, A.-L., Albert, R., Jeong, H.: Scale-free characteristics of random networks: the topology of the world wide web. Physica A 281(1), 69–77 (2000)

    Article  Google Scholar 

  5. Beck, M., Moore, T., Plank, J.S.: An end-to-end approach to globally scalable network storage. In: Proceedings of the 2002 conference on applications, technologies, architectures, and protocols for computer communications, pp. 339–346 (2002)

  6. Birman, K.P., Hayden, M., Ozkasap, O., Xiao, Z., Budiu, M., Minsky, Y.: Bimodal multicast. ACM Trans. Comput. Syst. 17(2), 41–88 (1999)

    Article  Google Scholar 

  7. Bollobás, B.: Modern Graph Theory. Springer, New York (1998)

  8. Bollobás, B., Riordan, O., Spencer, J., Tusnády, G.: The degree sequence of a scale-free random graph process. Random Struct Algorithms 18(3), 279–290 (2001)

    Article  MATH  Google Scholar 

  9. Carriero, N., Gelernter, D.: Linda in context. Commun. ACM 32(4), 444–458 (1989)

    Article  Google Scholar 

  10. Cartledge, C.: Preserve Me! (... if you can, using Unsupervised Small-World graphs.). http://ws-dl.blogspot.com/2013/10/2013-10-23-preserve-me-if-you-can-using.html/ (2013)

  11. Cartledge, C.L.: A Framework for Web Object Self-Preservation. PhD thesis, Old Dominion University, Norfolk, VA 23529, August (2014)

  12. Cartledge, C.L., Nelson, M.L.: Self-arranging preservation networks. In: Proceedings of the 8th ACM/IEEE-CS joint conference on digital libraries, pp. 445–445 (2008)

  13. Cartledge, C.L., Nelson, M.L.: Unsupervised creation of small world networks for the preservation of digital objects. In: Proceedings of the 9th ACM/IEEE-CS joint conference on digital libraries, pp. 349–352 (2009)

  14. Cartledge, C.L., Nelson, M.L.: Analysis of graphs for digital preservation suitability. In: Proceedings of the 21st ACM conference on hypertext and hypermedia, pp. 109–118. ACM (2010)

  15. Cartledge, C.L., Nelson, M.L.: Connectivity damage to a graph by the removal of an edge or vertex. Technical report. Old Dominion University, Computer Science Department, Norfolk, VA. arXiv:1103.3075 (2011)

  16. Ciancarini, P., Gorrieri, R., Zavattaro, G.: Towards a calculus for generative communication. Formal Methods Open Object-Based Distrib. Syst. 1, 283 (1997)

    Google Scholar 

  17. Cooper, B., Crespo, A., Garcia-Molina, H.: Implementing a reliable digital object archive. In: Proceedings of the 4th European conference on research and advanced technology for digital libraries, pp. 128–143 (2000)

  18. Cooper, B.F., Garcia-Molina, H.: Peer-to-peer data trading to preserve information. ACM Trans. Inf. Syst. 20(2), 133–170 (2002)

    Article  Google Scholar 

  19. Dabek, F., Kaashoek, M.F., Karger, D., Morris, R., Stoica, I.: Wide-area cooperative storage with CFS. In: Proceedings of the 18th annual ACM symposium on operating systems principles, October (2001)

  20. de la Rosa, J.L., Del Acebo, E., Trias, A., Aciar, S., Quisbert, H.: Crew intelligence systems for digital objects preservation. In: The 2nd swarm intelligence algorithms and applications symposium-SIAAS, vol. 9 (2009)

  21. de la Rosa, J.L., Olvera, J.A.: First studies on self-preserving digital objects. In: Artificial intelligence research and development: Proceedings of the 15th international conference of the Catalan Association for Artificial Intelligence and Applications, pp. 213–222 (2012)

  22. Duchon, P., Hanusse, N., Lebhar, E., Schabanel, N.: Could any graph be turned into a small-world? Theor. Comput. Sci. 355(1), 96–103 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  23. Duchon, P., Hanusse, N., Lebhar, E., Schabanel, N.: Towards small world emergence. In: ACM symposium on parallelism in algorithms and architectures, pp. 225–232 (2006)

  24. Gantz, J., Reinsel, D.: The Digital Universe in 2020: big data, bigger digital shadows, and biggest growth in the far east. IDC iView IDC Anal. Futur. 2007, 1–16 (2012)

  25. Gaume, B., Mathieu, F.: From random graph to small world by wandering. Technical report 6489, Unité de recherche INRIA Rocquencourt (2008)

  26. Gelernter, D., Carriero, N.: Coordination languages and their significance. Commun. ACM 35(2), 97–107 (1992)

    Article  Google Scholar 

  27. Goh, K.I., Kahng, B., Kim, D.: Universal behavior of load distribution in scale-free networks. Phys. Rev. Lett. 87(27), 278701 (2001)

    Article  Google Scholar 

  28. Hunter, J., Choudhury, S.: A semi-automated digital preservation system based on semantic web services. In: Proceedings of the 4th ACM/IEEE-CS joint conference on digital libraries, pp. 269–278 (2004)

  29. Ikeda, S., Kubo, I., Yamashita, M.: The hitting and cover times of random walks on finite graphs using local degree information. Theor. Comput. Sci. 410(1), 94–100 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  30. Kahn, R., Wilensky, R.: A framework for distributed digital object services. Int. J. Digit. Libr. 6(2), 115–123 (2006)

    Article  Google Scholar 

  31. Kleinberg, J.: The small-world phenomenon: an algorithmic perspective. In: Proceedings of the 32nd ACM symposium on theory of computing 32, 163–170 (2000)

  32. Klemm, K., Eguíluz, V.M.: Growing scale-free networks with small-world behavior. Phys. Rev. E. 65(5), 26107 (2002)

    Article  Google Scholar 

  33. Maniatis, P., Roussopoulos, M., Giuli, T.J., Rosenthal, D.S.H., Baker, M.: The LOCKSS peer-to-peer digital preservation system. ACM Trans. Comput. Syst. 23(1), 2–50 (2005)

    Article  Google Scholar 

  34. McCown, F., Nelson, M.L.: What happens when facebook is gone? In: Proceedings of the 9th ACM/IEEE-CS joint conference on digital libraries, pp. 251–254 (2009)

  35. Milian, M.: GeoCities’ Time has Expired. Yahoo Closing the Site Today. Los Angeles Times, Los Angeles, USA (2009)

  36. Miller, I., Freund, J.E.: Probability and Statistics for Engineers. Prentice-Hall, Englewood Cliffs, NJ (1977)

    MATH  Google Scholar 

  37. Nelson, M.L., Van de Sompel, H.: IJDL special issue on complex digital objects: Guest editors’ introduction. Int. J. Digit. Libr. 6(2), 113–114 (2006)

    Article  Google Scholar 

  38. Newman, M.E.J.: Models of the small world: a review. J. Stat. Phys. 101, 819 (2000)

    Article  MATH  Google Scholar 

  39. Nguyen, V., Martel, C.: Analyzing and characterizing small-world graphs. In: ACM-SIAM symposium on discrete algorithms, pp. 311–320 (2005)

  40. Payette, S., Staples, T.: The Mellon Fedora Project. In: Proceedings of the 6th European conference on research and advanced technology for digital libraries, pp. 406–421 (2002)

  41. Rajasekar, A., Wan, M., Moore, R.: MySRB and SRB: components of a data grid. In: Proceedings of the 11th IEEE international symposium on high performance distributed computing, pp. 301–310 (2002)

  42. Rajasekar, A., Wan, M., Moore, R., Schroeder, W.: A prototype rule-based distributed data management system. In: HPDC workshop on next generation distributed sata management (2006)

  43. Ratnasamy, S., Francis, P., Handley, M., Karp, R., Schenker, S.: A Scalable Content-Addressable Network. In: Proceedings of the 2001 conference on applications, technologies, architectures, and protocols for computer communications, pp. 161–172 (2001)

  44. Reich, V.: CLOCKSS—it takes a community. Ser. Libr. 54(1–2), 135–139 (2008)

  45. Reynolds, C.W.: Flocks, herds and schools: a distributed behavioral model. SIGGRAPH Comput. Graph. 21(4), 25–34 (1987)

    Article  Google Scholar 

  46. Rhea, S., Wells, C., Eaton, P., Geels, D., Zhao, B., Weatherspoon, H., Kubiatowicz, J.: Maintenance-free global data storage. IEEE Internet Comput. 5(5), 40–49 (2001)

    Article  Google Scholar 

  47. Rosenthal, D.S.H., Rosenthal, D.C., Miller, E.L., Adams, I.F., Storer, M.W., Zadok, E.: The economics of long-term digital storage. Paper presented at the Memory of the World in the Digital Age, Vancouver, BC (2012)

  48. Rosenthal, D.S.H.: Estimating storage costs. http://blog.dshr.org/2013/11/estimating-storage-costs.html (2013)

  49. Rosenthal, D.S.H., Robertson, T.S., Lipkis, T., Reich, V., Morabito, S.: Requirements for digital preservation systems: a bottom-up approach. Dlib Mag. 11 (2005)

  50. Rothenberg, J.: Avoiding Technological Quicksand: Finding a Viable Technical Foundation for Digital Preservation. A Report to the Council on Library and Information Resources. Council on Library and Information Resources, Washington, DC (1999)

    Google Scholar 

  51. Salaheldeen, H.M., Nelson, M.L.: Resurrecting My Revolution: Using Social Link Neighborhood in Bringing Context to the Disappearing Web. In: Proceedings of theory and practice of digital libraries, pp. 333–345 (2013)

  52. Smith, M.: DSpace: an institutional repository from the MIT libraries and Hewlett Packard Laboratories. In: Proceedings of the 6th European conference on research and advanced technology for digital libraries, pp. 543–549 (2002)

  53. Spector, L., Klein, J., Perry, C., Feinstein, M.: Emergence of collective behavior in evolving populations of flying agents. In: Genetic and Evolutionary Computation Conference, pp. 61–73. Springer (2003)

  54. Van de Sompel, H., Bekaert, J., Liu, X., Balakireva, L., Schwander, T.: aDORe: a modular, standards-based digital object repository. Comput. J. 48(5), 514–535 (2005)

    Article  Google Scholar 

  55. Walker, R.: Cyberspace When You’re Dead. The New York Times, NY, New York (2011)

    Google Scholar 

  56. Waters, D., Garrett, J.: Preserving Digital Information. Report of the Task Force on Archiving of Digital Information. The Commission on Preservation and Access, Washington, DC (1996)

    Google Scholar 

  57. Watts, D.J., Strogatz, S.H.: Collective dynamics of ‘small world’ networks. Nature 393, 440–442 (1998)

    Article  Google Scholar 

  58. Wohlsen, M.: Digital Data that Never Dies. Associated Press, NY, New York (2011)

    Google Scholar 

  59. Yin, S.: Flickr Permanently Deletes User’s Account, 4,000 Photos by Accident. PC Magazine, February (2011)

Download references

Acknowledgments

This work supported in part by the National Science Foundation (NSF), Project 370161.

Author information

Authors and Affiliations

  1. Computer Science Department, Old Dominion University, 4700 Elkhorn Ave, Suite 3300, Norfolk, VA, 23529-0162, USA

    Charles Cartledge & Michael L. Nelson

Authors
  1. Charles Cartledge
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael L. Nelson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Charles Cartledge.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cartledge, C., Nelson, M.L. When should I make preservation copies of myself?. Int J Digit Libr 16, 183–205 (2015). https://doi.org/10.1007/s00799-015-0155-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00799-015-0155-1

Keywords

Search

Navigation