Implementing a Reliable Digital Object Archive
An Archival Repository reliably stores digital objects for long periods of time (decades or centuries). The archival nature of the system requires new techniques for storing, indexing, and replicating digital objects. In this paper we discuss the specialized indexing needs of a write-once archive. We also present a reliability algorithm for effectively replicating sets of related objects. We describe a data import utility for archival repositories. Finally, we discuss and evaluate a prototype repository we have built, the Stanford Archival Vault (SAV).
Unable to display preview. Download preview PDF.
- Yuri Breitbart, Raghavan Komondoor, Rajeev Rastogi, S. Seshadri, and Avi Silberschatz. Update propagation protocols for replicated databases. In Proceedings of the ACM SIGMOD Conference, 1999.Google Scholar
- Yuan Chen, Jan Edler, Andrew Goldberg, Allan Gottlieb, Sumeet Sobti, and Peter Yianilos. A prototype implementation of archival intermemory. In Proceedings of the Fourth ACM DL Conference, 1999.Google Scholar
- Ann Chervenak, Vivekenand Vellanki, and Zachary Kurmas. Protecting file systems: A survey of backup techniques. In Proceedings Joint NASA and IEEE Mass Storage Conference, March 1998.Google Scholar
- Brian Cooper, Arturo Crespo, and Hector Garcia-Molina. Implementing a reliable digital object archive. http://dbpubs.stanford.edu/pub/2000-27, 2000. Extended version of paper.
- Brian Cooper and Hector Garcia-Molina. InfoMonitor: Unobtrusively archiving a World Wide Web server. http://www-db.stanford.edu/pub/papers/fmpaper.ps, 2000. Technical Report.
- Inktomi Corporation. Web surpasses one billion documents. http://-www.inktomi.com/new/press/billion.html, 2000.
- Arturo Crespo and Hector Garcia-Molina. Archival storage for digital libraries. In Proceedings of the Third ACM DL Conference, 1998.Google Scholar
- Arturo Crespo and Hector Garcia-Molina. Modeling archival repositories for digital libraries. In Proceedings of the Fourth European Conference on Research and Advanced Technology for Digital Libraries (ECDL), 2000.Google Scholar
- Jean Deken. Writ in water? an exploration of the gap between archival construct and practice in the machine-readable environment. In Working With Knowldge Conference, May 1998. Accessible at http://www.slac.stanford.edu/pubs/slacpubs/7000/slac-pub-7811.html.
- Ross Finlayson and David Cheriton. Log files: An extended file service exploiting write-once storage. In Proceedings of the 11th Symposium on Operating Systems Principles, November 1987.Google Scholar
- National Science Foundation. Workshop on Data Archival and Information Preservation: Executive summary. http://cecssrv1.cecs.missouri.edu/NSFWorkshop/execsum.html, 1999.
- Hector Garcia-Molina, Jeff Ullman, and Jennifer Widom. Database System Implementation. Prentice Hall, Upper Saddle River, New Jersey, 2000.Google Scholar
- John Garrett and Donald Waters. Preserving digital information: Report of the Task Force on Archiving of Digital Information, May 1996. Accessible at http://www.rlg.org/ArchTF/.
- Anja Haake and David Hicks. Verse: Towards hypertext versioning styles. In Hypertext’ 96, 1996.Google Scholar
- Joseph Halpern and Carl Lagoze. The Computing Research Repository: Promoting the rapid dissemination and archiving of computer science research. In Proceedings of the Fourth ACM DL Conference, 1999.Google Scholar
- John Hartman and John Ousterhout. The Zebra striped network file system. In Proceedings 14th Symposium on Operating Systems Principles, December 1993.Google Scholar
- Norman C. Hutchinson, Stephen Manley, Mike Federwisch, Guy Harris, Dave Hitz, Steven Kleiman, and Sean O’Malley. Logical vs. physical file system backup. In Proceedings of the Third USENIX Symposium on Operating Systems Design and Implementation (OSDI), 1999.Google Scholar
- Tivoli Systems Inc. Tivoli storage manager. http://www.tivoli.com/products/index/storage mgr/, 1999.
- Barbara Liskov, Sanjay Ghemawat, Robert Gruber, Paul Johnson, Liuba Shrira, and Michael Williams. Replication in the Harp file system. In Proceedings 13th Symposium on Operating Systems Principles, October 1991.Google Scholar
- Stanford Conservation Online. Electronic storage media.http://palimpsest.stanford.edu/bytopic/electronic-records/electronic-storage-media/, 2000.
- Michael Rabinovich, Narain Gehani, and Alex Kononov. Efficient update propagation in epidemic replicated databases. In Proceedings of the 5th International Conference on Extending Database Technology, 1996.Google Scholar
- Arcot Rajasekar, Richard Marciano, and Reagan Moore. Collection-based persistent archives. http://www.sdsc.edu/NARA/Publications/OTHER/Persistent/Persistent.html, 2000.
- Mendel Rosenblum and John K. Ousterhout. The design and implementation of a log-structured file system. In Proceedings 13th Symposium on Operating Systems Principles, October 1991.Google Scholar
- David Rosenthal and Vicky Reich. Permanent web publishing.http://lockss.stanford.edu/, 2000. To appear at Freenix, San Diego, CA, June 2000.
- Victorian Electronic Records Strategy. Victorian electronic records strategy final report. http://home.vicnet.net.au/~ provic/vers/final.htm, 1999.