The aDORe federation architecture: digital repositories at scale
- 92 Downloads
- 6 Citations
Abstract
The need to federate repositories emerges in two distinctive scenarios. In one scenario, scalability-related problems in the operation of a repository reach a point beyond which continued service requires parallelization and hence federation of the repository infrastructure. In the other scenario, multiple distributed repositories manage collections of interest to certain communities or applications, and federation is an approach to present a unified perspective across these repositories. The high-level, 3-Tier aDORe federation architecture can be used as a guideline to federate repositories in both cases. This paper describes the architecture, consisting of core interfaces for federated repositories in Tier-1, two shared infrastructure components in Tier-2, and a single-point of access to the federation in Tier-3. The paper also illustrates two large-scale deployments of the aDORe federation architecture: the aDORe Archive repository (over 100,000,000 digital objects) at the Los Alamos National Laboratory and the Ghent University Image Repository federation (multiple terabytes of image files).
Keywords
Interoperability Repository federation OAI-PMH OpenURLPreview
Unable to display preview. Download preview PDF.
References
- 1.Apps A.: The JISC information environment service registry. ASSIGNation 22(3), 9–11 (2005)Google Scholar
- 2.Bekaert, J., Hochstenbach, P., Van de Sompel, H.: Using MPEG-21 DIDL to represent complex digital objects in the Los Alamos National Laboratory Digital Library. D-Lib Mag. 9(11), (2003). doi: 10.1045/november2003-bekaert
- 3.Bekaert, J., Balakireva, L., Hochstenbach, P., Van de Sompel, H.: Using MPEG-21 and NISO OpenURL for the dynamic dissemination of complex digital objects in the Los Alamos National Laboratory Digital Library. D-Lib Mag. 9(11), (2004). doi: 10.1045/february2004-bekaert
- 4.Bekaert, J., Van de Sompel, H.: A standards-based solution for the accurate transfer of digital assets. D-Lib Mag. 11(6), (2005). doi: 10.1045/june2005-bekaert
- 5.Bekaert, J.: Standards-based interfaces for harvesting and obtaining assets from digital repositories. Ph.D. thesis, Ghent University (2006). Retrieved from http://hdl.handle.net/1854/4833
- 6.Bekaert, J., De Kooning, E., Van de Sompel, H.: Representing digital assets using MPEG-21 digital item declaration. Int. J. Digit. Libr. 6(2), 159–173 (2006). doi: 10.1007/s00799-005-0133-0 Google Scholar
- 7.Caplan P., Guenther R.: Practical preservation: the PREMIS experience. Libr. Trends 54(1), 111–124 (2005)CrossRefGoogle Scholar
- 8.Davis, J.R., Lagoze, C.: NCSTRL: design and deployment of a globally distributed digital library. J. Am. Soc. Inf. Sci. 31(3), 273–280 (1999). doi:10.1002/(SICI)1097-4571(2000)51:3<273::AID-ASI6>3.0.CO;2-6Google Scholar
- 9.DRIVER: Digital Repository Infrastructure Vision for European Research (2006). Retrieved from http://www.driver-repository.eu/
- 10.International Organization for Standardization: ISO/IEC 21000-2:2003. Information Technology—Multimedia Framework (MPEG-21)—Part 2: Digital Item Declaration, 1st edn. Geneva, Switzerland (2003)Google Scholar
- 11.International Organization for Standardization: ISO/IEC 21000-3:2003: Information Technology—Multimedia Framework (MPEG-21)—Part 3: Digital Item Identification, 1st edn. Geneva, Switzerland (2003)Google Scholar
- 12.International Press Telecommunications Council: “IPTC Core” Schema for XMP (2005). Retrieved from http://www.iptc.org/IPTC4XMP/
- 13.Japan Electronic Industries Development Association: Exchangeable Image File Format v 2.1 (1998). Retrieved from http://www.exif.org
- 14.Jerez, H., Liu, X., Hochstenbach, P., Van de Sompel, H.: The multi-faceted use of the OAI-PMH in the LANL repository. In: Joint Conference on Digital Libraries Proceedings, pp. 11–20 (2004). doi: 10.1109/JCDL.2004.1336089
- 15.Jerez, H., Manepalli, G., Blanchi, C., Lannom, L.: ADL-R: the First CORDRA Registry. D-Lib Mag. 12(2), (2006). doi: 10.1045/February2006-jerez
- 16.Joint Information Systems Committee: Information Environment Service Registry Metadata (2006). Retrieved from http://iesr.ac.uk/metadata/
- 17.Kahn, R., Wilensky, R.: A framework for distributed digital object services (1995). Retrieved from http://hdl.handle.net/cnri.dlib/tn95-01
- 18.Kahn, R., Wilensky, R.: A framework for distributed digital object services. Int. J. Digit. Libr. 6(2), 115–123 (1995). doi: 10.1007/s00799-005-0128-x CrossRefGoogle Scholar
- 19.Kindberg, T., Hawke, S.: RFC 4151: the ‘tag’ URI scheme (2005). Retrieved from http://www.ietf.org/rfc/rfc4151.txt
- 20.Kunze, J., Arvidson, A., Mohr, G., Stack, M.: The WARC File Format Version 0.9 (2006). Retrieved from http://archive-access.sourceforge.net/warc/warc_file_format-0.9.html
- 21.Kunze, J., Rodgers, R.P.C.: Internet draft: ARK identifier scheme (2007). Retrieved from http://www.ietf.org/internet-drafts/draft-kunze-ark-14.txt
- 22.Lagoze C., Davis J.R.: Dienst: an architecture for distributed document libraries. Commun. ACM 38(4), 47 (1995)CrossRefGoogle Scholar
- 23.Lagoze, C., Van de Sompel, H.: The open archives initiative: building a low-barrier interoperability framework. In: Proceedings of the 1st ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 54–62, (2001). doi: 10.1145/379437.379449
- 24.Lagoze, C., Van de Sompel, H., Nelson, M.L., Warner, S. (eds.): The open archives initiative protocol for metadata harvesting, 2nd edn. (2003). Retrieved from http://www.openarchives.org/OAI/2.0/openarchivesprotocol.htm
- 25.Lagoze, C., Payette, S., Shin, E., Wilper C.: Fedora: an architecture for complex objects and their relationships. Int. J. Digit. Libr. 6(2), 124–138 (2006). doi: 10.1007/s00799-005-0130-3 CrossRefGoogle Scholar
- 26.Lagoze, C., Van de Sompel, H., Nelson, M.L., Sanderson, R., Warner, S. (eds.): ORE specification—resource map profile of atom (2007). Retrieved from http://www.openarchives.org/ore/0.1/atom
- 27.Library of Congress, Preservation Metadata Maintenance Activity: PREMIS (2007). Retrieved from http://www.loc.gov/standards/premis/
- 28.Leach, P., Mealling, M., Salz, R.: RFC 4122: a Universally Unique IDentifier (UUID) URN Namespace. Retrieved from http://www.ietf.org/rfc/rfc4122.txt (2005)
- 29.Liu, X., Balakireva, L., Van de Sompel, H.: File-based storage of digital objects and constituent datastreams: XMLtapes and internet archive ARC files. Lect. Notes Comput. Sci. 3652, 254–265 (2005). doi: 10.1007/11551362_23 CrossRefGoogle Scholar
- 30.Los Alamos National Laboratory Research Library: aDORe Archive (2006). Retrieved from http://african.lanl.gov/aDORe/projects/adoreArchive/
- 31.Los Alamos National Laboratory Research Library: DIDLTools (2006). Retrieved from http://african.lanl.gov/aDORe/projects/DIDLTools/
- 32.Manepalli, G., Jerez, H., Nelson, M.L.: FeDCOR: an institutional CORDRA registry. D-Lib Mag. 12(2), (2006). doi: 10.1045/february2006-manepalli
- 33.McDonough, J.P.: METS: standardized encoding for digital library objects. Int. J. Digit. Libr. 6(2), 148–158 (2006). doi: 10.1007/s00799-005-0132-1 CrossRefGoogle Scholar
- 34.Nelson, M.L., Van de Sompel, H.: IJDL special issue on complex digital objects: Guest editors’ introduction. Int. J. Digit. Libr. 6(2), 113–114 (2006). doi: 10.1007/s00799-005-0127-y CrossRefGoogle Scholar
- 35.National Information Standards Organization. ANSI/NISO Z39.88-2004: The OpenURL Framework for Context-Sensitive Services. NISO Press, BethesdaGoogle Scholar
- 36.Rehak, D., Daniel, R., Lannom, R.: A model and infrastructure for federated learning content repositories. Interoperability of web-based educational systems workshop. In: CEUR Workshop Proceedings, vol. 143 (2005). Retrieved from http://cordra.net/cordra/information/publications/2005/www2005/cordrawww2005.pdf
- 37.Tansley, R., Bass, M., Stuve, D., Branschofsky, M., Chudnov, D., McClellan, G., Smith, M.: The Dspace institutional digital repository system: current functionality. In: Joint Conference on Digital Libraries Proceedings, pp. 87–97 (2003)Google Scholar
- 38.Tansley, R.: Building a distributed, standards-based repository federation. D-Lib Mag. 12(7/8), (2006). doi: 10.1045/july2006-tansley
- 39.Universiteitsbiliotheek Gent.: Topografische Collectie. Retrieved from http://adore.ugent.be/topo/ (2006)
- 40.Universiteitsbibliotheek Gent, Google.: Google and Ghent University Library to make hundreds of thousands of Dutch and French books available online. Press Release. Retrieved from http://lib.ugent.be/info/en/project-google.shtml
- 41.Van de Sompel, H., Payette, S., Ericksson, J., Lagoze, C., Warner, S.: Rethinking scholarly communication: building the system that scholars deserve. D-Lib Mag. 10(9), (2004). doi: 10.1045/september2004-vandesompel
- 42.Van de Sompel, H., Nelson, M.L., Lagoze, C., Warner, S.: Resource harvesting within the OAI-PMH framework. D-Lib Mag. 10(12), (2004). doi: 10.1045/december2004-vandesompel
- 43.Van de Sompel, H., Bekaert, J., Liu, X., Balakireva, L., Schwander, T.: aDORe: a modular, standards-based digital object repository. Comput. J. 48(5), 514–535 (2005). doi: 10.1093/comjnl/bxh114 CrossRefGoogle Scholar
- 44.Van de Sompel, H., Hammond, T., Neylon, E., Weibel, S.: RFC 4452: The “info” URI scheme for information assets with identifiers in public namespaces (2006). Retrieved from http://www.ietf.org/rfc/rfc4452.txt
- 45.Van de Sompel, H., Lagoze, C., Bekaert, J., Liu, X., Payette, S., Warner, S.: An interoperable fabric for scholarly value chains. D-Lib Mag. 12(10), (2006). doi: 10.1045/October2006-vandesompel
- 46.Van de Sompel, H., Lagoze, C.: Interoperability for the discovery, use, and re-use of units of scholarly communication. CT Watch Q. 3(3), (2007). Retrieved from http://www.ctwatch.org/quarterly/articles/2007/08/interoperability-for-the-discovery-use-and-re-use-of-units-of-scholarly-communication/
- 47.Warner, S., Bekaert, J., Lagoze, C., Liu, X., Payette, S., Van de Sompel, H.: Pathways: augmenting interoperability across scholarly repositories. Int. J. Digit. Libr. 7(1–2), 35–52 (2007). doi: 10.1007/s00799-007-0025-6 CrossRefGoogle Scholar