Digital Library Storage using iRODS Data Grids
Digital repository software provides a powerful and flexible infrastructure for managing and delivering complex digital resources and metadata. However, issues can arise in managing the very large, distributed data files that may constitute these resources. This paper describes an implementation approach that combines the Fedora digital repository software with a storage layer implemented as a data grid, using the iRODS middleware developed by DICE (Data Intensive Cyber Environments) as the successor to SRB. This approach allows us to use Fedoras flexible architecture to manage the structure of resources and to provide application- layer services to users. The grid-based storage layer provides efficient support for managing and processing the underlying distributed data objects, which may be very large (e.g. audio-visual material). The Rule Engine built into iRODS is used to integrate complex workflows at the data level that need not be visible to users, e.g. digital preservation functionality.
KeywordsDigital Library Digital Object Data Grid Digital Resource Digital Preservation
Unable to display preview. Download preview PDF.
- Tony Austin and Jenny Mitcham. Preservation and Management Strategies for Exceptionally Large Data Formats: ‘Big Data’. Technical report, English Heritage Project No: 3984. September 2007.Google Scholar
- C. Baru, R. Moore, A. Rajasekar and M. Wan. The SDSC Storage Resource Broker. In Proc. CASCON'98 Conference, Toronto, Canada, Nov.30–Dec.3, 1998.Google Scholar
- Tony Hey and Ann Trefethen. The data deluge: an e-Science perspective? In F. Berman, A. Hey, G. Fox (eds.), Grid Computing: Making the Global Infrastructure a Reality, John Wiley and Sons, Hoboken, NJ, 2003Google Scholar
- Mark Hedges, Tobias Blanke and Adil Hasan. Rule-based curation and preservation of data: A data grid approach using iRODS. Future Generation Computer Systems, (25), 2009.Google Scholar
- Carl Lagoze, Dean B. KraRt, Sandy Payette and Susan Jesuroga. What is a Digital Library Anymore, Anyway? D-Lib Magazine, (11), November 2005.Google Scholar
- E. Lyon. Dealing with Data: Roles, Rights, Responsibilities and Relationships. Technical report, Bath, UK, June 2007.Google Scholar
- A. Rajasekar, M. Wan, R. Moore and W. Schroeder. A Prototype Rule-based Distributed Data Management System. In HPDC workshop on “Next Generation Distributed Data Management”, Paris, France, May 2006.Google Scholar
- K. Thibodeau. Overview of Technological Approaches to Digital Preservation and Challenges in Coming Years. In Proceedings of The State of Digital Preservation: An International Perspective, Washington DC, USA, 2002.Google Scholar
- Andrew Treloar. Storage and Interoperability Work Package 2: Improve interoperability between Storage Resource Broker (SRB). based environments and Fedora Technical report, DART, 1. June 2007.Google Scholar