Data Vaults: A Symbiosis between Database Technology and Scientific File Repositories

  • Milena Ivanova
  • Martin Kersten
  • Stefan Manegold
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7338)


In this short paper we outline the data vault, a database-attached external file repository. It provides a true symbiosis between a DBMS and existing file-based repositories. Data is kept in its original format while scalable processing functionality is provided through the DBMS facilities. In particular, it provides transparent access to all data kept in the repository through an (array-based) query language using the file-type specific scientific libraries.

The design space for data vaults is characterized by requirements coming from various fields. We present a reference architecture for their realization in (commercial) DBMSs and a concrete implementation in MonetDB for remote sensing data geared at content-based image retrieval.


Query Processing Query Language External Data Reference Architecture Cache Manager 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alagiannis, I., Borovica, R., Branco, M., Idreos, S., Ailamaki, A.: NoDB: Efficient Query Execution on Raw Data Files. In: SIGMOD (2012)Google Scholar
  2. 2.
    Baumann, P.: Large-Scale Earth Science Services: A Case for Databases. In: ER (Workshops), pp. 75–84 (2006)Google Scholar
  3. 3.
    Baumann, P., et al.: The multidimensional database system RasDaMan. SIGMOD Rec. 27(2), 575–577 (1998)CrossRefGoogle Scholar
  4. 4.
    Cerra, D., Datcu, M.: Image Retrieval using Compression-based Techniques. In: International ITG Conference on Source and Channel Coding (2010)Google Scholar
  5. 5.
    Dumitru, C.O., Molina, D.E., et al.: TELEIOS WP3: KDD concepts and methods proposal: report and design recommendations,
  6. 6.
    FITS. Flexible Image Transport System,
  7. 7.
  8. 8.
    Hey, T., Tansley, S., Tolle, K.: The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research (2009)Google Scholar
  9. 9.
    Ivanova, M., Kersten, M., Nes, N., Gonçalves, R.: An Architecture for Recycling Intermediates in a Column-store. ACM Trans. Database Syst. 35(4), 24 (2010)CrossRefGoogle Scholar
  10. 10.
    Kunchithapadam, K., Zhang, W., et al.: Oracle Database Filesystem. In: SIGMOD, pp. 1149–1160 (2011)Google Scholar
  11. 11.
    MonetDB (2012),
  12. 12.
    Oracle. Oracle Spatial GeoRaster Developer’s Guide, 11g Release 2 (11.2)Google Scholar
  13. 13.
  14. 14.
    SEED. Standard for the exchange of earthquake data (May 2010),
  15. 15.
    SQL/MED. ISO/IEC 9075-9:2008 Information technology - Database languages - SQL - Part 9: Management of External Data (SQL/MED)Google Scholar
  16. 16.
    Stolte, E., von Praun, C., Alonso, G., Gross, T.R.: Scientific data repositories: Designing for a moving target. In: SIGMOD Conference, pp. 349–360 (2003)Google Scholar
  17. 17.
    Zhang, Y., Kersten, M., Ivanova, M., Nes, N.: SciQL: Bridging the Gap between Science and Relational DBMS. In: IDEAS, pp. 124–133 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Milena Ivanova
    • 1
  • Martin Kersten
    • 1
  • Stefan Manegold
    • 1
  1. 1.Centrum Wiskunde & Informatica (CWI)AmsterdamThe Netherlands

Personalised recommendations