Implementing a General Spatial Indexing Library for Relational Databases of Large Numerical Simulations

  • Gerard Lemson
  • Tamás Budavári
  • Alexander Szalay
Conference paper

DOI: 10.1007/978-3-642-22351-8_34

Part of the Lecture Notes in Computer Science book series (LNCS, volume 6809)
Cite this paper as:
Lemson G., Budavári T., Szalay A. (2011) Implementing a General Spatial Indexing Library for Relational Databases of Large Numerical Simulations. In: Bayard Cushing J., French J., Bowers S. (eds) Scientific and Statistical Database Management. SSDBM 2011. Lecture Notes in Computer Science, vol 6809. Springer, Berlin, Heidelberg

Abstract

Large multi-terabyte numerical simulations of different physical systems consist of billions of particles or grid points and hundreds to thousands of snapshots. Increasingly these data sets are stored in large object-relational databases. Most statistical analyses involve extracting various spatio-temporal subsets. Existing built-in spatial indexes in commercial systems lack essential features required for many applications in the physical sciences. We describe a library that we have implemented in several languages and platforms (Java/Oracle, C#/SQL Server) based on generic space-filling curves, implemented as plug-ins. The index provides a mapping of higher dimensional space into the standard linear B-tree index of any relational database. The architecture allows intersections with different geometric primitives. The library has been used for cosmological N-body simulations and isotropic turbulence, providing sub-second response time over datasets exceeding several tens of terabytes. The library can also address complex space-time challenges, like temporal look-back into past light-cones of cosmological simulations.

Keywords

spatial indexing numerical simulations relational databases 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Gerard Lemson
    • 1
  • Tamás Budavári
    • 2
  • Alexander Szalay
    • 2
  1. 1.MPAGermany
  2. 2.JHUUSA

Personalised recommendations