Advertisement

GeoWave: Utilizing Distributed Key-Value Stores for Multidimensional Data

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10411)

Abstract

To date, it has been difficult for modern geospatial software projects to take advantage of the benefits provided by distributed computing frameworks due to the implicit challenges of spatial and spatiotemporal data. Chief among these issues is preserving locality between multidimensional objects and the single dimensional sort order imposed by key-value stores. We will use the open source framework GeoWave to harness the scalability of various distributed frameworks and integrate them with geospatial queries, analytics, and map rendering. GeoWave performs dimensionality reduction by utilizing space–filling curves to convert n-dimensional data into a single dimension. This ensures that values close in multidimensional space are highly contiguous in the single dimensional keys of the datastore. By using various forms of geospatial data, we show that preserving locality in this way reduces the time needed to query, analyze, and render large amounts of data by multiple orders of magnitude.

References

  1. 1.
    Amazon DynamoDB: Amazon DynamoDB (2017). https://aws.amazon.com/dynamodb/
  2. 2.
    Apache Accumulo: Apache Accumulo (2017). https://accumulo.apache.org/
  3. 3.
    Apache Cassandra: Apache Cassandra (2017). http://cassandra.apache.org/
  4. 4.
    Apache HBase: Apache HBase (2017). https://hbase.apache.org/
  5. 5.
    Cloud BigTable: Cloud BigTable (2017). https://cloud.google.com/bigtable/
  6. 6.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Google Inc. (2004)Google Scholar
  7. 7.
    Eldawy, A., Mohamed, M.: The ecosystem of SpatialHadoop. SIGSPATIAL Spec. 6(3), 3–10 (2015)CrossRefGoogle Scholar
  8. 8.
    GeoServer: GeoServer (2017). http://geoserver.org/
  9. 9.
    Hamilton, C.H., Rau-Chaplin, A.: Compact Hilbert indices: space-filling curves for domains with unequal side lengths. Inf. Process. Lett. 105, 155–163 (2008)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Haverkort, H., Walderveen, F.: Locality and bounding-box quality of two-dimensional space-filling curves. Comput. Geom. 43, 131–147 (2010)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Indyk, P., Motwani, R., Raghavan, P., Vempala, S.: Locality-preserving hashing in multidimensional spaces, p. 618. ACM (1997)Google Scholar
  12. 12.
    Kim, H., Kang, S., Lee, S., Min, J.: The efficient algorithms for constructing enhanced quadtrees using MapReduce. IEICE Trans. Inf. Syst. 99(4), 918–926 (2016)CrossRefGoogle Scholar
  13. 13.
    Nishimura, S., Das, S., Agrawal, D.: MD-HBase: a scalable multi-dimensional data infrastructure for location aware. In: IEEE MDM 2011, vol. 1 (2011)Google Scholar
  14. 14.
    Paiva, J., Ruivo,, P., Romano, P., Rodrigues, L.: AUTOPLACER: scalable self-tuning data placement in distributed key-value stores. ACM Trans. Auton. Adapt. Syst. 9(4) (2014). Article No. 19Google Scholar
  15. 15.
    Uzaygezen: Uzaygezen (2017). https://github.com/aioaneid/uzaygezen

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Michael A. Whitby
    • 1
  • Rich Fecher
    • 1
  • Chris Bennight
    • 1
  1. 1.DigitalGlobeHerndonUSA

Personalised recommendations