Skip to main content

GeoWave: Utilizing Distributed Key-Value Stores for Multidimensional Data

Part of the Lecture Notes in Computer Science book series (LNISA,volume 10411)

Abstract

To date, it has been difficult for modern geospatial software projects to take advantage of the benefits provided by distributed computing frameworks due to the implicit challenges of spatial and spatiotemporal data. Chief among these issues is preserving locality between multidimensional objects and the single dimensional sort order imposed by key-value stores. We will use the open source framework GeoWave to harness the scalability of various distributed frameworks and integrate them with geospatial queries, analytics, and map rendering. GeoWave performs dimensionality reduction by utilizing space–filling curves to convert n-dimensional data into a single dimension. This ensures that values close in multidimensional space are highly contiguous in the single dimensional keys of the datastore. By using various forms of geospatial data, we show that preserving locality in this way reduces the time needed to query, analyze, and render large amounts of data by multiple orders of magnitude.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-64367-0_6
  • Chapter length: 18 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   64.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-64367-0
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   84.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.

(The source code to produce these results is publicly available (https://github.com/rfecher/dimensionality-experiments))

Fig. 12.

(The source code to produce these results is publicly available (https://github.com/rfecher/range-sensitivity-experiments))

References

  1. Amazon DynamoDB: Amazon DynamoDB (2017). https://aws.amazon.com/dynamodb/

  2. Apache Accumulo: Apache Accumulo (2017). https://accumulo.apache.org/

  3. Apache Cassandra: Apache Cassandra (2017). http://cassandra.apache.org/

  4. Apache HBase: Apache HBase (2017). https://hbase.apache.org/

  5. Cloud BigTable: Cloud BigTable (2017). https://cloud.google.com/bigtable/

  6. Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Google Inc. (2004)

    Google Scholar 

  7. Eldawy, A., Mohamed, M.: The ecosystem of SpatialHadoop. SIGSPATIAL Spec. 6(3), 3–10 (2015)

    CrossRef  Google Scholar 

  8. GeoServer: GeoServer (2017). http://geoserver.org/

  9. Hamilton, C.H., Rau-Chaplin, A.: Compact Hilbert indices: space-filling curves for domains with unequal side lengths. Inf. Process. Lett. 105, 155–163 (2008)

    MathSciNet  CrossRef  MATH  Google Scholar 

  10. Haverkort, H., Walderveen, F.: Locality and bounding-box quality of two-dimensional space-filling curves. Comput. Geom. 43, 131–147 (2010)

    MathSciNet  CrossRef  MATH  Google Scholar 

  11. Indyk, P., Motwani, R., Raghavan, P., Vempala, S.: Locality-preserving hashing in multidimensional spaces, p. 618. ACM (1997)

    Google Scholar 

  12. Kim, H., Kang, S., Lee, S., Min, J.: The efficient algorithms for constructing enhanced quadtrees using MapReduce. IEICE Trans. Inf. Syst. 99(4), 918–926 (2016)

    CrossRef  Google Scholar 

  13. Nishimura, S., Das, S., Agrawal, D.: MD-HBase: a scalable multi-dimensional data infrastructure for location aware. In: IEEE MDM 2011, vol. 1 (2011)

    Google Scholar 

  14. Paiva, J., Ruivo,, P., Romano, P., Rodrigues, L.: AUTOPLACER: scalable self-tuning data placement in distributed key-value stores. ACM Trans. Auton. Adapt. Syst. 9(4) (2014). Article No. 19

    Google Scholar 

  15. Uzaygezen: Uzaygezen (2017). https://github.com/aioaneid/uzaygezen

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Michael A. Whitby .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Whitby, M.A., Fecher, R., Bennight, C. (2017). GeoWave: Utilizing Distributed Key-Value Stores for Multidimensional Data. In: , et al. Advances in Spatial and Temporal Databases. SSTD 2017. Lecture Notes in Computer Science(), vol 10411. Springer, Cham. https://doi.org/10.1007/978-3-319-64367-0_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-64367-0_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-64366-3

  • Online ISBN: 978-3-319-64367-0

  • eBook Packages: Computer ScienceComputer Science (R0)