Skip to main content
SpringerLink
Log in

Big Data Storage Techniques for Spatial Databases: Implications of Big Data Architecture on Spatial Query Processing

  • Chapter
  • First Online:
Information Granularity, Big Data, and Computational Intelligence

Part of the book series: Studies in Big Data ((SBD,volume 8))

Abstract

Today, a large amount of data is being collected and stored. Data that has grown beyond traditional data management solutions has come to be known as big data. Solutions such as Hadoop have emerged to address the big data problem. However, spatial data presents its own challenges to storage and processing. Researchers have taken various approaches with Hadoop to handle spatial data efficiently. The approaches includes multi-stage map/reduce algorithms, generating on-demand indexes, and maintaining persistent indexes. This paper reviews the various approaches, categorizes the spatial queries reported in the testing, summarizes results, and identifies strengths and weaknesses with each approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aji, A., Sun, X., Vo, H., Liu, Q., Lee, R., Zhang, X., Saltz, J., Wang, F.: Demonstration of Hadoop-GIS: a spatial data warehousing system over MapReduce. (2013)

    Google Scholar 

  2. Aji, A., Wang, F.: High performance spatial query processing for large scale scientific data. In: Proceedings of the on SIGMOD/PODS 2012 PhD Symposium, ACM, pp. 9–14. (2012)

    Google Scholar 

  3. Aji, A., Wang, F., Saltz, J.H.: Towards building a high performance spatial query system for large scale medical imaging data. In: Proceedings of the 20th International Conference on Advances in Geographic Information Systems, ACM, pp. 309–318. (2012)

    Google Scholar 

  4. Aji, A., Wang, F., Vo, H., Lee, R., Liu, Q., Zhang, X., Saltz, J.: Hadoop GIS: a high performance spatial data warehousing system over MapReduce. Proc. VLDB Endowment 6(11), 1009–1020 (2013)

    Article  Google Scholar 

  5. Akdogan, A., Demiryurek, U., Banaei-Kashani, F., Shahabi, C.: Voronoi-based geospatial query processing with MapReduce. In: IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom), IEEE, pp. 9–16 (2010)

    Google Scholar 

  6. Apache HBase. http://hbase.apache.org

  7. Apache Hive. http://hive.apache.org

  8. Blackard, J.A., Dean, D., Anderson, C.: Covertype data set. http://archive.ics.uci.edu/ml/datasets/Covertype

  9. Borzsony, S., Kossmann, D., Stocker, K.: The skyline operator. In: IEEE Proceedings of 17th International Conference on Data Engineering, IEEE, pp. 421–430. (2001)

    Google Scholar 

  10. Cary, A., Yesha, Y., Adjouadi, M., Rishe, N.: Leveraging cloud computing in geodatabase management. In: IEEE International Conference on Granular Computing (GrC), IEEE, pp. 73–78. (2010)

    Google Scholar 

  11. Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)

    Article  Google Scholar 

  12. Eldawy, A., Li, Y., Mokbel, M.F., Janardan, R.: Cg_hadoop: Computational geometry in MapReduce. (2013)

    Google Scholar 

  13. Eldawy, A., Mokbel, M.F.: A demonstration of SpatialHadoop: an efficient MapReduce framework for spatial data. Proc. VLDB Endowment 6(12), 1230–1233 (2013)

    Article  Google Scholar 

  14. Ghemawat, S., Gobioff, H., Leung, S.T.: The Google file system. In: ACM SIGOPS Operating Systems Review, vol. 37, ACM, pp. 29–43 (2003)

    Google Scholar 

  15. Güting, R.H.: An introduction to spatial database systems. VLDB J. 3(4), 357–399 (1994)

    Article  Google Scholar 

  16. Guttman, A.: R-trees: a dynamic index structure for spatial searching. In: SIGMOD ‘84: Proceedings of the International Conference on Management of Data, ACM, pp. 47–57. New York, USA 1984

    Google Scholar 

  17. Liao, H., Han, J., Fang, J.: Multi-dimensional index on Hadoop distributed file system. In: IEEE Fifth International Conference on Networking, Architecture and Storage (NAS), IEEE, pp. 240–249. (2010)

    Google Scholar 

  18. Lu, W., Shen, Y., Chen, S., Ooi, B.C.: Efficient processing of k nearest neighbor joins using MapReduce. Proc. VLDB Endowment 5(10), 1016–1027 (2012)

    Article  Google Scholar 

  19. Nishimura, S., Das, S., Agrawal, D., Abbadi, A.E.: MD-Hbase: a scalable multi-dimensional data infrastructure for location aware services. In: 12th IEEE International Conference on Mobile Data Management (MDM), vol. 1, pp. 7–16. (2011)

    Google Scholar 

  20. Olston, C., Reed, B., Srivastava, U., Kumar, R., Tomkins, A.: Pig latin: a not-so-foreign language for data processing. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of data, ACM, pp. 1099–1110. (2008)

    Google Scholar 

  21. OpenStreetMap. http://www.openstreetmap.org

  22. PostGIS. http://postgis.net

  23. Schneider, M., Behr, T.: Topological relationships between complex spatial objects. ACM Trans. Database Syst. (TODS) 31(1), 39–81 (2006)

    Article  Google Scholar 

  24. SpatialHadoop. http://spatialhadoop.cs.umn.edu

  25. TIGER Files. http://www.census.gov/geo/www/tiger/

  26. Wang, K., Han, J., Tu, B., Dai, J., Zhou, W., Song, X.: Accelerating spatial data processing with MapReduce. In: IEEE 16th International Conference on Parallel and Distributed Systems (ICPADS), IEEE, pp. 229–236. (2010)

    Google Scholar 

  27. Wang, Y., Wang, S.: Research and implementation on spatial data storage and operation based on Hadoop platform. In: Second IITA International Conference on Geoscience and Remote Sensing (IITA-GRS), IEEE, vol. 2, pp. 275–278. (2010)

    Google Scholar 

  28. Zhang, C., Li, F., Jestes, J.: Efficient parallel kNN joins for large data in MapReduce. In: Proceedings of the 15th International Conference on Extending Database Technology, ACM, pp. 38–49. (2012)

    Google Scholar 

  29. Zhang, S., Han, J., Liu, Z., Wang, K., Feng, S.: Spatial queries evaluation with MapReduce. In: IEEE Eighth International Conference on Grid and Cooperative Computing. GCC’09, pp. 287–292. (2009)

    Google Scholar 

  30. Zhang, S., Han, J., Liu, Z., Wang, K., Xu, Z.: Sjmr: Parallelizing spatial join with MapReduce on clusters. In: IEEE International Conference on Cluster Computing and Workshops. CLUSTER’09, IEEE, pp. 1–8. (2009)

    Google Scholar 

  31. Zhong, Y., Han, J., Zhang, T., Li, Z., Fang, J., Chen, G.: Towards parallel spatial query processing for big spatial data. In: IEEE 26th International Symposium Workshops and PhD Forum of Parallel and Distributed Processing (IPDPSW), IEEE, pp. 2085–2094. (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Southern Illinois University, Edwardsville, IL, USA

    Roger Frye & Mark McKenney

Authors
  1. Roger Frye
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mark McKenney
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mark McKenney .

Editor information

Editors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Alberta, Edmonton, Alberta, Canada

    Witold Pedrycz

  2. Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan

    Shyi-Ming Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Frye, R., McKenney, M. (2015). Big Data Storage Techniques for Spatial Databases: Implications of Big Data Architecture on Spatial Query Processing. In: Pedrycz, W., Chen, SM. (eds) Information Granularity, Big Data, and Computational Intelligence. Studies in Big Data, vol 8. Springer, Cham. https://doi.org/10.1007/978-3-319-08254-7_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-08254-7_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-08253-0

  • Online ISBN: 978-3-319-08254-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation