Advertisement

Big Data Storage Techniques for Spatial Databases: Implications of Big Data Architecture on Spatial Query Processing

  • Roger Frye
  • Mark McKenney
Chapter
Part of the Studies in Big Data book series (SBD, volume 8)

Abstract

Today, a large amount of data is being collected and stored. Data that has grown beyond traditional data management solutions has come to be known as big data. Solutions such as Hadoop have emerged to address the big data problem. However, spatial data presents its own challenges to storage and processing. Researchers have taken various approaches with Hadoop to handle spatial data efficiently. The approaches includes multi-stage map/reduce algorithms, generating on-demand indexes, and maintaining persistent indexes. This paper reviews the various approaches, categorizes the spatial queries reported in the testing, summarizes results, and identifies strengths and weaknesses with each approach.

Keywords

Spatial data Big data MapReduce Query processing 

References

  1. 1.
    Aji, A., Sun, X., Vo, H., Liu, Q., Lee, R., Zhang, X., Saltz, J., Wang, F.: Demonstration of Hadoop-GIS: a spatial data warehousing system over MapReduce. (2013)Google Scholar
  2. 2.
    Aji, A., Wang, F.: High performance spatial query processing for large scale scientific data. In: Proceedings of the on SIGMOD/PODS 2012 PhD Symposium, ACM, pp. 9–14. (2012)Google Scholar
  3. 3.
    Aji, A., Wang, F., Saltz, J.H.: Towards building a high performance spatial query system for large scale medical imaging data. In: Proceedings of the 20th International Conference on Advances in Geographic Information Systems, ACM, pp. 309–318. (2012)Google Scholar
  4. 4.
    Aji, A., Wang, F., Vo, H., Lee, R., Liu, Q., Zhang, X., Saltz, J.: Hadoop GIS: a high performance spatial data warehousing system over MapReduce. Proc. VLDB Endowment 6(11), 1009–1020 (2013)CrossRefGoogle Scholar
  5. 5.
    Akdogan, A., Demiryurek, U., Banaei-Kashani, F., Shahabi, C.: Voronoi-based geospatial query processing with MapReduce. In: IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom), IEEE, pp. 9–16 (2010)Google Scholar
  6. 6.
  7. 7.
  8. 8.
    Blackard, J.A., Dean, D., Anderson, C.: Covertype data set. http://archive.ics.uci.edu/ml/datasets/Covertype
  9. 9.
    Borzsony, S., Kossmann, D., Stocker, K.: The skyline operator. In: IEEE Proceedings of 17th International Conference on Data Engineering, IEEE, pp. 421–430. (2001)Google Scholar
  10. 10.
    Cary, A., Yesha, Y., Adjouadi, M., Rishe, N.: Leveraging cloud computing in geodatabase management. In: IEEE International Conference on Granular Computing (GrC), IEEE, pp. 73–78. (2010)Google Scholar
  11. 11.
    Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)CrossRefGoogle Scholar
  12. 12.
    Eldawy, A., Li, Y., Mokbel, M.F., Janardan, R.: Cg_hadoop: Computational geometry in MapReduce. (2013)Google Scholar
  13. 13.
    Eldawy, A., Mokbel, M.F.: A demonstration of SpatialHadoop: an efficient MapReduce framework for spatial data. Proc. VLDB Endowment 6(12), 1230–1233 (2013)CrossRefGoogle Scholar
  14. 14.
    Ghemawat, S., Gobioff, H., Leung, S.T.: The Google file system. In: ACM SIGOPS Operating Systems Review, vol. 37, ACM, pp. 29–43 (2003)Google Scholar
  15. 15.
    Güting, R.H.: An introduction to spatial database systems. VLDB J. 3(4), 357–399 (1994)CrossRefGoogle Scholar
  16. 16.
    Guttman, A.: R-trees: a dynamic index structure for spatial searching. In: SIGMOD ‘84: Proceedings of the International Conference on Management of Data, ACM, pp. 47–57. New York, USA 1984Google Scholar
  17. 17.
    Liao, H., Han, J., Fang, J.: Multi-dimensional index on Hadoop distributed file system. In: IEEE Fifth International Conference on Networking, Architecture and Storage (NAS), IEEE, pp. 240–249. (2010)Google Scholar
  18. 18.
    Lu, W., Shen, Y., Chen, S., Ooi, B.C.: Efficient processing of k nearest neighbor joins using MapReduce. Proc. VLDB Endowment 5(10), 1016–1027 (2012)CrossRefGoogle Scholar
  19. 19.
    Nishimura, S., Das, S., Agrawal, D., Abbadi, A.E.: MD-Hbase: a scalable multi-dimensional data infrastructure for location aware services. In: 12th IEEE International Conference on Mobile Data Management (MDM), vol. 1, pp. 7–16. (2011)Google Scholar
  20. 20.
    Olston, C., Reed, B., Srivastava, U., Kumar, R., Tomkins, A.: Pig latin: a not-so-foreign language for data processing. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of data, ACM, pp. 1099–1110. (2008)Google Scholar
  21. 21.
  22. 22.
  23. 23.
    Schneider, M., Behr, T.: Topological relationships between complex spatial objects. ACM Trans. Database Syst. (TODS) 31(1), 39–81 (2006)CrossRefGoogle Scholar
  24. 24.
  25. 25.
  26. 26.
    Wang, K., Han, J., Tu, B., Dai, J., Zhou, W., Song, X.: Accelerating spatial data processing with MapReduce. In: IEEE 16th International Conference on Parallel and Distributed Systems (ICPADS), IEEE, pp. 229–236. (2010)Google Scholar
  27. 27.
    Wang, Y., Wang, S.: Research and implementation on spatial data storage and operation based on Hadoop platform. In: Second IITA International Conference on Geoscience and Remote Sensing (IITA-GRS), IEEE, vol. 2, pp. 275–278. (2010)Google Scholar
  28. 28.
    Zhang, C., Li, F., Jestes, J.: Efficient parallel kNN joins for large data in MapReduce. In: Proceedings of the 15th International Conference on Extending Database Technology, ACM, pp. 38–49. (2012)Google Scholar
  29. 29.
    Zhang, S., Han, J., Liu, Z., Wang, K., Feng, S.: Spatial queries evaluation with MapReduce. In: IEEE Eighth International Conference on Grid and Cooperative Computing. GCC’09, pp. 287–292. (2009)Google Scholar
  30. 30.
    Zhang, S., Han, J., Liu, Z., Wang, K., Xu, Z.: Sjmr: Parallelizing spatial join with MapReduce on clusters. In: IEEE International Conference on Cluster Computing and Workshops. CLUSTER’09, IEEE, pp. 1–8. (2009)Google Scholar
  31. 31.
    Zhong, Y., Han, J., Zhang, T., Li, Z., Fang, J., Chen, G.: Towards parallel spatial query processing for big spatial data. In: IEEE 26th International Symposium Workshops and PhD Forum of Parallel and Distributed Processing (IPDPSW), IEEE, pp. 2085–2094. (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Southern Illinois UniversityEdwardsvilleUSA

Personalised recommendations