Abstract
With the development of Earth observation technologies, geospatial raster data such as DEM and remote sensing data experienced explosive growth in last forty years, providing TB or even PB amount of data to research projects. However, storing and processing the large amount of data can be very challenge to many projects, especially these without access to high performance computers. Distributed computing can integrate existed computing and storage resources through the Internet, providing an alternative way to facilitate these projects on handling the raster data. This study presents a distributed DEM processing approach developed based on the Map/Reduce, which has been widely adopted in cloud computing applications. The approach allows users to store and process the raster type DEM data in a distributed storage regime to utilize computing and storage capabilities combined from many average computers, e.g., PC. The approach has been implemented in a prototyped system, which is developed by utilizing the Apache Hadoop. The prototype has been deployed in the experimental environment, and then 90-m resolution DEM for China and a DEM hillshade model have been ingested into the prototype to evaluate the prototype.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hulen, H., Graf, O., Fitzgerald, K., Watson, R.: Storage Area Networks and the High Performance Storage System. In: Tenth NASA Goddard Conference on Mass Storage Systems (April 2002)
Wang, M., Gong, J.Y., Li, D.R.: Design and Implementation of Large-Scale Image Database Management System. Editorial Board of Geomatics and Information Science of Wuhan University 28, 294–300 (2003), doi: CNKI:SUN:WHCH.0.2003-03-007
Jin, H.Z.: Research and Design of Cluster-Based Job Management System Oriented to Spatial Data Processing. Wuhan University (May 2005), doi: CNKI:CDMD:2.2006.034413
Ghemawat, S., Gobioff, H., Shun-Tak, L.: The Google File System. In: SOSP 2003: Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, vol. 37 (December 2003), doi:10.1145/945445.945450
Barroso, L.A., Dean, J., Holzle, U.: Web search for a planet: The Google cluster architecture: The Google Cluster Architecture. IEEE Computer Society 23, 22–28 (2003), doi:10.1109/MM.2003.1196112
Lammel, R.: Google’s MapReduce programming model—Revisited. Science of Computer Programming 70, 1–30 (2008), doi:10.1016/j.scico.2007.07.001
http://hadoop.apache.org/common/docs/r0.20.0/hdfs_user_guide.pdf
http://hadoop.apache.org/common/docs/r0.20.0/hdfs_user_guide.pdf
Wu, L.: Principles, Methods and Applications of Geographic Information Systems. Science Press, Beijing (2001)
Feng, M., Zhu, Y.Q., Zhang, M.Z., Zhao, H., Yu, M.L.: Design and Realization of Multi-source Remote Sensing Images Sharing Platform, vol. 10, pp. 102–108, doi: CNKI:SUN:DQXX.0.2008-01-018
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag GmbH Berlin Heidelberg
About this paper
Cite this paper
Yin, F., Feng, M., Song, J. (2013). Research on Mass Geospatial Raster Data Processing Based on Map/Reduce Model. In: Gaol, F. (eds) Recent Progress in Data Engineering and Internet Technology. Lecture Notes in Electrical Engineering, vol 156. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28807-4_37
Download citation
DOI: https://doi.org/10.1007/978-3-642-28807-4_37
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28806-7
Online ISBN: 978-3-642-28807-4
eBook Packages: EngineeringEngineering (R0)