Advertisement

Computational Infrastructure of SoilGrids 2.0

  • Luís M. de SousaEmail author
  • Laura Poggio
  • Gwen Dawes
  • Bas Kempen
  • Rik van den Bosch
Conference paper
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 554)

Abstract

SoilGrids maps soil properties for the entire globe at medium spatial resolution (250 m cell side) using state-of-the-art machine learning methods. The expanding pool of input data and the increasing computational demands of predictive models required a prediction framework that could deal with large data. This article describes the mechanisms set in place for a geo-spatially parallelised prediction system for soil properties. The features provided by GRASS GIS – mapset and region – are used to limit predictions to a specific geographic area, enabling parallelisation. The Slurm job scheduler is used to deploy predictions in a high-performance computing cluster. The framework presented can be seamlessly applied to most other geo-spatial process requiring parallelisation. This framework can also be employed with a different job scheduler, GRASS GIS being the main requirement and engine.

Keywords

Digital Soil Mapping High-performance computing GRASS GIS 

References

  1. 1.
    Arrouays, D., et al.: GlobalSoilMap: toward a fine-resolution global grid of soil properties. In: Advances in Agronomy, vol. 125, pp. 93–134. Academic Press (2014).  https://doi.org/10.1016/B978-0-12-800137-0.00003-0Google Scholar
  2. 2.
    Batjes, N.H., Ribeiro, E., van Oostrum, A.: Standardised soil profile data to support global mapping and modelling (wosis snapshot 2019). Earth Syst. Sci. Data Discussions 2019, 1–46 (2019).  https://doi.org/10.5194/essd-2019-164. https://www.earth-syst-sci-data-discuss.net/essd-2019-164/CrossRefGoogle Scholar
  3. 3.
    Bivand, R.: rgrass7: Interface between GRASS 7 geographical information system and R (2018). https://CRAN.R-project.org/package=rgrass7. r package version 0.1-12
  4. 4.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefGoogle Scholar
  5. 5.
    Goode, J.P.: The homolosine projection: a new device for portraying the earth’s surface entire. Ann. Assoc. Am. Geogr. 15(3), 119–125 (1925)CrossRefGoogle Scholar
  6. 6.
    GRASS Development Team: Geographic Resources Analysis Support System (GRASS GIS) software, version 7.6.0 (2019). http://www.grass.osgeo.org
  7. 7.
    McBratney, A., Santos, M., Minasny, B.: On digital soil mapping. Geoderma 117, 3–52 (2003).  https://doi.org/10.1016/S0016-7061(03)00223-4CrossRefGoogle Scholar
  8. 8.
    Meinshausen, N.: Quantile regression forests. J. Mach. Learn. Res. 7(Jun), 983–999 (2006) MathSciNetzbMATHGoogle Scholar
  9. 9.
    Minasny, B., McBratney, A.: Digital soil mapping: a brief history and some lessons. Geoderma 264(Part B), 301–311 (2016). Soil mapping, classification, and modelling: history and future directionsCrossRefGoogle Scholar
  10. 10.
    Momjian, B.: PostgreSQL: Introduction and Concepts. Addison-Wesley, New York (2001)Google Scholar
  11. 11.
    Neteler, M., Mitasova, H.: Open Source GIS: A GRASS GIS Approach, vol. 689. Springer, Berlin (2013)Google Scholar
  12. 12.
    OSGeo Foundation: VRT - GDAL virtual format. https://gdal.org/drivers/raster/vrt.html (2019). Accessed 2019 Oct 09
  13. 13.
    R Core Team: R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria (2019). http://www.R-project.org/. ISBN 3-900051-07-0
  14. 14.
    Reuther, A., et al.: Scheduler technologies in support of high performance data analysis. In: 2016 IEEE High Performance Extreme Computing Conference (HPEC), pp. 1–6. IEEE (2016)Google Scholar
  15. 15.
    Schwan, P., et al.: Lustre: building a file system for 1000-node clusters. In: Proceedings of the 2003 Linux Symposium, vol. 2003, pp. 380–386 (2003)Google Scholar
  16. 16.
    Snyder, J.P.: Flattening the Earth: Two Thousand Years of Map Projections. University of Chicago Press, Chicago (1997)Google Scholar
  17. 17.
    de Sousa, L.M., Poggio, L., Kempen, B.: Comparison of FOSS4G supported equal-area projections using discrete distortion indicatrices. ISPRS Int. J. Geo-Inf. 8(8), 351 (2019)CrossRefGoogle Scholar
  18. 18.
    Tange, O.: GNU Parallel 2018.  https://doi.org/10.5281/zenodo.1146014
  19. 19.
    Thomas Koenig: sysconf manual page, in Linux Programmer’s manual (2019). http://man7.org/linux/man-pages/man3/sysconf.3.html. Accessed 2019 Oct 15
  20. 20.
    Yoo, A.B., Jette, M.A., Grondona, M.: SLURM: simple linux utility for resource management. In: Feitelson, D., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 44–60. Springer, Heidelberg (2003).  https://doi.org/10.1007/10968987_3CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2020

Authors and Affiliations

  1. 1.ISRIC - World Soil InformationWageningenThe Netherlands
  2. 2.FB-ITWageningen University and ResearchWageningenThe Netherlands

Personalised recommendations