Advertisement

Geospatial Big Data for Environmental and Agricultural Applications

  • Athanasios Karmas
  • Angelos Tzotsos
  • Konstantinos Karantzalos
Chapter

Abstract

Earth observation (EO) and environmental geospatial datasets are growing at an unprecedented rate in size, variety and complexity, thus, creating new challenges and opportunities as far as their access, archiving, processing and analytics are concerned. Currently, huge imaging streams are reaching several petabytes in many satellite archives worldwide. In this chapter, we review the current state-of-the-art in big data frameworks able to access, handle, process, analyse and deliver geospatial data and value-added products. Operational services that feature efficient implementations and different architectures allowing in certain cases the online and near real-time processing and analytics are detailed. Based on the current status, state-of-the-art and emerging challenges, the present study highlights certain issues, insights and future directions towards the efficient exploitation of EO big data for important engineering, environmental and agricultural applications.

Keywords

Cloud Computing Geospatial Data Raster Data Normalize Difference Water Index Hadoop Distribute File System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Adamov A (2012) Distributed file system as a basis of data-intensive computing. In: 2012 6th International conference on application of information and communication technologies (AICT), pp 1–3. doi: 10.1109/ICAICT.2012.6398484
  2. 2.
    Aiordachioaie A, Baumann P (2010) Petascope: An open-source implementation of the ogc wcs geo service standards suite. In: Gertz M, Ludascher B (eds) Scientific and statistical database management. Lecture Notes in Computer Science, vol 6187, Springer, Berlin/Heidelberg, pp 160–168Google Scholar
  3. 3.
    Aji A, Wang F, Vo H, Lee R, Liu Q, Zhang X, Saltz J (2013) Hadoop gis: A high performance spatial data warehousing system over mapreduce. Proc VLDB Endowment 6(11):1009–1020. doi: 10.14778/2536222.2536227, http://dx.doi.org/10.14778/2536222.2536227
  4. 4.
    Asrar G, Kanemasu E, Yoshida M (1985) Estimates of leaf area index from spectral reflectance of wheat under different cultural practices and solar angles. Remote Sens Environ 17:1–11CrossRefGoogle Scholar
  5. 5.
    Assuncao MD, Calheiros RN, Bianchi S, Netto MA, Buyya R (2014) Big data computing and clouds: trends and future directions. J Parallel Distrib Comput. doi:http://dx.doi.org/10.1016/j.jpdc.2014.08.003, http://www.sciencedirect.com/science/article/pii/S0743731514001452
  6. 6.
    Babaee M, Datcu M, Rigoll G (2013) Assessment of dimensionality reduction based on communication channel model; application to immersive information visualization. In: 2013 IEEE international conference on big data, pp 1–6. doi: 10.1109/BigData.2013.6691726
  7. 7.
    Barroso L, Dean J, Holzle U (2003) Web search for a planet: the google cluster architecture. IEEE Micro 23(2):22–28. doi: 10.1109/MM.2003.1196112 CrossRefGoogle Scholar
  8. 8.
    Baumann P (1994) Management of multidimensional discrete data. Int J Very Large Data Bases 4(3):401–444CrossRefGoogle Scholar
  9. 9.
    Baumann P (1999) A database array algebra for spatio-temporal data and beyond. In: Next generation information technologies and systems, pp 76–93Google Scholar
  10. 10.
    Baumann P (2009) Array databases and raster data management. In: Ozsu T, Liu L (eds), Encyclopedia of database systems. Springer, New YorkGoogle Scholar
  11. 11.
    Baumann P (2010) The OGC web coverage processing service (WCPS) standard. GeoInformatica 14(4):447–479. doi: 10.1007/s10707-009-0087-2 MathSciNetCrossRefGoogle Scholar
  12. 12.
    Baumann P (2012) OGC WCS 2.0 Interface Standard-Core: Corrigendum (OGC 09-110r4)Google Scholar
  13. 13.
    Baumann P (2014) rasdaman: array databases boost spatio-temporal analytics. In: 2014 fifth international conference on computing for geospatial research and application (COM.Geo), pp 54–54Google Scholar
  14. 14.
    Baumann P, Nativi S (2012) Adding big earth data analytics to geoss. Group on Earth Observations Ninth Plenary Session – GEO-IX. Brazil, 22–23 NovemberGoogle Scholar
  15. 15.
    Baumann P, Dehmel A, Furtado P, Ritsch R, Widmann N (1998) The multidimensional database system rasdaman. In: Proceedings of the 1998 ACM SIGMOD international conference on management of data. ACM Press, New York, pp 575–577CrossRefGoogle Scholar
  16. 16.
    Begoli E, Horey J (2012) Design principles for effective knowledge discovery from big data. In: 2012 joint working IEEE/IFIP conference on IEEE software architecture (WICSA) and European conference on software architecture (ECSA), pp 215–218Google Scholar
  17. 17.
    Buehler K, McKee L (2006) The openGIS guide (third edition). In: Technical Committee, version 1, Engineering Specification Best Practices, OGIS TC Doc. 96-001Google Scholar
  18. 18.
    Cammalleri C, Anderson M, Gao F, Hain C, Ku W (2014) Mapping daily evapotranspiration at field scales over rainfed and irrigated agricultural areas using remote sensing data fusion. Agr Forest Meteorol 186(0):1–11Google Scholar
  19. 19.
    Cappelaere P, Sanchez S, Bernabe S, Scuri A, Mandl D, Plaza A (2013) Cloud implementation of a full hyperspectral unmixing chain within the nasa web coverage processing service for EO-1. IEEE J Sel Top Appl Earth Obs Remote Sens 6(2):408–418. doi: 10.1109/JSTARS.2013.2250256 CrossRefGoogle Scholar
  20. 20.
    CartoDB (Retrieved 2015) https://cartodb.com/platform
  21. 21.
    Chen J, Chen J, Liao A, Cao X, Chen L, Chen X, He C, Han G, Peng S, Lu M, Zhang W, Tong X, Mills J (2014) Global land cover mapping at 30m resolution: a POK-based operational approach. Int J Photogr Remote Sens. doi:http://dx.doi.org/10.1016/j.isprsjprs.2014.09.002
  22. 22.
    Choo J, Park H (2013) Customizing computational methods for visual analytics with big data. Computer Graphics and Applications, IEEE 33(4):22–28CrossRefGoogle Scholar
  23. 23.
    Davis B (1996) GIS: A Visual Approach. OnWord PressGoogle Scholar
  24. 24.
    de la Beaujardiere J (2006) OpenGIS Web Map Server Implementation Specification (OGC 06-042)Google Scholar
  25. 25.
    Dean J, Ghemawat S (2008) Mapreduce: Simplified data processing on large clusters. Commun ACM 51(1):107–113. doi  10.1145/1327452.1327492, http://doi.acm.org/10.1145/1327452.1327492
  26. 26.
    Demchenko Y, Zhao Z, Grosso P, Wibisono A, De Laat C (2012) Addressing big data challenges for scientific data infrastructure. In: 2012 IEEE 4th international conference on cloud computing technology and science (CloudCom). IEEE, New York, pp 614–617Google Scholar
  27. 27.
    Espinoza-Molina D, Datcu M (2013) Earth-observation image retrieval based on content, semantics, and metadata. IEEE IEEE Trans Geosci Remote Sens 51(11):5145–5159. doi: 10.1109/TGRS.2013.2262232 CrossRefGoogle Scholar
  28. 28.
    Evangelidis K, Ntouros K, Makridis S, Papatheodorou C (2014) Geospatial services in the cloud. Comput. Geosci. 63(0):116–122. doi:http://dx.doi.org/10.1016/j.cageo.2013.10.007, http://www.sciencedirect.com/science/article/pii/S0098300413002719
  29. 29.
    Foster I, Zhao Y, Raicu I, Lu S (2008) Cloud computing and grid computing 360-degree compared. In: Grid computing environments workshop, 2008 (GCE ’08), pp 1–10. doi: 10.1109/GCE.2008.4738445
  30. 30.
    Furht B, Escalante A (2011) Handbook of cloud computing. Springer, New YorkzbMATHGoogle Scholar
  31. 31.
    Garcia-Rojas A, Athanasiou S, Lehmann J, Hladky D (2013) Geoknow: leveraging geospatial data in the web of data. In: Open data on the web workshop, http://jens-lehmann.org/files/2013/odw_geoknow.pdf Google Scholar
  32. 32.
    gigaomcom (Retrieved 2015) Can you predict future traffic patterns? Nokia thinks it can. https://gigaom.com/2013/07/02/living-cities-lights-up-traffic-in-5-cities-with-interactive-data-visualization/
  33. 33.
    Glenn EP, Huete AR, Nagler PL, Nelson SG (2008) Relationship between remotely-sensed vegetation indices, canopy attributes and plant physiological processes: what vegetation indices can and cannot tell us about the landscape. Sensors 8(4):2136. doi: 10.3390/s8042136, http://www.mdpi.com/1424-8220/8/4/2136
  34. 34.
    Gray J (2008) Distributed computing economics. Queue 6(3):63–68. doi: 10.1145/1394127.1394131, http://doi.acm.org/10.1145/1394127.1394131
  35. 35.
    Habib S, Morozov V, Frontiere N, Finkel H, Pope A, Heitmann K (2013) Hacc: Extreme scaling and performance across diverse architectures. In: Proceedings of the international conference on high performance computing, networking, storage and analysis (SC ’13). ACM, New York, pp 6:1–6:10. doi: 10.1145/2503210.2504566, http://doi.acm.org/10.1145/2503210.2504566
  36. 36.
    Han J, Haihong E, Le G, Du J (2011) Survey on nosql database. In: 2011 6th international conference on pervasive computing and applications (ICPCA), pp 363–366. doi: 10.1109/ICPCA.2011.6106531
  37. 37.
    Han W, Yang Z, Di L, Yue P (2014) A geospatial web service approach for creating on-demand cropland data layer thematic maps. Transactions of the ASABE 57(1):239–247. doi:http://dx.doi.org/10.13031/trans.57.10020
  38. 38.
    Hansen MC, Potapov PV, Moore R, Hancher M, Turubanova SA, Tyukavina A, Thau D, Stehman SV, Goetz SJ, Loveland TR, Kommareddy A, Egorov A, Chini L, Justice CO, Townshend JRG (2013) High-resolution global maps of 21st-century forest cover change. Science 342(6160):850–853. doi: 10.1126/science.1244693 CrossRefGoogle Scholar
  39. 39.
    Hatfield JL, Prueger JH (2010) Value of using different vegetative indices to quantify agricultural crop characteristics at different growth stages under varying management practices. Remote Sens 2(2):562. doi: 10.3390/rs2020562, http://www.mdpi.com/2072-4292/2/2/562
  40. 40.
    Hunter PD, Tyler AN, Présing M, Kovács AW, Preston T (2008) Spectral discrimination of phytoplankton colour groups: the effect of suspended particulate matter and sensor spectral resolution. Remote Sens Environ 112(4):1527–1544. doi:http://dx.doi.org/10.1016/j.rse.2007.08.003, http://www.sciencedirect.com/science/article/pii/S0034425707004051, remote Sensing Data Assimilation Special Issue
  41. 41.
    Hwang K, Choi M (2013) Seasonal trends of satellite-based evapotranspiration algorithms over a complex ecosystem in East Asia. Remote Sens Environ 137(0):244–263Google Scholar
  42. 42.
    Idreos S, Kersten ML, Manegold S (2007) Database cracking. In: CIDR 2007, Third biennial conference on innovative data systems research, Asilomar, CA, January 7-10, 2007, Online Proceedings, pp 68–78, http://www.cidrdb.org/cidr2007/papers/cidr07p07.pdf
  43. 43.
    Idreos S, Groffen F, Nes N, Manegold S, Mullender S, Kersten M (2012) Monetdb: two decades of research in column-oriented database architectures. IEEE Data Eng Bull 35(1):40–45Google Scholar
  44. 44.
    Ivanova MG, Kersten ML, Nes NJ, Gonçalves RA (2010) An architecture for recycling intermediates in a column-store. ACM Trans Database Syst 35(4):24:1–24:43. doi: 10.1145/1862919.1862921, http://doi.acm.org/10.1145/1862919.1862921
  45. 45.
    Ivanova M, Kersten M, Manegold S (2012) Data vaults: A symbiosis between database technology and scientific file repositories. In: Ailamaki A, Bowers S (eds) Scientific and statistical database management. Lecture notes in computer science, vol. 7338. Springer, Berlin/Heidelberg, pp 485–494. doi: 10.1007/978-3-642-31235-9_32, http://dx.doi.org/10.1007/978-3-642-31235-9_32
  46. 46.
    Karantzalos K, Bliziotis D, Karmas A (2015) A scalable web geospatial service for near real-time, high-resolution land cover mapping. IEEE J Sel Top Appl Earth Obs Remote Sens Special Issue on ‘Big Data in Remote Sensing’ 8(10):4665–4674CrossRefGoogle Scholar
  47. 47.
    Karantzalos K, Karmas A, Tzotsos A (2015) RemoteAgri: processing online big earth observation data for precision agriculture. In: European conference on precision agricultureGoogle Scholar
  48. 48.
    Karmas A, Karantzalos K (2015) Benchmarking server-side software modules for handling and processing remote sensing data through rasdaman. In: (WHISPERS) IEEE workshop on hyperspectral image and signal processing: evolution in remote sensingGoogle Scholar
  49. 49.
    Karmas A, Karantzalos K, Athanasiou S (2014) Online analysis of remote sensing data for agricultural applications. In: OSGeo’s European conference on free and open source software for geospatialGoogle Scholar
  50. 50.
    Karmas A, Tzotsos A, Karantzalos K (2015) Scalable geospatial web services through efficient, online and near real-time processing of earth observation data. In: (BigData Service 2015) IEEE international conference on big data computing service and applicationsGoogle Scholar
  51. 51.
    Kopsiaftis G, Karantzalos K (2015) Vehicle detection and traffic density monitoring from very high resolution satellite video data. In: IEEE international geoscience and remote sensing symposium (IGARSS 2015)Google Scholar
  52. 52.
    Koubarakis M, Kontoes C, Manegold S (2013) Real-time wildfire monitoring using scientific database and linked data technologies. In: 16th international conference on extending database technologyGoogle Scholar
  53. 53.
    Kouzes R, Anderson G, Elbert S, Gorton I, Gracio D (2009) The changing paradigm of data-intensive computing. Computer 42(1):26–34. doi: 10.1109/MC.2009.26 CrossRefGoogle Scholar
  54. 54.
    Laney D (Retrieved 6 February 2001) 3d data management: controlling data volume, velocity and variety. GartnerGoogle Scholar
  55. 55.
    Lee C, Gasster S, Plaza A, Chang CI, Huang B (2011) Recent developments in high performance computing for remote sensing: a review. IEEE J Selected Top Appl Earth Obsand Remote Sens 4(3):508–527. doi: 10.1109/JSTARS.2011.2162643 CrossRefGoogle Scholar
  56. 56.
    Liu B, Blasch E, Chen Y, Shen D, Chen G (2013) Scalable sentiment classification for big data analysis using Naive Bayes Classifier. In: 2013 IEEE international conference on big data, pp 99–104. doi: 10.1109/BigData.2013.6691740
  57. 57.
    Ma Y, Wang L, Liu P, Ranjan R (2014) Towards building a data-intensive index for big data computing - a case study of remote sensing data processing. Information Sciences. doi:http://dx.doi.org/10.1016/j.ins.2014.10.006
  58. 58.
    Ma Y, Wang L, Zomaya A, Chen D, Ranjan R (2014) Task-tree based large-scale mosaicking for massive remote sensed imageries with dynamic dag scheduling. IEEE Trans Parallel Distrib Syst 25(8):2126–2137. doi: 10.1109/TPDS.2013.272 CrossRefGoogle Scholar
  59. 59.
    Ma Y, Wu H, Wang L, Huang B, Ranjan R, Zomaya A, Jie W (2014) Remote sensing big data computing: challenges and opportunities. Futur Gener Comput Syst. doi:http://dx.doi.org/10.1016/j.future.2014.10.029, http://www.sciencedirect.com/science/article/pii/S0167739X14002234
  60. 60.
    Menzies T, Zimmermann T (2013) Software analytics: so what? IEEE Softw 30(4):31–37CrossRefGoogle Scholar
  61. 61.
    MonetDB (Retrieved 2015) https://www.monetdb.org/home/features
  62. 62.
    Nebert D, Whiteside A, Vretanos P (2007) OpenGIS Catalogue Services Specification (OGC 07-006r1)Google Scholar
  63. 63.
    NGA (2014) Digitalglobe application a boon to raster data storage, processingGoogle Scholar
  64. 64.
  65. 65.
    Nikolaou C, Kyzirakos K, Bereta K, Dogani K, Giannakopoulou S, Smeros P, Garbis G, Koubarakis M, Molina D, Dumitru O, Schwarz G, Datcu M (2014) Big, linked and open data: applications in the German aerospace center. In: The semantic web: ESWC 2014 satellite events. Lecture notes in computer science. Springer International Publishing, New York, pp 444–449. doi: 10.1007/978-3-319-11955-7_64, http://dx.doi.org/10.1007/978-3-319-11955-7_64
  66. 66.
    OGC (Retrieved 20 June 2015) OGC abstract specifications. http://www.opengeospatial.org/standards/as
  67. 67.
    OGC (Retrieved 20 June 2015) OGC history. http://www.opengeospatial.org/ogc/historylong
  68. 68.
    Oosthoek J, Flahaut J, Rossi A, Baumann P, Misev D, Campalani P, Unnithan V (2013) Planetserver: innovative approaches for the online analysis of hyperspectral satellite data from Mars. Adv Space Res pp 219–244. doi:http://dx.doi.org/10.1016/j.asr.2013.07.002
  69. 69.
    Palmer SC, Hunter PD, Lankester T, Hubbard S, Spyrakos E, Tyler AN, Présing M, Horváth H, Lamb A, Balzter H, Tóth VR (2015) Validation of envisat {MERIS} algorithms for chlorophyll retrieval in a large, turbid and optically-complex shallow lake. Remote Sens Environ 157(0):158–169. doi:http://dx.doi.org/10.1016/j.rse.2014.07.024, http://www.sciencedirect.com/science/article/pii/S0034425714002739, [special Issue: Remote Sensing of Inland Waters]
  70. 70.
    Pelekis N, Theodoridis Y (2014) Mobility data management and exploration. Springer, New YorkCrossRefGoogle Scholar
  71. 71.
    Pettorelli N, Vik J, Mysterud A, Gaillard J, Tucker C, Stenseth N (2005) Using the satellite-derived ndvi to assess ecological responses to environmental change. Trends Ecol Evol 20:503–510CrossRefGoogle Scholar
  72. 72.
    Pijanowski BC, Tayyebi A, Doucette J, Pekin BK, Braun D, Plourde J (2014) A big data urban growth simulation at a national scale: configuring the GIS and neural network based land transformation model to run in a high performance computing (HPC) environment. Environ Model Software 51(0):250–268. doi:http://dx.doi.org/10.1016/j.envsoft.2013.09.015
  73. 73.
    Plaza AJ (2009) Special issue on architectures and techniques for real-time processing of remotely sensed images. J Real-Time Image Proc 4(3):191–193MathSciNetCrossRefGoogle Scholar
  74. 74.
    Plaza AJ, Chang CI (2007) High performance computing in remote sensing. Chapman & Hall/CRC Press, New YorkCrossRefGoogle Scholar
  75. 75.
    Repository CC (Retrieved 2015) https://github.com/cartodb/cartodb.jsGoogle Scholar
  76. 76.
    Rouse JW Jr, Haas RH, Schell JA, Deering DW (1974) Monitoring vegetation systems in the great Plains with Erts, vol.351. NASA Special Publication, Washington p 309Google Scholar
  77. 77.
    Russom P (2011) Big data analytics. TDWI best practices report, The Data Warehousing Institute (TDWI) ResearchGoogle Scholar
  78. 78.
    Sakr S, Liu A, Batista D, Alomari M (2011) A survey of large scale data management approaches in cloud environments. IEEE Commun Surv Tutorials 13(3):311–336. doi: 10.1109/SURV.2011.032211.00087 CrossRefGoogle Scholar
  79. 79.
    Sass G, Creed I, Bayley S, Devito K (2007) Understanding variation in trophic status of lakes on the boreal plain: a 20 year retrospective using landsat {TM} imagery. Remote Sens Environ 109(2):127–141CrossRefGoogle Scholar
  80. 80.
    Schut P (2007) OpenGIS web processing service (OGC 05-007r7)Google Scholar
  81. 81.
    Vouk M (2008) Cloud computing 2014; issues, research and implementations. In: 30th international conference on information technology interfaces, 2008 (ITI 2008), pp 31–40. doi: 10.1109/ITI.2008.4588381
  82. 82.
    Vretanos PPA (2010) OpenGIS Web Feature Service 2.0 Interface Standard (OGC 09-025r1 and ISO/DIS 19142)Google Scholar
  83. 83.
    Yu P (2013) On mining big data. In: Wang J, Xiong YH (ed) Web-age information management. Lecture notes in computer science. Springer, Berlin, HeidelbergGoogle Scholar
  84. 84.
    Yue P, Gong J, Di L, Yuan J, Sun L, Sun Z, Wang Q (2010) Geopw: laying blocks for the geospatial processing web. Trans GIS 14(6):755–772. doi: 10.1111/j.1467-9671.2010.01232.x, http://dx.doi.org/10.1111/j.1467-9671.2010.01232.x
  85. 85.
    Yue P, Di L, Wei Y, Han W (2013) Intelligent services for discovery of complex geospatial features from remote sensing imagery. ISPRS J Photogramm Remote Sens 83(0):151–164. doi:http://dx.doi.org/10.1016/j.isprsjprs.2013.02.015, http://www.sciencedirect.com/science/article/pii/S0924271613000580
  86. 86.
    Zeiler M (1999) Modeling our world: the ESRI guide to geodatabase design. ESRI Press, RedlandsGoogle Scholar
  87. 87.
    Zell E, Huff A, Carpenter A, Friedl L (2012) A user-driven approach to determining critical earth observation priorities for societal benefit. IEEE J Sel Top Appl Earth Obs Remote Sens 5(6):1594–1602. doi: 10.1109/JSTARS.2012.2199467 CrossRefGoogle Scholar
  88. 88.
    Zhang X, Seelan S, Seielstad G (2010) Digital northern great plains: a web-based system delivering near real time remote sensing data for precision agriculture. Remote Sens 2(3):861. doi: 10.3390/rs2030861, http://www.mdpi.com/2072-4292/2/3/861
  89. 89.
    Zhang Y, Kersten M, Ivanova M, Nes N (2011) Sciql: bridging the gap between science and relational dbms. In: Proceedings of the 15th symposium on international database engineering & Applications (IDEAS ’11). ACM, New York, NY, pp 124–133. doi: 10.1145/2076623.2076639, http://doi.acm.org/10.1145/2076623.2076639
  90. 90.
    Zhang Y, Scheers B, Kersten MNN Mand Ivanova (2011) Astronomical data processing using sciql, an sql based query language for array data. In: Astronomical data analysis software and systems XXI, vol 461, p 729Google Scholar
  91. 91.
    Zhao P, Foerster T, Yue P (2012) The geoprocessing web. Comput Geosci 47(0): 3–12. doi:http://dx.doi.org/10.1016/j.cageo.2012.04.021, http://www.sciencedirect.com/science/article/pii/S0098300412001446, towards a Geoprocessing Web
  92. 92.
    Zikopoulos P, Eaton C (2012) Understanding big data: analytics for enterprise class hadoop and streaming data. McGraw-Hill Companies, Inc., New YorkGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Athanasios Karmas
    • 1
  • Angelos Tzotsos
    • 1
  • Konstantinos Karantzalos
    • 1
  1. 1.Remote Sensing LaboratoryNational Technical University of AthensAthensGreece

Personalised recommendations