Skip to main content
SpringerLink
Log in

Big Data in Geophysics and Other Earth Sciences

  • REWIEVS
  • Published:
Izvestiya, Physics of the Solid Earth Aims and scope Submit manuscript

Abstract—The term “Big Data” has become very popular over the past decade. The frequency of its use in the research papers, reports, and broad press has been steadily increasing. This work describes the origin and development of the theory and practice of Big Data as a scientific discipline, outlines the main characteristics and methods for Big Data processing and analysis, discusses the formalism and family of Big Data V-characteristics, and presents the examples of the sources of the growing Big Data which have fundamental effect on the development of geophysics and related Earth sciences. The examples of the sources of Big Data in the Earth sciences are remote sensing, meteorology, geoecology (in terms of the global hierarchical network SMEAR (Stations Measuring Earth surfaces and Atmosphere Relations)), and seismic exploration. Besides, we discuss seismic monitoring data which can become Big Data when combined with other geophysical information and consider geomagnetic data which are not Big Data but nevertheless have a great scientific value.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.
Fig. 12.
Fig. 13.
Fig. 14.
Fig. 15.

Similar content being viewed by others

REFERENCES

  1. Ahmad, L., Habib Kanth, R., Parvaze, S., and Sheraz Mahdi, S., Experimental Agrometeorology: A Practical Manual, Cham: Springer, 2017a. https://doi.org/10.1007/978-3-319-69185-5

    Book  Google Scholar 

  2. Ahmad, L., Habib Kanth, R., Parvaze, S., and Sheraz Mahdi, S., Synoptic meteorology, in Experimental Agrometeorology: A Practical Manual, Cham: Springer, 2017b, pp. 119–121. https://doi.org/10.1007/978-3-319-69185-5_16

    Book  Google Scholar 

  3. Ammon, C.J., Lay, T., and Simpson, D.W., Great earthquakes and Global Seismic Network, Seismol. Res. Lett., 2010, vol. 81, no. 6, pp. 965–971. https://doi.org/10.1785/gssrl.81.6.965

    Article  Google Scholar 

  4. Armstrong, E.M., Bourassa, M.A., Cram, T.A., DeBellis, M., Elya, J., Greguska, F.R. III, Huang, T., Jacob, J.C., Ji, Z., Jiang, Y., Li, Y., Quach, N., McGibbney, L., Smith, S., Tsontos, V.M., et al., An Integrated Data Analytics Platform, Front. Mar. Sci., 2019, vol. 6, article 354. https://doi.org/10.3389/fmars.2019.00354

    Article  Google Scholar 

  5. Ashton, K., That ‘Internet of Things’ Thing, RFID Journal, 2009. https://www.rfidjournal.com/that-internet-of-things-thing-3. Cited March 17, 2021.

  6. Ball, J.E., Anderson, D.T., and Chan, C.S., Sr., Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community, J. Appl. Remote Sens., 2017, vol. 11, no. 4, Paper ID 042609. https://doi.org/10.1117/1.JRS.11.042609

  7. Balta, H., Velagic, J., Bosschaerts, W., De Cubber, G., and Siciliano, B., Fast statistical outlier removal based method for large 3D point clouds of outdoor environments, IFAC-PapersOnLine, 2018, vol. 51, no. 22, pp. 348–353. https://doi.org/10.1016/j.ifacol.2018.11.566

    Article  Google Scholar 

  8. Barry, R.M., Cavers, D.A., and Kneale, C.W., Recommended standards for digital tape formats, Geophysics, 1975, vol. 40, pp. 344–352.

    Article  Google Scholar 

  9. Baturin, Yu.M. and Shcherbinin, D.Yu., Photo and film technology on board domestic manned spacecraft (1961– 2000), Vopr. Istor. Estestvozn. Tekh., 2011, vol. 32, no. 3, pp. 87–104.

    Google Scholar 

  10. Baumann, P.R., History of Remote Sensing, Satellite Imagery, Part II, 2009. http://employees.oneonta.edu/baumanpr/geosat2/RS%20History%20II/RS-History-Part-2.html. Cited March 18, 2021.

  11. Baumann, P., Mazzetti, P., Ungar, J., et al., Big Data Analytics for Earth Sciences: the EarthServer approach, Int. J. Digital Earth, 2016, vol. 9, no. 1, pp. 3–29. https://doi.org/10.1080/17538947.2014.1003106

    Article  Google Scholar 

  12. Becker-Reshef, I., Vermote, E., Lindeman, M., and Justice, C., A generalized regression-based model for forecasting winter wheat yields in Kansas and Ukraine using MODIS data, Remote Sens. Environ., 2010, vol. 114, no. 6, pp. 1312–1323. https://doi.org/10.1016/j.rse.2010.01.010

    Article  Google Scholar 

  13. Besic, N., Figueras i Ventura, J., Grazioli, J., Gabella, M., Germann, U., and Berne, A., Hydrometeor classification through statistical clustering of polarimetric radar measurements: a semi-supervised approach, Atmos. Meas. Tech., 2016, vol. 9, no. 9, pp. 4425–4445. https://doi.org/10.5194/amt-9-4425-2016

    Article  Google Scholar 

  14. Big Data: Techniques and Technologies in Geoinformatics, Karimi, H.A., Ed., London: CRC Press, 2014.

    Google Scholar 

  15. Boman, K., Big data growth continues in seismic surveys. Rigzone, Sept. 2, 2015. https://www.rigzone.com/news/ oil_gas/a/140418/big_data_growth_continues_in_seismic_ surveys/. Cited March 18, 2021.

  16. Bondur, V.G., Modern approaches to processing large hyperspectral and multispectral aerospace data flows, Izv., Atmos. Ocean. Phys., 2014, vol. 50, no. 9, pp. 840–852. https://doi.org/10.7868/S0205961414010035

    Article  Google Scholar 

  17. Caers, J., Modeling Uncertainty in the Earth Sciences, Chichester: Wiley, 2011.

    Book  Google Scholar 

  18. Chen, D., Wang, L., Dou, M., and Liu, Z., Natural Disaster Monitoring with Wireless Sensor Networks: A Case Study of Data-intensive Applications upon Low-Cost Scalable Systems, Mobile Networks Appl., 2013, vol. 18, pp. 651–663. https://doi.org/10.1007/s11036-013-0456-9

    Article  Google Scholar 

  19. Chen, M., Mao, S., Zhang, Y., and Leung, V.C.M., Big Data. Related Technologies, Challenges, and Future Prospects, Cham: Springer, 2014. https://doi.org/10.1007/978-3-319-06245-7

    Book  Google Scholar 

  20. Chi, M., Plaza, A., Benediktsson, J.A., Sun, Z., Shen, J., and Zhu, Y., Big data for remote sensing: challenges and opportunities, Proc. IEEE, 2016, vol. 104, no. 11, pp. 2207–2219. https://doi.org/10.1109/JPROC.2016.2598228

    Article  Google Scholar 

  21. Chini, P., Giambene, G., and Kota, S., A survey on mobile satellite systems, Int. J. Satell. Commun. Networking, 2009, vol. 28, no. 1. https://doi.org/10.1002/sat.941

  22. Chowdhury, S. and Al-Zahrani, M., Water quality change in dam reservoir and shallow aquifer: analysis on trend, seasonal variability and data reduction, Environ. Monit. Assess., 2014, vol. 186, pp. 6127–6143. https://doi.org/10.1007/s10661-014-3844-0

    Article  Google Scholar 

  23. Chulliat, A., Alken, P., Nair, M., et al., The US/UK World Magnetic Model for 2015–2020: Technical Report, National Geophysical Data Center, NOAA, 2015. https://doi.org/10.7289/V5TB14V7

  24. Corizzo, R., Ceci, M., and Japkowicz, N., Anomaly detection and repair for accurate predictions in geo-distributed big data, Big Data Res., 2019, vol. 16, pp. 18–35. https://doi.org/10.1016/j.bdr.2019.04.001

    Article  Google Scholar 

  25. Cressie, N., Statistics for Spatial Data, New York: Wiley, 1993.

    Book  Google Scholar 

  26. Davis, T.N. and Sugiura, M., Auroral electrojet activity index AE and its universal time variations, J. Geophys. Res., 1966, vol. 71, no. 3, pp. 785–801. https://doi.org/10.1029/JZ071i003p00785

    Article  Google Scholar 

  27. de Jong, S.M., van der Meer, F.D., and Clevers, J.G. Basics of remote sensing, Ch. 1 of Remote Sensing Image Analysis: Including The Spatial Domain, de Jong, S.M. and van der Meer, F., Eds., Remote Sensing and Digital Image Processing Ser., vol. 5, Dordrecht: Springer, 2004, pp. 1–15. https://doi.org/10.1007/978-1-4020-2560-0_1

  28. Dedić, N. and Stanier, C., Towards differentiating business intelligence, big data, data analytics and knowledge discovery, in Innovations in Enterprise Information Systems Management and Engineering. ERP Future 2016, LNBIP vol. 285, Piazolo, F., Geist, V., Brehm, L., and Schmidt, R., Eds., Cham: Springer, 2017, pp. 114–122. https://doi.org/10.1007/978-3-319-58801-8_10

  29. Ding, A.J., Fu, C.B., Yang, X.Q., Sun, J.N., Petäjä, T., Kerminen, V.-M., Wang, T., Xie, Y., Herrmann, E., Zheng, L.F., Nie, W., Liu, Q., Wei, X.L., and Kulmala, M., Intense atmospheric pollution modifies weather: a case of mixed biomass burning with fossil fuel combustion pollution in eastern China, Atmos. Chem. Phys., 2013, vol. 13, no. 20, pp. 10545–10554. https://doi.org/10.5194/acp-13-10545-2013

    Article  Google Scholar 

  30. Dormy, E. and Le Mouël, J.-L., Geomagnetism and the dynamo: where do we stand?, C. R. Phys., 2008, vol. 9, no. 7, pp. 711–720. https://doi.org/10.1016/j.crhy.2008.07.003

    Article  Google Scholar 

  31. Earth Observations from Space: The First 50 Years of Scientific Achievements, National Research Council, Washington: National Academies Press, 2008. https://doi.org/10.17226/11991

  32. ESA—Swarm probes weakening of Earth’s magnetic field, 2020. http://www.esa.int/Applications/Observing_the_ Earth/Swarm/Swarm_probes_weakening_of_Earth_s_ magnetic_field. Cited March 18, 2021.

  33. Esch, T., Üreyen, S., Zeidler, J., Metz-Marconcini, A., Hirner, A., Asamer, H., Tum, M., Böttcher, M., Kuchar, S., Svaton, V., and Marconcini, M., Exploiting big earth data from space—first experiences with the timescan processing chain, Big Earth Data, 2018, vol. 2, no. 1, pp. 36–55. https://doi.org/10.1080/20964471.2018.1433790

    Article  Google Scholar 

  34. Feblowitz, J., Insights IDCE: Analytics in oil and gas: the big deal about big data, Proc. SPE Digital Energy Conference and Exhibition, Woodlands, 2013, Paper ID SPE-163717-MS. https://doi.org/10.2118/163717-MS

  35. Fernández-Martínez, J.L., Model reduction and uncertainty analysis in inverse problems, Leading Edge, 2015, vol. 9, pp. 1006–1016. https://doi.org/10.1190/tle34091006.1

    Article  Google Scholar 

  36. Fouedjio, F. and Klump, J., Exploring prediction uncertainty of spatial data in geostatistical and machine learning approaches, Environ. Earth Sci., 2019, vol. 78, no. 1, article 38. https://doi.org/10.1007/s12665-018-8032-z

    Article  Google Scholar 

  37. Franks, B., Taming the Big Data Tidal Wave, Hoboken: Wiley, 2012.

    Book  Google Scholar 

  38. Giuliani, G., Lacroix, P., Guigoz, Y., Roncella, R., Bigagli, L., Santoro, M., Mazzetti, P., Nativi, S., Ray, N., and Lehmann, A., Bringing GEOSS services into practice: a capacity building resource on spatial data infrastructures (SDI), Trans. GIS, 2017, vol. 21, no. 4, pp. 811–824. https://doi.org/10.1111/tgis.12209

    Article  Google Scholar 

  39. Gjerloev, J.W., The SuperMAG data processing technique, J. Geophys. Res.: Space Phys., 2012, vol. 117, no. 9, Paper ID A09213. https://doi.org/10.1029/2012JA017683

  40. Global Observing System (GOS), 2016. https://www.wmo.int/ pages/prog/www/OSY/GOS.html. Cited March 18, 2021.

  41. Global Telecommunication System (GTS) Main. World Meteorogical Organization (WMO), 2020. https://www. wmo.int/pages/prog/www/TEM/index_en.html. Cited March 18, 2021.

  42. Gomes, V.C., Queiroz, G.R., and Ferreira, K.R., An overview of platforms for big Earth observation data management and analysis, Remote Sens., 2020, vol. 12, no. 8, Paper ID 1253. https://doi.org/10.3390/rs12081253.

  43. Gonzalez, W.D., Joselyn, J.A., Kamide, Y., Kroehl, H.W., Rostoker, G., Tsurutani, B.T., and Vasyliunas, V.M., What is a geomagnetic storm?, J. Geophys. Res.: Space Phys., 1994, vol. 99, no. A4, pp. 5771–5792. https://doi.org/10.1029/93JA02867

    Article  Google Scholar 

  44. Gordon, A., Grace, W., Schwerdtfeger, P., and Byron-Scott, R., Dynamic Meteorology: A Basic Course, London: Routledge, 2016.

    Book  Google Scholar 

  45. Gorelick, N., Hancher, M., Dixon, M., Ilyushchenko, S., Thau, D., and Moore, R., Google Earth Engine: Planetary-scale geospatial analysis for everyone, Remote Sens. Environ., 2017, vol. 202, no. 3, pp. 18–27. https://doi.org/10.1016/j.rse.2017.06.031

    Article  Google Scholar 

  46. GOST (State Standard) 8.417-2002: State System for Ensuring the Uniformity of Measurements. Units of Quantities, 2003.

  47. Gultepe, I., Sharman, R., Williams, P.D., et al., A review of high impact weather for aviation meteorology, Pure Appl. Geophys., 2019, vol. 176, pp. 1869–1921. https://doi.org/10.1007/s00024-019-02168-6

    Article  Google Scholar 

  48. Guo, H., Big Earth data: A new frontier in Earth and information sciences, Big Earth Data, 2017, vol. 1, nos. 1–2, pp. 4–20. https://doi.org/10.1080/20964471.2017.1403062

    Article  Google Scholar 

  49. Gvishiani, A.D. and Lukianova, R.Yu., Estimating the influence of geomagnetic disturbances on the trajectory of the directional drilling of deep wells in the Arctic region, Izv., Phys. Solid Earth, 2018, vol. 54, no. 4, pp. 554–564. https://doi.org/10.1134/S0002333718040051

    Article  Google Scholar 

  50. Gvishiani, A. and Soloviev, A., Observations, Modeling and Systems Analysis in Geomagnetic Data Interpretation, Cham: Springer, 2020. https://doi.org/10.1007/978-3-030-58969-1

    Book  Google Scholar 

  51. Gvishiani, A., Soloviev, A., Krasnoperov, R., and Lukianova, R., Automated hardware and software system for monitoring the Earth’s magnetic environment, Data Sci. J., 2016, vol. 15, no. 18, pp. 1–24. https://doi.org/10.5334/dsj-2016-018

    Article  Google Scholar 

  52. Gvishiani, A.D., Soloviev, A.A., Sidorov, R.V., Krasnope-rov, R.I., Grudnev, A.A., Kudin, D.V., Karapetyan, Dzh.K., and Simonyan, A.O., Progress in organizing geomagnetic monitoring in Russia and the near abroad, Vest. Otd. Nauk Zemle RAN, 2018, vol. 10, Paper ID NZ4001. https://doi.org/10.2205/2018NZ000357

  53. Gvishiani, A.D., Kaftan, V.I., Krasnoperov, R.I., Tatarinov, V.N., and Vavilin, E.V., Geoinformatics and systems analysis in geophysics and geodynamics, Izv., Phys. Solid Earth, 2019a, vol. 55, no. 1, pp. 33–49. https://doi.org/10.31857/S0002-33372019142-60

    Article  Google Scholar 

  54. Gvishiani, A.D., Lukianova, R.Yu., and Soloviev, A.A., Geomagnetizm: ot yadra Zemli do Solntsa (Geomagnetism: from the Core of the Earth to the Sun), Moscow: RAN, 2019b.

  55. Gvishiani, A., Dobrovolsky, M., and Rybkina, A., Big Data and FAIR Data for Data Science, Ch. 6 of Resilience in the Digital Age, Roberts, F.S. and Sheremet, I.A., Eds. Lecture Notes in Computer Science Ser., vol. 12660, Cham: Springer, 2021. https://doi.org/10.1007/978-3-030-70370-7_6

  56. Hadjimitsis, D.G., Clayton, C.R.I., and Hope, V.S., An assessment of the effectiveness of atmospheric correction algorithms through the remote sensing of some reservoirs, Int. J. Remote Sens., 2004, vol. 25, no. 18, pp. 3651–3674. https://doi.org/10.1080/01431160310001647993

    Article  Google Scholar 

  57. Hari, P., Petäjä, T., Bäck, J., Kermine, V.-M., Lappalainen, H.K., Vihma, T., Laurila, T., Viisanen, Y., Vesala, T., and Kulmala, M., Conceptual design of a measurement network of the global change, Atmos. Chem. Phys., 2016, vol. 16, no. 2, pp. 1017–1028. https://doi.org/10.5194/acp-16-1017-2016

    Article  Google Scholar 

  58. Hashem, I.A.T., Yaqoob, I., Anuar, N.B., Mokhtar, S., Gani, A., and Khan, S.U., The rise of “big data” on cloud computing: Review and open research issues, Inf. Syst., 2015, vol. 47, pp. 98–115. https://doi.org/10.1016/j.is.2014.07.006

    Article  Google Scholar 

  59. Hassani, H., Huang, X., and Silva, E., Big data and climate change, Big Data Cognit. Comput., 2019, vol. 3, no. 1, article 12. https://doi.org/10.3390/bdcc3010012

    Article  Google Scholar 

  60. History of World Meteorogical Organization (WMO), 2020. https://public.wmo.int/en/about-us/who-we-are/history-of-wmo. Cited March 18, 2021.

  61. Hoeser, T., Bachofer, F., and Kuenzer, C., Object detection and image segmentation with deep learning on Earth observation data: a review. Part II: Applications, Remote Sens., 2020, vol. 12, no. 18, article 3053. https://doi.org/10.3390/rs12183053

    Article  Google Scholar 

  62. Holloway, J. and Mengersen, K., Statistical machine learning methods and remote sensing for sustainable development goals: a review, Remote Sens., 2018, vol. 10, no. 9, article 1365. https://doi.org/10.3390/rs10091365

    Article  Google Scholar 

  63. Hondula, D.M., Balling, R.C., Andrade, R., et al., Biometeorology for cities, Int. J. Biometeorol., 2017, vol. 61, pp. 59–69. https://doi.org/10.1007/s00484-017-1412-3

    Article  Google Scholar 

  64. Houghton, J., The Physics of Atmospheres, Cambridge: Cambridge Univ. Press, 2002.

    Google Scholar 

  65. Huang, T., NASA Sea Level Change Portal—it is not just another portal site, 2017a. https://trs.jpl.nasa.gov/bitstream/handle/2014/48852/CL%2317-6319.pdf. Cited March 18, 2021.

  66. Huang, T., Armstrong, E., Jacob, J., et al., An introduction to OceanWorks ocean science platform. Earth Science Technology Office (ESTF) 2017b. https://trs.jpl.nasa.gov/ bitstream/handle/2014/48094/CL%2317-2720.pdf. Cited March 18, 2021.

  67. Hulot, G., Finlay, C.C., Constable, C.G., Olsen, N., and Mandea, M., The magnetic field of planet Earth, Space Sci. Rev., 2010, vol. 152, nos. 1–4, pp. 159–222. https://doi.org/10.1007/s11214-010-9644-0

    Article  Google Scholar 

  68. Hulot, G., Vigneron, P., Léger, J.-M., et al., Swarm's absolute magnetometer experimental vector mode, an innovative capability for space magnetometry, Geophys. Res. Lett., 2015, vol. 42, no. 5, pp. 1352–1359.https://doi.org/10.1002/2014GL062700

  69. Hurwitz, J.S., Nugent, A., Halper, F., and Kaufman, M., Big Data for Dummies, New York: Wiley, 2013.

    Google Scholar 

  70. Inmon, W.H. and Linstedt, D., Data Architecture: A Primer for the Data Scientist. Big Data, Data Warehouse and Data Vault, Burlington: Morgan Kaufmann, 2014.

    Google Scholar 

  71. Internet of Things (IoT), Information Technology Gartner Glossary, 2020.

  72. Ishwarappa, J. and Anuradha, J., A brief introduction on Big Data 5Vs characteristics and Hadoop technology, Procedia Comput. Sci., 2015, vol. 48, pp. 319–324. https://doi.org/10.1016/j.procs.2015.04.188

    Article  Google Scholar 

  73. Islam, M. and Reza, S., The rise of big data and cloud computing, Internet Things Cloud Comput., 2019, vol. 7, no. 2, pp. 45–53. https://doi.org/10.11648/j.iotcc.20190702.12

    Article  Google Scholar 

  74. Jankowsky, J. and Sucksdorf, C., Guide for Magnetic Measurements and Observatory Practice, Warsaw: IAGA, 1996.

    Google Scholar 

  75. Jutz, S. and Milagro-Perez, M.P., Copernicus: the European Earth Observation programme, Rev. Teledetección, 2020, no. 56, pp. V–XI. https://doi.org/10.4995/raet.2020.14346

  76. Kang, X., Li, J., and Fan, X., Spatial-temporal visualization and analysis of Earth data under cesium digital Earth engine, Proc. 2018 2nd Int. Conf. on Big Data and Internet of Things (BDIOT 2018), New York: Association for Computing Machinery, 2018, pp. 29–32. https://doi.org/10.1145/3289430.3289447

  77. Karau, H., Konwinski, A., Wendell, P., and Zaharia, M., Learning Spark: Lightning-Fast Big Data Analysis, Sebastopol: O’Reilly Media, 2015.

    Google Scholar 

  78. Katzfuss, M. and Cressie, N., Spatio-temporal smoothing and EM estimation for massive remote-sensing data sets, J. Time Ser. Anal., 2011, vol. 32, no. 4, pp. 430–446. https://doi.org/10.1111/j.1467-9892.2011.00732.x

    Article  Google Scholar 

  79. Kawasaki, A., Yamamoto, A., Koudelova, P., Acierto, R., Nemoto, T., Kitsuregawa, M., and Koike, T., Data Integration and Analysis System (DIAS) contributing to climate change analysis and disaster risk reduction, Data Sci. J., 2017, vol. 16, no. 41, pp. 1–17. https://doi.org/10.5334/dsj-2017-041

    Article  Google Scholar 

  80. Kempler, S. and Mathews, T., Earth Science Data Analytics: definitions, techniques and skills, Data Sci. J., 2017, vol. 16, no. 6, pp. 1–8. https://doi.org/10.5334/dsj-2017-006

    Article  Google Scholar 

  81. Kingdon, A., Nayembil, M.L., Richardson, A.E., and Smith, A.G., A geodata warehouse: Using denormalisation techniques as a tool for delivering spatially enabled integrated geological information to geologists, Comput. Geosci., 2016, vol. 96, pp. 87–97. https://doi.org/10.1016/j.cageo.2016.07.016

    Article  Google Scholar 

  82. Kleppmann, M., Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems, Sebastopol: O’Reilly Media, 2017.

    Google Scholar 

  83. Konecny, G., Geoinformation: Remote Sensing, Photogrammetry and Geographic Information Systems, 2nd ed., London: Taylor and Francis, 2014.

    Book  Google Scholar 

  84. Krasnoperov, R., Peregoudov, D., Lukianova, R., Soloviev, A., and Dzeboev, B., Early Soviet satellite magnetic field measurements in the years 1964 and 1970, Earth Syst. Sci. Data, 2020, vol. 12, no. 1, pp. 555–561. https://doi.org/10.5194/essd-12-555-2020

    Article  Google Scholar 

  85. Laney, D., 3D Data Management: Controlling Data Volume, Velocity, and Variety, Meta Group, 2001. https:// studylib.net/doc/8647594/3d-data-management–controlling-data-volume–velocity–an. Cited March 18, 2021.

  86. LeHong, H. and Fenn, J., Key Trends to Watch in Gartner 2012 Emerging Technologies Hype Cycle, Forbes, 2012. https://www.forbes.com/sites/gartnergroup/2012/09/18/ key-trends-to-watch-in-gartner-2012-emerging-technologies-hype-cycle-2. Cited March 18, 2021.

  87. Liu, J.G. and Mason, P.J., Essential Image Processing and GIS for Remote Sensing, Chichester: Wiley, 2013.

    Google Scholar 

  88. Liu, P., A survey of remote-sensing big data, Front. Environ. Sci., 2015, vol. 3, p. 45. https://doi.org/10.3389/fenvs.2015.00045

    Article  Google Scholar 

  89. Lohr, S., The origins of ‘big data’: an etymological detective story, The New York Times, 2013. https://bits.blogs.nytimes.com/2013/02/01/the-origins-of-big-data-an-etymological-detective-story/. Cited March 18, 2021.

  90. Lopez, M.M. and Kalita, J., Deep Learning applied to NLP. arXiv preprint, 2017. arXiv:1703.03091

  91. MacLachlan, C., Arribas, A., Peterson, K.A., et al., Global Seasonal forecast system version 5 (GloSea5): A high-resolution seasonal forecast system, Q. J. R. Meteorol. Soc., 2015, vol. 141, pp. 1072–1084. https://doi.org/10.1002/qj.2396

    Article  Google Scholar 

  92. Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., and Hung Byers, A., Big data: The next frontier for innovation, competition, and productivity, McKinsey Global Institute (MGI), 2011.

    Google Scholar 

  93. Maraun, D., Shepherd, T.G., Widmann, M., Zappa, G., Walton, D., Gutierrez, J.M., Hagemann, S., Richter, I., Soares, P.M.M., Hall, A., and Mearns, L.O., Towards processinformed bias correction of climate change simulations, Nat. Clim. Change, 2017, vol. 7, pp. 764–773. https://doi.org/10.1038/NCLIMATE3418

    Article  Google Scholar 

  94. Marr, B., Big Data: Using SMART Big Data, Analytics and Metrics to Make Better Decisions and Improve Performance, Chichester: Wiley, 2015.

    Google Scholar 

  95. Marz, N. and Warren, J., Big Data: Principles and Best Practices of Scalable Realtime Data Systems, New York: Manning, 2015.

    Google Scholar 

  96. Maus, S., Rother, M., Stolle, C., Mai, W., Choi, S., Lühr, H., Cooke, D., and Roth, C., Third generation of the Potsdam Magnetic Model of the Earth (POMME), Geochem. Geophys. Geosyst., 2006, vol. 7, no. 7. https://doi.org/10.1029/2006GC001269

  97. Maus, S., Lühr, H., Rother, M., Hemant, K., Balasis, G., Ritter, P., and Stolle, C., Fifth-generation lithospheric magnetic field model from CHAMP satellite measurements, Geochem. Geophys. Geosyst., 2007a, vol. 8, no. 5. https://doi.org/10.1029/2006GC001521

  98. Maus, S., Sazonova, T., Hemant, K., Fairhead, J.D., and Ravat, D., National Geophysical Data Center candidate for the world digital magnetic anomaly map, Geochem. Geophys. Geosyst., 2007b, vol. 8, no. 6, pp. 1–10. https://doi.org/10.1029/2007GC001643

    Article  Google Scholar 

  99. Mayer-Schönberger, V. and Cukier, K., Big Data. A Revolution That Will Transform How We Live, Work, and Think, Boston: Houghton Mifflin Harcourt, 2013.

    Google Scholar 

  100. Mayer-Schönberger, V. and Cukier, K., Big Data: A Revolution That Will Transform How We Live, Work, and Think, London: Eamon Dolan/Mariner Books, 2014.

    Google Scholar 

  101. McInerney, M., NASA Earthdata Cloud, 2020. https://ntrs.nasa.gov/api/citations/20200001222/downloads/20200001222.pdf?attachment=true. Cited March 18, 2021.

  102. Menvielle, M. and Berthelier, A., The K-derived planetary indices: Description and availability, Rev. Geophys., 1991, vol. 29, no. 3, pp. 415–432. https://doi.org/10.1029/91RG00994

    Article  Google Scholar 

  103. Menvielle, M., Iyemori, T., Marchaudon, A., and Nose, M., Geomagnetic indices, in Geomagnetic Observations and Models, Mandea, M. and Korte, M., Eds., IAGA Special Sopron Book Ser. vol. 5, Dordrecht: Springer, 2011, pp. 183–228. https://doi.org/10.1007/978-90-481-9858-0_7

  104. Miller, A.A., Climatology, London: Taylor and Francis, 2019.

    Book  Google Scholar 

  105. Mitas, L. and Mitasova, H., Spatial interpolation, in Geographical Information Systems: Principles, Techniques, Management and Applications, Longley, P., Goodchild, M.F., Maguire, D.J., and Rhind, D.W., Eds., New York: Wiley, 1999, pp. 481–492.

    Google Scholar 

  106. Moon, N.H., Shin, M.Y., Moon, G.H., and Chun, J., Trends and prospects of forest meteorological studies based on the publications in Korean Journal of Agricultural and Forest Meteorology, Korean J. Agric. For. Meteorol., 2019, vol. 21, no. 3, pp. 121–134. https://doi.org/10.5532/KJAFM.2019.21.3.121

    Article  Google Scholar 

  107. Moore, J.M., Nonlinear filtering techniques for noisy geophysical data: Using big data to predict the future, Proc. AGU Fall Meeting 2014, San Fransisco, 2014, Paper ID NG23A-3789.

  108. Nativi, S., Mazzetti, P., andCraglia, M., A view-based model of data-cube to support big Earth data systems interoperability, Big Earth Data, 2017, vol. 1, nos. 1–2, pp. 75–99. https://doi.org/10.1080/20964471.2017.1404232

    Article  Google Scholar 

  109. Newman, S.-F., Seismographic Data Compression – Applying Modified Tunstall Coding, Tacoma: Inst. Technol., Univ. of Washington, 2006.

    Google Scholar 

  110. NIST Big Data Interoperability Framework: vol. 1, Definitions. Version 3, NIST Special Publication 1500-1r2, 2019. https://doi.org/10.6028/NIST.SP.1500-1r2

  111. Novick, K.A., Biederman, J.A., Desai, A.R., Litvak, M.E., Moore, D.J.P., Scott, R.L., and Torn, M.S., The AmeriFlux network: A coalition of the willing, Agric. For. Meteorol., 2018, vol. 249, pp. 444–456. https://doi.org/10.1016/j.agrformet.2017.10.009

    Article  Google Scholar 

  112. Odum, E.P., Fundamentals of Ecology, 3rd ed., Philadelphia: Saunders, 1971.

    Google Scholar 

  113. Ofli, F., Meier, P., Imran, M., Castillo, C., Tuia, D., Rey, N., Briant, J., Millet, P., Reinhard, F., Parkan, M., and Joost, S., Combining human computing and machine learning to make sense of big (aerial) data for disaster response, Big Data, 2016, vol. 4, no. 1, pp. 47–59. https://doi.org/10.1089/big.2014.0064

    Article  Google Scholar 

  114. Olsen, N. and Stolle, C., Satellite geomagnetism, Annu. Rev. Earth Planet. Sci., 2012, vol. 40, pp. 441–465. https://doi.org/10.1146/annurev-earth-042711-105540

    Article  Google Scholar 

  115. Olsen, N., Lühr, H., Sabaka, T.J., Mandea, M., Rother, M., Tøffner-Clausen, L., and Choi, S., CHAOS—a model of the Earth’s magnetic field derived from CHAMP, Ørsted, and SAC-C magnetic satellite data, Geophys. J. Int., 2006, vol. 166, no. 1, pp. 67–75. https://doi.org/10.1111/j.1365-246X.2006.02959.x

    Article  Google Scholar 

  116. Onay, C. and Öztürk, E., A review of credit scoring research in the age of Big Data, J. Financ. Regul. Compliance, 2018, vol. 26, no. 3, pp. 382–405. https://doi.org/10.1108/JFRC-06-2017-0054

    Article  Google Scholar 

  117. O’Neil, C., Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, New York: Crown, 2016.

    Google Scholar 

  118. Pavlikov, V.V., Ruzhentsev, N.V., Sobkolov, A.D., Salnikov, D.S., and Tsopa, A.I., Ground-based radiometric complex of millimeter wave band for meteorology and telecommunications applications, Telecommun. Radio Eng., 2017, vol. 76, no. 16, pp. 1477–1488. https://doi.org/10.1615/TelecomRadEng.v76.i16.70

    Article  Google Scholar 

  119. Pei, T., Song, C., Guo, S., et al., Big geodata mining: Objective, connotations and research issues, J. Geogr. Sci., 2020, vol. 30, no. 2, pp. 251–266. https://doi.org/10.1007/s11442-020-1726-7

    Article  Google Scholar 

  120. Pelton, J.N., Madry, S., and Camacho-Lara, S., Handbook of Satellite Applications, New York: Springer, 2013. https://doi.org/10.1007/978-1-4419-7671-0

    Book  Google Scholar 

  121. People and Pixels: Linking Remote Sensing and Social Science, National Research Council, Washington: National Academies Press, 1998. .https://doi.org/10.17226/5963

  122. Poblet, M., García-Cuesta, E., and Casanovas, P., Crowdsourcing tools for disaster management: A review of platforms and methods, in AI Approaches to the Complexity of Legal Systems, Casanovas, P., Pagallo, U., Palmirani, M., and Sartor, G., Eds., Berlin: Springer, 2014, pp. 261–274.

    Google Scholar 

  123. Potapov, P.V., Turubanova, S.A., Tyukavina, A., Krylov, A.M., McCarty, J.L., Radeloff, V.C., and Hansen, M.C., Eastern Europe’s forest cover dynamics from 1985 to 2012 quantified from the full Landsat archive, Remote Sens. Environ., 2015, vol., 159, pp. 28–43. https://doi.org/10.1016/j.rse.2014.11.027

    Article  Google Scholar 

  124. Preimesberger, C., Hadoop, Yahoo, 'Big Data’ brighten BI future, 2011. https://www.webcitation.org/67j1TSO8N?url= http://www.eweek.com/c/a/Data-Storage/TBA-Hadoop-Yahoo-Big-Data-Brightens-BI-Future-254079/. Cited March 18, 2021.

  125. Pyo, J., Duan, H., Baek, S., Kim, M.S., Jeon, T., Kwon, Y.S., Lee, H., and Cho, K.H., A convolutional neural network regression for quantifying cyanobacteria using hyperspectral imagery, Remote Sens. Environ., 2019, vol. 233, Paper ID 111350. https://doi.org/10.1016/j.rse.2019.111350

  126. Rasson, J. Observatories, instrumentation, in Encyclopedia of Geomagnetism and Paleomagnetism, Gubbins, D. and Herrero-Bervera, E., Eds., Dordrecht: Springer, 2007, pp. 711–713.

    Google Scholar 

  127. Rauber, R.M. and Nesbitt, S.W., Radar Meteorology: A First Course, New York: Wiley, 2018.

    Book  Google Scholar 

  128. Reay, S.J., Herzog, D.C., Alex, S., Kharin, E., McLean, S., Nosé, M., and Sergeeva, N., Magnetic observatory data and metadata: types and availability, in Geomagnetic Observations and Models, Mandea, M. and Korte, M., Eds., IAGA Special Sopron Book Ser., vol. 5, 2011, pp. 149–181. https://doi.org/10.1007/978-90-481-9858-0_7

    Book  Google Scholar 

  129. Rezai, A., Keshavarzi, P., and Moravej, Z., Key management issue in SCADA networks: a review, Eng. Sci. Technol., Int. J., 2017, vol. 20, no. 1, pp. 354–363. https://doi.org/10.1016/j.jestch.2016.08.011

    Article  Google Scholar 

  130. Richards, P.G. and Zavales, J., Seismological methods for monitoring a CTBT: The technical issues arising in early negotiations, in Monitoring a Comprehensive Test Ban Treaty, Husebye, E.S. and Dainty, A.M., Eds., NATO ASI Ser. E, vol. 303, 1996, pp. 53–81.

    Google Scholar 

  131. Roberts, P.H. and King, E.M., On the genesis of the Earth’s magnetism, Rep. Prog. Phys., 2013, vol. 76, no. 9, Paper ID 096801. https://doi.org/10.1088/0034-4885/76/9/096801

  132. Roden, R., Seismic interpretation in the age of big data, Proc. Conf.: SEG Technical Program Expanded Abstracts 2016, Dallas: SEG, 2016, pp. 163–165. https://doi.org/10.1190/segam2016-13612308.1

  133. Rodriguez-Galiano, V.F. and Chica-Rivas, M., Evaluation of different machine learning methods for land cover mapping of a Mediterranean area using multi-seasonal Landsat images and Digital Terrain Models, Int. J. Digital Earth, 2014, vol. 7, no. 6, pp. 492–509. https://doi.org/10.1080/17538947.2012.748848

    Article  Google Scholar 

  134. Salimi, S. and Hammad, A., Sensitivity analysis of probabilistic occupancy prediction model using big data, Build. Environ., 2020, vol. 172, Paper ID 106729. https://doi.org/10.1016/j.buildenv.2020.106729

  135. Schultz, M.G., Betancourt, C., Gong, B., Kleinert, F., Langguth, M., Leufen, L.H., Mozaffari, A., and Stadtler, S., Can deep learning beat numerical weather prediction?, Philos. Trans. R. Soc., A, 2021, vol. 379, no. 2194, Paper ID 20200097. https://doi.org/10.1098/rsta.2020.0097

  136. SMEAR, 2021. https://www.atm.helsinki.fi/SMEAR/. Cited March 18, 2021.

  137. Snijders, C., Matzat, U., and Reips, U.-D., “Big Data”: big gaps of knowledge in the field of internet science, Int. J. Internet Sci., 2012, vol. 7, no. 1, pp. 1–5.

    Google Scholar 

  138. Song, J., Gao, S., Zhu, Y., and Ma, C., A survey of remote sensing image classification based on CNNs, Big Earth Data, 2019, vol. 3, no. 3, pp. 232–254. https://doi.org/10.1080/20964471.2019.1657720

    Article  Google Scholar 

  139. Standard for the Exchange of Earthquake Data (SEED) Reference Manual: SEED Format Version 2.4. August, 2012, International Federation of Digital Seismograph Networks, Incorporated Research Institutions for Seismology (IRIS), U. S. Geological Survey, 2012.

  140. Stein, A., van der Meer, F.D., Gorte, B., Spatial Statistics for Remote Sensing, Dordrecht: Springer, 2002. https://doi.org/10.1007/0-306-47647-9

    Book  Google Scholar 

  141. St-Louis, B.J., Sauter, E.A., Trigg, D.F., et al., INTERMAGNET Technical Reference Manual. Version 4.6, Edinburgh: INTERMAGNET, 2012.

    Google Scholar 

  142. Stromann, O., Nascetti, A., Yousif, O., and Ban,Y., Dimensionality reduction and feature selection for object-based land cover classification based on Sentinel-1 and Sentinel-2 time series using Google Earth Engine, Remote Sens., 2020, vol. 12, no. 1, article 76. https://doi.org/10.3390/rs12010076

    Article  Google Scholar 

  143. Su, Z., Timmermans, W., Zeng, Y., et al., An overview of European efforts in generating climate data records, Bull. Am. Meteorol. Soc., 2018, vol. 99, pp. 349–359. https://doi.org/10.1175/BAMS-D-16-0074.1

    Article  Google Scholar 

  144. SuperMAG: Download Data. 2021. https://supermag.jhuapl.edu/mag/?fidelity=low&tab=customdownload. Cited March 18, 2021.

  145. Talwani, M. and Kessinger, W., Exploration geophysics, in Encyclopedia of Physical Science and Technology, 3rd ed., San Diego: Academic Press, 2003, pp. 709–726. https://doi.org/10.1016/B0-12-227410-5/00238-6

    Book  Google Scholar 

  146. Tao, R., Gong, Z., Ma, Q., and Thill, J.-C., Boosting computational effectiveness in big spatial flow data analysis with intelligent data reduction, ISPRS Int. J. Geo-Information, 2020, vol. 9, no. 5, article 299. https://doi.org/10.3390/ijgi9050299

    Article  Google Scholar 

  147. Teillet, P.M., Image correction for radiometric effects in remote sensing, Int. J. Remote Sens., 1986, vol. 7, no. 12, pp. 1637–1651. https://doi.org/10.1080/01431168608948958

    Article  Google Scholar 

  148. Thébault, E., Finlay, C.C., Beggan, C.D., et al., International Geomagnetic Reference Field: the 12th generation, Earth, Planets Space, 2015, vol. 67, article 79. https://doi.org/10.1186/s40623-015-0228-9

    Article  Google Scholar 

  149. Thomas, J.W. and Hoover, G.M., Exploration seismology, in Encyclopedia of Geology, 2nd ed., vol. 1, Amsterdam: Elsevier Acad. Press, 2021, pp. 656–663. https://doi.org/10.1016/B978-0-12-409548-9.12538-2

    Book  Google Scholar 

  150. Thorne, P.W., Allan, R.J., Ashcroft, L., et al., Toward an integrated set of surface meteorological observations for climate science and applications, Bull. Am. Meteorol. Soc., 2017, vol. 98, pp. 2689–2702. https://doi.org/10.1175/BAMS-D-16-0165.1

    Article  Google Scholar 

  151. Toth, C. and Jóźków, G., Remote sensing platforms and sensors: a survey, ISPRS Int. J. Photogramm. Remote Sens., 2016, vol. 115, pp. 22–36. https://doi.org/10.1016/j.isprsjprs.2015.10.004

    Article  Google Scholar 

  152. van Allen, J.A. and Frank, L.A., Radiation around the Earth to a radial distance of 107,400 km, Nature, 1959, vol. 183, pp. 430–434.

    Article  Google Scholar 

  153. van Allen, J.A., Ludwig, G.H., Ray, E.C., and McIlwain, C.E., Observation of high intensity radiation by satellites 1958 Alpha and Gamma, U.S. National Academy of Sciences, I.G.Y. Satellite Report Series 3, 1958, pp. 73–92. https://doi.org/10.2514/8.7396

  154. Vihma, T., Uotila, P., Sandven, S., et al., Towards an advanced observation system for the marine Arctic in the framework of the Pan-Eurasian Experiment (PEEX), Atmos. Chem. Phys., 2019, vol.19, no. 3, pp. 1941–1970. https://doi.org/10.5194/acp-19-1941-2019

    Article  Google Scholar 

  155. Voulodimos, A., Doulamis, N., Doulamis, A., and Protopapadakis, E., Deep learning for computer vision: a brief review, Comput. Intell. Neurosci., 2018, vol. 2018, Paper ID 7068349. https://doi.org/10.1155/2018/7068349

  156. Watson, J.C., Establishing evidence for internal structure using exploratory factor analysis, Meas. Eval. Couns. Dev., 2017, vol. 50, no. 4, pp. 232–238. https://doi.org/10.1080/07481756.2017.1336931

    Article  Google Scholar 

  157. Weyn, J.A., Durran, D.R., and Caruana, R., Can machines learn to predict weather? Using deep learning to predict gridded 500-hPa geopotential height from historical weather data, J. Adv. Model. Earth Syst., 2019, vol. 11, no. 8, pp. 2680–2693. https://doi.org/10.1029/2019MS001705

    Article  Google Scholar 

  158. Whaler, K., Geomagnetism in the satellite era, Astron. Geophys., 2007, vol. 48, no. 2, pp. 23–29.

    Article  Google Scholar 

  159. Wu, H. and Li, Z.-L., Scale issues in remote sensing: a review on analysis, processing and modeling, Sensors, 2009, vol. 9, pp. 1768–1793. https://doi.org/10.3390/s90301768

    Article  Google Scholar 

  160. Wu, S.R., Li, X., Apul, D., Breeze, V., Tang, Y., Fan, Y., and Chen, J., Agent-based modeling of temporal and spatial dynamics in life cycle sustainability assessment, J. Ind. Ecol., 2017, vol. 21, no. 6, pp. 1507–1521. https://doi.org/10.1111/jiec.12666

    Article  Google Scholar 

  161. Zhang, S., Yao, L., Sun, A., and Tay, Y., Deep learning based recommender system: a survey and new perspective, ACM Comput. Surv., 2019, vol. 52, no. 1, pp. 1–38. https://doi.org/10.1145/3285029

    Article  Google Scholar 

  162. Zhao, S., Wang, Q., Li, Y., Liu, S., Wang, Z., Zhu, L., and Wang, Z., An overview of satellite remote sensing technology used in China’s environmental protection, Earth Sci. Inf., 2017, vol. 10, pp. 137–148. https://doi.org/10.1007/s12145-017-0286-6

    Article  Google Scholar 

  163. Zilitinkevich, S.S., Elperin, T., Kleeorin, N., Rogachevskii, I., and Esau, I., A hierarchy of energy- and flux-budget (EFB) turbulence closure models for stably-stratified geophysical flows, Boundary-Layer Meteorol., 2013, vol. 146, no. 3, pp. 341–373. https://doi.org/10.1007/s10546-012-9768-8

    Article  Google Scholar 

Download references

ACKNOWLEDGMENTS

We acknowledge the use of data and services provided by the Data Sharing Core Facility “Analytical Geomagnetic Data Center” of the Geophysical Center of the Russian Academy of Sciences http://ckp.gcras.ru/.

We are grateful to Yuri Mikhailovich Baturin, Hero of the Russian Federation, Corresponding Member of the Russian Academy of Sciences, for providing the photographs from his personal archive.

Funding

The study was supported by the Russian Foundation for Basic Research under project no. 20-15-50125.

Author information

Authors and Affiliations

  1. Geophysical Center, Russian Academy of Sciences, 119296, Moscow, Russia

    A. D. Gvishiani, M. N. Dobrovolsky, B. V. Dzeranov & B. A. Dzeboev

  2. Schmidt Institute of Physics of the Earth, Russian Academy of Sciences, 123242, Moscow, Russia

    A. D. Gvishiani

  3. Geophysical Institute, Vladikavkaz Scientific Center, Russian Academy of Sciences, 362002, Vladikavkaz, Russia

    B. V. Dzeranov & B. A. Dzeboev

Authors
  1. A. D. Gvishiani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. N. Dobrovolsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. B. V. Dzeranov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. B. A. Dzeboev
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to B. A. Dzeboev.

Ethics declarations

The authors declare that they have no conflicts of interest.

Additional information

Translated by M. Nazarenko

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gvishiani, A.D., Dobrovolsky, M.N., Dzeranov, B.V. et al. Big Data in Geophysics and Other Earth Sciences. Izv., Phys. Solid Earth 58, 1–29 (2022). https://doi.org/10.1134/S1069351322010037

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1069351322010037

Keywords:

Search

Navigation