Skip to main content
SpringerLink
Log in

Scalable data processing and visualization service of Sentinel 5P for Earth Observations Data Cubes

  • Original Paper
  • Published:
Arabian Journal of Geosciences Aims and scope Submit manuscript

Abstract

Air pollution significantly affects human health and the environment. It is caused by the emission of diverse pollutants into the atmosphere. Most of the measurements for air quality are done through on-ground sensors that are single points and do not cover well a given territory. Currently, there are space missions that are aiming to complement these on-the-ground systems with more synoptic views. The management and processing of remote sensing and on-ground sensor data have become increasingly complex, requiring temporal and spatial analysis to extract meaningful information and identify patterns and trends of vast amounts of data from various sources. Data Cubes framework helps overcome processing challenges, effectively managing and analyzing large volumes of data in multiple dimensions, such as spatial, spectral, and temporal. The article presents a scalable EO processing and visualization service designed for Data Cubes to explore and analyze multidimensional array data obtained from the Sentinel 5P satellite by performing shared-memory parallel simulations. The service provides a comprehensive understanding of the studied region over a specific period through statistical analysis and visualization. A case study was conducted over the territory of Armenia from September 2018 to August 2019 to evaluate the performance and capabilities of the service.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

All data generated or analyzed during these studies are included in this present article.

References

  • Asmaryan S, Muradyan V, Tepanosyan G, Hovsepyan A, Saghatelyan A, Astsatryan H, Grigoryan H, Abrahamyan R, Guigoz Y, Giuliani G (2019) Paving the way towards an Armenian data cube. Data 4(3):117

    Article  Google Scholar 

  • Astsatryan H, Shoukourian Y, Vladimir S (2004) The armcluster project: brief introduction. In: Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications. pp 1291–1295

  • Astsatryan H, Hayrapetyan A, Narsisian W, Asmaryan S, Saghatelyan A, Muradyan V, Giuliani G, Guigoz Y, Ray N (2015a) An interoperable cloud-based scientific gateway for NDVI time series analysis. Comp Stand Inter 41:79–84

  • Astsatryan H, Hayrapetyan A, Narsisian W, Saribekyan A, Asmaryan S, Saghatelyan A, Muradyan V, Guigoz Y, Giuliani G, Ray N (2015b) An interoperable web portal for parallel geoprocessing of satellite image vegetation indices. Earth Sci Inform 8(2):453–460

  • Astsatryan H, Grigoryan H, Poghosyan A, Abrahamyan R, Asmaryan S, Muradyan V, Tepanosyan G, Guigoz Y, Giuliani G (2021a) Air temperature forecasting using artificial neural network for Ararat Valley. Earth Sci Inform 14(2):711–722

  • Astsatryan H, Narsisian W, Mirzoyan A, Sahakyan V (2021b) Research cloud computing ecosystem in Armenia. In: CEUR Workshop Proceedings. pp 117–121

  • Astsatryan H, Lalayan A, Giuliani G (2023) Scalable data processing platform for earth observation data repositories. Scalable Computing: Practice and Experience 24:35–44

    Google Scholar 

  • Baumann P, Mazzetti P, Ungar J, Barbera R, Barboni D, Beccati A, Bigagli L, Boldrini E, Bruno R, Calanducci A et al (2016) Big data analytics for earth sciences: the EarthServer approach. Int J Digit Earth 9(1):3–29

    Article  Google Scholar 

  • Brunekreef B, Holgate ST (2002) Air pollution and health. LANCET 360(9341):1233–1242

    Article  Google Scholar 

  • Dwyer JL, Roy DP, Sauer B, Jenkerson CB, Zhang HK, Lymburner L (2018) Analysis ready data: enabling analysis of the Landsat archive. Remote Sens-Basel 10(9):1363

    Article  Google Scholar 

  • Fan J, Yan J, Ma Y, Wang L (2017) Big data integration in remote sensing across a distributed metadata-based spatial infrastructure. Remote Sens-Basel 10(1):7

    Article  Google Scholar 

  • Giuliani G, Chatenoux B, Honeck E, Richard JP (2018) Towards sentinel-2 analysis ready data: a swiss data cube perspective. In: International Geoscience and Remote Sensing. IEEE, pp 8659–8662

  • Griffin D, Zhao X, McLinden CA, Boersma F, Bourassa A, Dammers E, Degenstein D, Eskes H, Fehr L, Fioletov V et al (2019) High-resolution mapping of nitrogen dioxide with TROPOMI: first results and validation over the Canadian oil sands. Geophys Res Lett 46(2):1049–1060

    Article  Google Scholar 

  • Hu H, Landgraf J, Detmers R, Borsdorff T, Aan de Brugh J, Aben I, Butz A, Hasekamp O (2018) Toward global mapping of methane with TROPOMI: first results and intersatellite comparison to GOSAT. Geophys Res Lett 45(8):3682–3689

    Article  Google Scholar 

  • Hunter JD (2007) Matplotlib: a 2D graphics environment. Comput Sci Eng 9(03):90–95

    Article  Google Scholar 

  • Ialongo I, Virta H, Eskes H, Hovila J, Douros J (2020) Comparison of TROPOMI/Sentinel-5 precursor NO\({}_{2}\) observations with ground-based measurements in Helsinki. Atmos Meas Tech 13(1):205–218

    Article  Google Scholar 

  • Katherine A, Ryan B, Sonntag W, Kavvada A, Friedl L (2017) Earth observation in service of the 2030 agenda for sustainable development. Geo-Spat Inf Sci 20(2):77–96

    Article  Google Scholar 

  • Kluyver T, Ragan-Kelley B, Pérez F, Granger BE, Bussonnier M, Frederic J, Kelley K, Hamrick JB, Grout J, Corlay S et al (2016) Jupyter Notebooks-a publishing format for reproducible computational workflows, vol 2016. IOS Press

  • Li J, Carlson BE, Lacis AA (2009) A study on the temporal and spatial variability of absorbing aerosols using total ozone mapping spectrometer and ozone monitoring instrument aerosol index data. J Geophys Res-Atmos 114(D9)

  • Loyola DG, Gimeno García S, Lutz R, Argyrouli A, Romahn F, Spurr RJ, Pedergnana M, Doicu A, Molina García V, Schüssler O (2018) The operational cloud retrieval algorithms from TROPOMI on board Sentinel-5 precursor. Atmos Meas Tech 11(1):409–427

  • World Health Organization et al (2003) Health aspects of air pollution with particulate matter, ozone and nitrogen dioxide: report on a who working group, Bonn, Germany 13–15 January 2003. Tech. rep., Copenhagen: WHO Regional Office for Europe

  • Rafaj P, Kiesewetter G, Gül T, Schöpp W, Cofala J, Klimont Z, Purohit P, Heyes C, Amann M, Borken-Kleefeld J et al (2018) Outlook for clean air in the context of sustainable development goals. Global Environ Chang 53:1–11

    Article  Google Scholar 

  • Rivest S (2001) Toward better support for spatial decision making: defining the characteristics of spatial on-line analytical processing (SOLAP). Geomatica 55(4):539–555

    Google Scholar 

  • Rocklin M (2015) Dask: parallel computation with blocked algorithms and task scheduling. In: Proceedings of the 14th Python in Science Conference, Citeseer, vol 130. p 136

  • Shoukourian YH, Sahakyan VG, Astsatryan HV (2013) E-infrastructures in Armenia: virtual research environments. In: Ninth International Conference on Computer Science and Information Technologies Revised Selected Papers. IEEE, pp 1–7

  • Sillman S, Logan JA, Wofsy SC (1990) The sensitivity of ozone to nitrogen oxides and hydrocarbons in regional ozone episodes. J Geophys Res-Atmos 95(D2):1837–1851

    Article  Google Scholar 

  • Veefkind J, Aben I, McMullan K, Förster H, De Vries J, Otter G, Claas J, Eskes H, De Haan J, Kleipool Q et al (2012) TROPOMI on the ESA sentinel-5 precursor: a GMES mission for global observations of the atmospheric composition for climate, air quality and ozone layer applications. Remote Sens Environ 120:70–83

  • Yijiang Z (2012) The conceptual design on spatial data cube. In: 2012 2nd International Conference on Consumer Electronics, Communications and Networks (CECNet). IEEE, pp 645–648

  • Yue P, Zhang M, Tan Z (2015) A geoprocessing workflow system for environmental monitoring and integrated modelling. Environ Modell Softw 69:128–140

    Article  Google Scholar 

  • Zheng Z, Yang Z, Wu Z, Marinello F (2019) Spatial variation of NO\({}_2\) and its impact factors in China: an application of Sentinel-5P products. Remote Sens-Basel 11(16):1939

Download references

Acknowledgements

The research was supported by the Science Committee of the Republic of Armenia by the project entitled “Scalable data processing platform for EO data repositories” (Nr. 22AA-1B015) and the University of Geneva Leading House and the State Committee of Science of the Republic of Armenia by the projects entitled “ADC4SD: Armenian Data Cube for Sustainable Development,” “Self-organized Swarm of UAVs Smart Cloud Platform Equipped with Multi-agent Algorithms and Systems” (Nr. 21AG-1B052), and “Remote sensing data processing methods using neural networks and deep learning to predict changes in weather phenomena” (Nr. 21SC-BRFFR-1B009).

Author information

Authors and Affiliations

  1. Institute for Informatics and Automation Problems of NAS RA, Yerevan, 0014, Armenia

    Hrachya Astsatryan, Hayk Grigoryan, Rita Abrahamyan & Arthur Lalayan

  2. Center for Ecological-Noosphere Studies of NAS RA, Yerevan, 0025, Armenia

    Shushanik Asmaryan

  3. Institute for Environmental Sciences, University of Geneva, Geneva, 1205, Châtelaine, Switzerland

    Gregory Giuliani & Yaniss Guigoz

Authors
  1. Hrachya Astsatryan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hayk Grigoryan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rita Abrahamyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Arthur Lalayan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shushanik Asmaryan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Gregory Giuliani
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yaniss Guigoz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hrachya Astsatryan.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Responsible Editor: Biswajeet Pradhan.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Astsatryan, H., Grigoryan, H., Abrahamyan, R. et al. Scalable data processing and visualization service of Sentinel 5P for Earth Observations Data Cubes. Arab J Geosci 16, 618 (2023). https://doi.org/10.1007/s12517-023-11672-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12517-023-11672-y

Keywords

Search

Navigation