Skip to main content
SpringerLink
Log in

Bayesian Latent Variable Co-kriging Model in Remote Sensing for Quality Flagged Observations

  • Published:
Journal of Agricultural, Biological and Environmental Statistics Aims and scope Submit manuscript

Abstract

Remote sensing data products often include quality flags that inform users whether the associated observations are of good, acceptable or unreliable qualities. However, such information on data fidelity is not consistently considered in remote sensing data analyses. Motivated by observations from the atmospheric infrared sounder (AIRS) instrument on board NASA’s Aqua satellite, we propose a latent variable co-kriging model with separable Gaussian processes to analyze large quality-flagged remote sensing data sets together with their associated quality information. We augment the posterior distribution by an imputation mechanism to decompose large covariance matrices into separate computationally efficient components taking advantage of their input structure. Within the augmented posterior, we develop a Markov chain Monte Carlo (MCMC) procedure that mostly consists of direct simulations from conditional distributions. In addition, we propose a computationally efficient recursive prediction procedure. We apply the proposed method to air temperature data from the AIRS instrument. We show that incorporating quality flag information in our proposed model substantially improves the prediction performance compared to models that do not account for quality flags.

Supplementary materials accompanying this paper appear online.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Banerjee S (2017) High-dimensional Bayesian geostatistics. Bayesian Anal 12:583–614. https://doi.org/10.1214/17-BA1056R

    Article  MathSciNet  MATH  Google Scholar 

  • Banerjee S, Carlin BP, Gelfand AE (2014) Hierarchical modeling and analysis for spatial data, 2nd edn. Chapman and Hall/CRC, New York

    MATH  Google Scholar 

  • Banerjee S, Gelfand AE, Finley AO, Sang H (2008) Gaussian predictive process models for large spatial data sets. J R Stat Soc Ser B (Stat Methodol) 70:825–848

    MathSciNet  MATH  Google Scholar 

  • Behrangi A, Fetzer EJ, Granger SL (2016) Early detection of drought onset using near surface temperature and humidity observed from space. Int J Remote Sens 37:3911–3923

    Google Scholar 

  • Bilionis I, Zabaras N, Konomi BA, Lin G (2013) Multi-output separable Gaussian process: towards an efficient, fully Bayesian paradigm for uncertainty quantification. J Comput Phys 241:212–239

    MathSciNet  MATH  Google Scholar 

  • Braverman A, Hobbs J, Teixeira J, Gunson M (2021) Post hoc uncertainty quantification for remote sensing observing systems. SIAM/ASA J Uncert Quant 9:1064–1093. https://doi.org/10.1137/19M1304283

    Article  MathSciNet  MATH  Google Scholar 

  • Cheng S, Konomi BA, Matthews JL, Karagiannis G, Kang EL (2021) Hierarchical Bayesian nearest neighbor co-kriging gaussian process models; an application to intersatellite calibration. Spat Stat 44:100516

    MathSciNet  Google Scholar 

  • Cohen WB, Maiersperger TK, Yang Z, Gower ST, Turner DP, Ritts WD, Berterretche M, Running SW (2003) Comparisons of land cover and LAI estimates derived from ETM+ and MODIS for four sites in North America: a quality assessment of 2000/2001 provisional MODIS products. Remote Sens Environ 88:233–25. https://www.sciencedirect.com/science/article/pii/S0034425703001810

  • Cressie N (1993) Statistics for spatial data. Wiley, New York

    MATH  Google Scholar 

  • Cressie N, Johannesson G (2008) Fixed rank kriging for very large spatial data sets. J R Stat Soc Ser B (Stat Methodol) 70:209–226

    MathSciNet  MATH  Google Scholar 

  • Datta A, Banerjee S, Finley AO, Gelfand AE (2016) Hierarchical nearest-neighbor Gaussian process models for large geostatistical datasets. J Am Stat Assoc 111:800–812

    MathSciNet  Google Scholar 

  • Genton MG (2007) Separable approximations of space-time covariance matrices. Environmetrics 18:681–695

    MathSciNet  Google Scholar 

  • Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. J Am Stat Assoc 102:359–378

    MathSciNet  MATH  Google Scholar 

  • Gramacy RB (2020) surrogates: Gaussian process modeling, design and optimization for the applied sciences. Chapman Hall/CRC, Boca Raton. http://bobby.gramacy.com/surrogates/

  • Gramacy RB, Apley DW (2015) Local Gaussian process approximation for large computer experiments. J Comput Gr Stat 24:561–578

    MathSciNet  Google Scholar 

  • Guillas S, Sarri A, Day SJ, Liu X, Dias F (2018) Functional emulation of high resolution tsunami modelling over Cascadia. Ann Appl Stat 12:2023–2053. https://doi.org/10.1214/18-AOAS1142

    Article  MathSciNet  MATH  Google Scholar 

  • Heaton MJ, Datta A, Finley A, Furrer R, Guhaniyogi R, Gerber F, Gramacy RB, Hammerling D, Katzfuss M, Lindgren F, Nychka D, Sun F, Zammit-Mangion A (2019) A case study competition among methods for analyzing large spatial data. J Agric Biol Environ Stat 24:398–425

    MathSciNet  MATH  Google Scholar 

  • Hobbs J, Braverman A, Cressie N, Granat R, Gunson M (2017) Simulation-based uncertainty quantification for estimating atmospheric CO\(_2\) from satellite data. SIAM/ASA J Uncert Quant 5:956–985

    MathSciNet  MATH  Google Scholar 

  • Kahn BH, Irion FW, Dang VT, Manning EM, Nasiri SL, Naud CM, Blaisdell JM, Schreier MM, Yue Q, Bowman KW, Fetzer EJ, Hulley GC, Liou KN, Lubin D, Ou SC, Susskind J, Takano Y, Tian B, Worden JR (2014) The atmospheric infrared sounder version 6 cloud products. Atmos Chem Phys 14:399–426

    Google Scholar 

  • Kalmus P, Wong S, Teixeira J (2015) The pacific subtropical cloud transition: A MAGIC assessment of AIRS and ECMWF thermodynamic structure. IEEE Geosci Remote Sens Lett 12:1586–1590

    Google Scholar 

  • Katzfuss M (2017) A multi-resolution approximation for massive spatial datasets. J Am Stat Assoc 112:201–214

    MathSciNet  Google Scholar 

  • Katzfuss M, Guinness J (2021) A general framework for Vecchia approximations of Gaussian processes. Stat Sci 36:124–141

    MathSciNet  MATH  Google Scholar 

  • Kennedy MC, O’Hagan A (2000) Predicting the output from a complex computer code when fast approximations are available. Biometrika 87:1–13

    MathSciNet  MATH  Google Scholar 

  • Konomi BA, Karagiannis G (2021) Bayesian analysis of multifidelity computer models with local features and nonnested experimental designs: application to the WRF model. Technometrics 63:510–522

    MathSciNet  Google Scholar 

  • Konomi BA, Sang H, Mallick BK (2014) Adaptive Bayesian nonstationary modeling for large spatial datasets using covariance approximations. J Comput Graph Stat 23:802–829

    MathSciNet  Google Scholar 

  • Lindgren F, Rue H, Lindström J (2011) An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach. J R Stat Soc Ser B (Stat Methodol) 73:423–498

    MathSciNet  MATH  Google Scholar 

  • Lu Y, Zhang F (2019) Toward ensemble assimilation of hyperspectral satellite observations with data compression and dimension reduction using principal component analysis. Mon Weather Rev 147:3505–3518

    Google Scholar 

  • Ma P, Kang EL (2020) A fused gaussian process model for very large spatial data. J Comput Graph Stat 29:479–489

    MathSciNet  MATH  Google Scholar 

  • Ma P, Kang EL (2020) Spatio-Temporal data fusion for massive sea surface temperature data from MODIS and AMSR-E instruments. Environmetrics 31:e2594 (E2594 env.2594)

    MathSciNet  Google Scholar 

  • Ma P, Karagiannis G, Konomi BA, Asher TG, Toro GR, Cox AT (2022) Multifidelity computer model emulation with high-dimensional output: an application to storm surge. J R Stat Soc Ser C (Appl Stat) 71:861–883

    MathSciNet  Google Scholar 

  • Ma P, Konomi BA, Kang EL (2019) An additive approximate Gaussian process model for large spatio-temporal data. Environmetrics 30:e2569

    MathSciNet  Google Scholar 

  • Ma P, Mondal A, Konomi BA, Hobbs J, Song JJ, Kang EL (2021) Computer model emulation with high-dimensional functional output in large-scale observing system uncertainty experiments. Technometrics 0:1–15

    Google Scholar 

  • Ming D, Williamson D, Guillas S (2021). Deep Gaussian process emulation using stochastic imputation

  • Nychka D, Bandyopadhyay S, Hammerling D, Lindgren F, Sain S (2015) A multiresolution Gaussian process model for the analysis of large spatial datasets. J Comput Graph Stat 24:579–599

    MathSciNet  Google Scholar 

  • Paciorek C, Schervish M (2006) Spatial modelling using a new class of nonstationary covariance functions. Environmetrics 17:483–506

    MathSciNet  Google Scholar 

  • Perdikaris P, Raissi M, Damianou A, Lawrence N, Karniadakis GE (2017) Nonlinear information fusion algorithms for data-efficient multi-fidelity modelling. Proc R Soc A Math Phys Eng Sci 473:20160751

    MATH  Google Scholar 

  • Peruzzi M, Banerjee S, Finley AO (2020) Highly scalable Bayesian geostatistical modeling via meshed gaussian processes on partitioned domains. J Am Stat Assoc 0:1–14

    MATH  Google Scholar 

  • Prata F, Lynch M (2019) Passive earth observations of volcanic clouds in the atmosphere. Atmosphere 10:199

    Google Scholar 

  • Qian PZ, Wu CJ (2008) Bayesian hierarchical modeling for integrating low-accuracy and high-accuracy experiments. Technometrics 50:192–204

    MathSciNet  Google Scholar 

  • Roberts GO, Rosenthal JS et al (2004) General state space Markov chains and MCMC algorithms. Probab Surv 1:20–71

    MathSciNet  MATH  Google Scholar 

  • Rougier J (2008) Efficient emulators for multivariate deterministic functions. J Comput Graph Stat 17:827–843

    MathSciNet  Google Scholar 

  • Sang H, Huang JZ (2012) A full scale approximation of covariance functions for large spatial data sets. J R Stat Soc Ser B (Stat Methodol) 74:111–132

    MathSciNet  MATH  Google Scholar 

  • Stein ML (1999) Interpolation of spatial data: some theory for kriging, 2nd edn. Springer, New York

    MATH  Google Scholar 

  • Stein ML, Chi Z, Welty LJ (2004) Approximating likelihoods for large spatial data sets. J R Stat Soc Ser B (Stat Methodol) 66:275–296

    MathSciNet  MATH  Google Scholar 

  • Susskind J, Barnet CD, Blaisdell JM (2003) Retrieval of atmospheric and surface parameters from AIRS/AMSU/HSB data in the presence of clouds. IEEE Trans Geosci Remote Sens 41:390–409

    Google Scholar 

  • Thrastarson HT, Manning E, Kahn B, Fetzer EJ, Yue Q, Wong S, Kalmus P, Payne V, Olsen ET, Wilson RC, Blaisdell J, Iredell L, Susskind J, Warner J, Cady-Pereira K (2021) AIRS/AMSU/HSB version 7 Level 2 product user guide. https://docserver.gesdisc.eosdis.nasa.gov/public/project/AIRS/V7_L2_Product_User_Guide.pdf

  • Tian B, Manning E, Roman J, Thrastarson H, Fetzer E, Monarrez R (2020) AIRS version 7 Level 3 product user guide. https://docserver.gesdisc.eosdis.nasa.gov/public/project/AIRS/V7_L3_Product_User_Guide.pdf

  • Turmon M, Braverman A (2019) Uncertainty quantification for JPL retrievals. Technical report, Pasadena. Jet propulsion laboratory, national aeronautics and space administration, CA http://hdl.handle.net/2014/45978

  • Vecchia AV (1988) Estimation and model identification for continuous spatial processes. J R Stat Soc Ser B (Methodol) 50:297–312

    MathSciNet  Google Scholar 

  • Waliser D, Gleckler PJ, Ferraro R, Taylor KE, Ames S, Biard J, Bosilovich MG, Brown O, Chepfer H, Cinquini L, Durack PJ, Eyring V, Mathieu P-P, Lee T, Pinnock S, Potter GL, Rixen M, Saunders R, Schulz J, Thépaut J-N, Tuma M (2020) Observations for model intercomparison project (Obs4MIPs): status for CMIP6. Geosci Model Dev 13:2945–2958

    Google Scholar 

  • Wunch D, Wennberg PO, Toon GC, Connor BJ, Fisher B, Osterman GB, Frankenberg C, Mandrake L, O’Dell C, Ahonen P, Biraud SC, Castano R, Cressie N, Crisp D, Deutscher NM, Eldering A, Fisher ML, Griffith DWT, Gunson M, Heikkinen P, Keppel-Aleks G, Kyrö E, Lindenmaier R, Macatangay R, Mendonca J, Messerschmidt J, Miller CE, Morino I, Notholt J, Oyafuso FA, Rettinger M, Robinson J, Roehl CM, Salawitch RJ, Sherlock V, Strong K, Sussmann R, Tanaka T, Thompson DR, Uchino O, Warneke T, Wofsy SC (2011) A method for evaluating bias in global measurements of \(\text{ CO}_{{2}}\) total columns from space. Atmos Chem Phys 11:12317–12337

    Google Scholar 

  • Zhou X, Kollias P, Lewis ER (2015) Clouds, precipitation, and marine boundary layer structure during the MAGIC Field campaign. J Clim 28:2420–2442

    Google Scholar 

  • Zhu Y, Kang EL, Bo Y, Tang Q, Cheng J, He Y (2015) A robust fixed rank kriging method for improving the spatial completeness and accuracy of satellite SST products. IEEE Trans Geosci Remote Sens 53:5021–5035

    Google Scholar 

Download references

Acknowledgements

The research of Konomi and Kang was partially supported by National Science Foundation Grant NSF DMS-2053668. Kang was also partially supported by Simons Foundation’s Collaboration Award (#317298 and #712755), and the Taft Research Center at the University of Cincinnati. Part of this work was performed at the Jet Propulsion Laboratory, California Institute of Technology, under contract with NASA. Support was provided by the Atmospheric Infrared Sounder (AIRS) mission. The authors thank Amy Braverman and Hai Nguyen for valuable discussions during the course of this work.

Author information

Authors and Affiliations

  1. Division of Statistics and Data Science, Department of Mathematical Sciences, University of Cincinnati, Cincinnati, USA

    Bledar A. Konomi & Emily L. Kang

  2. Department of Statistics, Yarmouk University, Irbid, Jordan

    Ayat Almomani

  3. Jet Propulsion Laboratory, California Institute of Technology, Pasadena, USA

    Jonathan Hobbs

Authors
  1. Bledar A. Konomi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emily L. Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ayat Almomani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jonathan Hobbs
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bledar A. Konomi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 331 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Konomi, B.A., Kang, E.L., Almomani, A. et al. Bayesian Latent Variable Co-kriging Model in Remote Sensing for Quality Flagged Observations. JABES 28, 423–441 (2023). https://doi.org/10.1007/s13253-023-00530-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13253-023-00530-9

Keywords

Search

Navigation