Evaluation of Air Quality Maps Using Cross-Validation: Metrics, Diagnostics and Optimization

  • Richard MénardEmail author
  • Martin Deshaies-Jacques
Conference paper
Part of the Springer Proceedings in Complexity book series (SPCOM)


Since 2002, Environment and Climate Change Canada (ECCC) has been producing hourly surface analyses of pollutants covering North America which have been used in numerous health impact studies. The analyses are produced using an optimum interpolation scheme that combines the operational air quality forecast model GEM-MACH outputs (CHRONOS model outputs were used prior to 2009) with real-time hourly observations of O3, PM2.5, PM10, NO2, and SO2. We examine how passive observations, that are not used to create the analysis (i.e. cross-validation), can be used to evaluate the analysis error and to optimize the input error statistics.


  1. 1.
    D.L. Crouse, P.A. Peters, P. Hystad, J.R. Brook, A. van Donkelaar, R.V. Martin, P.J. Villeneuve, M. Jerrett, M.S. Goldberg, C.A. Pope III, M. Brauer, R.D. Brook, A. Robichaud, R. Ménard, R.T. Burnett, Associations between ambient PM2.5, O3, and NO2 and mortality in the Canadian Census Health and Environment Cohort (CanCHEC) over a 16-year follow-up. Environ. Health Perspect. (Open Access) 123, 1180–1186 (2015). Scholar
  2. 2.
    D.L. Crouse, L. Pinault, A. Balram, P. Hystad, P.A. Peters, A. van Donkelaar, R.V. Martin, R. Ménard, A. Robichaud, P.J. Villeneuve, Urban greenness and mortality in Canada’s largest cities: a national cohort study. The Lancet, Planet Earth 1, e289–e297 (2017), Scholar
  3. 3.
    D.L. Crouse, A. Balram, P. Hystad, L. Pinault, M. van den Boshm, H. Chen, D. Rainham, E.M. Thomson, C.H. Close, A. van Donkelaar, R.V. Martin, R. Ménard, A. Robichaud, P.J. Villeneuve, Associations between living near water and risk of mortality among urban Canadians. Environ. Health Perspect. 126(7) (2018). Scholar
  4. 4.
    R. Ménard, A. Robichaud, The chemistry-forecast system at the Meteorological Service of Canada. In: The ECMWF Seminar Proceedings on Global Earth-System Monitoring, Reading, UK, 5–9 September 2005; pp. 297–308 (2005)Google Scholar
  5. 5.
    R. Ménard, Error covariance estimation methods based on analysis residuals: theoretical foundation and convergence properties derived from simplified observation networks. Q. J. R. Meteorol. Soc. 142, 257–273 (2016). Scholar
  6. 6.
    R. Ménard, M. Deshaies-Jacques, N. Gassett, A comparison of correlation-length estimation methods for the objective analysis of surface pollutants at Environment and Climate Change Canada. J. Air Waste Manag. Assoc. 66, 874–895 (2016). Scholar
  7. 7.
    R. Ménard, M. Deshaies-Jacques, Error covariance estimation methods based on analysis residuals and its application to air quality surface observation networks. In: C. Mensink, G. Kallos (Eds.) Air Pollution and Its Application XXV. Springer International AG, 249-253 (2017). Scholar
  8. 8.
    R. Ménard, M. Deshaies-Jacques, Evaluation of analysis by cross-validation. Part I: Using verification metrics. Atmosphere 9, 86 (2018a)., Scholar
  9. 9.
    R. Ménard, M. Deshaies-Jacques, Evaluation of analysis by cross-validation. Part II: Diagnostic and optimization of analysis error covariance. Atmosphere 9, 70 (2018b).,
  10. 10.
    A. Robichaud, R. Ménard, Multi-year objective analysis of warm season ground-level ozone and PM2.5 over North-America using real-time observations and Canadian operational air quality models. Atmos. Chem. Phys. 14, 1769–1800 (2014). Scholar
  11. 11.
    A. Robichaud, R. Ménard, Y. Zaïtseva, D. Anselmo, Multi-pollutant surface objective analyses and mapping of air quality health index over North America. Air Qual. Atmos. Health 9, 743–759 (2016). Scholar
  12. 12.
    D. Stieb, R. Ménard, M. Smith-Doiron, J. Yao, S. Henderson, L. Pinault, A. Robichaud, A. van Donkelaar, R.V. Martin, J. Brook, Spatial variability in ambient ozone and fine particles concentrations and population susceptibility among Canadian health regions. Can. J. Public Health (in print) (2019)Google Scholar

Copyright information

© Crown 2020

Authors and Affiliations

  1. 1.Air Quality Research DivisionEnvironment and Climate Change CanadaDorvalCanada

Personalised recommendations