Environmental Modeling & Assessment

, Volume 18, Issue 3, pp 249–257 | Cite as

Ensemble Techniques to Improve Air Quality Assessment: Focus on O3 and PM

  • A. Monteiro
  • I. Ribeiro
  • O. Tchepel
  • A. Carvalho
  • H. Martins
  • E. Sá
  • J. Ferreira
  • V. Martins
  • S. Galmarini
  • A. I. Miranda
  • C. Borrego


Five air quality models were applied over Portugal for July 2006 with an ensemble purpose. These models were used, with their own meteorology, parameterizations, boundary conditions and chemical mechanisms, but with the same emission data. The validation of the individual models and its ensemble for ozone (O3) and particulate matter was performed using monitoring data from 22 background stations over Portugal. After removing the bias from each model, different ensemble techniques were applied and compared. Besides the median, several weighted ensemble approaches were tested and intercompared: static (SLR) and dynamic (DLR) multiple linear regressions (using less-square optimization method) and the Bayesian Model Averaging (BMA) methodology. The goal of the comparison is to estimate to what extent the ensemble analysis is an improvement with respect to the single model results. The obtained results revealed that no one of the 4 tested ensembles clearly outperforms the others on the basis of statistical parameters and probabilistic analysis (reliability and resolution properties). Nevertheless, statistical results have shown that the application of the weights slightly improves ensemble performance when compared to those obtained from the median ensemble. The same statistical analysis together with the probabilistic measures demonstrates that the SLR and BMA methods are the best performers amongst the assessed methodologies.


Air quality modelling Ensemble techniques Static and dynamic approaches Minimum least square BMA methodology 



The authors acknowledge the Portuguese Environmental Protection Agency for the observational dataset support. Thanks are extended to the Portuguese ‘Ministério da Ciência, da Tecnologia e do Ensino Superior’ for the financing of ENSEMBLAIR (POCI/AMB/66707/2006) project, for the postdoc grants of H. Martins (SFRH/BPD/66874/2009) and J. Ferreira (SFRH/BPD/40620/2007) and also for the PhD grants of I. Ribeiro (SFRH/BD/60370/2009) and E. Sá (SFRH/BD/60474/2009).


  1. 1.
    Pagowski, M., Grell, G. A., McKeen, S. A., Dévényi, D., Wilczak, J. M., Bouchet, V., Gong, W., McHenry, J., Peckham, S., McQueen, J., Moffet, R., & Tang, Y. (2005). A simple method to improve ensembel-based ozone forecasts. Geophysical Research Letters, 32, L07814. doi: 10.1029/2004GL022305.CrossRefGoogle Scholar
  2. 2.
    Pagowski, M., Grell, G. A., Devenyi, D., Peckham, S., McKeen, S. A., Gong, W., Delle Monache, L., McHenry, J. N., McQueen, J., & Lee, P. (2006). Application of dynamic linear regression to improve the skill of ensemble-based deterministic ozone forecasts. Atmospheric Environment, 40, 3240e3250. doi: 10.1016/j.atmosenv.2006.02.006.Google Scholar
  3. 3.
    Mallet, V., & Sportisse, B. (2006). Ensemble-based air quality forecasts: a multimodel approach applied to ozone. Journal of Geophysical Research, 111, D18302. doi: 10.1029/2005JD006675.CrossRefGoogle Scholar
  4. 4.
    Delle Monache, L., Deng, X., Zhou, Y., & Stull, R. (2006a). Ozone ensemble forecasts: 1. A new ensemble design. Journal of Geophysical Research, 111, D05307. doi: 10.1029/2005JD006310.CrossRefGoogle Scholar
  5. 5.
    Delle Monache, L., Nipen, T., Deng, X., Zhou, Y., & Stull, R. (2006b). Ozone ensemble forecasts: 2. A Kalman filter predictor bias correction. Journal of Geophysical Research, 111, D05308. doi: 10.1029/2005JD006311.CrossRefGoogle Scholar
  6. 6.
    Wilczak, J., McKeen, S. A., Djalalova, I., et al. (2006). Bias-corrected ensemble and probabilistic forecasts of surface ozone over eastern North America during the summer of 2004. Journal of Geophysical Research, 111, D23S28. doi: 10.1029/2006JD007598.CrossRefGoogle Scholar
  7. 7.
    Zhang, F., Bei, N., Nielsen-Gammon, J. W., Li, G., Zhang, R., Stuart, A., & Aksoy, A. (2007). Impacts of meteorological uncertainties on ozone pollution predictability estimated through meteorological and photochemical ensemble forecasts. Journal of Geophysical Research, 112, D04304. doi: 10.1029/2006JD007429.CrossRefGoogle Scholar
  8. 8.
    McKeen, S., Chung, S. H., Wilczak, J., et al. (2007). The evaluation of several PM2.5 forecast models using data collected during the ICARTT/NEAQS 2004 field study. Journal of Geophysical Research, 112, D10S20. doi: 10.1029/2006JD007608.CrossRefGoogle Scholar
  9. 9.
    McKeen, S., et al. (2005). Assessment of an ensemble of seven real-time ozone forecasts over eastern North America during the summer of 2004. Journal of Geophysical Research, 110, D21307. doi: 10.1029/2005JD005858.CrossRefGoogle Scholar
  10. 10.
    van Loon, M., Vautard, R., Schaap, M., Bergström, R., Bessagnet, B., Brandt, J., Builtjes, P., Christensen, J., Cuvelier, C., Graff, A., Jonson, J., Krol, M., Langner, J., Roberts, P., Rouil, L., Stern, R., Tarrasón, L., Thunis, P., Vignati, E., White, L., & Winda, P. (2007). Evaluation of long-term ozone simulations from seven regional air quality models and their ensemble. Atmospheric Environment, 41(10), 2083–2097.CrossRefGoogle Scholar
  11. 11.
    Vautard, R., Builtjes, P., Thunis, P., Cuvelier, K., Bedogni, M., Bessagnet, B., Honoré, C., Moussiopoulos, N., Pirovano, G., Schaap, M., Stern, R., Tarrason, L., & Van Loon, M. (2007). Evaluation and intercomparison of ozone and PM10 simulations by several chemistry-transport models over 4 European cities within the City-Delta project. Atmospheric Environment, 41, 173–188.CrossRefGoogle Scholar
  12. 12.
    Vautard, R., Schaap, P. M., Bergström, R., et al. (2009). Skill and uncertainty of a regional air quality model ensemble. Atmospheric Environment, 43(31), 4822–4832.CrossRefGoogle Scholar
  13. 13.
    Riccio, A., Giunta, G., & Galmarini, S. (2007). Seeking for the rational of the median model: the optimal combination of multi-model ensemble. Atmospheric Chemistry and Physics, 7, 6085–6098.CrossRefGoogle Scholar
  14. 14.
    Galmarini, S., et al. (2004). Ensemble dispersion forecasting, part I: concept, approach and indicators. Atmospheric Environment, 38(28), 4607–4617.CrossRefGoogle Scholar
  15. 15.
    Galmarini, S., Bianconi, R., Addis, R., Andronopoulos, S., Astrup, P., Bartzis, J. C., Bellasio, R., Buckley, R., Champion, H., Chino, M., D’Amours, R., Davakis, E., Eleveld, H., Glaab, H., Manning, A., Mikkelsen, T., Pechinger, U., Polreich, E., Prodanova, M., Slaper, H., Syrakov, D., Terada, H., & Van der Auwera, L. (2004). Ensemble dispersion forecasting, part II: application and evaluation. Atmospheric Environment, 38(28), 4619–4632.CrossRefGoogle Scholar
  16. 16.
    Hou, D., Kalnay, E., & Droegemeier, K. K. (2001). Objective verification of the SAMEX’98 ensemble forecasts. Monthly Weather Review, 129, 73–91.CrossRefGoogle Scholar
  17. 17.
    Delle Monache, L., & Stull, R. (2003). An ensemble air quality forecast over western Europe during an ozone forecast. Atmospheric Environment, 37, 3469–3474.CrossRefGoogle Scholar
  18. 18.
    Djalalova, I., Wilczak, J., McKeen, S., Grell, G., Peckhama, S., Pagowski, M., DelleMonache, L., McQueen, J., Tang, Y., Leeg, P., McHenry, J., Gong, W., Bouchet, V., & Mathur, R. (2010). Ensemble and bias-correction techniques for air quality model forecasts of surface O3 and PM2.5 during the TEXAQS-II experiment of 2006. Atmospheric Environment, 44, 455–467.CrossRefGoogle Scholar
  19. 19.
    Monteiro, A., Ribeiro, I., Tchepel, O., Sá, E., Ferreira, J., Carvalho, A., Martins, V., Strunk, A., Galmarini, S., Elbern, H., Schaap, M., Builtjes, P., Miranda, A. I., Borrego, C. (2012). Bias correction techniques to improve air quality ensemble predictions: focus on O3 and PM over Portugal. Environmental Modelling and Assessment (in press).Google Scholar
  20. 20.
    ENVIRON. (2008). User’s guide to the comprehensive air quality model with extensions (CAMx) version 4.50 (May, 2008), Accessed Dec 2011.
  21. 21.
    Schmidt, H., Derognat, C., Vautard, R., & Beekmann, M. (2001). A comparison of simulated and observed ozone mixing ratios for the summer of 1998 in Western Europe. Atmospheric Environment, 35, 2449–2461.CrossRefGoogle Scholar
  22. 22.
    Bessagnet, B., Hodzic, A., Vautard, R., Beekmann, M., Cheinet, S., Honoré, C., Liousse, C., & Rouil, L. (2004). Aerosol modelling with CHIMERE—preliminary evaluation at the continental scale. Atmospheric Environment, 38, 2803–2817.CrossRefGoogle Scholar
  23. 23.
    Elbern, H., Strunk, A., Schmidt, H., & Talagrand, O. (2007). Emission rate and chemical state estimation by 4-dimensional variational inversion. Atmospheric Chemistry and Physics, 7, 3749–3769.CrossRefGoogle Scholar
  24. 24.
    Schaap, M., Timmermans, R. M. A., Sauter, F. J., Roemer, M., Velders, G. J. M., Boersen, G. A. C., Beck, J. P., & Builtjes, P. J. H. (2008). The LOTOS-EUROS model: description, validation and latest developments. International Journal of Environment and Pollution, 32(2), 270–290.CrossRefGoogle Scholar
  25. 25.
    Hurley, P., Physick, W., & Luhar, A. (2005). TAPM—a practical approach to prognostic meteorological and air pollution modelling. Environmental Modelling & Software, 20, 737–752.CrossRefGoogle Scholar
  26. 26.
    Dudhia, J. (1993). A nonhydrostatic version of the PennState/NCAR mesoscale model: validation tests and simulation of an Atlantic cyclone and clod front. Monthly Weather Review, 121, 1493–1513.CrossRefGoogle Scholar
  27. 27.
    Potempski, S., & Galmarini, S. (2009). Est modus in rebus: analytical properties of multi-model ensembles. Atmospheric Chemistry and Physics, 9, 9471–9489.CrossRefGoogle Scholar
  28. 28.
    Monteiro, A., Borrego, C., Miranda, A. I., Gois, V., Torres, P., Perez, A. T. (2007). Can air quality modelling improve emission inventories?. In: Proceedings of the 6th International Conference on Urban Air Quality, 26–30 March, Limassol, Cyprus (pp. 13–14).Google Scholar
  29. 29.
    Krishnamurti, T. N., Kishtawal, C. M., LaRow, T. E., Bachiochi, D. R., Zhang, Z., Willford, C. E., Gadfil, S., & Surendran, S. (1999). Improved weather and seasonal climate forecast from multimodel superensemble. Science, 285, 1548–1550.CrossRefGoogle Scholar
  30. 30.
    Raftery, A. E., Balabdaoui, F., Gneiting, T., & Polakowski, M. (2005). Using Bayesian model averaging to calibrate forecast ensembles. Monthly Weather Review, 133, 1155–1174.CrossRefGoogle Scholar
  31. 31.
    Raftery, A. E., & Zheng, Y. (2003). Long-run performance of Bayesian model averaging. Journal of the American Statistical Association, 98, 931–938.CrossRefGoogle Scholar
  32. 32.
    Duan, Q., Ajami, N. K., Gao, X., & Sorooshian, S. (2007). Multi-model hydrological prediction using Bayesian model averaging. Advances in Water Resources, 30(5), 1371–1386.CrossRefGoogle Scholar
  33. 33.
    Potempski, S., Galmarini, S., Riccio, A., & Giunta, G. (2010). Bayesian model averaging for emergency response atmospheric dispersion multimodel ensembles: is it really better? How many data are needed? Are the weights portable? Journal of Geophysical Research, 115, 9. doi: 10.1029/2010JD014210. D21309.CrossRefGoogle Scholar
  34. 34.
    Tilmes, S., Brandt, J., Flatoy, F., Bergrstrom, R., Flemming, J., Langner, J., Christensen, J. H., Frohn, L. M., Hov, O., Jacobsen, I., Reimer, E., Stern, R., & Zimmermann, J. (2002). Comparison of five Eulerian air pollution forecasting systems for the summer of 1999 using the German ozone monitoring data. Journal of Atmospheric Chemistry, 42, 91–121.CrossRefGoogle Scholar
  35. 35.
    Borrego, C., Monteiro, A., Ferreira, J., Miranda, A. I., Costa, A. M., Carvalho, A. C., & Lopes, M. (2008). Procedures for estimation of modelling uncertainty in air quality assessment. Environment International, 34, 613–620. doi: 10.1016/j.envint.2007.12.005.CrossRefGoogle Scholar
  36. 36.
    Monteiro, A., Strunk, A., Carvalho, A., Tchepel, O., Miranda, A. I., Borrego, C., Saavedra, S., Rodriguez, A., Souto, J., Casares, J., & Elbern, H. (2012). Investigating a high ozone episode in a rural mountain site. Environmental Pollution, 162, 176–189.CrossRefGoogle Scholar
  37. 37.
    Taylor, K. E. (2001). Summarizing multiple aspects of model performance in a single diagram. Journal of Geophysical Research, 106(D7), 7183–7192.CrossRefGoogle Scholar
  38. 38.
    Talagrand, O., Vautard, R., Strauss, B. (1998). Evaluation of probabilistic prediction systems. Proceedings of the Seminar on Predictability, Reading, UK, ECMWF (pp. 1–26).Google Scholar
  39. 39.
    Hamill, T. M. (2001). Interpretation of rank histograms for verifying ensemble forecasts. Monthly Weather Review, 129, 550–560.CrossRefGoogle Scholar
  40. 40.
    Mason, S. J., & Graham, N. E. (1999). Conditional probabilities, relative operating characteristics, and relative operative levels. Weather and Forecasting, 14, 713–725.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2012

Authors and Affiliations

  • A. Monteiro
    • 1
  • I. Ribeiro
    • 1
  • O. Tchepel
    • 1
  • A. Carvalho
    • 1
  • H. Martins
    • 1
  • E. Sá
    • 1
  • J. Ferreira
    • 1
  • V. Martins
    • 1
  • S. Galmarini
    • 2
  • A. I. Miranda
    • 1
  • C. Borrego
    • 1
  1. 1.CESAM & Department of Environment and PlanningUniversity of AveiroAveiroPortugal
  2. 2.IES/REM, Joint Research Center, European CommissionIspraItaly

Personalised recommendations