Skip to main content
SpringerLink
Log in

Deep particulate matter forecasting model using correntropy-induced loss

  • Original Article
  • Published:
Journal of Mechanical Science and Technology Aims and scope Submit manuscript

Abstract

Forecasting the particulate matter (PM) concentration in South Korea has become urgently necessary owing to its strong negative impact on human life. In most statistical or machine learning methods, independent and identically distributed data, for example, a Gaussian distribution, are assumed; however, time series such as air pollution and weather data do not meet this assumption. In this study, the detrended fluctuation analysis and power-law analysis are used in an analysis of the statistical characteristics of air pollution and weather data. Rigorous seasonality adjustment of the air pollution and weather data was performed because of their complex seasonality patterns and the heavy-tailed distribution of data even after deseasonalization. The maximum correntropy criterion for regression (MCCR) loss was applied to multiple models including conventional statistical models and state-of-the-art machine learning models. The results show that the MCCR loss is more appropriate than the conventional mean squared error loss for forecasting extreme values.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Abbreviations

C(r):

Autocorrelation

F(x):

Cumulative distribution function

\(\overline F (x)\) :

Complementary cumulative distribution function

h :

DFA fluctuation exponent

M i :

Predicted value

O i :

Actual value

res h :

Residuals

s y, smoothed :

Smoothed yearly seasonality

s w :

Weekly seasonality

s h :

Daily seasonality

V(s):

DFA fluctuation function

x(t):

Single column input time series

Y t :

Actual value

\({\overline Y _t}\) :

Predicted value

α :

Pareto index

β :

MCCR scale parameter

ξ :

Long-range dependence power law exponent

References

  1. EPA, Health and Environmental Effects of Particulate Matter (PM), United States Environmental Protection Agency, United States Government (2016).

  2. H. B. Kim et al., Long-term exposure to air pollutants and cancer mortality: a meta-analysis of cohort studies, Int. J. Environ. Res. Public Health, 15(11) (2018) 2608.

    Article  Google Scholar 

  3. C. A. Pope and D. W. Dockery, Health effects of fine particulate air pollution: lines that connect, J. Air Waste Manag. Assoc., 56(6) (2006) 709–742.

    Article  Google Scholar 

  4. OECD, The Economic Consequences of Outdoor Air Pollution, OECD Publishing, Paris (2016).

    Book  Google Scholar 

  5. AirKorea, Annual Report of Air Quality, Korea Environment Corporation (2019).

  6. WHO, Air Quality Guidelines, World Health Organization (2006).

  7. OECD, Air Pollution Exposure (Indicator), Organization for Economic Co-operation and Development (2021).

  8. P. A. Makar et al., Feedbacks between air pollution and weather, part 1: effects on weather, Atmos. Environ., 115 (2015) 442–469.

    Article  Google Scholar 

  9. I. Bouarar et al., Influence of anthropogenic emission inventories on simulations of air quality in China during winter and summer 2010, Atmos. Environ., 198 (2019) 236–256.

    Article  Google Scholar 

  10. H. J. Lee et al., Impacts of atmospheric vertical structures on transboundary aerosol transport from China to South Korea, Sci. Rep., 9 (2019) 13040.

    Article  Google Scholar 

  11. C. Jordan et al., Investigation of factors controlling PM25 variability across the South Korean Peninsula during KORUSAQ, Elem. Sci. Anth., 8 (28) (2020).

  12. K. P. Singh et al., Linear and nonlinear modeling approaches for urban air quality prediction, Sci. Total Environ., 426 (2012) 244–255.

    Article  Google Scholar 

  13. Y. Zhang et al., Real-time air quality forecasting, part I: history, techniques, and current status, Atmos. Environ., 60 (2012) 632–655.

    Article  Google Scholar 

  14. D. W. Byun and J. K. S. Ching, Science Algorithms of the EPA Models-3 Community Multiscale Air Quality (CMAQ) Modeling System, EPA/600/R-99/030, United States Environmental Protection Agency, United States Government (1999).

  15. I. Bey et al., Global modeling of tropospheric chemistry with assimilated meteorology: model description and evaluation, J. Geophys. Res. Atmos, 106(D19) (2001) 23073–23095.

    Article  Google Scholar 

  16. G. A. Grell et al., Fully coupled online chemistry within the WRF model, Atmos. Environ., 39(37) (2005) 6957–6975.

    Article  Google Scholar 

  17. A. Baklanov et al., Towards improving the simulation of meteorological fields in urban areas through updated/advanced surface fluxes description, Atmos. Chem. Phys., 8(3) (2008) 523–543.

    Article  Google Scholar 

  18. G. Zhou et al., Numerical air quality forecasting over eastern China: an operational application of WRF-Chem, Atmos. Environ., 153 (2017) 94–108.

    Article  Google Scholar 

  19. Z. Shang et al., A novel model for hourly PM25 concentration prediction based on CART and EELM, Sci. Total Environ., 651 (2019) 3043–3052.

    Article  Google Scholar 

  20. D. Guo, R. Guo and C. Thiart, Predicting air pollution using fuzzy membership grade Kriging, Comput. Environ. Urban Syst., 31(1) (2007) 33–51.

    Article  Google Scholar 

  21. W. Wang and Y. Guo, Air pollution PM25 data analysis in Los Angeles long beach with seasonal ARIMA model, 2009 Int. Conf. Energy Environ. Technol. (2009) 7–10.

  22. G. Lai et al., Modeling long- and short-term temporal patterns with deep neural networks, 41st Int. ACM SIGIR Conf. Res. Dev. Inf. Retrieval (2018).

  23. S. Y. Shih, F. K. Sun and H. Y. Lee, Temporal pattern attention for multivariate time series forecasting, Mach. Learn., 108(8) (2019) 1421–1441.

    Article  MathSciNet  Google Scholar 

  24. S. Li et al., Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting, arXiv: 1907.00235 (2019).

  25. J. E. Choi, H. Lee and J. Song, Forecasting daily PM10 concentrations in Seoul using various data mining techniques, Commun. Stat. Appl. Methods, 25(2) (2018) 199–215.

    Google Scholar 

  26. K. Cho et al., Air quality prediction using a deep neural network model, J. Korean Soc. Atmos. Environ., 35(2) (2019) 214–225.

    Article  Google Scholar 

  27. F. Franceschi, M. Cobo and M. Figueredo, Discovering relationships and forecasting PM10 and PM25 concentrations in Bogotá, Colombia, using artificial neural networks, principal component analysis, and k-means clustering, Atmos. Pollut. Res., 9(5) (2018) 912–922.

    Article  Google Scholar 

  28. Y. Bai et al., Hourly PM25 concentration forecast using stacked autoencoder model with emphasis on seasonality, J. Clean. Prod., 224 (2019) 739–750.

    Article  Google Scholar 

  29. W. Liu, P. P. Pokharel and J. C. Principe, Correntropy: properties and applications in non-Gaussian signal processing, IEEE Trans. Signal Process., 55(11) (2007) 5286–5298.

    Article  MathSciNet  Google Scholar 

  30. Y. Feng et al., Learning with the maximum correntropy criterion induced losses for regression, J. Mach. Learn. Res., 16(30) (2015) 993–1034.

    MathSciNet  MATH  Google Scholar 

  31. R. Cichowicz, G. Wielgosiński and W. Fetter, Dispersion of atmospheric air pollution in summer and winter season, Environ. Monit. Assess., 189(12) (2017) 605.

    Article  Google Scholar 

  32. O. Troyanskaya et al., Missing value estimation methods for DNA microarrays, Bioinformatics, 17(6) (2001) 520–525.

    Article  Google Scholar 

  33. F. Pedregosa et al., Scikit-learn: machine learning in python, J. Mach. Learn. Res., 12 (2011) 2825–2830.

    MathSciNet  MATH  Google Scholar 

  34. W. S. Cleveland, Robust locally weighted regression and smoothing scatterplots, J. Am. Stat. Assoc., 74(368) (1979) 829–836.

    Article  MathSciNet  Google Scholar 

  35. J. W. Kantelhardt et al., Detecting long-range correlations with detrended fluctuation analysis, Phys. A Stat. Mech. Its Appl., 295(3–4) (2001) 441–454.

    Article  Google Scholar 

  36. E. Koscielny-Bunde et al., Long-term persistence and multifractality of river runoff records: detrended fluctuation studies, J. Hydrol., 322(1–4) (2006) 120–137.

    Article  Google Scholar 

  37. A. Clauset, C. R. Shalizi and M. E. J. Newman, Power-law distributions in empirical data, SIAM Rev., 51(4) (2009) 661–703.

    Article  MathSciNet  Google Scholar 

  38. J. Alstott, E. Bullmore and D. Plenz, Powerlaw: a python package for analysis of heavy-tailed distributions, PLoS One, 9(1) (2014) e85777.

    Article  Google Scholar 

  39. G. E. Uhlenbeck and L. S. Ornstein, On the theory of the Brownian motion, Phys. Rev., 36 (1930) 823.

    Article  Google Scholar 

  40. M. T. Wojnowicz, The Ornstein-Uhlenbeck process in neural decision-making: mathematical foundations and simulations suggesting the adaptiveness of robustly integrating stochastic neural evidence, Master’s Thesis, University of Washington (2012).

  41. R. J. Hyndman and G. Athanasopoulos, Forecasting: Principles and Practice, Second Edition, OTexts, Melbourne, Australia (2018).

    Google Scholar 

  42. T. Chen and C. Guestrin, XGBoost: a scalable tree boosting system, Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min. (2016) 785–794.

  43. I. Goodfellow, Y. Bengio and A. Courville, Deep Learning, MIT Press (2015).

  44. G. Zhang, B. Eddy Patuwo and M. Y. Hu, Forecasting with artificial neural networks: the state of the art, Int. J. Forecast., 14(1) (1998) 35–62.

    Article  Google Scholar 

  45. S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural Comput., 9(8) (1997) 1735–1780.

    Article  Google Scholar 

  46. K. Cho et al., Learning phrase representations using RNN encoder-decoder for statistical machine translation, Proc. 2014 Conf. Empir. Methods Nat. Lang. Process. (2014) 1724–1734.

  47. D. Bahdanau, K. H. Cho and Y. Bengio, Neural machine translation by jointly learning to align and translate, 3rd Int. Conf. Learn. Represent. ICLR 2015 — Conf. Track Proc. (2015).

  48. N. Wu et al., Deep transformer models for time series forecasting: the influenza prevalence case, arXiv: 2001.08317 (2020).

  49. G. Zerveas et al., A transformer-based framework for multivariate time series representation learning, arXiv: 2010.02803 (2020).

  50. M. Bayram, T. Partal and G. O. Buyukoz, Numerical methods for simulation of stochastic differential equations, Adv. Differ. Equations, 17 (2018).

  51. S. Seabold and J. Perktold, Statsmodels: econometric and statistical modeling with python, Proc. 9th Python Sci. Conf. (2010).

  52. I. Sutskever, O. Vinyals and Q. V. Le, Sequence to sequence learning with neural networks, Adv. Neural Inf. Process. Syst. (2014) 3104–3112.

  53. A. Vaswani et al., Attention is all you need, Adv. Neural Inf. Process. Syst. (2017) 6000–6010.

  54. S. M. Kazemi et al., Time2Vec: learning a vector representation of time, arXiv: 1907.05321 (2019).

  55. D. Ding et al., Modeling extreme events in time series prediction, Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min. (2019).

  56. R. P. Ribeiro and N. Moniz, Imbalanced regression and extreme value prediction, Mach. Learn., 109(9) (2020) 1083–1835.

    MathSciNet  MATH  Google Scholar 

  57. D. Qi and A. J. Majda, Using machine learning to predict extreme events in complex systems, Proc. Natl. Acad. Sci. U. S. A., 117(1) (2020) 52–59.

    Article  MathSciNet  Google Scholar 

  58. T. Akiba et al., Optuna: a next-generation hyperparameter optimization framework, Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min. (2019).

  59. D. P. Kingma and J. L. Ba, Adam: a method for stochastic optimization, 3rd Int. Conf. Learn. Represent. ICLR 2015-Conf. Track Proc. (2015).

  60. A. Paszke et al., PyTorch: an imperative style, high-performance deep learning library, Adv. Neural Inf. Process. Syst. (2019) 8024–8035.

  61. J. Myoung et al., Optimization of the computing environment to improve the speed of the modeling (WRF and CMAQ) calculation of the national air quality forecast system, J. Environ. Sci. Int., 27(8) (2018) 723–735.

    Article  Google Scholar 

  62. S. Yu et al., New unbiased symmetric metrics for evaluation of air quality models, Atmos. Sci. Lett., 7(1) (2006) 26–34.

    Article  Google Scholar 

  63. Y. Li and Y. Liang, Learning overparameterized neural networks via stochastic gradient descent on structured data, Adv. Neural Inf. Process. Syst. (2018).

  64. A. Borovykh, C. W. Oosterlee and S. M. Bohté, Generalization in fully-connected neural networks for time series forecasting, J. Comput. Sci. (2019).

  65. B. K. Yi, H. V. Jagadish and C. Faloutsos, Efficient retrieval of similar time sequences under time warping, Proc. — Int. Conf. Data Eng. (1998).

Download references

Acknowledgments

This work was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIP) (2017R1E1A1A03070282).

Author information

Authors and Affiliations

  1. School of Mathematics and Computing, Yonsei University, Seoul, 03722, Korea

    Jongsu Kim & Changhoon Lee

  2. Department of Mechanical Engineering, Yonsei University, Seoul, 03722, Korea

    Changhoon Lee

Authors
  1. Jongsu Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Changhoon Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Changhoon Lee.

Additional information

Jongsu Kim received his B.S. (2011) in Atmospheric Science and Computer Science from Yonsei University, Seoul, Korea. He is a Ph.D. candidate at Yonsei University in the School of Mathematics and Computing. His research interests include time series forecasting using machine learning.

Changhoon Lee received his B.S. (1985) and M.S. (1987) from Seoul National University, Seoul, Korea and his Ph.D. (1993) from UC Berkeley, USA in Mechanical Engineering. He is a Professor in the Department of Computational Science & Engineering and Department of Mechanical Engineering, Yonsei University, Korea. His research interests include the fundamentals of turbulence, particle-turbulence interaction, numerical algorithms, air pollution modeling, and stochastic processes.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, J., Lee, C. Deep particulate matter forecasting model using correntropy-induced loss. J Mech Sci Technol 35, 4045–4063 (2021). https://doi.org/10.1007/s12206-021-0817-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12206-021-0817-4

Keywords

Search

Navigation