Skip to main content

Advertisement

SpringerLink
Log in

Global horizontal and direct normal solar irradiance modeling by the machine learning methods XGBoost and deep neural networks with CNN-LSTM layers: a case study using the GOES-16 satellite imagery

  • Original Research
  • Published:
International Journal of Energy and Environmental Engineering Aims and scope Submit manuscript

Abstract

Restrictive legislations on the use of fossil fuels encourage the research and development of clean and renewable energies. Renewable energy is characterized by random behavior, which hampers its integration into the current energy base system. Thus, estimating solar irradiation is important for the adoption of renewable energies into the current energy matrix. In this paper, two machine learning estimation models for global horizontal (GHI) and direct normal solar irradiance (DNI) are proposed: the first uses XGBoost and the second employs a convolutional neural network (CNN) combined with a long short-term memory (LSTM) network, forming the hybrid CNN-LSTM model. The case studies apply both models to process images from the GOES-16 satellite, taken from the city of Petrolina, Pernambuco, Brazil. Their results are compared against the reference Copernicus Atmosphere Monitoring Service, Solcast and the Physical Solar Model (PSM) provided by the National Solar Radiation Database. For the GHI estimation, the PSM model achieved the lowest RMSE, 147.23 W/m2, while for DNI estimation, the CNN-LSTM model performed best, with an RMSE equal to 238.22 W/m2. In this case, the proposed models achieved lower RMSE for DNI estimation when compared against the benchmark models, improving by 2.89% and 1.70% for the CNN-LSTM and XGBoost models, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data availability

All the developed code and the data used can be found on the following links: GOES-16 data grabbing: https://drive.google.com/drive/folders/1B-H9RiOWbgfSArvKsu9H1VzW0_jbst7r?usp = sharing.; https://drive.google.com/drive/folders/1B4juQ3oWvb4T34vPnuaJWpyHTDdII-Yy?usp = sharing. CAMS and Solcast performance evaluation: https://drive.google.com/drive/folders/1B-opvrVcDnnnDcOQqs5KMmpvXiC1Vtrf?usp = sharing. XGBoost and CNN-LSTM performance evaluation: https://drive.google.com/drive/folders/1B2OJQZ3hS7SdrQ8mCMIblSmoJkFz6d0Q?usp = sharing.

Notes

  1. More information about the GOES-R program can be found at https://www.goes-r.gov/mission/mission.html and https://www.nasa.gov/content/goes-r/index.html.

  2. More information about the SONDA project can be accessed at http://sonda.ccst.inpe.br/.

  3. More information about Solcast service can be found at https://solcast.com.

  4. More information about the CAMS service can be found at https://atmosphere.copernicus.eu/.

  5. More information about NSRDB service can be found at https://nsrdb.nrel.gov/.

References

  1. Agreement, P.: United nations. United Nations Treaty Collect, 1–27 (2015)

  2. Saidi, K., Omri, A.: The impact of renewable energy on carbon emissions and economic growth in 15 major renewable energy-consuming countries. Environ. Res. 186, 109567 (2020)

    Article  Google Scholar 

  3. IRENA: Renewable Energy Statistics 2021. International Renewable Energy Agency (IRENA), Abu Dhabi (2021)

  4. Ministry of Mines and Energy: Boletim Mensal de Monitoramento do Sistema Eletrico Brasileiro, Novembro de 2021(2021). Available at: https://www.gov.br/mme/pt-br/assuntos/secretarias/energia-eletrica/publicacoes/boletim-de-monitoramento-do-sistema-eletrico/2021/boletim-de-monitoramento-do-sistema-eletrico-novembro-2021.pdf. Accessed: 2022–01–17

  5. Abatan, A.A., Tett, S.F., Dong, B., Cunningham, C., Rudorff, C.M., Klingaman, N.P., de Abreu, R.C.: Drivers and physical processes of drought events over the state of São Paulo, Brazil. Clim. Dyn., 1–15 (2022)

  6. Vourlitis, G.L., Pinto, O.B., Jr., Dalmagro, H.J., de Arruda, P.E.Z., de Almeida Lobo, F., de Souza Nogueira, J.: Tree growth responses to climate variation in upland and seasonally flooded forests and woodlands of the cerrado-pantanal transition of Brazil. Forest Ecol. Manag. 505, 119917 (2022)

    Article  Google Scholar 

  7. Hunt, J.D., Nascimento, A., ten Caten, C.S., Tomé, F.M.C., Schneider, P.S., Thomazoni, A.L.R., de Castro, N.J., Brandão, R., de Freitas, M.A.V., Martini, J.S.C., et al.: Energy crisis in Brazil: impact of hydropower reservoir level on the river flow. Energy 239, 121927 (2022)

    Article  Google Scholar 

  8. Tupinambá-Simões, F., Bravo, F., Guerra-Hernández, J., Pascual, A.: Assessment of drought effects on survival and growth dynamics in eucalypt commercial forestry using remote sensing photogrammetry. A showcase in Mato Grosso Brazil. Forest Ecolo. Manag. 505, 119930 (2022)

    Article  Google Scholar 

  9. Freitas, A.A., Drumond, A., Carvalho, V.S., Reboita, M.S., Silva, B.C., Uvo, C.B.: Drought assessment in São Francisco river basin, Brazil: characterization through SPI and associated anomalous climate patterns. Atmosphere 13(1), 41 (2022)

    Article  Google Scholar 

  10. Larson, D.P., Li, M., Coimbra, C.F.: Scope: Spectral cloud optical property estimation using real-time goes-r longwave imagery. J. Renew. Sustain. Energy 12(2), 026501 (2020)

    Article  Google Scholar 

  11. Rajagukguk, R.A., Ramadhan, R.A., Lee, H.-J.: A review on deep learning models for forecasting time series data of solar irradiance and photovoltaic power. Energies 13(24), 6623 (2020)

    Article  Google Scholar 

  12. Kaur, A., Nonnenmacher, L., Pedro, H.T., Coimbra, C.F.: Benefits of solar forecasting for energy imbalance markets. Renew. Energy 86, 819–830 (2016)

    Article  Google Scholar 

  13. Law, E.W., Kay, M., Taylor, R.A.: Calculating the financial value of a concentrated solar thermal plant operated using direct normal irradiance forecasts. Sol. Energy 125, 267–281 (2016)

    Article  Google Scholar 

  14. Yang, D., Kleissl, J., Gueymard, C.A., Pedro, H.T., Coimbra, C.F.: History and trends in solar irradiance and pv power forecasting: a preliminary assessment and review using text mining. Sol. Energy 168, 60–101 (2018)

    Article  Google Scholar 

  15. Voyant, C., Notton, G., Kalogirou, S., Nivet, M.-L., Paoli, C., Motte, F., Fouilloy, A.: Machine learning methods for solar radiation forecasting: a review. Renew. Energy 105, 569–582 (2017)

    Article  Google Scholar 

  16. Rocha, P.C., Fernandes, J., Modolo, A., Lima, R.P., da Silva, M.V., Bezerra, C.D.: Estimation of daily, weekly and monthly global solar radiation using ANNs and a long data set: a case study of Fortaleza, in brazilian northeast region. Int. J. Energy Environ. Eng. 10(3), 319–334 (2019)

    Article  Google Scholar 

  17. Marzo, A., Trigo-Gonzalez, M., Alonso-Montesinos, J., Martínez-Durbán, M., López, G., Ferrada, P., Fuentealba, E., Cortés, M., Batlles, F.: Daily global solar radiation estimation in desert areas using daily extreme temperatures and extraterrestrial radiation. Renew. Energy 113, 303–311 (2017)

    Article  Google Scholar 

  18. Cornejo-Bueno, L., Casanova-Mateo, C., Sanz-Justo, J., Salcedo-Sanz, S.: Machine learning regressors for solar radiation estimation from satellite data. Sol. Energy 183, 768–775 (2019)

    Article  Google Scholar 

  19. Chen, J., Zhu, W., Yu, Q.: Estimating half-hourly solar radiation over the continental United States using goes-16 data with iterative random forest. Renew. Energy 178, 916–929 (2021)

    Article  Google Scholar 

  20. Jebli, I., Belouadha, F.-Z., Kabbaj, M.I., Tilioua, A.: Prediction of solar energy guided by pearson correlation using machine learning. Energy 224, 120109 (2021)

    Article  Google Scholar 

  21. Fogno Fotso, H.R., Aloyem Kaz´e, C.V., Djuidje Kenmo´e, G.: A novel hybrid model based on weather variables relationships improving applied for wind speed forecasting. Int. J. Energy Environ. Eng. 13, 1–14 (2021)

    Google Scholar 

  22. Rodríguez, F., Azcárate, I., Vadillo, J., Galarza, A.: Forecasting intra-hour solar photovoltaic energy by assembling wavelet based time-frequency analysis with deep learning neural networks. Int. J. Electr. Power Energy Syst. 137, 107777 (2022)

    Article  Google Scholar 

  23. Bastos, B.Q., Oliveira, F.L.C., Milidiú, R.L.: U-convolutional model for spatio-temporal wind speed forecasting. Int. J. Forecast. 37(2), 949–970 (2021)

    Article  Google Scholar 

  24. Feng, C., Zhang, J.: Solarnet: a sky image-based deep convolutional neural network for intra-hour solar forecasting. Sol. Energy 204, 71–78 (2020)

    Article  Google Scholar 

  25. Huertas-Tato, J., Galván, I.M., Aler, R., Rodríguez-Benítez, F.J., Pozo-Vázquez, D.: Using a multi-view convolutional neural network to monitor solar irradiance. Neural Comput. Appl. (2021). https://doi.org/10.1007/s00521-021-05959-y

    Article  Google Scholar 

  26. Dong, N., Chang, J.-F., Wu, A.-G., Gao, Z.-K.: A novel convolutional neural network framework based solar irradiance prediction method. Int. J. Electr. Power Energy Syst. 114, 105411 (2020)

    Article  Google Scholar 

  27. Feng, C., Zhang, J., Zhang, W., Hodge, B.-M.: Convolutional neural networks for intra-hour solar forecasting based on sky image sequences. Appl. Energy 310, 118438 (2022)

    Article  Google Scholar 

  28. Ghimire, S., Deo, R.C., Raj, N., Mi, J.: Deep solar radiation forecasting with convolutional neural network and long short-term memory network algorithms. Appl. Energy 253, 113541 (2019)

    Article  Google Scholar 

  29. Zang, H., Liu, L., Sun, L., Cheng, L., Wei, Z., Sun, G.: Short-term global horizontal irradiance forecasting based on a hybrid cnn-lstm model with spatiotemporal correlations. Renew. Energy 160, 26–41 (2020)

    Article  Google Scholar 

  30. Kumari, P., Toshniwal, D.: Long short term memory–convolutional neural network based deep hybrid approach for solar irradiance forecasting. Appl. Energy 295, 117061 (2021)

    Article  Google Scholar 

  31. Akhter, M.N., Mekhilef, S., Mokhlis, H., Ali, R., Usama, M., Muhammad, M.A., Khairuddin, A.S.M.: A hybrid deep learning method for an hour ahead power output forecasting of three different photovoltaic systems. Appl. Energy 307, 118185 (2022)

    Article  Google Scholar 

  32. Sengupta, M., Xie, Y., Lopez, A., Habte, A., Maclaurin, G., Shelby, J.: The national solar radiation data base (NSRDB). Renew. Sustain. Energy Rev. 89, 51–60 (2018)

    Article  Google Scholar 

  33. Habte, A., Sengupta, M., Lopez, A.: Evaluation of the National Solar Radiation Database (NSRDB): 1998–2015. Technical report National Renewable Energy Lab(NREL), Golden (2017)

    Google Scholar 

  34. Yang, D.: Kriging for NSRDB PSM version 3 satellite-derived solar irradiance. Sol. Energy 171, 876–883 (2018)

    Article  Google Scholar 

  35. Yang, D.: A correct validation of the national solar radiation data base (NSRDB). Renew. Sustain. Energy Rev. 97, 152–155 (2018)

    Article  Google Scholar 

  36. Bright, J.M.: Solcast: validation of a satellite-derived solar irradiance dataset. Sol. Energy 189, 435–449 (2019)

    Article  Google Scholar 

  37. Yang, D., Bright, J.M.: Worldwide validation of 8 satellite-derived and reanalysis solar radiation products: a preliminary evaluation and overall metrics for hourly data over 27 years. Sol. Energy 210, 3–19 (2020)

    Article  Google Scholar 

  38. Hillger, D.W., Schmit, T.: Quantization noise for goes-r abi bands. In: Proceedings of the 13th Conference on Satellite Meteorology and Oceanography Norfolk, VA, pp. 20–23, American Meteorological Society (2004)

  39. Padula, F., Cao, C.: Cwg analysis: Abi max/min radiance characterization and validation. In: Proceedings of the 13th Conference on Satellite Meteorology and Oceanography Norfolk, VA, pp. 20–23, American Meteorological Society (2015)

  40. Kalluri, S., Alcala, C., Carr, J., Griffith, P., Lebair, W., Lindsey, D., Race, R., Wu, X., Zierk, S.: From photons to pixels: processing data from the advanced baseline imager. Remote Sens. 10(2), 177 (2018)

    Article  Google Scholar 

  41. Peel, M.C., Finlayson, B.L., McMahon, T.A.: Updated world map of the köppen-geiger climate classification. Hydrol. Earth Syst. Sci. 11(5), 1633–1644 (2007). https://doi.org/10.5194/hess-11-1633-2007

    Article  Google Scholar 

  42. Google Earth V7.3.4.8248: Geographic location of Petrolina SONDA station. 9°4’8”S, 40°19’11”W. Eye altitude 28 km. Maxar Technologies 2021, CNES/Airbus 2021 (2021). Available at: https://earth.google. com/web/. Accessed: 2021–08–31 (2021)

  43. Gao, B., Huang, X., Shi, J., Tai, Y., Zhang, J.: Hourly forecasting of solar irradiance based on ceemdan and multi-strategy cnn-lstm neural networks. Renew. Energy 162, 1665–1683 (2020)

    Article  Google Scholar 

  44. Huang, X., Li, Q., Tai, Y., Chen, Z., Zhang, J., Shi, J., Gao, B., Liu, W.: Hybrid deep neural model for hourly solar irradiance forecasting. Renew. Energy 171, 1041–1060 (2021)

    Article  Google Scholar 

  45. Liu, D., Niu, D., Wang, H., Fan, L.: Short-term wind speed forecasting using wavelet transform and support vector machines optimized by genetic algorithm. Renew. Energy 62, 592–597 (2014)

    Article  Google Scholar 

  46. Wang, H., Wang, G., Li, G., Peng, J., Liu, Y.: Deep belief network based deterministic and probabilistic wind speed forecasting approach. Appl. Energy 182, 80–93 (2016)

    Article  Google Scholar 

  47. Kuhn, M., Johnson, K.: Applied Predictive Modeling, vol. 26. Springer, New York (2013)

    Book  MATH  Google Scholar 

  48. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  49. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  50. Wang, Y., Pan, Z., Zheng, J., Qian, L., Li, M.: A hybrid ensemble method for pulsar candidate classification. Astrophys. Space Sci. 364(8), 1–13 (2019)

    Article  MathSciNet  Google Scholar 

  51. Chen, T., Guestrin, C.: Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd Acm Sigkdd International Conference on Knowledge Discovery and Data Mining, pp. 785–794 (2016)

  52. Bondi, A.B.: Characteristics of scalability and their impact on performance. In: Proceedings of the 2nd International Workshop on Software and Performance, pp. 195–203 (2000)

  53. Si, Z., Yang, M., Yu, Y., Ding, T.: Photovoltaic power forecast based on satellite images considering effects of solar position. Appl. Energy 302, 117514 (2021)

    Article  Google Scholar 

  54. Fan, J., Wang, X., Zhang, F., Ma, X., Wu, L.: Predicting daily diffuse horizontal solar radiation in various climatic regions of China using support vector machine and tree-based soft computing models with local and extrinsic climatic data. J. Clean Product 248, 119264 (2020)

    Article  Google Scholar 

  55. Koo, Y., Oh, M., Kim, S.-M., Park, H.-D.: Estimation and mapping of solar irradiance for korea by using coms mi satellite images and an artificial neural network model. Energies 13(2), 301 (2020)

    Article  Google Scholar 

  56. Kamath, H.G., Srinivasan, J.: Validation of global irradiance derived from insat-3d over India. Sol. Energy 202, 45–54 (2020)

    Article  Google Scholar 

  57. Nonnenmacher, L., Kaur, A., Coimbra, C.F.: Verification of the suny direct normal irradiance model with ground measurements. Sol. Energy 99, 246–258 (2014)

    Article  Google Scholar 

  58. Zhu, T., Xie, L., Wei, H., Wang, H., Zhao, X., Zhang, K.: Clear-sky direct normal irradiance estimation based on adjustable inputs and error correction. J. Renew. Sustain. Energy 11(5), 056101 (2019)

    Article  Google Scholar 

  59. Kuhn, P.M., Garsche, D., Wilbert, S., Nouri, B., Hanrieder, N., Prahl, C., Zarzarlejo, L., Fern´andez, J., Kazantzidis, A., Schmidt, T., et al.: Shadow-camera based solar nowcasting system for shortest-term forecasts. Meteorol. Z. (2019). https://doi.org/10.1127/metz/2019/0954

    Article  Google Scholar 

  60. Paulescu, M., Mares, O., Paulescu, E., Stefu, N., Pacurar, A., Calinoiu, D., Gravila, P., Pop, N., Boata, R.: Nowcasting solar irradiance using the sunshine number. Energy Convers. Manag. 79, 690–697 (2014)

    Article  Google Scholar 

  61. Nouri, B., Blum, N., Wilbert, S., Zarzalejo, L.F.: A hybrid solar irradiance nowcasting approach: combining all sky imager systems and persistence irradiance models for increased accuracy. Sol. RRL (2021). https://doi.org/10.1002/solr.202100442

    Article  Google Scholar 

Download references

Acknowledgements

This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior—Brasil (CAPES)—Finance Code (Grant No. 001) and accomplished with the support of the Conselho Nacional de Desenvolvimento Científico e Tecnológico—Brasil (CNPq)—(Grant No. 305456/2019-9), both Brazilian governmental agencies. The authors would also like to thank Prof. Carlos F. M. Coimbra, head of the UCSD’s Coimbra Research Group for the valuable insights and for kindly hosting one of the authors (Paulo A. C. Rocha), providing physical and computational resources to accomplish this research. The author Paulo A. C. Rocha would like to thank Hugo T. C. Pedro from the Coimbra Research Group too, for all the attention on discussing the obtained results and giving relevant information regarding the GOES-16 imagery.

Author information

Authors and Affiliations

  1. Solar Energy and Natural Gas Laboratory, Mechanical Engineering Department, Technology Center, Federal University of Ceará, Fortaleza, Ceará, 60020-181, Brazil

    Paulo A. C. Rocha & Victor Oliveira Santos

Authors
  1. Paulo A. C. Rocha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Victor Oliveira Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paulo A. C. Rocha.

Ethics declarations

Conflicts of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rocha, P.A.C., Santos, V.O. Global horizontal and direct normal solar irradiance modeling by the machine learning methods XGBoost and deep neural networks with CNN-LSTM layers: a case study using the GOES-16 satellite imagery. Int J Energy Environ Eng 13, 1271–1286 (2022). https://doi.org/10.1007/s40095-022-00493-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40095-022-00493-6

Keywords

Search

Navigation