KloudNet: Deep Learning for Sky Image Analysis and Irradiance Forecasting

  • Dinesh Pothineni
  • Martin R. OswaldEmail author
  • Jan Poland
  • Marc Pollefeys
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11269)


We present a novel image-based approach for estimating irradiance fluctuations from sky images. Our goal is a very short-term prediction of the irradiance state around a photovoltaic power plant 5–10 min ahead of time, in order to adjust alternative energy sources and ensure a stable energy network. To this end, we propose a convolutional neural network with residual building blocks that learns to predict the future irradiance state from a small set of sky images. Our experiments on two large datasets demonstrate that the network abstracts upon local site-specific properties such as day- and month-dependent sun positions, as well as generic properties about moving, creating, dissolving clouds, or seasonal changes. Moreover, our approach significantly outperforms the established baseline and state-of-the-art methods.



This work received funding from the Horizon 2020 research and innovation programme under grant No. 637221 (Built2Spec).


  1. 1.
    Reuniwatt. Accessed 03 July 2018
  2. 2.
    SteadySun. Accessed 03 July 2018
  3. 3.
    Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016)
  4. 4.
    Ai, Y., Peng, Y., Wei, W.: A model of very short-term solar irradiance forecasting based on low-cost sky images. In: American Institute of Physics Conference Proceedings (2017)Google Scholar
  5. 5.
    Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (1994)CrossRefGoogle Scholar
  6. 6.
    Borkowski, J., Chai, A.T., Mo, T., Green, A.E.O.: Cloud effects on middle ultraviolet global radiation. Acta Geophysica Polonica 25(4), 287–301 (1977)Google Scholar
  7. 7.
    Chauvin, R., Nou, J., Thil, S., Traoré, A., Grieu, S.: Cloud detection methodology based on a sky-imaging system. Energy Procedia 69, 1970–1980 (2015). International Conference on Concentrating Solar Power and Chemical Energy Systems, SolarPACES, p. 2014CrossRefGoogle Scholar
  8. 8.
    Chollet, F., et al.: Keras (2015).
  9. 9.
    Chow, C., et al.: Intra-hour forecasting with a total sky imager at the UC San Diego solar energy testbed. Solar Energy 85, 2881–2893 (2011)CrossRefGoogle Scholar
  10. 10.
    Diagne, H.M., David, M., Lauret, P., Boland, J., Schmutz, N.: Review of solar irradiance forecasting methods and a proposition for small-scale insular grids. Renew. Sustain. Energy Rev. 27, 65–76 (2013)CrossRefGoogle Scholar
  11. 11.
    Elhoseiny, M., Huang, S., Elgammal, A.: Weather classification with deep convolutional neural networks. In: International Conference on Image Processing (2015)Google Scholar
  12. 12.
    Gevorgian, V., Booth, S.: Review of PREPA technical requirements for interconnecting wind and solar generation. Technical Report NREL/TP-5D00-57089, National Renewable Energy Laboratory (2013)Google Scholar
  13. 13.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016).
  14. 14.
    He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). Scholar
  15. 15.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). Scholar
  16. 16.
    Larraondo, P.R., Inza, I., Lozano, J.A.: Automating weather forecasts based on convolutional networks. In: ICML 17 Workshop on Deep Structured Prediction (2017)Google Scholar
  17. 17.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: ICML, pp. 448–456 (2015)Google Scholar
  18. 18.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: NIPS, pp. 1106–1114 (2012)Google Scholar
  19. 19.
    Lu, S., et al.: Machine learning based multi-physical-model blending for enhancing renewable energy forecast - improvement via situation dependent error correction. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 31–45 (2015)Google Scholar
  20. 20.
    Magnone, L., Sossan, F., Scolari, E., Paolone, M.: Cloud motion identification algorithms based on all-sky images to support solar irradiance forecast. In: Photovoltaic Specialists Conference (2017)Google Scholar
  21. 21.
    Marquez, R., Coimbra, C.F.: Intra-hour DNI forecasting based on cloud tracking image analysis. Solar Energy 91, 327–336 (2013)CrossRefGoogle Scholar
  22. 22.
    Mellit, A.: Artificial intelligence technique for modelling and forecasting of solar radiation data - a review. Int. J. Artif. Intell. Soft Comput. 1, 52–76 (2008)CrossRefGoogle Scholar
  23. 23.
    Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)CrossRefGoogle Scholar
  24. 24.
    Nguyen, H.T., Pearce, J.M.: Estimating potential photovoltaic yield with r.sun and the open source geographical resources analysis support system. Solar energy 84(5), 831–843 (2010)CrossRefGoogle Scholar
  25. 25.
    Onishi, R., Sugiyama, D.: Deep convolutional neural network for cloud coverage estimation from snapshot camera images. SOL Atmos. 13, 235–239 (2017)Google Scholar
  26. 26.
    Pelland, S., et al.: Photovoltaic and solar forecasting: state of the art. Technical report IEA PVPS T14–01:2013, International Energy Agency, October 2013Google Scholar
  27. 27.
    Perez, R., et al.: A new operational model for satellite-derived irradiances: description and validation. Solar Energy 73(5), 307–317 (2002)CrossRefGoogle Scholar
  28. 28.
    Richardson, W., Krishnaswami, H., Vega, R., Cervantes, M.: A low cost, edge computing, all-sky imager for cloud tracking and intra-hour irradiance forecasting. Sustainability 9, 1–17 (2017)CrossRefGoogle Scholar
  29. 29.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  30. 30.
    Ross, S., Gordon, G., Bagnell, D.: A reduction of imitation learning and structured prediction to no-regret online learning. In: Gordon, G., Dunson, D., Dudík, M. (eds.) International Conference on Artificial Intelligence and Statistics, vol. 15, pp. 627–635 (2011)Google Scholar
  31. 31.
    Sabburg, J., Wong, J.: Evaluation of a ground-based sky camera system for use in surface irradiance measurement. J. Atmos. Oceanic Technol. 16, 752–759 (1998)CrossRefGoogle Scholar
  32. 32.
    Schmidt, T., et al.: Short-term solar forecasting based on sky images to enable higher PV generation in remote electricity networks. Renew. Energy Environ. Sustain. 2, 23 (2017)CrossRefGoogle Scholar
  33. 33.
    Schmidt, T., Kalisch, J., Lorenz, E., Heinemann, D.: Evaluating the spatio-temporal performance of sky-imager-based solar irradiance analysis and forecasts. Atmos. Chem. Phys. 16, 3399–3412 (2016)CrossRefGoogle Scholar
  34. 34.
    Scolari, E., Sossan, F., Paolone, M.: Irradiance prediction intervals for PV stochastic generation in microgrid applications. Solar Energy 139, 116–129 (2016)CrossRefGoogle Scholar
  35. 35.
    Scolari, E., Sossan, F., Paolone, M.: Photovoltaic-model-based solar irradiance estimators: performance comparison and application to maximum power forecasting. IEEE Trans. Sustain. Energy 9, 35–44 (2018)CrossRefGoogle Scholar
  36. 36.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014).
  37. 37.
    Szegedy, C., et al.: Going deeper with convolutions. In: CVPR (2015)Google Scholar
  38. 38.
    Wolff, B., Kramer, O., Heinemann, D.: Selection of numerical weather forecast features for PV power predictions with random forests. In: Data Analytics for Renewable Energy Integration - 4th ECML PKDD Workshop, pp. 78–91 (2016)CrossRefGoogle Scholar
  39. 39.
    Wolff, B., Lorenz, E., Kramer, O.: Statistical learning for short-term photovoltaic power predictions. In: Lässig, J., Kersting, K., Morik, K. (eds.) Computational Sustainability. SCI, vol. 645, pp. 31–45. Springer, Cham (2016). Scholar
  40. 40.
    Yadav, A.K., Chandel, S.: Solar radiation prediction using artificial neural network techniques: a review. Renew. Sustain. Energy Rev. 33, 772–781 (2014)CrossRefGoogle Scholar
  41. 41.
    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Dinesh Pothineni
    • 1
  • Martin R. Oswald
    • 1
    Email author
  • Jan Poland
    • 2
  • Marc Pollefeys
    • 1
  1. 1.ETH ZürichZürichSwitzerland
  2. 2.ABB Corporate ResearchBadenSwitzerland

Personalised recommendations