Abstract
Since their inception, convolutional neural networks (CNNs) have been shown to have powerful feature extraction and learning capabilities, and the creation of deep residual networks (DRNs) was a milestone in the development of CNNs. However, residual networks mostly use convolution structures, which are widely applied to image recognition and classification problems. Therefore, when facing a load forecasting problem that involves nonlinear regression, will a DRN using a convolution structure still achieve great results? To answer this question, we present a network based on a DRN with a convolution structure to carry out short-term load forecasting, and we mainly focus on the effects of DRNs with different depths, widths and block structures for dealing with nonlinear regression problems. Through multiple sets of controlled experiments, we obtain the best network architecture and the corresponding hyperparameters for short-term load forecasting. The experimental results demonstrate that the model has higher prediction accuracy than existing models, and the DRN with a convolution structure can handle load forecasting while still achieving state-of-the-art results.
Similar content being viewed by others
Notes
In this paper, based on different datasets, we consider different holidays. In the ISO-NE dataset, we consider Christmas Eve, Thanksgiving Day, and Independence Day as major holidays. In the Malaysia dataset, we consider Spring Festival, Eid, and Christmas Eve as major holidays. To simplify the model, the rest of the holidays are considered as nonholidays.
Available at https://data.mendeley.com/datasets/f4fcrh4tn9/1
References
Pang Q, Zhang M (2010) Very short-term load forecasting based on neural network and rough set. In: 2010 international conference on intelligent computation technology and automation, vol 3, IEEE, pp 1132–1135
Chen K, Chen K, Wang Q, He Z, Hu J, He J (2018) Short-term load forecasting with deep residual networks. IEEE Transactions on Smart Grid
Feilat E, Bouzguenda M (2011) Medium-term load forecasting using neural network approach. In: 2011 IEEE PES Conference on Innovative Smart Grid Technologies-Middle East, IEEE, pp 1–5
Nalcaci G, Özmen A, Weber GW (2019) Long-term load forecasting: models based on mars, ann and lr methods. CEJOR 27(4):1033–1049
Li C, Li S, Liu Y (2016) A least squares support vector machine model optimized by moth-flame optimization algorithm for annual power load forecasting. Appl Intell 45(4):1166–1178
Xu W, Peng H, Zeng X, Zhou F, Tian X, Peng X (2019) A hybrid modelling method for time series forecasting based on a linear regression model and deep learning. Applied Intelligence
Hernández L, Baladrón C, Aguiar JM, Carro B, Sánchez-Esguevillas A, Lloret J (2014) Artificial neural networks for short-term load forecasting in microgrids environment. Energy 75:252–264
Panapakidis IP (2016) Application of hybrid computational intelligence models in short-term bus load forecasting. Expert Systems with Applications 54:105–120
Sengupta S, Basak S, Saikia P, Paul S, II RAP (2019) A review of deep learning with special emphasis on architectures, applications and recent trends
Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, Liu T, Wang X, Wang G, Cai J et al (2018) Recent advances in convolutional neural networks. Pattern Recogn 77:354–377
Di Persio L, Honchar O (2017) Analysis of recurrent neural networks for short-term energy load forecasting. In: AIP Conference Proceedings, vol 1906, AIP Publishing, pp 190006
Staudemeyer RC, Morris ER (2019) Understanding lstm–a tutorial into long short-term memory recurrent neural networks. arXiv:1909.09586
Zhao L, Zhou Y, Lu H, Fujita H (2018) Parallel computing method of deep belief networks and its application to traffic flow prediction. Knowledge Based Systems
Kong W, Dong ZY, Jia Y, Hill DJ, Xu Y, Zhang Y (2017) Short-term residential load forecasting based on lstm recurrent neural network. IEEE Trans Smart Grid 10(1):841–851
Duvenaud D, Rippel O, Adams R, Ghahramani Z (2014) Avoiding pathologies in very deep networks. In: Artificial intelligence and statistics, pp 202–210
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Xu B, Wang N, Chen T, Li M (2015) Empirical evaluation of rectified activations in convolutional network. arXiv:1505.00853
Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580
Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167
Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis (IJCV) 115(3):211–252
Liu T, Chen M, Zhou M, Du SS, Zhou E, Zhao T (2019) Towards understanding the importance of shortcut connections in residual networks. In: Advances in neural information processing systems, pp 7890–7900
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9
Zagoruyko S, Komodakis N (2016) Wide residual networks. arXiv:1605.07146
Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1492–1500
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708
He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision, Springer, pp 630–645
Lin M, Chen Q, Yan S (2013) Network in network. arXiv:1312.4400
Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp 315–323
Huang G, Li Y, Pleiss G, Liu Z, Hopcroft JE, Weinberger KQ (2017) Snapshot ensembles:, Train 1, get m for free. arXiv:1704.00109
Sadaei HJ, Silva PCDLE, Guimaraes FG, Lee MH (2019) Short-term load forecasting by using a combined method of convolutional neural networks and fuzzy time series. Energy 175:365–377
Acknowledgements
This work was supported in part by the Natural Science Foundation of China under Grant 61803056, in part by China Postdoctoral Science Foundation under Grant 2017M620374, in part by the Natural Science Foundation of Chongqing cstc2018jcyjAX0365, and in part by Fundamental Research Funds for the Central Universities under Grant XDJK2018B013. The statements made herein are solely the responsibility of the authors.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sheng, Z., Wang, H., Chen, G. et al. Convolutional residual network to short-term load forecasting. Appl Intell 51, 2485–2499 (2021). https://doi.org/10.1007/s10489-020-01932-9
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-020-01932-9