Random Hyper-parameter Search-Based Deep Neural Network for Power Consumption Forecasting
- 1.4k Downloads
In this paper, we introduce a deep learning approach, based on feed-forward neural networks, for big data time series forecasting with arbitrary prediction horizons. We firstly propose a random search to tune the multiple hyper-parameters involved in the method performance. There is a twofold objective for this search: firstly, to improve the forecasts and, secondly, to decrease the learning time. Next, we propose a procedure based on moving averages to smooth the predictions obtained by the different models considered for each value of the prediction horizon. We conduct a comprehensive evaluation using a real-world dataset composed of electricity consumption in Spain, evaluating accuracy and comparing the performance of the proposed deep learning with a grid search and a random search without applying smoothing. Reported results show that a random search produces competitive accuracy results generating a smaller number of models, and the smoothing process reduces the forecasting error.
KeywordsHyperparameters Time series forecasting Deep learning
The authors would like to thank the Spanish Ministry of Economy and Competitiveness for the support under the project TIN2017-88209-C2-1-R.
- 1.Bergstra, J., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyper-parameter optimization. In: Proceedings of the 24th International Conference on Neural Information Processing Systems, NIPS’11, pp. 2546–2554. Curran Associates Inc., New York (2011)Google Scholar
- 3.Candel, A., LeDell, E., Parmar, V., Arora, A.: Deep learning with H2O. H2O.ai, Inc., California (2017)Google Scholar
- 5.Dalto, M., Matusko, J., Vasak, M.: Deep neural networks for ultra-short-term wind forecasting. In: Proceedings of the IEEE International Conference on Industrial Technology (ICIT), pp. 1657–1663 (2015)Google Scholar
- 7.Ding, X., Zhang, Y., Liu, T., Duan, J.: Deep learning for event-driven stock prediction. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 2327–2334 (2015)Google Scholar
- 8.Ilievski, I., Akhtar, T., Feng, J., Shoemaker, C.A.: Efficient hyperparameter optimization for deep learning algorithms using deterministic RBF surrogates. In: Proceedings of the AAAI Conference on Artificial Intelligence (2017)Google Scholar
- 9.Klein, A., Falkner, S., Bartels, S., Hennig, P., Hutter, F.: Fast bayesian optimization of machine learning hyperparameters on large datasets. CoRR abs/1605.07079 (2016)Google Scholar
- 11.Loshchilov, I., Hutter, F.: CMA-ES for hyperparameter optimization of deep neural networks. arXiv preprint arXiv:1604.07269 (2016)
- 13.R Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2013). http://www.R-project.org/. ISBN 3-900051-07-0
- 14.Ruder, S.: An overview of gradient descent optimization algorithms. CoRR abs/1609.04747 (2016)Google Scholar
- 16.Young, S.R., Rose, D.C., Karnowski, T.P., Lim, S.H., Patton, R.M.: Optimizing deep learning hyper-parameters through an evolutionary algorithm. In: Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments, p. 4. ACM, New York (2015)Google Scholar