# Comparative analysis on hidden neurons estimation in multi layer perceptron neural networks for wind speed forecasting

## Abstract

In this paper methodologies are proposed to estimate the number of hidden neurons that are to be placed numbers in the hidden layer of artificial neural networks (ANN) and certain new criteria are evolved for fixing this hidden neuron in multilayer perceptron neural networks. On the computation of the number of hidden neurons, the developed neural network model is applied for wind speed forecasting application. There is a possibility of over fitting or under fitting occurrence due to the random selection of hidden neurons in ANN model and this is addressed in this paper. Contribution is done in developing various 151 different criteria and the evolved criteria are tested for their validity employing various statistical error means. Simulation results prove that the proposed methodology minimized the computational error and enhanced the prediction accuracy. Convergence theorem is employed over the developed criterion to validate its applicability for fixing the number of hidden neurons. To evaluate the effectiveness of the proposed approach simulations were carried out on collected real-time wind data. Simulated results confirm that with minimum errors the presented approach can be utilized for wind speed forecasting. Comparative analysis has been performed for the estimation of the number of hidden neurons in multilayer perceptron neural networks. The presented approach is compact, enhances the accuracy rate with reduced error and faster convergence.

### Keywords

Hidden neurons Multilayer perceptron networks Wind speed forecasting Convergence theorem### References

- Arai M (1993) Bounds on the number of hidden units in binary-valued three-layer neural networks. Neural Netw 6:855–860CrossRefGoogle Scholar
- Choi B, Lee J-H, Kim D-H (2008) Solving local minima problem with large number of hidden nodes on two layered feed forward artificial neural networks. Neurocomputing 71:3640–3643CrossRefGoogle Scholar
- Dass HK (2009) Advanced engineering mathematics, 1st edn 1988. S. CHAND & Company Ltd, New DelhiGoogle Scholar
- Doukin CA, Dargham JA, Chekima A (2010) Finding the number of hidden neurons for an MLP neural network using coarse to fine search technique. In: 10th International conference on information sciences signal processing and their applications (ISSPA), pp 606–609Google Scholar
- Fujita O (1998) Statistical estimation of the number of hidden units for feed forward neural network. Neural Netw 11:851–859CrossRefGoogle Scholar
- Gnana Sheela K, Deepa SN (2013) Review on methods to fix number of hidden neurons in neural networks. Math Probl Eng 2013:1–11CrossRefGoogle Scholar
- Hagiwara M (1994) A simple and effective method for removal of hidden units and weights. Neuro Comput 6:207–218Google Scholar
- Han M, Yin J (2008) The hidden neurons selection of the wavelet networks using support vector machines and ridge regression. Neuro Comput 72:471–479Google Scholar
- Huang G-B (2003) Learning capability and storage capacity of two-hidden layer feed forward networks. IEEE Trans Neural Netw 14:274–281CrossRefGoogle Scholar
- Huang S-C, Huang Y-F (1991) Bounds on the number of hidden neurons in multilayer perceptrons. IEEE Trans Neural Netw 2:47–55CrossRefGoogle Scholar
- Hunter D, Hao Y, Pukish III MS, Kolbusz J, Wilamowski BM (2012) Selection of proper neural network sizes and architecture—a comparative study. IEEE Trans Ind Inf 8:228–240CrossRefGoogle Scholar
- Jaeger H, Haas H (2004) Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304(5667):78–80CrossRefGoogle Scholar
- Jiang N, Zhang Z, Ma X, Wang J (2008) The lower bound on the number of hidden neurons in multi-valued multi threshold neural networks. In: Second international symposium on intelligent information technology application, vol 1, pp 103–107Google Scholar
- Karsoliya S (2012) Approximating number of hidden layer neuron in multiple hidden layer BPNN architecture. Int J Eng Trends Technol 31:714–717Google Scholar
- Keeni K, Nakayama K, Shimodaira H (1999) Estimation of initial weights and hidden units for fast learning of multilayer neural networks for pattern classification. In: International joint conference on neural networks, vol 3, pp 1652–1656Google Scholar
- Ke J, Liu X (2008) Empirical analysis of optimal hidden neurons in neural network modeling for stock prediction. In: Pacific-Asia workshop on computational intelligence and industrial application, vol 2, pp 828–832Google Scholar
- Li J-Y, Chow TWS, Yu Y-L (1995) The estimation theory and optimization algorithm for the number of hidden units in the higher-order feed forward neural network. In: Proceeding IEEE international conference on neural networks, vol 3, pp 1229–1233Google Scholar
- Li J, Zhang B, Mao C, Xie G, Li Y, Lu J (2010) Wind speed prediction based on the Elman recursion neural networks. In: International conference on modelling, identification and control, pp 728–732Google Scholar
- Madhiarasan M, Deepa SN (2016) A novel criterion to select hidden neuron numbers in improved back propagation networks for wind speed forecasting. Appl Intell 44(4):878–893CrossRefGoogle Scholar
- Mao KZ, Huang G-B (2005) Neuron selection for RBF neural network classifier based on data structure preserving criterion. IEEE Trans Neural Netw 16:1531–1540CrossRefGoogle Scholar
- Meng A, Ge J, Yin H, Chen S (2016) Wind speed forecasting based on wavelet packet decomposition and artificial neural networks trained by crisscross optimization algorithm. Energy Convers Manag 114:75–88CrossRefGoogle Scholar
- Morris AJ, Zhang J (1998) A sequential learning approach for single hidden layer neural network. Neural Netw 11:65–80CrossRefGoogle Scholar
- Murata N, Yoshizawa S, Amari S-I (1994) Network information criterion determining the number of hidden units for an artificial neural network model. IEEE Trans Neural Netw 5:865–872CrossRefGoogle Scholar
- Onoda T (1995) Neural network information criterion for the optimal number of hidden units. In: Proceeding IEEE international conference on neural networks, vol 1, pp 275–280Google Scholar
- Panchal G, Ganatra A, Kosta YP, Panchal D (2011) Behaviour analysis of multilayer perceptrons with multiple hidden neurons and hidden layers. Int J Comput Theory Eng 3:332–337CrossRefGoogle Scholar
- Peter SE, Kulkarni S, Raglend IJ, Simon SP (2013) Wavelet based spike propagation neural network (WSPNN) for wind power forecasting. Int Rev Model Simul (IREMOS) 6(5):1513–1522Google Scholar
- Qian G, Yong H (2013) Forecasting the rural per capita living consumption based on Matlab BP neural network. Int J Bus Soc Sci 4:131–137Google Scholar
- Ramadevi R, Sheela Rani B, Prakash V (2012) Role of hidden neurons in an Elman recurrent neural network in classification of cavitation signals. Int J Comput Appl 37:9–13Google Scholar
- Shibata K, Ikeda Y (2009) Effect of number of hidden neurons on learning in large-scale layered neural networks. In: ICROS-SICE international joint conference, pp 5008–5013Google Scholar
- Sivanandam SN, Sumathi S, Deepa SN (2008) Introduction to neural networks using Matlab 6.0, 1st edn. Tata McGraw Hill, New DelhiGoogle Scholar
- Sun J (2012) Learning algorithm and hidden node selection scheme for local coupled feed forward neural network classifier. Neuro Comput 79:158–163Google Scholar
- Tamura S, Tateishi M (1997) Capabilities of a four-layered feed forward neural network: four layer versus three. IEEE Trans Neural Netw 8:251–255CrossRefGoogle Scholar
- Teoh EJ, Tan KC, Xiang C (2006) Estimating the number of hidden neurons in a feed forward network using the singular value decomposition. IEEE Trans Neural Netw 17:1623–1629CrossRefGoogle Scholar
- Trenn S (2008) Multilayer perceptrons: approximation order and necessary number of hidden units. IEEE Trans Neural Netw 19:836–844CrossRefGoogle Scholar
- Urolagin S, Prema KV, Subba Reddy NV (2012) Generalization capability of artificial neural network incorporated with pruning method. Lect Notes Comput Sci 7135:171–178CrossRefGoogle Scholar
- Vora K, Yagnik S (2014) A new technique to solve local minima problem with large number of hidden nodes on feed forward neural network. Int J Eng Dev Res 2:1978–1981Google Scholar
- Wang J, Hu J (2015) A robust combination approach for short-term wind speed forecasting and analysis—combination of the ARIMA (autoregressive integrated moving average), ELM (extreme learning machine), SVM (support vector machine) and LSSVM (least square SVM) forecasts using a GPR (Gaussian process regression) model. Energy 93:41–56CrossRefGoogle Scholar
- Xu S, Chen L (2008) A novel approach for determining the optimal number of hidden layer neurons for FNN’s and its application in data mining. In: 5th International conference on information technology and application (ICITA), pp 683–686Google Scholar
- Yuan HC, Xiong FL, Huai XY (2003) A method for estimating the number of hidden neurons in feed-forward neural networks based on information entropy. Comput Electron Agric 40:57–64CrossRefGoogle Scholar
- Zeng X, Yeung DS (2006) Hidden neuron purning of multilayer perceptrons using a quantified sensitivity measure. Neuro Comput 69:825–837Google Scholar
- Zhang Z, Ma X, Yang Y (2003) Bounds on the number of hidden neurons in three-layer binary neural networks. Neural Netw 16:995–1002CrossRefGoogle Scholar