Artificial Intelligence Review

, Volume 48, Issue 4, pp 449–471 | Cite as

Comparative analysis on hidden neurons estimation in multi layer perceptron neural networks for wind speed forecasting

Article

Abstract

In this paper methodologies are proposed to estimate the number of hidden neurons that are to be placed numbers in the hidden layer of artificial neural networks (ANN) and certain new criteria are evolved for fixing this hidden neuron in multilayer perceptron neural networks. On the computation of the number of hidden neurons, the developed neural network model is applied for wind speed forecasting application. There is a possibility of over fitting or under fitting occurrence due to the random selection of hidden neurons in ANN model and this is addressed in this paper. Contribution is done in developing various 151 different criteria and the evolved criteria are tested for their validity employing various statistical error means. Simulation results prove that the proposed methodology minimized the computational error and enhanced the prediction accuracy. Convergence theorem is employed over the developed criterion to validate its applicability for fixing the number of hidden neurons. To evaluate the effectiveness of the proposed approach simulations were carried out on collected real-time wind data. Simulated results confirm that with minimum errors the presented approach can be utilized for wind speed forecasting. Comparative analysis has been performed for the estimation of the number of hidden neurons in multilayer perceptron neural networks. The presented approach is compact, enhances the accuracy rate with reduced error and faster convergence.

Keywords

Hidden neurons Multilayer perceptron networks Wind speed forecasting Convergence theorem 

References

  1. Arai M (1993) Bounds on the number of hidden units in binary-valued three-layer neural networks. Neural Netw 6:855–860CrossRefGoogle Scholar
  2. Choi B, Lee J-H, Kim D-H (2008) Solving local minima problem with large number of hidden nodes on two layered feed forward artificial neural networks. Neurocomputing 71:3640–3643CrossRefGoogle Scholar
  3. Dass HK (2009) Advanced engineering mathematics, 1st edn 1988. S. CHAND & Company Ltd, New DelhiGoogle Scholar
  4. Doukin CA, Dargham JA, Chekima A (2010) Finding the number of hidden neurons for an MLP neural network using coarse to fine search technique. In: 10th International conference on information sciences signal processing and their applications (ISSPA), pp 606–609Google Scholar
  5. Fujita O (1998) Statistical estimation of the number of hidden units for feed forward neural network. Neural Netw 11:851–859CrossRefGoogle Scholar
  6. Gnana Sheela K, Deepa SN (2013) Review on methods to fix number of hidden neurons in neural networks. Math Probl Eng 2013:1–11CrossRefGoogle Scholar
  7. Hagiwara M (1994) A simple and effective method for removal of hidden units and weights. Neuro Comput 6:207–218Google Scholar
  8. Han M, Yin J (2008) The hidden neurons selection of the wavelet networks using support vector machines and ridge regression. Neuro Comput 72:471–479Google Scholar
  9. Huang G-B (2003) Learning capability and storage capacity of two-hidden layer feed forward networks. IEEE Trans Neural Netw 14:274–281CrossRefGoogle Scholar
  10. Huang S-C, Huang Y-F (1991) Bounds on the number of hidden neurons in multilayer perceptrons. IEEE Trans Neural Netw 2:47–55CrossRefGoogle Scholar
  11. Hunter D, Hao Y, Pukish III MS, Kolbusz J, Wilamowski BM (2012) Selection of proper neural network sizes and architecture—a comparative study. IEEE Trans Ind Inf 8:228–240CrossRefGoogle Scholar
  12. Jaeger H, Haas H (2004) Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304(5667):78–80CrossRefGoogle Scholar
  13. Jiang N, Zhang Z, Ma X, Wang J (2008) The lower bound on the number of hidden neurons in multi-valued multi threshold neural networks. In: Second international symposium on intelligent information technology application, vol 1, pp 103–107Google Scholar
  14. Karsoliya S (2012) Approximating number of hidden layer neuron in multiple hidden layer BPNN architecture. Int J Eng Trends Technol 31:714–717Google Scholar
  15. Keeni K, Nakayama K, Shimodaira H (1999) Estimation of initial weights and hidden units for fast learning of multilayer neural networks for pattern classification. In: International joint conference on neural networks, vol 3, pp 1652–1656Google Scholar
  16. Ke J, Liu X (2008) Empirical analysis of optimal hidden neurons in neural network modeling for stock prediction. In: Pacific-Asia workshop on computational intelligence and industrial application, vol 2, pp 828–832Google Scholar
  17. Li J-Y, Chow TWS, Yu Y-L (1995) The estimation theory and optimization algorithm for the number of hidden units in the higher-order feed forward neural network. In: Proceeding IEEE international conference on neural networks, vol 3, pp 1229–1233Google Scholar
  18. Li J, Zhang B, Mao C, Xie G, Li Y, Lu J (2010) Wind speed prediction based on the Elman recursion neural networks. In: International conference on modelling, identification and control, pp 728–732Google Scholar
  19. Madhiarasan M, Deepa SN (2016) A novel criterion to select hidden neuron numbers in improved back propagation networks for wind speed forecasting. Appl Intell 44(4):878–893CrossRefGoogle Scholar
  20. Mao KZ, Huang G-B (2005) Neuron selection for RBF neural network classifier based on data structure preserving criterion. IEEE Trans Neural Netw 16:1531–1540CrossRefGoogle Scholar
  21. Meng A, Ge J, Yin H, Chen S (2016) Wind speed forecasting based on wavelet packet decomposition and artificial neural networks trained by crisscross optimization algorithm. Energy Convers Manag 114:75–88CrossRefGoogle Scholar
  22. Morris AJ, Zhang J (1998) A sequential learning approach for single hidden layer neural network. Neural Netw 11:65–80CrossRefGoogle Scholar
  23. Murata N, Yoshizawa S, Amari S-I (1994) Network information criterion determining the number of hidden units for an artificial neural network model. IEEE Trans Neural Netw 5:865–872CrossRefGoogle Scholar
  24. Onoda T (1995) Neural network information criterion for the optimal number of hidden units. In: Proceeding IEEE international conference on neural networks, vol 1, pp 275–280Google Scholar
  25. Panchal G, Ganatra A, Kosta YP, Panchal D (2011) Behaviour analysis of multilayer perceptrons with multiple hidden neurons and hidden layers. Int J Comput Theory Eng 3:332–337CrossRefGoogle Scholar
  26. Peter SE, Kulkarni S, Raglend IJ, Simon SP (2013) Wavelet based spike propagation neural network (WSPNN) for wind power forecasting. Int Rev Model Simul (IREMOS) 6(5):1513–1522Google Scholar
  27. Qian G, Yong H (2013) Forecasting the rural per capita living consumption based on Matlab BP neural network. Int J Bus Soc Sci 4:131–137Google Scholar
  28. Ramadevi R, Sheela Rani B, Prakash V (2012) Role of hidden neurons in an Elman recurrent neural network in classification of cavitation signals. Int J Comput Appl 37:9–13Google Scholar
  29. Shibata K, Ikeda Y (2009) Effect of number of hidden neurons on learning in large-scale layered neural networks. In: ICROS-SICE international joint conference, pp 5008–5013Google Scholar
  30. Sivanandam SN, Sumathi S, Deepa SN (2008) Introduction to neural networks using Matlab 6.0, 1st edn. Tata McGraw Hill, New DelhiGoogle Scholar
  31. Sun J (2012) Learning algorithm and hidden node selection scheme for local coupled feed forward neural network classifier. Neuro Comput 79:158–163Google Scholar
  32. Tamura S, Tateishi M (1997) Capabilities of a four-layered feed forward neural network: four layer versus three. IEEE Trans Neural Netw 8:251–255CrossRefGoogle Scholar
  33. Teoh EJ, Tan KC, Xiang C (2006) Estimating the number of hidden neurons in a feed forward network using the singular value decomposition. IEEE Trans Neural Netw 17:1623–1629CrossRefGoogle Scholar
  34. Trenn S (2008) Multilayer perceptrons: approximation order and necessary number of hidden units. IEEE Trans Neural Netw 19:836–844CrossRefGoogle Scholar
  35. Urolagin S, Prema KV, Subba Reddy NV (2012) Generalization capability of artificial neural network incorporated with pruning method. Lect Notes Comput Sci 7135:171–178CrossRefGoogle Scholar
  36. Vora K, Yagnik S (2014) A new technique to solve local minima problem with large number of hidden nodes on feed forward neural network. Int J Eng Dev Res 2:1978–1981Google Scholar
  37. Wang J, Hu J (2015) A robust combination approach for short-term wind speed forecasting and analysis—combination of the ARIMA (autoregressive integrated moving average), ELM (extreme learning machine), SVM (support vector machine) and LSSVM (least square SVM) forecasts using a GPR (Gaussian process regression) model. Energy 93:41–56CrossRefGoogle Scholar
  38. Xu S, Chen L (2008) A novel approach for determining the optimal number of hidden layer neurons for FNN’s and its application in data mining. In: 5th International conference on information technology and application (ICITA), pp 683–686Google Scholar
  39. Yuan HC, Xiong FL, Huai XY (2003) A method for estimating the number of hidden neurons in feed-forward neural networks based on information entropy. Comput Electron Agric 40:57–64CrossRefGoogle Scholar
  40. Zeng X, Yeung DS (2006) Hidden neuron purning of multilayer perceptrons using a quantified sensitivity measure. Neuro Comput 69:825–837Google Scholar
  41. Zhang Z, Ma X, Yang Y (2003) Bounds on the number of hidden neurons in three-layer binary neural networks. Neural Netw 16:995–1002CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  1. 1.Anna University, Regional CampusCoimbatoreIndia

Personalised recommendations