Kernelized LARS–LASSO for constructing radial basis function neural networks
- 406 Downloads
Model structure selection is of crucial importance in radial basis function (RBF) neural networks. Existing model structure selection algorithms are essentially forward selection or backward elimination methods that may lead to sub-optimal models. This paper proposes an alternative selection procedure based on the kernelized least angle regression (LARS)–least absolute shrinkage and selection operator (LASSO) method. By formulating the RBF neural network as a linear-in-the-parameters model, we derive a l 1-constrained objective function for training the network. The proposed algorithm makes it possible to dynamically drop a previously selected regressor term that is insignificant. Furthermore, inspired by the idea of LARS, the computing of output weights in our algorithm is greatly simplified. Since our proposed algorithm can simultaneously conduct model structure selection and parameter optimization, a network with better generalization performance is built. Computational experiments with artificial and real world data confirm the efficacy of the proposed algorithm.
KeywordsKernelized LARS–LASSO Radial basis function (RBF) Mode structure selection
This paper is supported by the National Natural Science Foundation of China under Grant 61273233, the Research Foundation for the Doctoral Program of Higher Education under Grant 20090002110035, the Project of China Ocean Association under Grant DY125-25-02, and Tsinghua University Initiative Scientific Research Program under Grants 2010THZ07002 and 2011THZ07132.
- 9.Frank A, Asuncion A (2010) UCI machine learning repository. URL: http://archive.ics.uci.edu/ml10
- 10.Furnival G, Wilson R Jr (1974) Regressions by leaps and bounds. Technometrics 16(4):499–511Google Scholar
- 13.Huang G, Song S, Wu C (2012) Orthogonal least squares algorithm for training cascade neural networks. IEEE Trans Circuits Syst I Regul Pap 59(12). doi: 10.1109/TCSI.2012.2189060
- 16.Liu Z, Li W, Sun W (2011) A novel method of short-term load forecasting based on multiwavelet transform and multiple neural networks. Neural Comput Appl 1–7. doi: 10.1007/s00521-011-0715-2
- 19.Qian D, Zhao D, Yi J, Liu X (2011) Neural sliding-mode load frequency controller design of power systems. Neural Comput Appl 1–8. doi: 10.1007/s00521-011-0709-0
- 21.Sheikhan M, Pardis R, Gharavian D (2012) State of charge neural computational models for high energy density batteries in electric vehicles. Neural Comput Appl 1–10. doi: 10.1007/s00521-012-0883-8
- 23.Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B (Methodol) 58(1):267–288Google Scholar
- 25.Wang G, Yeung D, Lochovsky F (2007) The kernel path in kernelized lasso. In: International conference on artificial intelligence and statistics, pp 580–587Google Scholar