A Novel Input Stochastic Sensitivity Definition of Radial Basis Function Neural Networks and Its Application to Feature Selection
For a well-trained radial basis function neural network, this paper proposes a novel input stochastic sensitivity definition and gives its computational formula assuming the inputs are modelled by normal distribution random variables. Based on this formula, one can calculate the magnitude of sensitivity for each input (i.e. feature), which indicates the degree of importance of input to the output of neural network. When there are redundant inputs in the training set, one always wants to remove those redundant features to avoid a large network. This paper shows that removing redundant features or selecting significant features can be completed by choosing features with sensitivity over a predefined threshold. Numerical experiment shows that the new approach to feature selection performs well.
KeywordsFeature Selection Hide Neuron Radial Basis Function Neural Network Computational Formula Redundant Feature
Unable to display preview. Download preview PDF.
- 1.Hoff, M.E.: Learning Phenomena in Networks of Adaptive Switching Circuits. Technical Report 1554-1 of Stanford Electronic Laboratory, Stanford, CA (1962)Google Scholar
- 5.Ng, W.W.Y., Yeung, D.S., Ran, Q., Tsang, E.C.C.: Statistical Output Sensitivity to Input and Weight Perturbations of Radial Basis Function Neural Networks. In: Proceedings of IEEE on System, Man and Cybernetics, vol. 2, pp. 503–508 (2002)Google Scholar
- 6.Ng, W.W.Y., Yeung, D.S.: Input Dimensionality Reduction for Radial Basis Neural Network Classification Problems Using Sensitivity Measure. In: Proceedings of the First International Conference on Machine Learning and Cybernetics, Beijing, China, vol. 4, pp. 2214–2219 (2002)Google Scholar
- 7.Ng, W.W.Y., Yeung, D.S., Cloete, I.: Input Sample Selection for RBF Neural Network Classification Problems Using Sensitivity Measure. In: Proceedings of IEEE on System, Man and Cybernetics, vol. 3, pp. 2593–2598 (2003)Google Scholar
- 12.Halmos, P.R.: Measure Theory, New York (1950)Google Scholar
- 14.Wang, X.Z., Zhang, H.: An Upper Bound of Input Perturbation for RBFNNs Sensitivity Analysis. In: Proceedings of the Fourth International Conference on Machine Learning and Cybernetics, vol. 8, pp. 4704–4709 (2005)Google Scholar