Abstract
In the past two decades, the use of the weight decay regularizer for improving the generalization ability of neural networks has been extensively investigated. However, most existing results focus on the fault-free neural networks only. This papers extends the analysis on the generalization ability for networks with multiplicative weight noise. Our analysis result allows us not only to estimate the generalization ability of a faulty network, but also to select a good model from various settings. Simulated experiments are performed to verify theoretical result.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Moody, J.E.: Note on generalization, regularization, and architecture selection in nonlinear learning systems. In: First IEEE-SP Workshop on Neural Networks for Signal Processing, pp. 1–10 (1991)
Chen, S., Hong, X., Harris, C.J., Sharkey, P.M.: Sparse modelling using orthogonal forward regression with press statistic and regularization. In: IEEE Trans. Systems, Man and Cybernetics, Part B, pp. 898–911 (2004)
Bernier, J.L., Ortega, J., Rodriguez, M.M., Rojas, I., Prieto, A.: An accurate measure for multilayer perceptron tolerance to weight deviations. Neural Processing Letters 10(2), 121–130 (1999)
Leung, C.S., Young, G.H., Sum, J., Kan, W.K.: On the regularization of forgetting recursive least square. IEEE Transactions on Neural Networks 10, 1482–1486 (1999)
Anitha, D., Himavathi, S., Muthuramalingam, A.: Feedforward neural network implementation in fpga using layer multiplexing for effective resource utilization. IEEE Transactions on Neural Networks 18, 880–888 (2007)
Moussa, M., Savich, A.W., Areibi, S.: The impact of arithmetic representation on implementing mlp-bp on fpgas: A study. IEEE Transactions on Neural Networks 18, 240–252 (2007)
Kaneko, T., Liu, B.: Effect of coefficient rounding in floating-point digital filters. IEEE Trans. on Aerospace and Electronic Systems AE-7, 995–1003 (1970)
Fedorov, V.V.: Theory of optimal experiments. Academic Press, London (1972)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Leung, C.S., Sum, P.F., Wang, H. (2009). Analysis on Generalization Error of Faulty RBF Networks with Weight Decay Regularizer. In: Köppen, M., Kasabov, N., Coghill, G. (eds) Advances in Neuro-Information Processing. ICONIP 2008. Lecture Notes in Computer Science, vol 5507. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03040-6_39
Download citation
DOI: https://doi.org/10.1007/978-3-642-03040-6_39
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03039-0
Online ISBN: 978-3-642-03040-6
eBook Packages: Computer ScienceComputer Science (R0)