Localized Bayes Estimation for Non-identifiable Models
Hierarchical learning machines such as neural networks are now being used in many applications. Although the Bayes ensemble learning gives the good generalization performance in such hierarchical learning machines, it is difficult to realize the posterior distribution because of the singularities in the parameter space. In this paper, we propose a new learning algorithm which enables us to construct a localized posterior distribution. We call this method Localized Bayes estimation and theoretically show that it attains the smaller generalization error in reduced rank approximations.
KeywordsPosterior Distribution Predictive Distribution Markov Chain Monte Carlo Method Generalization Error Good Generalization Performance
Unable to display preview. Download preview PDF.
- 2.Nakajima, S., Watanabe, S.: Generalization Error and Free Energy of Linear Neural Networks in Variational Bayes Approach. In: Proc. of ICONIP 2005, pp. 55–60 (2005)Google Scholar
- 3.Aoyagi, M., Watanabe, S.: The generalization error of reduced rank regression in Bayesian estimation. In: Proc. of ISAITA 2004, pp. 1068–1073 (2004)Google Scholar