Localized Bayes Estimation for Non-identifiable Models

  • Shingo Takamatsu
  • Shinichi Nakajima
  • Sumio Watanabe
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4232)


Hierarchical learning machines such as neural networks are now being used in many applications. Although the Bayes ensemble learning gives the good generalization performance in such hierarchical learning machines, it is difficult to realize the posterior distribution because of the singularities in the parameter space. In this paper, we propose a new learning algorithm which enables us to construct a localized posterior distribution. We call this method Localized Bayes estimation and theoretically show that it attains the smaller generalization error in reduced rank approximations.


Posterior Distribution Predictive Distribution Markov Chain Monte Carlo Method Generalization Error Good Generalization Performance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Watanabe, S.: Algebraic analysis for nonidentifiable learning machines. Neural Computation 13(4), 899–933 (2001)MATHCrossRefGoogle Scholar
  2. 2.
    Nakajima, S., Watanabe, S.: Generalization Error and Free Energy of Linear Neural Networks in Variational Bayes Approach. In: Proc. of ICONIP 2005, pp. 55–60 (2005)Google Scholar
  3. 3.
    Aoyagi, M., Watanabe, S.: The generalization error of reduced rank regression in Bayesian estimation. In: Proc. of ISAITA 2004, pp. 1068–1073 (2004)Google Scholar
  4. 4.
    Fukumizu, K.: Generalization error of linear neural networks in unidentifiable cases. In: Watanabe, O., Yokomori, T. (eds.) ALT 1999. LNCS (LNAI), vol. 1720, pp. 51–62. Springer, Heidelberg (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Shingo Takamatsu
    • 1
  • Shinichi Nakajima
    • 2
  • Sumio Watanabe
    • 3
  1. 1.Tokyo Institute of TechnologyKanagawaJapan
  2. 2.Nikon CorporationSaitamaJapan
  3. 3.PI Lab.Tokyo Institute of TechnologyKanagawaJapan

Personalised recommendations