Asymptotic Behavior of Stochastic Complexity of Complete Bipartite Graph-Type Boltzmann Machines
In singular statistical models, it was shown that Bayes learning is effective. However, on Bayes learning, calculation containing the Bayes posterior distribution requires huge computational costs. To overcome the problem, mean field approximation (or equally variational Bayes method) was proposed. Recently, the generalization error and stochastic complexity in mean field approximation have been theoretically studied. In this paper, we treat the complete bipartite graph-type Boltzmann machines and derive the upper bound of the asymptotic stochastic complexity in mean field approximation.
KeywordsLearning Model Fisher Information Hide Unit Generalization Error True Distribution
Unable to display preview. Download preview PDF.
- 2.Aoyagi, M., Watanabe, S.: The generalization error of reduced rank regression in bayesian estimation. In: Proc. of ISITA 2004, Italy, pp. 1068–1073 (2004)Google Scholar
- 4.Nakajima, S., Watanabe, S.: Generalization Error and Free Energy of Linear Neural Networks in Variational Bayes Approach. In: Proc. of ICONI 2005, Taiwan, pp. 55–60 (2005)Google Scholar
- 5.Watanabe, K., Watanabe, S.: Lower bounds of stochastic complexities in variational Bayes learning of gaussian mixture models. In: Proc. IEEE conference on Cybernetics and Intelligent Systems, pp. 99–104 (2004)Google Scholar
- 6.Hosino, T., Watanabe, K., Watanabe, S.: Stochastic Complexity of Variational Bayesian Hidden Markov Models. In: Proc. of IJCNN 2005, Canada (2005)Google Scholar
- 7.Hosino, T., Watanabe, K., Watanabe, S.: Stochastic Complexity of Stochastic Context Free Grammer on Variational Bayesian method. IEICE Technicalreport, NC2005-49 (October 2005)Google Scholar
- 8.Nakano, N., Watanabe, S.: Stochastic Complexity of Layered Neural Networks in Mean Field Approximation. In: Proc. of ICONI 2005, Taiwan, pp. 332–337 (2005)Google Scholar
- 9.Nishiyama, Y., Watanabe, S.: Asymptotic Behavior of Free Energy of General Boltzmann Machines in Mean Field Approximation, IEICE Technical report, (July 2006) (to appear)Google Scholar