Experimental Study of Ergodic Learning Curve in Hidden Markov Models
Part of the
Lecture Notes in Computer Science
book series (LNCS, volume 5506)
A number of learning machines used in information science are not regular, but rather singular, because they are non-identifiable and their Fisher information matrices are singular. Even for singular learning machines, the learning theory was developed for the case in which training samples are independent. However, if training samples have time-dependency, then learning theory is not yet established. In the present paper, we define an ergodic generalization error for a time-dependent sequence and study its behavior experimentally in hidden Markov models. The ergodic generalization error is clarified to be inversely proportional to the number of training samples, but the learning coefficient depends strongly on time-dependency.
KeywordsTraining Sample Hide Markov Model True Parameter Hide State Predictive Distribution
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Aoyagi, M., Watanabe, S.: Stochastic complexities of reduced rank regression in Bayesian estimation. Neural Networks 18(7), 924–933 (2005)CrossRefzbMATHGoogle Scholar
Watanabe, S.: Algebraic analysis for nonidentifiable learning machines. Neural Computation 13(4), 899–933 (2001)CrossRefzbMATHGoogle Scholar
Watanabe, S.: Algebraic geometrical methods for hierarchical learning machines. Neural Networks 14(8), 1049–1060 (2001)CrossRefGoogle Scholar
Yamazaki, K., Watanabe, S.: Singularities in mixture models and upper bounds of stochastic complexity. Int. J. Neural Networks 16, 1029–1038 (2003)CrossRefzbMATHGoogle Scholar
Yamazaki, K., Watanabe, S.: Algebraic geometry and stochastic complexity of hidden Markov models. Neurocomputing 69(1), 62–84 (2005)CrossRefGoogle Scholar
© Springer-Verlag Berlin Heidelberg 2009