Abstract
Generative topographic mapping (GTM) is a statistical model to extract a hidden smooth manifold from data, like the self-organizing map (SOM). Although a deterministic search algorithm for the hyperparameters regulating the smoothness of the manifold has been proposed previously, it is based on approximations that are valid only on abundant data. Thus, it often fails to obtain suitable estimates on small data. In this paper, to improve the hyperparameter search in GTM, we construct a Gibbs sampler on the model, which generates random sample series following the posteriors on the hyperparameters. Reliable estimates are obtained from the samples. In addition, we obtain another deterministic algorithm using the ensemble learning. From the result of an experimental comparison of these algorithms, an efficient method for reliable estimation in GTM is suggested.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Kohonen, T.: 1995, Self-Organizing Maps. Springer, Berlin.
Durbin, R. and Willshaw, D.: 1987, An analogue approach to the traveling salesman problem using an elastic net method. Nature, 326, 689-691.
Durbin, R., Szeliski, R. and Yuille, A.: 1989, An analysis of the elastic net approach to the traveling salesman problem. Neural Computation, 1, 348-358.
Simic, P. D.: 1990, Statistical mechanics as the underlying theory of ‘elastic’ and ‘neural’ optimisations. Network, 1,89-103.
Durbin, R. and Mitchison, G.: 1990, A dimension reduction framework for understanding cortical maps. Nature, 343, 644-647.
Utsugi, A.: 1996, Topology selection for self-organizing maps. Network, 7, 727-740.
Utsugi, A.: 1997, Hyperparameter selection for self-organizing maps. Neural Computation, 9, 623-635.
Utsugi, A.: 1998, Density estimation by mixture models with smoothing priors. Neural Computation, 10, 2115-2135.
Bishop, C.M., Svensén, M. and Williams, C.K.I.: 1998, GTM: the generative topographic mapping. Neural Computation, 10, 215-234.
Bishop, C.M., Svensén, M. and Williams, C.K.I.: 1998, Developments of the generative topographic mapping. Neurocomputing, 21, 203-224.
MacKay, D.J.C.: 1992, A practical Bayesian framework for backprop networks. Neural Computation, 4, 448-472.
Kass, R.E. and Raftery, A.E.: 1995, Bayes factors. Journal of the American Statistical Association, 90, 773-795.
Tanner, M.A.: 1996, Tools for statistical inference: methods for exploration of posterior istribution and likelihood functions. Springer-Verlag, New York, 3rd edition.
Waterhouse, S., MacKay, D. and Robinson, T.: 1996, Bayesian methods for mixtures of experts. In: D.S. Touretzky, M.C. Mozer and M.E. Haaslmo, eds, Advances in Neural Information Processing Systems 8, MIT Press, Cambridge, pp. 351-357.
MacKay, D.J.C.: 1999, Comparison of approximate methods for handling hyperparameters. Neural Computation, 11, 1035-1068.
Buja, A., Hastie, T. and Tibshirani, R.: 1989, Linear smoothers and additive models. The Annals of Statistics, 17, 453-555.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Utsugi, A. Bayesian Sampling and Ensemble Learning in Generative Topographic Mapping. Neural Processing Letters 12, 277–290 (2000). https://doi.org/10.1023/A:1026567325853
Issue Date:
DOI: https://doi.org/10.1023/A:1026567325853