Abstract
Bayesian Ying-Yang (BYY) learning is proposed as a unified statistical learning framework firstly in (Xu, 1995) and systematically developed in past years. Its consists of a general BYY system and a fundamental harmony learning principle as a unified guide for developing new parameter learning algorithms, new regularization techniques, new model selection criteria, as well as a new learning approach that implements parameter learning with model selection made automatically during learning (Xu, 1999a&b; 2000a&b). This paper goes further beyond the scope of BYY learning, and provides new results and new understandings on harmony learning from perspectives of conventional parametric models, BYY systems and some general properties of information geometry.
The work described in this paper was fully supported by a grant from the Research Grant Council of the Hong Kong SAR (project No: CUHK4383/99E).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Amari, S.-I., Cichocki, A., and Yang, H. (1996), “A new learning algorithm for blind separation of sources”, in D. S. Touretzky, et al, eds, Advances in Neural Information Processing 8, MIT Press: Cambridge, MA, 757–763, 1996.
Barlow, H.B. (1989), “Unsupervised learning”, Neural Computation, 1, 295–311, 1989.
Dayan, P., & Zemel, R.S., (1995) “Competition and multiple cause models”, Neural Computation 7, pp565–579, 1995.
Dayan, P. & Hinton, G., E., (1996), “Varieties of Helmholtz machine”, Neural Networks 9, No.8, 1385–1403.
Hinton, G. E., Dayan, P., Frey, B. and Neal, R.M.(1995), “The wake-sleep algorithm for unsupervised learning neural networks”, Science 268, 1158–1160, 1995.
Jacobs, R.A., Jordan, M.I., Nowlan, S., and Hinton, G., E. (1991), “Adaptive mixtures of local experts”, Neural Computation, 3, 79–87, 1991.
Jordan, M.I., and Jacobs, R.A. (1994), “Hierarchical mixtures of experts and the EM algorithm”, Neural Computation 6, 181–214, 1994.
Xu, L. (2000a), “BYYΣ-Π Factor Systems and Harmony Learning: Recent Advances”, Proc. 7th International Conference on Neural Information Processing (ICONIP-2000), Nov. 14–18, Taejon, KOREA.
Xu, L.(2000b), “Temporal BYY Learning for State Space Approach, Hidden Markov Model and Blind Source Separation”, IEEE Trans on Signal Processing, Vol. 48, No.7, pp 2132–2144, 2000.
Xu, L. (1999a), “Bayesian Ying-Yang Unsupervised and Supervised Learning: Theory and Applications”, Proc. 1999 Chinese Conf. on Neural Networks and Signal Processing, 12–29, Shantou, China, Nov., 1999.
Xu, L., (1999b), “BYY Data Smoothing Based Learning on A Small Size of Sam-ples”, and “Bayesian Ying-Yang Theory for Empirical Learning, Regularization and Model Selection: General Formulation”, Proc. 1999 Intl. Joint Conf. on Neural Networks, Vol.1 of 6, pp 546–551 and pp 552–557, USA, July 10–16,1999.
Xu, L.(1999c), “Bayesian Ying-Yang Supervised Learning, Modular Models, and Three Layer Nets”, Proc. 1999 Intl. Joint Conf. on Neural Networks, Vol.1 of 6, pp 540–545, USA, July 10–16, 1999.
Xu, L., (1998a), “RBF Nets, Mixture Experts, and Bayesian Ying-Yang Learning”, Neurocomputing, Vol. 19, No.1-3, 223–257, 1998.
Xu, L., (1998b), “Bayesian Kullback Ying-Yang Dependence Reduction Theory”, Neurocomputing, Vol. 22, No.1-3, 81–112, 1998.
Xu, L.(1997), “Bayesian Ying-Yang Machine, Clustering and Number of Clusters”, Pattern Recognition Letters, Vol.18, No.11-13, 1167–1178, 1997.
Xu, L., (1995) “A Unified Learning Scheme: Bayesian-Kullback YING-YANG Machine”, Advances in Neural Information Processing Systems 8, eds., D. S. Touretzky, et al, MIT Press, 444–450, 1996. A part of its preliminary version on Proc. ICONIP95, Peking, Oct 30-Nov. 3, 977–988, 1995.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Xu, L. (2000). Best Harmony Learning. In: Leung, K.S., Chan, LW., Meng, H. (eds) Intelligent Data Engineering and Automated Learning — IDEAL 2000. Data Mining, Financial Engineering, and Intelligent Agents. IDEAL 2000. Lecture Notes in Computer Science, vol 1983. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44491-2_18
Download citation
DOI: https://doi.org/10.1007/3-540-44491-2_18
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-41450-6
Online ISBN: 978-3-540-44491-6
eBook Packages: Springer Book Archive