Abstract
This paper presents a model characterizing unsupervised learning from an information theoretic point of view. Under some hypothesis, it defines a theoretical quality criterion, which corresponds to the informational limit that bounds the learning ability of any clustering algorithm. This quality criterion depends on the information content of the learning set. It is relevant when examples are sparsely described, i.e. when most of the descriptors are missing. This theoretical limit of any unsupervised learning algorithm is then compared to the actual learning quality of different clustering algorithms (EM, COBWEB and PRESS). This empirical comparison is based on the use of artificial data sets, which are randomly degraded. Finally, the paper shows that the results of PRESS, an algorithm specifically designed to learn from sparsely described examples, are very closed to the theoretical upper bound quality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
DEMPSTER, A.P., LAIRD, N.M. and RUBIN, D.B. (1977): Maximum likelihood from incomplete data via the EM algorithm. Royal Statistical Society B 39(1), 1–38.
FISHER, D.H. (1987): Knowledge acquisition via incremental conceptual clustering. Machine Learning (2), 139–172.
HE, J., TAN, A.-H., TAN, C.-L. and SUNG, S.-Y. (2002): On Qualitative Evaluation of Clustering Systems. Information Retrieval and Clustering, Kluwer Academic Publishers.
LIPPMANN, W. (1922): Public Opinion. Wading River, Long Island.
ROSCH, E. (1975): Cognitive representations of semantic categories. Journal of Experimental Psychology: General, number 104, 192–232.
SREBRO, N., SHAKHNAROVICH, G. and ROWEIS, S. (2005): When is Clustering Hard? In: PASCAL Workshop on Statistics and Optimization of Clustering Workshop.
SREBRO, N., SHAKHNAROVICH, G. and ROWEIS, S. (2006): An investigation of computational and informational limits in Gaussian mixture clustering. In: 23rd International Conference on Machine Learning (ICML) (preliminary version appeared as UTML-TR-2006-002, February 2006).
VELCIN, J. and GANASCIA, J.-G. (2005): Stereotype extraction with default clustering. In: Proceedings of the 19 th International Joint Conference on Artificial Intelligence. Edinburgh, Scotland.
WITTEN, I.H. and FRANK, E. (2005): Data Mining: Practical machine learning tools and techniques. 2nd Edition, Morgan Kaufmann, San Francisco.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Ganascia, JG., Velcin, J. (2007). Unsupervised Learning Informational Limit in Case of Sparsely Described Examples. In: Brito, P., Cucumel, G., Bertrand, P., de Carvalho, F. (eds) Selected Contributions in Data Analysis and Classification. Studies in Classification, Data Analysis, and Knowledge Organization. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73560-1_32
Download citation
DOI: https://doi.org/10.1007/978-3-540-73560-1_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73558-8
Online ISBN: 978-3-540-73560-1
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)