Symmetries from Uniform Space Covering in Stochastic Discrimination
Studies on ensemble methods for classification suffer from the difficulty of modeling the complementary strengths of the components. Kleinberg’s theory of stochastic discrimination (SD) addresses this rigorously via mathematical notions of enrichment, uniformity, and projectability of a model ensemble. We explain these concepts via a very simple numerical example that captures the basic principles of the SD theory and method. We focus on a fundamental symmetry in point set covering that is the key observation leading to the foundation of the theory. We believe a better understanding of the SD method will lead to developments of better tools for analyzing other ensemble methods.
KeywordsFeature Space Machine Intelligence Model Ensemble Ensemble Method Ensemble Learning
Unable to display preview. Download preview PDF.
- 1.Berlind, R.: An Alternative Method of Stochastic Discrimination with Applications to Pattern Recognition, Doctoral Dissertation, Department of Mathematics, State University of New York at Buffalo (1994)Google Scholar
- 3.Chen, D.: Estimates of Classification Accuracies for Kleinberg’s Method of Stochastic Discrimination in Pattern Recognition, Doctoral Dissertation, Department of Mathematics, State University of New York at Buffalo (1998)Google Scholar
- 5.Freund, Y., Schapire, R.E.: Experiments with a New Boosting Algorithm. In: Proceedings of the Thirteenth International Conference on Machine Learning, Bari, Italy, July 3-6, pp. 148–156 (1996)Google Scholar
- 7.Ho, T.K.: Random Decision Forests. In: Proceedings of the 3rd International Conference on Document Analysis and Recognition, Montreal, Canada, August 14-18, pp. 278–282 (1995)Google Scholar
- 8.Ho, T.K.: Multiple classifier combination: Lessons and next steps. In: Kandel, A., Bunke, H. (eds.) Hybrid Methods in Pattern Recognition. World Scientific, Singapore (2002)Google Scholar
- 9.Ho, T.K., Kleinberg, E.M.: Building Projectable Classifiers of Arbitrary Complexity. In: Proceedings of the 13th International Conference on Pattern Recognition, Vienna, Austria, August 25-30, pp. 880–885 (1996)Google Scholar
- 11.Ho, T.K.: Nearest Neighbors in Random Subspaces. In: Proceedings of the Second International Workshop on Statistical Techniques in Pattern Recognition, Sydney, Australia, August 11-13, pp. 640–648 (1998)Google Scholar
- 12.Ho, T.K., Hull, J.J., Srihari, S.N.: Decision Combination in Multiple Classifier Systems. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-16(1), 66–75 (1994)Google Scholar
- 19.Lam, L., Suen, C.Y.: Application of majority voting to pattern recognition. IEEE Transactions on Systems, Man, and Cybernetics SMC-27(5), 553–568 (1997)Google Scholar