A New Notion of Weakness in Classification Theory
The notion of a weak classifier, as one which is “a little better” than a random one, was introduced first for 2-class problems . The extensions to K-class problems are known. All are based on relative activations for correct and incorrect classes and do not take into account the final choice of the answer. A new understanding and definition is proposed here. It takes into account only the final choice of classification that must be taken. It is shown that for a K class classifier to be called “weak”, it needs to achieve lower than 1/K risk value. This approach considers only the probability of the final answer choice, not the actual activations.
Unable to display preview. Download preview PDF.
- 1.Kearns, M., Valiant, L.: Cryptographic limitations on learning Boolean formulae and finite automata. Journal of the Association for Computing Machinery 41(1), 67–95Google Scholar
- 2.Bax, E.: Validation of Voting Committees. Neural Computation 4(10), 975–986 (1998)Google Scholar
- 3.Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, New York (2001)Google Scholar
- 4.Tresp, V.: Committee Machines. In: Hu, Y.H., Hwang, J.-N. (eds.) Handbook for Neural Network Signal Processing. CRC Press, Boca Raton (2001)Google Scholar
- 5.Kittler, J., Hojjatoleslami, A., Windeatt, T.: Strategies for combining classifiers employing shared and distinct pattern representations. Pattern Recognition Letters 18, 1373–1377 (1997)Google Scholar
- 6.Schapire, R.E.: The strength of weak learnability. Machine Learning 5, 197–227 (1990)Google Scholar
- 7.Eibl, G., Pfeiffer, K.-P.: Multiclass boosting for weak classifiers. Journal of Machine Learning 6, 189–210 (2005)Google Scholar
- 8.Freund, Y., Schapire, R.E.: A decision theoretic generalization of online learning and an application to boosting. Journal of Computer and System Sciences 55, 119–139 (1997)Google Scholar
- 9.Podolak, I.T.: Hierarchical Classifier with Overlapping Class Groups. Expert Systems with Applications 34(1), 673–682 (2008)Google Scholar
- 12.Podolak, I.T., Roman, A.: Improving the accuracy of a hierarchical classifier (in preparation)Google Scholar