Optimal “Anti-Bayesian” Parametric Pattern Classification for the Exponential Family Using Order Statistics Criteria
This paper reports some pioneering results in which optimal parametric classification is achieved in a counter-intuitive manner, quite opposed to the Bayesian paradigm. The paper, which builds on the results of , demonstrates (with both theoretical and experimental results) how this can be done for some distributions within the exponential family. To be more specific, within a Bayesian paradigm, if we are allowed to compare the testing sample with only a single point in the feature space from each class, the optimal Bayesian strategy would be to achieve this based on the (Mahalanobis) distance from the corresponding means, which in one sense, is the most central point in the respective distribution. In this paper, we shall show that we can obtain optimal results by operating in a diametrically opposite way, i.e., a so-called “anti-Bayesian” manner. Indeed, we shall show that by working with a very few (sometimes as small as two) points distant from the mean, one can obtain remarkable classification accuracies. These points, in turn, are determined by the Order Statistics of the distributions, and the accuracy of our method, referred to as Classification by Moments of Order Statistics (CMOS), attains the optimal Bayes’ bound! In this paper, we shall show the claim for two uni-dimensional members of the exponential family. The theoretical results, which have been verified by rigorous experimental testing, also present a theoretical foundation for the families of Border Identification (BI) reported algorithms.
KeywordsClassification using Order Statistics (OS) Moments of OS
Unable to display preview. Download preview PDF.
- 1.Thomas, A., Oommen, B.J.: Optimal “Anti-Bayesian” Parametric Pattern Classification Using Order Statistics Criteria (accepted for Publication, 2012)Google Scholar
- 2.Garcia, S., Derrac, J., Cano, J.R., Herrera, F.: Prototype Selection for Nearest Neighbor Classification: Taxonomy and Empirical Study. IEEE Transactions on Pattern Analysis and Machine IntelligenceGoogle Scholar
- 3.Triguero, I., Derrac, J., Garcia, S., Herrera, F.: A Taxonomy and Experimental Study on Prototype Generation for Nearest Neighbor Classification. IEEE Transactions on Systems, Man and Cybernetics - Part C: Applications and ReviewsGoogle Scholar
- 4.Thomas, A., Oommen, B.J.: The Foundational Theory of Optimal “Anti-Bayesian” Parametric Pattern Classification Using Order Statistics Criteria (to be submitted, 2012)Google Scholar
- 5.Thomas, A.: Pattern Classification using Novel Order Statistics and Border Identification Methods. PhD thesis, School of Computer Science, Carleton University (to be submitted, 2013)Google Scholar
- 6.Ahsanullah, M., Nevzorov, V.B.: Order Statistics: Examples and Exercises. Nova Science Publishers, Inc. (2005)Google Scholar