Optimal “Anti-Bayesian” Parametric Pattern Classification for the Exponential Family Using Order Statistics Criteria

  • A. Thomas
  • B. John Oommen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7324)


This paper reports some pioneering results in which optimal parametric classification is achieved in a counter-intuitive manner, quite opposed to the Bayesian paradigm. The paper, which builds on the results of [1], demonstrates (with both theoretical and experimental results) how this can be done for some distributions within the exponential family. To be more specific, within a Bayesian paradigm, if we are allowed to compare the testing sample with only a single point in the feature space from each class, the optimal Bayesian strategy would be to achieve this based on the (Mahalanobis) distance from the corresponding means, which in one sense, is the most central point in the respective distribution. In this paper, we shall show that we can obtain optimal results by operating in a diametrically opposite way, i.e., a so-called “anti-Bayesian” manner. Indeed, we shall show that by working with a very few (sometimes as small as two) points distant from the mean, one can obtain remarkable classification accuracies. These points, in turn, are determined by the Order Statistics of the distributions, and the accuracy of our method, referred to as Classification by Moments of Order Statistics (CMOS), attains the optimal Bayes’ bound! In this paper, we shall show the claim for two uni-dimensional members of the exponential family. The theoretical results, which have been verified by rigorous experimental testing, also present a theoretical foundation for the families of Border Identification (BI) reported algorithms.


Classification using Order Statistics (OS) Moments of OS 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Thomas, A., Oommen, B.J.: Optimal “Anti-Bayesian” Parametric Pattern Classification Using Order Statistics Criteria (accepted for Publication, 2012)Google Scholar
  2. 2.
    Garcia, S., Derrac, J., Cano, J.R., Herrera, F.: Prototype Selection for Nearest Neighbor Classification: Taxonomy and Empirical Study. IEEE Transactions on Pattern Analysis and Machine IntelligenceGoogle Scholar
  3. 3.
    Triguero, I., Derrac, J., Garcia, S., Herrera, F.: A Taxonomy and Experimental Study on Prototype Generation for Nearest Neighbor Classification. IEEE Transactions on Systems, Man and Cybernetics - Part C: Applications and ReviewsGoogle Scholar
  4. 4.
    Thomas, A., Oommen, B.J.: The Foundational Theory of Optimal “Anti-Bayesian” Parametric Pattern Classification Using Order Statistics Criteria (to be submitted, 2012)Google Scholar
  5. 5.
    Thomas, A.: Pattern Classification using Novel Order Statistics and Border Identification Methods. PhD thesis, School of Computer Science, Carleton University (to be submitted, 2013)Google Scholar
  6. 6.
    Ahsanullah, M., Nevzorov, V.B.: Order Statistics: Examples and Exercises. Nova Science Publishers, Inc. (2005)Google Scholar
  7. 7.
    Morris, K.W., Szynal, D.: A goodness-of-fit for the Uniform Distribution based on a Characterization. Journal of Mathematical Science 106, 2719–2724 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Lin, G.D.: Characterizations of Continuous Distributions via Expected values of two functions of Order Statistics. Sankhya: The Indian Journal of Statistics 52, 84–90 (1990)zbMATHGoogle Scholar
  9. 9.
    Too, Y., Lin, G.D.: Characterizations of Uniform and Exponential Distributions. Academia Sinica 7(5), 357–359 (1989)MathSciNetGoogle Scholar
  10. 10.
    Grudzien, Z., Szynal, D.: Characterizations of Distributions by Moments of Order Statistics when the Sample Size is Random. Applications Mathematicae 23, 305–318 (1995)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Nadarajah, S.: Explicit Expressions for Moments of Order Statistics. Statistics and Probability Letters 78, 196–205 (2008)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • A. Thomas
    • 1
  • B. John Oommen
    • 1
  1. 1.School of Computer ScienceCarleton UniversityOttawaCanada

Personalised recommendations