Advertisement

Optimal Mean-Precision Classifier

  • David M. J. Tax
  • Marco Loog
  • Robert P. W. Duin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5519)

Abstract

For pattern recognition problems where a small set of relevant objects should be retrieved from a (very) large set of irrelevant objects, standard evaluation criteria are often insufficient. For these situations often the precision-recall curve is used. An often-employed scalar measure derived from this curve is the mean precision, that estimates the average precision over all values of the recall. This performance measure, however, is designed to be non-symmetric in the two classes and it appears not very simple to optimize. This paper presents a classifier that approximately maximizes the mean precision by a collection of simple linear classifiers.

Keywords

Pattern recognition performance evaluation information retrieval precision-recall graph 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [And82]
    Anderson, J.A.: Logistic discrimination. In: Kirshnaiah, P.R., Kanal, L.N. (eds.) Classification, Pattern Recognition and Reduction of Dimensionality. Handbook of Statistics, vol. 2, pp. 169–191. North Holland, Amsterdam (1982)CrossRefGoogle Scholar
  2. [Bra97]
    Bradley, A.P.: The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition 30(7), 1145–1159 (1997)CrossRefGoogle Scholar
  3. [BS05]
    Brefeld, U., Scheffer, T.: AUC miximizing support vector learning. In: Proceedings of ICML 2005 workshop on ROC analysis in Machine Learning (2005)Google Scholar
  4. [DHS01]
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Wiley & Sons, Chichester (2001)zbMATHGoogle Scholar
  5. [FFHO02]
    Ferri, C., Flach, P., Hernandez-Orallo, J.: Learning decision trees using the area under the ROC curve. In: Proceedings of the ICML (2002)Google Scholar
  6. [Fla03]
    Flach, P.: The geometry of ROC space: understanding machine learning metrics through ROC isometrics. In: Proceedings of the international conference on Machine learning 2003, pp. 194–201 (2003)Google Scholar
  7. [NHBM98]
    Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)Google Scholar
  8. [Par62]
    Parzen, E.: On estimation of a probability density function and mode. Annals of Mathenatical Statistics 33, 1065–1076 (1962)MathSciNetCrossRefzbMATHGoogle Scholar
  9. [SM83]
    Salton, G., McGill, M.J.: Introduction to Modern Information Retrieval. McGraw-Hill, New York (1983)zbMATHGoogle Scholar
  10. [Vap98]
    Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)zbMATHGoogle Scholar
  11. [vR79]
    van Rijsbergen, C.J.: Information Retrieval, 2nd edn. Butterwort (1979)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • David M. J. Tax
    • 1
  • Marco Loog
    • 1
  • Robert P. W. Duin
    • 1
  1. 1.Information and Communication Theory GroupDelft University of TechnologyDelftThe Netherlands

Personalised recommendations