Estimating the ROC Curve of Linearly Combined Dichotomizers

  • Claudio Marrocco
  • Mario Molinara
  • Francesco Tortorella
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3617)

Abstract

A well established technique to improve the classification performances is to combine more classifiers. In the binary case, an effective instrument to analyze the dichotomizers under different class and cost distributions providing a description of their performances at different operating points is the Receiver Operating Characteristic (ROC) curve. To generate a ROC curve, the outputs of the dichotomizers have to be processed. An alternative way that makes this analysis more tractable with mathematical tools is to use a parametric model and, in particular, the binormal model that gives a good approximation to many empirical ROC curves. Starting from this model, we propose a method to estimate the ROC curve of the linear combination of two dichotomizers given the ROC curves of the single classifiers. A possible application of this approach has been successfully tested on real data set.

Keywords

Support Vector Machine True Positive Rate Multi Layer Perceptron Cumulative Distribution Function Pima Indian Diabetes 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Provost, F., Fawcett, T., Kohavi, R.: The case against accuracy estimation for comparing induction algorithms. In: Proc. 15th Intl. Conf. on Machine Learning, pp. 445–453. Morgan Kaufmann, San Francisco (1998)Google Scholar
  2. 2.
    Egan, J.P.: Signal detection theory and ROC analysis. Series in Cognition and Perception. Academic Press, New York (1975)Google Scholar
  3. 3.
    Metz, C.E.: ROC methodology in radiologic imaging. Invest. Radiol. 21, 720–733 (1986)CrossRefGoogle Scholar
  4. 4.
    Bradley, A.P.: The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition 30, 1145–1159 (1997)CrossRefGoogle Scholar
  5. 5.
    Provost, F., Fawcett, T.: Robust classification for imprecise environments. Machine Learning 42, 203–231 (2001)MATHCrossRefGoogle Scholar
  6. 6.
    Tortorella, F.: A ROC-based Reject Rule for Dichotomizers. Pattern Recognition Letters 26, 167–180 (2005)CrossRefGoogle Scholar
  7. 7.
    Fawcett, T.: ROC graphs: notes and practical considerations for data mining researchers. HP Labs Tech Report HPL-2003-4 (2003)Google Scholar
  8. 8.
    Metz, C.E., Herman, B.A., Shen, J.H.: Maximum-likelihood estimation of ROC curves from continuously-distributed data. Statistics in Medicine 17, 1033–1053 (1998)CrossRefGoogle Scholar
  9. 9.
    Pepe, M.S.: The Statistical Evaluation of Medical Tests for Classification and Prediction. Oxford Statistical Science Series. Oxford University Press, Oxford (2003)MATHGoogle Scholar
  10. 10.
    Papoulis, A.: Probability, Random Variables, and Stochastic Processes. McGraw-Hill, New York (2001)Google Scholar
  11. 11.
    Cortes, C., Mohri, M.: AUC Optimization vs. Error Rate Minimization. Advances in Neural Information Processing Systems. In: NIPS 2003 (2004)Google Scholar
  12. 12.
    Flake, G.W., Pearlmuter, B.A.: Differentiating Functions of the Jacobian with Respect to the Weights. In: Solla, S.A., Leen, T.K., Müller, K. (eds.) Advances in Neural Information Processing Systems, vol. 12. The MIT Press, Cambridge (2000)Google Scholar
  13. 13.
    Blake, C., Keogh, E., Merz, C.J.: UCI Repository of Machine Learning Databases (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
  14. 14.
    Metz, C.E., Pan, X.: Proper binormal ROC curves: theory and maximum-likelihood estimation. J. Math. Psych. 43, 1–33 (1999)MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Claudio Marrocco
    • 1
  • Mario Molinara
    • 1
  • Francesco Tortorella
    • 1
  1. 1.Dipartimento di Automazione, Elettromagnetismo, Ingegneria dell’Informazione e Matematica IndustrialeUniversità degli Studi di CassinoCassinoItaly

Personalised recommendations