Efficient AUC Learning Curve Calculation
A learning curve of a performance measure provides a graphical method with many benefits for judging classifier properties. The area under the ROC curve (AUC) is a useful and increasingly popular performance measure. In this paper, we consider the computational aspects of calculating AUC learning curves. A new method is provided for incrementally updating exact AUC curves and for calculating approximate AUC curves for datasets with millions of instances. Both theoretical and empirical justifications are given for the approximation. Variants for incremental exact and approximate AUC curves are provided as well.
KeywordsTrue Positive Receiver Operator Characteristic Receiver Operator Characteristic Curve Anchor Point Positive Class
Unable to display preview. Download preview PDF.
- 1.Bandos, A., Rockette, H.E., Gur, D.: Resampling Methods for the Area Under the ROC Curve. In: ROCML 2006 (2006)Google Scholar
- 2.Beck, J., Schultz, E.: The use of ROC curves in test performance evaluation. Archives of Pathology and Laboratory Medicine 110, 13–20 (1986)Google Scholar
- 3.Egan, J.P.: Signal Detection Theory and ROC analysis. Academic Press, New York (1975)Google Scholar
- 4.Fawcett, T.: ROC Graphs: Notes and Practical Considerations for Data Mining Researchers. Technical Report HPL-2003-4, HP LabsGoogle Scholar
- 5.Flach, P.A.: The geometry of ROC space: understanding machine learning metrics through ROC isometrics. In: ICML (2003)Google Scholar
- 6.Lachiche, N., Flach, P.A.: Improving accuracy and cost of two-class and multi-class probabilistic classifiers using ROC curves. In: ICML (2003)Google Scholar
- 7.Macskassy, S.A., Provost, F., Rosset, S.: Pointwise ROC Confidence Bounds: An Empirical Evaluation. In: ROCML 2005 (2005)Google Scholar
- 9.Witten, I.H., Frank, E.: Data mining: Practical machine learning tools and techniques with Java implementations. Morgan Kaufmann, San Francisco (2000)Google Scholar