Skip to main content
SpringerLink
Log in

Multi-Objective Optimisation for Receiver Operating Characteristic Analysis

  • Chapter
Multi-Objective Machine Learning

Part of the book series: Studies in Computational Intelligence ((SCI,volume 16))

Abstract

Receiver operating characteristic (ROC) analysis is now a standard tool for the comparison of binary classifiers and the selection operating parameters when the costs of misclassification are unknown.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. N.M. Adams and D.J. Hand. Comparing classifiers when the misallocation costs are uncertain. Pattern Recognition, 32:1139–1147, 1999.

    Article  Google Scholar 

  2. M. Anastasio, M. Kupinski, and R. Nishikawa. Optimization and FROC analysis of rule-based detection schemes using a multiobjective approach. IEEE Transactions on Medical Imaging, 17:1089–1093, 1998.

    Article  Google Scholar 

  3. C. Bishop. Neural Networks for Pattern Recognition. Clarendon Press, Oxford, 1995.

    Google Scholar 

  4. A.P. Bradley. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition, 30:1145–1159, 1997.

    Article  Google Scholar 

  5. C.A. Coello Coello. A Comprehensive Survey of Evolutionary-Based Multiobjective Optimization Techniques. Knowledge and Information Systems. An International Journal, 1(3):269–308, 1999.

    Google Scholar 

  6. K. Deb. Multi-Objective Optimization Using Evolutionary Algorithms. Wiley, Chichester, 2001.

    MATH  Google Scholar 

  7. K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. Fast and Elitist Multiobjective Genetic Algorithm: NSGA–II. IEEE Transactions on Evolutionary Computation, 6(2):182–197, 2002.

    Article  Google Scholar 

  8. R.O. Duda and P.E. Hart. Pattern Classification and Scene Analysis. Wiley, New York, 1973.

    MATH  Google Scholar 

  9. C. Elkan. The foundations of cost-sensitive learning. In IJCAI, pages 973–978, 2001.

    Google Scholar 

  10. R.M. Everson and J.E. Fieldsend. Multi-class ROC analysis from a multiobjective optimisation perspective. Technical Report 421, Department of Computer Science, University of Exeter, April 2005.

    Google Scholar 

  11. R.M. Everson and J.E. Fieldsend. Multi-objective optimisation of safety related systems: An application to short term conflict alert. IEEE Transactions on Evolutionary Computation, 2006. (In press).

    Google Scholar 

  12. C. Ferri, J. Hernández-Orallo, and M.A. Salido. Volume under the ROC surface for multi-class problems. In ECML 2003, pages 108–120, 2003.

    Google Scholar 

  13. J.E. Fieldsend and R.M. Everson. ROC Optimisation of Safety Related Systems. In J. Hernández-Orallo, C. Ferri, N. Lachiche, and P. Flach, editors, Proceedings of ROCAI 2004, part of the 16th European Conference on Artificial Intelligence (ECAI), pages 37–44, 2004.

    Google Scholar 

  14. J.E. Fieldsend and R.M. Everson. Visualisation of multi-class ROC surfaces. In Proceedings of ROCML 2005, part of the 22nd International Conference on Machine Learning (ICML 2005), forthcoming, 2005.

    Google Scholar 

  15. J.E. Fieldsend, R.M. Everson, and S. Singh. Using Unconstrained Elite Archives for Multi-Objective Optimisation. IEEE Transactions on Evolutionary Computation, 7(3):305–323, 2003.

    Article  Google Scholar 

  16. P. Flach, H. Blockeel, C. Ferri, J. Hernández-Orallo, and J. Struyf. Decision support for data mining: Introduction to ROC analysis and its applications. In D. Mladenic, N. Lavrac, M. Bohanec, and S. Moyle, editors, Data Mining and Decision Support: Integration and Collaboration, pages 81–90. Kluwer, 2003.

    Google Scholar 

  17. C.M. Fonseca and P.J. Fleming. An Overview of Evolutionary Algorithms in Multiobjective Optimization. Evolutionary Computation, 3(1):1–16, 1995.

    Google Scholar 

  18. D.J. Hand and R.J. Till. A simple generalisation of the area under the ROC curve for multiple class classification problems. Machine Learning, 45:171–186, 2001.

    Article  MATH  Google Scholar 

  19. J.A. Hanley and B.J. McNeil. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 82(143):29–36, 1982.

    Google Scholar 

  20. J. Hernández-Orallo, C. Ferri, N. Lachiche, and P. Flach, editors. ROC Analysis in Artificial Intelligence, 1st International Workshop, ROCAI-2004, Valencia, Spain, 2004.

    Google Scholar 

  21. M. T. Jensen. Reducing the Run-Time Complexity of Multiobjective EAs: The NSGA-II and Other Algorithms. IEEE Transactions on Evolutionary Computation, 7(5):503–515, 2003.

    Article  Google Scholar 

  22. J. Knowles and D. Corne. The Pareto Archived Evolution Strategy: A new baseline algorithm for Pareto multiobjective optimisation. In Proceedings of the 1999 Congress on Evolutionary Computation, pages 98–105, Piscataway, NJ, 1999. IEEE Service Center.

    Chapter  Google Scholar 

  23. J.D. Knowles and D. Corne. Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy. Evolutionary Computation, 8(2):149–172, 2000.

    Article  Google Scholar 

  24. T. Kohonen. Self-Organising Maps. Springer, 1995.

    Google Scholar 

  25. M.A. Kupinski and M.A. Anastasio. Multiobjective Genetic Optimization of Diagnostic Classifiers with Implications for Generating Receiver Operating Characterisitic Curves. IEEE Transactions on Medical Imaging, 18(8):675–685, 1999.

    Article  Google Scholar 

  26. M. Laumanns, L. Thiele, and E. Zitzler. Running Time Analysis of Multiobjective Evolutionary Algorithms on Pseudo-Boolean Functions. IEEE Transactions on Evolutionary Computation, 8(2):170–182, 2004.

    Article  Google Scholar 

  27. M. Laumanns, L. Thiele, E. Zitzler, E. Welzl, and K. Deb. Running Time Analysis of Multi-objective Evolutionary Algorithms on a Simple Discrete Optimization Problem. In J.J. Merelo Guervós, P. Adamidis, H-G Beyer, J-L Fernández-Villacañas, and H-P Schwefel, editors, Parallel Problem Solving from Nature–PPSN VII, Lecture Notes in Computer Science, pages 44–53. Springer- Verlag, 2002.

    Google Scholar 

  28. D. Lowe and M. E. Tipping. Feed-forward neural networks and topographic mappings for exploratory data analysis. Neural Computing and Applications, 4:83–95, 1996.

    Article  Google Scholar 

  29. D. Mossman. Three-way ROCs. Medical Decision Making, 19(1):78–89, 1999.

    Article  Google Scholar 

  30. F. Provost and T. Fawcett. Analysis and visualisation of classifier performance: Comparison under imprecise class and cost distributions. In Proceedings of the Third International Conference on Knowledge Discovery and Data Mining, pages 43–48, Menlo Park, CA, 1997. AAAI Press.

    Google Scholar 

  31. F. Provost and T. Fawcett. Robust classification systems for imprecise environments. In Proceedings of the Fifteenth National Conference on Artificial Intelligence, pages 706–7, Madison, WI, 1998. AAAI Press.

    Google Scholar 

  32. B.D. Ripley. Neural networks and related methods for classification (with discussion). Journal of the Royal Statistical Society Series B, 56(3):409–456, 1994.

    MATH  MathSciNet  Google Scholar 

  33. M.J.J. Scott, M. Niranjan, and R.W. Prager. Parcel: feature subset selection in variable cost domains. Technical Report CUED/F-INFENG/TR.323, Cambridge University Engineering Department, 1998.

    Google Scholar 

  34. F. Tortorella, editor. Pattern Recognition Letters: Special Issue on ROC Analysis in Pattern Recognition, volume 26, 2006.

    Google Scholar 

  35. D. Van Veldhuizen and G. Lamont. Multiobjective Evolutionary Algorithms: Analyzing the State-of-the-Art. Evolutionary Computation, 8(2):125–147, 2000.

    Article  Google Scholar 

  36. X. Yao, Y. Liu, and G. Lin. Evolutionary Programming Made Faster. IEEE Transactions on Evolutionary Computation, 3(2):82–102, 1999.

    Article  Google Scholar 

  37. E. Zitzler and L. Thiele. Multiobjective Evolutionary Algorithms: A Comparative Case Study and the Strength Pareto Approach. IEEE Transactions on Evolutionary Computation, 3(4):257–271, 1999.

    Article  Google Scholar 

  38. M.H. Zweig and G. Campbell. Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clinical Chemistry, 39:561–577, 1993.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Engineering, Computer Science and Mathematics, University of Exeter, Exeter, EX4 4QF, UK

    Richard M. Everson & Jonathan E. Fieldsend

Authors
  1. Richard M. Everson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jonathan E. Fieldsend
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Honda Research Institute Europe GmbH, Carl-Legien-Str. 30, Offenbach, 63073, Germany

    Yaochu Jin Dr.

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer

About this chapter

Cite this chapter

Everson, R.M., Fieldsend, J.E. (2006). Multi-Objective Optimisation for Receiver Operating Characteristic Analysis. In: Jin, Y. (eds) Multi-Objective Machine Learning. Studies in Computational Intelligence, vol 16. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-33019-4_23

Download citation

  • DOI: https://doi.org/10.1007/3-540-33019-4_23

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-30676-4

  • Online ISBN: 978-3-540-33019-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation