Classification and Survival Analysis Using Multi-objective Evolutionary Algorithms

  • Christian Setzkorn
Chapter
Part of the Studies in Computational Intelligence book series (SCI, volume 98)

Model extraction from data has at least three objectives. It aims to produce accurate, comprehensible and interesting models. Hence, multi-objective evolutionary algorithms are a natural choice to tackle this problem. They are capable of optimizing several incommensurable objectives in as single run without making any assumptions about the importance of each objective. This chapter proposes several multi-objective evolutionary algorithms to tackle three different model extraction tasks. The first approach performs supervised classification whilst overcoming some of the shortcomings of existing approaches. The second and third approach tackle two survival analysis problems. All approaches are evaluated on artificial, benchmark and real-world medical data sets.

Keywords

Variation Operator Uveal Melanoma Lifetime Distribution Rule System Fuzzy Rule System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Freitas A A, Rozenberg G (2002) Data mining and knowledge discovery with evolutionary algorithms. Springer Verlag, Berlin, Heidelberg, New YorkMATHGoogle Scholar
  2. 2.
    Sobin L H, Wittekind C (2002) Classification of malignant tumours. Wiley-LissGoogle Scholar
  3. 3.
    Russell S J, Norvig P (1994) Artificial intelligence: A modern approach. Prentice HallGoogle Scholar
  4. 4.
    Holland J H (1975) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. University of Michigan Press, Ann ArborGoogle Scholar
  5. 5.
    Michie D, Spiegelhalter D J, Taylor C C (1994) Machine learning, neural and statistical classification. Ellis HorwoodGoogle Scholar
  6. 6.
    Krause P, Clark D (1993) Representing uncertain knowledge: An artificial intelligence approach. Kluwer Academic PublishersGoogle Scholar
  7. 7.
    Michalewicz Z, Fogel D B (2005) How to solve it: modern heuristics. 2nd Edition, Springer, BerlinGoogle Scholar
  8. 8.
    Gordon A D (1981) Classification. Chapman and HallGoogle Scholar
  9. 9.
    Bishop C M (1995) Neural networks for pattern recognition. Oxford University Press, OxfordGoogle Scholar
  10. 10.
    Kleinbaum D G (1996) Survival analysis: A self-learning text. SpringerGoogle Scholar
  11. 11.
    Afifi A, Clark V A, May S (2003) Computer-aided multivariate analysis. Chapman and HallGoogle Scholar
  12. 12.
    Kalbfleisch J D, Prentice R L (1980) The statistical analysis of failure time data. WileyGoogle Scholar
  13. 13.
    Evans M, Hastings N, Peacock B (1993) Statistical distributions. John Wiley and SonsGoogle Scholar
  14. 14.
    Lawless J F (2002) Statistical models and methods for lifetime data. Wiley, New YorkGoogle Scholar
  15. 15.
    Damato B E (2000) Ocular tumours : diagnosis and treatment. Butterworth HeinemannGoogle Scholar
  16. 16.
    Klir G J, Clair U S, Yuan B (1997) Fuzzy set theory: foundations and applications. Prentice Hall, Upper Saddle River, NJMATHGoogle Scholar
  17. 17.
    Deb K (2001) Multi-objective optimization using evolutionary algorithms. Wiley ChichesterGoogle Scholar
  18. 18.
    Quinlan J R (1994) C4.5 : programs for machine learning. Morgan KaufmannGoogle Scholar
  19. 19.
    Duda R O, Hart P E, Stork H G (2000) Pattern classification. Wiley-Interscience, New YorkGoogle Scholar
  20. 20.
    Silverman B W (1999) Density estimation for statistics and data analysis. Chapman and HallGoogle Scholar
  21. 21.
    Fayyad U M, Piatetsky-Shapiro G, Smyth P (1996) From data mining to knowledge discovery in databases. Advances in Knowledge Discovery and Data Mining, AAAI Press/The MIT Press 1–36Google Scholar
  22. 22.
    Setzkorn C, Taktak A F, Damato B (2006) Evolving oblique decision trees for survival analysis. Poster Proceedings of the 6th Industrial Conference on Data Mining, Springer 144–158Google Scholar
  23. 23.
    Freitas A A (2000) Evolutionary algorithms. Handbook of Data Mining and Knowledge DiscoveryGoogle Scholar
  24. 24.
    Provost F, Fawcett T, Kohavi R (1998) The case against accuracy estimation for comparing induction algorithms. Proc. 15th International Conf. on Machine Learning, Morgan Kaufmann, San Francisco, CA 445–453Google Scholar
  25. 25.
    Fonseca C M, Fleming P J (1995) Multiobjective genetic algorithms made easy: selection, sharing, and mating restriction. Proceedings of the First International Conference on Genetic Algorithms in Engineering Systems: Innovations and Applications, Sheffield, UK 42–52Google Scholar
  26. 26.
    Holland J H, Reitman J S (2002) Cognitive systems based on adaptive algorithms. In: D. A. Waterman and F. Hayes-Roth (eds) Pattern Directed Inference Systems, Academic Press 313–329Google Scholar
  27. 27.
    Holland J H (1986) Escaping brittleness: The possibility of general-purpose learning algorithms applied to rule-based systems. In: R. S. Michalski and J. G. Carbonell and T. M. Mitchell (eds) Machine Learning: An Artificial Intelligence Approach, Volume II, Morgan Kaufmann 593–623Google Scholar
  28. 28.
    Fawcett T (2001) Using rule sets to maximize ROC performance. Proceedings of the IEEE International Conference on Data Mining, IEEE Computer Society 131–138Google Scholar
  29. 29.
    Spears W M (1995) Adapting crossover in evolutionary algorithms. In: J. R. McDonnell and R. G. Reynolds and D. B. Fogel (eds) Proc. of the Fourth Annual Conference on Evolutionary Programming, MIT Press, Cambridge, MA 367–384Google Scholar
  30. 30.
    Laumanns M, Thiele L, Zitzler E, Deb K (2002) Archiving with guaranteed convergence and diversity in multi-objective optimization. In: Langdon W B, Cantú-Paz E, Mathias K, Roy R, Davis D, Poli R, Balakrishnan K, Honavar V, Rudolph G, Wegener J, Bull L Potter M A, Schultz A C, Miller J F, Burke E, Jonoska N (eds) GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, Morgan Kaufmann Publishers, San Francisco, CA 94104, USA 439–447Google Scholar
  31. 31.
    Setzkorn C, Paton R C (2004) JavaSpaces-an affordable technology for the simple implementation of reusable parallel evolutionary algorithms. In: López J A, Benfenati E, Dubitzky W (eds) Knowledge Exploration in Life Science Informatics - KELSI 2004 (LNAI 3303), Springer-Verlag, New York 151–161Google Scholar
  32. 32.
    Laumanns M, Zitzler E, Thiele L (2000) A unified model for multi-objective evolutionary algorithms with elitism. Proceedings of the 2000 Congress on Evolutionary Computation (CEC 2000). IEEE Press, Piscataway, New Jersey 46–53CrossRefGoogle Scholar
  33. 33.
    Hoffmann F, Pfister G (1995) A new learning method for the design of hierarchical fuzzy controllers using messy genetic algorithms. Proceedings of the Sixth International Fuzzy Systems Association World Congress (IFSA’95). Sao Paulo, Brazil 249–252Google Scholar
  34. 34.
    Clearwater S, Provost F (1990) RL4: A tool for knowledge-based induction. Proceedings of the Second International IEEE Conference on Tools for Artificial Intelligence 24–30Google Scholar
  35. 35.
    Duch W, Jankowski N, Grabczewski K, Adamczak R (2000) Optimization and interpretation of rule-based classifiers. Intelligent Information Systems, Advances in Soft Computing 1–14Google Scholar
  36. 36.
    Nürnberger A, Klose A, Kruse R (2000) Analyzing borders between partially contradicting fuzzy classification rules. Proceedings of 19th International Conference of the North American Fuzzy Information Processing Society (NAFIPS 2000) 59–63Google Scholar
  37. 37.
    Cramer N L (1985) A representation for the adaptive generation of simple sequential programs. Proceedings of the International Conference on Genetic Algorithms and Their Applications 183–187Google Scholar
  38. 38.
    Koza J R (1998) Genetic programming. In: Williams James G, Kent A (eds) Encyclopedia of Computer Science and Technology, Marcel-Dekker 29–43Google Scholar
  39. 39.
    Zitzler E, Laumanns M, Thiele L (2001) SPEA2: Improving the strength pareto evolutionary algorithm. EUROGEN 2001 - Evolutionary Methods for Design, Optimisation and Control with Applications to Industrial Problems 19–26Google Scholar
  40. 40.
    Burges C J C (1998) A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2(2):121–167CrossRefGoogle Scholar
  41. 41.
    Provost F J, Aronis J M (1996) Scaling up inductive learning with massive parallelism. Machine Learning 23(1):33–46Google Scholar
  42. 42.
    Dhar V, Chou D, Provost F J (2000) Discovering interesting patterns for investment decision making with GLOWER - A genetic learner overlaid with entropy reduction. Data Mining and Knowledge Discovery 4(4):251–280MATHCrossRefGoogle Scholar
  43. 43.
    Eiben A E, Hinterding R, Michalewicz Z (1999) Parameter control in evolutionary algorithms. IEEE Trans. on Evolutionary Computation 3(2):124–141CrossRefGoogle Scholar
  44. 44.
    Grohman W M, Dhawan A P (2001) Fuzzy convex set-based pattern classification for analysis of mammographic microcalcifications. Pattern Recognition 34:1469–1482MATHCrossRefGoogle Scholar
  45. 45.
    Nauck D, Kruse R (1999) Obtaining interpretable fuzzy classification rules from medical data. Artificial Intelligence in Medicine 16:149–169CrossRefGoogle Scholar
  46. 46.
    Russo M (1997) FuGeNeSys - a fuzzy genetic neural system for fuzzy modeling. IEEE Transactions On Fuzzy Systems 6(3):373–388CrossRefGoogle Scholar
  47. 47.
    Jain A, Duin P, Mao J (2000) Statistical pattern recognition: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(1):4–37CrossRefGoogle Scholar
  48. 48.
    Damato B E (2005) Current management of uveal melanoma. European Journal of Cancer Supplements 3(3):433–435CrossRefGoogle Scholar
  49. 49.
    Singer J D, Willett J B (1993) It’s about time: Using discrete-time survival analysis to study duration and the timing of events. Journal of Educational Statistics 18(2):155–195CrossRefGoogle Scholar
  50. 50.
    Setzkorn C, Taktak A F, Damato B (2006) On the use of multi-objective evolutionary algorithms for survival analysis. BioSystems (in press).Google Scholar
  51. 51.
    Freireich E J et al.(1963) The Effect of 6-mercaptopurine on the duration of steroid-induced remissions in acute leukemia. Blood 21:699–716Google Scholar
  52. 52.
    Setzkorn C, Paton R C (2005) On the use of multi-objective evolutionary algorithms for the induction of fuzzy classification rule systems. BioSystems 81(2):101–112CrossRefGoogle Scholar
  53. 53.
    Eleuteri A, Tagliaferri R, Milano L, De Placido S, De Laurentiis M (2003) A novel neural network-based survival analysis model. Neural Networks 16(5–6):855–864CrossRefGoogle Scholar
  54. 54.
    Hand D J, Till R J (2001) A simple generalization of the area under the ROC curve for multiple class classification problems. Machine Learning 45:171–186MATHCrossRefGoogle Scholar
  55. 55.
    Hanley J, McNeil B J (1982) The meaning and use of the area under a receiver operating characteristic ROC curve. Radiology 143:29–36Google Scholar
  56. 56.
    Bradley A (1997) The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition 30(7):1145–1159CrossRefGoogle Scholar
  57. 57.
    Cordón O, del Jesus M J, Herrera F, Lozano M (1999) A proposal on reasoning methods in fuzzy rule-based classification systems. International Journal of Approximate Reasoning 20:21–45Google Scholar
  58. 58.
    Steimann F (1997) Fuzzy set theory in medicine. Artificial Intelligence in Medicine 11(1):1–7CrossRefGoogle Scholar
  59. 59.
    Zadeh L A (1965) Fuzzy sets. Information and Control 8(3): 338–353MATHCrossRefMathSciNetGoogle Scholar
  60. 60.
    Biganzoli E, Boracchi P, Mariani L, Marubini E (1998) Feed forward neural networks for the analysis of censored survival data: a partial logistic regression approach. Statistics in Medicine 17(10):1169–1186CrossRefGoogle Scholar
  61. 61.
    Clark T G, Bradburn M J, Love S B, Altman D G (2003) Survival analysis part IV: Further concepts and methods in survival analysis. British Journal of Cancer 89:781–786CrossRefGoogle Scholar
  62. 62.
    De Jong K A (1975) An analysis of the behaviour of a class of genetic adaptive systems. PhD thesis, University of MichiganGoogle Scholar
  63. 63.
    Setzkorn C (2005) On the use of multi-objective evolutionary algorithms for classification rule induction. PhD Thesis, University of Liverpool, Department of Computer Science Liverpool, United KingdomGoogle Scholar
  64. 64.
    Smith S F (1980) A learning system based on genetic algorithm. PhD Thesis, University of PittburghGoogle Scholar
  65. 65.
    Setzkorn C, Paton R C (2003) MERBIS - A multi-objective evolutionary rule base induction system. ULCS-03-016, University of LiverpoolGoogle Scholar
  66. 66.
    Sebag M, Azé J, Lucas N (2004) ROC-based evolutionary learning: application to medical data mining. Artificial Evolution, 384–396Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Christian Setzkorn
    • 1
  1. 1.National Center for Zoonosis ResearchUniversity of LiverpoolUK

Personalised recommendations