Induction of Decision Multi-trees Using Levin Search

  • C. Ferri-Ramírez
  • J. Hernández-Orallo
  • M. J. Ramírez-Quintana
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2329)


In this paper, we present a method for generating very expressiv e decision trees over a functional logic language. The generation of the tree folio ws a short-to-long search which is guided by the MDL principle. Once a solution is found, the construction of the tree goes on in order to obtain more solutions ordered as well by description length. The result is a multi-tree which is populated taking into consideration computational resources according to a Levin search. Some experiments show that the method pays off in practice.


Machine Learning Decision-tree Induction Inductive Logic Programming (ILP) Levin search Minimum Description Length (MDL) 


  1. 1.
    E. Bauer and R. Kohavi. An empirical comparison of voting classification algorithms: Bagging, boosting and variants. Machine Learning, 36:105–139, 1999.CrossRefGoogle Scholar
  2. 2.
    Leo Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth Publishing Company, 1984.Google Scholar
  3. 3.
    T. Dean and M. Boddy. An analysis of time-dependent planning. In Proc. of the 1th National Conference on Artificial Intelligence, pages 49–54, 1988.Google Scholar
  4. 4.
    C. Ferri, J. Hernández, and M.J. Ramírez. The FLIP system homepage., 2000.
  5. 5.
    C. Ferri, J. Hernández, and M.J. Ramírez. Learning MDL-guided Decision Trees for Constructor-Based Languages. In WIP track of 11th Int. Conf. on Inductive Logic Progr,ILP01, pages 39–50, 2001.Google Scholar
  6. 6.
    Y. Freund and R.E. Schapire. Experiments with a new boosting algorithm. In Proc. of the 13th Int. Conf. on Machine Learning (ICML’1996), pages 148–156. Morgan Kaufmann, 1996.Google Scholar
  7. 7.
    M. Hanus. The Integration of Functions into Logic Programming: From Theory to Practice. Journal of Logic Programming, 19–20:583-628, 1994.Google Scholar
  8. 8.
    SPSS Inc. Clementine homepage,
  9. 9.
    L.A. Levin. Universal Search Problems. Problems Inform. Transmission, 9:265–266, 1973.Google Scholar
  10. 10.
    M. Li and P. Vitányi. An Introduction to Kolmogorov Complexity and its Applications. 2nd Ed. Springer-Verlag, 1997.Google Scholar
  11. 11.
    M. Mehta, J. Rissanen, and R. Agrawal. MDL-Based Decision Tree Pruning. In Proc. of the 1st Int. Conf. on Knowledge Discovery and Data Mining (KDD’95), pages 216–221, 1995.Google Scholar
  12. 12.
    N.J. Nilsson. Artficial Intelligence: a new synthesis. Morgan Kaufmann, 1998.Google Scholar
  13. 13.
    University of California. UCI Machine Learning Repository Content Summary.
  14. 14.
    N.C. Berkman P.E. Utgoff and J.A. Clouse. Decision tree induction based on efficient tree restructuring. Machine Learning, 29(l):5–44, 1997.zbMATHGoogle Scholar
  15. 15.
    B. Pfahringer. Compression-based discretization of continuous attributes. In Proc. 12th International Conference on Machine Learning, pages 456–463. Morgan Kaufmann, 1995.Google Scholar
  16. 16.
    J. R. Quinlan. Induction of Decision Trees. In Read, in Machine Learning. M. Kaufmann, 1990.Google Scholar
  17. 17.
    J. R. Quinlan. Learning Logical Definitions from Relations. M.L.J, 5(3):239–266, 1990.Google Scholar
  18. 18.
    J. R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993.Google Scholar
  19. 19.
    J. R. Quinlan. Bagging, Boosting, and C4.5. In Proc. of the 13th Nat. Conf. on A.I. and the Eighth Innovative Applications of A.I. Conf., pages 725–730. AAAI Press / MIT Press, 1996.Google Scholar
  20. 20.
    J. R. Quinlan and R. L. Rivest. Inferring Decision Trees Using The Minimum Description Length Principle. Information and Computation, 80:227–248, 1989.zbMATHCrossRefMathSciNetGoogle Scholar
  21. 21.
    J. Rissanen. Modelling by shortest data description. Automatica, 14:465–471, 1978.zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • C. Ferri-Ramírez
    • 1
  • J. Hernández-Orallo
    • 1
  • M. J. Ramírez-Quintana
    • 1
  1. 1.DSIC, UPVValenciaSpain

Personalised recommendations