Induction of Decision Multi-trees Using Levin Search
In this paper, we present a method for generating very expressiv e decision trees over a functional logic language. The generation of the tree folio ws a short-to-long search which is guided by the MDL principle. Once a solution is found, the construction of the tree goes on in order to obtain more solutions ordered as well by description length. The result is a multi-tree which is populated taking into consideration computational resources according to a Levin search. Some experiments show that the method pays off in practice.
KeywordsMachine Learning Decision-tree Induction Inductive Logic Programming (ILP) Levin search Minimum Description Length (MDL)
- 2.Leo Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth Publishing Company, 1984.Google Scholar
- 3.T. Dean and M. Boddy. An analysis of time-dependent planning. In Proc. of the 1th National Conference on Artificial Intelligence, pages 49–54, 1988.Google Scholar
- 4.C. Ferri, J. Hernández, and M.J. Ramírez. The FLIP system homepage. http://www.dsic.upv.es/~flip/, 2000.
- 5.C. Ferri, J. Hernández, and M.J. Ramírez. Learning MDL-guided Decision Trees for Constructor-Based Languages. In WIP track of 11th Int. Conf. on Inductive Logic Progr,ILP01, pages 39–50, 2001.Google Scholar
- 6.Y. Freund and R.E. Schapire. Experiments with a new boosting algorithm. In Proc. of the 13th Int. Conf. on Machine Learning (ICML’1996), pages 148–156. Morgan Kaufmann, 1996.Google Scholar
- 7.M. Hanus. The Integration of Functions into Logic Programming: From Theory to Practice. Journal of Logic Programming, 19–20:583-628, 1994.Google Scholar
- 8.SPSS Inc. Clementine homepage, http://www.spss.com/clementine/.
- 9.L.A. Levin. Universal Search Problems. Problems Inform. Transmission, 9:265–266, 1973.Google Scholar
- 10.M. Li and P. Vitányi. An Introduction to Kolmogorov Complexity and its Applications. 2nd Ed. Springer-Verlag, 1997.Google Scholar
- 11.M. Mehta, J. Rissanen, and R. Agrawal. MDL-Based Decision Tree Pruning. In Proc. of the 1st Int. Conf. on Knowledge Discovery and Data Mining (KDD’95), pages 216–221, 1995.Google Scholar
- 12.N.J. Nilsson. Artficial Intelligence: a new synthesis. Morgan Kaufmann, 1998.Google Scholar
- 13.University of California. UCI Machine Learning Repository Content Summary. http://www.ics.uci.edu/~mlearn/MLSummary.html.
- 15.B. Pfahringer. Compression-based discretization of continuous attributes. In Proc. 12th International Conference on Machine Learning, pages 456–463. Morgan Kaufmann, 1995.Google Scholar
- 16.J. R. Quinlan. Induction of Decision Trees. In Read, in Machine Learning. M. Kaufmann, 1990.Google Scholar
- 17.J. R. Quinlan. Learning Logical Definitions from Relations. M.L.J, 5(3):239–266, 1990.Google Scholar
- 18.J. R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993.Google Scholar
- 19.J. R. Quinlan. Bagging, Boosting, and C4.5. In Proc. of the 13th Nat. Conf. on A.I. and the Eighth Innovative Applications of A.I. Conf., pages 725–730. AAAI Press / MIT Press, 1996.Google Scholar