Advertisement

The effect of numeric features on the scalability of inductive learning programs

  • Georgios Paliouras
  • David S. Brée
Papers
Part of the Lecture Notes in Computer Science book series (LNCS, volume 912)

Abstract

The behaviour of a learning program as the quantity of data increases affects to a large extent its applicability on real-world problems. This paper presents the results of a theoretical and experimental investigation of the scalability of four well-known empirical concept learning programs. In particular it examines the effect of using numeric features in the training set. The theoretical part of the work involved a detailed worst-case computational complexity analysis of the algorithms. The results of the analysis deviate substantially from previously reported estimates, which have mainly examined discrete and finite feature spaces. In order to test these results, a set of experiments was carried out, involving one artificial and two real data sets. The artificial data set introduces a near-worst-case situation for the examined algorithms, while the real data sets provide an indication of their average-case behaviour.

Keywords

empirical concept learning scalability decision trees numeric features 

References

  1. 1.
    J. Catlett. Megainduction: a test flight. In Proceedings of the Eighth International Workshop in Machine Learning, pages 596–599, 1991.Google Scholar
  2. 2.
    P. Clark and T. Niblett. The CN2 Algorithm. Machine Learning, 3(4):261–283, 1989.Google Scholar
  3. 3.
    T. V. de Merckt. Decision Trees in Numerical Attribute Spaces. In Proceedings of the Int. Joint Conf. on Artificial Intelligence, pages 1016–1021, 1993.Google Scholar
  4. 4.
    P. Errington and J. Graham. Application of Artificial Neural Networks to Chromosome Classification. Cytometry, 14:627–639, 1993.Google Scholar
  5. 5.
    U. Fayyad and K. Irani. Mutli-Interval Discretization of Continuous-Valued Attributes for Classification Learning. In Proceedings of the Int. Joint Conf. on Artificial Intelligence, pages 1022–1027, 1993.Google Scholar
  6. 6.
    D. Fisher and K. McKusick. An Empirical Comparison of ID3 and Back-propagation. In Proceedings of the Int. Joint Conf. on Artificial Intelligence, pages 788–793, 1991.Google Scholar
  7. 7.
    P. Frey and D. Slate. Letter Recognition Using Holland-Style Adaptive Classifiers. Machine Learning, 6:161–182, 1991.Google Scholar
  8. 8.
    M. Gams and N. Lavrač. Review of Five Empirical Learning Systems Within a Proposed Schemata. In Proceedings of 2nd European Workshop on Machine Learning, pages 46–66, 1987.Google Scholar
  9. 9.
    R. Michalski. A Theory and Methodology of Inductive Learning. In R. Michalski, J. Carbonell, and T. Mitchell, editors, Machine Learning: An Artificial Intelligence Approach, pages 83–138. Kaufmann, 1983.Google Scholar
  10. 10.
    R. Michalski, I. Mozetic, J. Hong, and N. Lavrač. The Multi-purpose incremental learning system AQ15 and its testing application to three medical domains. In AAAI Proceedings, pages 1041–1045, 1986.Google Scholar
  11. 11.
    D. Michie, D. Spiegelhalter, and C. Taylor. Machine Learning, Neural and Statistical Classification. Ellis Harwood, 1994.Google Scholar
  12. 12.
    R. Mooney, J. Shavlik, G. Towell, and A. Gove. An Experimental Comparison of Symbolic and Connectionist Learning Algorithms. In Proceedings of the Int. Joint Conf. on Artificial Intelligence, pages 775–780, 1991.Google Scholar
  13. 13.
    P. Murphy and D. Aha. UCI Repository of machine learning databases. Machine Readable data repository, 1994.Google Scholar
  14. 14.
    S. Murthy, S. Kasif, S. Salzberg, and R. Beigel. OC1: Randomised Induction of Oblique Decision Trees. In AAAI Proceedings, pages 322–327, 1993.Google Scholar
  15. 15.
    P. O'Rorke. A Comparative Study of Inductive Learning Systems AQ11P and ID3 Using a Chess-End Game Problem. Technical Report ISG 82-2, Computer Science Department, University of Illinois at Urbana-Champaign, 1982.Google Scholar
  16. 16.
    G. Paliouras. The Scalability of Machine Learning Algorithms. Master's thesis, Department of Computer Science, University of Manchester, 1993.Google Scholar
  17. 17.
    J. Quinlan. Learning Efficient Classification Procedures and Their Application to Chess End Games. In R. Michalski, J. Carbonell, and T. Mitchell, editors, Machine Learning: An Artificial Intelligence Approach, pages 463–482. Kaufmann, 1983.Google Scholar
  18. 18.
    J. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, Inc., San Mateo, CA, 1993.Google Scholar
  19. 19.
    L. Rendell. A New Basis for State-Space Learning Systems and a Successful Implementation. Artificial Intelligence, 20(4):369–392, 1983.Google Scholar
  20. 20.
    L. Rendell. Conceptual Knowledge Acquisition in Search. In L. Bolc, editor, Computational Models of Learning, pages 89–159. Springer Verlag, 1987.Google Scholar
  21. 21.
    L. Rendell, H. Cho, and R. Seshu. Improving the Design of Similarity-Based Rule-Learning Systems. International Journal of Expert Systems, 2:97–133, 1989.Google Scholar
  22. 22.
    P. Utgoff. Incremental Induction of Decision Trees. Machine Learning, 4(2):161–186, 1989.Google Scholar
  23. 23.
    S. Weiss and I. Kapouleas. An Empirical Comparison of Pattern Recognition, Neural Nets, and Machine Learning Classification Methods. In Proceedings of the Int. Joint Conf. on Artificial Intelligence, pages 781–787, 1991.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • Georgios Paliouras
    • 1
  • David S. Brée
    • 1
  1. 1.Department of Computer ScienceUniversity of ManchesterManchesterUK

Personalised recommendations