Applied Intelligence

, Volume 7, Issue 1, pp 39–55

Overcoming the Myopia of Inductive Learning Algorithms with RELIEFF

  • Igor Kononenko
  • Edvard Šimec
  • Marko Robnik-Šikonja
Article

DOI: 10.1023/A:1008280620621

Cite this article as:
Kononenko, I., Šimec, E. & Robnik-Šikonja, M. Applied Intelligence (1997) 7: 39. doi:10.1023/A:1008280620621

Abstract

Current inductive machine learning algorithms typically use greedy search with limited lookahead. This prevents them to detect significant conditional dependencies between the attributes that describe training objects. Instead of myopic impurity functions and lookahead, we propose to use RELIEFF, an extension of RELIEF developed by Kira and Rendell [10, 11], for heuristic guidance of inductive learning algorithms. We have reimplemented Assistant, a system for top down induction of decision trees, using RELIEFF as an estimator of attributes at each selection step. The algorithm is tested on several artificial and several real world problems and the results are compared with some other well known machine learning algorithms. Excellent results on artificial data sets and two real world problems show the advantage of the presented approach to inductive learning.

learning from examplesestimating attributesimpurity functionRELIEFFempirical evaluation

Copyright information

© Kluwer Academic Publishers 1997

Authors and Affiliations

  • Igor Kononenko
    • 1
  • Edvard Šimec
    • 1
  • Marko Robnik-Šikonja
    • 1
  1. 1.University of LjubljanaLjubljanaSlovenia E-mail