Estimating attributes: Analysis and extensions of RELIEF

  • Igor Kononenko
Regular Papers
Part of the Lecture Notes in Computer Science book series (LNCS, volume 784)


In the context of machine learning from examples this paper deals with the problem of estimating the quality of attributes with and without dependencies among them. Kira and Rendell (1992a,b) developed an algorithm called RELIEF, which was shown to be very efficient in estimating attributes. Original RELIEF can deal with discrete and continuous attributes and is limited to only two-class problems. In this paper RELIEF is analysed and extended to deal with noisy, incomplete, and multi-class data sets. The extensions are verified on various artificial and one well known real-world problem.


Information Gain Gini Index Training Instance Original Attribute Independent Attribute 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Breiman L., Friedman J.H., Olshen R.A., Stone C.J.: Classification and Regression Trees. Wadsforth International Group 1984Google Scholar
  2. 2.
    Hunt, E., Martin, J. & Stone, P.: Experiments in Induction. New York: Academic Press 1966Google Scholar
  3. 3.
    Kira, K. & Rendell, L.: A practical approach to feature selection. In: Proc. Intern. Conf. on Machine Learning. (Aberdeen, July 1992) D. Sleeman & P. Edwards (eds.), Morgan Kaufmann 1992, pp. 249–256Google Scholar
  4. 4.
    Kira K. & Rendell L.: The feature selection problem: traditional methods and new algorithm. In: Proc. AAAI'92. San Jose, CA, July 1992Google Scholar
  5. 5.
    Kononenko I., Bratko I., Roškar E.: Experiments in inductive learning of medical diagnostic rules. In: Proc. International School for the Synthesis of Expert Knowledge Workshop. Bled, Slovenia, August 1984.Google Scholar
  6. 6.
    Kononenko I.: Semi-naive Bayesian classifier. In: Proc. European Working Session on Learning, (Porto, March 1991), Y. Kodratoff (ed.), Springer Verlag 1991, pp.206–219Google Scholar
  7. 7.
    Mantaras R.L.: ID3 Revisited: A distance based criterion for attribute selection. In: Proc. Int. Symp. Methodologies for Intelligent Systems. Charlotte, North Carolina, U.S.A., Oct. 1989Google Scholar
  8. 8.
    Quinlan, R.: Induction of decision trees. Machine learning 1, 81–106 (1986)Google Scholar
  9. 9.
    Smyth P. & Goodman R.M.: Rule induction using information theory. In. G. Piatetsky-Shapiro & W. Frawley (eds.): Knowledge Discovery in Databases. MIT Press 1990Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1994

Authors and Affiliations

  • Igor Kononenko
    • 1
  1. 1.Faculty of Electrical Engineering & Computer ScienceUniversity of LjubljanaLjubljanaSlovenia

Personalised recommendations