Noise detection and elimination applied to noise handling in a KRK chess endgame

  • Dragan Gamberger
  • Nada Lavrač
Experiments and Applications
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1314)


Compression measures used in inductive learners, such as measures based on the MDL (Minimum Description Length) principle, provide a theoretically justified basis for grading candidate hypotheses. Compression-based induction is appropriate also for handling of noisy data. This paper shows that a simple compression measure can be used to detect noisy examples. A technique is proposed in which noisy examples are detected and eliminated from the training set, and a hypothesis is then built from the set of remaining examples. The separation of noise detection and hypothesis formation has the advantage that noisy examples do not influence hypothesis construction as opposed to most standard approaches to noise handling in which the learner typically tries to avoid overfitting the noisy example set. Experimental results in a KRK (king-rook-king) chess endgame domain show the potential of this novel approach to noise handling.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    B. Cestnik and I. Bratko. On estimating probabilities in tree pruning. In Proc. 5th European Working Session on Learning, 138–150, 1991.Google Scholar
  2. 2.
    P. Clark and T. Niblett. The CN2 induction algorithm. Machine Learning, 3(4):261–283, 1989.Google Scholar
  3. 3.
    K.A. De Jong and W.M. Spears. Learning concept classification rules using genetic algorithms. In Proceedings of the 12th International Joint Conference on Artificial Intelligence (IJCAI-91), Morgan Kaufmann, (1991) 651–656.Google Scholar
  4. 4.
    S. Džeroski, N. Lavrac. Inductive learning in deductive databases. IEEE Transactions on Knowledge and Data Engineering, 5(6), 939–949, 1993.Google Scholar
  5. 5.
    D. Gamberger. A minimization approach to propositional inductive learning. In Proceedings of the 8th European Conference on Machine Learning (ECML-95), Springer, (1995) 151–160.Google Scholar
  6. 6.
    D. Gamberger. Specific rule induction for medical domains. In Proc. Computer-Aided Data Analysis in Medicine, CADAM-95, 136–145. US Scientific Publishing IJS-SP-95-1, 1995.Google Scholar
  7. 7.
    D. Gamberger and N. Lavrač. Towards a theory of relevance in inductive concept learning. Technical report IJS-DP-7310. J. Stefan Institute, Ljubljana, 1995.Google Scholar
  8. 8.
    D. Gamberger, N. Lavrač and S. Džeroski. Noise elimination in inductive concept learning: A case study in medical diagnosis. In Proc. Seventh International Workshop on Algorithmic Learning Theory ALT'96, Springer 1996 (in press).Google Scholar
  9. 9.
    N. Lavrač, S. Dzeroski and M. Grobelnik. Learning nonrecursive definitions of relations with LINUS. In Proc. Fifth European Working Session on Learning, pages 265–281, Springer, Berlin, 1991.Google Scholar
  10. 10.
    N. Lavrač and S. Džeroski. Inductive learning of relations from noisy examples. In S. Muggleton (ed.) Inductive Logic Programming, 495–516. Academic Press, 1992.Google Scholar
  11. 11.
    N. Lavrač and S. Džeroski. Inductive Logic Programming: Techniques and Applications. Ellis Horwood (Simon & Schuster), Ellis Horwood Series in Artificial Intelligence. UK: Chichester, 1994.Google Scholar
  12. 12.
    N. Lavrač, D. Gamberger and S. Džeroski. An approach to dimensionality reduction in learning from deductive databases. In Proceedings of the 5th International Workshop on Inductive Logic Programming (ILP-95), Technical report, Katholieke Universiteit Leuven, 1995.Google Scholar
  13. 13.
    N. Lavrač, S. Džroski and I. Bratko. Handling imperfect data in inductive logic programming. In L. De Raedt (ed.) Advances in Inductive Logic Programming, 48–64. IOS Press, 1996.Google Scholar
  14. 14.
    J. Mingers. An empirical comparison of pruning methods for decision tree induction. Machine Learning, 4(2):227–243, 1989.Google Scholar
  15. 15.
    J. Mingers. An empirical comparison of selection measures for decision-tree induction. Machine Learning, 3(4):319–342, 1989.Google Scholar
  16. 16.
    S.H. Muggleton, M. Bain, J. Hayes-Michie and D.Michie. An experimental comparison of human and machine learning formalisms. In Proc. Sixth International Workshop on Machine Learning, 113–118, Morgan Kaufmann, San Mateo, CA, 1989.Google Scholar
  17. 17.
    S. Muggleton, A. Srinivasan and M. Bain. Compression, significance and accuracy. In Proc. 9th International Conference on Machine Learning, 338–347. Morgan Kaufmann, 1992.Google Scholar
  18. 18.
    T. Niblett T and I. Bratko. Learning decision rules in noisy domains. In M. Bramer (ed.) Research and Development in Expert Systems III, 24–25. Cambridge University Press, 1986.Google Scholar
  19. 19.
    R. Quinlan. Simplifying decision trees. International Journal of Man-Machine Studies, 27(3):221–234, 1987.Google Scholar
  20. 20.
    J.R. Quinlan. Learning logical definitions from relations. Machine Learning, 5(3): 239–266, 1990.Google Scholar
  21. 21.
    J. Rissanen. Modeling by shortest data description. Automatica, 14: 465–471, 1978.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1997

Authors and Affiliations

  • Dragan Gamberger
    • 1
  • Nada Lavrač
    • 2
  1. 1.Rudjer Boškovié InstituteZagrebCroatia
  2. 2.Jožef Stefan InstituteLjubljanaSlovenia

Personalised recommendations