Evaluation of learning systems : An artificial data-based approach

  • H. Lounis
  • G. Bisson
Part 8: Applications
Part of the Lecture Notes in Computer Science book series (LNCS, volume 482)


Experimentation has an important role in determining the capacities and restrictions of machine learning (ML) systems. In this paper we present the definition of some sensitivity and evaluation criteria which can be used to perform an evaluation of learning systems. Moreover, in order to overcome some of the limitations of real data sets, we introduce the specification of a parametrable generator of artificial learning sets which allows us to make easily complete experiments to discover some empirical rules of behavior for ML algorithms. Finally, we give some results obtained with different algorithms, showing that artificial data bases approach is an interesting direction to explore.


Evaluation of Machine Learning Algorithms Sensitivity Criteria Evaluation Criteria Artificial Data Base Parametrable Generator Modelization of Learning Domains 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bisson, Laublet 1989, Méthode d'évaluation des systèmes d'apprentissage, Rapport de fin d'étude, ONERA-GIA, August 1989.Google Scholar
  2. Bisson, Laublet 1990, Theoritical and empirical study of learning systems, Cognitiva 1990 Madrid, 1990.Google Scholar
  3. Bisson 1991, Learning of Rule Systems in a First Order Representation, LRI Internal report 1991Google Scholar
  4. Bonelli 1990, A.PARODI, S.SEN, S.WILSON, NewBOOLE: A fast GBML system, 7th ICML, Austin, 1990.Google Scholar
  5. Boswell 1990 a, Manual for CN2, The Turing Institute, January 1990.Google Scholar
  6. Boswell 1990 b, Manual for NewID version 2.1, The Turing Institute, January 1990.Google Scholar
  7. Cannat 1988, Machine Learning Applied to Air Trafic Control, Proc. Human-Machine Interaction Artificial Intelligence Aeronautics & Space, pp.265–274, Toulouse, 1988.Google Scholar
  8. Elomaa 1989, N.HOLSTI, An experimental comparison of inducing decision trees and decision lists in noisy domains, EWSL Montpellier, 1989.Google Scholar
  9. Ganascia 1987, AGAPE et CHARADE: deux techniques d'apprentissage symbolique appliquées à la construction de bases de connaissances, Thèse d'état soutenue 4/87,Université Paris Sud.Google Scholar
  10. Ganascia, Helft 1988, Evaluation des Systèmes d'Apprentissage, 3éme Journées Françaises de l'Apprentissage 1988 Cassis, p 3–20.Google Scholar
  11. Kibler, Langley 1988, Machine Learning as an Experimental Science, EWSL 1988, Glasgow, pp 81–92.Google Scholar
  12. Manago, Kodratoff 1987, Noise and Knowledge Acquisition, Proceeding of 10th IJCAI, Los Altos, 1987Google Scholar
  13. Martin 1989, Reducing Redundant Learning, in Proceedings of the 6o International Workshop on Machine Learning, Ithaca 1989, p 396–99.Google Scholar
  14. Minton 1985, Selectively Generalizing Plans for Problem Solving, IJCAI 85 p596–599.Google Scholar
  15. MLT 1990, Laboratoire de Marcoussis, British Aerospace, Université de Paris-Sud, Evaluation Methods for Attribute-Based Algorithms, Deliverable 7.2, Projet ESPRIT 2154, Aout 1990.Google Scholar
  16. Quinlan 1986, The effect of noise on concept learning, Machine learning 2 an artificial intelligence approach, TIOGA 1986.Google Scholar
  17. Quinlan 1988, Symplifying decision trees, in B.Gaines, J.Boose Knowledge Acquisition for Knowledge-Based Systems, Academic Press 1988.Google Scholar
  18. Rendell 1989, H. CHO, R. SESHU Improving the Design of Similarity-Based Rule-Learning Systems, International Journal of Expert System, pp 97–133, volume 2, Number 1, 1989.Google Scholar
  19. Rumelhart 1986, G.E. HINTON, R.J. WILLIAMS, Learning internal representation by error propagation. Parallel distributed processing: Explorations in the microstructures of cognition, MIT Press, vol 1, pp 318–362, 1986.Google Scholar
  20. Shavlik 1989, An Empirical Analysis of EBL Approaches for Learning Plan Schemata, in Proceedings of the 6o International Workshop on Machine Learning, Ithaca 1989, p183–87.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1991

Authors and Affiliations

  • H. Lounis
    • 1
  • G. Bisson
    • 1
  1. 1.LRI, Equipe Inférence et ApprentissageUniversité Paris-SudOrsay CedexFrance

Personalised recommendations