An approach to measuring theory quality

  • Edgar Sommer
Data Mining
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1076)


The quality of theories produced with the help of machine learning algorithms is usually measured in terms of accuracy and coverage. This paper reopens the issue of understandability of induced theories, which, while prominent in the early days of ML, seems to have fallen from favor in the sequel. This issue is especially relevant in the broader context of using ML as an aide in design and maintenance of knowledge bases for knowledge based systems. The guiding question is: beyond accuracy, what constitutes a good theory? An attempt at surveying relevant work in the fields of linguistics and cognitive psychology is made. The sympathetic reader will find this somewhat motivates the author's personal intuitions about the quality of a theory, hinging on understandability. These intuitions, in turn, point toward some simple criteria that may help in measuring quality. By way of consolation for those who do not share the author's intuitions, the criteria proposed here are objective in the sense that the measurements they provide may be evaluated from a number of contrary perspectives. Some empirical results are given in the context of theory restructuring: redundancy elimination and introduction of new intermediate concepts.


Logical Theory Inductive Logic Programming Syntactic Form Inferential Structure Artificial Intelligence Approach 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [Barsalou 83]
    L. W. Barsalou. Ad hoc categories. Memory & Cognition, 11(3):211–227, 1983.Google Scholar
  2. [Biere 89]
    Bernd Ulrich Biere. Verständlich-Machen. Max Niemeyer Verlag, Tübingen, 1989.Google Scholar
  3. [Bock 78]
    M. Bock. Wort-, Satz-, Textverarbeitung. Kohlhammer, Stuttgart, 1978.Google Scholar
  4. [Bruynooghe 82]
    Maurice Bruynooghe. Adding redundancy to obtain more reliable and readable Prolog programs. In Michel Van Caneghem (ed.), Proceedings of the First International Logic Programming Conference, pp. 129–133, Marseille, France, 1982. ADDP-GIA.Google Scholar
  5. [Buntine 88]
    W. Buntine. Generalized Subsumption and its Applications to Induction and Redundancy. Artificial Intelligence, 36:149–176, 1988.Google Scholar
  6. [Carbonell 89]
    Jaime G. Carbonell. Paradigms for Machine Learning. Artificial Intelligence, 40:1–9, 1989.Google Scholar
  7. [Emde89]
    Werner Emde. An Inference Engine for Representing Multiple Theories. In K. Monk (ed.), Knowledge Representation and Organization in Machine Learning, pp. 148–176. Springer, New York, Berlin, Tokyo, 1989. Also: KIT-Report Nr. 64, TU Berlin, 1988.Google Scholar
  8. [Emde 91]
    Werner Emde. Modellbildung, Wissensrevision und Wissensrepräsentation im Maschinellen Lernen. Informatik-Fachberichte 281. Springer Verlag, Berlin, New York, 1991. PhD thesis.Google Scholar
  9. [Emde 94]
    Werner Emde. Inductive Learning from very few Classified Examples. In Proc. 7th European Conference on Machine Learning (ECML-94), 1994. also available as GMD Tech. Report.Google Scholar
  10. [Fu/Buchanan 85]
    Li-Min Fu and Brace Buchanan. Learning Intermediate Concepts in Constructing a Hierarchical Knowledge Base. In Proc. 9th International Joint Conference on Artificial Intelligence, pp. 659–666, San Mateo, CA, 1985. Morgan Kaufman.Google Scholar
  11. [Gernsbacher 90]
    M. A. Gernsbacher. Language Comprehension as Structure Building. Lawrence Erlbaum, Hillsdale NJ, 1990.Google Scholar
  12. [Hörmann 81]
    Hans Hörmann. To Mean — To Understand: Problems of Psychological Semantics. Springer-Verlag, 1981.Google Scholar
  13. [Kietz/Wrobel 91]
    Jörg-Uwe Kietz and Stefan Wrobel. Controlling the Complexity of Learning in Logic through Syntactic and Task-Oriented Models. In Stephen Muggleton (ed.), Proc. 1st Int. Workshop on ILP, pp. 107–126, Viana de Castelo, Portugal, 1991. Also in S.Muggleton (ed.), Inductive Logic Programming, Academic Press, 1992.Google Scholar
  14. [Kintsch 74]
    W. Kintsch. The Representation of Meaning in Memory. Erlbaum, Potomac, MD, 1974.Google Scholar
  15. [Kintsch/Keenan 73]
    W. Kintsch and J Keenan. Reading rate and retention as a function of the number of propositions in the base structure of sentences. Cognitive Psychology, 5:257–274, 1973.Google Scholar
  16. [Kodratoff 94]
    Yves Kodratoff. Guest Editor's Introduction (The Comprehensibility Manifesto). AI Communications, 7(2):83–85, 1994.Google Scholar
  17. [Kutschera 67]
    Franz von Kutschera. Elementare Logik. Wien, 1967.Google Scholar
  18. [Kutschera 72]
    Franz von Kutschera. Wissenschaftstheorie, volume II. Wilhelm Fink Verlag, München, 1972.Google Scholar
  19. [Lesgold 72]
    A. M. Lesgold. Pronominalization: a Device for Unifying Sentences in Memory. Journal of Verbal Learning and Verbal Behavior, 11:316–323, 1972.Google Scholar
  20. [Mandler 67]
    G. Mandler. Organisation and Memory. In K. W. Spence and J. T. Spence (eds.), The Psychology of Learning and Motivation, volume 2, pp. 189–196. Academic Press, New York, 1967.Google Scholar
  21. [Michalski 83]
    Ryszard S. Michalski. A Theory and Methodology of Inductive Learning. In Machine Learning — An Artificial Intelligence Approach, volume I, pp. 83–134. Morgan Kaufman, San Mateo, CA, 1983.Google Scholar
  22. [Michie 86]
    Donald Michie. The superarticulacy phenomenon in the context of software manufacture. Proceedings of the Royal Society of London, A 405:185–212, 1986.Google Scholar
  23. [Morik et al. 93]
    K. Morik, S. Wrobel, Jörg-Uwe Kietz, and W. Emde. Knowledge Acquisition and Machine Learning. Academic Press, London, 1993.Google Scholar
  24. [Muggleton 87]
    Stephen Muggleton. Structuring Knowledge by Asking Questions. In Ivan Bratko and Nada Lavrač(eds.), Progress in Machine Learning—Proc. Second European Working Session on Learning (EWSL), Wilmslow, UK, 1987. Sigma Press.Google Scholar
  25. [Muggleton/Buntine 88]
    Stephen Muggleton and Wray Buntine. Machine Invention of First-Order Predicates by Inverting Resolution. In Proc. Fifth Intern. Conf. on Machine Learning, San Mateo, CA, 1988. Morgan Kaufman.Google Scholar
  26. [Muggleton/deRaedt 94]
    Stephen Muggleton and Luc deRaedt. Inductive Logic Programming: Theory and Methods. Journal of Logic Programming, 19/20:629–680, 1994.Google Scholar
  27. [Muggleton/Feng 90]
    Stephen Muggleton and Cao Feng. Efficient Induction of Logic Programs. In Proc. First Conf. on Algorithmic Learning Theory, Tokyo, 1990. Ohmsha Publishers.Google Scholar
  28. [Nedellec 95]
    Claire Nedellec (ed.). Proc. IJCAI Workshop on Machine Learning and Comprehensibility,, 1995.Google Scholar
  29. [Piatetsky-Shapiro/Frawley 91]
    G. Piatetsky-Shapiro and W. Frawley. Knowledge discovery in databases. The MIT press, 1991. (editors).Google Scholar
  30. [Plotkin 70]
    Gordon D. Plotkin. A note on inductive generalization. In B. Meltzer and D. Michie (eds.), Machine Intelligence, volume 5, chapter 8, pp. 153–163. American Elsevier, 1970.Google Scholar
  31. [Popper 33]
    Karl Popper. Die beiden Grundprobleme der Erkenntnistheorie: aufgrund von Ms. aus d. Jahren 1930–1933, volume 18 of Die Einheit der Gesellschaftswissenschaften. Mohr, Tübingen, 1933. edited by Troels Eggers Hansen, appeared 1979.Google Scholar
  32. [Quinlan 90]
    J. Ross Quinlan. Learning Logical Definitions from Relations. Machine Learning, 5(3):239–266, 1990.Google Scholar
  33. [Sanford/Garrod 81]
    A. J. Sanford and S. C. Garrod. Understanding Written Language. Wiley and Sons, Chichester, 1981.Google Scholar
  34. [Smolensky 87]
    P. Smolensky. Connectionist AI, Symbolic AI, and the Brain. Artificial Intelligence Review, 1:95–109, 1987.Google Scholar
  35. [Sommer et al. 94]
    E. Sommer, K. Morik, J.M. Andre, and M. Uszynski. What On-line Learning Can Do for Knowledge Acquisition. Knowledge Acquisition, 6:435–460, 1994.Google Scholar
  36. [Sommer et al. 96]
    E. Sommer, Werner Emde, Jörg-Uwe Kietz, and Stefan Wrobel. Mobal 42 User Guide ((always) Draft). Arbeitspapiere der gmd, GMD, 1996. Available via WWW Scholar
  37. [Sommer 94a]
    E. Sommer. Learning Relations without Closing the World. In Proc. of the European Conference on Machine Learning (ECML-94), Berlin, 1994. Springer-Verlag.Google Scholar
  38. [Sommer 94b]
    E. Sommer. Restructuring in Horn clause knowledge bases. Technical report, ESPRIT Project ILP (6020), 1994. ILP Deliverable GMD 2.1.Google Scholar
  39. [Sommer 94c]
    E. Sommer. Rulebase Stratification: an Approach to theory restructuring. In Proc. 4th Intl. Workshop on Inductive Logic Programming (ILP-94), 1994. Available via WWW Scholar
  40. [Sommer 95a]
    E. Sommer. An Approach to Quantifying the Quality of Induced Theories. In Claire Nedellec (ed.), Proc. IJCAI Workshop on Machine Learning and Comprehensibility, 1995. Available via WWW http: // html.Google Scholar
  41. [Sommer 95b]
    E. Sommer. Fender: An approach to theory restructuring. In Stefan Wrobel and Nada Lavrac (eds.), Proc. of the European Conference on Machine Learning (ECML-95), volume 912 of Lecture Notes in Artificial Intelligence, Berlin, 1995. Springer-Verlag.Google Scholar
  42. [Sommer 95c]
    E. Sommer. Mobal's theory restructuring tool RRT. Technical report, ESPRIT Project ILP (6020), 1995. ILP Deliverable GMD 2.2.Google Scholar
  43. [Sommer 96]
    E. Sommer. Theory Restructuring. NN, 1996. (submitted).Google Scholar
  44. [Wrobel 94]
    Stefan Wrobel. Concept Formation and Knowledge Revision. Kluwer Academic Publishers, Dordrecht, Netherlands, 1994.Google Scholar
  45. [Wygotski 64]
    L. S. Wygotski. Denken und Sprechen. Conditio humana. S. Fischer, 1964. (First published in Russian 1934, english translation “Thought and language” 1962 by MIT Press).Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1996

Authors and Affiliations

  • Edgar Sommer
    • 1
  1. 1.AI Division (I3.KI)GMD (German National Research Center for Information Technology)Sankt AugustinGermany

Personalised recommendations