Advertisement

Machine Learning

, Volume 10, Issue 1, pp 79–110 | Cite as

Induction over the unexplained: Using overly-general domain theories to aid concept learning

  • Raymond J. Mooney
Article

Abstract

This paper describes and evaluates an approach to combining empirical and explanation-based learning calledInduction Over the Unexplained (IOU). IOU is intended for learning concepts that can be partially explained by an overly-general domain theory. An eclectic evaluation of the method is presented which includes results from all three major approaches: empirical, theoretical, and psychological. Empirical results show that IOU is effective at refining overly-general domain theories and that it learns more accurate concepts from fewer examples than a purely empirical approach. The application of theoretical results from PAC learnability theory explains why IOU requires fewer examples. IOU is also shown to be able to model psychological data demonstrating the effect of background knowledge on human learning.

Keywords

Combining explanation-based and empirical learning knowledge-base refinement theory specialization 

References

  1. Ahn, W., & Brewer, W.F. (1988). Similarity-based and explanation-based learning of explanatory and nonexplanatory information.Proceedings of the Tenth Annual Conference of the Cognitive Science Society (pp. 524–530). Hillsdale, NJ: Erlbaum.Google Scholar
  2. Ahn, W., Mooney, R.J., Brewer, W.F., & DeJong, G.F. (1987). Schema acquisition from one example: Psychological evidence for explanation-based learning.Proceedings of the Ninth Annual Conference of the Cognitive Science Society (pp. 50–57). Hillsdale, NJ: Erlbaum.Google Scholar
  3. Bergadano, F., & Giordana, A. (1988). A knowledge intensive approach to concept induction.Proceedings of the Fifth International Conference on Machine Learning (pp. 305–317). San Mateo, CA: Morgan Kaufman.Google Scholar
  4. Blum, A., & Rivest, R.L. (1988). Training a 3-node neural net is NP-complete.Proceedings of the 1988 Workshop on Computational Learning Theory (pp. 9–18). San Mateo, CA: Morgan Kaufman.Google Scholar
  5. Bruner, J.S., Goodnow, J., & Austin, G.A. (1956).A study in thinking. New York: Wiley.Google Scholar
  6. Cohen, W.W. (1990). Learning from textbook knowledge: A case study.Proceedings of the Eighth National Conference on Artificial Intelligence (pp. 743–748). Cambridge, MA: MIT Press.Google Scholar
  7. Danyluk, A. (1989). Finding new rules for incomplete theories: Explicit biases for induction with contextual information.Proceedings of the Sixth International Workshop on Machine Learning (pp. 34–36). San Mateo, CA: Morgan Kaufman.Google Scholar
  8. DeJong, G.F., & Mooney, R.J. (1986). Explanation-based learning: An alternative view.Machine Learning, 1 145–176.Google Scholar
  9. Dietterich, T.G., London, B., Clarkson, K., & Dromey, G. (1982). Learning and inductive inference. In R. Cohen & E.A. Feigenbaum (Eds.),Handbook of Artificial Intelligence: Volume III. San Mateo, CA: Morgan Kaufman.Google Scholar
  10. Dowling, W.F., & Gallier, J.H. (1984). Linear-time algorithms for testing the satisfiability of propositional Horn formulae.Journal of Logic Programming, 3 267–284.Google Scholar
  11. Ehrenfeucht, A., Haussler, D., Kearns, M., & Valiant, L. (1989). A general lower bound on the number of examples needed for learning.Information and Computation, 82 267–284.Google Scholar
  12. Fisher, D.H. (1987). Knowledge acquisition via incremental conceptual clustering.Machine Learning, 2 139–172.Google Scholar
  13. Flann, N.S., & Dietterich, T.G. (1989). A study of explanation-based methods for inductive learning.Machine Learning, 4 187–226.Google Scholar
  14. Ginsberg, A. (1988). Theory revision via prior operationalization.Proceedings of the Seventh National Conference on Artificial Intelligence (pp. 743–748). San Mateo, CA: Morgan Kaufman.Google Scholar
  15. Ginsberg, A. (1990). Theory reduction, theory revision, and retranslation.Proceedings of the Eighth National Conference on Artificial Intelligence (pp. 743–748). Cambridge, MA: MIT Press.Google Scholar
  16. Gold, E.M. (1967). Language identification in the limit.Information and Control, 10 447–474.Google Scholar
  17. Hall, R. (1988). Learning by failing to explain: Using partial explanations to learn in incomplete or intractable domains.Machine Learning, 3 45–78.Google Scholar
  18. Haussler, D. (1988). Quantifying inductive bias: Artificial intelligence algorithms and Valiant's learning framework.Artificial Intelligence, 36 177–221.Google Scholar
  19. Hirsh, H. (1990).Incremental version space merging: A general framework for concept learning. Hingham, MA: Kluwer.Google Scholar
  20. Kearns, M., Li, M., Pitt, L., & Valiant, L.G. (1987). Recent results on Boolean concept learning.Proceedings of the Fourth International Machine Learning Workshop (pp. 337–352). San Mateo, CA: Morgan Kaufman.Google Scholar
  21. Kibler, D., & Langley, P. (1988). Machine learning as an experimental science.Proceedings of the Third European Working Session on Learning (pp. 81–92). London: Pitman.Google Scholar
  22. Langley, P. (1989). Editorial: Toward a unified science of machine learning.Machine Learning, 3 253–259.Google Scholar
  23. Lebowitz, M. (1986). Integrated learning: Controlling explanation.Cognitive Science, 10 219–240.Google Scholar
  24. Medin, D.L., Wattenmaker, W.D., & Michalski, R.S. (1987). Constraints and preferences in inductive learning: An experimental study of human and machine performance.Cognitive Science, 11 299–239.Google Scholar
  25. Michalski, R.S. (1983). A theory and methodology of inductive learning. In R.S. Michalski, J.G. Carbonell, & T.M. Mitchell (Eds.),Machine Learning: An Artificial Intelligence Approach. Palo Alto, CA: Tioga.Google Scholar
  26. Michalski, R.S., & Chilausky, R.L. (1980). Learning by being told and learning from examples: An experimental comparison of the two methods of knowledge acquisition in the context of developing an expert system for soybean disease diagnosis.Policy Analysis and Information Systems, 4 125–160.Google Scholar
  27. Mingers, J. (1989). An empirical comparison of pruning methods for decision tree induction.Machine Learning, 4 227–243.Google Scholar
  28. Minton, S. (1988). Quantitative results concerning the utility of explanation-based learning.Proceedings of the Seventh National Conference on Artificial Intelligence (pp. 564–569). San Mateo, CA: Morgan Kaufman.Google Scholar
  29. Mitchell, T.M., Keller, R., & Kedar Cabelli, S. (1986). Explanation-based generalization: A unifying view.Machine Learning, 1 47–80.Google Scholar
  30. Mooney, R., Ourston, D., & Wu, S.Y. (1989). Induction over the unexplained: A new approach to combining empirical and explanation-based learning (Technical Report AI89-110). Austin, TX: University of Texas, Artificial Intelligence Laboratory.Google Scholar
  31. Muggleton, S., & Buntine, W. (1988). Machine invention of first-order predicates by inverting resolution.Proceedings of the Fifth International Machine Learning Conference (pp. 339–352). San Mateo, CA: Morgan Kaufman.Google Scholar
  32. Murphy, G.L., & Medin, D.L. (1985). The role of theories in conceptual coherence.Psychological Review, 92 289–316.Google Scholar
  33. Ourston, D., & Mooney, R. (1990). Changing the rules: A comprehensive approach to theory refinement.Proceedings of the Eighth National Conference on Artificial Intelligence (pp. 815–820). Cambridge, MA: MIT Press.Google Scholar
  34. Pazzani, M.J. (1990).Creating a memory of causal relationships: An integration of empirical and explanation-based learning methods. Hillsdale, NJ: Earlbaum.Google Scholar
  35. Pazzani, M.J., & Sarrett, W. (1990). Average case analysis of conjunctive learning algorithms.Proceedings of the Seventh International Machine Learning Conference (pp. 339–347). San Mateo, CA: Morgan Kaufman.Google Scholar
  36. Pazzani, M.J., & Silverstein, G. (1990). Feature selection and hypothesis selection models of induction.Proceedings of the Twelfth Annual Conference of the Cognitive Science Society (pp. 221–228). Hillsdale, NJ: Earlbaum.Google Scholar
  37. Porter, B., Bareiss, R., & Holte, R.C. (1990). Concept learning and heuristic classification in weak-theory domains.Artificial Intelligence, 45 229–263.Google Scholar
  38. Quinlan, J.R. (1986). Induction of decision trees.Machine Learning, 1 81–106.Google Scholar
  39. Quinlan, J.R. (1990). Learning logical definitions from relations.Machine Learning, 5 239–266.Google Scholar
  40. Reinke, R. (1984). Knowledge acquisition and refinement tools for the ADVISE meta-expert system. Master's thesis, Department of Computer Science, University of Illinois, Urbana, IL.Google Scholar
  41. Rosenbloom, P., & Newell, A. (1987). Learning by chunking: A production system model of practice. In D. Klahr, P. Langley, & R. Neches (Eds.),Production system models of learning and development. Cambridge, MA: MIT Press.Google Scholar
  42. Rumelhart, D.E., Hinton, G.E., & Williams, J.R. (1986). Learning internal representations by error propagation. In D.E. Rumelhart & J.L. McClelland (Eds.),Parallel Distributed Processing (Vol. 1). Cambridge, MA: MIT Press.Google Scholar
  43. Segre, A.M. (Ed.) (1989). Combining empirical and explanation-based learning.Proceedings of the Sixth International Workshop on Machine Learning (pp. 2–92). San Mateo, CA: Morgan Kaufman.Google Scholar
  44. Shavlik, J.W., Mooney, R.J., & Towell, G.G. (1991). Symbolic and neural learning algorithms: An experimental comparison.Machine Learning, 6 11–143.Google Scholar
  45. Skorstad, J., Falkenhainer, B., & Gentner, D. (1987). Analogical processing: A simulation and empirical corroboration.Proceedings of the Sixth National Conference on Artificial Intelligence (pp. 322–326). San Mateo: Morgan Kaufman.Google Scholar
  46. Stepp, R.E. (1984). Conjunctive conceptual clustering: A methodology and experimentation. Doctoral dissertation, Department of Computer Science, University of Illinois, Urbana, IL.Google Scholar
  47. Towell, G.G., Shavlik, J.W., & Noordewier, M.O. (1990). Refinement of approximate domain theories by knowledge-based neural networks.Proceedings of the Eighth National Conference on Artificial Intelligence (pp. 861–866). Cambridge, MA: MIT Press.Google Scholar
  48. Utgoff, P.E. (1989). Incremental induction of decision trees.Machine Learning, 4 161–186.Google Scholar
  49. Valiant, L.G. (1984). A theory of the learnable.Communications of the Association for Computing Machinery, 27 1134–1142.Google Scholar
  50. Wilkins, D.C. (1988). Knowledge base refinement using apprenticeship learning techniques.Proceedings of the Seventh National Conference on Artificial Intelligence (pp. 646–651). San Mateo, CA: Morgan Kaufman.Google Scholar
  51. Winston, P.H., Binford, T.O., Katz, B., & Lowry, M. (1983). Learning physical descriptions from functional definitions, examples, and precedents.Proceedings of the Third National Conference on Artificial Intelligence (pp. 433–439). San Mateo, CA: Morgan Kaufman.Google Scholar
  52. Wisniewski, E.J. (1989). Learning from examples: The effect of different conceptual roles.Proceedings of the Eleventh Annual Conference of the Cognitive Science Society (pp. 980–986), Hillsdale, NJ: Earlbaum.Google Scholar

Copyright information

© Kluwer Academic Publishers 1993

Authors and Affiliations

  • Raymond J. Mooney
    • 1
  1. 1.Department of Computer SciencesUniversity of TexasAustin

Personalised recommendations