Machine Learning

, Volume 1, Issue 2, pp 177–226

A general framework for induction and a study of selective induction

  • Larry Rendell
Article

Abstract

This paper has two major parts. The first is an extensive analysis of the problem of induction, and the second part is a detailed study of selective induction. Throughout the paper we integrate a number of notions, mainly from artificial intelligence, but also from pattern recognition and cognitive psychology. The result is a synthetic view which exploits uncertainty, task-guidance, and biases such as language restriction. Some of the main themes and contributions are as follows. (1) Practical induction is really a problem of efficacy and efficiency (power). (2) Search in a space of hypothetical concepts is governed by acredibility function which combines various knowledge sources in a single subjective probability or belief measure μ. (3) The amount of knowledge supplied by various sources can often be quantified; these sources include various biases and the learning system itself. (4) Induction is equivalent to discovery of autility function u, which captures the purpose or goal of induction. (5) The difficulty of induction may be characterized by the form of u. Smooth or coherent functions mean selective induction, which has had the most attention in machine learning. (6) Systems for selective induction are more similar than commonly understood. By juxtaposing them we can discover similarities and improvements. (7) Our analysis suggests a number of incipient principles for powerful induction.

Key words

induction uncertain and incremental learning concept formation 

References

  1. Anderberg, M.R. (1973).Cluster analysis for applications. New York: Academic Press.Google Scholar
  2. Angluin, D., & Smith, C.H. (1983). Inductive inference: Theory and methods.ACM Computing Surveys 15, 237–269.Google Scholar
  3. Banerji, R.B. (1985). The logic of learning: A basis for pattern recognition and for improvement of performance.Advances in Computers,24, 177–216.Google Scholar
  4. Bruner, J.S., Goodnow, J.J., & Austin, G.A. (1956).A study of thinking. New York: Wiley.Google Scholar
  5. Cheeseman, P. (1985). In defense of probability.Proceedings of the Ninth International Joint Conference on Artificial Intelligence, (pp. 1003–1009). Los Angeles: Morgan-Kaufmann.Google Scholar
  6. Christensen, R. (1964).Foundations of inductive reasoning. Berkeley: Entropy, Ltd.Google Scholar
  7. Dietterich, T.G., London, B., Clarkson, K., & Dromey, G. (1982). Learning and inductive inference. In P.R. Cohen & E.A. Feigenbaum (Ed.),The handbook of artificial intelligence. Los Altos: Kaufmann.Google Scholar
  8. Duda, R.O., & Hart, P.E. (1973).Pattern classification and scene analysis. New York: Wiley.Google Scholar
  9. Easterlin, J.D., & Langley, P. (1985). A framework for concept formation.Proceedings of the Seventh Annual Conference of the Cognitive Science Society, (pp. 267–271). Irvine, CA.Google Scholar
  10. Erman, L.D., Hayes-Roth, F., Lesser, V.R., & Reddy, D.R. (1980). The HEARSAY-II speech understanding system: Integrating knowledge to resolve uncertainty.Computing Surveys,12, 213–253.Google Scholar
  11. Glass, A.L., & Holyoak, K.J. (1975). Alternative conceptions of semantic memory.Cognition,3, 313–339.Google Scholar
  12. Holland, J.H. (1975).Adaptation in natural and artificial systems. Ann Arbor: University of Michigan Press.Google Scholar
  13. Holte, R.C. (1986). Alternative information structures in incremental learning systems.Proceedings of the European Working Session on Learning.Google Scholar
  14. Hunt, E.B., Marin, J., & Stone, P.J. (1966).Experiments in induction. New York: Academic Press.Google Scholar
  15. Kanal, L., & Chandrasekaran, B. (1972). On linguistic, statistical and mixed models for pattern recognition. In S. Watanabe (Ed.),Frontiers of pattern recognition (pp. 163–192), New York: Academic Press.Google Scholar
  16. Koestler, A. (1964).The act of creation: A study of the conscious and unconscious in science and art. New York: Macmillan (hard cover). (Some other editions omit the important Book Two).Google Scholar
  17. Langley, P. (1985). Learning search strategies through discrimination.International Journal of Man-Machine Studies,18, 513–541.Google Scholar
  18. Lenat, D.B. (1983). The role of heuristics in learning by discovery: Three case studies. In R.S. Michalski, J.G. Carbonell, & T.M. Mitchell (Eds.),Machine learning: An artificial intelligence approach (pp. 243–306). Palo Alto: Tioga.Google Scholar
  19. Matheus, C.J. (1985). Views on concept representation and implications for machine learning. Unpublished manuscript.Google Scholar
  20. Medin, D.L., & Smith, E.E. (1984). Concepts and concept formation.Annual Review of Psychology,35, 113–138.Google Scholar
  21. Medin, D.L., & Wattenmaker, W.D. (1985). Category cohesiveness, theories, and cognitive archeology. In U. Neisser (Ed.),Concepts reconsidered: The ecological and intellectual bases of categories.Google Scholar
  22. Michalski, R.S. (1983). A theory and methodology of inductive learning.Artificial Intelligence,20(2), 111–161. Reprinted in R.S. Michalski, J.G. Carbonell, & T.M. Mitchell (Eds.),Machine learning: An artificial intelligence approach (pp. 83–134). Palo Alto: Tioga.Google Scholar
  23. Michalski, R.S., & Chilausky, R.L. (1980). Learning by being told and learning from examples: An experimental comparison of the two methods of knowledge acquisition in the context of developing an expert system for soybean disease diagnosis.International Journal Policy Analysis and Information Systems,4, 125–161.Google Scholar
  24. Michalski, R.S., & Stepp, R.E. (1983). Learning from observation: Conceptual clustering, In R.S. Michalski, J.G. Carbonell, & T.M. Mitchell (Eds.),Machine learning: An artificial intelligence approach (pp. 331–363). Palo Alto: Tioga.Google Scholar
  25. Michie, D. (1977). A theory of advice. In E.W. Elcock, & D. Michie (Eds.),Machine intelligence, 8 (pp. 151–168). American Elsevier.Google Scholar
  26. Mitchell, T.M. (1978).Version spaces: An approach to concept learning. Unpublished doctoral dissertation, Stanford University, Stanford, California.Google Scholar
  27. Mitchell, T.M. (1980).The need for biases in learning generalizations, (Technical Report No. CBMTR-117). New Brunswick: Rutgers University, Dept. of Computer Science.Google Scholar
  28. Mitchell, T.M. (1982). Generalization as search.Artificial Intelligence,21, 203–226.Google Scholar
  29. Murphy, G.L., & Medin, D.L. (1985). The role of theories in conceptual coherence.Psychological Review, 92.Google Scholar
  30. O'Rorke, P. (1982).A comparative study of inductive learning systems AQ11P and ID-3 using a chess endgame test problem (Technical Report No. UIUCDCS-F-82–899 and ISG 82–2). Urbana-Champaign: University of Illinois, Dept. of Computer Science.Google Scholar
  31. Quinlan, J.R. (1983). Learning efficient classification procedures and their application to chess end games. In R.S. Michalski, J.G. Carbonell, & T.M. Mitchell (Eds.),Machine learning: An artificial intelligence approach (pp. 463–482). Palo Alto: Tioga.Google Scholar
  32. Quinlan, J.R. (1986). Induction of decision trees.Machine Learning 1, 81–106.Google Scholar
  33. Rendell, L.A. (1977).A locally optimal solution of the fifteen puzzle produced by an automatic evaluation function generator (Technical Report No. CS-77–36). Waterloo, Ontario, Canada: University of Waterloo, Dept of Computer Science.Google Scholar
  34. Rendell, L.A. (1981).An adaptive plan for state-space problems. Unpublished doctoral dissertation (Technical Report No. CS-81-13), University of Waterloo, Dept. of Computer Science, Waterloo, Ontario, Canada.Google Scholar
  35. Rendell, L.A. (1983a). A new basis for state-space learning systems and a successful implementation,Artificial Intelligence,20, 369–392.Google Scholar
  36. Rendell, L.A. (1983b).Conceptual knowledge acquisition in search (Technical Report No. CIS-83–15). Guelph, Ontario, Canada: University of Guelph, Dept. of Computing and Information Science. Also L. Bolc (Ed.).Computational models of learning, Springer-Verlag).Google Scholar
  37. Rendell, L.A. (1985a). Genetic plans and the probabilistic learning system: Synthesis and results,Proceedings International Conference on Genetic Algorithms and their Applications (pp. 60–73).Google Scholar
  38. Rendell, L.A. (1985b). Substantial constructive induction using layered information compression: Tractable feature formation in search,Proceedings of the Ninth International Joint Conference on Artificial Intelligence (pp. 650–658).Google Scholar
  39. Rendell, L.A. (1986). Uncertain and incremental concept learning: Efficiency through model and data synergy. Unpublished manuscript.Google Scholar
  40. Ritchie, G.D., & Hanna, F.K. (1984). AM: A case study in AI methodology.Artificial Intelligence,23, 249–268.Google Scholar
  41. Samuel, A.L. (1967). Some studies in machine learning using the game of checkers II — recent progress.IBM Journal of Research & Development,11, 601–617.Google Scholar
  42. Tou, T.T., & Gonzalez, R.C. (1974).Pattern recognition principles, Reading: Addison-Wesley.Google Scholar
  43. Utgoff, P.E., & Mitchell, T.M. (1982). Acquisition of appropriate bias for inductive concept learning.Proceedings of the National Conference on Artificial Intelligence, 414–417.Google Scholar
  44. Watanabe, S. (1969).Knowing and guessing: A formal and quantitative study, New York: Wiley.Google Scholar
  45. Watanabe, S. (1972). Pattern recognition as information compression. In Watanabe, S. (Ed.).Frontiers of pattern recognition (pp. 561–567). New York: Academic Press.Google Scholar
  46. Winston, P.H. (1984).Artificial intelligence (2nd ed.). Reading: Addison Wesley.Google Scholar
  47. Wise, B.P., & Henrion, M. (1985). A framework for comparing uncertain inference systems to probability.Proceedings of Uncertainty and Probability in Artificial Intelligence Workshop (pp. 99–108).Google Scholar

Copyright information

© Kluwer Academic Publishers 1986

Authors and Affiliations

  • Larry Rendell
    • 1
  1. 1.Department of Computer ScienceUniversity of Illinois at Urbana-ChampaignUrbanaU.S.A.

Personalised recommendations