Abstract
Induction of a concept description given noisy instances is difficult and is further exacerbated when the concepts may change over time. This paper presents a solution which has been guided by psychological and mathematical results. The method is based on a distributed concept description which is composed of a set of weighted, symbolic characterizations. Two learning processes incrementally modify this description. One adjusts the characterization weights and another creates new characterizations. The latter process is described in terms of a search through the space of possibilities and is shown to require linear space with respect to the number of attribute-value pairs in the description language. The method utilizes previously acquired concept definitions in subsequent learning by adding an attribute for each learned concept to instance descriptions. A program called STAGGER fully embodies this method, and this paper reports on a number of empirical analyses of its performance. Since understanding the relationships between a new learning method and existing ones can be difficult, this paper first reviews a framework for discussing machine learning systems and then describes STAGGER in that framework.
Article PDF
Similar content being viewed by others
References
Anderson, J. R. (1983). The architecture of cognition. Cambridge, MA: Harvard University Press.
Bruner, J. S., Goodnow, J. J., & Austin, G. A. (1956). A study of thinking. New York: John Wiley & Sons.
Dietterich, T. G., & Michalski, R. S. (1981).Inductive learning of structural descriptions:Evaluation criteria and comparative review of selected methods.Artificial Intelligence,16, 257-294.
Duda, R., Gaschnig, J., & Hart, P. (1979). Model design in the Prospector consultant system for mineral exploration. In D. Michie (Ed. ), Expert systems in the micro electronic age. Edinburgh: Edinburgh University Press.
Easterlin, J. D. (1985). A survery of concept formation in machine learning. Unpublished manuscript, University of California, Department of Information and Computer Science, Irvine, CA.
Fine, T. L. (1973). Theories of probability. New York: Academic Press.
Fitzgerald, R. D. (1963). Effects of partial reinforcement with acid on the classically conditioned salivary response in dogs. Journal of Comparative and Physiological Psychology, 56, 1056-1060.
Gamzu, E., & Williams, D. R. (1971). Classical conditioning of a complex skeletal response. Scienceq, 171, 923-925.
Gluck, M. A., & Corter, J. E. (1985). Information, uncertainty, and the utility of categories, Proceedings of the Seventh Annual Conference of the Cognitive Science Society (pp. 283-287). Irvine, CA: Lawrence Erlbaum Associates.
Granger, R. H., Jr., & Schlimmer, J. C. (1985a). Combining numeric and symbolic learning techniques, Proceedings of the Third International Machine Learning Workshop (pp. 44-49). Skytop, PA.
Granger, R. H., Jr., & Schlimmer, J. C. (1985b). Learning salience among features through contingency in the CEL framework. Proceedings of the Seventh Annual Conference of the Cognitive Science Society (pp. 65-79). Irvine, CA: Lawrence Erlbaum Associates.
Granger, R. H., Jr., & Schlimmer, J. C. (in press). The computation of contingency in classical conditioning. In G. H. Bower (Ed. ), The psychology of learning and motivation:Advances in research and theory. New York: Academic Press.
Hampson, S., & Kibler, D. (1983). A Boolean complete neural model of adaptive behavior. Biological Cybernetics, 49, 9-19.
Holland, J. H. (1975). Adaptation in natural and artificial systems. Ann Arbor, MI: University of Michigan Press.
Iba, W. F. (1984). Machine learning in terms of concepts in terms of previously learned concepts. Unpublished manuscript, University of California, Board of Computer and Information Sciences, Santa Cruz, CA.
Langley, P. (in press). A general theory of discrimination learning. In D. Klahr, P. Langley, & R. Neches (Eds.), Production system models of learning and development. Cambridge, MA: MIT Press.
Langley, P., & Carbonell, J. G. (in press). Language acquisition and machine learning. In B. MacWhinney (Ed.), Mechanisms of language acquisition. Hillsdale, NJ: Lawrence Erlbaum Associates.
Lowerre, B., & Reddy, R. (1980). The HARPY speech understanding system. In W. Lea (Ed. ), Trends in speech recognition. Englewood Cliffs, NJ: Prentice-Hall.
Mauldin, M. L. (1984). Maintaining diversity in genetic search. Proceedings of the National Conference on Artificial Intelligence (pp. 247-250). Austin, TX: Morgan Kaufmann.
Michalski, R. S. (1983). A theory and methodology of inductive learning. Artificial Intelligence, 20, 111-161.
Mitchell, T. M. (1982). Generalization as search. Artificial Intelligence, 18, 203-226.
Mitchell, T. M., Utgoff, P. E., & Banerji, R. (1983). Learning by experimentation:Acquiring and refining problem-solving heuristics. In R. S. Michalski, J. G. Carbonell, & T. M. Mitchell (Eds. ), Machine learning:An artificial intelligence approach. Palo Alto, CA: Tioga.
Quintan, J. R. (1979). Discovering rules by induction from large collections of examples. In D. Michie (Ed. ), Expert systems in the micro electronic age. Edinburgh: Edinburgh University Press.
Quinlan, J. R. (1983). Learning from noisy data. Proceedings of the Second International Machine Learning Workshop (pp. 58-64). Urbana-Champaign, IL.
Quinlan, J. R. (1985). Induction of decision trees (Technical Report 85. 6). New South Wales, Australia: New South Wales Institute of Technology, School of Computing Sciences.
Rescorla, R. A. (1968). Probability of shock in the presence and absence of CS in fear conditioning. Journal of Comparative and Physiological Psychology, 66, 1-5.
Sammut, C., & Banerji, R. B. (1986). Learning concepts by asking questions. In R. S. Michalski, J. G. Carbonell, & T. M. Mitchell (Eds. ), Machine learning:An artificial intelligence approach (Vol. 2). Los Altos, CA: Morgan Kaufmann.
Schlimmer, J. C. (1986). A comparison of feature salience measures (Technical Report 86-13). Irvine, CA: University of California, Department of Information and Computer Science.
Siegel, S., & Domjan, M. (1971). Backward conditioning as an inhibitory procedure. Learning and Motivation, 2, 1-11.
Smith, E. E., & Medin, D. L., (1981). Categories and concepts. Cambridge, MA: Harvard University Press.
Wasserman, E. A., Chatlosh, D. L., & Neunaber, D. J. (1983). Perception of causal relations in humans:Factors affecting judgments of response-outcome contingencies under free-operant procedures. Learning and Motivation, 14, 406-432.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Schlimmer, J.C., Granger, R.H. Incremental Learning from Noisy Data. Machine Learning 1, 317–354 (1986). https://doi.org/10.1023/A:1022810614389
Issue Date:
DOI: https://doi.org/10.1023/A:1022810614389