Machine Learning

, Volume 12, Issue 1–3, pp 49–67 | Cite as

Experience selection and problem choice in an exploratory learning system

  • Paul D. Scott
  • Shaul Markovitch
Article

Abstract

A fully autonomous exploratory learning system must perform two tasks that are not required of supervised learning systems: experience selection and problem choice. Experience selection is the process of choosing informative training examples from the space of all possible examples. Problem choice is the process of identifying defects in the domain theory and determining which should be remedied next. These processes are closely related because the degree to which a specific experience is informative depends on the particular defects in the domain theory that the system is attempting to remedy. In this article we propose a general control structure for exploratory learning in which problem choice by an information-theoretic “curiosity” heuristic: the problem chosen then guides the selection of training examples. An implementation of an exploratory learning system based on this control structure is described, and a series of experimental results are presented.

Keywords

Exploratory learning experience selection problem choice machine discovery curiosity 

References

  1. Aha, D.W., Kibler, D., & Albert, M.K. (1991). Instance-based learning algorithms.Machine Learning, 6, 37–66.Google Scholar
  2. Carbonell, J.G., & Gil, Y. (1987). Learning by experimentation. InProceedings of the Fourth International Workshop on Machine Learning, Irvine, California (pp. 256–265), San Mateo, CA: Morgan Kaufmann.Google Scholar
  3. Langley, P. (1987). A general theory of discrimination learning. In D. Klahr, P. Langley, & R. Neches (Eds.),Production system models of learning and development, pp. 99–161. Cambridge, MA: MIT Press.Google Scholar
  4. Langley, P., Simon, H.A., Bradshaw, G.L., & Żytkow, J.M. (1987).Scientific discovery: Computational explorations of the creative process. Cambridge, MA: MIT Press.Google Scholar
  5. Langley, P., & Żytkow, J.M. (1989). Data-driven approaches to empirical discovery.Artificial Intelligence, 40(1–3).Google Scholar
  6. Markovitch, S., & Scott, P.D. (1989). Information filters and their implementation in the SYLLOG system.Proceedings of the Sixth International Workshop on Machine Learning (pp. 404–407), Ithaca, New York. Los Altos, CA: Morgan Kaufmann.Google Scholar
  7. Markovitch, S., & Scott, P.D. (1993). Information filtering: Selection mechanisms in learning systems.Machine Learning, 10, 113–151.Google Scholar
  8. Mitchell, T.M. (1982). Generalization as search.Artificial Intelligence, 18, 203–226.Google Scholar
  9. Quinlan, J.R. (1986). The effect of noise on concept learning. In R.S. Michalski, J.G. Carbonell, & T.M. Mitchell (Eds.),Machine learning: An artificial intelligence approach (Vol. 2). San Mateo, CA: Morgan Kaufmann.Google Scholar
  10. Rajamoney, S.A. (1989). Exemplar-based theory rejection: An approach to the experience consistency problem.Proceedings of the Sixth International Workshop on Machine Learning (pp. 284–289), Ithaca, New York. Los Altos, CA: Morgan Kaufmann.Google Scholar
  11. Rajamoney, S., DeJong, G., & Flatings, B. (1985). Towards a model of conceptual knowledge acquisition through directed experimentation.Proceedings of the Ninth International Conference on Artificial Intelligence (pp. 688–690). Los Angeles: Morgan Kaufmann.Google Scholar
  12. Ruff, R.A., & Dietterich, T.G. (1989). What good are experiments?Proceedings of the Sixth International Workshop on Machine Learning (pp. 109–112), Ithaca, New York. Los Altos, CA: Morgan Kaufmann.Google Scholar
  13. Scott, P.D., & Markovitch, S. (1989a). Uncertainty based selection of learning experiences.Proceedings of the Sixth International Workshop on Machine Learning (pp. 358–361), Ithaca, New York. Los Altos, CA: Morgan Kaufmann.Google Scholar
  14. Scott, P.D., & Markovitch, S. (1989b). Learning novel domains through curiosity and conjecture.Proceedings of the Eleventh International Conference on Artificial Intelligence (pp. 669–674). Detroit: Morgan Kaufmann.Google Scholar
  15. Scott, P.D., & Markovitch, S. (1991). Representation generation in an exploratory learning system. In D. Fisher & M. Pizzani (Eds.),Concept formation: Knowledge and experience in unsupervised learning. San Mateo, CA: Morgan Kaufmann.Google Scholar
  16. Shalin, V.L., Wisniewski, E.J., Levi, K.R., & Scott, P.D. (1987). A formal analysis of machine learning systems for knowledge acquisition.Proceedings of the Second AAAI Knowledge Acquisition For Knowledge Based Systems Workshop. Banff, Canada. An extended version appears in B.R. Gaines & J.H. Boose (Eds.),Machine learning and uncertain reasoning: Knowledge based systems (Vol. 3). San Diego, CA: Academic Press, 1990.Google Scholar
  17. Shannon, C.E., & Weaver, W. (1949).The mathematical theory of communication. Urbana, IL: University of Illinois Press.Google Scholar
  18. Shen, W.M., & Simon, H.A. (1989). Rule creation and rule learning through environmental exploration.Proceedings of the Eleventh International Conference on Artificial Intelligence (pp. 675–680). Detroit: Morgan Kaufmann.Google Scholar
  19. Simon, H.A., & Lea, G. (1974). Problem solving and rule induction: A unified view. In L.W. Gregg (Ed.),Knowledge and cognition, pp. 105–127. New Jersey: Lawrence Erlbaum.Google Scholar
  20. Winston, P.H. (1975). Learning structural descriptions from examples. In P.H. Winston (Ed.),The psychology of computer vision. New York: McGraw-Hill.Google Scholar
  21. Żytkow, J.M. (1987). Combining many searches in the FAHRENHEIT discovery system.Proceedings of the Fourth International Workshop on Machine Learning (pp. 281–287), Irvine, California. Los Altos, CA: Morgan Kaufmann.Google Scholar
  22. Żytkow, J.M., & Zhu, J. (1991). Automated empirical discovery in a numerical space.Proceedings of the Third Annual Chinese Machine Learning Workshop '91.Google Scholar

Copyright information

© Kluwer Academic Publishers 1993

Authors and Affiliations

  • Paul D. Scott
    • 1
  • Shaul Markovitch
    • 2
  1. 1.Department of Computer ScienceUniversity of EssexColchesterUnited Kingdom
  2. 2.Computer Science Department, TechnionHaifaIsrael

Personalised recommendations