On concept space and hypothesis space in case-based learning algorithms

  • A D Griffiths
  • D G Bridge
Part of the Lecture Notes in Computer Science book series (LNCS, volume 912)


In order to learn more about the behaviour of case-based reasoners as learning systems, we formalise a simple case-based learner as a PAC learning algorithm. We show that the case-based representation 〈CB, σ〉 is rich enough to express any boolean function. We define a family of simple case-based learning algorithms which use a single, fixed similarity measure and we give necessary and sufficient conditions for the consistency of these learning algorithms in terms of the chosen similarity measure. Finally, we consider the way in which these simple algorithms, when trained on target concepts from a restricted concept space, often output hypotheses which are outside the chosen concept space. A case study investigates this relationship between concept space and hypothesis space and concludes that the case-based algorithm studied is a less than optimal learning algorithm for the chosen, small, concept space.


  1. 1.
    D W Aha, D Kibler, and M K Albert. Instance-based learning algorithms. Machine Learning, 6:37–66, 1991.Google Scholar
  2. 2.
    M K Albert and D W Aha. Analyses of instance-based learning algorithms. In AAAI-91: Proceedings of the Ninth National Conference on Artificial Intelligence, pages 553–558, 1991.Google Scholar
  3. 3.
    M Anthony and N Biggs. Computational Learning Theory. Cambridge University Press, 1992.Google Scholar
  4. 4.
    A Blumer, A Ehrenfeucht, D Haussler, and M K Warmuth. Learnability and the Vapnik-Chervonenkis dimension. Journal of the ACM, 36(4):929–965, Oct 1989.Google Scholar
  5. 5.
    S Cost and S Salzberg. A weighted nearest neighbour algorithm for learning with symbolic features. Machine Learning, 10(1):37–66, Mar 1993.Google Scholar
  6. 6.
    W H E Day and D P Faith. A model in partial orders for comparing objects by dualistic measures. Mathemetical Biosciences, 78(2):179–192, 1986.Google Scholar
  7. 7.
    A M Dearden and M D Harrison. The engineering of case memory systems, submitted to the Journal of Intelligent Information Systems.Google Scholar
  8. 8.
    C Globig and S Wess. Symbolic learning and nearest-neighbour classification. In Proceedings of the 17th Annual Conference of the Gesellschaft fur Klassification e. V. University of Kaiserslautern, March 3–5, 1993. Springer-Verlag, 1994.Google Scholar
  9. 9.
    D Haussler. Quantifying inductive bias: AI learning algorithms and Valiant's learning framework. Artificial Intelligence, 36:177–221, 1988.Google Scholar
  10. 10.
    D Haussler. Probably approximately correct learning. In AAAI-90 Proceedings of the Eight National Conference on Artificial Intelligence, Boston, MA, pages 1101–1108. American Association for Artificial Intelligence, 1990.Google Scholar
  11. 11.
    K P Jantke. Case-based learning and inductive inference. GOSLER report 08/92, FB Mathematik & Informatik, TH Leipzig, 1992.Google Scholar
  12. 12.
    K P Jantke and S Lange. Case-based representation and learning of pattern languages. In EWCBR-93 Working Notes of the first European Workshop on Case-Based Reasoning, volume 1, pages 139–144. University of Kaiserslautern, 1993.Google Scholar
  13. 13.
    E L Rissland, J Kolodner, and D Waltz. Case-based reasoning. In Proceedings of DARPA Case-Based Reasoning Workshop May 1989, pages 1–13. Morgan Kaufmann, 1989.Google Scholar
  14. 14.
    P. Turney. Theoretical analyses of cross-validation error and voting in instance-based learning. Technical Report NRC-35073, Knowledge Systems Laboratory, Institute for Information Technology, National Research Council (Canada), 1993.Google Scholar
  15. 15.
    S Wess and C Globig. Case-based and symbolic classification algorithms — A case study using version space. In Topics in CBR: Selected papers from EWCBR-93, LNCS vol. 837, pages 77–91. Springer-Verlag, 1994.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • A D Griffiths
    • 1
  • D G Bridge
    • 1
  1. 1.Department of Computer ScienceUniversity of YorkYorkUK

Personalised recommendations