Abstract
In this paper we present a probabilistic framework for case-based reasoning in data-intensive domains, where only weak prior knowledge is available. In such a probabilistic viewpoint the attributes are interpreted as random variables, and the case base is used to approximate the underlying joint probability distribution of the attributes. Consequently structural case adaptation (and parameter adjustment in particular) can be viewed as prediction based on the full probability model constructed from the case history. The methodology addresses several problems encountered in building case-based reasoning systems. It provides a computationally efficient structural adaptation algorithm, avoids over-fitting by using Bayesian model selection and uses directly probabilities as measures of similarity. The methodology described has been implemented in the D-SIDE software package, and the approach is validated by presenting empirical results of the method's classification prediction performance for a set of public domain data sets.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
D. Aha, D. Kibler, and M. Albert. Instance-based learning algorithms. Machine Learning, 6:37–66, 1991.
R. Barletta. An introduction to case-based reasoning. AI Expert, pages 42–49, 1991.
J.M. Bernardo and A.F.M Smith. Bayesian theory. John Wiley, 1994.
B. Cestnik and I. Bratko. On estimating probabilities in tree pruning. In Y. Kodratoff, editor, Machine Learning EWSL-91, pages 138–150. Springer-Verlag, 1991.
P. Cheeseman, J. Kelly, M. Self, J. Stutz, W. Taylor, and D. Freeman. Autoclass: A Bayesian classification system. In Proceedings of the Fifth International Conference on Machine Learning, pages 54–64, Ann Arbor, June 1988.
D.M. Chickering and D. Heckerman. Efficient approximations for the marginal likelihood of incomplete data given a Bayesian network. Technical Report MSR-TR-96-08, Microsoft Research, Advanced Technology Division, 1996.
W. Clancey. Heuristic classification. Artificial Intelligence, 27:289–350, 1985.
T. Cover and P. Hart. Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13:21–27, 1967.
M.H. DeGroot. Optimal statistical decisions. McGraw-Hill, 1970.
A.P. Dempster, N.M. Laird, and D.B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39(1):1–38, 1977.
B.S. Everitt and D.J. Hand. Finite Mixture Distributions. Chapman and Hall, London, 1981.
N. Friedman and M. Goldszmidt. Building classifiers using Bayesian networks. In Proceedings of AAAI-96 (to appear), 1996.
N. Friedman and M. Goldszmidt. Discretizing continuous attributes while learning Bayesian networks. In L. Saitta, editor, Machine Learning: Proceedings of the Thirteenth International Conference (to appear). Morgan Kaufmann Publishers, 1996.
A. Gelman, J. Carlin, H. Stern, and D. Rubin. Bayesian Data Analysis. Chapman & Hall, 1995.
R.C. Holte. Very simple classification rules perform well on most commonly used datasets. Machine Learning, 11:63–91, 1993.
G.H. John and P. Langley. Estimating continuous distributions in Bayesian classifiers. In P. Besnard and S. Hanks, editors, Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence, pages 338–345. Morgan Kaufmann Publishers, 1995.
M.I. Jordan and R.A. Jacobs. Hierarchical mixtures of experts and the EM algorithm. Neural Computation, 6:181–214, 1994.
R.E. Kass and A.E. Raftery. Bayes factors. Technical Report 254, Department of Statistics, University of Washington, 1994.
M. Keane. Analogical asides on case-based reasoning. In S. Wess, K.-D. Althoff, and M Richter, editors, Topics in Case-Based Reasoning, volume 837 of Lecture Notes in Artificial Intelligence, pages 21–32. Springer-Verlag, 1994.
J. Kolodner. Case-Based Reasoning. Morgan Kaufmann Publishers, San Mateo, 1993.
I. Kononenko. Successive naive Bayesian classifier. Informatica, 17:167–174, 1993.
I. Kononenko and I. Bratko. Information-based evaluation criterion for classifier's performance. Machine Learning, 6:67–80, 1991.
P. Kontkanen, P. Myllymäki, and H. Tirri. Unsupervised Bayesian learning of discrete finite mixtures. Manuscript, submitted for publication.
P. Kontkanen, P. Myllymäki, and H. Tirri. Comparing Bayesian model class selection criteria by discrete finite mixtures. In Proceedings of the ISIS (Information, Statistics and Induction in Science) Conference, Melbourne, Australia, August 1996. (To appear.).
P. Kontkanen, P. Myllymäki, and H. Tirri. Constructing Bayesian finite mixture models by the EM algorithm. Technical Report C-1996-9, University of Helsinki, Department of Computer Science, February 1996.
P. Koton. Using experience in learning and problem solving. PhD thesis, Massachusetts Institute of Technology, 1989.
R.J.A. Little and D.B. Rubin. Statistical analysis with missing data. Wiley, 1987.
D. Michie, D.J. Spiegelhalter, and C.C. Taylor, editors. Machine Learning, Neural and Statistical Classification. Ellis Horwood, London, 1994.
P. Myllymäki and H. Tirri. Bayesian case-based reasoning with neural networks. In Proceedings of the IEEE International Conference on Neural Networks, volume 1, pages 422–427, San Francisco, March 1993. IEEE, Piscataway, NJ.
P. Myllymäki and H. Tirri. Massively parallel case-based reasoning with probabilistic similarity metrics. In S. Wess, K.-D. Althoff, and M Richter, editors, Topics in Case-Based Reasoning, volume 837 of Lecture Notes in Artificial Intelligence, pages 144–154. Springer-Verlag, 1994.
P. Myllymäki and H. Tirri. Constructing computationally efficient Bayesian models via unsupervised clustering. In A. Gammerman, editor, Probabilistic Reasoning and Bayesian Belief Networks, pages 237–248. Alfred Waller Publishers, Suffolk, 1995.
C. Owens. Integrating feature extraction and memory search. Machine Learning, 10(3):311–340, 1993.
J.R. Quinlan. Improved use of continuous attributes in C4.5. Journal of Artificial Intelligence Research, 4:77–90, 1996.
R. Schank. Dynamic Memory: A theory of reminding and learning in computers and people. Cambridge University Press, 1982.
D.W. Scott. Multivariate Density Estimation. Theory, Practice, and Visualization. John Wiley & Sons, New York, 1992.
D. Skalak. Prototype and feature selection by sampling and random mutation hill climbing algorithms. In Machine Learning: Proceedings of the Eleventh International Conference, pages 293–301, 1994.
H. Tirri, P. Kontkanen, and P. Myllymäki. Probabilistic instance-based learning. In L. Saitta, editor, Machine Learning: Proceedings of the Thirteenth International Conference, pages 507–515. Morgan Kaufmann Publishers, 1996.
D.M. Titterington, A.F.M. Smith, and U.E. Makov. Statistical Analysis of Finite Mixture Distributions. John Wiley & Sons, New York, 1985.
I. Watson and F. Marir. Case-based reasoning: A review. The Knowledge Engineering Review, 9(4):327–354, 1994.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1996 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tirri, H., Kontkanen, P., Myllymäki, P. (1996). A bayesian framework for case-based reasoning. In: Smith, I., Faltings, B. (eds) Advances in Case-Based Reasoning. EWCBR 1996. Lecture Notes in Computer Science, vol 1168. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0020627
Download citation
DOI: https://doi.org/10.1007/BFb0020627
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-61955-0
Online ISBN: 978-3-540-49568-0
eBook Packages: Springer Book Archive