Abstract
Ensembles of classifiers will produce lower errors than the member classifiers if there is diversity in the ensemble. One means of producing this diversity in nearest neighbour classifiers is to base the member classifiers on different feature subsets. In this paper we show four examples where this is the case. This has implications for the practice of feature subset selection (an important issue in CBR and data-mining) because it shows that, in some situations, there is no single best feature subset to represent a problem. We show that if diversity is emphasised in the development of the ensemble that the ensemble members appear to be local learners specializing in sub-domains of the problem space. The paper concludes with some proposals on how analysis of ensembles of local learners might provide insight on problem-space decomposition for hierarchical CBR.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aha, D.W. (1998) Feature weighting for lazy learning algorithms. In: H. Liu and H. Motoda (Eds.) Feature Extraction, Construction and Selection: A Data Mining Perspective. Norwell MA: Kluwer.
Bonzano, A., Cunningham, P., Smyth, B., (1997) Using introspective learning to improve retrieval in CBR: A case study in air traffic control International Conference on Case-Based Reasoning, Providence, Lecture Notes in Computer Science, SpringerVerlag, E. Plaza & D. Leake (eds), pp291–302.
Breiman, L., (1996) Bagging predictors. Machine Learning, 24:123–140.
Cherkauer, K.J. (dy1995) Stuffing Mind into Computer: Knowledge and Learning for Intelligent Systems. Informatica 19:4 (501–511) Nov. 1995
Condorcet, Marquis J. A. (1781) Sur les elections par scrutiny, Histoire de l’Academie Royale des Sciences, 31–34.
Cunningham, P., Carney, J., (2000) Diversity versus Quality in Classification Ensembles based on Feature Selection, 11th European Conference on Machine Learning (ECML 2000), Lecture Notes in Artificial Intelligence, R. López de Mántaras and E. Plaza, (eds) pp109–116, Springer Verlag.
Guerra-Salcedo, C., Whitley, D., (1999a). Genetic Approach for Feature Selection for Ensemble Creation. in GECCO-99: Proceedings of the Genetic and Evolutionary Computation Conference, Banzhaf, W., Daida, J., Eiben, A. E., Garzon, M. H., Honavar, V., Jakiela, M., & Smith, R. E. (eds.). Orlando, Florida USA, pp236–243, San Francisco, CA: Morgan Kaufmann.
Guerra-Salcedo, C., Whitley, D., (1999b). Feature Selection Mechanisms for Ensemble Creation: A Genetic Search Perspective, in Data Mining with Evolutionary Algorithms: Research Directions. Papers from the AAAI Workshop. Alex A. Freitas (Ed.) Technical Report WS-99-06. AAAI Press, 1999.
Hansen, L.K., Salamon, P., (1990) Neural Network Ensembles, IEEE Pattern Analysis and Machine Intelligence, 1990. 12, 10, 993–1001.
Ho, T.K., (1998a) The Random Subspace Method for Constructing Decision Forests, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20, 8, 832–844.
Ho, T.K., (1998b) Nearest Neighbours in Random Subspaces, Proc. Of 2nd International Workshop on Statistical Techniques in Pattern Recognition, A. Amin, D. Dori, P. Puil, H. Freeman, (eds.) pp640–648, Springer Verlag LNCS 1451.
Kohavi, P. Langley, Y. Yun, (1997) The Utility of Feature Weighting in NearestNeighbor Algorithms, European Conference on Machine Learning, ECML’97, Prague, Czech Republic, 1997, poster.
Krogh, A., Vedelsby, J., (1995) Neural Network Ensembles, Cross Validation and Active Learning, in Advances in Neural Information Processing Systems 7, G. Tesauro, D. S. Touretsky, T. K. Leen, eds., pp231–238, MIT Press, Cambridge MA.
Liu Y., Yao X. (1999) Ensemble learning via negative correlation, Neural Networks 12, 1999.
Newell, A., & Simon, H.A., (1976) Computer Science as Empirical Enquiry: Symbols and Search. Communications of ACM, 19(3), 1976, pp.113–126.
Nitzan, S.I., Paroush, J., (1985) Collective Decision Making. Cambridge: Cambridge University Press.
Opitz D., Shavlik J., (1996) Generating Accurate and diverse members of a Neural Network Ensemble, Advances in Neural Information Processing Systems, pp. 535–543, Denver, CO. MIT Press. 1996.
Richter, M. M. (1998). Introduction (to Case-Based Reasoning). in Case-based reasoning technology: from foundations to applications, Lenz, M., Bartsch-Spörl, B., Burkhard, H.-D. & Wess, S. (eds.) (1998). Springer-Verlag, LNAI 1400, pp1–16.
Smyth B., Cunningham P., (1992) Déjà Vu: A Hierarchical Case-Based Reasoning System for Software Design, in Proceedings of European Conference on Artificial Intelligence, ed. Bernd Neumann, John Wiley, pp587–589, Vienna Austria, August 1992.
Smyth, B., Keane, M., & Cunningham, P., (2000) Hierarchical Case-Based Reasoning: Integrating Case-Based and Decompositional Problem-Solving Techniques for Plant-Control Software Design, to appear in IEEE Transactions on Knowledge and Data Engineering.
Tumer, K., and Ghosh, J., (1996) Error Correlation and Error Reduction in Ensemble Classifiers, Connection Science, Special issue on combining artificial neural networks: ensemble approaches, Vol. 8, No. 3 & 4, pp 385–404.
van de Laar, P., Heskes, T., (2000) Input selection based on an ensemble, Neurocomputing, 34:227–238.
Wettschereck, D., & Aha, D. W. (Eds.) (1997). ECML-97 MLNet Workshop Notes: Case-Based Learning: Beyond Classification of Feature Vectors (Technical Report AIC-97-005). Washington, DC: Naval Research Laboratory, Navy Center for Applied Research in Artificial Intelligence.
Wettschereck, D., Aha, D. W., & Mohri, T. (1997). A review and comparative evaluation of feature weighting methods for lazy learning algorithms. Artificial Intelligence Review, 11, 273–314.
Zenobi, G., & Cunningham, P., (2001) Using ambiguity in preparing ensembles of classifiers based on different feature subsets to minimize generalization error, submitted to ECML 2001.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cunningham, P., Zenobi, G. (2001). Case Representation Issues for Case-Based Reasoning from Ensemble Research. In: Aha, D.W., Watson, I. (eds) Case-Based Reasoning Research and Development. ICCBR 2001. Lecture Notes in Computer Science(), vol 2080. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44593-5_11
Download citation
DOI: https://doi.org/10.1007/3-540-44593-5_11
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42358-4
Online ISBN: 978-3-540-44593-7
eBook Packages: Springer Book Archive