Abstract
Learning algorithms differ in the degree to which they process their inputs prior to their use in performance tasks. Many algorithms eagerly compile input samples and use only the compilations to make decisions. Others are lazy: they perform less precompilation and use the input samples to guide decision making. The performance of many lazy learners significantly degrades when samples are defined by features containing little or misleading information. Distinguishing feature relevance is a critical issue for these algorithms, and many solutions have been developed that assign weights to features. This chapter introduces a categorization framework for feature weighting approaches used in lazy similarity learners and briefly surveys some examples in each category.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Aha, D. W. (1991). Incremental constructive induction: An instance-based approach. In Proceedingsof the Eighth International Workshop on Machine Learning, pages 117–121, Evanston, IL. Morgan Kaufmann.
Aha, D. W. (1992). Tolerating noisy, irrelevant, and novel attributes in instance-based learning algorithms. International Journal of Man-Machine Studies, 36:267–287.
Aha, D. W., editor (1997). Lazy Learning.Kluwer, Norwell, MA.
Aha, D. W. and Bankert, R. L. (1996). A comparative evaluation of sequential feature selection algorithms. In Fisher, D. and Lenz, J.-H., editors, Artificial Intelligence and Statistics V.Springer, New York.
Aha, D. W. and Goldstone, R. L. (1992). Concept learning and flexible weighting. In Proceedings of the Fourteenth Annual Conference of the Cognitive Science Society, pages 534–539, Bloomington, IN. Lawrence Erlbaum.
Atkeson, C., Moore, A., and Schaal, S. (1997). Locally weighted learning. AI Review, 11:11–73.
Barletta, R. and Mark, W. (1988). Explanation-based indexing of cases. In Proceedings of the Seventh National Conference on Artificial Intelligence,pages 541–546, St. Paul, MN. Morgan Kaufmann.
Barsalou, L. W. (1983). Ad hoc categories. Memory &1 Cognition, 11:211–227.
Bonzano, A., Cunningham, P., and Smyth, B. (1997). Using introspective learning to improve retrieval in CBR: A case study in air traffic control. In Proceedings of the Second ICCBR Conference, pages 291–302, Providence, RI. Springer.
Borrajo, D. and Veloso, M. (1997). Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review, 11:371–405.
Cain, T., Pazzani, M. J., and Silverstein, G. (1991). Using domain knowledge to influence similarity judgement. In Proceedings of a Case-Based Reasoning Workshop,pages 191–202, Washington, DC. Morgan Kaufmann.
Cardie, C. (1993). Using decision trees to improve case-based learning. In Proceedings of the Tenth ICML, pages 25–32, Amherst, MA. Morgan Kaufmann.
Cardie, C. and Howe, N. (1997). Improving minority-class prediction using case-specific feature weights. In Proceedings of the Fourteenth ICML,pages 57–65, Nashville, TN. Morgan Kaufmann.
Chang, E. I. and Lippman, R. P. (1991). Using genetic algorithms to improve pattern classification performance. In Lippman, R., Moody, J., and Touretzsky, D. S., editors, Advances in Neural Information Processing Systems 3.Morgan Kaufmann, Denver, CO.
Creecy, R. H., Masand, B. M., Smith, S. J., and Waltz, D. L. (1992). Trading mips and memory for knowledge engineering. CACM, 35:48–64.
Daelemans, W. and van den Bosch, A. (1992). Generalization performance of backpropagation learning on a syllabification task. In Proceedings of TWLT3:Connectionism and Natural Language Processing,pages 27–37, Enschede, The Netherlands. Unpublished.
Dasarathy, B. V. (1991). Nearest neighbor(NN) norms: NN pattern classification techniques.IEEE Computer Society Press, Los Alamitos, CA.
Datta, P. and Kibler, D. (1997). Learning symbolic prototypes. In Proceedings of the Fourteenth ICML, pages 158–166, Tahoe City, CA. Morgan Kaufmann.
Domingos, P. (1997). Context-sensitive feature selection for lazy learners. AI Review, 11:227–253.
Dudani, S. (1975). The distance-weighted k-nearest neighbor rule. IEEE Transactions on Systems man, and Cybernetics, 6:325–327.
Fix, E. and J. L. Hodges, Jr. (1951). Discriminatory analysis, nonparametric discrimination, consistency properties. Technical Report 4, United States Air Force, School of Aviation Medicine.
Fox, S. and Leake, D. L. (1995). Using introspective reasoning to refine indexing. In Proceedings of the Fourteenth IJCAI, pages 391–397, Montreal. Morgan Kaufmann.
Friedman, J. H. (1994). Flexible metric nearest neighbor classification. Technical report, Stanford University, Department of Statistics.
Friedman, J. H., Kohavi, R., and Yun, Y. (1996). Lazy decision trees. In Proceeding of the Thirteenth National Conference on Artificial Intelligence, pages 717–724, Portland, OR. AAAI Press.
Fukunaga, K. and Flick, T. (1982). A parametrically-defined nearest neighbor distance measure. Pattern Recognition Letters, 1:3–5.
Fukunaga, K. and Flick, T. (1984). An optimal global nearest neighbor metric. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6:314–318.
Griffiths, A. G. and Bridge, D. G. (1997). PAC analyses of a `similarity learning’ IBL algorithm. In Proceedings of the Second ICCBR Conference, pages 445–454, Providence, RI. Springer.
Güvenir, H. A. and Sirin, I. (1996). Classification by feature partitioning. Machine Learning, 23:47–68.
Hastie, T. and Tibshirani, R. (1996). Discriminant adaptive nearest neighborclassification. IEEE Pattern Analysis and Machine Intelligence, 18:607–616.
Hastings, J., Branting, L. K., and Lockwood, J. A. (1995). Case adaptation using an incomplete causal model. In Proceedings of the First ICCBR Conference, pages 181–192, Sesimbra, Portugal. Springer.
Howe, N. and Cardie, C. (1997). Examining locally varying weights for nearest neighbor algorithms. In Proceedings of the Second ICCBR Conference, pages 455–466, Providence, RI. Springer.
J. D. Kelly, Jr. and Davis, L. (1991). A hybrid genetic algorithm for classification. In Proceedings of the Twelfth IJCAI,pages 645–650, Sydney, Australia. Morgan Kaufmann.
John, G., Kohavi, R., and Pfleger, K. (1994). Irrelevant features and the subset selection problem. In Proceedings of the Eleventh ICML, pages 121–129, New Brunswick, NJ. Morgan Kaufmann.
Kibler, D. and Aha, D. W. (1987). Learning representative exemplars of concepts: An initial case study. In Proceedings of the Fourth International Workshop on Machine Learning, pages 24–30, Irvine, CA. Morgan Kaufmann.
Kira, K. and Rendell, L. A. (1992). The feature selection problem: Traditional methods and a new algorithm. In Proceedings of the Tenth IJCAI,pages 129–134, San Jose, CA. AAAI Press.
Kohavi, R., Langley, P., and Yun, Y. (1997). The utility of feature weighting in nearest neighbor algorithms. In van Someren, M. and Widmer, G., editors, Poster Papers: Ninth ECML.Unpublished, Prague, Czech Republic.
Kononenko, I. (1994). Estimating attributes: Analysis and extensions of relief. In Proceedings of the Seventh ECML, pages 171–182, Catania, Italy. Springer.
Kontkanen, P., Myllymaki, P., Silander, T., and Tirri, H. (1998). Bayes optimal instance-based learning. In Proceedings of the Tenth ECML, Chemnitz, Germany. Springer.
Lachiche, N. and Marquis, P. (1998). Scope classification: An instance-based learning algorithm with a rule-based characterization. In Proceedings of the Tenth ECML, Chemnitz, Germany. Springer.
Langley, P. and Iba, W. (1993). Average-case analysis of a nearest neighbor algorithm. In Proceedings of the Thirteenth IJCAI, pages 889–894, Chambery, France. Morgan Kaufmann.
Leng, B. and Trott, J. (1997). Automatically tuning question weights in CBR Express 2.x. Unpublished Manuscript.
Ling, X. C. and Wang, H. (1997). Towards optimal weights setting for the 1-nearest neighbour learning algorithm. AI Review, 11:255–272.
Lowe, D. (1995). Similarity metric learning for a variable-kernal classifier. Neural Computation, 7:72–85.
Maron, O. and Moore, A. W. (1997). The racing algorithm: Model selection for lazy learners. AI Review,11:192–225.
Mitchell, T. (1997). Machine Learning.McGraw-Hill, New York.
Mohri, T. and Tanaka, H. (1994). An optimal weighting criterion of case indexing for both numeric and symbolic attributes. In Aha, D. W., editor, Case-Based Reasoning: Papers from the 1994 Workshop.AAAI Press, Menlo Park, CA.
Myles, J. and Hand, D. (1990). The multi-class metric problem in nearest neighbor discrimination rules. Pattern Recognition, 23:1291–1297.
Muñoz Avila, H. M. and Hüllen, J. (1996). Feature weighting by explaining casebased planning episodes. In Proceedings of the Third European Workshop on Case-Based Reasoning, pages 280–294, Lausanne, Switzerland. Springer.
Ricci, F. and Aha, D. W. (1998). Error-correcting output codes for local learners. In Proceedings of the Tenth ECML, Chemnitz, Germany. Springer.
Ricci, F. and Avesani, P. (1995). Learning a local similarity metric for casebased reasoning. In Proceedings of the First ICCBR Conference, pages 301–312, Sesimbra, Portugal. Springer.
Salzberg, S. L. (1991). A nearest hyperrectangle learning method. Machine Learning, 6:251–276.
Satoh, K. and Okamoto, S. (1994). Toward PAC-learning of weights from qualitative distance information. In Aha, D. W., editor, Case-Based Reasoning: Papers from the 1994 Workshop.AAAI Press, Menlo Park, CA.
Shimazu, H., Shibata, A., and Nihei, K. (1994). Case-based retrieval interface adapted to customer-initiated dialogues in help desk operations. In Proceedings of the Twelfth National Conference on Artificial Intelligence, pages 513–518, Seattle, WA. AAAI Press.
Short, R. and Fukunaga, K. (1981). The optimal distance measure for nearest neighbor classification. IEEE Transactions on Information Theory, 27:622–627.
Skalak, D. (1994). Prototype and feature selection by sampling and random mutation hill climbing algorithms. In Proceedings of the Eleventh ICML, pages 293–301, New Brunswick, NJ. Morgan Kaufmann.
Smyth, B. and Cunningham, P. (1995). A comparison of incremental case-based reasoning and inductive learning. In Haton, J. P. and Keane, M., editors, Advances in Case-Based Reasoning.Springer, Berlin.
Stanfill, C. and Waltz, D. (1986). Toward memory-based reasoning. CA CM, 29:1213–1228.
Tan, M. (1993). Cost-sensitive learning of classification knowledge and its application in robotics. Machine Learning, 13:7–34.
Trott, J. R. and Leng, B. (1997). An engineering approach for troubleshooting case bases. In Proceedings of the Second ICCBR Conference, pages 178–189, Providence, RI. Springer.
Turney, P. D. (1995). Cost-sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm. Journal of Artificial Intelligence Research, 2:369–409.
Wettschereck, D. (1994). A study of distance-based machine learning algorithms. Doctoral dissertation, Oregon State University, Department of Computer Science.
Wettschereck, D., Aha, D. W., and Mohri, T. (1997). A review and empirical comparison of feature weighting methods for a class of lazy learning algorithms. AI Review, 11:273–314.
Wilke, W. and Bergmann, R. (1996). Considering decision costs while learning of feature weights. In Proceedings of the Third European Workshop on Case-Based Reasoning, pages 460–472, Lausanne, Switzerland. Springer.
Wilson, D. R. and Martinez, T. R. (1996). Instance-based learning with genetically derived attribute weights. In Proceedings of the International Conference on Artificial Intelligence, Expert Systems, and Neural Networks, pages 11–14. Unpublished.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer Science+Business Media New York
About this chapter
Cite this chapter
Aha, D.W. (1998). Feature Weighting for Lazy Learning Algorithms. In: Liu, H., Motoda, H. (eds) Feature Extraction, Construction and Selection. The Springer International Series in Engineering and Computer Science, vol 453. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-5725-8_2
Download citation
DOI: https://doi.org/10.1007/978-1-4615-5725-8_2
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4613-7622-4
Online ISBN: 978-1-4615-5725-8
eBook Packages: Springer Book Archive