Abstract
This paper presents a theory of learning called nested generalized exemplar (NGE) theory, in which learning is accomplished by storing objects in Euclidean n-space, En, as hyperrectangles. The hyperrectangles may be nested inside one another to arbitrary depth. In contrast to generalization processes that replace symbolic formulae by more general formulae, the NGE algorithm modifies hyperrectangles by growing and reshaping them in a well-defined fashion. The axes of these hyperrectangles are defined by the variables measured for each example. Each variable can have any range on the real line; thus the theory is not restricted to symbolic or binary values.
This paper describes some advantages and disadvantages of NGE theory, positions it as a form of exemplarbased learning, and compares it to other inductive learning theories. An implementation has been tested in three different domains, for which results are presented below: prediction of breast cancer, classification of iris flowers, and prediction of survival times for heart attack patients. The results in these domains support the claim that NGE theory can be used to create compact representations with excellent predictive accuracy.
Article PDF
Similar content being viewed by others
References
Aha, D. (1989). Incremental, instance-based learning of independent and graded concept descriptions. Proceedings of the Sixth International Workshop on Machine Learning (pp. 387–391). Ithaca, NY: Morgan Kaufman.
Aha, D., & Kibler, D. (1989). Noise-tolerant instance-based learning algorithms. Proceedings of the Eleventh International Joint Conference on Artificial Intelligence (pp. 794–799). Detroit, Michigan: Morgan Kaufmann.
Aha, D., Kibler, D., & Albert, M. (1991). Instance-based learning algorithms. Machine Learning, 6, 37–66.
Ashley, D., & Rissland, E. (1987). But, see, accord: Generating “Blue Book” citations in HYPO. A case-based system for trade secreta law. Proceedings of the First International Conference on Artificial Intelligence and Law (pp. 67–74). Boston, MA: ACM Press.
Bareiss, R. (1988). Protos: A unified approach to concept representation, classification, and learning. Ph.D. Thesis, University of Texas at Austin. (Technical Report CS-88-10). Nashville, TN: Vanderbilt University, Department of Computer Science.
Bareiss, R., Porter, B., & Murray, K. (1989). Gaining autonomy during knowledge acquisition. (Technical Report AI89–96). Austin, TX: University of Texas, Artificial Intelligence Laboratory.
Barr, R., & Caplan, L. (1987). Category representations and their implications for category structure. Memory and Cognition, 15, 397–418.
Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. (1989). Learnability and the Vapnik-Chervonenkis dimension. Journal of the ACM, 36, 929–965.
Breiman, L., Friedman, J., Olshen, R., & Stone, C. (1984). Classification and regression trees. Belmont: Wadsworth.
Buchanan, B., & Mitchell, T. (1978). Model-directed learning of production rules. In D.Waterman, & F.Hayes-Roth (Eds.), Pattern-directed inference systems. New York: Academic Press.
Bundy, A., Silver, B., & Plummer, D. (1985). An analytical comparison of some rule-learning programs. Artificial Intelligence, 27, 137–181.
Cover, T., & Hart, P. (1967). Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13, 21–27.
Crawford, S. (1989). Extensions to the CART algorithm. The International Journal of Man-Machine Studies, 31, 197–217.
Dietterich, T., & Michalski, R. (1983). A comparative review of selected methods of learning from examples. In R.Michalski, J.Carbonell, & T.Mitchell (Eds.), Machine learning. San Mateo, CA: Morgan Kaufmann.
Everitt, B. (1980). Cluster analysis. Hampshire, England: Gower Publishing Co. Ltd.
Fisher, D. (1989). Noise-tolerant conceptual clustering. Proceedings of IJCAI-89 (pp. 825–830), Detroit, MI: Morgan Kaufmann Publishers.
Fisher, R.A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7, 179–188.
Helmbold, D., Sloan, R., & Warmuth, M. (1989). Learning nested differences of intersection closed concept classes. Proceedings of the 1989 Workshop on Computational Learning Theory. San Mateo, CA: Morgan Kaufmann.
Kan, G., Visser, C., Koolen, J., & Dunning, A. (1986). Short and long term predictive value of wall motion score in acute myocardial infarction. British Heart Journal, 56, 422–427.
Kasif, S. (1989). Personal communication.
Kibler, D., & Aha, D. (1987). Learning representative exemplars of concepts: An initial case study. Proceedings of the Fourth International Workshop on Machine Learning (pp. 24–30). Irvine, CA: Morgan Kaufmann.
Kinney, E. (1988). Personal communication.
Kolodner, J. (1980). Retrieval and organizational strategies in conceptual memory: A computer model. Ph.D. Thesis (Research Report 187). Department of Computer Science, Yale University, New Haven, CT.
Kolodner, J., & Simpson, R. (1984). Problem solving and dynamic memory. Proceedings of the First Annual Workshop on Theoreteical Issues in Conceptual Information Processing (pp. 1–10). Atlanta, GA: Georgia Institute of Technology.
Medin, D. (1983). Structural principles in categorization. In J.Tighe, & B.Shepp (Eds.), Perception, cognition, and development. Hillsdale, NJ: L. Erlbaum Associates.
Medin, D., & Schaffer, M. (1978). Context theory of classification learning. Psychological Review, 85, 207–238.
Michalski, R., Mozetic, I., Hong, J., & Lavrac, N. (1986). The multi-purpose incremental learning system AQ15 and its testing application to three medical domains. Proceedings of AAAI-86 (pp. 1041–1045). Philadelphia, PA: Morgan Kaufmann.
Osherson, D., & Smith, E. (1981). On the adequacy of prototype theory as a theory of concepts. Cognition, 9, 35–58.
Porter, B., Bareiss, R., & Holte, R. (1989). Knowledge acquisition and heuristic classification in weak-theory domains. (Technical Report AI89–96). Austin, TX: University of Texas, Artificial Intelligence Laboratory.
Quinlan, J.R. (1986). Induction of decision trees. Machine Learning, 1, 81–106.
Reed, S. (1972). Pattern recognition and categorization. Cognitive Psychology, 3, 382–407.
Rissland, E., & Ashley, K. (1987). A case-based system for trade secrets law. Proceedings of the First International Conference on Artificial Intelligence and Law (pp. 60–66). Boston, MA: ACM Press.
Salzberg, S. (1985). Heuristics for inductive learning. Proceedings of IJCAI-85 (pp. 603–610). Los Angeles, CA: Morgan Kaufmann.
Salzberg, S. (1986). Pinpointing good hypotheses with heuristics. In W.Gale (Ed.), Artificial Intelligence and Statistics, Reading, MA: Addison-Wesley Publishing Co.
Salzberg, S. (1989). Learning with nested generalized exemplars. Ph.D. Thesis (Technical Report TR-14-89). Cambridge, MA: Harvard University, Department of Computer Science.
Smith, E., & Osherson, D. (1984). Conceptual combination with prototype concepts. Cognitive Science, 8, 337–361.
Thornton, C. (1987). Hypercuboid formation behaviour of two learning algorithms. Proceedings of IJCAI-87 (pp. 301–303). Milan, Italy: Morgan Kaufmann.
Vere, S. (1980). Multilevel counterfactuals for generalizations of relational concepts and productions. Artificial Intelligence, 14, 138–164.
Valiant, L. (1984). A theory of the learnable. Communications of the ACM, 27, 1134–1142.
Warmuth, M. (1989). Personal communication.
Weiss, S., & Kapouleas, I. (1989). An empirical comparison of pattern recognition, neural nets, and machine learning classification methods. Proceedings of IJCAI-89 (pp. 781–787). Detroit, MI: Morgan Kaufmann.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Salzberg, S. A nearest hyperrectangle learning method. Mach Learn 6, 251–276 (1991). https://doi.org/10.1007/BF00114779
Issue Date:
DOI: https://doi.org/10.1007/BF00114779