IOGA: An instance-oriented genetic algorithm

  • Richard S. Forsyth
Modifications and Extensions of Evolutionary Algorithms Further Modifications and Extensionds
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1141)


Instance-based methods of classification are easy to implement, easy to explain and relatively robust. Furthermore, they have often been found in empirical studies to be competitive in accuracy with more sophisticated classification techniques (Aha et al., 1991; Weiss & Kulikowski, 1991; Fogarty, 1992; Michie et al., 1994). However, a twofold drawback of the simplest instance-based classification method (1-NNC) is that it requires the storage of all training instances and the use of all attributes or features on which those instances are measured — thus failing to exhibit the cognitive economy which is the hallmark of successful learning (Wolff, 1991). Previous researchers have proposed ways of adapting the basic 1-NNC algorithm either to select only a subset of training cases (‘prototypes’) or to discard redundant and/or ‘noisy’ attributes, but not to do both at once. The present paper describes a program (IOGA) that uses an evolutionary algorithm to select prototypical cases and relevant attributes simultaneously, and evaluates it empirically by application to a set of test problems from a variety of fields. These trials show that very considerable economization of storage can be achieved, coupled with a modest gain in accuracy.


Dimensionality Reduction Evolutionary Computing Feature Selection Nearest-Neighbour Classification 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Ackley, D.H. (1987). An Empirical Study of Bit Vector Function Optimization. In: L. Davis, ed., Genetic Algorithms & Simulated Annealing. Pitman, London.Google Scholar
  2. Afifi, A.A. & Azen, S.P. (1979). Statistical Analysis: a Computer Oriented Approach, 2nd. edition, Academic Press, New York.Google Scholar
  3. Aha, D.W., Kibler, D. & Albert, M.K. (1991). Instance-Based Learning Algorithms. Machine Learning, 6, 37–66.Google Scholar
  4. Allaway, S.L., Ritchie, C.D., Robinson, D. & Smolski, O.R. (1988). Detection of Alcohol-Induced Fatty Liver by Computerized Tomography. J. Royal Soc. Medicine, 81, 149–151.Google Scholar
  5. Althoff, K-D., Auriol, E., Barletta, R. & Manago, M. (1995). A Review of Industrial Case-Based Reasoning Tools. AI Intelligence, Oxford.Google Scholar
  6. Anderson, E. (1935). The Irises of the Gaspe Peninsula. Bulletin of the American Iris Society, 59, 2–5.Google Scholar
  7. Andrews, D.F. & Herzberg, A.M. (1985). Data: a Collection of Problems from Many Fields for the Student and Research Worker. Springer-Verlag, New York.Google Scholar
  8. Batchelor, B.G. (1974). Practical Approaches to Pattern Classification. Plenum Press, London.Google Scholar
  9. Batchelor, B.G. (1978) ed. Pattern Recognition: Ideas in Practice. Plenum Press, N.Y.Google Scholar
  10. Breiman, L., Friedman, J.H., Olshen, R.A. & Stone, C.J. (1984). Classification and Regression Trees. Wadsworth, Monterey, California.Google Scholar
  11. Chang, C.L. (1974). Finding Prototypes for Nearest Neighbour Classifiers. IEEE Trans. on Computers, C-23(11), 1179–1184.Google Scholar
  12. Darwin, C.R. & Wallace, A.R. (1858). On the Tendency of Species to Form Varieties; and on the Perpetuation of Varieties and Species by Natural Means of Selection. Paper presented to the London Linnean Society, 1st July 1858. In: D.C. Porter & P.W.Google Scholar
  13. Graham (1993). The Portable Darwin. Penguin, London, 86–104.Google Scholar
  14. Dasarathy, B.V. (1991) ed. Nearest Neighbour (NN) Norms: NN Pattern Classification Techniques. IEEE Computer Society Press, Los Alamitos, California.Google Scholar
  15. Davis, L. (1991) ed. Handbook of Genetic Algorithms. Van Nostrand Reinhold, New York.Google Scholar
  16. Devijer, P.A. & Kittler, J. (1982). Pattern Recognition: a Statistical Approach. Prentice-Hall, New Jersey.Google Scholar
  17. Fisher, R.A. (1936). The Use of Multiple Measurements in Taxonomic Problems. Annals of Eugenics, 7, 179–188.Google Scholar
  18. Fix, E. & Hodges, J.L. (1951). Discriminatory Analysis — Nonparametric Discrimination: Consistency Properties. Project 21-49-004, Report No. 4, USAF School of Aviation Medicine, Randolf Field, Texas, 261–279.Google Scholar
  19. Flury, B. & Riedwyl, H. (1988). Multivariate Statistics: a Practical Approach. Chapman & Hall, London.Google Scholar
  20. Fogarty, T.C. (1992). First Nearest Neighbor Classification on Frey & Slate's Letter Recognition Problem. Machine Learning, 9, 387–388.Google Scholar
  21. Forsyth, R.S. (1989) ed. Machine Learning: Principles & Techniques. Chapman & Hall, London.Google Scholar
  22. Forsyth, R.S. (1990). Neural Learning Algorithms: Some Empirical Trials. Proc. 3rd International Conf. on Neural Networks & their Applications, Neuro-Nimes-90. EC2, Nanterre.Google Scholar
  23. Forsyth, R.S. (1995). Stylistic Structures: a Computational Approach to Text Classification. Doctoral Thesis, University of Nottingham.Google Scholar
  24. Fukunaga, K. & Mantock, J.M. (1984). Nonparametric Data Reduction. IEEE Trans. on Pattern Analysis & Machine Intelligence, PAMI-6(1), 115–118.Google Scholar
  25. Gabor, G. (1975). The eta-NN Method: a Sequential Feature Selection for Nearest Neighbour Decision Rule. In: I. Csiszar & P. Elias, eds., Topics in Information Theory. North-Holland, Amsterdam.Google Scholar
  26. Geva, S. & Sitte, J. (1991). Adaptive Nearest Neighbor Pattern Classification. IEEE Trans. on Neural Networks, NN-2(2), 318–322.CrossRefGoogle Scholar
  27. Goldberg, D.E. (1989). Genetic Algorithms in Search, Optimization & Machine Learning. Addison-Wesley, Reading, Mass.Google Scholar
  28. Hand, D.J. & Batchelor, B.G. (1978). An Edited Nearest Neighbour Rule. Information Sciences, 14, 171–180.CrossRefGoogle Scholar
  29. Hart, P.E. (1968). The Condensed Nearest Neighbour Rule. IEEE Trans. on Info. Theory, IT-14(3), 515–516.CrossRefGoogle Scholar
  30. Holland, J.H. (1975). Adaptation in Natural & Artficial Systems. Univ. Michigan Press, Ann Arbor.Google Scholar
  31. James, M. (1985). Classification Algorithms. Collins, London.Google Scholar
  32. Kelly, J.D. & Davis, L. (1991). Hybridizing the Genetic Algorithm and the K Nearest Neighbors Classification Algorithm. In: R.K. Belew & L.B. Booker, eds., Proc. Fourth Internat. Conf. on Genetic Algorithms. Morgan-Kaufmann, San Mateo, California, 377–383.Google Scholar
  33. Kohonen, T. (1988). Self-Organization & Associative Memory, 2nd. edition. Springer-Verlag, Berlin.Google Scholar
  34. Kolodner, J.L. (1993). Case-Based Reasoning. Morgan Kaufmann, California.Google Scholar
  35. Manly, B.F.J. (1994). Multivariate Statistical Methods: a Primer. Chapman & Hall, London.Google Scholar
  36. McKenzie, D.P. & Forsyth, R.S. (1995). Classification by Similarity: An Overview of Statistical Methods of Case-Based Reasoning. Computers in Human Behavior, 11(2), 273–288.CrossRefGoogle Scholar
  37. McLachlan, G. (1992). Discriminant Analysis and Statistical Pattern Recognition. Wiley, New York.Google Scholar
  38. Michie, D., Spiegelhalter, D.J. & Taylor, C.C. (1994) eds. Machine Learning, Neural and Statistical Classification. Ellis Horwood, Chichester.Google Scholar
  39. Mosteller, F. & Tukey, J.W. (1977). Data Analysis and Regression. Addison-Wesley, Reading, Mass.Google Scholar
  40. Mosteller, F. & Wallace, D.L. (1984). Applied Bayesian and Classical Inference: the Case of the Federalist Papers. Springer-Verlag, New York.Google Scholar
  41. Pei, M., Goodman, E.D., Punch, W.F. & Ding, Y. (1995). Genetic Algorithms for Classification & Feature Extraction. Technical Report: Michican State Univeristy, GA Research Group, Engineering Faculty.Google Scholar
  42. Quinlan, J.R. (1987). Simplifying Decision Trees. Int. J. Man-Machine Studies, 27, 221–234.Google Scholar
  43. Reaven, G.M. & Miller, R.G. (1979). An Attempt to Define the Nature of Chemical Diabetes using a Multidimensional Analysis. Diabetologia, 16, 17–24.CrossRefPubMedGoogle Scholar
  44. Rechenberg, I. (1973). Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution, Frommann-Halzboog, Stuttgart.Google Scholar
  45. Ritter, G.L., Woodruff, H.B., Lowry, S.R. & Isenhour, T.L. (1974). An Algorithm for a Selective Nearest Neighbour Decision Rule. IEEE Trans. on Info. Theory, IT-21(6), 665–669.Google Scholar
  46. Siedlecki, W. & Sklansky, J. (1989). A Note on Genetic Algorithms for Large-scale Feature Selection. Pattern Recognition Letters, 10, 335–347.CrossRefGoogle Scholar
  47. Smith, J.E., Fogarty, T.C. & Johnson, I.R. (1994). Genetic Selection of Features for Clustering and Classification. IEE Colloquium on Genetic Algorithms in Image Processing & Vision. London.Google Scholar
  48. Swonger, C.W. (1972). Sample Set Condensation for a Condensed Nearest Neighbour Decision Rule for Pattern Recognition. In: S. Watanabe, ed., Frontiers of Pattern Recognition. Academic Press.Google Scholar
  49. Tomek, I. (1976). An Experiment with the Edited Nearest-Neighbour Rule. IEEE Trans. on Systems, Man & Cybernetics, SMC-6(6), 448–452.Google Scholar
  50. Ullman, J.R. (1974). Automatic Selection of Reference Data for Use in a Nearest Neighbour Method of Pattern Classification. IEEE Trans. on Info. Theory, IT-20(4), 541–543.CrossRefGoogle Scholar
  51. Weiss, S.M. & Kulikowski, C.A. (1991). Computer Systems that Learn. Morgan Kaufmann, San Mateo, CA.Google Scholar
  52. Whitley, D. (1989). The GENITOR Algorithm and Selective Pressure: why Rank-Based Allocation of Reproductive Trials is Best. Proc. Third Internat. Conf. on GAs, 116–121, Morgan-Kaufmann, Palo Alto, CA.Google Scholar
  53. Wolff, J.G. (1991). Towards a Theory of Cognition and Computing. Ellis Horwood, Chichester.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1996

Authors and Affiliations

  • Richard S. Forsyth
    • 1
  1. 1.Department of Mathematical SciencesUniversity of the West of EnglandBristolUK

Personalised recommendations