Advertisement

Download PDF

Machine Learning

, Volume 6, Issue 2, pp 161–182 | Cite as

Letter recognition using Holland-style adaptive classifiers

  • Peter W. Frey
  • David J. Slate
Article

Abstract

Machine rule induction was examined on a difficult categorization problem by applying a Holland-style classifier system to a complex letter recognition task. A set of 20,000 unique letter images was generated by randomly distorting pixel images of the 26 uppercase letters from 20 different commercial fonts. The parent fonts represented a full range of character types including script, italic, serif, and Gothic. The features of each of the 20,000 characters were summarized in terms of 16 primitive numerical attributes. Our research focused on machine induction techniques for generating IF-THEN classifiers in which the IF part was a list of values for each of the 16 attributes and the THEN part was the correct category, i.e., one of the 26 letters of the alphabet. We examined the effects of different procedures for encoding attributes, deriving new rules, and apportioning credit among the rules. Binary and Gray-code attribute encodings that required exact matches for rule activation were compared with integer representations that employed fuzzy matching for rule activation. Random and genetic methods for rule creation were compared with instance-based generalization. The strength/specificity method for credit apportionment was compared with a procedure we call “accuracy/utility”.

Keywords

Category learning parallel rule-based systems exemplar-based induction apportionment of credit fuzzy-match rule activation 
Download to read the full article text

References

  1. AckleyD.H., HintonG.E., & SejnowskiT.J. (1985). A learning algorithm for Boltzmann machines.Cognitive Science,9, 147–169.Google Scholar
  2. Anderson, J.A. (1983). Cognitive and psychological computation with neural models.IEEE Transactions on Systems, Man, and Cybernetics, SMC-13, 799–815.Google Scholar
  3. BookerL.B. (1988). Classifier systems that learn internal world models.Machine Learning,3, 161–192.Google Scholar
  4. CaruanaR.A., & SchafferD. (1988). Representation and hidden bias: gray vs. binary coding for genetic algorithms.Proceedings of the Fifth International Conference on Machine Learning (pp. 153–161). Ann Arbor, MI: Morgan Kaufmann Publishers.Google Scholar
  5. CharnessN. (1981). Aging and skilled problem-solving.Journal of Experimental Psychology: General,110, 21–38.Google Scholar
  6. ChaseW.G., & SimonH.A. (1973). Perception in chess.Cognitive Psychology,4, 55–81.Google Scholar
  7. DavisL., & YoungD.K. (1988). Classifier systems with Hamming weights.Proceedings of the Fifth International Conference on Machine Learning (pp. 162–173). Ann Arbor, MI: Morgan Kaufmann Publishers.Google Scholar
  8. deGrootA.D. (1965).Thought and choice in chess. The Hague: Mouton, 2nd edition, 1978.Google Scholar
  9. GoldbergD.E. (1989).Genetic algorithms in search, optimization, and machine learning. Reading, MA: Addison-Wesley Publishing.Google Scholar
  10. HollandJ.H. (1975).Adaptation in natural and artificial systems. Ann Arbor, MI: University of Michigan Press.Google Scholar
  11. HollandJ.H. (1980). Adaptive algorithms for discovering and using general patterns in growing knowledge bases.International Journal of Policy Analysis and Information Systems,4, 217–240.Google Scholar
  12. HollandJ.H. (1986). Escaping brittleness: The possibilities of general purpose machine learning algorithms applied to parallel rule-based systems. In R.S.Michalski, J.G.Carbonell, & T.M.Mitchell (Eds.),Machine learning: An artificial intelligence approach (Vol. II). San Mateo, CA: Morgan Kaufmann Publishers.Google Scholar
  13. HollandJ.H., HolyoakK.J., NisbettR.E., & ThagardP.R. (1986).Induction: Processes of inference, learning, and discovery. Cambridge, MA: The MIT Press.Google Scholar
  14. HuntE.B., MarinJ., & StoneP.J. (1966).Experiments in induction. New York: Academic Press.Google Scholar
  15. QuinlanJ.R. (1979). Discovering rules by induction from large collections of examples. In D.Michie (Ed.),Expert systems in the micro electronic age. Edinburgh: Edinburgh University Press.Google Scholar
  16. QuinlanJ.R. (1986). Induction of decision trees.Machine Learning,1, 81–106.Google Scholar
  17. RobertsonJ.R. (1988). Population size in classifier systems.Proceedings of the Fifth International Conference on Machine Learning (pp. 142–152). Ann Arbor, MI: Morgan Kaufmann Publishers.Google Scholar
  18. RumelhartD.E., & ZipserD. (1985). Feature discovery by competitive learning.Cognitive Science,9, 75–112.Google Scholar
  19. StanfillC., & WaltzD. (1986). Toward memory-based reasoning.Communications of the ACM,29, 1213–1228.Google Scholar
  20. SuttonR. (1988). Learning to predict by the method of temporal differences.Machine Learning,3, 9–44.Google Scholar
  21. WilsonS.W. (1985). Knowledge growth in an artificial animal.Proceedings of an International Conference on Genetic Algorithms and Their Applications (pp. 16–23). Pittsburgh, PA: Lawrence Erlbaum Associates.Google Scholar
  22. WilsonS.W. (1987). Classifier systems and the animat problem.Machine Learning,2, 199–228.Google Scholar
  23. WilsonS.W. (1988). Bid competition and specificity reconsidered.Complex Systems,2, 705–723.Google Scholar

Copyright information

© Kluwer Academic Publishers 1991

Authors and Affiliations

  • Peter W. Frey
    • 1
  • David J. Slate
    • 2
  1. 1.Department of PsychologyNorthwestern UniversityEvanston
  2. 2.Pattern Recognition GroupOdesta CorporationEvanston

Personalised recommendations