Generating Classifier Ensembles from Multiple Prototypes and Its Application to Handwriting Recognition

  • Simon Günter
  • Horst Bunke
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2364)

Abstract

There are many examples of classification problems in the literature where multiple classifier systems increase the performance over single classifiers. Normally one of the two following approaches is used to create a multiple classifier system. 1. Several classifiers are developed completely independent of each other and combined in a last step. 2. Several classifiers are created out of one base classifier by using so called classifier ensemble creation methods. In this paper algorithms which combine both approaches are introduced and they are experimentally evaluated in the context of an hidden Markov model (HMM) based handwritten word recognizer.

Keywords

Multiple Classifier System Ensemble Creation Method AdaBoost Hidden Markov Model (HMM) Handwriting Recognition 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    A. Brakensiek, J. Rottland, A. Kosmala, and G. Rigoll. Off-line handwriting recognition using various hybrid modeling techniques and character n-grams. In 7th International Workshop on Frontiers in Handwritten Recognition, pages 343–352, 2000.Google Scholar
  2. 2.
    Leo Breiman. Bagging predictors. Machine Learning, (2):123–140, 1996.Google Scholar
  3. 3.
    T. G. Dietterich. Ensemble methods in machine learning. In F. Roli, editors. First International Workshop on Multiple Classifier Systems, Cagliari, Italy, 2000. Springer. [12]}, pages 1–15.Google Scholar
  4. 4.
    T. G. Dietterich and G. Bakiri. Solving multiclass learning problems via errorcorrecting output codes. Journal of Artifical Intelligence Research, 2:263–286, 1995.MATHGoogle Scholar
  5. 5.
    T.G. Dietterich and E.B. Kong. Machine learning bias, statistical bias, and statistical variance of decision tree algorithms. Technical report, Departement of Computer Science, Oregon State University, 1995.Google Scholar
  6. 6.
    R. Duin and D. Tax. Experiments with classifier combination rules. In F. Roli, editors. First International Workshop on Multiple Classifier Systems, Cagliari, Italy, 2000. Springer. [12]}, pages 16–29.Google Scholar
  7. 7.
    Yoav Freund and Robert E. Schapire. A descision-theoretic generalisation of online learning and an application to boosting. Journal of Computer and Systems Sciences, 55(1):119–139, 1997.MATHCrossRefGoogle Scholar
  8. 8.
    T. K. Ho. The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(8):832–844, 1998.CrossRefGoogle Scholar
  9. 9.
    S. Impedovo, P. Wang, and H. Bunke, editors. Automatic Bankcheck Processing. World Scientific Publ. Co, Singapore, 1997.Google Scholar
  10. 10.
    A. Kaltenmeier, T. Caesar, J.M. Gloger, and E. Mandler. Sophisticated topology of hidden Markov models for cursive script recognition. In Proc. of the 2nd Int. Conf. on Document Analysis and Recognition, Tsukuba Science City, Japan, pages 139–142, 1993.Google Scholar
  11. 11.
    G. Kim, V. Govindaraju, and S.N. Srihari. Architecture for handwritten text recognition systems. In S.-W. Lee, editor, Advances in Handwriting Recognition, pages 163–172. World Scientific Publ. Co., 1999.Google Scholar
  12. 12.
    J. Kittler and F. Roli, editors. First International Workshop on Multiple Classifier Systems, Cagliari, Italy, 2000. Springer.Google Scholar
  13. 13.
    J. Kittler and F. Roli, editors. Second International Workshop on Multiple Classifier Systems, Cambridge, UK, 2001. Springer.Google Scholar
  14. 14.
    D. Lee and S. Srihari. Handprinted digit recognition: A comparison of algorithms. In Third International Workshop on Frontiers in Handwriting Recognition, pages 153–162, 1993.Google Scholar
  15. 15.
    U. Marti and H. Bunke. A full English sentence database for off-line handwriting recognition. In Proc. of the 5th Int. Conf. on Document Analysis and Recognition, Bangalore, India, pages 705–708, 1999.Google Scholar
  16. 16.
    U.-V. Marti and H. Bunke. Using a statistical language model to improve the performance of an HMM-based cursive handwriting recognition system. Int. Journal of Pattern Recognition and Art. Intelligence, 15:65–90, 2001.CrossRefGoogle Scholar
  17. 17.
    D. Partridge and W. B. Yates. Engineering multiversion neural-net systems. Neural Computation, 8(4):869–893, 1996.CrossRefGoogle Scholar
  18. 18.
    L. Rabiner. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257–285, 1989.CrossRefGoogle Scholar
  19. 19.
    J.-C. Simon. Off-line cursive word recognition. Special Issue of Proc. of the IEEE, 80(7):1150–1161, July 1992.Google Scholar
  20. 20.
    C. Suen and L. Lam. Multiple classifier combination methodologies for different output level. In F. Roli, editors. First International Workshop on Multiple Classifier Systems, Cagliari, Italy, 2000. Springer. [12]}, pages 52–66.Google Scholar
  21. 21.
    C.Y. Suen, C. Nadal, R. Legault, T.A. Mai, and L. Lam. Computer recognition of unconstrained handwritten numerals. Special Issue of Proc. of the IEEE, 80(7):1162–1180, 1992.Google Scholar
  22. 22.
    L. Xu, A. Krzyzak, and C. Suen. Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Transactions on Systems, Man and Cybernetics, 22(3):418–435, 1992.CrossRefGoogle Scholar
  23. 23.
    S. J. Young, J. Jansen, J. J. Odell, D. Ollason, and P. C. Woodland. The HTK Hidden Markov Model Toolkit Book. Entropic Cambridge Research Laboratory, http://htk.eng.cam.ac.uk/, 1995.
  24. 24.
    M. Zimmermann and H. Bunke. Hidden markov model length optimization for handwriting recognition systems. Technical Report IAM-01-003, Department of Computer Science, University of Bern, 2001.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Simon Günter
    • 1
  • Horst Bunke
    • 1
  1. 1.Department of Computer ScienceUniversity of BernBernSwitzerland

Personalised recommendations