Ensembles of Classifiers for Handwritten Word Recognition Specialized on Individual Handwriting Style

  • Simon Günter
  • Horst Bunke
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3163)


The study of multiple classifier systems has become an area of intensive research in pattern recognition recently. Also in handwriting recognition, systems combining several classifiers have been investigated. Recently, new methods for the generation of multiple classifier systems, called ensemble methods, have been proposed in the field of machine learning, which generate an ensemble of classifiers from a single classifier automatically. In this paper a new ensemble method is proposed. It is characterized by training each individual classifier on a particular writing style. The new ensemble method is tested on two large scale handwritten word recognition tasks.


handwritten word recognition ensemble method writing style hidden Markov model (HMM) 


  1. 1.
    Proc of the 7th Int. Conf. on Document Analysis and Recognition, Edinburgh, Scotland (2003)Google Scholar
  2. 2.
    Brakensiek, A., Rottland, J., Kosmala, A., Rigoll, G.: Off-line handwriting recognition using various hybrid modeling techniques and character n-grams. In: 7th International Workshop on Frontiers in Handwritten Recognition, pp. 343–352 (2000)Google Scholar
  3. 3.
    Breiman, L.: Bagging predictors. Machine Learning (2), 123–140 (1996)Google Scholar
  4. 4.
    Bunke, H., Wang, P. (eds.): Handbook of Character Recognition and Document Image Analysis. World Scientific, Singapore (1997)Google Scholar
  5. 5.
    Dietterich, T.G.: Ensemble methods in machine learning. In: [19], pp. 1–15Google Scholar
  6. 6.
    Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via errorcorrecting output codes. Journal of Artifical Intelligence Research 2, 263–286 (1995)zbMATHGoogle Scholar
  7. 7.
    Dietterich, T.G., Kong, E.B.: Machine learning bias, statistical bias, and statistical variance of decision tree algorithms. Technical report, Departement of Computer Science, Oregon State University (1995)Google Scholar
  8. 8.
    Freund, Y., Schapire, R.E.: A decision-theoretic generalisation of online learning and an application to boosting. Journal of Computer and Systems Sciences 55(1), 119–139 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Günter, S., Bunke, H.: Optimizing the number of states, training iterations and Gaussians in an HMM-based handwritten word recognizer. In: [1], vol. 1, pp. 472–476Google Scholar
  10. 10.
    Günter, S., Bunke, H.: Creation of classifier ensembles for handwritten word recognition using feature selection algorithms. In: Proc. of the 8th International Conference on Frontiers in Handwriting Recognition, Niagara-onthe-Lake, Canada, pp. 183–188 (2002)Google Scholar
  11. 11.
    Günter, S., Bunke, H.: A new combination scheme for HMM-based classifiers and its application to handwriting recognition. In: 16th International Conference on Pattern Recognition, Quebec, Canada, vol. 2, pp. 332–337 (2002)Google Scholar
  12. 12.
    Günter, S., Bunke, H.: Ensembles of classifiers for handwritten word recognition. Int. Journal on Document Analysis and Recognition 5(4), 224–232 (2003)CrossRefGoogle Scholar
  13. 13.
    Hertel, C., Bunke, H.: A set of novel features for writer identification. In: Proc. of the 4th Int. Conference on Audio-and Video-Based Biometric Person Authentication AVBPA, Guildford, UK (2003)Google Scholar
  14. 14.
    Ho, T.K.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)CrossRefGoogle Scholar
  15. 15.
    Impedovo, S., Wang, P., Bunke, H. (eds.): Automatic Bankcheck Processing. World Scientific Publ. Co, Singapore (1997)zbMATHGoogle Scholar
  16. 16.
    Jain, A., Murty, M., Flynn, P.: Data clustering: A survey. ACM Computing Survey 31, 264–323 (1999)CrossRefGoogle Scholar
  17. 17.
    Kaltenmeier, A., Caesar, T., Gloger, J.M., Mandler, E.: Sophisticated topology of hidden Markov models for cursive script recognition. In: Proc. of the 2nd Int. Conf. on Document Analysis and Recognition, Tsukuba Science City, Japan, pp. 139–142 (1993)Google Scholar
  18. 18.
    Kim, G., Govindaraju, V., Srihari, S.N.: Architecture for handwritten text recognition systems. In: Lee, S.-W. (ed.) Advances in Handwriting Recognition, pp. 163–172. World Scientific Publ. Co., Singapore (1999)CrossRefGoogle Scholar
  19. 19.
    Kittler, J., Roli, F. (eds.): First International Workshop on Multiple Classifier Systems. Springer, Heidelberg (2000)Google Scholar
  20. 20.
    Kittler, J., Roli, F. (eds.): Third International Workshop on Multiple Classifier Systems, Cagliari, Italy. Springer, Heidelberg (2002)Google Scholar
  21. 21.
    Kundu, A.: Handwritten word recognition using hidden Markov model. In: [4], pp. 157–182Google Scholar
  22. 22.
    Lee, D., Srihari, S.: Handprinted digit recognition: A comparison of algorithms. In: Third International Workshop on Frontiers in Handwriting Recognition, pp. 153–162 (1993)Google Scholar
  23. 23.
    Marti, U., Bunke, H.: The IAM-database: An English sentence database for offline handwriting recognition. Int. Journal of Document Analysis and Recognition 5, 39–46 (2002)zbMATHCrossRefGoogle Scholar
  24. 24.
    Marti, U.-V., Bunke, H.: Using a statistical language model to improve the performance of an HMM-based cursive handwriting recognition system. Int. Journal of Pattern Recognition and Art. Intelligence 15, 65–90 (2001)CrossRefGoogle Scholar
  25. 25.
    Nishimura, H., Kobayashi, M., Maruyama, M., Nakano, Y.: Off-line character recognition using HMM by multiple directional feature extraction and voting with bagging algorithm. In: 5th International Conference on Document Analysis and Recognition, Bangalore, India, pp. 49–52 (1999)Google Scholar
  26. 26.
    Partridge, D., Yates, W.B.: Engineering multiversion neural-net systems. Neural Computation 8(4), 869–893 (1996)CrossRefGoogle Scholar
  27. 27.
    Rabiner, L., Juang, B.-H.: Fundamentals of Speech Recognition. Prentice-Hall, Englewood Cliffs (1993)Google Scholar
  28. 28.
    Simon, J.-C.: Off-line cursive word recognition. Special Issue of Proc. of the IEEE 80(7), 1150–1161 (1992)Google Scholar
  29. 29.
    Suen, C.Y., Nadal, C., Legault, R., Mai, T.A., Lam, L.: Computer recognition of unconstrained handwritten numerals. Special Issue of Proc. of the IEEE 80(7), 1162–1180 (1992)Google Scholar
  30. 30.
    Windeatt, T., Roli, F. (eds.): 4th Int. Workshop on Multiple Classifier Systems, Guildford, United Kingdom. Springer, Heidelberg (2003)Google Scholar
  31. 31.
    Xu, L., Krzyzak, A., Suen, C.: Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Transactions on Systems, Man and Cybernetics 22(3), 418–435 (1992)CrossRefGoogle Scholar
  32. 32.
    Young, S.J., Jansen, J., Odell, J.J., Ollason, D., Woodland, P.C.: The HTK Hidden Markov Model Toolkit Book. Entropic Cambridge Research Laboratory (1995),
  33. 33.
    Zimmermann, M., Bunke, H.: Automatic segmentation of the IAM off-line database for handwritten English text. In: Proc. of 16th Int. Conference on Pattern Recognition, Quebec, Canada, vol. 4, pp. 35–39 (2002)Google Scholar
  34. 34.
    Zimmermann, M., Bunke, H.: Automatic segmentation of the IAM off-line database for handwritten English text. In: Proc. of the 16th Int. Conference on Pattern Recognition, Quebec, Canada, vol. 4, pp. 35–39 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Simon Günter
    • 1
  • Horst Bunke
    • 1
  1. 1.Department of Computer ScienceUniversity of BernBernSwitzerland

Personalised recommendations