Evaluation of Classical and Novel Ensemble Methods for Handwritten Word Recognition

  • Simon Günter
  • Horst Bunke
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3138)

Abstract

Handwritten text recognition is one of the most difficult problems in the field of pattern recognition. Recently, a number of classifier creation and combination methods, known as ensemble methods, have been proposed in the field of machine learning. They have shown improved recognition performance over single classifiers. In this paper a number of ensemble methods for handwritten word recognition are described, experimentally evaluated, and compared to each other. Those methods include classical, general purpose ensemble methods as well as novel ensemble methods specifically developed by the authors for handwritten word recognition. The aim of the paper is to investigate the potential of ensemble methods for improving the performance of handwriting recognition systems. The base recognition systems used in this paper are hidden Markov model classifiers.

Keywords

ensemble methods handwritten word recognition hidden Markov model (HMM) 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Proc of the 7th Int. Conf. on Document Analysis and Recognitio, Edinbourgh, Scotland (2003)Google Scholar
  2. 2.
    Breiman, L.: Bagging predictors. Machine Learning (2), 123–140 (1996)Google Scholar
  3. 3.
    Dietterich, T.G.: Ensemble methods in machine learning. In: [17], pp. 1–15Google Scholar
  4. 4.
    Dietterich, T.G., Kong, E.B.: Machine learning bias, statistical bias, and statistical variance of decision tree algorithms. Technical report, Departement of Computer Science, Oregon State University (1995)Google Scholar
  5. 5.
    Freund, Y., Schapire, R.E.: A decision-theoretic generalisation of online learning and an application to boosting. Journal of Computer and Systems Sciences 55(1), 119–139 (1997)MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Günter, S., Bunke, H.: Generating classifier ensembles from multiple prototypes and its application to handwriting recognition. In: [18], pp. 179–188Google Scholar
  7. 7.
    Günter, S., Bunke, H.: New boosting algorithms for classification problems with large number of classes applied to a handwritten word recognition task. In: [27], pp. 326–335Google Scholar
  8. 8.
    Günter, S., Bunke, H.: Optimizing the number of states, training iterations and Gaussians in an HMM-based handwritten word recognizer. In: [1], vol. 1, pp. 472–476Google Scholar
  9. 9.
    Günter, S., Bunke, H.: Creation of classifier ensembles for handwritten word recognition using feature selection algorithms. In: Proc. of the 8th International Conference on Frontiers in Handwriting Recognition, pp. 183–188 (2002)Google Scholar
  10. 10.
    Günter, S., Bunke, H.: A new combination scheme for HMM-based classifiers and its application to handwriting recognition. In: 16th International Conference on Pattern Recognition, Quebec, Canada, vol. 2, pp. 332–337 (2002)Google Scholar
  11. 11.
    Günter, S., Bunke, H.: Ensembles of classifiers for handwritten word recognition. International Journal of Document Analysis and Recognition 5(4), 224–232 (2003)CrossRefGoogle Scholar
  12. 12.
    Ho, T.K.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)CrossRefGoogle Scholar
  13. 13.
    Impedovo, S., Wang, P., Bunke, H. (eds.): Automatic Bankcheck Processing. World Scientific Publ. Co., Singapore (1997)MATHGoogle Scholar
  14. 14.
    Jain, A., Murty, M., Flynn, P.: Data clustering: A survey. ACM Computing Survey 31, 264–323 (1999)CrossRefGoogle Scholar
  15. 15.
    Kaltenmeier, A., Caesar, T., Gloger, J.M., Mandler, E.: Sophisticated topology of hidden Markov models for cursive script recognition. In: Proc. of the 2nd Int. Conf. on Document Analysis and Recognition, Tsukuba Science City, Japan, pp. 139–142 (1993)Google Scholar
  16. 16.
    Kim, G., Govindaraju, V., Srihari, S.N.: Architecture for handwritten text recognition systems. In: Lee, S.-W. (ed.) Advances in Handwriting Recognition, pp. 163–172. World Scientific Publ. Co., Singapore (1999)CrossRefGoogle Scholar
  17. 17.
    In: Kittler, J., Roli, F. (eds.) First International Workshop on Multiple Classifier Systems, Cagliari, Italy, Springer, Heidelberg (2000)Google Scholar
  18. 18.
    In: Kittler, J., Roli, F. (eds.) Third International Workshop on Multiple Classifier Systems, Cagliari, Italy, Springer, Heidelberg (2002)Google Scholar
  19. 19.
    Lam, L., Huang, Y.-S., Suen, C.: Combination of multiple classifier decisions for optical character recognition. In: Bunke, H., Wang, P. (eds.) Handbook of Character Recognition and Document Image Analysis, pp. 79–101. World Scientific, Singapore (1997)Google Scholar
  20. 20.
    Lee, D., Srihari, S.: Handprinted digit recognition: A comparison of algorithms. In: Third International Workshop on Frontiers in Handwriting Recognition, pp. 153–162 (1993)Google Scholar
  21. 21.
    Marti, U., Bunke, H.: The IAM-database: An English sentence database for offline handwriting recognition. Int. Journal of Document Analysis and Recognition 5, 39–46 (2002)MATHCrossRefGoogle Scholar
  22. 22.
    Marti, U.-V., Bunke, H.: Using a statistical language model to improve the performance of an HMM-based cursive handwriting recognition system. Int. Journal of Pattern Recognition and Art. Intelligence 15, 65–90 (2001)CrossRefGoogle Scholar
  23. 23.
    Partridge, D., Yates, W.B.: Engineering multiversion neural-net systems. Neural Computation 8(4), 869–893 (1996)CrossRefGoogle Scholar
  24. 24.
    Rabiner, L.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–285 (1989)CrossRefGoogle Scholar
  25. 25.
    Simon, J.-C.: Off-line cursive word recognition. Special Issue of Proc. of the IEEE 80(7), 1150–1161 (1992)Google Scholar
  26. 26.
    Suen, C.Y., Nadal, C., Legault, R., Mai, T.A., Lam, L.: Computer recognition of unconstrained handwritten numerals. Special Issue of Proc. of the IEEE 80(7), 1162–1180 (1992)Google Scholar
  27. 27.
    In: Windeatt, T., Roli, F. (eds.) 4th Int. Workshop on Multiple Classifier Systems, Guildford, United Kingdom, Springer, Heidelberg (2003)Google Scholar
  28. 28.
    Xu, L., Krzyzak, A., Suen, C.: Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Transactions on Systems, Man and Cybernetics 22(3), 418–435 (1992)CrossRefGoogle Scholar
  29. 29.
    Zimmermann, M., Bunke, H.: Automatic segmentation of the IAM off-line database for handwritten English text. In: Proc. of the 16th Int. Conference on Pattern Recognition, Quebec, Canada, vol. 4, pp. 35–39 (2002)Google Scholar
  30. 30.
    Zimmermann, M., Bunke, H.: Hidden markov model length optimization for handwriting recognition systems. In: Proc. of the 8th International Workshop on Frontiers in Handwriting Recognition, pp. 369–374 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Simon Günter
    • 1
  • Horst Bunke
    • 1
  1. 1.Department of Computer ScienceUniversity of BernBernSwitzerland

Personalised recommendations