Ensembles of Classifiers Derived from Multiple Prototypes and Their Application to Handwriting Recognition

  • Simon Günter
  • Horst Bunke
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3077)


There are many examples of classification problems in the literature where multiple classifier systems increase the performance over single classifiers. Normally one of the following two approaches is used to create a multiple classifier system: 1. Several classifiers are developed completely independent of each other and combined in a last step. 2. Several classifiers are created out of one prototype classifier by using so called classifier ensemble methods. In this paper a novel algorithm which combines both approaches is introduced. This new algorithm is experimentally evaluated in the context of hidden Markov model (HMM) based handwritten word recognizers and compared to previously introduced methods which also combine both approaches.


Handwriting Recognition Hidden Markov Model (HMM) Multiple Classifier System Ensemble Method 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Proc of the 7th Int. Conf. on Document Analysis and Recognition, Edinburgh, Scotland (2003)Google Scholar
  2. 2.
    Brakensiek, J., Rottland, A.: Off-line handwriting recognition using various hybrid modeling techniques and character n-grams. In: 7th International Workshop on Frontiers in Handwritten Recognition, pp. 343–352 (2000)Google Scholar
  3. 3.
    Breiman, L.: Bagging predictors. Machine Learning (2), 123–140 (1996)Google Scholar
  4. 4.
    Dietterich, T.G.: Ensemble methods in machine learning. In: [17], pp. 1–15Google Scholar
  5. 5.
    Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via errorcorrecting output codes. Journal of Artifical Intelligence Research 2, 263–286 (1995)zbMATHGoogle Scholar
  6. 6.
    Dietterich, T.G., Kong, E.B.: Machine learning bias, statistical bias, and statistical variance of decision tree algorithms. Technical report, Departement of Computer Science, Oregon State University (1995)Google Scholar
  7. 7.
    Freund, Y., Schapire, R.E.: A decision-theoretic generalisation of online learning and an application to boosting. Journal of Computer and Systems Sciences 55(1), 119–139 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Günter, S., Bunke, H.: Generating classifier ensembles from multiple prototypes and its application to handwriting recognition. In: [18], pp. 179–188Google Scholar
  9. 9.
    Günter, S., Bunke, H.: New boosting algorithms for classification problems with large number of classes applied to a handwritten word recognition task. In: [28], pp. 326–335Google Scholar
  10. 10.
    Günter, S., Bunke, H.: Optimizing the number of states, training iterations and Gaussians in an HMM-based handwritten word recognizer. In: [1], vol. 1, pp. 472–476Google Scholar
  11. 11.
    Ho, T.K.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)CrossRefGoogle Scholar
  12. 12.
    Hull, D.: Using statistical testing in the evaluation of retrieval experiments. In: Research and Development in Information Retrieval, pp. 329–338 (1993)Google Scholar
  13. 13.
    Impedovo, S., Wang, P., Bunke, H. (eds.): Automatic Bankcheck Processing. World Scientific Publ. Co., Singapore (1997)zbMATHGoogle Scholar
  14. 14.
    Kaltenmeier, A., Caesar, T., Gloger, J.M., Mandler, E.: Sophisticated topology of hidden Markov models for cursive script recognition. In: Proc. of the 2nd Int. Conf. on Document Analysis and Recognition, Tsukuba Science City, Japan, pp. 139–142 (1993)Google Scholar
  15. 15.
    Kim, G., Govindaraju, V., Srihari, S.N.: Architecture for handwritten text recognition systems. In: Lee, S.-W. (ed.) Advances in Handwriting Recognition, pp. 163–172. World Scientific Publ. Co., Singapore (1999)CrossRefGoogle Scholar
  16. 16.
    Kirby, M.: Geometric Data Analysis: An Empirical Approach to Dimensionality Reduction and the Study of Patterns. John Wiley and Sons, New York (2001)zbMATHGoogle Scholar
  17. 17.
    Kittler, J., Roli, F. (eds.): MCS 2000. LNCS, vol. 1857. Springer, Heidelberg (2000)Google Scholar
  18. 18.
    Roli, F., Kittler, J. (eds.): MCS 2002. LNCS, vol. 2364. Springer, Heidelberg (2002)zbMATHGoogle Scholar
  19. 19.
    Lee, D., Srihari, S.: Handprinted digit recognition: A comparison of algorithms. In: Third International Workshop on Frontiers in Handwriting Recognition, pp. 153–162 (1993)Google Scholar
  20. 20.
    Marti, U., Bunke, H.: The IAM-database: An English sentence database for offline handwriting recognition. Int. Journal of Document Analysis and Recognition 5, 39–46 (2002)zbMATHCrossRefGoogle Scholar
  21. 21.
    Marti, U.-V., Bunke, H.: Using a statistical language model to improve the performance of an HMM-based cursive handwriting recognition system. Int. Journal of Pattern Recognition and Art. Intelligence 15, 65–90 (2001)CrossRefGoogle Scholar
  22. 22.
    Partridge, D., Yates, W.B.: Engineering multiversion neural-net systems. Neural Computation 8(4), 869–893 (1996)CrossRefGoogle Scholar
  23. 23.
    Rabiner, L.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–285 (1989)CrossRefGoogle Scholar
  24. 24.
    Simon, J.-C.: Off-line cursive word recognition. Special Issue of Proc. of the IEEE 80(7), 1150–1161 (1992)Google Scholar
  25. 25.
    Suen, C.Y., Nadal, C., Legault, R., Mai, T.A., Lam, L.: Computer recognition of unconstrained handwritten numerals. Special Issue of Proc. of the IEEE 80(7), 1162–1180 (1992)Google Scholar
  26. 26.
    A. Vinciarelli. Offline Cursive Handwriting: From Word to Text Recognition. PhD thesis, University of Bern, Switzerland, 2003. Google Scholar
  27. 27.
    Vinciarelli, A., Bengio, S., Bunke, H.: Offline recognition of large vocabulary cursive handwritten text. In: [1], vol. 2, pp. 1101–1105Google Scholar
  28. 28.
    Windeatt, T., Roli, F. (eds.): MCS 2003. LNCS, vol. 2709. Springer, Heidelberg (2003)zbMATHGoogle Scholar
  29. 29.
    Xu, L., Krzyzak, A., Suen, C.: Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Transactions on Systems, Man and Cybernetics 22(3), 418–435 (1992)CrossRefGoogle Scholar
  30. 30.
    Young, S.J., Jansen, J., Odell, J.J., Ollason, D., Woodland, P.C.: The HTK Hidden Markov Model Toolkit Book. Entropic Cambridge Research Laboratory (1995),
  31. 31.
    Zimmermann, M., Bunke, H.: Hidden Markov model length optimization for handwriting recognition systems. In: Proc. of the 8th International Workshop on Frontiers in Handwriting Recognition, pp. 369–374 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Simon Günter
    • 1
  • Horst Bunke
    • 1
  1. 1.Department of Computer ScienceUniversity of BernBernSwitzerland

Personalised recommendations