Hybrid HMM/ANN Models for Bimodal Online and Offline Cursive Word Recognition

  • S. España-Boquera
  • J. Gorbe-Moya
  • F. Zamora-Martínez
  • M. J. Castro-Bleda
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6388)


The recognition performance of current automatic offline handwriting transcription systems is far from being perfect. This is the reason why there is a growing interest in assisted transcription systems, which are more efficient than correcting by hand an automatic transcription. A recent approach to interactive transcription involves multi-modal recognition, where the user can supply an online transcription of some of the words. In this paper, a description of the bimodal engine, which entered the “Bi-modal Handwritten Text Recognition” contest organized during the 2010 ICPR, is presented. The proposed recognition system uses Hidden Markov Models hybridized with neural networks (HMM/ANN) for both offline and online input. The N-best word hypothesis scores for both the offline and the online samples are combined using a log-linear combination, achieving very satisfying results.


Recognition System Handwriting Recognition Slant Angle Online Sample Handwritten Text 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Vilar, J.M., Castro-Bleda, M.J., Zamora-Martínez, F., España-Boquera, S., Gordo, A., Llorens, D., Marzal, A., Prat, F., Gorbe, J.: A Flexible System for Document Processing and Text Transcription. In: Meseguer, P. (ed.) CAEPIA 2009. LNCS (LNAI), vol. 5988, pp. 291–300. Springer, Heidelberg (2010)Google Scholar
  2. 2.
    Pastor, M., Vidal, E., Casacuberta, F.: A bi-modal handwritten text corpus. Technical report, Instituto Tecnológico de Informática, Universidad Politécnica de Valencia, Spain (2009)Google Scholar
  3. 3.
    Marti, U.V., Bunke, H.: The IAM-database: an English sentence database for offline handwriting recognition. International Journal of Document Analysis and Recognition 5, 39–46 (2002)CrossRefzbMATHGoogle Scholar
  4. 4.
    Liwicki, M., Bunke, H.: IAM-OnDB - an on-line English sentence database acquired from handwritten text on a whiteboard. In: International Conference on Document Analysis and Recognition, pp. 956–961 (2005)Google Scholar
  5. 5.
    Gorbe-Moya, J., España-Boquera, S., Zamora-Martínez, F.: Castro-Bleda, M.J.: Handwritten Text Normalization by using Local Extrema Classification. In: Proc. 8th International Workshop on Pattern Recognition in Information Systems, Barcelona, Spain, pp. 164–172. Insticc Press (June 2008)Google Scholar
  6. 6.
    España-Boquera, S., Castro-Bleda, M.J., Gorbe-Moya, J., Zamora-Martínez, F.: Improving Offline Handwritten Text Recognition with Hybrid HMM/ANN Models. IEEE Trans. Pattern Anal. Mach. Intell. (August 2010) (preprints), doi: 10.1109/TPAMI.2010.141Google Scholar
  7. 7.
    Toselli, A.H., Juan, A., González, J., Salvador, I., Vidal, E., Casacuberta, F., Keysers, D., Ney, H.: Integrated Handwriting Recognition and Interpretation using Finite-State Models. International Journal of Pattern Recognition and Artificial Intelligence 18(4), 519–539 (2004)CrossRefGoogle Scholar
  8. 8.
    España-Boquera, S., Zamora-Martínez, F., Castro-Bleda, M.J., Gorbe-Moya, J.: Efficient BP Algorithms for General Feedforward Neural Networks. In: Mira, J., Álvarez, J.R. (eds.) IWINAC 2007. LNCS, vol. 4527, pp. 327–336. Springer, Heidelberg (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • S. España-Boquera
    • 1
  • J. Gorbe-Moya
    • 1
  • F. Zamora-Martínez
    • 2
  • M. J. Castro-Bleda
    • 1
  1. 1.Departamento de Sistemas Informáticos y ComputaciónUniversidad Politécnica de ValenciaValenciaSpain
  2. 2.Departamento de Ciencias Físicas, Matemáticas y de la ComputaciónUniversidad CEU-Cardenal HerreraValenciaSpain

Personalised recommendations