Advertisement

Keyword Spotting with Convolutional Deep Belief Networks and Dynamic Time Warping

  • Baptiste Wicht
  • Andreas Fischer
  • Jean Hennebert
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9887)

Abstract

To spot keywords on handwritten documents, we present a hybrid keyword spotting system, based on features extracted with Convolutional Deep Belief Networks and using Dynamic Time Warping for word scoring. Features are learned from word images, in an unsupervised manner, using a sliding window to extract horizontal patches. For two single writer historical data sets, it is shown that the proposed learned feature extractor outperforms two standard sets of features.

Keywords

Document Image Dynamic Time Warping Mean Average Precision Template Image Word Image 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Chen, F.R., Wilcox, L.U., Bloomberg, D.S.: Word spotting in scanned images using Hidden Markov Models. In: Proceedings of the IEEE International Conference on Acoustics Speech and Signal Processing, vol. 5, pp. 1–4. IEEE (1993)Google Scholar
  2. 2.
    Choisy, C.: Dynamic handwritten keyword spotting based on the NSHP-HMM. In: Proceedings of the IEEE International Conference on Document Analysis and Recognition, vol. 1, pp. 242–246. IEEE (2007)Google Scholar
  3. 3.
    Fischer, A., Keller, A., Frinken, V., Bunke, H.: Lexicon-free handwritten word spotting using character HMMs. Pattern Recogn. Lett. 33, 934–942 (2012)CrossRefGoogle Scholar
  4. 4.
    Fischer, A., Wüthrich, M., Liwicki, M., Frinken, V., Bunke, H., Viehhauser, G., Stolz, M.: Automatic transcription of handwritten medieval documents. In: Proceedings of the International Conference on Virtual Systems and Multimedia, pp. 137–142. IEEE (2009)Google Scholar
  5. 5.
    Forsyth, D., Jaety, E., Teh, Y.W., Maire, M., Bock, R.B., Vesom, G.: Making latin manuscripts searchable using gHMMs. In: Proceedings of the Advances in Neural Information Processing Systems, vol. 17, p. 385. MIT Press (2005)Google Scholar
  6. 6.
    Frinken, V., Fischer, A., Manmatha, R., Bunke, H.: A novel word spotting method based on recurrent neural networks. IEEE Trans. Pattern Anal. Mach. Intell. 34, 211–224 (2012)CrossRefGoogle Scholar
  7. 7.
    Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Comput. 14, 1771–1800 (2002)CrossRefzbMATHGoogle Scholar
  8. 8.
    Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Honglak, L., Chaitanya, E., Ng, A.Y.: Sparse deep belief net model for visual area V2. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 873–880 (2008)Google Scholar
  10. 10.
    Kuo, S.S., Agazzi, O.E.: Keyword spotting in poorly printed documents using pseudo 2-D Hidden Markov Models. IEEE Trans. Pattern Anal. Mach. Intell. 16, 842–848 (1994)CrossRefGoogle Scholar
  11. 11.
    Lavrenko, V., Rath, T.M., Manmatha, R.: Holistic word recognition for handwritten historical documents. In: Proceedings of the International Workshop on Document Image Analysis for Libraries, pp. 278–287. IEEE (2004)Google Scholar
  12. 12.
    Lee, H., Grosse, R., Ranganath, R., Ng, A.Y.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Proceedings of the International Conference on Machine Learning, pp. 609–616. ACM (2009)Google Scholar
  13. 13.
    Manmatha, R., Croft, W.: Word spotting: indexing handwritten archives. In: Intelligent Multimedia Information Retrieval Collection, pp. 43–64 (1997)Google Scholar
  14. 14.
    Manmatha, R., Han, C., Riseman, E.M.: Word spotting: a new approach to indexing handwriting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 631–637. IEEE (1996)Google Scholar
  15. 15.
    Marti, U.V., Bunke, H.: Using a statistical language model to improve the performance of an HMM-based cursive handwriting recognition system. Int. J. Pattern Recogn. Artif. Intell. 15, 65–90 (2001)CrossRefGoogle Scholar
  16. 16.
    Myers, C., Rabiner, L., Rosenberg, A.: An investigation of the use of dynamic time warping for word spotting and connected speech recognition. In: Proceedings of the IEEE International Conference on Acoustics Speech and Signal Processing, vol. 5, pp. 173–177. IEEE (1980)Google Scholar
  17. 17.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the International Conference on Machine Learning, pp. 807–814 (2010)Google Scholar
  18. 18.
    Rath, T.M., Manmatha, R.: Word image matching using dynamic time warping. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 521–527. IEEE (2003)Google Scholar
  19. 19.
    Rath, T.M., Manmatha, R.: Word spotting for historical documents. Int. J. Doc. Anal. Recogn. (IJDAR) 9, 139–152 (2007)CrossRefGoogle Scholar
  20. 20.
    Rodrıguez, J.A., Perronnin, F.: Local gradient histogram features for word spotting in unconstrained handwritten documents. In: Proceedings of the International Conference on Frontiers in Handwriting Recognition, pp. 7–12 (2008)Google Scholar
  21. 21.
    Rose, R.C., Paul, D.B.: A Hidden Markov Model based keyword recognition system. In: Proceedings of the International Conference on Acoustics Speech, and Signal Processing, pp. 129–132. IEEE (1990)Google Scholar
  22. 22.
    Sakoe, H., Chiba, S.: Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. Acoust. Speech Signal Process. 26, 43–49 (1978)CrossRefzbMATHGoogle Scholar
  23. 23.
    Vinciarelli, A.: A survey on off-line cursive word recognition. Pattern Recogn. 35, 1433–1446 (2002)CrossRefzbMATHGoogle Scholar
  24. 24.
    Wicht, B., Hennebert, J.: Camera-based Sudoku recognition with deep belief network. In: Proceedings the of IEEE International Conference of Soft Computing and Pattern Recognition, pp. 83–88. IEEE (2014)Google Scholar
  25. 25.
    Wicht, B., Hennebert, J.: Mixed handwritten and printed digit recognition in Sudoku with convolutional deep belief network. In: Proceedings of the IEEE International Conference on Document Analysis and Recognition. IEEE (2015)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Baptiste Wicht
    • 1
    • 2
  • Andreas Fischer
    • 1
    • 2
  • Jean Hennebert
    • 1
    • 2
  1. 1.University of Applied Science of Western SwiterzlandDelémontSwitzerland
  2. 2.University of FribourgFribourgSwitzerland

Personalised recommendations