Advertisement

Using Word Mover’s Distance with Spatial Constraints for Measuring Similarity Between Mongolian Word Images

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10637)

Abstract

In the framework of bag-of-visual-words, visual words are independent each other, which results in discarding spatial relations and lacking semantic information of visual words. To capture semantic information of visual words, a deep learning procedure similar to word embedding technique is used for mapping visual words to embedding vectors in a semantic space. And then, word mover’s distance (WMD) is utilized to measure similarity between two word images, which calculates the minimum traveling distance from the visual embeddings of one word image to another one. Moreover, word images are partitioned into several sub-regions with equal sizes along rows and columns in advance. After that, WMDs can be computed from the corresponding sub-regions of the two word images, separately. Thus, the similarity between the two word images is the sum of these WMDs. Experimental results show that the proposed method outperforms various baseline and state-of-the-art methods, including spatial pyramid matching, latent Dirichlet allocation, average visual word embeddings and the original word mover’s distance.

Keywords

Visual word embeddings Word mover’s distance Spatial information Keyword spotting Query-by-example 

Notes

Acknowledgement

This paper is supported by the National Natural Science Foundation of China under Grant 61463038.

References

  1. 1.
    Rath, T.M., Manmatha, R.: Word spotting for historical manuscripts. Int. J. Doc. Anal. Recogn. 9(2), 139–152 (2007)CrossRefGoogle Scholar
  2. 2.
    Rath, T.M., Manmatha, R.: Features for word spotting in historical manuscripts. In: Proceedings of ICDAR 2003, pp. 218–222. IEEE Press, New York (2003)Google Scholar
  3. 3.
    Rath, T.M., Manmatha, R.: Word image matching using dynamic time warping. In: Proceedings of CVPR 2003, pp. 521–527. IEEE Press, New York (2003)Google Scholar
  4. 4.
    Shekhar, R., Jawahar, C.V.: Word image retrieval using bag of visual words. In: Proceedings of DAS 2012, pp. 297–301. IEEE Press, New York (2012)Google Scholar
  5. 5.
    Aldavert, D., Rusinol, M., Toledo, R., Llados, J.: A study of bag-of-visual-words representations for handwritten keyword spotting. Int. J. Doc. Anal. Recogn. 18(3), 223–234 (2015)CrossRefGoogle Scholar
  6. 6.
    Mikolov, T., Sutskever, I., Chen, K., Coorado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of NIPS 2013, pp. 3111–3119. MIT Press, Massachusetts (2013)Google Scholar
  7. 7.
    Kusner, M.J., Sun, Y., Kolkin, N.I., Weinberger, K.Q.: From word embeddings to document distances. Proc. Mach. Learn. Res. 37, 957–966 (2015)Google Scholar
  8. 8.
    Fornes, A., Frinken, V., Fischer, A., Almazan, J., Jackson, G., Bunke, H.: A keyword spotting approach using blurred shape model-based descriptors. In: Proceedings of HIP 2011, pp. 83–89. ACM Press, New York (2011)Google Scholar
  9. 9.
    Aldavert, D., Rusinol, M., Toledo, R., Llados, J.: Integrating visual and textual cues for query-by-string word spotting. In: Proceedings of ICDAR 2013, pp. 511–515. IEEE Press, New York (2013)Google Scholar
  10. 10.
    Rothacker, L., Fink, G.A.: Segmentation-free query-by-string word spotting with bag-of-features HMMs. In: Proceedings of ICDAR 2015, pp. 661–665. IEEE Press, New York (2015)Google Scholar
  11. 11.
    Wei, H.X., Gao, G.L., Su, X.D.: A multiple instances approach to improving keyword spotting on historical Mongolian document images. In: Proceedings of ICDAR 2015, pp. 121–125. IEEE Press, New York (2015)Google Scholar
  12. 12.
    Wei, H.X., Zhang, H., Gao, G.L.: Representing word image using visual word embeddings and RNN for keyword spotting on historical document images. In: Proceedings of ICME 2017, pp. 1374–1379. IEEE Press, New York (2017)Google Scholar
  13. 13.
    Wei, H.X., Gao, G.L.: Visual language model for keyword spotting on historical Mongolian document images. In: Proceedings of CCDC 2017, pp. 1765–1770. IEEE Press, New York (2017)Google Scholar
  14. 14.
    Wei, H., Gao, G., Su, X.: LDA-based word image representation for keyword spotting on historical Mongolian documents. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds.) ICONIP 2016. LNCS, vol. 9950, pp. 432–441. Springer, Cham (2016). doi: 10.1007/978-3-319-46681-1_52 CrossRefGoogle Scholar
  15. 15.
    Zamani, H., Croft, W.B.: Embeddings-based query language models. In: Proceedings of ICTIR 2016, pp. 147–156. ACM Press, New York (2016)Google Scholar
  16. 16.
    Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Proceedings of EMNLP 2014, pp. 1532–1543. ACL Press, Stroudsburg (2014)Google Scholar
  17. 17.
    Nalisnick, E., Mitra, B., Craswell, N., Caruana, R.: Improving document ranking with dual word embeddings. In: Proceedings of WWW 2016, pp. 83–84. ACM Press, New York (2016)Google Scholar
  18. 18.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of CVPR 2006, pp. 2169–2178. IEEE Press, New York (2006)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Hongxi Wei
    • 1
  • Hui Zhang
    • 1
  • Guanglai Gao
    • 1
  • Xiangdong Su
    • 1
  1. 1.School of Computer ScienceInner Mongolia UniversityHohhotChina

Personalised recommendations