Transliteration Equivalence Using Canonical Correlation Analysis

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5993)


We address the problem of Transliteration Equivalence, i.e. determining whether a pair of words in two different languages (e.g. Auden, ऑडेन) are name transliterations or not. This problem is at the heart of Mining Name Transliterations (MINT) from various sources of multilingual text data including parallel, comparable, and non-comparable corpora and multilingual news streams. MINT is useful in several cross-language tasks including Cross-Language Information Retrieval (CLIR), Machine Translation (MT), and Cross-Language Named Entity Retrieval. We propose a novel approach to Transliteration Equivalence using language-neutral representations of names. The key idea is to consider name transliterations in two languages as two views of the same semantic object and compute a low-dimensional common feature space using Canonical Correlation Analysis (CCA). Similarity of the names in the common feature space forms the basis for classifying a pair of names as transliterations. We show that our approach outperforms state-of-the-art baselines in the CLIR task for Hindi-English (3 collections) and Tamil-English (2 collections).


Information Retrieval Cross-Language Information Retrieval Machine Translation Cross-Language Named Entity Retrieval Machine Transliteration Mining Canonical Correlation Analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barr, C., Jones, R., Regelson, M.: The linguistic structure of english web-search queries. In: Conference on Empirical Methods in Natural Language Processing, EMNLP (October 2008)Google Scholar
  2. 2.
    Guo, J., Xu, G., Cheng, X., Li, H.: Named entity recognition in query. In: SIGIR 2009: Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pp. 267–274. ACM, New York (2009)CrossRefGoogle Scholar
  3. 3.
    Al-Onaizan, Y., Knight, K.: Machine transliteration of names in arabic text. In: Proceedings of the AaCL 2002 workshop on Computational approaches to semitic languages, pp. 1–13 (2002)Google Scholar
  4. 4.
    Al-Onaizan, Y., Knight, K.: Translating named entities using monolingual and bilingual resources. In: ACL 2002: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 400–408 (2001)Google Scholar
  5. 5.
    Virga, P., Khudanpur, S.: Transliteration of proper names in cross-lingual information retrieval. In: Proceedings of the ACL 2003 workshop on Multilingual and mixed-language named entity recognition, Morristown, NJ, USA, pp. 57–64. Association for Computational Linguistics (2003)Google Scholar
  6. 6.
    Udupa, R., Saravanan, K., Bakalov, A., Bhole, A.: “They are out there, if you know where to look”: Mining transliterations of oov query terms for cross-language information retrieval. In: ECIR, pp. 437–448 (2009)Google Scholar
  7. 7.
    Udupa, R., Saravanan, K., Kumaran, A., Jagarlamudi, J.: Mint: A method for effective and scalable mining of named entity transliterations from large comparable corpora. In: EACL, pp. 799–807 (2009)Google Scholar
  8. 8.
    Kondrak, G.: Identifying cognates by phonetic and semantic similarity. In: NAACL 2001: Second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies 2001, Morristown, NJ, USA, pp. 1–8. Association for Computational Linguistics (2001)Google Scholar
  9. 9.
    Hardoon, D.R., Szedmák, S., Shawe-Taylor, J.: Canonical correlation analysis: An overview with application to learning methods. Neural Computation 16(12), 2639–2664 (2004)zbMATHCrossRefGoogle Scholar
  10. 10.
    Li, H., Sim, K.C., Kuo, J.S., Dong, M.: Semantic transliteration of personal names. In: ACL (2007)Google Scholar
  11. 11.
    Pirkola, A., Toivonen, J., Keskustalo, H., Järvelin, K.: Fite-trt: a high quality translation technique for oov words. In: SAC 2006: Proceedings of the, ACM Symposium on Applied computing, pp. 1043–1049. ACM, New York (2006)Google Scholar
  12. 12.
    Klementiev, A., Roth, D.: Named entity transliteration and discovery from multilingual comparable corpora. In: HLT-NAACL (2006)Google Scholar
  13. 13.
    Fung, P.: A pattern matching method for finding noun and proper noun translations from noisy parallel corpora. In: Proceedings of the 33rd annual meeting on Association for Computational Linguistics, Morristown, NJ, USA, pp. 236–243. Association for Computational Linguistics (1995)Google Scholar
  14. 14.
    Quirk, C., Udupa, R., Menezes, A.: Generative models of noisy translations with applications to parallel fragment extraction. In: Proc. of the MT Summit XI, pp. 321–327 (2007)Google Scholar
  15. 15.
    Rapp, R.: Automatic identification of word translations from unrelated english and german corpora. In: Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, Morristown, NJ, USA, pp. 519–526. Association for Computational Linguistics (1999)Google Scholar
  16. 16.
    Ballesteros, L., Croft, B.: Dictionary methods for cross-lingual information retrieval. In: Proceedings of the 7th International DEXA Conference on Database and Expert Systems Applications, pp. 791–801 (1996)Google Scholar
  17. 17.
    Mandl, T., Womser-Hacker, C.: How do named entities contribute to retrieval effectiveness? In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 833–842. Springer, Heidelberg (2005)Google Scholar
  18. 18.
    Mandl, T., Womser-Hacker, C.: The effect of named entities on effectiveness in cross-language information retrieval evaluation. In: SAC 2005: Proceedings of the, ACM symposium on Applied computing, pp. 1059–1064. ACM, New York (2005)CrossRefGoogle Scholar
  19. 19.
    Gaussier, É., Renders, J.M., Matveeva, I., Goutte, C., Déjean, H.: A geometric view on bilingual lexicon extraction from comparable corpora. In: ACL, pp. 526–533 (2004)Google Scholar
  20. 20.
    Nardi, A., Peters, C.: Working notes for the clef 2006 workshop (2006)Google Scholar
  21. 21.
    Nardi, A., Peters, C.: Working notes for the clef 2007 workshop (2007)Google Scholar
  22. 22.
    Udupa, R., Jagarlamudi, J., Saravanan, K.: Microsoft research india at fire 2008: Hindi-english cross-language information retrieval (2008)Google Scholar
  23. 23.
    Och, F.J., Ney, H.: A systematic comparison of various statistical alignment models. Comput. Linguist. 29(1), 19–51 (2003)CrossRefGoogle Scholar
  24. 24.
    Zhai, C., Lafferty, J.: A study of smoothing methods for language models applied to information retrieval. ACM Trans. Inf. Syst. 22(2), 179–214 (2004)CrossRefGoogle Scholar
  25. 25.
    Porter, M.F.: An algorithm for suffix stripping, pp. 313–316 (1997)Google Scholar
  26. 26.
    Khapra, M., Bhattacharyya, P.: Improving transliteration accuracy using word-origin detection and lexicon lookup. In: Proceedings of the 2009 Named Entities Workshop: Shared Task on Transliteration (NEWS 2009), Suntec, Singapore, August 2009, pp. 84–87. Association for Computational Linguistics (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  1. 1.Microsoft ResearchIndia
  2. 2.Indian Institute of TechnologyBombay

Personalised recommendations