Morphology to the Rescue Redux: Resolving Borrowings and Code-Mixing in Machine Translation

  • Esmé Manandise
  • Claudia Gdaniec
Part of the Communications in Computer and Information Science book series (CCIS, volume 100)


In the IBM LMT machine translation system, derivational morphological rules recognize and analyze words that are not found in its source lexicons, and generate default transfers for these unlisted words. Unfound words with no inflectional or derivational affixes are by default nouns. These rules are now expanded to provide lexical coverage of a particular set of words created on the fly in emails by bilingual Spanish-English speakers. What characterizes the approach is the generation of additional default parts of speech, and the use of morphological, semantic, and syntactic features from both source and target lexicons for analysis and transfer. A built-in rule-based strategy to handle language borrowing and code-mixing allows for the recognition of words with variable and unpredictable frequency of occurrence, which would remain otherwise unfound, thus affecting the accuracy of parsing and the quality of translation output.


Unfound words rule-based morphology derivational morphology parsing code-mixing code-switching borrowing scoring unsupervised email machine translation languages in contact Spanish-English 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Gardner-Chloros, P., Edwards, M.: Assumptions behind grammatical approaches to code-switching: when the blueprint is a red herring. Transactions of the Philological Society 102(1), 103–129 (2004)CrossRefGoogle Scholar
  2. 2.
    Solorio, T., Liu, Y.: Learning to Predict Code-switching Points. In: Proceedings of Empirical Methods on Natural Language Processing, pp. 973–981 (2008)Google Scholar
  3. 3.
    A Parallel Corpus for Statistical Machine Translation,
  4. 4.
    McCord, M., Wolff, S.: The Lexicon and Morphology for LMT. IBM Research Division Research Report, RC 13403 (1988)Google Scholar
  5. 5.
    McCord, M.C., Bernth, A.: The LMT Transformational System. Machine Translation and the Information Soup. In: Proceedings of the 3rd AMTA Conference, pp. 344–354. Springer, Heidelberg (1998)Google Scholar
  6. 6.
    McCord, M.C.: Slot Grammar: A system for simple construction of practical natural language grammars. In: Studer, R. (ed.) Natural Language and Logic: International Scientific Symposium, pp. 118–145. Springer, Berlin (1990)CrossRefGoogle Scholar
  7. 7.
    Gdaniec, C., Manandise, E., McCord, M.: Derivational Morphology to the Rescue: How It Can Help Resolve Unfound Words in MT. In: Hutchins, J. (ed.) Proceedings, MT Summit VIII, Santiago (2001); CD edn., compiled by Hutchins, J.Google Scholar
  8. 8.
    Gdaniec, C., Manandise, E.: Using Word Formation Rules to Extend MT Lexicons. In: Richardson, S.D. (ed.) AMTA 2002. LNCS (LNAI), vol. 2499, pp. 64–73. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  9. 9.
    Cartoni, B.: Lexical Morphology in Machine Translation: a Feasibility Study. In: Proceedings of of the 12th Conference of the European Chapter of the ACL, pp. 130–138 (2009)Google Scholar
  10. 10.
    Meni, A., Goldberg, Y., Gabay, D., Elhadad, M.: Unsupervised Lexicon-Based Resolution of Unknown Words for Full Morphological Analysis. In: Proceedings of ACL 2008, HLT, pp. 728–736 (2008)Google Scholar
  11. 11.
    Solorio, T., Liu, Y., Medina, B.: Part-of-speech Tagging English-Spanish Code-switched Text. In: Proceedings of Empirical Methods on Natural Language Processing (2008)Google Scholar
  12. 12.
    Franco, J.C., Solorio, T.: Baby-Steps towards Building a Spanglish Language Model. In: Gelbukh, A. (ed.) CICLing 2007. LNCS, vol. 4394, pp. 75–84. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  13. 13.
    Goyal, P., Mital, M.R., Mukerjee, A., Raina, A.M., Sharma, D., Vikram, K.: Saarthaka: A Bilingual Parser for Hindi, English and Code-Switching Structures. In: Proceedings of EACL 2003, European Chapter of the Association for Computational Linguistics, Budapest, pp. 15–24 (2003)Google Scholar
  14. 14.
    Joshi, A.: Processing of Sentences with Intra-sentential Code-switching. In: Horecky, J. (ed.) COLING 1982. North-Holland Publishing Company © Academia, Amsterdam (1982)Google Scholar
  15. 15.
    Sinha, R.M.K., Thakur, A.: Machine Translation of Bi-lingual Hindi-English (Hinglish) Text. In: Proceedings of the 10th Conference on Machine Translation, Phuket, Thailand, pp. 149–156 (2005)Google Scholar
  16. 16.
    Alex, B., Dubey., A., Keller, F.: Using Foreign Inclusion Detection to Improve Parsing Performance. In: Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 151–160 (2007)Google Scholar
  17. 17.
    Rinsche, A.: Towards a MT Evaluation Methodology. In: Proceedings of the Fifth International Conference on Theoretical and Methodological Issues in Machine Translation, Kyoto, Japan, July 14-16 (1993),

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Esmé Manandise
    • 1
  • Claudia Gdaniec
    • 2
  1. 1.IBM Thomas J. Watson Research CenterYorktown HeightsUSA
  2. 2.South Westphalia University of Applied SciencesSoestGermany

Personalised recommendations