Advertisement

Class-Based Language Models for Chinese-English Parallel Corpus

  • Junfei Guo
  • Juan Liu
  • Michael Walsh
  • Helmut Schmid
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7817)

Abstract

This paper addresses using novel class-based language models on parallel corpora, focusing specifically on English and Chinese languages. We find that the perplexity of Chinese is generally much higher than English and discuss the possible reasons. We demonstrate the relative effectiveness of using class-based models over the modified Kneser-Ney trigram model for our task. We also introduce a rare events clustering and a polynomial discounting mechanism, which is shown to improve results. Our experimental results on parallel corpora indicate that the improvement due to classes are similar for English and Chinese. This suggests that class-based language models should be used for both languages.

Keywords

Language Model Machine Translation Chinese Word Statistical Machine Translation Computational Linguistics 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Manning, C.D., Schütze, H.: Foundations of statistical natural language processing. MIT Press, Cambridge (1999)zbMATHGoogle Scholar
  2. 2.
    Koehn, P.: Statistical Machine Translation, 1st edn. Cambridge University Press, New York (2010)zbMATHGoogle Scholar
  3. 3.
    Schütze, H.: Integrating history-length interpolation and classes in language modeling. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 1516–1525. Association for Computational Linguistics, Portland (2011)Google Scholar
  4. 4.
    Gao, J., Goodman, J., Li, M., Lee, K.F.: Toward a unified approach to statistical language modeling for chinese, vol. 1(1), pp. 3–33 (March 2002)Google Scholar
  5. 5.
    Manning, C.D., Raghavan, P., Schtze, H.: Introduction to Information Retrieval. Cambridge University Press, New York (2008)zbMATHCrossRefGoogle Scholar
  6. 6.
    Chang, P.C., Galley, M., Manning, C.D.: Optimizing chinese word segmentation for machine translation performance. In: Proceedings of the Third Workshop on Statistical Machine Translation, StatMT 2008, pp. 224–232. Association for Computational Linguistics, Stroudsburg (2008)CrossRefGoogle Scholar
  7. 7.
    Chen, S.F., Goodman, J.: An empirical study of smoothing techniques for language modeling. In: Proceedings of the 34th Annual Meeting on Association for Computational Linguistics, ACL 1996, pp. 310–318. Association for Computational Linguistics, Stroudsburg (1996)CrossRefGoogle Scholar
  8. 8.
    Brown, P.F., deSouza, P.V., Mercer, R.L., Pietra, V.J.D., Lai, J.C.: Class-based n-gram models of natural language. Computational Linguistics 18, 467–479 (1992)Google Scholar
  9. 9.
    Dupont, P., Rosenfeld, R.: Lattice based language models. Technical report (1997)Google Scholar
  10. 10.
    Eisele, A., Chen, Y.: Multiun: A multilingual corpus from united nation documents. In: Chair, N.C.C., Choukri, K., Maegaard, B., Mariani, J., Odijk, J., Piperidis, S., Rosner, M., Tapias, D. (eds.) Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC 2010). European Language Resources Association (ELRA), Valletta (2010)Google Scholar
  11. 11.
    Stolcke, A.: Srilm - an extensible language modeling toolkit. In: Hansen, J.H.L., Pellom, B.L. (eds.) INTERSPEECH. ISCA (2002)Google Scholar
  12. 12.
    Luo, X., Roukos, S.: An iterative algorithm to build chinese language models. In: Proceedings of the 34th Annual Meeting on Association for Computational Linguistics, ACL 1996, pp. 139–143. Association for Computational Linguistics, Stroudsburg (1996)CrossRefGoogle Scholar
  13. 13.
    Gao, J., Goodman, J.T., Miao, J.: The Use of Clustering Techniques for Language Modeling Application to Asian LanguagesGoogle Scholar
  14. 14.
    Luo, J., Lamel, L., Gauvain, J.L.: Modeling characters versuswords for mandarin speech recognition. In: Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2009, pp. 4325–4328. IEEE Computer Society, Washington, DC (2009)CrossRefGoogle Scholar
  15. 15.
    Schütze, H.: Distributional part-of-speech tagging. In: Proceedings of the Seventh Conference on European Chapter of the Association for Computational Linguistics, EACL 1995, pp. 141–148. Morgan Kaufmann Publishers Inc., San Francisco (1995)CrossRefGoogle Scholar
  16. 16.
    Yokoyama, T., Shinozaki, T., Iwano, K., Furui, S.: Unsupervised class-based language model adaptation for spontaneous speech recognition. In: Proceedings of 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2003), vol. 1, pp. I-236–I-239 (April 2003)Google Scholar
  17. 17.
    Momtazi, S., Klakow, D.: A word clustering approach for language model-based sentence retrieval in question answering systems. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management, CIKM 2009, pp. 1911–1914. ACM, New York (2009)CrossRefGoogle Scholar
  18. 18.
    Maltese, G., Bravetti, P., Crépy, H., Grainger, B.J., Herzog, M., Palou, F.: Combining word- and class-based language models: a comparative study in several languages using automatic and manual word-clustering techniques. In: Dalsgaard, P., Lindberg, B., Benner, H., Tan, Z.H. (eds.) INTERSPEECH, pp. 21–24. ISCA (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Junfei Guo
    • 1
    • 2
  • Juan Liu
    • 1
  • Michael Walsh
    • 2
  • Helmut Schmid
    • 2
  1. 1.School of ComputerWuhan UniversityChina
  2. 2.Institute for Natural Language ProcessingUniversity of StuttgartGermany

Personalised recommendations