Abstract
In this paper, we investigate the adaptation of language modeling for conversational Mandarin-English Code-Switching (CS) speech and its effect on speech recognition performance. First, we investigate the prediction of code switches based on textual features with focus on Part-of-Speech (POS) tags. We show that the switching attitude is speaker dependent and utilize this finding to cluster the training speakers into classes with similar switching attitude. Second, we apply recurrent neural network language models which integrate the POS information into the input layer and factorize the output layer into languages for modeling CS. Furthermore, we adapt the background N-Gram and RNN language model to the different Code-Switching attitudes of the speaker clusters which lead to significant reductions in terms of perplexity. Finally, using these adapted language models we rerun the speech recognition system for each speaker and achieve improvements in terms of mixed error rate.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Adel, H., Vu, N.T., Kraus, F., Schlippe, T., Schultz, T., Li, H.: Recurrent neural network language modeling for code switching conversational speech. In: ICASSP (2012)
Auer, P.: Code-switching in conversation. Routledge (1999)
Auer, P.: From codeswitching via language mixing to fused lects toward a dynamic typology of bilingual speech. International Journal of Bilingualism 3(4), 309–332 (1999)
Bokamba, E.G.: Are there syntactic constraints on code-mixing? World Englishes 8(3), 277–292 (1989)
Chan, J.Y.C., Ching, P., Lee, T., Cao, H.: Automatic speech recognition of Cantonese-English code-mixing utterances. In: Proc. of ICSLP (2006)
Kombrink, S., Mikolov, T., Karafiát, M., Burget, L.: Recurrent neural network based language modeling in meeting recognition. In: Proc. of ICSLP (2011)
Lyu, D.C., Tan, T.P., Chng, E.S., Li, H.: An analysis of a Mandarin-English code-switching speech corpus: Seame. In: ICSLP (2010)
Marcus, M.P., Marcinkiewicz, M.A., Santorini, B.: Building a large annotated corpus of English: The penn treebank. Computational Linguistics 19(2), 313–330 (1993)
Mikolov, T., Karafiát, M., Burget, L., Cernocky, J., Khudanpur, S.: Recurrent neural network based language model. In: Proc. of ICSLP (2010)
Mikolov, T., Kombrink, S., Burget, L., Cernocky, J., Khudanpur, S.: Extensions of recurrent neural network language model. In: ICASSP, pp. 5528–5531 (2011)
Mikolov, T., Kombrink, S., Deoras, A., Burget, L., Cernockỳ, J.: Rnnlm–recurrent neural network language modeling toolkit. In: Proc. of the 2011 ASRU Workshop, pp. 196–201 (2011)
Muysken, P.: Bilingual speech: A typology of code-mixing, vol. 11. Cambridge University Press (2000)
Poplack, S.: Syntactic structure and social function of code-switching, vol. 2. Centro de Estudios Puertorriqueños, City University of New York (1978)
Poplack, S.: Sometimes i’ll start a sentence in spanish y termino en español: toward a typology of code-switching. Linguistics 18(7-8), 581–618 (1980)
Shi, Y., Wiggers, P., Jonker, C.M.: Towards recurrent neural networks language models with linguistic and contextual features. In: ICSLP (2012)
Solorio, T., Liu, Y.: Learning to predict code-switching points. In: Proc. of the EMNLP, pp. 973–981. Association for Computational Linguistics (2008)
Steinbach, M., Karypis, G., Kumar, V., et al.: A comparison of document clustering techniques. In: KDD Workshop on Text Mining, vol. 400, pp. 525–526 (2000)
Stolcke, A., et al.: Srilm-an extensible language modeling toolkit. In: Proc. of ICSLP, vol. 2, pp. 901–904 (2002)
Toutanova, K., Klein, D., Manning, C.D., Singer, Y.: Feature-rich part-of-speech tagging with a cyclic dependency network. In: Proc. of the HLT-NAACL, pp. 173–180 (2003)
Toutanova, K., Manning, C.D.: Enriching the knowledge sources used in a maximum entropy part-of-speech tagger. In: Proc. of the 2000 Joint SIGDAT Conference EMNLP/VLC, pp. 63–70 (2000)
Vu, N.T., Lyu, D.C., Weiner, J., Telaar, D., Schlippe, T., Blaicher, F., Chng, E., Schultz, T., Li, H.: A first speech recognition system for Mandarin-English code-switch conversational speech. In: ICASSP, pp. 4889–4892 (2012)
Xue, N., Xia, F., Chiou, F.D., Palmer, M.: The Penn Chinese treebank: Phrase structure annotation of a large corpus. Natural Language Engineering 11(2), 207 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Vu, N.T., Adel, H., Schultz, T. (2013). An Investigation of Code-Switching Attitude Dependent Language Modeling. In: Dediu, AH., MartÃn-Vide, C., Mitkov, R., Truthe, B. (eds) Statistical Language and Speech Processing. SLSP 2013. Lecture Notes in Computer Science(), vol 7978. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39593-2_26
Download citation
DOI: https://doi.org/10.1007/978-3-642-39593-2_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39592-5
Online ISBN: 978-3-642-39593-2
eBook Packages: Computer ScienceComputer Science (R0)