Adjusting Occurrence Probabilities of Automatically-Generated Abbreviated Words in Spoken Dialogue Systems
Users often abbreviate long words when using spoken dialogue systems, which results in automatic speech recognition (ASR) errors. We define abbreviated words as sub-words of an original word and add them to the ASR dictionary. The first problem we face is that proper nouns cannot be correctly segmented by general morphological analyzers, although long and compound words need to be segmented in agglutinative languages such as Japanese. The second is that, as vocabulary size increases, adding many abbreviated words degrades the ASR accuracy. We have developed two methods, (1) to segment words by using conjunction probabilities between characters, and (2) to adjust occurrence probabilities of generated abbreviated words on the basis of the following two cues: phonological similarities between the abbreviated and original words and frequencies of abbreviated words in Web documents. Our method improves ASR accuracy by 34.9 points for utterances containing abbreviated words without degrading the accuracy for utterances containing original words.
Index TermsSpoken dialogue systems abbreviated words adjusting occurrence probabilities
Unable to display preview. Download preview PDF.
- 1.Zweig, G., Nguyen, P., Ju, Y., Wang, Y., Yu, D., Acero, A.: The Voice-Rate Dialog System for Consumer Ratings. In: Proc. Interspeech, pp. 2713–2716 (2007)Google Scholar
- 2.Jan, E.E., Maison, B., Mangu, L., Zweig, G.: Automatic Construction of Unique Signatures and Confusable Sets for Natural Language Directory Assistance Applications. In: Proc. Eurospeech, pp. 1249–1252 (2003)Google Scholar
- 3.Katsumaru, M., Komatani, K., Ogata, T., Okuno, H.G.: Expanding Vocabulary for Recognizing User’s Abbreviations of Proper Nouns without Increasing ASR Error Rates in Spoken Dialogue Systems. In: Proc. Interspeech, pp. 187–190 (2008)Google Scholar
- 4.Komatani, K., Ueno, S., Kawahara, T., Okuno, H.G.: User Modeling in Spoken Dialogue Systems for Flexible Guidance Generation. In: Proc. Eurospeech, pp. 745–748 (2003)Google Scholar
- 5.Kudo, T., Yamamoto, K., Matsumoto, Y.: Applying conditional random fields to Japanese morphological analysis. In: Proc. EMNLP, pp. 230–237 (2004), http://mecab.sourceforge.net/
- 7.Clarkson, P.R., Rosenfeld, R.: Statistical Language Modeling Using the CMU-Cambridge Toolkit. In: Proc. ESCA Eurospeech, pp. 2707–2710 (1997), http://svr-www.eng.cam.ac.uk/~prc14/toolkit.html
- 8.Kawahara, T., Lee, A., Takeda, K., Itou, K., Shikano, K.: Recent progress of open-source LVCSR Engine Julius and Japanese model repository. In: Proc. ICSLP, pp. 3069–3072 (2004)Google Scholar