Interactive Computer Aids for Acquiring Proficiency in Mandarin

  • Stephanie Seneff
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4274)


It is widely recognized that one of the best ways to learn a foreign language is through spoken dialogue with a native speaker. However, this is not a practical method in the classroom due to the one-to-one student/teacher ratio it implies. A potential solution to this problem is to rely on computer spoken dialogue systems to role play a conversational partner. This paper describes several multilingual dialogue systems specifically designed to address this need. Students can engage in dialogue with the computer either over the telephone or through audio/typed input at a Web page. Several different domains are being developed, in which a student’s conversational interaction is assisted by a software agent functioning as a “tutor” which can provide them with translation assistance at any time. Thus, two recognizers are running in parallel, one for English and one for Chinese. Some of the research issues surrounding high-quality spoken language translation and dialogue interaction with a non-native speaker are discussed.


Language Learn Dialogue System Implicit Feedback Dialogue Manager Conversational Interaction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Brown, P.F., Cocke, J., Della Pietra, S.A., Della Pietra, V.J., Jelinek, F., Mercer, R.L., Roossin, P.S.: A statistical approach to machine translation. Computational Linguistics 16(2), 79–85 (1990)Google Scholar
  2. 2.
    Baptist, L., Seneff, S.: Genesis-II: A Versatile System for Language Generation in Conversational System Applications. In: Proc. ICSLP 2000, Beijing, China, October 2000, vol. III, pp. 271–274 (2000)Google Scholar
  3. 3.
    Cowan, B.: PLUTO: A Preprocessor for Multilingual Spoken Language Generation. S.M. thesis, MIT Department of Electrical Engineering and Computer Science (February 2004)Google Scholar
  4. 4.
    Dong, B., Zhao, Q., Zhang, J., Yan, Y.: Automatic Assessment of Pronunciation Quality. In: ISCSLP 2004, Hong Kong, pp. 137–140 (2004)Google Scholar
  5. 5.
    Glass, J.: A Probabilistic Framework for Segment-Based Speech Recognition. Computer, Speech, and Language 17, 137–152 (2003)CrossRefGoogle Scholar
  6. 6.
    Ehsani, F., Knodt, E.: Speech Technology in Computer-aided Language Learning: Strengths and Limitations of a new CALL Paradigm Language Learning & Technology 2(1), 45–60 (1998)Google Scholar
  7. 7.
    Eskenazi, M.: Using Automatic Speech Processing for Foreign Language Pronunciation Tutoring: Some Issues and a Prototype. Language Learning and Technolgy 2(2), 62–76 (1999)Google Scholar
  8. 8.
    Holland, V.M., Kaplan, J.D., Sabol, M.A.: Preliminary Tests of Language Learning in a Speech-Interactive Graphics Microworld. Calico Journal 16(3), 339–359 (1998)Google Scholar
  9. 9.
  10. 10.
  11. 11.
    Jia, J.: The study of the application of a web-based chatbot system on the teaching of foreign languages. In: Proceedings of SITE 2004, pp. 1201–1207. AACE Press, USA (2004)Google Scholar
  12. 12.
    Jia, J.: CSIEC (Computer Simulator in Educational Communication): A Virtual Context-Adaptive Chatting Partner for Foreign Language Learners. In: Proceedings of ICALT 2004, USA, pp. 690–692. IEEE Computer Society Press, Los Alamitos (2004)Google Scholar
  13. 13.
    Johnson, W.L., Marsella, S., Mote, N., Si, M., Vihjalmsson, H., Wu, S.: Balanced Perception and Action in the Tactical Language Training System. In: International Conference on Autonomous and Multi-agent Systems (2004)Google Scholar
  14. 14.
    Johnson, W.L., Marsella, S., Vihjalmsson, H.: The DARWARS Tactical Language Training System. In: Interservice/Industry Training, Simulation, and Education Conference, I/ITSEC (2004)Google Scholar
  15. 15.
    Kim, J.-M., Wang, C., Peabody, M., Seneff, S.: An Interactive English Pronunciation Dictionary for Korean Learners. In: Interspeech 2004, pp. 1145–1148 (2004)Google Scholar
  16. 16.
    Koehn, P., Och, F.J., Marcu, D.: Statistical Phrase-Based Translation. In: Proc. HLT-NAACL (2003)Google Scholar
  17. 17.
    Leather, J.: Perceptual and productive learning of Chinese lexical tone by Dutch and English speakers. In: Leather, J., James, A. (eds.) New Sounds 1990, University of Amsterdam, pp. 72–97 (1990)Google Scholar
  18. 18.
    Lee, J.S.Y., Seneff, S.: Automatic Grammar Correction for Second-Language Learners. In: Proc. INTERSPEECH (2006) (to appear)Google Scholar
  19. 19.
    Litman, D.: Spoken Dialogue for Intelligent Tutoring Systems; Opportunities and Challenges. In: Proc. HLT-NAACL, Keynote speech (2006)Google Scholar
  20. 20.
    Meng, H., Ching, P.C., Chan, S.F., Wong, Y.F., Chan, C.C.: ISIS: An Adaptive Trilingual Conversational System with Interleaving, Interaction, and Delegation Dialogues. ACM Transactions on Computer-Human Interaction (TOCHI) 11(3), 268–299 (2004)CrossRefGoogle Scholar
  21. 21.
    Neri, A., Cucchiarini, C., Strik, H.: Feedback in computer assisted pronunciation training: technology push or demand pull? In: Proceedings of ICSLP, pp. 1209–1212 (2002)Google Scholar
  22. 22.
    Peabody, M., Seneff, S., Wang, C.: Mandarin Tone Acquisition through Typed Dialogues. In: InSTIL Symposium on Computer Assisted Language Learning, Venice, Italy, pp. 173–176 (2004)Google Scholar
  23. 23.
    Peabody, M., Seneff, S.: Towards Automatic Tone Correction in Non-native Mandarin. Submitted to ISCSLP 2006 (2006)Google Scholar
  24. 24.
    Sato, S.: CTM: an example-based translation aid system using the character-based match retrieval method. In: Proc. COLING (1992)Google Scholar
  25. 25.
    Seneff, S., Hurley, E., Lau, R., Pao, C., Schmid, P., Zue, V.: Galaxy-II: A Reference Architecture for Conversational System Development. In: ICSLP 1998, Sydney, Australia, December 1998, pp. 931–934 (1998)Google Scholar
  26. 26.
    Seneff, S., Wang, C., Peabody, M., Zue, V.: Second Language Acquisition through Human Computer Dialogue. In: Proc. ISCSLP 2004, Hong Kong (2004)Google Scholar
  27. 27.
    Seneff, S.: TINA: A Natural Language System for Spoken Language Applications. Computational Linguistics 18(1), 61–86 (1992)Google Scholar
  28. 28.
    Seneff, S., Wang, C., Zhang, J.: Spoken Conversational Interaction for Language Learning. In: Proc. InSTIL Symposium on Computer Assisted Language Learning, Venice, pp. 151–154 (2004)Google Scholar
  29. 29.
    Seneff, S.: Response Planning and Generation in the Mercury Flight Reservation System. Computer Speech and Language 16, 283–312 (2002)CrossRefGoogle Scholar
  30. 30.
    Wang, C., Seneff, S.: High-quality Speech Translation for Language Learning. In: STIL Symposium on Computer Assisted Language Learning, Venice, Italy, pp. 99–102 (2004)Google Scholar
  31. 31.
    Yao, T.C., Liu Yao, Y.: Integrated Chinese, 2nd edn. Cheng and Tsui Company, Boston (2005)Google Scholar
  32. 32.
    Seneff, S., Wang, C., Lee, J.S.Y.: Combining Linguistic and Statistical Methods for Bi-directional English Chinese Translation in the Flight Domain. In: AMTA (2006) (to appear)Google Scholar
  33. 33.
    Zue, V., Seneff, S., Glass, J., Polifroni, J., Pao, C., Hazen, T.J., Hetherington, L.: Jupiter: A Telephone-Based Conversational Interface for Weather Information. IEEE Trans. Speech and Audio Proc. 8(1), 85–96 (2000)CrossRefGoogle Scholar
  34. 34.
    Zue, V., Glass, J.: Conversational Interfaces: Advances and Challenges. Proc. IEEE 88(8), 1166–1180 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Stephanie Seneff
    • 1
  1. 1.Computer Science and Artificial Intelligence LaboratoryMITCambridgeUSA

Personalised recommendations