Advertisement

The Vowel Game: Continuous Real-Time Visualization for Pronunciation Learning with Vowel Charts

  • Annu Paganus
  • Vesa-Petteri Mikkonen
  • Tomi Mäntylä
  • Sami Nuuttila
  • Jouni Isoaho
  • Olli Aaltonen
  • Tapio Salakoski
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4139)

Abstract

Learning to pronounce new speech sounds is difficult. Visual feedback helps in identifying the errors and indicating the achieved progress. The Vowel Game uses a visualization method that symbolizes the vocal tract. This instructs the user on how to adjust e.g. the tongue position during pronunciation. It gives information about the correctness and goodness of the uttered vowel. Preliminary evaluation suggests that continuous real-time feedback can be obtained, but the effect on learning remains to be tested.

Keywords

Visual Feedback Vocal Tract Speech Sound Spectral Envelope Collaborative Virtual Environment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Alais, D., Carlile, S.: Synchronizing to real events: Subjective audiovisual alignment scales with perceived auditory depth and speed of sound. Proc. Natl. Acad. Sci. USA. 102(6), 2244–2247 (2005)CrossRefGoogle Scholar
  2. 2.
    Carey, M.: CALL visual feedback for pronunciation of vowels: KAY Sona Speech. CALICO Journal 21(3) (2004)Google Scholar
  3. 3.
    Day, P.N., O’ Brian Holt, P., Russell, G.T.: Modelling the Effects of Delayed Visual Feedback in Real-Time Operator Control Loops: A Cognitive Perspective. In: Proceedings of XVII European Annual Conference on Human Decision Making and Manual Control, Loughborough, October 1999. Group D Publications Ltd., pp. 70–79 (1999)Google Scholar
  4. 4.
    Deller, J.R., Proakis, J.G., Hansen, J.H.L.: Discrete-Time Processing of Speech Signals. Macmillan, New York (1993)Google Scholar
  5. 5.
    Dowd, A., Smith, J.R., Wolfe, J.: Learning to pronounce vowel sounds in a foreign language using acoustic measurements of the vocal tract as feedback in real time. Language and Speech 41, 1–20 (1998)Google Scholar
  6. 6.
    Fitzgerald, M., Gruenwald, A., Stoker, R.: Software review – Video Voice Speech Training System. Volta Review 89, 171–173 (1989)Google Scholar
  7. 7.
    Hatzis, A.: Optical Logo-Therapy (OLT): Computer-Based Audio-Visual Feedback Using Interactive Visual Displays for Speech Training. PhD thesis, Department of Computer Science, University of Sheffield (1999)Google Scholar
  8. 8.
    Iverson, P., Kuhl, P.K., Akahane-Yamada, R., Diesch, E., Tohkura, Y., Kettermann, A., Siebert, C.: A perceptual interference account of acquisition difficulties for non-native phonemes. Cognition 87, B47–B57 (2003)Google Scholar
  9. 9.
    Jakobson, R., Fant, G., Halle, M.: Preliminaries to speech analysis: The distinctive features and their correlates. MIT Press, Cambridge (1969)Google Scholar
  10. 10.
    Kuhl, P.K.: Human adults and human infants show a "perceptual magnet effect” for the prototypes of speech categories, monkeys do not. Perception and Psychophysics 50(2), 93–107 (1991)CrossRefGoogle Scholar
  11. 11.
    Liberman, A.M., Harris, K.S., Hoffman, H.S., Griffith, B.C.: The discrimination of speech sounds within and across phoneme boundaries. Journal of Experimental Psychology 54, 358–368 (1957)CrossRefGoogle Scholar
  12. 12.
    Massaro, D.W., Light, J.: Using Visible Speech to Train Perception and Production of Speech for Individuals With Hearing Loss. Journal of Speech, Language and Hearing Research 47, 304–320 (2004)CrossRefGoogle Scholar
  13. 13.
    Paige, A., Zue, V.: Calculation of Vocal Tract Length. IEEE Transactions on Audio and Electroacoustics 18(3), 268–270 (1970)CrossRefGoogle Scholar
  14. 14.
    Regan, M., Pose, R.: Priority rendering with a virtual reality address recalculation pipeline. In: Proceedings of the 21st Annual Conference on Computer Graphics and interactive Techniques SIGGRAPH 1994. ACM Press, New York (1994)Google Scholar
  15. 15.
    The Turku Vowel Test. Dept. of Phonetics, University of Turku (3.4.2006), http://fon.utu.fi/
  16. 16.
    Vaghi, I., Greenhalgh, C., Benford, S.: Coping with inconsistency due to network delays in collaborative virtual environments. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST 1999, London, United Kingdom, December 20 - 22, 1999, pp. 42–49. ACM Press, New York (1999)CrossRefGoogle Scholar
  17. 17.
    Vicsi, K., Roach, P., Öster, A.-M., Kacic, Z., Csatári, F., Sfakianaki, A., Veronik, R.: A multilingual, Multimodal, Speech Training System SPECO. In: Proceedings of Eurospeech 2001, pp. 2807–2810 (2001)Google Scholar
  18. 18.
    Zhang, F.: Using interactive feedback tool to enhance pronunciation in language learning. In: Mishra, S., Sharma, R.C. (eds.) Interactive Multimedia in Education and Training, pp. 377–399. Idea Group Publishing, USA (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Annu Paganus
    • 1
  • Vesa-Petteri Mikkonen
    • 2
  • Tomi Mäntylä
    • 1
  • Sami Nuuttila
    • 1
  • Jouni Isoaho
    • 1
  • Olli Aaltonen
    • 2
  • Tapio Salakoski
    • 1
  1. 1.Department of Information TechnologyUniversity of TurkuTurkuFinland
  2. 2.Department of PhoneticsUniversity of TurkuTurkuFinland

Personalised recommendations