A Game System for Speech Rehabilitation

  • Mark Shtern
  • M. Brandon Haworth
  • Yana Yunusova
  • Melanie Baljko
  • Petros Faloutsos
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7660)


Speech disorders are common in children and adults. For a number of these individuals, traditional speech therapy will not be successful, leaving them unable to (or poorly able) to communicate through speech. By combining recent advances in motion tracking technology with state of the art interactive gaming technology, we aim to develop a novel clinical tool that can potentially overcome the shortcomings of traditional methods. There is strong evidence that entertaining and clever visualizations not only can provide necessary augmented feedback to facilitate motor skill acquisition but, also, can be very engaging and effective teaching tools. For the first time, we want to explore the practically infinite possibilities for interactive visualizations of game engines in speech rehabilitation focused on the tongue, the primary articulator, and develop a commercial grade clinical rehabilitation tool with full security, reporting, and data management abilities.


Wave System Speech Therapy Speech Disorder Game Engine Game System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Engwall, O., Bälter, O.: Pronunciation feedback from real and virtual language teachers. Computer Assisted Language Learner 3, 235–262 (2007)CrossRefGoogle Scholar
  2. 2.
    Vicsi, K., Roach, P., Oster, A., Kacic, Z., Barczikay, A., Tantoa, A., Csatari, F., Bakcsi, Z., Sfakianaki, A.: A multilingual teaching and training system for children with speech disorders. International Journal of Speech Technology 3, 289–300 (2000)zbMATHCrossRefGoogle Scholar
  3. 3.
    Piper, A., O’Brien, E., Morris, M., Winograd, T.: Sides: a cooperative tabletop computer game for social skills development. In: Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work, CSCW 2006, pp. 1–10. ACM, New York (2006)CrossRefGoogle Scholar
  4. 4.
    Berry, J.: Accuracy of the NDI wave speech research system. Speech Language, and Hearing Research 54, 1295–1301 (2011)CrossRefGoogle Scholar
  5. 5.
    Massaro, D., Bigler, S., Chen, T., Perlman, M., Ouni, S.: Pronunciation training: the role of eye and ear. In: Proceedings of Interspeech 2008, pp. 2623–2626 (2008)Google Scholar
  6. 6.
    Kuhl, P.K., Meltzoff, A.N.: The bimodal perception of speech in infancy. Science 218, 1138–1141 (1982)CrossRefGoogle Scholar
  7. 7.
    Lucero, J., Maciel, S., Johns, D., Munhall, K.: Empirical modeling of human face kinematics during speech using motion clustering. The Journal of the Acoustical Society of America (January 2005)Google Scholar
  8. 8.
    Mcgurck, H., Macdonald, J.W.: Hearing lips and seeing voices. Nature 264(246-248) (1976)Google Scholar
  9. 9.
    Davis, S.M., Drichta, C.E.: Biofeedback theory and application in allied health: speech pathology. Biofeedback Self Regul. 5(2), 159–174 (1980)CrossRefGoogle Scholar
  10. 10.
    Maryn, Y., De Bodt, M., Van Cauwenberge, P.: Effects of biofeedback in phonatory disorders and phonatory performance: a systematic literature review. Appl. Psychophysiol. Biofeedback 31(1), 65–83 (2006)CrossRefGoogle Scholar
  11. 11.
    Mandel, A., Nymark, J., Balmer, S., Grinnell, D., O’Riain, M.: Electromyographic versus rhythmic positional biofeedback in computerized gait retraining with stroke patients. Arch. Phys. Med. Rehabil. 71(9), 649–654 (1990)Google Scholar
  12. 12.
    Moreland, J., Thomson, M.A.: Efficacy of electromyographic biofeedback compared with conventional physical therapy for upper-extremity function in patients following stroke: a research overview and meta-analysis. Phys. Ther. 74(6), 534–543 (1994); discussion 544–547Google Scholar
  13. 13.
    Beacutedard, P., Proteau, L.: On-line vs. off-line utilization of peripheral visual afferent information to ensure spatial accuracy of goal-directed movements. Exp. Brain Res. 158(1), 75–85 (2004)Google Scholar
  14. 14.
    Franklin, D.W., So, U., Burdet, E., Kawato, M.: Visual feedback is not necessary for the learning of novel dynamics. PLoS ONE 2(12), e1336 (2007)Google Scholar
  15. 15.
    McNeil, M., Katz, W., Fossett, T., Garst, D., Szuminsky, N., Carter, G., Lim, K.: Effects of on-line augmented kinematic and perceptual feedback on treatment of speech movement in apraxia of speech. Folia Phoniatr. Logop. 62, 127–133 (2010)CrossRefGoogle Scholar
  16. 16.
    Hardcastle, W., Gibbon, F., Jones, W.: Visual display of tongue palate contact: electropalatography in the assessment and remediation of speech disorders. Br. J. Disord. Commun. 26, 41–74 (1991)CrossRefGoogle Scholar
  17. 17.
    Engwall, O.: Analysis of and feedback on phonetic features in pronunciation training with a virtual teacher. Computer Assisted Language Learning 25(1), 37–64 (2012), QC 20120416Google Scholar
  18. 18.
    Dourish, P.: Where the action is: the foundations of embodied interaction. MIT Press, Cambridge (2001)Google Scholar
  19. 19.
    Ludlow, C.L., Hoit, J., Kent, R., Ramig, L.O., Shrivastav, R., Strand, E., Yorkston, K., Sapienza, C.M.: Translating principles of neural plasticity into research on speech motor control recovery and rehabilitation. J. Speech Lang. Hear. Res. 51(1), 240–258 (2008)CrossRefGoogle Scholar
  20. 20.
    Yunusova, Y., Green, J., Mefferd, A.: Accuracy assessment for AG500, electromagnetic articulograph. Speech Language, and Hearing Research 52, 547–555 (2009)CrossRefGoogle Scholar
  21. 21.
    Yunusova, Y., Baljko, M., Pintilie, G., Rudy, K., Faloutsos, P., Daskalogiannakis, J.: Acquisition of the 3d surface of the palate by in-vivo digitization. Speech Communication 54, 923–931 (2012)CrossRefGoogle Scholar
  22. 22.
    Unity Inc.: Unity3D,

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Mark Shtern
    • 1
  • M. Brandon Haworth
    • 1
  • Yana Yunusova
    • 2
  • Melanie Baljko
    • 1
  • Petros Faloutsos
    • 1
  1. 1.York UniversityTorontoCanada
  2. 2.University of TorontoTorontoCanada

Personalised recommendations