Skip to main content
SpringerLink
Log in

Opti-Speech-VMT: Implementation and Evaluation

  • Conference paper
  • First Online:
Body Area Networks. Smart IoT and Big Data for Intelligent Health Management (BODYNETS 2021)

Included in the following conference series:

  • 578 Accesses

Abstract

We describe Opti-Speech-VMT, a prototype tongue tracking system that uses electromagnetic articulography to permit visual feedback during oral movements.Opti-Speech-VMT is specialized for visuomotor tracking (VMT) experiments in which participants follow an oscillating virtual target in the oral cavity using a tongue sensor. The algorithms for linear, curved, and custom trajectories are outlined, and new functionality is briefly presented. Because latency can potentially affect accuracy in VMT tasks, we examined system latency at both the API and total framework levels. Using a video camera, we compared the movement of a sensor (placed on an experimenter’s finger) against an oscillating target displayed on a computer monitor. The average total latency was 87.3 ms, with 69.8 ms attributable to the API, and 17.4 ms to Opti-Speech-VMT. These results indicate minimal reduction in performance due to Opti-Speech-VMT, and suggest the importance of the EMA hardware and signal processing optimizations used.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Katz, W., et al.: Opti-Speech: a real-time, 3D visual feedback system for speech training. In: INTERSPEECH, pp. 1174–1178 (2014)

    Google Scholar 

  2. Wang, J., Green, J.R., Samal, A.: Individual articulator’s contribution to phoneme production. In: IEEE International Conference on Acoustics, Speech and Signal Proceedings, pp. 7785–7789, May 2013

    Google Scholar 

  3. Wang, J., Samal, A., Rong, P., Green, J.R.: An optimal set of flesh points on tongue and lips for speech-movement classification. J. Speech Lang. Hear. Res. 59(1), 15–26 (2016)

    Article  Google Scholar 

  4. Katz, W.F., Mehta, S.: Visual feedback of tongue movement for novel speech sound learning. Front. Hum. Neurosci. 9, 612 (2015)

    Article  Google Scholar 

  5. Watkins, C.H.: Sensor driven real-time animation for feedback during physical therapy, (Masters Thesis), The University of Texas at Dallas (2015)

    Google Scholar 

  6. Mental, R.L.: Using Realistic Visual Biofeedback for the Treatment of Residual Speech Sound Errors, (Doctoral Dissertation), Case Western Reserve University (2018)

    Google Scholar 

  7. Fazel, V., Katz, W.F.: Visuomotor pursuit tracking accuracy for intraoral tongue movement. J. Acoust. Soc. Am. 140(4), 3224 (2016)

    Article  Google Scholar 

  8. Fazel, V.: Lingual speech motor control assessed by a novel visuomotor tracking paradigm, (Doctoral Dissertation), The University of Texas at Dallas (2021)

    Google Scholar 

  9. Bernhardt, M.B., et al.: Ultrasound as visual feedback in speech habilitation: exploring consultative use in rural British Columbia. Canada. Clin. Linguist. Phonetics 22(2), 149–162 (2008)

    Article  Google Scholar 

  10. Preston, J.L., Leece, M.C., Maas, E.: Intensive treatment with ultrasound visual feedback for speech sound errors in childhood apraxia. Front. Hum. Neurosci. 10(2016), 440 (2016)

    Google Scholar 

  11. Preston, J.L., et al.: Ultrasound visual feedback treatment and practice variability for residual speech sound errors. J. Speech Lang. Hear. Res. 57(6), 2102–2115 (2014)

    Article  Google Scholar 

  12. Haldin, C., et al.: Speech recovery and language plasticity can be facilitated by sensori-motor fusion training in chronic non-fluent aphasia. A case report study. Clin. Linguist. Phonetics 32(7), 595–621 (2018)

    Google Scholar 

  13. Laporte, C., Ménard, L.: Multi-hypothesis tracking of the tongue surface in ultrasound video recordings of normal and impaired speech. Med. Image Anal. 44, 98–114 (2018)

    Article  Google Scholar 

  14. Karimi, E., Menard, L., Laporte, C.: Fully-automated tongue detection in ultrasound images. Comput. Biol. Med. 111, 103335 (2019)

    Article  Google Scholar 

  15. Mozaffari, M.H., Lee, W.-S.: Encoder-decoder CNN models for automatic tracking of tongue contours in real-time ultrasound data. Methods 179, 26–36 (2020)

    Article  Google Scholar 

  16. Shtern, M., Haworth, M.B., Yunusova, Y., Baljko, M., Faloutsos, P.: A game system for speech rehabilitation. In: Kallmann, M., Bekris, K. (eds.) MIG 2012. LNCS, vol. 7660, pp. 43–54. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-34710-8_5

    Chapter  Google Scholar 

  17. Tilsen, S., Das, D., McKee, B.: Real-time articulatory biofeedback with electromagnetic articulography. Linguist. Vanguard 1(1), 39–55 (2015)

    Article  Google Scholar 

  18. Suemitsu, A., Dang, J., Ito, T., Tiede, M.: A real-time articulatory visual feedback approach with target presentation for second language pronunciation learning. J. Acoust. Soc. Am. 138(4), EL382-7 (2015). PMID: 26520348, PMCID: PMC4608962. https://doi.org/10.1121/1.4931827

  19. James, K., et al.: Watch your Tongue: A point-tracking visualisation system in Blender

    Google Scholar 

  20. Katz, W.F., Bharadwaj, S.V., Carstens, B.: Electromagnetic articulography treatment for an adult with Broca’s aphasia and apraxia of speech. J. Speech Lang. Hear. Res. 42(6), 1355–1366 (1999)

    Article  Google Scholar 

  21. Kulpa, J.D., Pfordresher, P.Q.: Effects of delayed auditory and visual feedback on sequence production. Exp. Brain Res. 224(1), 69–77 (2013)

    Article  Google Scholar 

  22. Friston, S., Karlstrum, P., Steed, A.: The effects of low latency on pointing and steering tasks. IEEE Trans. Vis. Comput. Graph. 22(5), 1605–1615 (2016)

    Article  Google Scholar 

  23. Chesters, J., Baghai-Ravary, K., Mottonen, R.: The effects of delayed auditory and visual feedback on speech production. J. Acoust. Soc. Am. 137(2), 873–883 (2015). https://doi.org/10.1121/1.4906266

    Article  Google Scholar 

  24. Bradski, B.: The OpenCV Library. Dr. Dobb’s J, Software Tools (2000)

    Google Scholar 

  25. Berry, J.: Accuracy of the NDI wave speech research system. J. Speech Lang. Hear. Res 54, 1295–1301 (2011)

    Article  Google Scholar 

  26. Sigona, F., Stella, M., Stella, A.P., Bernardini, P., Fivela, B.G., Grimaldi, M.: Assessing the position tracking reliability of Carstens’ AG500 and AG501 electromagnetic articulographs during constrained movements and speech tasks. Speech Commun. 104, 73–88 (2018)

    Article  Google Scholar 

  27. Glotfelty, A., Katz, W.F.: The role of visibility in silent speech tongue movements: a kinematic study of consonants. J. Speech Lang. Hear. Res 2021, 1–8 (2021)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Texas at Dallas, Richardson, TX, 75080, USA

    Hiranya G. Kumar, Anthony R. Lawn, B. Prabhakaran & William F. Katz

  2. Topaz Labs, 14285 Midway Road, Suite 125, Addison, TX, 75001, USA

    Anthony R. Lawn

Authors
  1. Hiranya G. Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anthony R. Lawn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. B. Prabhakaran
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. William F. Katz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hiranya G. Kumar .

Editor information

Editors and Affiliations

  1. University of Glasgow, Glasgow, UK

    Masood Ur Rehman

  2. University of Glasgow, Glasgow, UK

    Ahmed Zoha

Rights and permissions

Reprints and permissions

Copyright information

© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kumar, H.G., Lawn, A.R., Prabhakaran, B., Katz, W.F. (2022). Opti-Speech-VMT: Implementation and Evaluation. In: Ur Rehman, M., Zoha, A. (eds) Body Area Networks. Smart IoT and Big Data for Intelligent Health Management. BODYNETS 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 420. Springer, Cham. https://doi.org/10.1007/978-3-030-95593-9_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-95593-9_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-95592-2

  • Online ISBN: 978-3-030-95593-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation