Myo-To-Speech - Evolving Fuzzy-Neural Network Prediction of Speech Utterances from Myoelectric Signals

  • Mario MalcangiEmail author
  • Giovanni Felisati
  • Alberto Saibene
  • Enrico Alfonsi
  • Mauro Fresia
  • Roberto Maffioletti
  • Hao Quan
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 893)


Voice rehabilitation is needed after several diseases, when a subject’s vocal ability is compromised by surgical interference or removal of phonation organs (e.g. the larynx), by neural degeneration or by neurological injury to the motor component of the motor-speech system in the phonation area of the brain (e.g. dysarthria in Parkinson disease). A novel approach to voice rehabilitation consists of predicting the phonetic control sequence of the voice-production apparatus (larynx, tongue, etc.) by drawing inferences on the basis of myoelectric (EMG) signals captured by a set of contact electrodes, applied to the neck area of a subject with important phonatory alteration (e.g. laryngectomised) and intact neural control. The inference paradigm is based on an EFuNN (Evolving Fuzzy Neural Network) that has been trained to use the sampled EMG signal to predict the phoneme that corresponds to the motor control of the sublingual muscle movements monitored at phonation time. A phoneme-to-speech synthesizer generates audio output corresponding to the utterance the subject has tried to enunciate.


EFuNN Evolving Fuzzy Neural Network Voice dysarthria Voice rehabilitation Myoelectric signal 



A special acknowledgment is due to Prof. Nikola Kasabov, Auckland University of Technology, Director KEDRI – Knowledge Engineering and Discovery Research Institute, for his invaluable suggestions on how to get the most from the EFuNN’s evolving capabilities.

Acknowledgment is also due to Jan Hein Broeders (Analog Devices’ healthcare business-development manager for EMEA) for his precious support and expertise in hardware prototyping, especially for the analog front-end (AFE) subsystem.


  1. 1.
    Balasubramanian, T.: Voice rehabilitation following total laryngectomy. Otoryngology Online J. 5(1), 5 (2015)Google Scholar
  2. 2.
    Goldstain, E.A., Heaton, J.T., Kobler, B., Stanley, G.B., Hillman, R.E.: Design and implementation of a hand-free electrolarynx device controlled by neck strap muscle electromyographic activity. IEEE Trans. Biomed. Eng. 51, 325–332 (2004)CrossRefGoogle Scholar
  3. 3.
    Titze, I.R.: The physics of small-amplitude oscillation of the vocal folds. J. Acoust. Soc. Am. 83, 1536–1552 (1988). PMID 3372869CrossRefGoogle Scholar
  4. 4.
    Lucero, J.C.: The minimum lung pressure to sustain vocal fold oscillation. J. Acoust. Soc. Am. 98, 779–784 (1995)CrossRefGoogle Scholar
  5. 5.
    Lucero, J.C.: Optimal glottal configuration for ease of phonation. J. Voice 12, 151–158 (1998)CrossRefGoogle Scholar
  6. 6.
    Mor, N., Simonyan, K., Blitzer, A.: Central voice production and pathophysiology of spasmodic dysphonia. Laryngoscope 128(1), 177–183 (2018). Epub 23 May 2017, Review (2018)CrossRefGoogle Scholar
  7. 7.
    Jürgens, U.: Neural pathways underlying vocal control. Neurosci. Biobehav. Rev. 26, 235–258 (2002)CrossRefGoogle Scholar
  8. 8.
    Jürgens, U.: A study of the central control of vocalization using the squirrel monkey. Med. Eng. Phys. 24, 473–477 (2002)CrossRefGoogle Scholar
  9. 9.
    Vihma, Mn., de Boysson-Bardies, B.: The nature and origins of ambient language influence on infant vocal production and early words. Phonetica 51, 159–169 (1994)CrossRefGoogle Scholar
  10. 10.
    Mac Neilage, P.F.: The frame/content theory of evolution of speech production. Behav. Brain Sci. 21, 499–511 (1998)Google Scholar
  11. 11.
    Kuypers, H.G.: Cortico-bulbar connexions to the pons and lower brainstem in man: an anatomical study. Brain 81, 364–388 (1958)CrossRefGoogle Scholar
  12. 12.
    Haslinger, B., Erhard, P., Dresel, C., Castrop, F., Roettinger, M., Ceballos-Baumann, A.O.: Silent event-related, fMRI reveals reduced sensorimotor activation in laryngeal dystonia. Neurology 65, 1562–1569 (2005)CrossRefGoogle Scholar
  13. 13.
    Simonyan, K., Ludlow, C.L.: Abnormal activation of the primary somatosensory cortex in spasmodic dysphonia: an fMRI study. Cereb. Cortex 20, 2749–2759 (2010)CrossRefGoogle Scholar
  14. 14.
    Rödel, R.M., et al.: Human cortical motor representation of the larynx as assessed by transcranial magnetic stimulation (TMS). Laryngoscope 114, 918–922 (2004)CrossRefGoogle Scholar
  15. 15.
    Ludlow, C.L.: Spasmodic dysphonia: a laryngeal control disorder specific to speech. J. Neurosci. 31(3), 793–797 (2011)CrossRefGoogle Scholar
  16. 16.
    Kasabov, N.: Evolving Connectionist Systems: The Knowledge Engineering Approach. Springer, Heidelberg (2007). Scholar
  17. 17.
    Kasabov, N.: EFuNN, IEEE Tr SMC (2001)Google Scholar
  18. 18.
    Kasabov, N.: Evolving fuzzy neural networks – algorithms, applications and biological motivation. In: Yamakawa, T., Matsumoto, G. (eds.) Methodologies for the Conception, Design and Application of the Soft Computing, pp. 271–274. World Computing (1998)Google Scholar
  19. 19.
  20. 20.
    Zahner, M., Janke, M., Wand, M., Schultz, T.: Conversion from facial myoelectric signals to speech: a unit selection approach. In: Interspeech, pp. 1184–1188 (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Mario Malcangi
    • 1
    Email author
  • Giovanni Felisati
    • 2
  • Alberto Saibene
    • 2
  • Enrico Alfonsi
    • 3
  • Mauro Fresia
    • 3
  • Roberto Maffioletti
    • 1
  • Hao Quan
    • 1
  1. 1.Computer Science DepartmentUniversità degli Studi di MilanoMilanItaly
  2. 2.Department of Heath ScienceUniversità degli Studi di MilanoMilanItaly
  3. 3.IRCCS Mondino FoundationPaviaItaly

Personalised recommendations