Abstract
Automatic speech recognition is the main form of man–machine communication. Recently, several studies have shown the ability to automatically recognize speech based on electromyography (EMG) signals of the facial muscles using machine learning methods. The objective of this study was to utilize machine learning methods for automatic identification of speech based on EMG signals. EMG signals from three facial muscles were measured from four healthy female subjects while pronouncing seven different words 50 times. Short time Fourier transform features were extracted from the EMG data. Principle component analysis (PCA) and locally linear embedding (LLE) methods were applied and compared for reducing the dimensions of the EMG data. K-nearest-neighbors was used to examine the ability to identify different word sets of a subject based on his own dataset, and to identify words of one subject based on another subject's dataset, utilizing an affine transformation for aligning between the reduced feature spaces of two subjects. The PCA and LLE achieved average recognizing rate of 81% for five words-sets in the single-subject approach. The best average recognition success rates for three and five words-sets were 88.8% and 74.6%, respectively, for the multi-subject classification approach. Both the PCA and LLE achieved satisfactory classification rates for both the single-subject and multi-subject approaches. The multi-subject classification approach enables robust classification of words recorded from a new subject based on another subject’s dataset and thus can be applicable for people who have lost their ability to speak.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Betts BJ, Binsted K, Jorgensen C (2006) Small-vocabulary speech recognition using surface electromyography. Interact Comput 18(6):1242–1259
Chan AD, Englehart K, Hudgins B, Lovely DF (2001) Myo-electric signals to augment speech recognition. Med Biol Eng Compu 39(4):500–504
Chan AD, Englehart K, Hudgins B, Lovely DF (2002) A multi-expert speech recognition system using acoustic and myoelectric signals. In: Proceedings of the Second Joint 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society—Engineering in Medicine and Biology 1: 72–73 IEEE
Denby B, Schultz T, Honda K, Hueber T, Gilbert JM, Brumberg JS (2010) Silent speech interfaces. Speech Commun 52(4):270–287
Dhakal P, Damacharla P, Javaid AY, Devabhaktuni V (2019) A near real-time automatic speaker recognition architecture for voice-based user interface. Mach Learn Knowl Extr 1(1):504–520
Ding R, Larson CR, Logemann JA, Rademaker AW (2002) Surface electromyographic and electroglottographic studies in normal subjects under two swallow conditions: normal and during the Mendelsohn manuever. Dysphagia 17(1):1–12
Jolliffe IT (1986) Principal components in regression analysis. Principal component analysis. Springer, New York, pp 129–155
Jong NS, Phukpattaranont P (2019) A speech recognition system based on electromyography for the rehabilitation of dysarthric patients: a Thai syllable study. Biocybern Biomed Eng 39(1):234–245
Jorgensen C, Binsted K (2005) Web browser control using EMG based sub vocal speech recognition. In: Proceedings of the 38th Annual Hawaii International Conference on System Sciences 294c–294c IEEE.
Jorgensen C, Dusan S (2010) Speech interfaces based upon surface electromyography. Speech Commun 52(4):354–366
Jorgensen C, Lee DD, Agabont S (2003) Sub auditory speech recognition based on EMG signals. In: Proceedings of the International Joint Conference on Neural Networks 4:3128–3133 IEEE
Jou SC, Schultz T, Walliczek M, Kraft F, Waibel A (2006) Towards continuous speech recognition using surface electromyography. In: Ninth International Conference on Spoken Language Processing
Konrad P (2005) The ABC of EMG: A practical introduction to kinesiological electromyography, 30–35
Lafon S, Keller Y, Coifman RR (2006) Data fusion and multicue data matching by diffusion maps. IEEE Trans Pattern Anal Mach Intell 28(11):1784–1797
Lapatki BG, Stegeman DF, Jonas IE (2003) A surface EMG electrode for the simultaneous observation of multiple facial muscles. J Neurosci Methods 123(2):117–128
Lee HY, Hong JS, Lee KC, Shin YK, Cho SR (2015) Changes in hyolaryngeal movement and swallowing function after neuromuscular electrical stimulation in patients with dysphagia. Ann Rehabil Med 39(2):199
Liu Y, Zhang Y, Yu Z, Zeng M (2016) Incremental supervised locally linear embedding for machinery fault diagnosis. Eng Appl Artif Intell 50:60–70
Manabe H, Zhang Z (2004) Multi-stream HMM for EMG-based speech recognition. In: The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society 2:4389–4392 IEEE
Meltzner GS, Sroka J, Heaton JT, Gilmore LD, Colby G, Roy S, Chen N, Luca CJ (2008) Speech recognition for vocalized and subvocal modes of production using surface EMG signals from the neck and face. In: Ninth Annual Conference of the International Speech Communication Association
Meltzner GS, Heaton JT, Deng Y, De Luca G, Roy SH, Kline JC (2018) Development of sEMG sensors and algorithms for silent speech recognition. J Neural Eng 15(4):046031
Morse MS, O’Brien EM (1986) Research summary of a scheme to ascertain the availability of speech information in the myoelectric signals of neck and head muscles using surface electrodes. Comput Biol Med 16(6):399–410
Pearson K (1901) On lines of closes fit to system of points in space, London, E dinb. Dublin Philos Mag J Sci 2:559–572
Phinyomark A, Scheme E (2018) EMG pattern recognition in the era of big data and deep learning. Big Data Cogn Comput 2(3):21
Rabin N, Golan M, Singer G, Kleper D (2019) Modeling and analysis of students’ performance trajectories using diffusion maps and kernel two-sample tests. Eng Appl Artif Intell 85:492–503
Rabin N, Kahlon M, Malayev S, Ratnovsky A (2020) Classification of human hand movements based on EMG signals using nonlinear dimensionality reduction and data fusion techniques. Expert Syst Appl 149:113281
Ratnovsky A, Carmeli YN, Elad D, Zaretsky U, Dollberg S, Mandel D (2013) Analysis of facial and inspiratory muscles performance during breastfeeding. Technol Health Care 21(5):511–520
Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326
Srisuwan N, Phukpattaranont P, Limsakul C (2018) Comparison of feature evaluation criteria for speech recognition based on electromyography. Med Biol Eng Comp 56(6):1041–1051
Sugie N, Tsunoda K (1985) A speech prosthesis employing a speech synthesizer-vowel discrimination from perioral muscle activities and vowel production. IEEE Trans Biomed Eng 7:485–490
Tsai AC, Luh JJ, Lin TT (2015) A novel STFT-ranking feature of multi-channel EMG for motion pattern recognition. Expert Syst Appl 42(7):3327–3341
Wand M, Schultz T (2009) Towards speaker-adaptive speech recognition based on surface electromyography. In: Biosignals, pp 155–162
Wand M, Schultz T (2011) Session-independent EMG-based Speech Recognition. In: Biosignals pp. 295–300.
Wand M, Schultz T (2014) Towards real-life application of EMG-based speech recognition by using unsupervised adaptation. In: Fifteenth Annual Conference of the International Speech Communication Association
Wand M, Schmidhuber J (2016) Deep neural network frontend for continuous EMG-based speech recognition. In: Interspeech, pp 3032–3036
Wand M, Janke M, Schultz T (2014) Tackling speaking mode varieties in EMG-based speech recognition. IEEE Trans Biomed Eng 61(10):2515–2526
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ratnovsky, A., Malayev, S., Ratnovsky, S. et al. EMG-based speech recognition using dimensionality reduction methods. J Ambient Intell Human Comput 14, 597–607 (2023). https://doi.org/10.1007/s12652-021-03315-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-021-03315-5