Advertisement

Speech Intelligibility Predicted from Neural Entrainment of the Speech Envelope

  • Jonas Vanthornhout
  • Lien Decruy
  • Jan Wouters
  • Jonathan Z. Simon
  • Tom Francart
Research Article

Abstract

Speech intelligibility is currently measured by scoring how well a person can identify a speech signal. The results of such behavioral measures reflect neural processing of the speech signal, but are also influenced by language processing, motivation, and memory. Very often, electrophysiological measures of hearing give insight in the neural processing of sound. However, in most methods, non-speech stimuli are used, making it hard to relate the results to behavioral measures of speech intelligibility. The use of natural running speech as a stimulus in electrophysiological measures of hearing is a paradigm shift which allows to bridge the gap between behavioral and electrophysiological measures. Here, by decoding the speech envelope from the electroencephalogram, and correlating it with the stimulus envelope, we demonstrate an electrophysiological measure of neural processing of running speech. We show that behaviorally measured speech intelligibility is strongly correlated with our electrophysiological measure. Our results pave the way towards an objective and automatic way of assessing neural processing of speech presented through auditory prostheses, reducing confounds such as attention and cognitive capabilities. We anticipate that our electrophysiological measure will allow better differential diagnosis of the auditory system, and will allow the development of closed-loop auditory prostheses that automatically adapt to individual users.

Keywords

Neural decoding Auditory evoked potentials EEG measures Objective measures Speech understanding 

Notes

Acknowledgements

The authors thank Lise Goris and Eline Verschueren for their help with the data acquisition.

Compliance with Ethical Standards

Before each experiment, the subjects signed an informed consent form approved by the Medical Ethics Committee UZ KU Leuven/Research (KU Leuven) with reference S59040.

Conflict of Interest

The authors declare that they have no conflict of interest.

References

  1. Aiken SJ, Picton TW (2008) Human cortical responses to the speech envelope. Ear Hear 29(2):139–157.  https://doi.org/10.1097/AUD.0b013e31816453dc CrossRefPubMedGoogle Scholar
  2. Anderson S, Parbery-Clark A, White-Schwoch T, Kraus N (2013) Auditory brainstem response to complex sounds predicts self-reported speech-in-noise performance. J Speech Lang Hear Res 56(1):31–43.  https://doi.org/10.1044/1092-4388(2012/12-0043) CrossRefPubMedGoogle Scholar
  3. Biesmans W, Das N, Francart T, Bertrand A (2017) Auditory-inspired speech envelope extraction methods for improved eeg-based auditory attention detection in a cocktail party scenario. IEEE Trans Neural Syst Rehabil Eng 25(5):402–412.  https://doi.org/10.1109/TNSRE.2016.2571900
  4. Di Liberto GM, O’Sullivan JA, Lalor EC (2015) Low-frequency cortical entrainment to speech reflects phoneme-level processing. Curr Biol 25(19):2457–2465.  https://doi.org/10.1016/j.cub.2015.08.030 CrossRefPubMedGoogle Scholar
  5. Dillon H (2012) Hearing aids. Thieme, StuttgartGoogle Scholar
  6. Ding N, Simon JZ (2011) Neural coding of continuous speech in auditory cortex during monaural and dichotic listening. J Neurophysiol 107(1):78–89.  https://doi.org/10.1152/jn.00297.2011 CrossRefPubMedPubMedCentralGoogle Scholar
  7. Ding N, Simon JZ (2012) Emergence of neural encoding of auditory objects while listening to competing speakers. Proc Natl Acad Sci 109(29):11,854–11,859.  https://doi.org/10.1073/pnas.1205381109 CrossRefGoogle Scholar
  8. Ding N, Simon JZ (2013) Adaptive temporal encoding leads to a background-insensitive cortical representation of speech. J Neurosci 33(13):5728–5735.  https://doi.org/10.1523/JNEUROSCI.5297-12.2013 CrossRefPubMedPubMedCentralGoogle Scholar
  9. Ding N, Simon JZ (2014) Cortical entrainment to continuous speech: functional roles and interpretations. Front Hum Neurosci 8:311CrossRefPubMedPubMedCentralGoogle Scholar
  10. Ding N, Chatterjee M, Simon JZ (2014) Robust cortical entrainment to the speech envelope relies on the spectro-temporal fine structure. NeuroImage 88:41–46.  https://doi.org/10.1016/j.neuroimage.2013.10.054 CrossRefPubMedGoogle Scholar
  11. Doelling KB, Arnal LH, Ghitza O, Poeppel D (2014) Acoustic landmarks drive delta–theta oscillations to enable speech comprehension by facilitating perceptual parsing. NeuroImage 85:761–768.  https://doi.org/10.1016/j.neuroimage.2013.06.035 CrossRefPubMedGoogle Scholar
  12. Drullman R, Festen JM, Plomp R (1994a) Effect of reducing slow temporal modulations on speech reception. J Acoust Soc Am 95(5):2670–2680.  https://doi.org/10.1121/1.409836 CrossRefPubMedGoogle Scholar
  13. Drullman R, Festen JM, Plomp R (1994b) Effect of temporal envelope smearing on speech reception. J Acoust Soc Am 95(2):1053–1064.  https://doi.org/10.1121/1.408467 CrossRefPubMedGoogle Scholar
  14. Edwards E, Chang EF (2013) Syllabic (2–5 hz) and fluctuation (1–10 hz) ranges in speech and auditory processing. Hear Res 305:113–134.  https://doi.org/10.1016/j.heares.2013.08.017 CrossRefPubMedGoogle Scholar
  15. Francart T, van Wieringen A, Wouters J (2008) APEX 3: a multi-purpose test platform for auditory psychophysical experiments. J Neurosci Methods 172(2):283–293.  https://doi.org/10.1016/j.jneumeth.2008.04.020 CrossRefPubMedGoogle Scholar
  16. Horton C, Srinivasan R, D’Zmura M (2014) Envelope responses in single-trial eeg indicate attended speaker in a “cocktail party”. J Neural Eng 11(4):046,015.  https://doi.org/10.1088/1741-2560/11/4/046015 CrossRefGoogle Scholar
  17. Hullett PW, Hamilton LS, Mesgarani N, Schreiner CE, Chang EF (2016) Human superior temporal gyrus organization of spectrotemporal modulation tuning derived from speech stimuli. J Neurosci 36(6):2014–2026.  https://doi.org/10.1523/JNEUROSCI.1779-15.2016 CrossRefPubMedPubMedCentralGoogle Scholar
  18. Kong YY, Somarowthu A, Ding N (2015) Effects of spectral degradation on attentional modulation of cortical auditory responses to continuous speech. J Assoc Res Otolaryngol 16(6):783–796.  https://doi.org/10.1007/s10162-015-0540-x CrossRefPubMedPubMedCentralGoogle Scholar
  19. Lalor EC, Pearlmutter BA, Reilly RB, McDarby G, Foxe JJ (2006) The vespa: a method for the rapid estimation of a visual evoked potential. NeuroImage 32(4):1549–1561.  https://doi.org/10.1016/j.neuroimage.2006.05.054 CrossRefPubMedGoogle Scholar
  20. Lalor EC, Power AJ, Reilly RB, Foxe JJ (2009) Resolving precise temporal processing properties of the auditory system using continuous stimuli. J Neurophysiol 102(1):349–359.  https://doi.org/10.1152/jn.90896.2008 CrossRefPubMedGoogle Scholar
  21. Luts H, Jansen S, Dreschler W, Wouters J (2015) Development and normative data for the Flemish/Dutch matrix test. KU Leuven. https://lirias.kuleuven.be/bitstream/123456789/474335/1/Documentation+Flemish-Dutch+Matrix_December2014.pdf. Accessed 5 Feb 2018
  22. McGee TJ, Clemis JD (1980) The approximation of audiometric thresholds by auditory brain stem responses. Otolaryngol Head Neck Surg 88(3):295–303.  https://doi.org/10.1177/019459988008800319 CrossRefPubMedGoogle Scholar
  23. O’Sullivan JA, Power AJ, Mesgarani N, Rajaram S, Foxe JJ, Shinn-Cunningham BG, Slaney M, Shamma SA, Lalor EC (2015) Attentional selection in a cocktail party environment can be decoded from single-trial eeg. Cereb Cortex 25(7):1697–1706.  https://doi.org/10.1093/cercor/bht355
  24. Pasley BN, David SV, Mesgarani N, Flinker A, Shamma SA, Crone NE, Knight RT, Chang EF (2012) Reconstructing speech from human auditory cortex. PLoS Biol 10(1):e1001,251.  https://doi.org/10.1371/journal.pbio.1001251 CrossRefGoogle Scholar
  25. Peelle JE, Davis MH (2012) Neural oscillations carry speech rhythm through to comprehension. Front Psychol 3:320CrossRefPubMedPubMedCentralGoogle Scholar
  26. Picton TW, Dimitrijevic A, Perez-Abalo MC, Van Roon P (2005) Estimating audiometric thresholds using auditory steady-state responses. J Am Acad Audiol 16(3):140–156.  https://doi.org/10.3766/jaaa.16.3.3 CrossRefPubMedGoogle Scholar
  27. Presacco A, Simon JZ, Anderson S (2016) Evidence of degraded representation of speech in noise, in the aging midbrain and cortex. J Neurophysiol 116(5):2346–2355.  https://doi.org/10.1152/jn.00372.2016 CrossRefPubMedPubMedCentralGoogle Scholar
  28. Shannon RV, Zeng FG, Kamath V, Wygonski J, Ekelid M (1995) Speech recognition with primarily temporal cues. Science 270(5234):303–304.  https://doi.org/10.1126/science.270.5234.303 CrossRefPubMedGoogle Scholar
  29. Søndergaard PL, Majdak P (2013) The auditory modeling toolbox. In: Blauert J (ed) The technology of binaural listening. Springer, Berlin, Heidelberg, pp 33–56.  https://doi.org/10.1007/978-3-642-37762-4 CrossRefGoogle Scholar
  30. Søndergaard PL, Torrésani B, Balazs P (2012) The linear time frequency analysis toolbox. Int J Wavelets Multiresolution Inf Process 10(4):1250032.  https://doi.org/10.1142/S0219691312500324
  31. Woodfield A, Akeroyd MA (2010) The role of segmentation difficulties in speech-in-speech understanding in older and hearing-impaired adults. J Acoust Soc Am 128(1):EL26–EL31.  https://doi.org/10.1121/1.3443570 CrossRefPubMedGoogle Scholar
  32. Yang M, Sheth SA, Schevon CA, McKhann II GM, Mesgarani N (2015) Speech reconstruction from human auditory cortex with deep neural networks. In: Sixteenth Annual Conference of the International Speech Communication Association, Dresden, Germany, pp 1121–1125Google Scholar

Copyright information

© Association for Research in Otolaryngology 2018

Authors and Affiliations

  1. 1.Department of Neurosciences, ExpORLKU Leuven - University of LeuvenLeuvenBelgium
  2. 2.Department of Electrical and Computer EngineeringUniversity of MarylandCollege ParkUSA
  3. 3.Department of BiologyUniversity of MarylandCollege ParkUSA
  4. 4.Institute for Systems ResearchUniversity of MarylandCollege ParkUSA

Personalised recommendations