Abstract
Speech intelligibility is currently measured by scoring how well a person can identify a speech signal. The results of such behavioral measures reflect neural processing of the speech signal, but are also influenced by language processing, motivation, and memory. Very often, electrophysiological measures of hearing give insight in the neural processing of sound. However, in most methods, non-speech stimuli are used, making it hard to relate the results to behavioral measures of speech intelligibility. The use of natural running speech as a stimulus in electrophysiological measures of hearing is a paradigm shift which allows to bridge the gap between behavioral and electrophysiological measures. Here, by decoding the speech envelope from the electroencephalogram, and correlating it with the stimulus envelope, we demonstrate an electrophysiological measure of neural processing of running speech. We show that behaviorally measured speech intelligibility is strongly correlated with our electrophysiological measure. Our results pave the way towards an objective and automatic way of assessing neural processing of speech presented through auditory prostheses, reducing confounds such as attention and cognitive capabilities. We anticipate that our electrophysiological measure will allow better differential diagnosis of the auditory system, and will allow the development of closed-loop auditory prostheses that automatically adapt to individual users.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aiken SJ, Picton TW (2008) Human cortical responses to the speech envelope. Ear Hear 29(2):139–157. https://doi.org/10.1097/AUD.0b013e31816453dc
Anderson S, Parbery-Clark A, White-Schwoch T, Kraus N (2013) Auditory brainstem response to complex sounds predicts self-reported speech-in-noise performance. J Speech Lang Hear Res 56(1):31–43. https://doi.org/10.1044/1092-4388(2012/12-0043)
Biesmans W, Das N, Francart T, Bertrand A (2017) Auditory-inspired speech envelope extraction methods for improved eeg-based auditory attention detection in a cocktail party scenario. IEEE Trans Neural Syst Rehabil Eng 25(5):402–412. https://doi.org/10.1109/TNSRE.2016.2571900
Di Liberto GM, O’Sullivan JA, Lalor EC (2015) Low-frequency cortical entrainment to speech reflects phoneme-level processing. Curr Biol 25(19):2457–2465. https://doi.org/10.1016/j.cub.2015.08.030
Dillon H (2012) Hearing aids. Thieme, Stuttgart
Ding N, Simon JZ (2011) Neural coding of continuous speech in auditory cortex during monaural and dichotic listening. J Neurophysiol 107(1):78–89. https://doi.org/10.1152/jn.00297.2011
Ding N, Simon JZ (2012) Emergence of neural encoding of auditory objects while listening to competing speakers. Proc Natl Acad Sci 109(29):11,854–11,859. https://doi.org/10.1073/pnas.1205381109
Ding N, Simon JZ (2013) Adaptive temporal encoding leads to a background-insensitive cortical representation of speech. J Neurosci 33(13):5728–5735. https://doi.org/10.1523/JNEUROSCI.5297-12.2013
Ding N, Simon JZ (2014) Cortical entrainment to continuous speech: functional roles and interpretations. Front Hum Neurosci 8:311
Ding N, Chatterjee M, Simon JZ (2014) Robust cortical entrainment to the speech envelope relies on the spectro-temporal fine structure. NeuroImage 88:41–46. https://doi.org/10.1016/j.neuroimage.2013.10.054
Doelling KB, Arnal LH, Ghitza O, Poeppel D (2014) Acoustic landmarks drive delta–theta oscillations to enable speech comprehension by facilitating perceptual parsing. NeuroImage 85:761–768. https://doi.org/10.1016/j.neuroimage.2013.06.035
Drullman R, Festen JM, Plomp R (1994a) Effect of reducing slow temporal modulations on speech reception. J Acoust Soc Am 95(5):2670–2680. https://doi.org/10.1121/1.409836
Drullman R, Festen JM, Plomp R (1994b) Effect of temporal envelope smearing on speech reception. J Acoust Soc Am 95(2):1053–1064. https://doi.org/10.1121/1.408467
Edwards E, Chang EF (2013) Syllabic (2–5 hz) and fluctuation (1–10 hz) ranges in speech and auditory processing. Hear Res 305:113–134. https://doi.org/10.1016/j.heares.2013.08.017
Francart T, van Wieringen A, Wouters J (2008) APEX 3: a multi-purpose test platform for auditory psychophysical experiments. J Neurosci Methods 172(2):283–293. https://doi.org/10.1016/j.jneumeth.2008.04.020
Horton C, Srinivasan R, D’Zmura M (2014) Envelope responses in single-trial eeg indicate attended speaker in a “cocktail party”. J Neural Eng 11(4):046,015. https://doi.org/10.1088/1741-2560/11/4/046015
Hullett PW, Hamilton LS, Mesgarani N, Schreiner CE, Chang EF (2016) Human superior temporal gyrus organization of spectrotemporal modulation tuning derived from speech stimuli. J Neurosci 36(6):2014–2026. https://doi.org/10.1523/JNEUROSCI.1779-15.2016
Kong YY, Somarowthu A, Ding N (2015) Effects of spectral degradation on attentional modulation of cortical auditory responses to continuous speech. J Assoc Res Otolaryngol 16(6):783–796. https://doi.org/10.1007/s10162-015-0540-x
Lalor EC, Pearlmutter BA, Reilly RB, McDarby G, Foxe JJ (2006) The vespa: a method for the rapid estimation of a visual evoked potential. NeuroImage 32(4):1549–1561. https://doi.org/10.1016/j.neuroimage.2006.05.054
Lalor EC, Power AJ, Reilly RB, Foxe JJ (2009) Resolving precise temporal processing properties of the auditory system using continuous stimuli. J Neurophysiol 102(1):349–359. https://doi.org/10.1152/jn.90896.2008
Luts H, Jansen S, Dreschler W, Wouters J (2015) Development and normative data for the Flemish/Dutch matrix test. KU Leuven. https://lirias.kuleuven.be/bitstream/123456789/474335/1/Documentation+Flemish-Dutch+Matrix_December2014.pdf. Accessed 5 Feb 2018
McGee TJ, Clemis JD (1980) The approximation of audiometric thresholds by auditory brain stem responses. Otolaryngol Head Neck Surg 88(3):295–303. https://doi.org/10.1177/019459988008800319
O’Sullivan JA, Power AJ, Mesgarani N, Rajaram S, Foxe JJ, Shinn-Cunningham BG, Slaney M, Shamma SA, Lalor EC (2015) Attentional selection in a cocktail party environment can be decoded from single-trial eeg. Cereb Cortex 25(7):1697–1706. https://doi.org/10.1093/cercor/bht355
Pasley BN, David SV, Mesgarani N, Flinker A, Shamma SA, Crone NE, Knight RT, Chang EF (2012) Reconstructing speech from human auditory cortex. PLoS Biol 10(1):e1001,251. https://doi.org/10.1371/journal.pbio.1001251
Peelle JE, Davis MH (2012) Neural oscillations carry speech rhythm through to comprehension. Front Psychol 3:320
Picton TW, Dimitrijevic A, Perez-Abalo MC, Van Roon P (2005) Estimating audiometric thresholds using auditory steady-state responses. J Am Acad Audiol 16(3):140–156. https://doi.org/10.3766/jaaa.16.3.3
Presacco A, Simon JZ, Anderson S (2016) Evidence of degraded representation of speech in noise, in the aging midbrain and cortex. J Neurophysiol 116(5):2346–2355. https://doi.org/10.1152/jn.00372.2016
Shannon RV, Zeng FG, Kamath V, Wygonski J, Ekelid M (1995) Speech recognition with primarily temporal cues. Science 270(5234):303–304. https://doi.org/10.1126/science.270.5234.303
Søndergaard PL, Majdak P (2013) The auditory modeling toolbox. In: Blauert J (ed) The technology of binaural listening. Springer, Berlin, Heidelberg, pp 33–56. https://doi.org/10.1007/978-3-642-37762-4
Søndergaard PL, Torrésani B, Balazs P (2012) The linear time frequency analysis toolbox. Int J Wavelets Multiresolution Inf Process 10(4):1250032. https://doi.org/10.1142/S0219691312500324
Woodfield A, Akeroyd MA (2010) The role of segmentation difficulties in speech-in-speech understanding in older and hearing-impaired adults. J Acoust Soc Am 128(1):EL26–EL31. https://doi.org/10.1121/1.3443570
Yang M, Sheth SA, Schevon CA, McKhann II GM, Mesgarani N (2015) Speech reconstruction from human auditory cortex with deep neural networks. In: Sixteenth Annual Conference of the International Speech Communication Association, Dresden, Germany, pp 1121–1125
Acknowledgements
The authors thank Lise Goris and Eline Verschueren for their help with the data acquisition.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Before each experiment, the subjects signed an informed consent form approved by the Medical Ethics Committee UZ KU Leuven/Research (KU Leuven) with reference S59040.
Conflict of Interest
The authors declare that they have no conflict of interest.
Additional information
Financial support was provided by the KU Leuven Special Research Fund under grant OT/14/119 to Tom Francart. Research funded by a PhD grant of the Research Foundation Flanders (FWO). This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 637424), from the YouReCa Junior Mobility Programme under grant JUMO/15/034 to Jonas Vanthornhout, from NIH grant R01-DC-014085 to Jonathan Z. Simon and from FWO project grant G.0662.13.
Rights and permissions
About this article
Cite this article
Vanthornhout, J., Decruy, L., Wouters, J. et al. Speech Intelligibility Predicted from Neural Entrainment of the Speech Envelope. JARO 19, 181–191 (2018). https://doi.org/10.1007/s10162-018-0654-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10162-018-0654-z