Predictions of Speech Chimaera Intelligibility Using Auditory Nerve Mean-Rate and Spike-Timing Neural Cues

  • Michael R. Wirtzfeld
  • Rasha A. Ibrahim
  • Ian C. Bruce
Research Article


Perceptual studies of speech intelligibility have shown that slow variations of acoustic envelope (ENV) in a small set of frequency bands provides adequate information for good perceptual performance in quiet, whereas acoustic temporal fine-structure (TFS) cues play a supporting role in background noise. However, the implications for neural coding are prone to misinterpretation because the mean-rate neural representation can contain recovered ENV cues from cochlear filtering of TFS. We investigated ENV recovery and spike-time TFS coding using objective measures of simulated mean-rate and spike-timing neural representations of chimaeric speech, in which either the ENV or the TFS is replaced by another signal. We (a) evaluated the levels of mean-rate and spike-timing neural information for two categories of chimaeric speech, one retaining ENV cues and the other TFS; (b) examined the level of recovered ENV from cochlear filtering of TFS speech; (c) examined and quantified the contribution to recovered ENV from spike-timing cues using a lateral inhibition network (LIN); and (d) constructed linear regression models with objective measures of mean-rate and spike-timing neural cues and subjective phoneme perception scores from normal-hearing listeners. The mean-rate neural cues from the original ENV and recovered ENV partially accounted for perceptual score variability, with additional variability explained by the recovered ENV from the LIN-processed TFS speech. The best model predictions of chimaeric speech intelligibility were found when both the mean-rate and spike-timing neural cues were included, providing further evidence that spike-time coding of TFS cues is important for intelligibility when the speech envelope is degraded.


intelligibility envelope temporal fine structure recovered envelope mean-rate spike-timing chimaera 



The authors thank Laurel Carney and Hubert de Bruin for advice on the experiment design; Sue Becker for the use of her amplifier, headphones, and testing room; Malcolm Pilgrim and Timothy Zeyl for assistance with running the experiment; Dan Bosnyak and Dave Thompson for assistance with the acoustic calibration; Jason Boulet and the anonymous reviewers for very helpful comments on earlier versions of the manuscript; and the subjects for their participation. This research was supported by the Natural Sciences and Engineering Research Council of Canada (Discovery Grant No. 261736), and the human experiments were approved by the McMaster Research Ethics Board (#2010 051).

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.


  1. Apoux F, Yoho SE, Youngdahl CL, Healy E (2013) Can envelope recovery account for speech recognition based on temporal fine structure? Proceedings of Meetings on Acoustics 19(1):050072CrossRefGoogle Scholar
  2. Baer T, Moore BCJ, Gatehouse S (1993) Spectral contrast enhancement of speech in noise for listeners with sensorineural hearing impairment: effects on intelligibility, quality, and response times. J Rehabil Res Dev 30(1):49–72PubMedGoogle Scholar
  3. Bentsen T, Harte JM, Dau T (2011) Human cochlear tuning estimates from stimulus-frequency otoacoustic emissions. J Acoust Soc Am 129(6):3797–3807CrossRefPubMedGoogle Scholar
  4. Bondy J, Bruce IC, Becker S, Haykin S (2004) Predicting speech intelligibility from a population of neurons. In: Thrun S, Saul L, Schölkopf B (eds) Advances in neural information processing systems 16. MIT Press, Cambridge, MA, pp 1409–1416Google Scholar
  5. Bruce IC (2004) Physiological assessment of contrast-enhancing frequency shaping and multiband compression in hearing aids. Physiol Meas 25(4):945–956CrossRefPubMedGoogle Scholar
  6. Bruce IC, Dinath F, Zeyl T (2007) Insights into optimal phonemic compression from a computational model of the auditory periphery. In: Auditory Signal Processing in Hearing-Impaired Listeners, Internationl Symposium on Audiological and Auditory Research (ISAAR), p 73–81Google Scholar
  7. Bruce IC, Léger AC, Moore BC, Lorenzi C (2013) Physiological prediction of masking release for normal-hearing and hearing-impaired listeners. Proceedings of Meetings on Acoustics: ICA 2013 Montreal, Acoustical Society of America 133(5):1–8Google Scholar
  8. Bruce IC, Léger AC, Wirtzfeld MR, Moore BC, Lorenzi C (2015) Spike-time coding and auditory-nerve degeneration best explain speech intelligibility in noise for normal and near-normal low-frequency hearing. In: Abstracts of the 38th ARO Midwinter Research MeetingGoogle Scholar
  9. Burnham KP, Anderson DR (2002) Model selection and multimodel inference, a practical information-theoretic approach, 2nd edn. Springer, New YorkGoogle Scholar
  10. Chi T, Gao Y, Guyton MC, Ru P, Shamma S (1999) Spectro-temporal modulation transfer functions and speech intelligibility. J Acoust Soc Am 106(5):2719–2732CrossRefPubMedGoogle Scholar
  11. Davis MH, Johnsrude IS, Hervais-Adelman A, Taylor K, McGettigan C (2005) Lexcial information drives perceptual learning of distorted speech: evidence from the comprehension of noise-vocoded sentences. J Exp Psychol 134(2):222–241CrossRefGoogle Scholar
  12. Delgutte B (1997) Auditory neural processing of speech. The handbook of phonetic sciences pp:507–538Google Scholar
  13. Dinath F, Bruce IC (2008) Hearing aid gain prescriptions balance restoration of auditory nerve mean-rate and spike-timing representations of speech. In: Proceedings of 30th International IEEE Engineering in Medicine and Biology Conference, IEEE, Piscataway, NJ, p 1793–1796Google Scholar
  14. Drullman R (1995) Temporal envelope and fine structure cues for speech intelligibility. J Acoust Soc Am 97(1):585–592CrossRefPubMedGoogle Scholar
  15. Dudley H (1939) The vocoder. Bell Labs Record 17:122–126Google Scholar
  16. Elhilali M, Chi T, Shamma SA (2003)A spectro-temporal modulation index (STMI) for assessment of speech intelligibility. Speech Comm 41(2, 3):331–348Google Scholar
  17. Flanagan JL (1980) Parametric coding of speech spectra. J Acoust Soc Am 68(2):412–419CrossRefGoogle Scholar
  18. Fogerty D, Humes LE (2012)The role of vowel and consonant fundamental frequency, envelope, and temporal fine structure cues to the intelligibility of words and sentences. J Acoust Soc Am 131(2):1490–1501Google Scholar
  19. Franck BAM, Sidonne C, van Kreveld-Bos GM, Dreschler WA, Verschuure H (1999) Evaluation of spectral enhancement in hearing aids, combined with phonemic compression. J Acoust Soc Am 106(3):1452–1464CrossRefPubMedGoogle Scholar
  20. French NR, Steinberg JC (1947) Factors governing the intelligibility of speech sounds. J Acoust Soc Am 19:90–119CrossRefGoogle Scholar
  21. Ghitza O (2001) On the upper cutoff frequency of the auditory critical-band envelope detectors in the context of speech perception. J Acoust Soc Am 110(3):1628–1640CrossRefPubMedGoogle Scholar
  22. Gilbert G, Lorenzi C (2006) The ability of listeners to use recovered envelope cues from speech fine structure. J Acoust Soc Am 119(4):2438–2444CrossRefPubMedGoogle Scholar
  23. Gilbert G, Bergeras I, Voillery D, Lorenzi C (2007) Effects of periodic interruptions on the intelligibility of speech based on temporal fine-structure or envelope cues. J Acoust Soc Am 122(3):1336–1339CrossRefPubMedGoogle Scholar
  24. Greenwood DD (1990) A cochlear frequency-position function for several species–29 years later. J Acoust Soc Am 87(6):2592–2605CrossRefPubMedGoogle Scholar
  25. Hartline HK (1974) Studies on the excitation and inhibition in the retina, Edited by Floyd Ratliff. The Rockefeller University Press, New YorkGoogle Scholar
  26. Heinz MG, Swaminathan J (2009) Quantifying envelope and fine-structure coding in auditory nerve responses to chimaeric speech. J Assoc Res Otolaryngol 10(3):407–423CrossRefPubMedPubMedCentralGoogle Scholar
  27. Hines A, Harte N (2010) Speech intelligibility from image processing. Speech Comm 52(9):736–752CrossRefGoogle Scholar
  28. Hines A, Harte N (2012) Speech intelligibility prediction using a neurogram similarity index measure. Speech Comm 54(2):306–320CrossRefGoogle Scholar
  29. Hopkins K, Moore BCJ, Stone MA (2010) The effects of the addition of low-level, low-noise noise on the intelligibility of sentences processed to remove temporal envelope information. J Acoust Soc Am 128(4):2150–2161CrossRefPubMedGoogle Scholar
  30. Hossain ME, Jassim WA, Zilany MSA (2016) Reference-free assessment of speech intelligibility using bispectrum of an auditory neurogram. PLoS One 11(3):e0150,415CrossRefGoogle Scholar
  31. Ibrahim RA, Bruce IC (2010) Effects of peripheral tuning on the auditory nerve’s representation of speech envelope and temporal fine structure cues. In: Lopez-Poveda EA, Palmer AR, Meddis R (eds) The neurophysiological basis of auditory perception. Springer, New York, pp 429–438CrossRefGoogle Scholar
  32. Jackson BS, Carney LH (2005) The spontaneous-rate histogram of the auditory nerve can be explained by only two or three spontaneous rates and long-range dependence. J Assoc Res Otolaryngol 6(2):148–159CrossRefPubMedPubMedCentralGoogle Scholar
  33. Jassim WA, Zilany MS (2016) Speech quality assessment using 2d neurogram orthogonal moments. Speech Comm 80:34–48CrossRefGoogle Scholar
  34. Johnson DH (1980) The relationship between spike rate and synchrony in responses of auditory-nerve fibers to single tones. J Acoust Soc Am 68(4):1115–1122CrossRefPubMedGoogle Scholar
  35. Jørgensen S, Ewert SD, Dau T (2013) A multi-resolution envelope-power based model for speech intelligibility. J Acoust Soc Am 134(1):436–446CrossRefPubMedGoogle Scholar
  36. Joris PX, Yin TCT (1992) Responses to amplitude-modulated tones in the auditory nerve of the cat. J Acoust Soc Am 91(1):215–232CrossRefPubMedGoogle Scholar
  37. Joris PX, Schreiner CE, Rees A (2004) Neural processing of amplitude-modulated sounds. Physiol Rev 84(2):541–577CrossRefPubMedGoogle Scholar
  38. Joris PX, Bergevin C, Kalluri R, McLaughlin M, Michelet P, van der Heijden M, Shera CA (2011) Frequency selectivity in old-world monkeys corroborates sharp cochlear tuning in humans. Proc Natl Acad Sci 108(42):17,516–17,520CrossRefGoogle Scholar
  39. Kates JM, Arehart KH (2014) The hearing-aid speech perception index (HASPI). Speech Comm 65:75–93CrossRefGoogle Scholar
  40. Kiang NYS, Watanabe T, Thomas EC, Clark LF (1965) Discharge patterns of single fibers in the cat’s auditory nerve. Res. Monogr. No. 35, M.I.T. Press, CambridgeGoogle Scholar
  41. Léger AC, Desloge JG, Braida LD, Swaminathan J (2015a) The role of recovered envelope cues in the identification of temporal fine-structure speech for hearing-impaired listeners. J Acoust Soc Am 137(1):505–508CrossRefPubMedPubMedCentralGoogle Scholar
  42. Léger AC, Reed CM, Desloge JG, Swaminathan J, Braida LD (2015b)Consonant identification in noise using Hilbert-transform temporal fine-structure speech and recovered-envelope speech for listeners with normal and impaired hearing. J Acoust Soc Am 138(1):389–403Google Scholar
  43. Liberman MC (1978) Auditory-nerve response from cats raised in a low-noise chamber. J Acoust Soc Am 63(2):442–455CrossRefPubMedGoogle Scholar
  44. Logan BF Jr (1977) Information in the zero crossings of bandpass signals. Bell Syst Tech J 56(4):487–510CrossRefGoogle Scholar
  45. Lopez-Poveda EA, Eustaquio-Martin A (2013) On the controversy about the sharpness of human cochlear tuning. J Assoc Res Otolaryngol 14(5):673–686CrossRefPubMedPubMedCentralGoogle Scholar
  46. Lorenzi C, Gilbert G, Carn H, Garnier S, Moore BCJ (2006) Speech perception problems of the hearing impaired reflect inability to use temporal fine structure. Proc Natl Acad Sci U S A 103(49):18,866–18,869CrossRefGoogle Scholar
  47. Lyzenga J, Festen JM, Houtgast T (2002) A speech enhancement scheme incorporating spectral expansion evaluated with simulated loss of frequency selectivity. J Acoust Soc Am 112(3):1145–1157CrossRefPubMedGoogle Scholar
  48. Mesgarani N, David SV, Fritz JB, Shamma SA (2008) Phoneme representation and classification in primary auditory cortex. J Acoust Soc Am 123(2):899–909CrossRefPubMedGoogle Scholar
  49. Miller RL, Schilling JR, Franck KR, Young ED (1997)Effects of acoustic trauma on the representation of the vowel /ε/ in cat auditory nerve fibers. J Acoust Soc Am 101(6):3602–3616Google Scholar
  50. Moore BCJ (2008) The role of temporal fine structure processing in pitch perception, masking, and speech perception for normal-hearing and hearing-impaired people. J Assoc Res Otolaryngol 9(4):399–406CrossRefPubMedPubMedCentralGoogle Scholar
  51. Nie K, Stickney G, Zeng FG (2005) Encoding frequency modulation to improve cochlear implant performance in noise. IEEE Trans Biomed Eng 52(1):64–73CrossRefPubMedGoogle Scholar
  52. Nie K, Atlas L, Rubinstein J (2008) Single sideband encoder for music coding in cochlear implants. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2008), p 4209–4212Google Scholar
  53. Paliwal K, Wójcicki K (2008) Effect of analysis window duration on speech intelligibilty. IEEE Signal Processing Letters 15:785–788CrossRefGoogle Scholar
  54. Pascal J, Bourgeade A, Lagier M, Legros C (1998) Linear and nonlinear model of the human middle ear. J Acoust Soc Am 104(3):1509–1516CrossRefPubMedGoogle Scholar
  55. Rice SO (1973) Distortion produced by band limitation of an FM wave. Bell Syst Tech J 52(5):605–626CrossRefGoogle Scholar
  56. Rose JE, Brugge JF, Anderson DJ, Hind JE (1967) Phase-locked response to low-frequency tones in single auditory nerve fibers of the squirrel monkey. J Neurophsiology 30(4):769–793Google Scholar
  57. Rosen S (1992) Temporal information in speech: acoustic, auditory and linguistic aspects. Philos Trans: Biol Sci 336(1278):367–373CrossRefGoogle Scholar
  58. Ruggero MA, Temchin AN (2005) Unexceptional sharpness of frequency tuning in the human cochlea. Proc Natl Acad Sci U S A 102(51):18,614–18,619CrossRefGoogle Scholar
  59. Sachs MB, Young ED (1979) Encoding of steady-state vowels in the auditory nerve: representation in terms of discharge rate. J Acoust Soc Am 66(2):470–479CrossRefPubMedGoogle Scholar
  60. Sachs MB, Young ED (1980) Effects of nonlinearities on speech encoding in the auditory nerve. J Acoust Soc Am 68(3):858–875CrossRefPubMedGoogle Scholar
  61. Sachs MB, Voigt HF, Young ED (1983) Auditory nerve representation of vowels in background noise. J Neurophysiol 50(1):27–45PubMedGoogle Scholar
  62. Shamma SA (1985) Speech processing in the auditory system II: Lateral inhibition and the central processing of speech evoked activity in the auditory nerve. J Acoust Soc Am 78(5):1622–1632Google Scholar
  63. Shamma SA (1998) Spatial and temporal processing in the auditory system. In: Koch C, Segev I (eds) Methods of neuronal modeling: from ions to networks, 2nd edn. MIT Press, Cambridge, MA, pp 411–460 Google Scholar
  64. Shamma S, Lorenzi C (2013) On the balance of envelope and temporal fine structure in the encoding of speech in the early auditory system. J Acoust Soc Am 133(5):2818–2833CrossRefPubMedPubMedCentralGoogle Scholar
  65. Shannon RV, Zeng FG, Kamath V, Wygonski J, Ekelid M (1995) Speech recognition with primarily temporal cues. Science 270(5234):303–304CrossRefPubMedGoogle Scholar
  66. Sheft S, Ardoint M, Lorenzi C (2008) Speech identification based on temporal fine structure cues. J Acoust Soc Am 124(1):562–575CrossRefPubMedPubMedCentralGoogle Scholar
  67. Shera CA, Guinan JJ Jr, Oxenham AJ (2002) Revised estimates of human cochlear tuning from otoacoustic and behavioral measurements. Proc Natl Acad Sci 99(5):3318–3323CrossRefPubMedPubMedCentralGoogle Scholar
  68. Shera CA, Guinan JJ Jr, Oxenham AJ (2010) Otoacoustic estimation of cochlear tuning: validation in the chinchilla. J Assoc Res Otolaryngol 11(3):343–365CrossRefPubMedPubMedCentralGoogle Scholar
  69. Simpson AM, Moore BCJ, Glasberg BR (1990) Spectral enhancement to improve the intelligibility of speech in noise for hearing-impaired listeners. Acta Otolaryngol Suppl 469:101–107PubMedGoogle Scholar
  70. Sit JJ, Simonson AM, Oxenham AJ, Faltys MA, Sarpeshkar R (2007) A low-power asynchronous interleaved sampling algorithm for cochlear implants that encodes envelope and phase information. IEEE Trans Biomed Eng 54(1):138–149CrossRefPubMedGoogle Scholar
  71. Smith ZM, Delgutte B, Oxenham AJ (2002) Chimaeric sounds reveal dichotomies in auditory perception. Nature 416(6876):87–90CrossRefPubMedPubMedCentralGoogle Scholar
  72. Stone MA, Moore BCJ (1992) Spectral feature enhancement for people with sensorineural hearing impairment: effects on speech intelligibility and quality. J Rehabil Res Dev 29(2):39–56CrossRefPubMedGoogle Scholar
  73. Studebaker GA (1985) A “rationalized” arcsine transform. J Speech Hear Res 28(3):455–462CrossRefPubMedGoogle Scholar
  74. Swaminathan J, Heinz MG (2012) Psychophysiological analyses demonstrate the importance of neural envelope coding for speech perception in noise. J Neurosci 32(5):1747–1756CrossRefPubMedPubMedCentralGoogle Scholar
  75. Swaminathan J, Reed CM, Desloge JG, Braida LD, Delhorne LA (2014) Consonant idenfication using temporal fine structure and recovered envelope cues. J Acoust Soc Am 135(4):2078–2090CrossRefPubMedPubMedCentralGoogle Scholar
  76. Tillman TW, Carhart R (1966)An expanded test for speech discrimination utilizing CNC monosyllabic words. Brooks Air Force Base, TX Northwestern University Auditory Test No. 6, USAF School of Aerospace Medicine Technical Report, p 1–12Google Scholar
  77. Voelcker HB (1966) Toward a unified theory of modulation, part I: phase-envelope relationships. Proc IEEE 54(3):340–353CrossRefGoogle Scholar
  78. Voigt HF, Sachs MB, Young ED (1982) Representation of whispered vowels in discharge patterns of auditory-nerve fibers. Hear Res 8(1):49–58CrossRefPubMedGoogle Scholar
  79. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612CrossRefPubMedGoogle Scholar
  80. Wiener FM, Ross DA (1946) The pressure distribution in the auditory canal in a progressive sound field. J Acoust Soc Am 18(2):401–408CrossRefGoogle Scholar
  81. Wirtzfeld MW (2017) Predicting speech intelligibility and quality from model auditory nerve fiber mean-rate and spike-timing activity. PhD thesis, McMaster University, Hamilton, ON, CanadaGoogle Scholar
  82. Young ED, Oertel D (2003) The cochlear nucleus. In: Shepherd GM (ed) Synaptic organization of the brain. Oxford University Press, NY, chap 4, p 125–163Google Scholar
  83. Young ED, Sachs MB (1979) Representation of steady-state vowels in the temporal aspects of the discharge patterns of populations of auditory-nerve fibers. J Acoust Soc Am 66(5):1381–1403CrossRefPubMedGoogle Scholar
  84. Zeng FG, Nie K, Liu S, Stickney G, Rio ED, Kong YY, Chen H (2004) On the dichotomy in auditory perception between temporal envelope and fine structure cues. J Acoust Soc Am 116(3):1351–1354CrossRefPubMedGoogle Scholar
  85. Zilany MSA, Bruce IC (2006) Modeling auditory-nerve responses for high sound pressure levels in the normal and impaired auditory periphery. J Acoust Soc Am 120(3):1446–1466CrossRefPubMedGoogle Scholar
  86. Zilany MSA, Bruce IC (2007a) Predictions of speech intelligibility with a model of the normal and impaired auditory-periphery. In: Proceedings of 3rd International IEEE EMBS Conference on Neural Engineering, IEEE, Piscataway, NJGoogle Scholar
  87. Zilany MSA, Bruce IC (2007b) Representation of the vowel /ε/ in normal and impaired auditory nerve fibers: model predictions of responses in cats. J Acoust Soc Am 122(1):402–417Google Scholar
  88. Zilany MSA, Bruce IC, Nelson PC, Carney LH (2009) A phenomenological model of the synapse between the inner hair cell and auditory nerve: long-term adaptation with power-law dynamics. J Acoust Soc Am 126(5):2390–2412CrossRefPubMedPubMedCentralGoogle Scholar
  89. Zilany MSA, Bruce IC, Carney LH (2014) Updated parameters and expanded simulation options for a model of the auditory periphery. J Acoust Soc Am 135(1):283–286CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Association for Research in Otolaryngology 2017

Authors and Affiliations

  • Michael R. Wirtzfeld
    • 1
  • Rasha A. Ibrahim
    • 1
  • Ian C. Bruce
    • 1
  1. 1.Department of Electrical and Computer EngineeringMcMaster UniversityHamiltonCanada

Personalised recommendations