Voice Processing and Voice-Identity Recognition

  • Samuel Robert MathiasEmail author
  • Katharina von Kriegstein
Part of the Springer Handbook of Auditory Research book series (SHAR, volume 69)


The human voice is the most important sound source in our environment, not only because it produces speech, but also because it conveys information about the speaker. In many situations, listeners understand the speech message and recognize the speaker with minimal effort. Psychophysical studies have investigated which voice qualities (such as vocal timbre) distinguish speakers and allow listeners to recognize speakers. Glottal and vocal tract characteristics strongly influence perceived similarity between speakers and serve as cues for voice-identity recognition. However, the importance of a particular voice quality for voice-identity recognition depends on the speaker and the stimulus. Voice-identity recognition relies on a network of brain regions comprising a core system of auditory regions within the temporal lobe (including regions dedicated to processing glottal and vocal tract characteristics and regions that play more abstract roles) and an extended system of nonauditory regions representing information associated with specific voice identities (e.g., faces and names). This brain network is supported by early, direct connections between the core voice system and an analogous core face system. Precisely how all these brain regions work together to accomplish voice-identity recognition remains an open question; answering it will require rigorous testing of hypotheses derived from theoretical accounts of voice processing.


Congenital phonagnosia Core face system Core voice system Glottal-pulse rate Vocal recognition Vocal timbre Vocal tract length 







distance measure


fusiform face area


functional magnetic resonance imaging


facial recognition units


glottal-pulse rate


Hechl’s gyrus


harmonics-to-noise ratio


inferior frontal gyrus


inferior parietal lobe


just noticeable difference








person-identity nodes


planum temporale


superior temporal gyrus


superior temporal sulcus


perceptual threshold


temporal voice areas


ventrolateral prefrontal cortex


voice recognition units


vocal-tract length


  1. Agus TR, Paquette S, Suied C et al (2017) Voice selectivity in the temporal voice area despite matched low-level acoustic cues. Sci Rep 7(1):11526PubMedPubMedCentralCrossRefGoogle Scholar
  2. Andics A, Gácsi M, Faragó T et al (2014) Voice-sensitive regions in the dog and human brain are revealed by comparative fMRI. Curr Biol 24(5):574–578PubMedCrossRefGoogle Scholar
  3. Baumann O, Belin P (2010) Perceptual scaling of voice identity: common dimensions for different vowels and speakers. Psychol Res 74(1):110–120PubMedCrossRefGoogle Scholar
  4. Bartlett FC (1932) Remembering: a study in experimental and social psychology. Cambridge University Press, CambridgeGoogle Scholar
  5. Belin P, Bestelmeyer PEG, Latinus M, Watson R (2011) Understanding voice perception. Br J Psychol 102(4):711–725PubMedCrossRefGoogle Scholar
  6. Belin P, Zatorre RJ, Ahad P (2002) Human temporal-lobe response to vocal sounds. Brain Res Cogn Brain Res 13(1):17–26PubMedCrossRefGoogle Scholar
  7. Belin P, Zatorre RJ (2003) Adaptation to speaker’s voice in right anterior temporal lobe. Neuroreport 14(16):2105–2109PubMedCrossRefGoogle Scholar
  8. Belin P, Zatorre RJ, Lafaille P et al (2000) Voice-sensitive areas in human auditory cortex. Nature 403(6767):309–312CrossRefGoogle Scholar
  9. Blank H, Anwander A, von Kriegstein K (2011) Direct structural connections between voice- and face-recognition areas. J Neurosci 31(36):12906–12915PubMedCrossRefGoogle Scholar
  10. Blank H, Wieland N, von Kriegstein K (2014) Person recognition and the brain: merging evidence from patients and healthy individuals. Neurosci Biobehav Rev 47:717–734PubMedCrossRefGoogle Scholar
  11. Bodamer J (1947) Die Prosop-Agnosie (Prosopagnosia) Archiv für Psychiatrie und Nervenkrankheiten (Archive for Psychiatry and Neurological Diseases) 179(1–2):6–53Google Scholar
  12. Bruce V, Young A (1986) Understanding face recognition. Br J Psychol 77(3):305–327PubMedCrossRefPubMedCentralGoogle Scholar
  13. Ellis H, Jones D, Mosdell N (1997) Intra- and inter-modal repetition priming of familiar faces and voices. Br J Psychol 88(1):143–156PubMedCrossRefPubMedCentralGoogle Scholar
  14. Fecteau S, Armony JL, Joanette Y, Belin P (2004) Is voice processing species-specific in human auditory cortex? An fMRI study. NeuroImage 23(3):840–848PubMedCrossRefPubMedCentralGoogle Scholar
  15. Fitch WT, Giedd J (1999) Morphology and development of the human vocal tract: a study using magnetic resonance imaging. J Acoust Soc Am 106(3):1511–1522PubMedCrossRefPubMedCentralGoogle Scholar
  16. Formisano E, De Martino F, Bonte M, Goebel R (2008) “Who” is saying “what”? Brain-based decoding of human voice and speech. Science 322(5903):970–973CrossRefGoogle Scholar
  17. Fouquet M, Pisanski K, Mathevon N, Reby D (2016) Seven and up: individual differences in male voice fundamental frequency emerge before puberty and remain stable throughout adulthood. R Soc Open Sci. Scholar
  18. Frühholz S, Trost W, Kotz SA (2016) The sound of emotions — Towards a unifying neural network perspective of affective sound processing. Neurosci Biobehav Rev 68:96–110PubMedCrossRefPubMedCentralGoogle Scholar
  19. Gainotti G, Barbier A, Marra C (2003) Slowly progressive defect in recognition of familiar people in a patient with right anterior temporal atrophy. Brain 126(4):792–803PubMedCrossRefPubMedCentralGoogle Scholar
  20. Garrido L, Eisner F, McGettigan C et al (2009) Developmental phonagnosia: a sensitive deficit of vocal identity recognition. Neuropsychologia 47:123–131PubMedCrossRefPubMedCentralGoogle Scholar
  21. Gaudrain E, Li S, Ban V, Patterson RD (2009) The role of glottal pulse rate and vocal tract length in the perception of speaker identity. Paper presented at Interspeech 2009: 10th annual conference of the international speech communication association, 1–5, 148–151Google Scholar
  22. Gilbert HR, Weismer GG (1974) The effects of smoking on the speaking fundamental frequency of adult women. J Psycholinguist Res 3(3):225–231CrossRefGoogle Scholar
  23. Gray H (1918) Anatomy of the human body. Lea Febiger, PhiladelphiaCrossRefGoogle Scholar
  24. Griffiths TD, Hall DA (2012) Mapping pitch representation in neural ensembles with fMRI. J Neurosci 32(39):13343–13347PubMedCrossRefPubMedCentralGoogle Scholar
  25. Hailstone JC, Ridgway GR, Bartlett JW et al (2011) Voice processing in dementia: a neuropsychological and neuroanatomical analysis. Brain 134:2535–2547PubMedPubMedCentralCrossRefGoogle Scholar
  26. Hautamäki R, Kinnunen T, Hautamäki V, Laukkanen A-M (2015) Automatic versus human speaker verification: the case of voice mimicry. Speech Comm 72:13–31CrossRefGoogle Scholar
  27. Haxby JV, Hoffman EA, Gobbini MI (2000) The distributed human neural system for face perception. Trends Cogn Sci 4(6):223–233PubMedCrossRefPubMedCentralGoogle Scholar
  28. Hickok G, Costanzo M, Capasso R, Miceli G (2011) The role of Broca’s area in speech perception: evidence from aphasia revisited. Brain Lang 119(3):214–220PubMedPubMedCentralCrossRefGoogle Scholar
  29. Hillenbrand J, Getty LA, Clark MJ, Wheeler K (1995) Acoustic characteristics of American English vowels. J Acoust Soc Am 97(5):3099–3111PubMedCrossRefGoogle Scholar
  30. Hillenbrand JM, Clark MJ (2009) The role of f0 and formant frequencies in distinguishing the voices of men and women. Atten Percept Psychophys 71(5):1150–1166PubMedCrossRefGoogle Scholar
  31. Hölig C, Föcker J, Best A et al (2017) Activation in the angular gyrus and in the pSTS is modulated by face primes during voice recognition. Hum Brain Mapp 38(5):2553–2565PubMedCrossRefGoogle Scholar
  32. Hollien H, Shipp T (1972) Speaking fundamental frequency and chronologic age in males. J Speech Lang Hear Res 15(1):155–159CrossRefGoogle Scholar
  33. Jiang J, Liu F, Wan X, Jiang CM (2015) Perception of melodic contour and intonation in autism spectrum disorder: evidence from Mandarin speakers. J Autism Dev Disord 45:2067–2075PubMedCrossRefGoogle Scholar
  34. Johnson K (2005) Speaker normalization in speech perception. In: Pisoni DP, Remez RR (eds) The handbook of speech perception. Blackwell Publishing Ltd, Malden, pp 363–389CrossRefGoogle Scholar
  35. Kanwisher N, Yovel G (2006) The fusiform face area: a cortical region specialized for the perception of faces. Philos Trans R Soc Lond Ser B Biol Sci 361(1476):2109–2128CrossRefGoogle Scholar
  36. Kell AJ, Yamins DL, Shook EN et al (2018) A task-optimized neural network replicates human auditory behavior predicts brain responses and reveals a cortical processing hierarchy. Neuron 98:630–644PubMedCrossRefGoogle Scholar
  37. Kitaoka N, Enami D, Nakagawa S (2014) Effect of acoustic and linguistic contexts on human and machine speech recognition. Comput Speech Lang 28(3):769–787CrossRefGoogle Scholar
  38. Kreiman J, Vanlancker-Sidtis D, Gerratt BR (2005) Perception of voice quality. In: Pisoni DP, Remez RR (eds) The handbook of speech perception. Blackwell Publishing Ltd., Malden, pp 338–362CrossRefGoogle Scholar
  39. Kreiman J, Gerratt BR (1998) Validity of rating scale measures of voice quality. J Acoust Soc Am 104(3):1598–1608PubMedCrossRefPubMedCentralGoogle Scholar
  40. Kreitewolf J, Gaudrain E, von Kriegstein K (2014) A neural mechanism for recognizing speech spoken by different speakers. NeuroImage 91:375–385PubMedCrossRefPubMedCentralGoogle Scholar
  41. Kreitewolf J, Mathias SR, von Kriegstein K (2017) Implicit talker training improves comprehension of auditory speech in noise. Front Psychol.
  42. Künzel HJ (1989) How well does average fundamental frequency correlate with speaker height and weight? Phonetica 46(1–3):117–125PubMedCrossRefPubMedCentralGoogle Scholar
  43. Latinus M, Belin P (2011) Anti-voice adaptation suggests prototype-based coding of voice identity. Front Psychol 2:175Google Scholar
  44. Latinus M, McAleer P, Bestelmeyer PEG, Belin P (2013) Norm-based coding of voice identity in human auditory cortex. Curr Biol 23(12):1075–1080PubMedPubMedCentralCrossRefGoogle Scholar
  45. Laver J (1980) The phonetic description of voice quality. Cambridge University Press, CambridgeGoogle Scholar
  46. Lavner Y, Gath I, Rosenhouse J (2000) The effects of acoustic modifications on the identification of familiar voices speaking isolated vowels. Speech Comm 30:9–26CrossRefGoogle Scholar
  47. Lavner Y, Rosenhouse J, Gath I (2001) The prototype model in speaker identification by human listeners. Int J Speech Technol 4(1):63–74CrossRefGoogle Scholar
  48. López S, Riera P, Assaneo MF et al (2013) Vocal caricatures reveal signatures of speaker identity. Sci Rep.
  49. Luzzi S, Coccia M, Polonara G et al (2018) Sensitive associative phonagnosia after right anterior temporal stroke. Neuropsychologia 116:154–161. Scholar
  50. Maguinness C, Roswandowitz C, von Kriegstein K (2018) Understanding the mechanisms of familiar voice-identity recognition in the human brain. Neuropsychologia 166:179–193CrossRefGoogle Scholar
  51. Mathias SR, von Kriegstein K (2014) How do we recognise who is speaking. Front Biosci S6:92–109CrossRefGoogle Scholar
  52. Mullennix JW, Ross A, Smith C, Kuykendall K, Conrad J, Barb S (2011) Typicality effects on memory for voice: implications for earwitness testimony. Appl Cogn Psychol 25(1):29–34CrossRefGoogle Scholar
  53. Murray T, Singh S (1980) Multidimensional analysis of male and female voices. J Acoust Soc Am 68(5):1294–1300CrossRefGoogle Scholar
  54. Neuner F, Schweinberger SR (2000) Neuropsychological impairments in the recognition of faces voices and personal names. Brain Cogn 44(3):342–366PubMedCrossRefPubMedCentralGoogle Scholar
  55. Nosofsky RM (1986) Choice similarity and the context theory of classification. J Exp Psychol Learn Mem Cogn 10:104–114CrossRefGoogle Scholar
  56. O’Scalaidhe SP, Wilson FA, Goldman-Rakic PS (1997) Areal segregation of face-processing neurons in prefrontal cortex. Science 278(5340):1135–1138CrossRefGoogle Scholar
  57. Petkov CI, Kayser C, Steudel T et al (2008) A voice region in the monkey brain. Nat Neurosci 11(3):367–374PubMedCrossRefPubMedCentralGoogle Scholar
  58. Pernet CR, McAleer P, Latinus M et al (2015) The human voice areas: spatial organization and inter-individual variability in temporal and extra-temporal cortices. NeuroImage 119:164–174PubMedPubMedCentralCrossRefGoogle Scholar
  59. Perrodin C, Kayser C, Logothetis NK, Petkov CI (2011) Voice cells in the primate temporal lobe. Curr Biol 21(16):1408–1415PubMedPubMedCentralCrossRefGoogle Scholar
  60. Peterson GE, Barney HL (1952) Control methods used in a study of the vowels. J Acoust Soc Am 24(4):175–184CrossRefGoogle Scholar
  61. Plack CJ, Oxenham AJ (2005) The psychophysics of pitch. In: Plack CJ, Oxenham AJ, Popper AN, Fay RR (eds) Pitch: neural coding and perception. Springer Handbook of Auditory Research, vol 24. Springer, New York, pp 7–55CrossRefGoogle Scholar
  62. Remez RE, Fellowes JM, Rubin PE (1997) Talker identification based on phonetic information. J Exp Psychol Hum Percept Perform 23(3):651–666PubMedCrossRefPubMedCentralGoogle Scholar
  63. Romanski LM, Goldman-Rakic PS (2002) An auditory domain in primate prefrontal cortex. Nat Neurosci 5(1):15–16PubMedPubMedCentralCrossRefGoogle Scholar
  64. Roswandowitz C, Kappes C, Obrig H, von Kriegstein K (2018a) Obligatory and facultative brain regions for voice-identity recognition. Brain 141(1):234–247PubMedCrossRefPubMedCentralGoogle Scholar
  65. Roswandowitz C, Maguinness C, von Kriegstein K (2018b) Deficits in voice-identity processing: acquired and developmental phonagnosia. In: Frühholz S, Belin P (eds) The oxford handbook of voice perception. Oxford University Press, OxfordGoogle Scholar
  66. Roswandowitz C, Mathias SR, Hintz F et al (2014) Two cases of sensitive developmental voice-recognition impairments. Curr Biol 24(19):2348–2353PubMedCrossRefPubMedCentralGoogle Scholar
  67. Roswandowitz C, Schelinski S, von Kriegstein K (2017) Developmental phonagnosia: linking neural mechanisms with the behavioural phenotype. NeuroImage 155:97–112PubMedCrossRefPubMedCentralGoogle Scholar
  68. Saslove H, Yarmey AD (1980) Long-term auditory memory: Speaker identification. J Appl Psychol 65(1):111–116PubMedCrossRefPubMedCentralGoogle Scholar
  69. Schall S, Kiebel SJ, Maess B, von Kriegstein K (2013) Early auditory sensory processing of voices is facilitated by visual mechanisms. NeuroImage 77:237–245PubMedCrossRefGoogle Scholar
  70. Schall S, Kiebel SJ, Maess B, von Kriegstein K (2014) Voice identity recognition: functional division of the right STS and its behavioral relevance. J Cogn Neurosci 27(2):280–291CrossRefGoogle Scholar
  71. Schall S, Kiebel SJ, Maess B, von Kriegstein K (2015) Voice identity recognition: functional division of the right STS and its behavioral relevance. J Cogn Neurosci 27(2):280–291PubMedCrossRefPubMedCentralGoogle Scholar
  72. Schelinski S, Roswandowitz C, von Kriegstein K (2017) Voice identity processing in autism spectrum disorder. Autism Res 10(1):155–168PubMedCrossRefGoogle Scholar
  73. Sheffert SM, Pisoni DB, Fellowes JM, Remez RE (2002) Learning to recognize talkers from natural sinewave and reversed speech samples. J Exp Psychol Hum Percept Perform 28(6):1447–1469PubMedPubMedCentralCrossRefGoogle Scholar
  74. Smith DRR, Patterson RD (2005) The interaction of glottal-pulse rate and vocal-tract length in judgements of speaker size, sex, and age. J Acoust Soc Am 118(5):3177–3186PubMedPubMedCentralCrossRefGoogle Scholar
  75. Smith DRR, Patterson RD, Turner R et al (2005) The processing and perception of size information in speech sounds. J Acoust Soc Am 117(1):305–318PubMedPubMedCentralCrossRefGoogle Scholar
  76. Stevenage SV, Clarke G, McNeill A (2012) The “other-accent” effect in voice recognition. J Cogn Psychol 24(6):647–653CrossRefGoogle Scholar
  77. Stoicheff ML (1981) Speaking fundamental frequency characteristics of nonsmoking female adults. J Speech Lang Hear Res 24(3):437–441CrossRefGoogle Scholar
  78. Sugihara T, Diltz MD, Averbeck BB, Romanski LM (2006) Integration of auditory and visual communication information in the primate ventrolateral prefrontal cortex. J Neurosci 26(43):11138–11147PubMedPubMedCentralCrossRefGoogle Scholar
  79. Talavage TM, Johnsrude IS, Gonzalez-Castillo J (2012) In: Poeppel D, Overath T, Popper AN, Fay RR (eds) The human auditory cortex. Springer handbook of auditory research, vol 43. Springer, New York, pp 129–164Google Scholar
  80. Titze I (1989) Physiologic and acoustic differences between male and female voices. J Acoust Soc Am 85(4):1699–1707PubMedCrossRefPubMedCentralGoogle Scholar
  81. van Lancker D, Kreiman J, Emmorey K (1985) Familiar voice recognition: patterns and parameters. Part I Recognition of backward voices. J Phon 13:19–38Google Scholar
  82. van Lancker DR, Canter GJ (1982) Impairment of voice and face recognition in patients with hemispheric damage. Brain Cogn 1:185–195PubMedCrossRefPubMedCentralGoogle Scholar
  83. van Lancker DR, Kreiman J, Cummings J (1989) Voice perception deficits: neuroanatomical correlates of phonagnosia. J Clin Exp Neuropsychol 11(5):665–674PubMedCrossRefPubMedCentralGoogle Scholar
  84. von Kriegstein K (2011) A multisensory perspective on human auditory communication. In: Murray MM, Wallace MT (eds) The neural bases of multisensory processes. CRC Press, Boca Raton, pp 683–700CrossRefGoogle Scholar
  85. von Kriegstein K, Dogan O, Grüter M et al (2008) Simulation of talking faces in the human brain improves auditory speech recognition. Proc Natl Acad Sci U S A 105(18):6747–6752Google Scholar
  86. von Kriegstein K, Kleinschmidt A, Giraud A (2006) Voice recognition and cross-modal responses to familiar speakers’ voices in prosopagnosia. Cereb Cortex 16(9):1314–1322Google Scholar
  87. von Kriegstein K, Eger E, Kleinschmidt A, Giraud A-L (2003) Modulation of neural responses to speech by directing attention to voices or verbal content. Cogn Brain Res 17(1):48–55CrossRefGoogle Scholar
  88. von Kriegstein K, Giraud A-L (2004) Distinct functional substrates along the right superior temporal sulcus for the processing of voices. NeuroImage 22(2):948–955PubMedCrossRefPubMedCentralGoogle Scholar
  89. von Kriegstein K, Giraud A-L (2006) Implicit multisensory associations influence voice recognition. PLoS Biol 4(10). Scholar
  90. von Kriegstein K, Kleinschmidt A, Sterzer P, Giraud A-L (2005) Interaction of face and voice areas during speaker recognition. J Cogn Neurosci 17(3):367–376CrossRefGoogle Scholar
  91. von Kriegstein K, Kleinschmidt A, Giraud A (2006) Voice recognition and cross-modal responses to familiar speakers’ voices in prosopagnosia. Cereb Cortex 16(9):1314–1322Google Scholar
  92. von Kriegstein K, Smith DRR, Patterson RD et al (2007) Neural representation of auditory size in the human voice and in sounds from other resonant sources. Curr Biol 17(13):1123–1128CrossRefGoogle Scholar
  93. von Kriegstein K, Smith DRR, Patterson RD et al (2010) How the human brain recognizes speech in the context of changing speakers. J Neurosci 30(2):629–638CrossRefGoogle Scholar
  94. Wester M (2012) Talker discrimination across languages. Speech Comm 54:781–790CrossRefGoogle Scholar
  95. Wilding J, Cook S (2000) Sex differences and individual consistency in voice identification. Percept Mot Skills 91(2):535–538PubMedCrossRefPubMedCentralGoogle Scholar
  96. Xu X, Biederman I, Shilowich BE et al (2015) Developmental phonagnosia: Neural correlates and a behavioral marker. Brain Lang 149:106–117PubMedCrossRefPubMedCentralGoogle Scholar
  97. Yarmey AD (2007) The psychology of speaker identification and earwitness memory. In: Lindsay RCL, Ross DF, Read JD, Toglia MP (eds) The handbook of eyewitness psychology vol II: memory for people. Lawrence Erlbaum Associates, Mahwah, pp 101–136Google Scholar
  98. Zäske R, Hasan BAS, Belin P (2017) It doesn’t matter what you say: fMRI correlates of voice learning and recognition independent of speech content. Cortex 94:100–112PubMedPubMedCentralCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Samuel Robert Mathias
    • 1
    Email author
  • Katharina von Kriegstein
    • 2
    • 3
  1. 1.Neurocognition, Neurocomputation and Neurogenetics DivisionYale University School of MedicineNew HavenUSA
  2. 2.Technische Universität DresdenDresdenGermany
  3. 3.Max Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany

Personalised recommendations