Skip to main content
SpringerLink
Log in

Neural Integration of Audiovisual Sensory Inputs in Macaque Amygdala and Adjacent Regions

  • Original Article
  • Published:
Neuroscience Bulletin Aims and scope Submit manuscript

Abstract

Integrating multisensory inputs to generate accurate perception and guide behavior is among the most critical functions of the brain. Subcortical regions such as the amygdala are involved in sensory processing including vision and audition, yet their roles in multisensory integration remain unclear. In this study, we systematically investigated the function of neurons in the amygdala and adjacent regions in integrating audiovisual sensory inputs using a semi-chronic multi-electrode array and multiple combinations of audiovisual stimuli. From a sample of 332 neurons, we showed the diverse response patterns to audiovisual stimuli and the neural characteristics of bimodal over unimodal modulation, which could be classified into four types with differentiated regional origins. Using the hierarchical clustering method, neurons were further clustered into five groups and associated with different integrating functions and sub-regions. Finally, regions distinguishing congruent and incongruent bimodal sensory inputs were identified. Overall, visual processing dominates audiovisual integration in the amygdala and adjacent regions. Our findings shed new light on the neural mechanisms of multisensory integration in the primate brain.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Noesselt T, Tyll S, Boehler CN, Budinger E, Heinze HJ, Driver J. Sound-induced enhancement of low-intensity vision: Multisensory influences on human sensory-specific cortices and thalamic bodies relate to perceptual enhancement of visual detection sensitivity. J Neurosci 2010, 30: 13609–13623.

    CAS  PubMed  PubMed Central  Google Scholar 

  2. Orioli G, Bremner AJ, Farroni T. Multisensory perception of looming and receding objects in human newborns. Curr Biol 2018, 28: R1294–R1295.

    CAS  PubMed  Google Scholar 

  3. Baumann O, Greenlee MW. Neural correlates of coherent audiovisual motion perception. Cereb Cortex 2007, 17: 1433–1443.

    PubMed  Google Scholar 

  4. Khandhadia AP, Murphy AP, Romanski LM, Bizley JK, Leopold DA. Audiovisual integration in macaque face patch neurons. Curr Biol 2021, 31: 1826-1835.e3.

    CAS  PubMed  PubMed Central  Google Scholar 

  5. Lewis R, Noppeney U. Audiovisual synchrony improves motion discrimination via enhanced connectivity between early visual and auditory areas. J Neurosci 2010, 30: 12329–12339.

    CAS  PubMed  PubMed Central  Google Scholar 

  6. von Saldern S, Noppeney U. Sensory and striatal areas integrate auditory and visual signals into behavioral benefits during motion discrimination. J Neurosci 2013, 33: 8841–8849.

    Google Scholar 

  7. Brang D, Taich ZJ, Hillyard SA, Grabowecky M, Ramachandran VS. Parietal connectivity mediates multisensory facilitation. Neuroimage 2013, 78: 396–401.

    PubMed  Google Scholar 

  8. Kaminiarz A, Schlack A, Hoffmann KP, Lappe M, Bremmer F. Visual selectivity for heading in the macaque ventral intraparietal area. J Neurophysiol 2014, 112: 2470–2480.

    PubMed  Google Scholar 

  9. Schlack A, Sterbing-D’Angelo SJ, Hartung K, Hoffmann KP, Bremmer F. Multisensory space representations in the macaque ventral intraparietal area. J Neurosci 2005, 25: 4616–4625.

    CAS  PubMed  PubMed Central  Google Scholar 

  10. Roitman JD, Shadlen MN. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J Neurosci 2002, 22: 9475–9489.

    CAS  PubMed  PubMed Central  Google Scholar 

  11. Stein BE, Stanford TR. Multisensory integration: Current issues from the perspective of the single neuron. Nat Rev Neurosci 2008, 9: 255–266.

    CAS  PubMed  Google Scholar 

  12. Stein BE, Stanford TR, Rowland BA. Multisensory integration and the society for neuroscience: Then and now. J Neurosci 2020, 40: 3–11.

    CAS  PubMed  PubMed Central  Google Scholar 

  13. Collignon O, Girard S, Gosselin F, Roy S, Saint-Amour D, Lassonde M. Audio-visual integration of emotion expression. Brain Res 2008, 1242: 126–135.

    CAS  PubMed  Google Scholar 

  14. Kragel PA, Čeko M, Theriault J, Chen D, Satpute AB, Wald LW, et al. A human colliculus-pulvinar-amygdala pathway encodes negative emotion. Neuron 2021, 109: 2404-2412.e5.

    CAS  PubMed  PubMed Central  Google Scholar 

  15. Liu X, Huang H, Snutch TP, Cao P, Wang L, Wang F. The superior colliculus: Cell types, connectivity, and behavior. Neurosci Bull 2022, 38: 1519–1540.

    PubMed  PubMed Central  Google Scholar 

  16. Rafal RD, Koller K, Bultitude JH, Mullins P, Ward R, Mitchell AS, et al. Connectivity between the superior colliculus and the amygdala in humans and macaque monkeys: Virtual dissection with probabilistic DTI tractography. J Neurophysiol 2015, 114: 1947–1962.

    PubMed  PubMed Central  Google Scholar 

  17. Shang C, Liu Z, Chen Z, Shi Y, Wang Q, Liu S, et al. BRAIN CIRCUITS. A parvalbumin-positive excitatory visual pathway to trigger fear responses in mice. Science 2015, 348: 1472–1477.

    CAS  PubMed  Google Scholar 

  18. Wei P, Liu N, Zhang Z, Liu X, Tang Y, He X, et al. Processing of visually evoked innate fear by a non-canonical thalamic pathway. Nat Commun 2015, 6: 6756.

    CAS  PubMed  Google Scholar 

  19. Yang Y, Liu DQ, Huang W, Deng J, Sun Y, Zuo Y, et al. Selective synaptic remodeling of amygdalocortical connections associated with fear memory. Nat Neurosci 2016, 19: 1348–1355.

    CAS  PubMed  Google Scholar 

  20. Morrow J, Mosher C, Gothard K. Multisensory neurons in the primate amygdala. J Neurosci 2019, 39: 3663–3675.

    CAS  PubMed  PubMed Central  Google Scholar 

  21. Dotson NM, Hoffman SJ, Goodell B, Gray CM. A large-scale semi-chronic microdrive recording system for non-human Primates. Neuron 2017, 96: 769-782.e2.

    CAS  PubMed  Google Scholar 

  22. Shan L, Huang H, Zhang Z, Wang Y, Gu F, Lu M, et al. Mapping the emergence of visual consciousness in the human brain via brain-wide intracranial electrophysiology. Innovation (Camb) 2022, 3: 100243.

    PubMed  Google Scholar 

  23. Zhang Z, Shan L, Wang Y, Li W, Jiang M, Liang F, et al. Primate preoptic neurons drive hypothermia and cold defense. Innovation (Camb) 2023, 4: 100358.

    PubMed  Google Scholar 

  24. Oostenveld R, Fries P, Maris E, Schoffelen JM. FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput Intell Neurosci 2011, 2011: 156869.

    PubMed  Google Scholar 

  25. Stolk A, Griffin S, van der Meij R, Dewar C, Saez I, Lin JJ, et al. Integrated analysis of anatomical and electrophysiological human intracranial data. Nat Protoc 2018, 13: 1699–1723.

    CAS  PubMed  PubMed Central  Google Scholar 

  26. Jung B, Taylor PA, Seidlitz J, Sponheim C, Perkins P, Ungerleider LG, et al. A comprehensive macaque fMRI pipeline and hierarchical atlas. Neuroimage 2021, 235: 117997.

    PubMed  Google Scholar 

  27. Hartig R, Glen D, Jung B, Logothetis NK, Paxinos G, Garza-Villarreal EA, et al. The subcortical atlas of the Rhesus macaque (SARM) for neuroimaging. Neuroimage 2021, 235: 117996.

    PubMed  Google Scholar 

  28. Kremer Y, Flakowski J, Rohner C, Lüscher C. Context-dependent multiplexing by individual VTA dopamine neurons. J Neurosci 2020, 40: 7489–7509.

    CAS  PubMed  PubMed Central  Google Scholar 

  29. Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 2012, 482: 85–88.

    CAS  PubMed  PubMed Central  Google Scholar 

  30. Keshavarzi S, Bracey EF, Faville RA, Campagner D, Tyson AL, Lenzi SC, et al. Multisensory coding of angular head velocity in the retrosplenial cortex. Neuron 2022, 110: 532–543.e9.

    CAS  PubMed  PubMed Central  Google Scholar 

  31. Paxinos G, Petrides MHE. The Rhesus Monkey Brain in Stereotaxic Coordinates. 4th ed. London: Academic Press, 2021.

  32. Diehl MM, Romanski LM. Responses of prefrontal multisensory neurons to mismatching faces and vocalizations. J Neurosci 2014, 34: 11233–11243.

    CAS  PubMed  PubMed Central  Google Scholar 

  33. Romanski LM, Hwang J. Timing of audiovisual inputs to the prefrontal cortex and multisensory integration. Neuroscience 2012, 214: 36–48.

    CAS  PubMed  Google Scholar 

  34. Sugihara T, Diltz MD, Averbeck BB, Romanski LM. Integration of auditory and visual communication information in the primate ventrolateral prefrontal cortex. J Neurosci 2006, 26: 11138–11147.

    CAS  PubMed  PubMed Central  Google Scholar 

  35. Bremen P, Massoudi R, van Wanrooij MM, van Opstal AJ. Audio-visual integration in a redundant target paradigm: A comparison between Rhesus macaque and man. Front Syst Neurosci 2017, 11: 89.

    PubMed  PubMed Central  Google Scholar 

  36. Duhamel JR, Colby CL, Goldberg ME. Ventral intraparietal area of the macaque: Congruent visual and somatic response properties. J Neurophysiol 1998, 79: 126–136.

    CAS  PubMed  Google Scholar 

  37. Barraclough NE, Xiao D, Baker CI, Oram MW, Perrett DI. Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J Cogn Neurosci 2005, 17: 377–391.

    PubMed  Google Scholar 

  38. Beauchamp MS, Lee KE, Argall BD, Martin A. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 2004, 41: 809–823.

    CAS  PubMed  Google Scholar 

  39. Dahl CD, Logothetis NK, Kayser C. Modulation of visual responses in the superior temporal sulcus by audio-visual congruency. Front Integr Neurosci 2010, 4: 10.

    PubMed  PubMed Central  Google Scholar 

  40. Ghazanfar AA, Maier JX, Hoffman KL, Logothetis NK. Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. J Neurosci 2005, 25: 5004–5012.

    CAS  PubMed  PubMed Central  Google Scholar 

  41. Stein BE, Stanford TR, Rowland BA. Development of multisensory integration from the perspective of the individual neuron. Nat Rev Neurosci 2014, 15: 520–535.

    CAS  PubMed  PubMed Central  Google Scholar 

  42. Werner S, Noppeney U. Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cereb Cortex 2010, 20: 1829–1842.

    PubMed  Google Scholar 

  43. Wallace MT, Stein BE. Development of multisensory neurons and multisensory integration in cat superior colliculus. J Neurosci 1997, 17: 2429–2444.

    CAS  PubMed  PubMed Central  Google Scholar 

  44. Haber SN, Knutson B. The reward circuit: Linking primate anatomy and human imaging. Neuropsychopharmacology 2010, 35: 4–26.

    PubMed  Google Scholar 

  45. Rolls ET. Emotion and decision-making explained: A précis. Cortex 2014, 59: 185–193.

    PubMed  Google Scholar 

  46. Rolls ET. The texture and taste of food in the brain. J Texture Stud 2020, 51: 23–44.

    PubMed  Google Scholar 

  47. Rolls ET, Thorpe SJ, Maddison SP. Responses of striatal neurons in the behaving monkey. 1. Head of the caudate nucleus. Behav Brain Res 1983, 7: 179–210.

    CAS  PubMed  Google Scholar 

  48. Strait CE, Sleezer BJ, Hayden BY. Signatures of value comparison in ventral striatum neurons. PLoS Biol 2015, 13: e1002173.

    PubMed  PubMed Central  Google Scholar 

  49. Williams GV, Rolls ET, Leonard CM, Stern C. Neuronal responses in the ventral striatum of the behaving macaque. Behav Brain Res 1993, 55: 243–252.

    CAS  PubMed  Google Scholar 

  50. Kodsi MH, Swerdlow NR. Prepulse inhibition in the rat is regulated by ventral and caudodorsal striato-pallidal circuitry. Behav Neurosci 1995, 109: 912–928.

    CAS  PubMed  Google Scholar 

  51. Kodsi MH, Swerdlow NR. Ventral pallidal GABA-A receptors regulate prepulse inhibition of acoustic startle. Brain Res 1995, 684: 26–35.

    CAS  PubMed  Google Scholar 

  52. Swerdlow NR, Braff DL, Geyer MA. GABAergic projection from nucleus accumbens to ventral pallidum mediates dopamine-induced sensorimotor gating deficits of acoustic startle in rats. Brain Res 1990, 532: 146–150.

    CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We thank Dr. Yong-Shuai Ge for his kind assistance with the CT scans. This work was supported by the National Natural Science Foundation of China (U20A2017 and 31830037), Guangdong Basic and Applied Basic Research Foundation (2020A1515010785, 2020A1515111118, and 2022A1515010134), the Youth Innovation Promotion Association of the Chinese Academy of Sciences (2017120), the Shenzhen-Hong Kong Institute of Brain Science–Shenzhen Fundamental Research Institutions (NYKFKT2019009), Shenzhen Technological Research Center for Primate Translational Medicine (F-2021-Z99-504979), the Strategic Research Program of the Chinese Academy of Sciences (XDBS01030100 and XDB32010300), Scientific and Technological Innovation 2030 (2021ZD0204300), and the Fundamental Research Funds for the Central Universities.

Author information

Author notes

  1. Liang Shan, Liu Yuan, and Bo Zhang contributed equally to this work.

Authors and Affiliations

  1. CAS Key Laboratory of Brain Connectome and Manipulation, the Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China

    Liang Shan, Liu Yuan, Bo Zhang, Jian Ma, Xiao Xu & Ji Dai

  2. Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen, 518055, China

    Liang Shan & Ji Dai

  3. University of Chinese Academy of Sciences, Beijing, 100049, China

    Liu Yuan, Fei Gu, Yi Jiang & Ji Dai

  4. State Key Laboratory of Brain and Cognitive Science, CAS Center for Excellence in Brain Science and Intelligence Technology, Institute of Psychology, Chinese Academy of Sciences, Beijing, 100101, China

    Fei Gu & Yi Jiang

  5. Chinese Institute for Brain Research, Beijing, 102206, China

    Yi Jiang

  6. Shenzhen Technological Research Center for Primate Translational Medicine, Shenzhen, 518055, China

    Ji Dai

  7. Key Laboratory of Brain Science, Zunyi Medical University, Zunyi, 563000, China

    Bo Zhang

Authors
  1. Liang Shan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liu Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jian Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiao Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Fei Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yi Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Ji Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yi Jiang or Ji Dai.

Ethics declarations

Conflict of interest

The authors declare that there are no conflicts of interest.

Supplementary Information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shan, L., Yuan, L., Zhang, B. et al. Neural Integration of Audiovisual Sensory Inputs in Macaque Amygdala and Adjacent Regions. Neurosci. Bull. 39, 1749–1761 (2023). https://doi.org/10.1007/s12264-023-01043-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12264-023-01043-8

Keywords

Search

Navigation