Abstract
Extensive studies suggest that the brain integrates multisensory signals in a Bayesian optimal way. In this work, we consider how the couplings in a neural network model are shaped by the prior information when it performs optimal multisensory integration and encodes the whole profile of the posterior. To process stimuli of two modalities, a biologically plausible neural network model consists of two modules, one for each modality, and crosstalks between the two modules are carried out through feedforward cross-links and reciprocal connections. We found that the reciprocal couplings are crucial to optimal multisensory integration in that their pattern is shaped by the correlation in the joint prior distribution of sensory stimuli. Our results show that a decentralized architecture based on reciprocal connections is able to accommodate complex correlation structures across modalities and utilize this prior information in optimal multisensory integration.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alais, D., Burr, D.: No direction-specific bimodal facilitation for audiovisual motion detection. Cogn. Brain Res. 19(2), 185–194 (2004)
Ernst, M.O., Banks, M.S.: Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415(6870), 429–433 (2002)
Gu, Y., Angelaki, D.E., DeAngelis, G.C.: Neural correlates of multisensory cue integration in macaque MSTd. Nat. Neurosci. 11(10), 1201–1210 (2008)
Girshick, A.R., Landy, M.S., Simoncelli, E.P.: Cardinal rules: visual orientation perception reflects knowledge of environmental statistics. Nat. Neurosci. 14(7), 926–932 (2011)
Fischer, B.J., Peña, J.L.: Owl’s behavior and neural representation predicted by Bayesian inference. Nat. Neurosci. 14(8), 1061–1066 (2011)
Körding, K.P., Wolpert, D.M.: Bayesian integration in sensorimotor learning. Nature 427(6971), 244–247 (2004)
Ghazanfar, A.A., Schroeder, C.E.: Is neocortex essentially multisensory? Trends Cogn. Sci. 10(6), 278–285 (2006)
Sato, Y., Toyoizumi, T., Aihara, K.: Bayesian inference explains perception of unity and ventriloquism aftereffect: identification of common sources of audiovisual stimuli. Neural Comput. 19(12), 3335–3355 (2007)
Shams, L., Ma, W.J., Beierholm, U.: Sound-induced flash illusion as an optimal percept. NeuroReport 16(17), 1923–1927 (2005)
Shams, L., Beierholm, U.R.: Causal inference in perception. Trends Cogn. Sci. 14(9), 425–432 (2010)
Durante, F., Sempi, C.: Principles of Copula Theory. Taylor & Francis, Boca Raton (2015)
Sklar, A.: Random variables, joint distribution functions, and copulas. Kybernetika 9(6), 449–460 (1973)
Körding, K.P., Beierholm, U., Ma, W.J., Quartz, S., Tenenbaum, J.B., Shams, L.: Causal inference in multisensory perception. PLoS One 2(9), e943 (2007)
Zhang, W.H., Chen, A., Rasch, M.J., Wu, S.: Decentralized multisensory information integration in neural systems. J. Neurosci. 36(2), 532–547 (2016)
Magosso, E., Cuppini, C., Ursino, M.: A neural network model of ventriloquism effect and aftereffect. PLoS One 7(8), e42503 (2012)
Amari, S.: Dynamics of pattern formation in lateral-inhibition type neural fields. Biol. Cybern. 27(2), 77–87 (1977)
Ohshiro, T., Angelaki, D.E., DeAngelis, G.C.: A normalization model of multisensory integration. Nat. Neurosci. 14(6), 775–782 (2011)
Carandini, M., Heeger, D.J.: Normalization as a canonical neural computation. Nat. Rev. Neurosci. 13(1), 51–62 (2012)
Van Atteveldt, N., Murray, M.M., Thut, G., Schroeder, C.E.: Multisensory integration: flexible use of general operations. Neuron 81(6), 1240–1253 (2014)
Fung, C.C.A., Wong, K.Y.M., Wu, S.: A moving bump in a continuous manifold: a comprehensive study of the tracking dynamics of continuous attractor neural networks. Neural Comput. 22(3), 752–792 (2010)
Williams, R.J., Zipser, D.: Gradient-based learning algorithms for recurrent networks and their computational complexity. In: Back-Propagation: Theory, Architectures and Applications, pp. 433–486 (1995)
Seung, H.S.: Learning continuous attractors in recurrent networks. In: Jordan, M.I., Kearns, M.J., Solla, S.A. (eds.) Advances in Neural Information Processing Systems 10, pp. 654–660. MIT Press, Cambridge (1998)
Acknowledgments
This work is supported by the Research Grants Council of Hong Kong (N_HKUST606/12, 605813 and 16322616) and National Basic Research Program of China (2014CB846101) and the Natural Science Foundation of China (31261160495).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Wang, H., Zhang, WH., Wong, K.Y.M., Wu, S. (2017). How the Prior Information Shapes Neural Networks for Optimal Multisensory Integration. In: Cong, F., Leung, A., Wei, Q. (eds) Advances in Neural Networks - ISNN 2017. ISNN 2017. Lecture Notes in Computer Science(), vol 10262. Springer, Cham. https://doi.org/10.1007/978-3-319-59081-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-59081-3_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-59080-6
Online ISBN: 978-3-319-59081-3
eBook Packages: Computer ScienceComputer Science (R0)