Abstract
We assess the challenges of studying action and language mechanisms in the brain, both singly and in relation to each other to provide a novel perspective on neuroinformatics, integrating the development of databases for encoding – separately or together – neurocomputational models and empirical data that serve systems and cognitive neuroscience.
Similar content being viewed by others
References
Aihara, T., Takeda, Y., Takeda, K., Yasuda, W., Sato, T., et al. (2012). Cortical current source estimation from electroencephalography in combination with near-infrared spectroscopy as a hierarchical prior. NeuroImage, 59, 4006–4021.
Alday, P., Schlesewsky, M., Bornkessel-Schlesewsky, I. (2013). Towards a computational model of actor-based language comprehension. Neuroinformatics. doi:10.1007/s12021-013-9198-x.
Arbib, M. A. (1981). Perceptual structures and distributed motor control. In V. B. Brooks (Ed.), Handbook of physiology — the nervous system II. Motor control (pp. 1449–1480). Bethesda: American Physiological Society.
Arbib, M. A. (2012). How the brain got language: the mirror system hypothesis. New York & Oxford: Oxford University Press.
Arbib, M. A., & Bota, M. (2003). Language evolution: neural homologies and neuroinformatics. Neural Networks, 16, 1237–1260.
Arbib, M. A., Billard, A. G., Iacoboni, M., & Oztop, E. (2000). Synthetic brain imaging: grasping, mirror neurons and imitation. Neural Networks, 13, 975–997.
Arbib, M. A., Ghanesh, V., Gasser, B. (2014a). Dyadic brain modeling, ontogenetic ritualization of gesture in apes, and the contributions of primate mirror neuron systems. Philosophical Transactions of the Royal Society B, to appear.
Arbib, M. A., Plangprasopchok, A., Bonaiuto, J. J., Schuler R. E. (2014b). A Neuroinformatics of brain modeling and its implementation in the Brain Operation Database BODB. Neuroinformatics. doi:10.1007/s12021-013-9209-y.
Ascoli, G. A. (2006). The ups and downs of neuroscience shares. Neuroinformatics, 4, 213–216. doi:10.1385/NI:4:3:213.
Aziz-Zadeh, L., Sheng, T., Liew, S.-L., Damasio, H. (2011). Understanding otherness: the neural bases of action comprehension and pain empathy in a congenital amputee. Cerebral Cortex, 4, 811–819.
Bakker, R., Potjans, T. C., & Diesmann, M. (2011). Macaque structural connectivity revisited: CoCoMac 2.0. BMC Neuroscience, 12(supplement 1), 72.
Bandrowski, A., Gupta, A., Gupta, A., Larson, S., Martone, M. (2012). Exploring mammalian brain connectivity using NeuroLex. Neuroinformatics 2012 Abstracts: http://www.neuroinformatics2012.org/abstracts/exploring-mammalian-brain-connectivity-using-neurolex.
Barrès, V., Lee, J. (2013). Template construction grammar: from visual scene description to language comprehension and agrammatism. Neuroinformatics. doi:10.1007/s12021-013-9197-y.
Barrès, V., Simons, A., & Arbib, M. A. (2013). Synthetic event-related potentials: a computational bridge between neurolinguistic models and experiments. Neural Networks, 37, 66–92.
Bohland, J. W., Bokil, H., Allen, C. B., & Mitra, P. P. (2009a). The brain atlas concordance problem: quantitative comparison of anatomical parcellations. PloS One, 4, e7200.
Bohland, J. W., Bullock, D., & Guenther, F. H. (2009b). Neural representations and mechanisms for the performance of simple speech sequences. Journal of Cognitive Neuroscience, 22, 1504–1529.
Bohland, J., Myers, E., Kim, E. (2013). An informatics approach to integrating genetic and neurological data in speech and language neuroscience. Neuroinformatics. doi:10.1007/s12021-013-9201-6.
Bonaiuto, J. J., & Arbib, M. A. (2010). Extending the mirror neuron system model, II: what did I just do? A new role for mirror neurons. Biological Cybernetics, 102, 341–359.
Bonaiuto, J. J., & Arbib, M. A. (2014). Modeling the BOLD correlates of competitive neural dynamics. Neural Networks, 49, 1–10.
Bonaiuto, J. B., Rosta, E., & Arbib, M. A. (2007). Extending the mirror neuron system model, I : audible actions and invisible grasps. Biological Cybernetics, 96, 9–38.
Bornkessel, I., & Schlesewsky, M. (2006). The extended argument dependency model: a neurocognitive approach to sentence comprehension across languages. Psychological Review, 113, 787–821.
Bornkessel-Schlesewsky, I., & Schlesewsky, M. (2009). The role of prominence information in the real-time comprehension of transitive constructions: a cross-linguistic approach. Language and Linguistics Compass, 3, 19–58.
Bornkessel-Schlesewsky, I., Kretzschmar, F., Tune, S., Wang, L., Gençc, S., et al. (2011). Think globally: cross-linguistic variation in electrophysiological activity during sentence comprehension. Brain and Language, 117, 133–152.
Bota, M., & Arbib, M. A. (2004). Integrating databases and expert systems for the analysis of brain structures: connections, similarities, and homologies. Neuroinformatics, 2, 19–58.
Bota, M., & Swanson, L. W. (2008). BAMS neuroanatomical ontology: design and implementation. Front Neuroinformatics, 2, 2. doi:10.3389/neuro.11.002.2008.
Ciccarese, P., Wu, E., Wong, G., Ocana, M., Kinoshita, J., et al. (2008). The SWAN biomedical discourse ontology. Journal of Biomedical Informatics, 41, 739–751.
Clifton, C., & Staub, A. (2008). Parallelism and competition in syntactic ambiguity resolution. Language and Linguistics Compass, 2, 234–250.
Craik, K. J. W. (1943). The nature of explanation. Cambridge: Cambridge University Press.
De Schutter, E. (2008). Why Are computational neuroscience and systems biology so separate? PLoS Computational Biology, 4, e1000078.
Debener, S., Ullsperger, M., Siegel, M., Fiehler, K., Cramon, D. Y., & Engel, A. K. (2005). Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring. Journal of Neuroscience, 25, 11730–11737.
Debener, S., Ullsperger, M., Siegel, M., & Engel, A. K. (2006). Single-trial EEG–fMRI reveals the dynamics of cognitive function. Trends in Cognitive Sciences, 10, 558–563.
Demiris, Y., Aziz-Zadeh, L. A., Bonaiuto, J. J. (2013). Information processing in the mirror neuron system in primates and machines. Neuroinformatics. doi:10.1007/s12021-013-9200-7.
Denys, K., Vanduffel, W., Fize, D., Nelissen, K., Peuskens, H., et al. (2004). The processing of visual shape in the cerebral cortex of human and nonhuman primates: A functional magnetic resonance imaging study. Journal of Neuroscience, 24, 2551–2565.
Derrfuss, J., & Mar, R. A. (2009). Lost in localization: the need for a universal coordinate database. Neurolmage, 48, 1–7.
di Pellegrino, G., Fadiga, L., Fogassi, L., Gallese, V., & Rizzolatti, G. (1992). Understanding motor events: a neurophysiological study. Experimental Brain Research, 91, 176–180.
Ferreira, F., Ferraro, V., & Bailey, K. G. D. (2002). Good-enough representations in language comprehension. Current Directions in Psychological Science, 11, 11–15.
Fillmore, C. J. (1976). Frame semantics and the nature of language. Annals of the New York Academy of Sciences, 280, 20–32.
Fisher, S. (2010). Genetic susceptibility to stuttering. New England Journal of Medicine, 362, 750–752.
Fisher, S. E., & Ridley, M. (2013). Culture, genes, and the human revolution. Science, 340, 929–930.
Fisher, S., & Scharff, C. (2009). FOXP2 as a molecular window into speech and language. Trends in Genetics, 25.
Fox, P. T., Laird, A. R., Fox, S. P., Fox, P. M., Uecker, A. M., et al. (2005). BrainMap taxonomy of experimental design: description and evaluation. Human Brain Mapping, 25, 185–198.
Friederici, A. D. (2002). Towards a neural basis of auditory sentence processing. Trends in Cognitive Sciences, 6, 78–84.
Frishkoff, G. A., Sydes, J., Mueller, K., Frank, R., Curran, T, et al. (2011). Minimal Information for Neural Electromagnetic Ontologies (MINEMO): a standards-compliant method for analysis and integration of event-related potentials (ERP) data. Standards in Genomic Sciences, 5(2): 211–223.
Friston, K. J. (2002). Beyond phrenology: what can neuroimaging tell us about distributed circuitry? Annual Review of Neuroscience, 25, 221–250.
Friston, K. J., & Dolan, R. J. (2010). Computational and dynamic models in neuroimaging. NeuroImage, 52, 752–765.
Friston, K. J., Buechel, C., Fink, G. R., Morris, J., Rolls, E., & Dolan, R. J. (1997). Psychophysiological and modulatory interactions in neuroimaging. NeuroImage, 6, 218–229.
Friston, K. J., Ashburner, J., Kiebel, S. J., Nichols, T. E., Penny, W. D. (Eds.) (2007). Statistical parametric mapping: the analysis of functional brain images. London: Academic Press.
Fujii, N., Hihara, S., & Iriki, A. (2008). Social cognition in premotor and parietal cortex. Social Neuroscience, 3, 250–260.
Gallese, V., Fadiga, L., Fogassi, L., & Rizzolatti, G. (1996). Action recognition in the premotor cortex. Brain, 119, 593–609.
Gardner, D., Akil, H., Ascoli, G., Bowden, D., Bug, W., et al. (2008). The neuroscience information framework: a data and knowledge environment for neuroscience. Neuroinformatics, 6, 149–160.
Gasser, B., Cartmill, E., & Arbib, M. A. (2013). Ontogenetic ritualization of primate gesture as a case study in dyadic brain modeling. Neuroinformatics. doi:10.1007/s12021-013-9182-5.
Gibson, E. (1998). Linguistic complexity: locality of syntactic dependencies. Cognition, 68, 1–76.
Gleeson, P., Crook, S., Cannon, R. C., Hines, M. L., Billings, G. O., et al. (2010). NeuroML: a language for describing data driven models of neurons and networks with a high degree of biological detail. PLoS Computational Biology, 6, e1000815.
Goddard, N. H., Ml, H., Howell, F., Cornelis, H., Shankar, K., & Beeman, D. (2001). Towards NeuroML: model description methods for collaborative modelling in neuroscience. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 356, 1209–1228.
Hagoort, P. (2005). On Broca, brain, and binding: a new framework. Trends in Cognitive Sciences, 9, 416–423.
Hawkins, J. A. (1999). Processing complexity and filler-gap dependencies across grammars. Language & Cognitive Processes, 75, 244–285.
Hickok, G. (2009). The functional neuroanatomy of language. Physics of Life Reviews, 6, 121–143.
Horwitz, B., Tagamets, M. A., & McIntosh, A. R. (1999a). Neural modeling, functional brain imaging, and cognition. Trends in Cognitive Sciences, 3, 91–98.
Horwitz, B., Tagamets, M. A., & McIntosh, A. R. (1999b). Neural modeling, functional brain imaging, and cognition. TRENDS in Cognitive Sciences, 3, 91–98.
Jones, A. R., Overly, C. C., & Sunkin, S. M. (2009). The allen brain atlas: 5 years and beyond. Nature Reviews. Neuroscience, 10, 821–828.
Kempen, G. (2013). Prolegomena to a neurocomputational architecture for human grammatical encoding and decoding. Neuroinformatics. doi:10.1007/s12021-013-9191-4.
Kempen, G., & Hoenkamp, E. (1987). An incremental procedural grammar for sentence formulation. Cognitive Science, 11, 201–258.
Kennedy, D. N. (2006). Where’s the beef? Missing data in the information age. Neuroinformatics, 4, 271–273. doi:10.1385/NI:4:4:271.
Kennedy, D. N. (2010). Making connections in the connectome era. Neuroinformatics, 8, 61–62.
Kennedy, D. N. (2012). The benefits of preparing data for sharing even when you don’t. Neuroinformatics, 10, 223–224.
Kennedy, D. N., Ascoli, G. A., & De Schutter, E. (2011). Next steps in data publishing. Neuroinformatics, 9, 317–320. doi:10.1007/s12021-011-9131-0.
Kotter, R. (2004). Online retrieval, processing, and visualization of primate connectivity data from the CoCoMac database. Neuroinformatics, 2, 127–144.
Kouzes, R. T., Myers, J. D., & Wulf, W. A. (1996). Collaboratories: doing science on the Internet. Computer, 29, 40–46.
Laird, A. R., Lancaster, J., & Fox, P. T. (2005). BrainMap: the social evolution of a human brain mapping database. Neuroinformatics, 3, 65–77.
Laird, A. R., Eickhoff, S. B., Kurth, F., Fox, P. M., Uecker, A. M., et al. (2009). ALE meta-analysis workflows Via the brainmap database: progress towards a probabilistic functional brain atlas. Front Neuroinformatics, 3, 23. doi:10.3389/neuro.11.023.2009.
Laird, A. R., Eickhoff, S. B., Fox, P. M., Uecker, A. M., Ray, K. L., et al. (2011). The BrainMap strategy for standardization, sharing, and meta-analysis of neuroimaging data. BMC Research Notes, 4, 349.
Lewis, R. L., Vasishth, S., & Van Dyke, J. A. (2006). Computational principles of working memory in sentence comprehension. Trends in Cognitive Sciences, 10, 447–454.
Li, P., Zhao, X., & MacWhinney, B. (2007). Dynamic self-organization and early lexical development in children. Cognitive Science, 31, 581–612.
Lohrey, J. M., Killeen, N. E. B., & Egan, G. F. (2009). An integrated object model and method framework for subject-centric e-Research applications. Frontiers in Neuroinformatics, 3, 19.
Luo, X.-Z., Kennedy, D., & Cohen, Z. (2009). Neuroimaging Informatics Tools and Resources Clearinghouse (NITRC) resource announcement. Neuroinformatics, 7, 55–56.
MacWhinney, B. (1987). The competition model. In B. MacWhinney (Ed.), Mechanisms of language acquisition (pp. 249–308). Hillsdale: Lawrence Erlbaum.
MacWhinney, B. (2000). The childes project: tools for analyzing talk (3rd ed.). Mahwah: Lawrence Erlbaum Associates.
MacWhinney, B. (2008). How mental models encode embodied linguistic perspectives. In R. Klatzky, B. MacWhinney, & M. Behrmann (Eds.), Embodiment, ego-space, and action (pp. 369–410). Mahwah: Lawrence Erlbaum.
MacWhinney. (2011). The logic of the unified model. In S. Gass & A. Mackey (Eds.), Handbook of second language acquisition. New York: Routledge.
MacWhinney, B., Fromm, D., Forbes, M., & Holland, A. (2011). AphasiaBank: methods for studying discourse. Aphasiology, 25, 1286–1307.
Mantini, D., Hasson, U., Betti, V., Perrucci, M. G., Romani, G. L., et al. (2012). Interspecies activity correlations reveal functional correspondence between monkey and human brain areas. Nature Methods, 9, 277–282.
Marenco, L., Ascoli, G. A., Martone, M. E., Shepherd, G. M., & Miller, P. L. (2008). The NIF LinkOut broker: a web resource to facilitate federated data integration using NCBI identifiers. Neuroinformatics, 6, 219–227.
Marenco, L., Wang, R., Shepherd, G. M., & Miller, P. (2010). The NIF DISCO framework: facilitating automated integration of neuroscience content on the web. Neuroinformatics, 8, 101–112.
Matsumoto, R., Nair, D. R., Bingaman, W., Ikeda, A., Shibasaki, H., & Lüders, H. O. (2005). Interareal connectivity in the human language system: a cortico-cortical evoked potential study. International Congress Series, 1278, 397–400.
McElree, B., Foraker, S., & Dyer, L. (2003). Memory structures that subserve sentence comprehension. Journal of Memory and Language, 48, 67–91.
McIntosh, A. R. (2012). Tracing the route to path analysis in neuroimaging. NeuroImage, 62, 887–890.
Miller, G. A. (1995). WordNet: a lexical database for english. Communications of the ACM, 38, 39–41.
Nelissen, K., Luppino, G., Vanduffel, W., Rizzolatti, G., & Orban, G. A. (2005). Observing others: multiple action representation in the frontal lobe. Science, 310, 332–336.
Nielsen, F. Å. (2005). Mass meta-analysis in Talairach space. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems (17th ed., pp. 985–992). Cambridge: MIT Press.
Nielsen, F. Å. (2013). Brede tools and federating online neuroinformatics databases. Neuroinformatics.doi:10.1007/s12021-013-9183-4.
Orban, G. A., Van Essen, D., & Vanduffel, W. (2004). Comparative mapping of higher visual areas in monkeys and humans. Trends in Cognitive Sciences, 8, 315–324.
Oztop, E., & Arbib, M. A. (2002). Schema design and implementation of the grasp-related mirror neuron system. Biological Cybernetics, 87, 116–140.
Oztop, E., Wolpert, D., & Kawato, M. (2005). Mental state inference using visual control parameters. Brain Research. Cognitive Brain Research, 22, 129–151.
Oztop, E., Kawato, M., & Arbib, M. A. (2013). Mirror neurons: functions, mechanisms and models. Neuroscience Letters, 540, 43–55.
Palmer, M. (2009). Semlink: linking PropBank, VerbNet and FrameNet. Proceedings of the Generative Lexicon Conference. Sept. 2009, Pisa, Italy: GenLex-09: 580–90.
Peeters, R., Simone, L., Nelissen, K., Fabbri-Destro, M., Vanduffel, W., et al. (2009). The representation of tool use in humans and monkeys: common and uniquely human features. The Journal of Neuroscience, 29, 11523–39.
Poizner, H., Klima, E., & Bellugi, U. (1987). What the hands reveal about the brain. Cambridge: MIT Press.
Rizzolatti, G., & Arbib, M. A. (1998). Language within our grasp. Trends in Neurosciences, 21, 188–194.
Rizzolatti, G., & Sinigaglia, C. (2008). Mirrors in the brain: how our minds share actions, emotions, and experience (Translated from the Italian by Frances Anderson). Oxford: Oxford University Press.
Russ, T. A., Ramakrishnan, C., Hovy, E. H., Bota, M., & Burns, G. A. (2011). Knowledge engineering tools for reasoning with scientific observations and interpretations: a neural connectivity use case. BMC Bioinformatics, 12, 351.
Sagotsky, J. A., Zhang, L., Wang, Z., Martin, S., & Deisboeck, T. S. (2008). Life Sciences and the web: a new era for collaboration. Molecular Systems Biology, 4.
Santos, G. S., Nagasaka, Y., Fujii, N., & Nakahara, H. (2011). Encoding of social state information by neuronal activities in the macaque caudate nucleus. Social Neuroscience, 7, 42–58.
Sato, M. A., Yoshioka, T., Kajihara, S., Toyama, K., Goda, N., et al. (2004). Hierarchical Bayesian estimation for MEG inverse problem. NeuroImage, 23, 806–826.
Silberman, Y., Bentin, S., & Miikkulainen, R. (2007). Semantic boost on episodic associations: an empirically-based computational model. Cognitive Science, 31, 645–671.
Sobolev, A., Leonhardt, A., Rautenberg, P., Kellner, C., Herz, A., Wachtler, T. (2012). Cloud services and a data API for electrophysiology. Neuroinformatics 2012: http://www.neuroinformatics2012.org/abstracts/cloud-services-and-a-data-api-for-electrophysiology.
Stein, L. (2002). Creating a bioinformatics nation. Nature, 417, 119–120.
Traxler, M. J., & Gernsbacher, M. A. (Eds.). (2006). Handbook of psycholinguistics. London: Academic.
Turner, J., & Laird, A. R. (2012). The cognitive paradigm ontology: design and application. Neuroinformatics, 10, 57–66.
Ullman, M. T. (2004). Contributions of memory circuits to language: the declarative/procedural model. Cognition, 92, 231–270.
Vanduffel, W., Fize, D., Peuskens, H., Denys, K., Sunaert, S., et al. (2002). Extracting 3D from motion: differences in human and monkey intraparietal cortex. Science, 298, 413–415.
Vosse, T., & Kempen, G. (2000). Syntactic structure assembly in human parsing: a computational model based on competitive inhibition and a lexicalist grammar. Cognition, 75, 105–143.
Vosse, T., & Kempen, G. (2009). The unification space implemented as a localist neural net: predictions and error-tolerance in a constraint-based parser. Cognitive Neurodynamics, 3, 331–346.
Waldrop, M. M. (2008). Science 2.0. Scientific American, 298, 68–73.
Wolpert, D. M., & Kawato, M. (1998). Multiple paired forward and inverse models for motor control. Neural Networks, 11, 1317–1329.
Acknowledgement, Discussion Groups and Workshop Participants
The present paper is based on discussions held on Wednesday July 27, as part of the Workshop on “Action, Language and Neuroinformatics” held on July 25–27, 2011, in Los Angeles under the aegis of the USC Brain Project of the University of Southern California, and organized by Michael A. Arbib. The Workshop was supported in part by the National Science Foundation under Grant No. 0924674.
There were six discussion groups, three (1a,b,c) meeting in parallel in the morning and three (2a,b,c) in the afternoon. The participants of the Workshop were divided into three groups, one each on Action, Language, and Neuroinformatics. The discussion groups were then organized as follows. The name of the rapporteur for each session is marked in Bold. The present paper is based on the integration of their six reports.
1a. Half the Action group + half the Neuroinformatics group: assessing the model-data integration and neuroinformatics needs of the Action group: Bonaiuto, Oztop, Demiris, Vanduffel, Arbib, Marques, Bohland.
1b. Half the Neurolinguistics group + the other half of the Neuroinformatics group: assessing the model-data integration and neuroinformatics needs of the Neurolinguistics group: Bornkessel-Schlesewsky, Small, MacWhinney, Miikkulainen, Nielsen, Fox, Barrès, Schuler.
1c. The other half of the Action group + the other half of the Neurolinguistics group: Defining shared modeling challenges and the development of a shared conceptual framework. Kemmerer, Aziz-Zadeh, Cartmill, Gasser, Grosvald, Wood, Kempen, Lee, Schilling.
2a. Action group: What are the key data and/or conceptual issues ripe for modeling; what are the key lines of modeling that hold most promise to address these data/issues? Oztop, Demiris, Vanduffel, Cartmill, Arbib, Aziz-Zadeh, Gasser, Schilling, Wood.
2b. Neurolinguistics group: What are the key data and/or conceptual issues ripe for modeling; what are the key lines of modeling that hold most promise to address these data/issues? MacWhinney, Kempen, Grosvald, Small, Bornkessel-Schlesewsky, Miikkulainen. Kemmerer, Lee, Barres
2c. Neuroinformatics group: What tools are ripe for sharing, or should be ripened? What are promising lines for federation? Nielsen, Marques, Bohland, Bonaiuto, Bota, Fox, Schuler.
Brief biosketches of the participants, access to selected papers, and abstracts of their talks may be found at the Workshop Website: http://uscbp.usc.edu/mediawiki/index.php/2011_Workshop.
Author information
Authors and Affiliations
Corresponding author
Appendix: A Tabulation of Present and Future Resources
Appendix: A Tabulation of Present and Future Resources
Rights and permissions
About this article
Cite this article
Arbib, M.A., Bonaiuto, J.J., Bornkessel-Schlesewsky, I. et al. Action and Language Mechanisms in the Brain: Data, Models and Neuroinformatics. Neuroinform 12, 209–225 (2014). https://doi.org/10.1007/s12021-013-9210-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12021-013-9210-5