Advertisement

Psychological Research PRPF

, Volume 73, Issue 4, pp 527–544 | Cite as

Brain mechanisms for predictive control by switching internal models: implications for higher-order cognitive functions

  • Hiroshi ImamizuEmail author
  • Mitsuo Kawato
Original Article

Abstract

Humans can guide their actions toward the realization of their intentions. Flexible, rapid and precise realization of intentions and goals relies on the brain learning to control its actions on external objects and to predict the consequences of this control. Neural mechanisms that mimic the input–output properties of our own body and other objects can be used to support prediction and control, and such mechanisms are called internal models. We first summarize functional neuroimaging, behavioral and computational studies of the brain mechanisms related to acquisition, modular organization, and the predictive switching of internal models mainly for tool use. These mechanisms support predictive control and flexible switching of intentional actions. We then review recent studies demonstrating that internal models are crucial for the execution of not only immediate actions but also higher-order cognitive functions, including optimization of behaviors toward long-term goals, social interactions based on prediction of others’ actions and mental states, and language processing. These studies suggest that a concept of internal models can consistently explain the neural mechanisms and computational principles needed for fundamental sensorimotor functions as well as higher-order cognitive functions.

Keywords

Functional Connectivity Internal Model Grip Force Inferior Parietal Lobule Superior Parietal Lobule 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgments

We thank Toshinori Yoshioka for developing the software used for the three-dimensional display of multiple brain activities (Figs. 4, 7; Supplemental movie). This MATLAB(R) based software is freely available at: http://www.cns.atr.jp/multi_color.

Supplementary material

Supplemental movie (MPG 29.1 MB).

References

  1. Baron-Cohen, S. (1997). Mindblindness: An essay on autism and theory of mind (learning, development and conceptual change). Cambridge: MIT Press.Google Scholar
  2. Blakemore, S. J., & Decety, J. (2001). From the perception of action to the understanding of intention. Nature Reviews Neuroscience, 2(8), 561–567.PubMedCrossRefGoogle Scholar
  3. Blakemore, S. J., Frith, C. D., & Wolpert, D. M. (2001). The cerebellum is involved in predicting the sensory consequences of action. Neuroreport, 12(9), 1879–1884.PubMedCrossRefGoogle Scholar
  4. Blakemore, S. J., Wolpert, D. M., & Frith, C. D. (1998). Central cancellation of self-produced tickle sensation. Nature Neuroscience, 1(7), 635–640.PubMedCrossRefGoogle Scholar
  5. Bonda, E., Petrides, M., Ostry, D., & Evans, A. (1996). Specific involvement of human parietal systems and the amygdala in the perception of biological motion. Journal of Neuroscience, 16(11), 3737–3744.PubMedGoogle Scholar
  6. Brashers-Krug, T., Shadmehr, R., & Bizzi, E. (1996). Consolidation in human motor memory. Nature, 382(6588), 252–255.PubMedCrossRefGoogle Scholar
  7. Bursztyn, L. L., Ganesh, G., Imamizu, H., Kawato, M., & Flanagan, J. R. (2006). Neural correlates of internal-model loading. Current Biology, 16(24), 2440–2445.PubMedCrossRefGoogle Scholar
  8. Clower, D. M., West, R. A., Lynch, J. C., & Strick, P. L. (2001). The inferior parietal lobule is the target of output from the superior colliculus, hippocampus, and cerebellum. Journal of Neuroscience, 21(16), 6283–6291.PubMedGoogle Scholar
  9. Daw, N. D., Niv, Y., & Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience, 8(12), 1704–1711.PubMedCrossRefGoogle Scholar
  10. Decety, J., Jackson, P. L., Sommerville, J. A., Chaminade, T., & Meltzoff, A. N. (2004). The neural bases of cooperation and competition: An fMRI investigation. Neuroimage, 23(2), 744–751.PubMedCrossRefGoogle Scholar
  11. Diedrichsen, J., Criscimagna-Hemminger, S. E., & Shadmehr, R. (2007). Dissociating timing and coordination as functions of the cerebellum. Journal of Neuroscience, 27(23), 6291–6301.PubMedCrossRefGoogle Scholar
  12. Doya, K. (1999). What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? Neural Networks, 12(7–8), 961–974.PubMedCrossRefGoogle Scholar
  13. Doya, K. (2000). Complementary roles of basal ganglia and cerebellum in learning and motor control. Current Opinion in Neurobiology, 10(6), 732–739.PubMedCrossRefGoogle Scholar
  14. Doya, K., Okada, G., Ueda, K., Okamoto, Y., & Yamawaki, S. (2001). Pediction of short- and long-term reward: A functional MRI study with a Markov decision problem. Paper presented at the Annual Meeting Society for Neuroscience.Google Scholar
  15. Ebner, T. J., & Pasalar, S. (2008). Cerebellum predicts the future motor state. Cerebellum, 7(4), 583–588.PubMedCrossRefGoogle Scholar
  16. Flanagan, J. R., Nakano, E., Imamizu, H., Osu, R., Yoshioka, T., & Kawato, M. (1999). Composition and decomposition of internal models in motor learning under altered kinematic and dynamic environments. Journal of Neuroscience, 19(20), RC34.PubMedGoogle Scholar
  17. Flanagan, J. R., & Wing, A. M. (1997). The role of internal models in motion planning and control: Evidence from grip force adjustments during movements of hand-held loads. Journal of Neuroscience, 17(4), 1519–1528.PubMedGoogle Scholar
  18. Friederici, A. D., Bahlmann, J., Heim, S., Schubotz, R. I., & Anwander, A. (2006). The brain differentiates human and non-human grammars: Functional localization and structural connectivity. Proceedings of the National Academy of Sciences of the USA, 103(7), 2458–2463.PubMedCrossRefGoogle Scholar
  19. Friston, K. J., Harrison, L., & Penny, W. (2003). Dynamic causal modelling. Neuroimage, 19, 1273–1302.PubMedCrossRefGoogle Scholar
  20. Frith, C. D., Blakemore, S. J., & Wolpert, D. M. (2000). Abnormalities in the awareness and control of action. Philosophical Transactions of the Royal Society of London Series B Biological Sciences, 355(1404), 1771–1788.CrossRefGoogle Scholar
  21. Frith, C. D., & Frith, U. (1999). Interacting minds—A biological basis. Science, 286(5445), 1692–1695.PubMedCrossRefGoogle Scholar
  22. Gallese, V., Fadiga, L., Fogassi, L., & Rizzolatti, G. (1996). Action recognition in the premotor cortex. Brain, 119(Pt 2), 593–609.PubMedCrossRefGoogle Scholar
  23. Gandolfo, F., Mussa-Ivaldi, F. A., & Bizzi, E. (1996). Motor learning by field approximation. Proceedings of the National Academy of Sciences of the USA, 93(9), 3843–3846.PubMedCrossRefGoogle Scholar
  24. Ghahramani, Z., & Wolpert, D. M. (1997). Modular decomposition in visuomotor learning. Nature, 386(6623), 392–395.PubMedCrossRefGoogle Scholar
  25. Gomi, H., Shidara, M., Takemura, A., Inoue, Y., Kawano, K., & Kawato, M. (1998). Temporal firing patterns of purkinje cells in the cerebellar ventral paraflocculus during ocular following responses in monkeys I. Simple spikes (in process citation). Journal of Neurophysiology, 80(2), 818–831.PubMedGoogle Scholar
  26. Graydon, F. X., Friston, K. J., Thomas, C. G., Brooks, V. B., & Menon, R. S. (2005). Learning-related fMRI activation associated with a rotational visuo-motor transformation. Brain Research Cognitive Brain Research, 22, 373–383.PubMedCrossRefGoogle Scholar
  27. Grodd, W., Hulsmann, E., Lotze, M., Wildgruber, D., & Erb, M. (2001). Sensorimotor mapping of the human cerebellum: fMRI evidence of somatotopic organization. Human Brain Mapping, 13(2), 55–73.PubMedCrossRefGoogle Scholar
  28. Haruno, M., & Kawato, M. (in press). Activity in the superior temporal sulcus highlights learning competence in an interaction game. Journal of Neuroscience.Google Scholar
  29. Haruno, M., Kuroda, T., Doya, K., Toyama, K., Kimura, M., Samejima, K., et al. (2004). A neural correlate of reward-based behavioral learning in caudate nucleus: A functional magnetic resonance imaging study of a stochastic decision task. Journal of Neuroscience, 24(7), 1660–1665.PubMedCrossRefGoogle Scholar
  30. Haruno, M., Wolpert, D. M., & Kawato, M. (2001). Mosaic model for sensorimotor learning and control. Neural Computation, 13(10), 2201–2220.PubMedCrossRefGoogle Scholar
  31. Hauser, M. D., Chomsky, N., & Fitch, W. T. (2002). The faculty of language: What is it, who has it, and how did it evolve? Science, 298(5598), 1569–1579.PubMedCrossRefGoogle Scholar
  32. Higuchi, S., Imamizu, H., Chaminade, T., & Kawato, M. (2004). Broca’s area during tool—use and linguistic processing. Paper presented at the Annual Meeting Society for Neuroscience.Google Scholar
  33. Higuchi, S., Imamizu, H., & Kawato, M. (2007). Cerebellar activity evoked by common tool-use execution and imagery tasks: An fMRI study. Cortex, 43(3), 350–358.PubMedCrossRefGoogle Scholar
  34. Hoshi, E., Tremblay, L., Feger, J., Carras, P. L., & Strick, P. L. (2005). The cerebellum communicates with the basal ganglia. Nature Neuroscience, 8(11), 1491–1493.PubMedCrossRefGoogle Scholar
  35. Hurley, S. (2008). The shared circuits model (SCM): How control, mirroring, and simulation can enable imitation, deliberation, and mindreading. The Behavioral and Brain Sciences, 31(1), 1–22. (discussion 22–58).PubMedCrossRefGoogle Scholar
  36. Iacoboni, M., Koski, L. M., Brass, M., Bekkering, H., Woods, R. P., Dubeau, M. C., et al. (2001). Reafferent copies of imitated actions in the right superior temporal cortex. Proceedings of the National Academy of Sciences of the USA, 98(24), 13995–13999.PubMedCrossRefGoogle Scholar
  37. Imamizu, H., Higuchi, S., Toda, A., & Kawato, M. (2007a). Reorganization of brain activity for multiple internal models after short but intensive training. Cortex, 43(3), 338–349.PubMedCrossRefGoogle Scholar
  38. Imamizu, H., & Kawato, M. (2008). Neural correlates of predictive and postdictive switching mechanisms for internal models. Journal of Neuroscience, 28(42), 10751–10765.PubMedCrossRefGoogle Scholar
  39. Imamizu, H., Kuroda, T., Miyauchi, S., Yoshioka, T., & Kawato, M. (2003). Modular organization of internal models of tools in the human cerebellum. Proceedings of the National Academy of Sciences of the USA, 100(9), 5461–5466.PubMedCrossRefGoogle Scholar
  40. Imamizu, H., Kuroda, T., Yoshioka, T., & Kawato, M. (2004). Functional magnetic resonance imaging examination of two modular architectures for switching multiple internal models. Journal of Neuroscience, 24(5), 1173–1181.PubMedCrossRefGoogle Scholar
  41. Imamizu, H., Miyauchi, S., Tamada, T., Sasaki, Y., Takino, R., Putz, B., et al. (2000). Human cerebellar activity reflecting an acquired internal model of a new tool. Nature, 403(6766), 192–195.PubMedCrossRefGoogle Scholar
  42. Imamizu, H., Sugimoto, N., Osu, R., Tsutsui, K., Sugiyama, K., Wada, Y., et al. (2007b). Explicit contextual information selectively contributes to predictive switching of internal models. Experimental Brain Research, 181(3), 395–408.CrossRefGoogle Scholar
  43. Ito, M. (1984). The cerebellum and neural motor control. New York: Raven Press.Google Scholar
  44. Jacobs, R. A., Jordan, M. I., Nowlan, S. J., & Hinton, G. E. (1991). Adaptive mixture of local experts. Neural Computation, 3, 79–87.CrossRefGoogle Scholar
  45. Johansson, R. S., & Westling, G. (1988). Coordinated isometric muscle commands adequately and erroneously programmed for the weight during lifting task with precision grip. Experimental Brain Research, 71(1), 59–71.Google Scholar
  46. Karniel, A., & Mussa-Ivaldi, F. A. (2002). Does the motor control system use multiple models and context switching to cope with a variable environment? Experimental Brain Research, 143(4), 520–524.CrossRefGoogle Scholar
  47. Kawato, M. (1999). Internal models for motor control and trajectory planning. Current Opinion in Neurobiology, 9(6), 718–727.PubMedCrossRefGoogle Scholar
  48. Kawato, M., Furukawa, K., & Suzuki, R. (1987). A hierarchical neural-network model for control and learning of voluntary movement. Biological Cybernetics, 57(3), 169–185.PubMedCrossRefGoogle Scholar
  49. Kawato, M., Kuroda, T., Imamizu, H., Nakano, E., Miyauchi, S., & Yoshioka, T. (2003). Internal forward models in the cerebellum: fMRI study on grip force and load force coupling. Progress in Brain Research, 142, 171–188.PubMedCrossRefGoogle Scholar
  50. Kawato, M., & Samejima, K. (2007). Efficient reinforcement learning: Computational theories, neuroscience and robotics. Current Opinion in Neurobiology, 17(2), 205–212.PubMedCrossRefGoogle Scholar
  51. Kawawaki, D., Shibata, T., Goda, N., Doya, K., & Kawato, M. (2006). Anterior and superior lateral occipito-temporal cortex responsible for target motion prediction during overt and covert visual pursuit. Neuroscience Research, 54(2), 112–123.PubMedCrossRefGoogle Scholar
  52. Kerns, J. G., Cohen, J. D., MacDonald, A. W., 3rd, Cho, R. Y., Stenger, V. A., & Carter, C. S. (2004). Anterior cingulate conflict monitoring and adjustments in control. Science, 303(5660), 1023–1026.PubMedCrossRefGoogle Scholar
  53. Kitazawa, S., Kimura, T., & Yin, P. B. (1998). Cerebellar complex spikes encode both destinations and errors in arm movements. Nature, 392(6675), 494–497.PubMedCrossRefGoogle Scholar
  54. Krakauer, J. W., Ghilardi, M. F., & Ghez, C. (1999). Independent learning of internal models for kinematic and dynamic control of reaching. Nature Neuroscience, 2(11), 1026–1031.PubMedCrossRefGoogle Scholar
  55. Krakauer, J. W., Ghilardi, M. F., Mentis, M., Barnes, A., Veytsman, M., Eidelberg, D., et al. (2004). Differential cortical and subcortical activations in learning rotations and gains for reaching: A PET study. Journal of Neurophysiology, 91(2), 924–933.PubMedCrossRefGoogle Scholar
  56. Kravitz, J. H., & Yaffe, F. L. (1972). Conditionned adaptation to prismatic displacement with a tone as the conditioal stimulus. Perception & Psychophysics, 12(3), 305–308.Google Scholar
  57. Maquet, P., Schwartz, S., Passingham, R., & Frith, C. (2003). Sleep-related consolidation of a visuomotor skill: Brain mechanisms as assessed by functional magnetic resonance imaging. Journal of Neuroscience, 23(4), 1432–1440.PubMedGoogle Scholar
  58. Martin, A., & Chao, L. L. (2001). Semantic memory and the brain: Structure and processes. Current Opinion in Neurobiology, 11(2), 194–201.PubMedCrossRefGoogle Scholar
  59. Miall, R. C. (2003). Connecting mirror neurons and forward models. Neuroreport, 14(17), 2135–2137.PubMedCrossRefGoogle Scholar
  60. Miall, R. C., Keating, J. G., Malkmus, M., & Thach, W. T. (1998). Simple spike activity predicts occurrence of complex spikes in cerebellar Purkinje cells. Nature Neuroscience, 1(1), 13–15.PubMedCrossRefGoogle Scholar
  61. Miall, R. C., Reckess, G. Z., & Imamizu, H. (2001). The cerebellum coordinates eye and hand tracking movements. Nature Neuroscience, 4(6), 638–644.PubMedCrossRefGoogle Scholar
  62. Miall, R. C., Weir, D. J., Wolpert, D. M., & Stein, J. F. (1993). Is the cerebellum a Smith predictor? Journal of Motor Behavior, 25, 203–216.PubMedGoogle Scholar
  63. Middleton, F. A., & Strick, P. L. (1997). Dentate output channels: Motor and cognitive components. In C. I. de Zeeuw, P. Strata, & J. Voogd (Eds.), The cerebellum: From structure to control (pp. 553–566). Amsterdam: Elsevier Science BV.Google Scholar
  64. Middleton, F. A., & Strick, P. L. (2001). Cerebellar projections to the prefrontal cortex of the primate. Journal of Neuroscience, 21(2), 700–712.PubMedGoogle Scholar
  65. Milner, T. E., Franklin, D. W., Imamizu, H., & Kawato, M. (2007). Central control of grasp: Manipulation of objects with complex and simple dynamics. Neuroimage, 36(2), 388–395.PubMedCrossRefGoogle Scholar
  66. Obayashi, S., Suhara, T., Kawabe, K., Okauchi, T., Maeda, J., Akine, Y., et al. (2001). Functional brain mapping of monkey tool use. Neuroimage, 14(4), 853–861.PubMedCrossRefGoogle Scholar
  67. O’Reilly, J. X., Mesulam, M. M., & Nobre, A. C. (2008). The cerebellum predicts the timing of perceptual events. Journal of Neuroscience, 28(9), 2252–2260.PubMedCrossRefGoogle Scholar
  68. Osu, R., Hirai, S., Yoshioka, T., & Kawato, M. (2004). Random presentation enables subjects to adapt to two opposing forces on the hand. Nature Neuroscience, 7(2), 111–112.PubMedCrossRefGoogle Scholar
  69. Oztop, E., Kawato, M., & Arbib, M. (2006). Mirror neurons and imitation: A computationally guided review. Neural Network, 19(3), 254–271.CrossRefGoogle Scholar
  70. Oztop, E., Wolpert, D., & Kawato, M. (2005). Mental state inference using visual control parameters. Brain Research Cognitive Brain Research, 22(2), 129–151.PubMedCrossRefGoogle Scholar
  71. Raichle, M. E., Fiez, J. A., Videen, T. O., MacLeod, A. M., Pardo, J. V., Fox, P. T., et al. (1994). Practice-related changes in human brain functional anatomy during nonmotor learning. Cerebral Cortex, 4(1), 8–26.PubMedCrossRefGoogle Scholar
  72. Sakai, K. L. (2005). Language acquisition and brain development. Science, 310(5749), 815–819.PubMedCrossRefGoogle Scholar
  73. Sasaki, K., Oka, H., Kawaguchi, S., Jinnai, K., & Yasuda, T. (1977). Mossy fibre and climbing fibre responses produced in the cerebellar cortex by stimulation of the cerebral cortex in monkeys. Experimental Brain Research, 29(3–4), 419–428.Google Scholar
  74. Schmid, A., Rees, G., Frith, C., & Barnes, G. (2001). An fMRI study of anticipation and learning of smooth pursuit eye movements in humans. Neuroreport, 12(7), 1409–1414.PubMedCrossRefGoogle Scholar
  75. Schultz, W., Apicella, P., & Ljungberg, T. (1993). Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. Journal of Neuroscience, 13(3), 900–913.PubMedGoogle Scholar
  76. Schultz, J., Imamizu, H., Kawato, M., & Frith, C. D. (2004). Activation of the human superior temporal gyrus during observation of goal attribution by intentional objects. Journal of Cognitive Neuroscience, 16(10), 1695–1705.PubMedCrossRefGoogle Scholar
  77. Shadmehr, R., & Holcomb, H. H. (1997). Neural correlates of motor memory consolidation. Science, 277(5327), 821–825.PubMedCrossRefGoogle Scholar
  78. Shidara, M., Kawano, K., Gomi, H., & Kawato, M. (1993). Inverse-dynamics model eye movement control by Purkinje cells in the cerebellum. Nature, 365(6441), 50–52.PubMedCrossRefGoogle Scholar
  79. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning. Cambridge, MA: MIT Press.Google Scholar
  80. Tamada, T., Miyauchi, S., Imamizu, H., Yoshioka, T., & Kawato, M. (1999). Cerebro-cerebellar functional connectivity revealed by the laterality index in tool-use learning. Neuroreport, 10(2), 325–331.PubMedCrossRefGoogle Scholar
  81. Tankersley, D., Stowe, C. J., & Huettel, S. A. (2007). Altruism is associated with an increased neural response to agency. Nature Neuroscience, 10(2), 150–151.PubMedCrossRefGoogle Scholar
  82. Wolpert, D. M., Doya, K., & Kawato, M. (2003). A unifying computational framework for motor control and social interaction. Philosophical Transactions of the Royal Society of London Series B Biological Sciences, 358(1431), 593–602.CrossRefGoogle Scholar
  83. Wolpert, D. M., Ghahramani, Z., & Jordan, M. I. (1995). An internal model for sensorimotor integration. Science, 269(5232), 1880–1882.PubMedCrossRefGoogle Scholar
  84. Wolpert, D. M., & Kawato, M. (1998). Multiple paired forward and inverse models for motor control. Neural Networks, 11, 1317–1329.PubMedCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  1. 1.Biological Information and Communications Technology GroupNational Institute of Information and Communications TechnologyKeihanna Science CityJapan
  2. 2.Computational Neuroscience LaboratoriesAdvanced Telecommunications Research Institute InternationalKeihanna Science CityJapan

Personalised recommendations