Biological Cybernetics

, Volume 102, Issue 3, pp 227–260 | Cite as

Action and behavior: a free-energy formulation

  • Karl J. FristonEmail author
  • Jean Daunizeau
  • James Kilner
  • Stefan J. Kiebel
Open Access
Original Paper


We have previously tried to explain perceptual inference and learning under a free-energy principle that pursues Helmholtz’s agenda to understand the brain in terms of energy minimization. It is fairly easy to show that making inferences about the causes of sensory data can be cast as the minimization of a free-energy bound on the likelihood of sensory inputs, given an internal model of how they were caused. In this article, we consider what would happen if the data themselves were sampled to minimize this bound. It transpires that the ensuing active sampling or inference is mandated by ergodic arguments based on the very existence of adaptive agents. Furthermore, it accounts for many aspects of motor behavior; from retinal stabilization to goal-seeking. In particular, it suggests that motor control can be understood as fulfilling prior expectations about proprioceptive sensations. This formulation can explain why adaptive behavior emerges in biological agents and suggests a simple alternative to optimal control theory. We illustrate these points using simulations of oculomotor control and then apply to same principles to cued and goal-directed movements. In short, the free-energy formulation may provide an alternative perspective on the motor control that places it in an intimate relationship with perception.


Computational Motor Control Bayesian Hierarchical Priors 

List of symbols

\({{\bf \Psi}\supseteq \{ \tilde{\bf{x}},{\tilde {\bf v}},{\boldsymbol \theta},{\boldsymbol \gamma}\}}, {\Psi \supseteq \{\tilde {x},\tilde {v},\theta ,\gamma \}}\)

Unknown causes of sensory input; variables in bold denote true values and those in italics denote variables assumed by the agent or model

\({\tilde{x}(t) = [x, {x}', {x}'',\ldots]^T, \dot{\tilde{x}}(t) = f(\tilde{x}, \tilde{v}, \theta)+\tilde{w}}\)

Generalised hidden-states that act on an agent. These are time-varying quantities that include all high-order temporal derivatives; they represent a point in generalised coordinates of motion that encodes a path or trajectory

\({\tilde{v}(t) = [v, {v}', {v}'',\ldots]^T}\)

Generalised forces or causal states that act on hidden states

\({\tilde{s}(t) = g(\tilde{x}, \tilde{v}, \theta) + \tilde{z}}\)

Generalised sensory states caused by hidden states

\({\theta \supseteq \{\theta_1 ,\theta_2, \ldots\}}\)

Parameters of the equations of motion and sensory mapping

\({\gamma \supseteq \{\gamma^s, \gamma^x, \gamma^v\}}\)

Parameters of the precision of random fluctuations \({\Pi(\gamma^i) : i \in s, x, v}\)

\({\tilde{w}(t) = [w, {w}', {w}'', \ldots]^T}\)

Generalised random fluctuations of the motion of hidden states

\({\tilde{z}(t) = [z, {z}', {z}'', \ldots]^T}\)

Generalised random fluctuations of sensory states

\({\tilde{n}(t) = [n, {n}', {n}'', \ldots]^T}\)

Generalised random fluctuations of causal states

\({\Pi^i := \Pi(\gamma^i) = \Sigma(\gamma^i)^{-1}}\)

Precisions or inverse covariances of generalised random fluctuations

\({{\bf g}(\tilde{\bf{x}}, {\tilde{\bf v}}, {\bf \theta})}\) , \({{\bf f}(\tilde{\bf{x}}, {\tilde{\bf v}}, \tilde{a}, {\bf \theta})}\)

Sensory mapping and equations of motion generating sensory states

\({g(\tilde{x}, \tilde{v}, \theta)}\) , \({f(\tilde{x}, \tilde{v}, \theta)}\)

Sensory mapping and equations of motion modeling sensory states


Policy: a scalar function of generalised sensory and internal states

\({p(\tilde{\bf{x}}|m)}\) , \({p(\tilde{s}|m)}\)

Ensemble densities; the density of the hidden and sensory states of agents at equilibrium with their environment.

\({D(q\vert \vert p) = \left\langle{{\rm ln}(q/p)}\right\rangle_q}\)

Kullback-Leibler divergence or cross-entropy between two densities

\({\langle \rangle_q }\)

Expectation or mean of under the density q


Model or agent; entailing the form of a generative model

\({H(X) = \left\langle {\ln p(\tilde{\bf{x}}\vert m)}\right\rangle_p H(S)=\left\langle {\ln p(\tilde {s}\vert m)}\right\rangle_p}\)

Entropy of generalised hidden and sensory states

\({}{-\ln p(\tilde{s}\vert m)}\)

Surprise or self-information of generalised sensory states

\({F(\tilde{s},\mu ) \ge -\ln p(\tilde{s}\vert m)}\)

Free-energy bound on surprise


Recognition density on causes Ψ with sufficient statistics μ

\({\mu =\{\tilde {\mu}(t),\mu _\theta ,\mu _\gamma \}} \tilde {\mu}=\{\tilde {\mu}_x ,\tilde {\mu}_v \}\)

Conditional or posterior expectation of the causes Ψ; these are the sufficient statistics of the Gaussian recognition density

\({\tilde{\eta}(t) = [\eta, {\eta}', {\eta}'', \ldots ]^T}\)

Prior expectation of generalised causal states

\({\xi_i = \Pi_i\tilde{\varepsilon}_i : i \in s, x, v}\)

Precision-weighted generalised prediction errors

\({{\tilde {\varepsilon}} = \left[ \begin{array}{l} \tilde {\varepsilon}_s =\tilde {s}-g(\mu)\\ \tilde {\varepsilon}_x =D\tilde {\mu}_x -f(\mu)\\ \tilde{\varepsilon}_v =\tilde {\mu}_v -\tilde {\eta}\end{array} \right]}\)

Generalised prediction error on sensory states, the motion of hidden states and forces or causal states.



We would like to thank Neil Burgess and Florentin Wörgötter for very helpful reviews of a previous version of this work.

Open Access

This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.


  1. Abbott LF, Varela JA, Sen K, Nelson SB (1997) Synaptic depression and cortical gain control. Science 275(5297): 220–224CrossRefPubMedGoogle Scholar
  2. Andersen RA (1989) Visual and eye movement factions of the posterior parietal cortex. Annu Rev Neurosci 12: 377–405CrossRefPubMedGoogle Scholar
  3. Anosov DV (2001) Ergodic theory. In Hazewinkel M (ed) Encyclopaedia of mathematics. Kluwer Academic Publishers. ISBN 978-1556080104Google Scholar
  4. Ballard DH, Hinton GE, Sejnowski TJ (1983) Parallel visual computation. Nature 306: 21–26CrossRefPubMedGoogle Scholar
  5. Barlow HB (1969) Pattern recognition and the responses of sensory neurons. Ann N Y Acad Sci 156: 872–881CrossRefPubMedGoogle Scholar
  6. Bays PM, Wolpert DM (2007) Computational principles of sensorimotor control that minimize uncertainty and variability. J Physiol 578(Pt 2): 387–396PubMedGoogle Scholar
  7. Bell CC, Grant K (1989) Corollary discharge inhibition and preservation of temporal information in a sensory nucleus of mormyrid electric fish. J Neurosci 9(3): 1029–1044PubMedGoogle Scholar
  8. Bellman R (1952) On the theory of dynamic programming. Proc Natl Acad Sci USA 38: 716–719CrossRefPubMedGoogle Scholar
  9. Bernard C (1974) Lectures on the phenomena common to animals and plants (trans: Hoff HE, Guillemin R, Guillemin L). Charles C Thomas, Springfield. ISBN 978-0398028572Google Scholar
  10. Berret B, Darlot C, Jean F, Pozzo T, Papaxanthis C, Gauthier JP (2008) The inactivation principle: mathematical solutions minimizing the absolute work and biological implications for the planning of arm movements. PLoS Comput Biol 4(10): e1000194CrossRefPubMedGoogle Scholar
  11. Bruyn JL, Mason AH (2009) Temporal coordination during bimanual reach-to-grasp movements: the role of vision. Q J Exp Psychol (Colchester) 5: 1–15Google Scholar
  12. Bütefisch CM, Davis BC, Sawaki L, Waldvogel D, Classen J, Kopylev L, Cohen LG (2002) Modulation of use-dependent plasticity by d-amphetamine. Ann Neurol 51(1): 59–68CrossRefPubMedGoogle Scholar
  13. Camerer CF (2003) Behavioural studies of strategic thinking in games. Trends Cogn Sci 7(5): 225–231CrossRefPubMedGoogle Scholar
  14. Crauel H, Flandoli F (1994) Attractor for random dynamical systems. Probab Theory Relat Fields 100: 365–393CrossRefGoogle Scholar
  15. Daw ND, Doya K (2006) The computational neurobiology of learning and reward. Curr Opin Neurobiol 16(2): 199–204CrossRefPubMedGoogle Scholar
  16. Dayan P, Hinton GE, Neal RM (1995) The Helmholtz machine. Neural Comput 7: 889–904CrossRefPubMedGoogle Scholar
  17. Deneve S (2008) Bayesian spiking neurons I: inference. Neural Comput 20(1): 91–117CrossRefPubMedGoogle Scholar
  18. Diedrichsen J, Dowling N (2009) Bimanual coordination as task-dependent linear control policies. Hum Mov Sci [Epub ahead of print]Google Scholar
  19. Diedrichsen J, Verstynen T, Hon A, Zhang Y, Ivry RB (2007) Illusions of force perception: the role of sensori-motor predictions, visual information, and motor errors. J Neurophysiol 97(5): 3305– 3313CrossRefPubMedGoogle Scholar
  20. Disney AA, Aoki C, Hawken MJ (2007) Gain modulation by nicotine in macaque v1. Neuron 56(4): 701–713CrossRefPubMedGoogle Scholar
  21. Doya K (2002) Metalearning and neuromodulation. Neural Netw 15(4–6): 495–506CrossRefPubMedGoogle Scholar
  22. Evans DJ (2003) A non-equilibrium free-energy theorem for deterministic systems. Mol Phys 101: 1551–1554CrossRefGoogle Scholar
  23. Feldman AG (2009) New insights into action-perception coupling. Exp Brain Res 194(1): 39–58CrossRefPubMedGoogle Scholar
  24. Feldman AG, Levin MF (1995) The origin and use of positional frames of reference in motor control. Behav Brain Sci 18: 723–806CrossRefGoogle Scholar
  25. Feynman RP (1972) Statistical mechanics. Benjamin, ReadingGoogle Scholar
  26. Fourneret P, Jeannerod M (1998) Limited conscious monitoring of motor performance in normal subjects. Neuropsychologia 36(11): 1133–1140CrossRefPubMedGoogle Scholar
  27. Friston K (2005) A theory of cortical responses. Philos Trans R Soc Lond B Biol Sci 360(1456): 815–836CrossRefPubMedGoogle Scholar
  28. Friston K (2008) Hierarchical models in the brain. PLoS Comput Biol 4(11):e1000211. PMID: 18989391Google Scholar
  29. Friston K (2010) The free-energy principle: a unified brain theory. Nat Rev Neurosci 11(2): 127–138CrossRefPubMedGoogle Scholar
  30. Friston K, Stephan KE (2007) Free-energy and the brain. Synthese 159: 417–458CrossRefPubMedGoogle Scholar
  31. Friston KJ, Tononi G, Reeke GN Jr, Sporns O, Edelman GM (1994) Value-dependent selection in the brain: simulation in a synthetic neural model. Neuroscience 59(2): 229–243CrossRefPubMedGoogle Scholar
  32. Friston K, Kilner J, Harrison L (2006) A free-energy principle for the brain. J Physiol (Paris) 100(1–3): 70–87CrossRefGoogle Scholar
  33. Friston KJ, Trujillo-Barreto N, Daunizeau J (2008) DEM: a variational treatment of dynamic systems. NeuroImage 41(3): 849–885CrossRefPubMedGoogle Scholar
  34. Friston KJ, Daunizeau J, Kiebel SJ (2009) Reinforcement learning or active inference?. PLoS One 4(7): e6421CrossRefPubMedGoogle Scholar
  35. Gontar V (2000) Entropy principle of extremality as a driving force in the discrete dynamics of complex and living systems. Chaos Solitons Fractals 11: 231–236CrossRefGoogle Scholar
  36. Gottlieb GL (1998) Rejecting the equilibrium-point hypothesis. Motor Control 2(1): 10–12PubMedGoogle Scholar
  37. Grafton ST, Hamilton AF (2007) Evidence for a distributed hierarchy of action representation in the brain. Hum Mov Sci 26(4): 590–616CrossRefPubMedGoogle Scholar
  38. Grafton ST, Schmitt P, Van Horn J, Diedrichsen J (2008) Neural substrates of visuomotor learning based on improved feedback control and prediction. Neuroimage 39(3): 1383–1395CrossRefPubMedGoogle Scholar
  39. Helmholtz H (1860/1962). Handbuch der physiologischen optik, vol 3 (English trans: Southall JPC, ed). Dover, New YorkGoogle Scholar
  40. Hinton GE, von Camp D (1993) Keeping neural networks simple by minimizing the description length of weights. In: Proceedings of COLT-93, pp 5–13Google Scholar
  41. Huffman KJ, Krubitzer L (2001) Area 3a: topographic organization and cortical connections in marmoset monkeys. Cereb Cortex 11: 849–867CrossRefPubMedGoogle Scholar
  42. Jax SA, Rosenbaum DA (2007) Hand path priming in manual obstacle avoidance: evidence that the dorsal stream does not only control visually guided actions in real time. J Exp Psychol Hum Percept Perform 33(2): 425–441CrossRefPubMedGoogle Scholar
  43. Jones DS (1979) Elementary information theory. Clarendon Press, New YorkGoogle Scholar
  44. Kawato M, Hayakawa H, Inui T (1993) A forward-inverse optics model of reciprocal connections between visual cortical areas. Network 4: 415–422CrossRefGoogle Scholar
  45. Kersten D, Mamassian P, Yuille A (2004) Object perception as Bayesian inference. Annu Rev Psychol 55: 271–304CrossRefPubMedGoogle Scholar
  46. Kiebel SJ, Daunizeau J, Friston KJ (2008) A hierarchy of time-scales and the brain. PLoS Comput Biol 4(11):e1000209. PMID: 19008936Google Scholar
  47. Knill DC, Pouget A (2004) The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci 27(12): 712–719CrossRefPubMedGoogle Scholar
  48. Körding KP, Wolpert DM (2004) Bayesian integration in sensorimotor learning. Nature 427(6971): 244–247CrossRefPubMedGoogle Scholar
  49. Körding KP, Beierholm U, Ma WJ, Quartz S, Tenenbaum JB, Shams L (2007) Causal inference in multisensory perception. PLoS ONE 2(9): e943CrossRefPubMedGoogle Scholar
  50. Kreisel SH, Hennerici MG, Bäzner H (2007) Pathophysiology of stroke rehabilitation: the natural course of clinical recovery, use-dependent plasticity and rehabilitative outcome. Cerebrovasc Dis 23(4): 243–255CrossRefPubMedGoogle Scholar
  51. Kulvicius T, Porr B, Wörgötter F (2007) Development of receptive fields in a closed-loop behavioral system. Neurocomputing 70: 2046–2049CrossRefGoogle Scholar
  52. Lee TS, Mumford D (2003) Hierarchical Bayesian inference in the visual cortex. J Opt Soc Am A Opt Image Sci Vis 20: 1434–1448CrossRefPubMedGoogle Scholar
  53. Linsker R (1990) Perceptual neural organization: some approaches based on network models and information theory. Annu Rev Neurosci 13: 257–281CrossRefPubMedGoogle Scholar
  54. Liu D, Todorov E (2007) Evidence for the flexible sensorimotor strategies predicted by optimal feedback control. J Neurosci 27(35): 9354–9368CrossRefPubMedGoogle Scholar
  55. Mackay DJC (1992) Information-based objective functions for active data selection. Neural Comput 4: 590–604CrossRefGoogle Scholar
  56. MacKay DJC (1995) Free-energy minimization algorithm for decoding and cryptoanalysis. Electron Lett 31: 445–447CrossRefGoogle Scholar
  57. Manoonpong P, Geng T, Kulvicius T, Porr B, Wörgötter F (2007) Adaptive, fast walking in a biped robot under neuronal control and learning. PLoS Comput Biol 3(7): e134CrossRefPubMedGoogle Scholar
  58. Maturana HR, Varela F (1972) De máquinas y seres vivos. Editorial Universitaria, Santiago. English version: “Autopoiesis: the organization of the living,” in Maturana HR, Varela FG (1980) Autopoiesis and cognition. Reidel, DordrechtGoogle Scholar
  59. Montague PR, Dayan P, Person C, Sejnowski TJ (1995) Bee foraging in uncertain environments using predictive Hebbian learning. Nature 377(6551): 725–728CrossRefPubMedGoogle Scholar
  60. Mumford D (1992) On the computational architecture of the neocortex. II. The role of cortico-cortical loops. Biol Cybern 66: 241–251CrossRefPubMedGoogle Scholar
  61. Mussa Ivaldi FA, Morasso P, Zaccaria R (1988) A distributed model for representing and regularizing motor redundancy. Biol Cybern 60(1): 1–16PubMedGoogle Scholar
  62. Nakano E, Imamizu H, Osu R, Uno Y, Gomi H, Yoshioka T, Kawato M (1999) Quantitative examinations of internal representations for arm trajectory planning: minimum commanded torque change model. J Neurophysiol 81(5): 2140–2155PubMedGoogle Scholar
  63. Neal RM, Hinton GE (1998) A view of the EM algorithm that justifies incremental, sparse, and other variants. In: Jordan MI (eds) Learning in graphical models. Kluwer Academic Publishers, Dordrecht, pp 355–368Google Scholar
  64. Olshausen BA, Field DJ (1996) Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381: 607–609CrossRefPubMedGoogle Scholar
  65. O’Reilly RC, Noelle DC, Braver TS, Cohen JD (2002) Prefrontal cortex and dynamic categorization tasks: representational organization and neuromodulatory control. Cereb Cortex 12(3): 246–257CrossRefPubMedGoogle Scholar
  66. Paulignan Y, MacKenzie C, Marteniuk R, Jeannerod M (1991) Selective perturbation of visual input during prehension movements. 1. The effects of changing object position. Exp Brain Res 83(3): 502–512CrossRefPubMedGoogle Scholar
  67. Paulin MG (2005) Evolution of the cerebellum as a neuronal machine for Bayesian state estimation. J Neural Eng 2(3): S219–S234CrossRefPubMedGoogle Scholar
  68. Porr B, Wörgötter F (2003) Isotropic sequence order learning. Neural Comput 15(4): 831–864CrossRefPubMedGoogle Scholar
  69. Prinz AA (2006) Insights from models of rhythmic motor systems. Curr Opin Neurobiol 16(6): 615–620CrossRefPubMedGoogle Scholar
  70. Rao RP, Ballard DH (1998) Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive field effects. Nat Neurosci 2: 79–87CrossRefGoogle Scholar
  71. Redgrave P, Prescott TJ, Gurney K (1999) The basal ganglia: a vertebrate solution to the selection problem?. Neuroscience 89(4): 1009–1023CrossRefPubMedGoogle Scholar
  72. Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF (eds) Classical conditioning II: current research and theory. Appleton Century Crofts, New York, pp 64–99Google Scholar
  73. Schaal S, Mohajerian P, Ijspeert A (2007) Dynamics systems vs. optimal control—a unifying view. Prog Brain Res 165: 425–445CrossRefPubMedGoogle Scholar
  74. Scheidt RA, Reinkensmeyer DJ, Conditt MA, Rymer WZ, Mussa-Ivaldi FA (2000) Persistence of motor adaptation during constrained, multi-joint, arm movements. J Neurophysiol 84: 853–862PubMedGoogle Scholar
  75. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275: 1593–1599CrossRefPubMedGoogle Scholar
  76. Schweitzer F (2003) Brownian agents and active particles: collective dynamics in the natural and social sciences. Springer Series in Synergetics, 1st ed 2003, 2nd printing 2007. ISBN: 978-3-540-73844-2Google Scholar
  77. Shadmehr R, Krakauer JW (2008) A computational neuroanatomy for motor control. Exp Brain Res 185(3): 359–381CrossRefPubMedGoogle Scholar
  78. Sutton RS (1996) Generalization in reinforcement learning: successful examples using sparse coarse coding. Adv Neural Inf Process Syst 8: 1038–1044Google Scholar
  79. Sutton RS, Barto AG (1981) Toward a modern theory of adaptive networks: expectation and prediction. Psychol Rev 88(2): 135– 170CrossRefPubMedGoogle Scholar
  80. Tatler BW, Wade NJ (2003) On nystagmus, saccades, and fixations. Perception 32(2): 167–184CrossRefPubMedGoogle Scholar
  81. Todorov E (2006) Linearly-solvable Markov decision problems. In: Scholkopf B et al (eds) Advances in neural information processing systems, vol 19. MIT Press, Cambridge, pp 1369–1376Google Scholar
  82. Todorov E, Jordan MI (1998) Smoothness maximization along a predefined path accurately predicts the speed profiles of complex arm movements. J Neurophysiol 80(2): 696–714PubMedGoogle Scholar
  83. Todorov E, Jordan MI (2002) Optimal feedback control as a theory of motor coordination. Nat Neurosci 5(11): 1226–1235CrossRefPubMedGoogle Scholar
  84. Toussaint M (2009) Probabilistic inference as a model of planned behavior. Künstliche Intelligenz Ger Artif Intell J (in press)Google Scholar
  85. Tschacher W, Haken H (2007) Intentionality in non-equilibrium systems? The functional aspects of self-organized pattern formation. New Ideas Psychol 25: 1–15CrossRefGoogle Scholar
  86. Tseng YW, Diedrichsen J, Krakauer JW, Shadmehr R, Bastian AJ (2007) Sensory prediction errors drive cerebellum-dependent adaptation of reaching. J Neurophysiol 98(1): 54–62CrossRefPubMedGoogle Scholar
  87. Verschure PF, Voegtlin T (1998) A botom up approach towards the acquisition and expression of sequential representations applied to a behaving real-world device: Distributed Adaptive Control III. Neural Netw 11(7–8): 1531–1549CrossRefPubMedGoogle Scholar
  88. Verschure PF, Voegtlin T, Douglas RJ (2003) Environmentally mediated synergy between perception and behavior in mobile robots. Nature 425: 620–624CrossRefPubMedGoogle Scholar
  89. Voss M, Ingram JN, Wolpert DM, Haggard P (2008) Mere expectation to move causes attenuation of sensory signals. PLoS ONE 3(8): e2866CrossRefPubMedGoogle Scholar
  90. Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8: 279–292Google Scholar
  91. Wei K, Körding KP (2009) Relevance of error: what drives motor adaptation?. J Neurophysiol 101(2): 655–664CrossRefPubMedGoogle Scholar
  92. Wolpert DM, Miall RC (1996) Forward models for physiological motor control. Neural Netw 9(8): 1265–1279CrossRefPubMedGoogle Scholar
  93. Wolpert DM, Ghahramani Z, Jordan MI (1995) An internal model for sensorimotor integration. Science 269(5232): 1880–1882CrossRefPubMedGoogle Scholar
  94. Wörgötter F, Porr B (2005) Temporal sequence learning, prediction, and control: a review of different models and their relation to biological mechanisms. Neural Comput 17(2): 245–319CrossRefPubMedGoogle Scholar
  95. Yu AJ, Dayan P (2005) Uncertainty, neuromodulation and attention. Neuron 46: 681–692CrossRefPubMedGoogle Scholar

Copyright information

© The Author(s) 2010

Open AccessThis is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (, which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Authors and Affiliations

  • Karl J. Friston
    • 1
    Email author
  • Jean Daunizeau
    • 1
  • James Kilner
    • 1
  • Stefan J. Kiebel
    • 1
  1. 1.The Wellcome Trust Centre for Neuroimaging, Institute of NeurologyUniversity College LondonLondonUK

Personalised recommendations