Optimal isn’t good enough

Abstract

The notion that biological systems come to embody optimal solutions seems consistent with the competitive drive of evolution. It has been used to interpret many examples of sensorimotor behavior. It is attractive from the viewpoint of control engineers because it solves the redundancy problem by identifying the one optimal motor strategy out of many similarly acceptable possibilities. This perspective examines whether there is sufficient basis to apply the formal engineering tools of optimal control to a reductionist understanding of biological systems. For an experimental biologist, this translates into whether the theory of optimal control generates nontrivial and testable hypotheses that accurately predict novel phenomena, ideally at deeper levels of structure than the observable behavior. The methodology of optimal control is applicable when there is (i) a single, known cost function to be optimized, (ii) an invertible model of the plant, and (iii) simple noise interfering with optimal performance. None of these is likely to be true for biological organisms. Furthermore, their motivation is usually good-enough rather than globally optimal behavior. Even then, the performance of a biological organism is often much farther from optimal than the physical limits of its hardware because the brain is continuously testing the acceptable limits of performance as well as just performing the task. This perspective considers an alternative strategy called “good-enough” control, in which the organism uses trial-and-error learning to acquire a repertoire of sensorimotor behaviors that are known to be useful, but not necessarily optimal. This leads to a diversity of solutions that tends to confer robustness on the individual organism and its evolution. It is also more consistent with the capabilities of higher sensorimotor structures, such as cerebral cortex, which seems to be designed to classify and recall complex sets of information, thereby allowing the organism to learn from experience, rather than to compute new strategies online. Optimal control has been a useful metaphor for understanding some superficial aspects of motor psychophysics. Reductionists who want to understand the underlying neural mechanisms need to move on.

This is a preview of subscription content, access via your institution.

References

  1. Anderson FC, Pandy MG (1999) A dynamic optimization solution for vertical jumping in three dimensions. Comput Methods Biomech Biomed Eng 2: 201–231

    Article  Google Scholar 

  2. Bernstein NA (1967) The co-ordination and regulation of movement. In: Whiting HTA (ed) (Translation) Human motor actions: Bernstein reassessed. Elsevier, Amsterdam

  3. Bjursten LM, Norrsell K, Norrsell U (1976) Behavioural repertory of cats without cerebral cortex from infancy. Exp Brain Res 25: 115–130

    CAS  PubMed  Article  Google Scholar 

  4. Boyd SP, Vandenberghe L (2004) Convex optimization. Cambridge University Press, New York

    Book  Google Scholar 

  5. Buchanan TS, Shreeve DA (1996) An evaluation of optimization techniques for the prediction of muscle activation patterns during isometric tasks. J Biomech Eng 118: 565–574

    CAS  PubMed  Article  Google Scholar 

  6. Churchland MM, Afshar A, Shenoy KV (2006) A central source of movement variability. Neuron 52: 1085–1096

    CAS  PubMed Central  PubMed  Article  Google Scholar 

  7. Crowninshield RD, Brand RA (1981) A physiologically based criterion of muscle force prediction in locomotion. J Biomech 14: 793–801

    CAS  PubMed  Article  Google Scholar 

  8. Culler E, Mettler FA (1934) Conditioned behavior in a decorticate dog. J Comp Psychol 18: 291–303

    Article  Google Scholar 

  9. de Rugy A, Loeb GE, Carroll TJ (2012) Muscle coordination is habitual rather than optimal. J Neurosci 32: 7384–7391

    PubMed  Article  Google Scholar 

  10. Diamond IT (1979) The subdivisions of neocortex: a proposal to revise the traditional view of sensory, motor, and association areas. Prog Psychobiol Physiol Psychol 8: 1–43

    Google Scholar 

  11. Flanders M (2011) What is the biological basis of sensorimotor integration?. Biol Cybern 104: 1–8

    PubMed Central  PubMed  Article  Google Scholar 

  12. Flash T, Hogan N (1985) The coordination of arm movements: an experimentally confirmed mathematical model. J Neurosci 5: 1688–1703

    CAS  PubMed  Google Scholar 

  13. Friston K (2010) The free-energy principle: a unified brain theory. Nat Rev Neurosci 11: 127–138

    CAS  PubMed  Article  Google Scholar 

  14. Gallistel CR, Fairhurst S, Balsam P (2004) The learning curve: implications of a quantitative analysis. Proc Natl Acad Sci USA 101: 13124–13131

    CAS  PubMed Central  PubMed  Article  Google Scholar 

  15. Gill PE, Murray W, Wright MH (1982) Practical optimization. Emerald Group Publishing Limited, West Yorkshire

    Google Scholar 

  16. Goaillard J-M, Taylor AL, Schulz DJ, Marder E (2009) Functional consequences of animal-to-animal variation in circuit parameters. Nat Neurosci 12: 1424–1430

    CAS  PubMed Central  PubMed  Article  Google Scholar 

  17. Grashow R, Brookings T, Marder E (2010) Compensation for variable intrinsic neuronal excitability by circuit-synaptic interactions. J Neurosci 30: 9145–9156

    CAS  PubMed Central  PubMed  Article  Google Scholar 

  18. Hamilton AFd, Jones KE (2004) The scaling of motor noise with muscle strength and motor unit number in humans. Exp Brain Res 157: 417–430

    PubMed  Article  Google Scholar 

  19. Harper D (2010) optimal. In: Online Etymology Dictionary. http://www.etymonline.com/index.php?allowed_in_frame=0&search=optimal&searchmode=none

  20. He J, Levine WS, Loeb GE (1991) Feedback gains for correcting small perturbations to standing posture. IEEE Trans Autom Control 36: 322–332

    Article  Google Scholar 

  21. Jones KE, Hamilton AFd, Wolpert DM (2002) Sources of signal-dependent noise during isometric force production. J Neurophysiol 88: 1533–1544

    PubMed  Google Scholar 

  22. Kawato M (1999) Internal models for motor control and trajectory planning. Curr Opin Neurobiol 9: 718–727

    CAS  PubMed  Article  Google Scholar 

  23. Kawato M, Gomi H (1992) A computational model of four regions of the cerebellum based on feedback-error learning. BiolCybern 68: 95–103

    CAS  Google Scholar 

  24. Lagarias JC, Reeds JA, Wright MH, Wright PE (1998) Convergence properties of the Nelder-Mead simplex method in low dimensions. SIAM J OPTIM 9: 112–147

    Article  Google Scholar 

  25. Li Y, Levine WS, Loeb GE (2012) A two-joint human posture control model with realistic neural delays. IEEE Trans Neural Syst Rehabil Eng. doi:10.1109/TNSRE.2012.2199333

  26. Loeb GE (1983) Finding common ground between robotics and physiology. Trends Neurosci 6: 203–204

    Article  Google Scholar 

  27. Loeb GE (1993) The distal hindlimb musculature of the cat: interanimal variability of locomotor activity and cutaneous reflexes. Exp Brain Res 96: 125–140

    CAS  PubMed  Article  Google Scholar 

  28. Loeb GE (1999) Asymmetry of hindlimb muscle activity and cutaneous reflexes after tendon transfers in kittens. J Neurophysiol 82: 3392–3405

    CAS  PubMed  Google Scholar 

  29. Loeb GE, Levine WS, He J (1990) Understanding sensorimotor feedback through optimal control. Cold Spring Harbor Symp Quant Biol 55: 791–803

    CAS  PubMed  Article  Google Scholar 

  30. Marr D (1982) Vision. W.H. Freeman & Co, New York

    Google Scholar 

  31. McCrea DA, Rybak IA (2008) Organization of mammalian locomotor rhythm and pattern generation. Brain Res Rev 57: 134–146

    PubMed Central  PubMed  Article  Google Scholar 

  32. Osu R, Gomi H (1999) Multijoint muscle regulation mechanisms examined by measured human arm stiffness and EMG signals. J Neurophysiol 81: 1458–1468

    CAS  PubMed  Google Scholar 

  33. Pandy MG, Anderson FC, Hull DG (1992) A parameter optimization approach for the optimal control of large-scale musculoskeletal systems. J Biomech Eng 114: 450–460

    CAS  PubMed  Article  Google Scholar 

  34. Pandy MG, Zajac FE, Sim E, Levine WS (1990) An optimal control model for maximum-height human jumping. J Biomech 23: 1185–1198

    CAS  PubMed  Article  Google Scholar 

  35. Partridge LD (1982) The good enough calculi of evolving control systems: evolution is not engineering. Am J Physiol 242: R173–R177

    CAS  PubMed  Google Scholar 

  36. Pedotti A, Krishnan VV, Stark L (1978) Optimization of muscle-force sequencing in human locomotion. Math Biosci 38: 57–76

    Article  Google Scholar 

  37. Pellionisz AJ, Peterson BW (1988) A tensorial model of neck motor activation. In: Peterson BW, Richmond FJR (eds) Control of head movement. Oxford University Press, New York, pp 178–186

  38. Perfiliev S, Isa T, Johnels B, Steg G, Wessberg J (2010) Reflexive limb selection and control of reach direction to moving targets in cats, monkeys, and humans. J Neurophysiol 104: 2423–2432

    PubMed  Article  Google Scholar 

  39. Pierrot-Deseilligny E, Burke DC (2005) The circuitry of the human spinal cord: its role in motor control and movement disorders. Cambridge University Press, Cambridge

    Book  Google Scholar 

  40. Raikova RT, Gabriel DA, Aladjov H (2005) Experimental and modelling investigation of learning a fast elbow flexion in the horizontal plane. J Biomech 38: 2070–2077

    PubMed  Article  Google Scholar 

  41. Raphael G, Tsianos GA, Loeb GE (2010) Spinal-like regulator facilitates control of a two-degree-of-freedom wrist. J Neurosci 30: 9431–9444

    CAS  PubMed  Article  Google Scholar 

  42. Rathelot JA, Strick PL (2009) Subdivisions of primary motor cortex based on cortico-motoneuronal cells. PNAS 106: 918–923

    CAS  PubMed Central  PubMed  Article  Google Scholar 

  43. Sabes PN, Jordan MI (1997) Obstacle avoidance and a perturbation sensitivity model for motor planning. J Neurosci (New York) 17: 7119–7128

    CAS  Google Scholar 

  44. Slifkin AB, Newell KM (1998) Is variability in human performance a reflection of system noise. Curr Dir Psychol Sci 7: 170–177

    Article  Google Scholar 

  45. Tang LS, Goeritz ML, Caplan JS, Taylor AL, Fisek M, Marder E (2010) Precise temperature compensation of phase in a rhythmic motor pattern. PLoS Biol 8: e1000469

    PubMed Central  PubMed  Article  Google Scholar 

  46. Terekhov A, Zatsiorsky V (2011) Analytical and numerical analysis of inverse optimization problems: conditions of uniqueness and computational methods. Biol Cybern 104: 75–93

    PubMed Central  PubMed  Article  Google Scholar 

  47. Todorov E, Jordan MI (2002) Optimal feedback control as a theory of motor coordination. Nat Neurosci 5: 1226–1235

    CAS  PubMed  Article  Google Scholar 

  48. Tsianos GA, Raphael G, Loeb GE (2011) Modeling the potentiality of spinal-like circuitry for stabilization of a planar arm system. Prog Brain Res 194: 203–213

    PubMed  Article  Google Scholar 

  49. Tsianos GA, Rustin C, Loeb GE (2012) Mammalian muscle model for predicting force and energetics during physiological behaviors. IEEE Trans Neural Syst Rehabil Eng 20: 117–133

    PubMed  Article  Google Scholar 

  50. Valero-Cuevas FJ, Hoffmann H, Kurse MU, Kutch JJ, Theodorou EA (2009) Computational models for neuromuscular function. IEEE Rev Biomed Eng 2: 110–135

    PubMed Central  PubMed  Article  Google Scholar 

  51. Venkataraman P (2009) Applied optimization with MATLAB programming, Chapt 6. Wiley, Somerset

  52. Yuan G, Chang K, Hsieh C, Lin C (2010) A comparison of optimization methods for large-scale L1-regularized linear classification. Linear Classif J Mach Learn Res 2: 3183–3234

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Gerald E. Loeb.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Loeb, G.E. Optimal isn’t good enough. Biol Cybern 106, 757–765 (2012). https://doi.org/10.1007/s00422-012-0514-6

Download citation

Keywords

  • Sensorimotor
  • Optimal control
  • Spinal cord
  • Internal model
  • Motor learning
  • Feedback
  • Cost
  • Motor noise