Skip to main content
SpringerLink
Log in

Sure enough: efficient Bayesian learning and choice

  • Original Paper
  • Published:
Animal Cognition Aims and scope Submit manuscript

Abstract

Probabilistic decision-making is a general phenomenon in animal behavior, and has often been interpreted to reflect the relative certainty of animals’ beliefs. Extensive neurological and behavioral results increasingly suggest that animal beliefs may be represented as probability distributions, with explicit accounting of uncertainty. Accordingly, we develop a model that describes decision-making in a manner consistent with this understanding of neuronal function in learning and conditioning. This first-order Markov, recursive Bayesian algorithm is as parsimonious as its minimalist point-estimate, Rescorla–Wagner analogue. We show that the Bayesian algorithm can reproduce naturalistic patterns of probabilistic foraging, in simulations of an experiment in bumblebees. We go on to show that the Bayesian algorithm can efficiently describe the behavior of several heuristic models of decision-making, and is consistent with the ubiquitous variation in choice that we observe within and between individuals in implementing heuristic decision-making. By describing learning and decision-making in a single Bayesian framework, we believe we can realistically unify descriptions of behavior across contexts and organisms. A unified cognitive model of this kind may facilitate descriptions of behavioral evolution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Abrahams MV (1989) Foraging guppies and the ideal free distribution: the influence of information on patch choice. Ethology 82(2):116–126

    Article  Google Scholar 

  • Alatalo RV, Hglund J, Lundberg A, Sutherland WJ (1992) Evolution of black grouse leks: female preferences benefit males in larger leks. Behav Ecol 3(1):53–59

    Article  Google Scholar 

  • Behrens TE, Woolrich MW, Walton ME, Rushworth MF (2007) Learning the value of information in an uncertain world. Nat Neurosci 10(9):1214–1221

    Article  CAS  PubMed  Google Scholar 

  • Biernaskie JM, Walker SC, Gegear RJ (2009) Bumblebees learn to forage like Bayesians. Am Nat 174(3):413–423

    Article  PubMed  Google Scholar 

  • Bouton ME (2002) Context, ambiguity, and unlearning: sources of relapse after behavioral extinction. Biol Psychiatry 52(10):976–986

    Article  PubMed  Google Scholar 

  • Bowers JS, Davis CJ (2012a) Bayesian just-so stories in psychology and neuroscience. Psychol Bull 138(3):389–414

    Article  PubMed  Google Scholar 

  • Bowers JS, Davis CJ (2012b) Is that what Bayesians believe? Reply to Griffiths, Chater, Norris, and Pouget (2012). Psychol Bull 138(3):423

    Article  PubMed  Google Scholar 

  • Bröder A (2000) Assessing the empirical validity of the take-the-best heuristic as a model of human probabilistic inference. J Exp Psychol Learn 26(1):1332–1346

    Article  Google Scholar 

  • Cartoni E, Puglisi-Allegra S, Baldassarre G (2013) The three principles of action: a Pavlovian-instrumental transfer hypothesis. Front Behav Neurosci 7:153

    Article  PubMed  PubMed Central  Google Scholar 

  • Chater N, Oaksford M, Hahn U, Heit E (2010) Bayesian models of cognition. Wiley Interdiscip Rev Cogn Sci 1(6):811–823

    Article  PubMed  Google Scholar 

  • Cnaani J, Thomson JD, Papaj DR (2006) Flower choice and learning in foraging bumblebees: effects of variation in nectar volume and concentration. Ethology 112(3):278–285

    Article  Google Scholar 

  • Courville AC, Daw ND, Touretzky DS (2006) Bayesian theories of conditioning in a changing world. Trends Cogn Sci 10(7):294–300

    Article  PubMed  Google Scholar 

  • Dayan P, Kakade S, Montague PR (2000) Learning and selective attention. Nat Neurosci 3(Suppl 11):1218–1223

    Article  CAS  PubMed  Google Scholar 

  • Devenport LD (1998) Spontaneous recovery without interference: why remembering is adaptive. Anim Learn Behav 26(2):172–181

    Article  Google Scholar 

  • Devenport L, Hill T, Wilson M, Ogden E (1997) Tracking and averaging in variable environments: a transition rule. J Exp Psychol Anim B 23(4):450–460

    Article  Google Scholar 

  • Dunsmoor J, Niv Y, Daw N, Phelps E (2015) Rethinking extinction. Neuron 88(1):47–63

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Fischer BJ, Pea JL (2011) Owls behavior and neural representation predicted by Bayesian inference. Nat Neurosci 14(8):1061–1066

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Foley BR, Saltz JB, Nuzhdin SV, Marjoram P (2015) A Bayesian approach to social structure uncovers cryptic regulation of group dynamics in Drosophila melanogaster. Am Nat 185(6):797–808

    Article  PubMed  PubMed Central  Google Scholar 

  • Fusi S, Asaad WF, Miller EK, Wang XJ (2007) A neural circuit model of flexible sensorimotor mapping: learning and forgetting on multiple timescales. Neuron 54(2):319–333

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB (2013) Bayesian data analysis, 3rd edn. Chapman and Hall, London

    Google Scholar 

  • Gershman SJ (2015) A unifying probabilistic view of associative learning. PLoS Comput Biol 11(11):e1004567

    Article  PubMed  PubMed Central  Google Scholar 

  • Gershman SJ, Blei DM, Niv Y (2010) Context, learning, and extinction. Psychol Rev 117(1):197–209

    Article  PubMed  Google Scholar 

  • Gibbon J, Church RM, Meck WH (1984) Scalar timing in memory. Ann NY Acad Sci 423:52–77

    Article  CAS  PubMed  Google Scholar 

  • Gigerenzer G, Gaissmaier W (2011) Heuristic decision making. Annu Rev Psychol 62:451–482

    Article  PubMed  Google Scholar 

  • Green RF (2006) A simpler, more general method of finding the optimal foraging strategy for Bayesian birds. Oikos 112(2):274–284

    Article  Google Scholar 

  • Greggers U, Menzel R (1993) Memory dynamics and foraging strategies of honeybees. Behav Ecol Sociobiol 32(1):17–29

    Article  Google Scholar 

  • Griffiths TL, Chater N, Norris D, Pouget A (2012) How the Bayesians got their beliefs (and what those beliefs actually are): comment on Bowers and Davis (2012). Psychol Bull 138(3):415–422

    Article  PubMed  Google Scholar 

  • Harvey N, Bolger F (2001) Collecting information: optimizing outcomes, screening options, or facilitating discrimination? Q J Exp Psychol A 54(1):269–301

    Article  CAS  PubMed  Google Scholar 

  • Hausmann D, Läge D (2008) Sequential evidence accumulation in decision making: the individual desired level of confidence can explain the extent of information acquisition. Judgm Decis Mak 3(3):229–243

    Google Scholar 

  • Hutchinson JMC, Gigerenzer G (2005) Simple heuristics and rules of thumb: where psychologists and behavioural biologists might meet. Behav Process 69(2):97–124

    Article  Google Scholar 

  • Iwasa Y, Higashi M, Yamamura N (1981) Prey distribution as a factor determining the choice of optimal foraging strategy. Am Nat 117(5):710–723

    Article  Google Scholar 

  • Janmaat KR, Ban SD, Boesch C (2013) Ta chimpanzees use botanical skills to discover fruit: what we can learn from their mistakes. Anim Cogn 16(6):851–860

    Article  PubMed  Google Scholar 

  • Jones M, Love BC (2011) Bayesian fundamentalism or enlightenment? On the explanatory status and theoretical contributions of Bayesian models of cognition. Behav Brain Sci 34(4):169–188 (discussion 188–231)

    Article  PubMed  Google Scholar 

  • Jovani R, Mavor R (2011) Group size versus individual group size frequency distributions: a nontrivial distinction. Anim Behav 82:1027–1036

    Article  Google Scholar 

  • Kaelbling L, Littman M, Moore A (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285

    Google Scholar 

  • Kakade S, Dayan P (2002) Acquisition and extinction in autoshaping. Psychol Rev 109(3):533–544

    Article  PubMed  Google Scholar 

  • Kauffman A, Parsons L, Stein G, Wills A, Kaletsky R, Murphy C (2011) C. elegans positive butanone learning, short-term, and long-term associative memory assays. JoVE 49:e2490

    Google Scholar 

  • Kensinger BJ, Luttbeg B (2014) The limitations of inferring decision rule use from individuals sampling behaviour: a computational test of old and new algorithms. Evol Ecol Res 16(2):179–194

    Google Scholar 

  • Knill DC, Pouget A (2004) The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci 27(12):712–719

    Article  CAS  PubMed  Google Scholar 

  • Körding K (2007) Decision theory: what should the nervous system do? Science 318(5850):606–610

    Article  PubMed  Google Scholar 

  • Kraemer PJ, Golding JM (1997) Adaptive forgetting in animals. Psychon Bull Rev 4(4):480–491

    Article  Google Scholar 

  • Krakauer DC, Rodríguez-Gironés MA (1995) Searching and learning in a random environment. J. Theor. Biol. 177(4):417–429

    Article  Google Scholar 

  • Kruschke J (2008) Bayesian approaches to associative learning: from passive to active learning. Learn Behav 36(3):210–226

    Article  PubMed  Google Scholar 

  • Le Pelley ME, Mitchell CJ, Beesley T, George DN, Wills AJ (2016) Attention and associative learning in humans: an integrative review. Psychol Bull 142(10):131

    Article  Google Scholar 

  • Liu CC, Watanabe T (2012) Accounting for speed-accuracy tradeoff in perceptual learning. Vision Res 61:10714

    Google Scholar 

  • Luttbeg B (1996) A comparative Bayes tactic for mate assessment and choice. Behav Ecol 7(4):451–460

    Article  Google Scholar 

  • Luttbeg B (2002) Assessing the robustness and optimality of alternative decision rules with varying assumptions. Anim Behav 63(4):805–814

    Article  Google Scholar 

  • Luttbeg B, Langen TA (2004) Comparing alternative models to empirical data: cognitive models of western scrub-jay foraging behavior. Am Nat 163(2):263–276

    Article  PubMed  Google Scholar 

  • Luttbeg B, Warner RR (1999) Reproductive decision-making by female peacock wrasses: flexible versus fixed behavioral rules in variable environments. Behav Ecol 10(6):666–674

    Article  Google Scholar 

  • Ma WJ, Jazayeri M (2014) Neural coding of uncertainty and probability. Annu Rev Neurosci 37(1):205–220

    Article  CAS  PubMed  Google Scholar 

  • Mackenzie A, Reynolds J, Brown V, Sutherland W (1995) Variation in male mating success on leks. Am Nat 145(4):633–652

    Article  Google Scholar 

  • Maia TV (2009) Reinforcement learning, conditioning, and the brain: successes and challenges. Cogn Affect Behav Neurosci 9(4):343–364

    Article  PubMed  Google Scholar 

  • Matzel LD, Schachtman TR, Miller RR (1985) Recovery of an overshadowed association achieved by extinction of the overshadowing stimulus. Learn Motiv 16(4):398–412

    Article  Google Scholar 

  • McNamara JM, Houston AI (1987) Memory and the efficient use of information. J Theor Biol 125(4):385–395

    Article  CAS  PubMed  Google Scholar 

  • McNamara JM, Green RF, Olsson O (2006) Bayes theorem and its applications in animal behaviour. Oikos 112(2):243–251

    Article  Google Scholar 

  • Menzel R (1993) Associative learning in honey bees. Apidologie 24(3):157–168

    Article  Google Scholar 

  • Michelena P, Sibbald AM, Erhard HW, McLeod JE (2009) Effects of group size and personality on social foraging: the distribution of sheep across patches. Behav Ecol 20(1):145–152

    Article  Google Scholar 

  • Newell BR, Shanks DR (2003) Take the best or look at the rest? Factors influencing one-reason decision making. J Exp Psychol Learn 29(1):53–65

    Article  Google Scholar 

  • Newell BR, Weston NJ, Shanks DR (2003) Empirical tests of a fast-and-frugal heuristic: not everyone takes-the-best. Organ Behav Hum Decision 91(1):82–96

    Article  Google Scholar 

  • Olsson O, Brown JS (2006) The foraging benefits of information and the penalty of ignorance. Oikos 112(2):260–273

    Article  Google Scholar 

  • Olsson O, Holmgren NMA (1998) The survival-rate-maximizing policy for Bayesian foragers: wait for good news. Behav Ecol 9(4):345–353

    Article  Google Scholar 

  • Oster G, Heinrich B (1976) Why do bumblebees major? A mathematical model. Ecol Monogr 46(2):129

    Article  Google Scholar 

  • Payzan-LeNestour E, Bossaerts P (2011) Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings. PLoS Comput Biol 7(1):e1001048

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Pelé M, Sueur C (2013) Decision-making theories: linking the disparate research areas of individual and collective cognition. Anim Cogn 16(4):543–556

    Article  PubMed  Google Scholar 

  • Pouget A, Beck JM, Ma WJ, Latham PE (2013) Probabilistic brains: knowns and unknowns. Nat Neurosci 16(9):1170–1178

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Real L (1990) Search theory and mate choice. Am Nat 136(3):376–404

    Article  Google Scholar 

  • Real L, Ellner S, Harder LD (1990) Short-term energy maximization and risk-aversion in bumble bees: a reply to Possingham. Ecology 71(4):1625–1628

    Article  Google Scholar 

  • Real LA (1981) Uncertainty and pollinator-plant interactions: the foraging behavior of bees and wasps on artificial flowers. Ecology 62(1):20–26

    Article  Google Scholar 

  • Reger ML, Poulos AM, Buen F, Giza CC, Hovda DA, Fanselow MS (2012) Concussive brain injury enhances fear learning and excitatory processes in the amygdala. Biol Psychiatry 71(4):335–343

    Article  PubMed  Google Scholar 

  • Rescorla RA (2004) Spontaneous recovery. Learn Mem 11(5):501–509

    Article  PubMed  Google Scholar 

  • Roche J, Stubbs D, Glanz W (1996) Assessment and choice: an operant simulation of foraging in patches. J Exp Anal Behav 66(3):327–347

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Roesch MR, Esber GR, Li J, Daw ND, Schoenbaum G (2012) Surprise! Neural correlates of Pearce–Hall and Rescorla–Wagner coexist within the brain. Eur J Neurosci 35(7):1190–1200

    Article  PubMed  PubMed Central  Google Scholar 

  • Rushworth M, Behrens T (2008) Choice, uncertainty and value in prefrontal and cingulate cortex. Nat Neurosci 11(4):389–397

    Article  CAS  PubMed  Google Scholar 

  • Saltz J (2011) Natural genetic variation in social environment choice: context-dependent gene-environment correlation in Drosophila melanogaster. Evolution 65:2325–2334

    Article  PubMed  Google Scholar 

  • Schultz W, Dickinson A (2000) Neuronal coding of prediction errors. Annu Rev Neurosci 23:473–500

    Article  CAS  PubMed  Google Scholar 

  • Selonen V, Hanski IK (2010) Decision making in dispersing Siberian flying squirrels. Behav Ecol 21(2):219–225

    Article  Google Scholar 

  • Stephens D (1985) How important are partial preferences? Anim Behav 33(2):667–669

    Article  Google Scholar 

  • Todd PM, Gigerenzer G (2000) Précis of Simple heuristics that make us smart. Behav Brain Sci 23(5):727–741

    Article  CAS  PubMed  Google Scholar 

  • Valone T (2006) Are animals capable of Bayesian updating? An empirical review. Oikos 112(2):252–259

    Google Scholar 

  • Vossel S, Mathys C, Daunizeau J, Bauer M, Driver J, Friston KJ, Stephan KE (2014) Spatial attention, precision, and Bayesian inference: a study of saccadic response speed. Cereb Cortex 24(6):1436–1450

    Article  PubMed  Google Scholar 

  • White KG (2001) Forgetting functions. Anim Learn Behav 29(3):193–207

    Article  Google Scholar 

  • Wilson RC, Nassar MR, Gold JI (2010) Bayesian online learning of the hazard rate in change-point problems. Neural Comput 22(9):245–276

    Article  Google Scholar 

  • Wystrach A, Mangan M, Webb B (2015) Optimal cue integration in ants. Proc R Soc B 282(1816):20151484

    Article  PubMed  PubMed Central  Google Scholar 

  • Yang T, Shadlen MN (2007) Probabilistic reasoning by neurons. Nature 447(7148):1075–1080

    Article  CAS  PubMed  Google Scholar 

  • Yi L (2007) Applications of timing theories to a peak procedure. Behav Process 75(2):188–198

    Article  Google Scholar 

  • Yu AJ, Dayan P (2005) Uncertainty, neuromodulation, and attention. Neuron 46(4):681–692

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank Stephen Hamblin for the germ of the idea that became this paper, and for extensive discussion on the heuristics and foraging literature.

Funding

This study was funded by the National Institute of Health (Grant Number R01MH100879) and the National Science Foundation (DMS 1101060).

Author information

Authors and Affiliations

  1. The Department of Molecular and Computational Biology, The University of Southern California, 1050 Childs Way, Los Angeles, CA, 90089, USA

    Brad R. Foley

  2. Preventative Medicine, Keck School of Medicine, The University of Southern California, 2001 N. Soto Street, Los Angeles, CA, 90032, USA

    Paul Marjoram

Authors
  1. Brad R. Foley
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul Marjoram
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brad R. Foley.

Ethics declarations

Conflict of interest

Author Foley declares that he has no conflict of interest. Author Marjoram declares that he has no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (r 11 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Foley, B.R., Marjoram, P. Sure enough: efficient Bayesian learning and choice. Anim Cogn 20, 867–880 (2017). https://doi.org/10.1007/s10071-017-1107-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10071-017-1107-5

Keywords

Search

Navigation