Abstract
In the framework of modern ideas, decision making and, hence, the choice of the optimal behavior, take place on the level of specific neuron networks. Those networks accumulate information on the possible behavioral alternatives, estimate all pro et contra and make the optimal decision. In formal network models, the standard one, describing the dynamics of that process, is the attractor dynamics, where the switching from one decision to another one is the transfer of the system from one attractor to another. That attractor model is formal, is not practically associated with the neurophysiology and is based on the numerical solving some ad hoc equations. More prominent place in the hierarchy of models is hold by analytic models, one of them is considered in the present work.
In that model, the dynamical process of decision making is likened (as in the attractor model) to moving the system over the energy landscape in the presence of strong enough noises. This process could be readily presented by the ball movement over the rough hummocky surface (in such a simulator, more or less intensive surface shaking could play the role of the noise). Different actual decisions correspond to landscape wells of different depths being separated by energy barriers from each other. In the process of decision making, the ball overcomes those barriers, due to noises, and moves from one well to another one. If the probability of the ball transfer to one of the wells is superior to probabilities of other transfers, that process results in the unique decision; conversely, different decisions could be made with comparable probabilities.
As against to attractor models with the explicit consideration of noise effects, we use the known Arrhenius-Kramers formula which associates the mean system life-time in the certain quasi-stationary state with the height of the energy barrier and the average noise energy. Such an approach allows to do without using artificial and non-physiologic equations of the system motion and to obtain the simple analytic description of the decision making process. That model is applied to analyze one of standard value-oriented games, where participants gather remunerations of different values from two alternative sources. Within the framework of the considered model, it is possible to describe analytically the process of moving from one decision to another and other details of the process. That complements our knowledge about the decision making.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Below, by the engram we mean a limited section of the neural network that stores important information for the solution or a prototype of the solution. It is believed that the functional role of engram neurons in the process of the task solution is to give a signal about how useful a particular solution is, but it is not known, how they do it.
- 2.
By a quasi-stationary state we mean a state that is not stationary, but evolves slowly. In our case, this requires \(\tau \gg \tau _0\), that is \(\varDelta /\langle \varPhi \rangle \gtrsim 1\) (see (1)).
- 3.
Due to the hysteresis behavior of the system considered (see below), parameters \(D_{LL}\), corresponding to these switches, differ slightly from each other.
- 4.
References
Armbruster, D.J.N., Ueltzhoffer, K., Basten, U., Fiebach, C.J.: Prefrontal cortical mechanisms underlying individual differences in cognitive flexibility and stability. J. Cogn. Neurosci. 24(12), 2385–2399 (2012). https://doi.org/10.1162/jocn_a_00286
Bogacz, R.: Optimal decision-making theories: linking neurobiology with behaviour. Trends Cogn. Sci. 11(3), 118–125 (2007)
Loewenstein, G., Elster, J.: Choice over Time. Russell Sage Foundation, New York (1992)
Dreisbach, G., Goschke, T.: How positive affect modulates cognitive control: reduced perseveration at the cost of increased distractibility. J. Exp. Psychol. Learn. Memory Cogn. 30(2), 343–353 (2004). https://doi.org/10.1037/0278-7393.30.2.343
Dreisbach, G., Haider, H.: Thats what task sets are for: shielding against irrelevant information. Psychol. Res. 72(4), 355–361 (2008). https://doi.org/10.1007/s00426-007-0131-5
Durstewitz, D., Seamans, J.K.: The dual-state theory of prefrontal cortex dopamine function with relevance to Catechol-O-Methyltransferase genotypes and Schizophrenia. Biol. Psychiatry 64(9), 739–749 (2008). https://doi.org/10.1016/j.biopsych.2008.05.015
Fischer, R., Hommel, B.: Deep thinking increases task-set shielding and reduces shifting flexibility in dual-task performance. Cognition 123(2), 303–307 (2012). https://doi.org/10.1016/j.cognition.2011.11.015
Glimcher, P.W.: The neurobiology of visual-saccadic decision making. Annu. Rev. Neurosci. 26, 133–179 (2003)
Glimcher, P.W., Rustichini, A.: Neuroeconomics: the consilience of brain and decision. Science 306, 447–452 (2004)
Goschke, T.: Intentional reconfiguration and involuntary persistence in task set switching. In: Monsell, S., Driver, J. (eds.) Control of Cognitive Processes: Attention and Performance XVIII, pp. 331–355. MIT Press Ltd., Cambridge (2000)
Goschke, T.: Voluntary action and cognitive control from a cognitive neuroscience perspective. In: Maasen, S., Prinz, W., Roth, G. (eds.) Voluntary Action: Brains, Minds, and Sociality, pp. 49–85. Oxford University Press, New York (2003)
Goschke, T.: Volition in action: intentions, control dilemmas and the dynamic regulation of intentional control. In: Action Science: Foundations of an Emerging Discipline, pp. 409–434. MIT Press, Cambridge (2013)
Hommel, B., Colzato, L.S.: The social transmission of metacontrol policies: mechanisms underlying the interpersonal transfer of persistence and flexibility. Neurosci. Biobehav. Rev. 81, 43–58 (2017). https://doi.org/10.1016/j.neubiorev.2017.01.009
Kalenscher, T., Pennartz, C.M.A.: Is a bird in the hand worth two in the future? The neuroeconomics of intertemporal decision-making. Prog. Neurobiol. 84, 284–315 (2008)
Knutson, B., Taylor, J., Kaufman, M., Peterson, R., Glover, G.: Distributed neural representation of expected value. J. Neurosci. 25(19), 4806–4812 (2005)
Kuhnen, C.M., Knutson, B.: The neural basis of financial risk taking. Neuron 47(5), 763–770 (2005)
Laing, C.R., Chow, C.C.: A spiking neuron model for binocular rivalry. J. Comp. Neurosci. 12, 39–53 (2002)
Logan, G.D.: Executive control of thought and action: in search of the wild homunculus. Curr. Dir. Psychol. Sci. 12(2), 45–48 (2003)
Meilikhov, E.Z., Farzetdinova, R.M.: Bistable perception of ambiguous images: simple Arrhenius model. Cogn. Neurodyn. 13(6), 613–621 (2019). https://doi.org/10.1007/s11571-019-09554-9
Meiran, N.: Task switching and cognitive self control. In: Hassin, R., Ochsner, K.N., Trope, Y. (eds.) Self Control in Society, Mind and Brain, pp. 202–220. Oxford University Press, New York (2010). https://doi.org/10.1093/acprof:oso/9780195391381.001.0001
Miller, E.K., Cohen, J.D.: An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24(1), 167–202 (2001)
Monsell, S.: Task switching. Trends Cogn. Sci. 7(3), 134–140 (2003). https://doi.org/10.1016/S1364-6613(03)00028-7
Moreno-Bote, R., Rinzel, J., Rubin, N.: Noise-induced alternations in an attractor network model of perceptual bistability. J. Neurophys. 98, 1125–1139 (2007)
Lago-Fernández, L.F., Deco, G.: A model of binocular rivalry based on competition in IT. Neurocomputing 44, 503–507 (2002)
McClure, S.M., Laibson, D.I., Loewenstein, G., Cohen, J.D.: Separate neural systems value immediate and delayed monetary rewards. Science 306, 503–507 (2004)
Poo, M., et al.: What is memory? The present state of the engram. BMC Biol. 14, 1 (2016). https://doi.org/10.1186/s12915-016-0261-6
Pignatelli, M., et al.: Engram cell excitability state determines the efficacy of memory retrieval. Neuron 101, 1–11 (2019)
Ratcliff, R., Tharap, A., Mckoon, G.: A diffusion model analysis of the effects of aging on brightness discrimination. Percept. Psychophys. 65(4), 523–535 (2003)
Rilling, J.K., King-Casas, B., Sanfey, A.G.: The neurobiology of social decision-making. Curr. Opin. Neurobiol. 18, 159–165 (2008)
Scherbaum, S., Frisch, S., Leiberg, S., Lade, S.J., Goschke, T., Dshemuchadse, M.: Process dynamics in delay discounting decisions: an attractor dynamics approach. Judgm. Decis. Mak. 11(5), 472–495 (2016)
Scherbaum, S., Haber, P., Morley, K., Underhill, D., Moustafa, A.A.: Biased and less sensitive: a gamified approach to delay discounting in heroin addiction. J. Clin. Exp. Neuropsychol. 40(2), 139–150 (2018). https://doi.org/10.1080/13803395.2017.1324022
Senftleben, U., Schoemann, M., Schwenke, D., Richter, S., Dshemuchadse, M., Scherbaum, S.: Choice perseveration in value-based decision making: the impact of intertrial interval and mood. Acta Psychol. 198, 102876 (2019a)
Senftleben, U., Schoemann, M., Scherbaum, S.: To stay or not to stay: the stability-flexibility trade-off in value-based decision making (2019b). https://doi.org/10.31234/osf.io/d95a6
Ueltzhoffer, K., Armbruster, D.J.N., Fiebach, C.J.: Stochastic dynamics underlying cognitive stability and flexibility. PLoS Comput. Biol. 11(6), 1–46 (2015). https://doi.org/10.1371/journal.pcbi.1004331
Wilson, H.R., Blake, R., Lee, S.H.: Dynamics of travelling waves in visual perception. Nature 412(6850), 907–910 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Meilikhov, E., Farzetdinova, R. (2022). Value-Based Decision Making – Simple Analytic Theory. In: Kryzhanovsky, B., Dunin-Barkowski, W., Redko, V., Tiumentsev, Y., Klimov, V.V. (eds) Advances in Neural Computation, Machine Learning, and Cognitive Research V. NEUROINFORMATICS 2021. Studies in Computational Intelligence, vol 1008. Springer, Cham. https://doi.org/10.1007/978-3-030-91581-0_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-91581-0_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-91580-3
Online ISBN: 978-3-030-91581-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)