Abstract
Relative to other neuromodulators, serotonin (5-HT) has received far less attention in machine learning and active inference. We will review prior work interpreting 5-HT1a signaling as an uncertainty parameter with opponency to dopamine. We will then discuss how 5-HT2a receptors may promote more exploratory policy selection by enhancing imaginative planning (as sophisticated affective inference). Finally, we will briefly comment on how qualitatively different effects may be observed across low and high levels of 5-HT2a signaling, where the latter may help agents to change self-adversarial policies and break free of maladaptive absorbing states in POMDPs.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Barnes, N.M., et al.: International union of basic and clinical pharmacology. CX. Classification of receptors for 5-hydroxytryptamine; pharmacology and function. Pharmacol. Rev. 73(1), 310–520 (2021). https://doi.org/10.1124/pr.118.015552
Moutkine, I., Collins, E.L., Béchade, C., Maroteaux, L.: Evolutionary considerations on 5-HT2 receptors. Pharmacol. Res. 140, 14–20 (2019). https://doi.org/10.1016/j.phrs.2018.09.014
Zhang, G., Stackman, R.W.: The role of serotonin 5-HT2A receptors in memory and cognition. Front. Pharmacol. 6, 225 (2015). https://doi.org/10.3389/fphar.2015.00225
Johnson, M.W., Hendricks, P.S., Barrett, F.S., Griffiths, R.R.: Classic psychedelics: an integrative review of epidemiology, therapeutics, mystical experience, and brain network function. Pharmacol. Ther. 197, 83–102 (2019). https://doi.org/10.1016/j.pharmthera.2018.11.010
Kraehenmann, R.: Dreams and psychedelics: neurophenomenological comparison and therapeutic implications. Curr. Neuropharmacol. 15(7), 1032–1042 (2017). https://doi.org/10.2174/1573413713666170619092629
Carhart-Harris, R., Nutt, D.: Serotonin and brain function: a tale of two receptors. J. Psychopharmacol. Oxf. Engl. 31(9), 1091–1120 (2017). https://doi.org/10.1177/0269881117725915
Shao, L.-X., et al.: Psilocybin induces rapid and persistent growth of dendritic spines in frontal cortex in vivo. Neuron 109(16), 2535–2544 (2021). https://doi.org/10.1016/j.neuron.2021.06.008
Friston, K.J., FitzGerald, T., Rigoli, F., Schwartenbeck, P., Pezzulo, G.: Active inference: a process theory. Neural Comput. 29(1), 1–49 (2017). https://doi.org/10.1162/NECO_a_00912
Friston, K.J.: The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. 11(2), 127–138 (2010). https://doi.org/10.1038/nrn2787
Friston, K.J., Rosch, R., Parr, T., Price, C., Bowman, H.: Deep temporal models and active inference. Neurosci. Biobehav. Rev. 77, 388–402 (2017). https://doi.org/10.1016/j.neubiorev.2017.04.009
Carhart-Harris, R.L., Friston, K.J.: The default-mode, ego-functions and free-energy: a neurobiological account of Freudian ideas. Brain J. Neurol. 133(Pt 4), 1265–1283 (2010). https://doi.org/10.1093/brain/awq010
Safron, A.: The radically embodied conscious cybernetic Bayesian brain: from free energy to free will and back again. Entropy 23(6), 783 (2021). https://doi.org/10.3390/e23060783
Safron, A.: An integrated world modeling theory (IWMT) of consciousness: combining integrated information and global neuronal workspace theories with the free energy principle and active inference framework; toward solving the hard problem and characterizing agentic causation. Front. Artif. Intell. 3, 30 (2020). https://doi.org/10.3389/frai.2020.00030
Friston, K., Da Costa, L., Hafner, D., Hesp, C., Parr, T.: Sophisticated Inference. https://arxiv.org/abs/2006.04120v1. Accessed 18 Jun 2020
Hesp, C., Tschantz, A., Millidge, B., Ramstead, M., Friston, K., Smith, R.: Sophisticated affective inference: simulating anticipatory affective dynamics of imagining future events. In: Active Inference, pp. 179–186. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-64919-7_18
Barron, H.C., Auksztulewicz, R., Friston, K.: Prediction and memory: a predictive coding account. Prog. Neurobiol. 192, 101821 (2020). https://doi.org/10.1016/j.pneurobio.2020.101821
Çatal, O., Verbelen, T., Van de Maele, T., Dhoedt, B., Safron, A.: Robot navigation as hierarchical active inference. Neural Netw. 142, 192–204 (2021). https://doi.org/10.1016/j.neunet.2021.05.010
Johnston, A., McBain, C.J., Fisahn, A.: 5-Hydroxytryptamine1A receptor- activation hyperpolarizes pyramidal cells and suppresses hippocampal gamma oscillations via Kir3 channel activation. J. Physiol. 592(19), 4187–4199 (2014). https://doi.org/10.1113/jphysiol.2014.279083
Mannella, F., Gurney, K., Baldassarre, G.: The nucleus accumbens as a nexus between values and goals in goal-directed behavior: a review and a new hypothesis. Front. Behav. Neurosci. 7, 135 (2013). https://doi.org/10.3389/fnbeh.2013.00135
FitzGerald, T.H.B., Dolan, R.J., Friston, K.J.: Dopamine, reward learning, and active inference. Front. Comput. Neurosci. 9, 136 (2015). https://doi.org/10.3389/fncom.2015.00136
Friston, K.J., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., Dolan, R.J.: The anatomy of choice: dopamine and decision-making. Philos. Trans. R. Soc. B Biol. Sci. 369(1655), 20130481 (2014). https://doi.org/10.1098/rstb.2013.0481
Moran, R.J., et al.: The protective action encoding of serotonin transients in the human brain. Neuropsychopharmacology 43(6), 1425−1435 (2018). https://doi.org/10.1038/npp.2017.304
Grossman, C.D., Bari, B.A., Cohen, J.Y.: Serotonin neurons modulate learning rate through uncertainty. bioRxiv 103, 922 (2020). https://doi.org/10.1101/2020.10.24.353508
Ohmura, Y., et al.: Disruption of model-based decision making by silencing of serotonin neurons in the dorsal raphe nucleus. Curr. Biol. 31(11), 2446–2454 (2021). https://doi.org/10.1016/j.cub.2021.03.048
Boureau, Y.-L., Dayan, P.: Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacology 36(1), 74−97 (2011). https://doi.org/10.1038/npp.2010.151
Daw, N.D., Kakade, S., Dayan, P.: Opponent interactions between serotonin and dopamine. Neural Netw. Off. J. Int. Neural Netw. Soc. 15(4–6), 603–616 (2002). https://doi.org/10.1016/s0893-6080(02)00052-7
Doya, K., Miyazaki, K.W., Miyazaki, K.: Serotonergic modulation of cognitive computations. Curr. Opin. Behav. Sci. 38, 116–123 (2021). https://doi.org/10.1016/j.cobeha.2021.02.003
Bromberg-Martin, E.S., Hikosaka, O., Nakamura, K.: Coding of task reward value in the dorsal raphe nucleus. J. Neurosci. Off. J. Soc. Neurosci. 30(18), 6262–6272 (2010). https://doi.org/10.1523/JNEUROSCI.0015-10.2010
Yagishita, S.: Transient and sustained effects of dopamine and serotonin signaling in motivation-related behavior. Psychiatry Clin. Neurosci. 74(2), 91–98 (2020). https://doi.org/10.1111/pcn.12942
Hayes, S.C.: A Liberated Mind: How to Pivot Toward What Matters. Penguin, London (2019)
Atasoy, S., Deco, G., Kringelbach, M.L.: Playing at the edge of criticality: expanded whole-brain repertoire of connectome-harmonics. In: Tomen, N., Herrmann, J.M., Ernst, U. (eds.) The Functional Role of Critical Dynamics in Neural Systems. SSBN, vol. 11, pp. 27–45. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20965-0_2
Davis, A.K., Barrett, F.S., Griffiths, R.R.: Psychological flexibility mediates the relations between acute psychedelic effects and subjective decreases in depression and anxiety. J. Context. Behav. Sci. 15, 39–45 (2020). https://doi.org/10.1016/j.jcbs.2019.11.004
Gerraty, R.T., Davidow, J.Y., Foerde, K., Galvan, A., Bassett, D.S., Shohamy, D.: Dynamic flexibility in striatal-cortical circuits supports reinforcement learning. J. Neurosci. 38(10), 2442–2453 (2018). https://doi.org/10.1523/JNEUROSCI.2084-17.2018
Ha, D., Schmidhuber, J.: World Models (Mar 2018). ArXiv180310122 Cs Stat. https://doi.org/10.5281/zenodo.1207631
Voigt, J.-P., Fink, H.: Serotonin controlling feeding and satiety. Behav. Brain Res. 277, 14–31 (2015). https://doi.org/10.1016/j.bbr.2014.08.065
Hjorth, O.R., et al.: Expression and co-expression of serotonin and dopamine transporters in social anxiety disorder: a multitracer positron emission tomography study. Mol. Psychiatry 26(8), 1–10 (2019). https://doi.org/10.1038/s41380-019-0618-7
Fotopoulou, A., Tsakiris, M.: Mentalizing homeostasis: the social origins of interoceptive inference–replies to commentaries. Neuropsychoanalysis 19(1), 71–76 (2017)
Ciaunica, A., Constant, A., Preissl, H., Fotopoulou, A.: The first prior: from co-embodiment to co-homeostasis in early life. PsyArXiv. https://doi.org/10.31234/osf.io/twubr. Accessed 5 Jan 2021
Pavlova, I.V., Rysakova, M.P.: Effects of administration of serotonin 5- HT1A receptor ligands into the amygdala on the behavior of rats with different manifestations of conditioned reflex fear. Neurosci. Behav. Physiol. 48(3), 267–278 (2018). https://doi.org/10.1007/s11055-018-0560-1
Dayan, P., Huys, Q.J.M.: Serotonin in affective control. Annu. Rev. Neurosci. 32(1), 95–126 (2009). https://doi.org/10.1146/annurev.neuro.051508.135607
Colyn, L., Venzala, E., Marco, S., Perez-Otaño, I., Tordera, R.M.: Chronic social defeat stress induces sustained synaptic structural changes in the prefrontal cortex and amygdala. Behav. Brain Res. 373, 112079 (2019). https://doi.org/10.1016/j.bbr.2019.112079
Badcock, P.B., Davey, C.G., Whittle, S., Allen, N.B., Friston, K.J.: The depressed brain: an evolutionary systems theory. Trends Cogn. Sci. 21(3), 182–194 (2017). https://doi.org/10.1016/j.tics.2017.01.005
Schultz, W.: Neuronal reward and decision signals: from theories to data. Physiol. Rev. 95(3), 853–951 (2015). https://doi.org/10.1152/physrev.00023.2014
Dalley, J.W., Roiser, J.P.: Dopamine, serotonin and impulsivity. Neuroscience 215, 42–58 (2012). https://doi.org/10.1016/j.neuroscience.2012.03.065
Di Pietro, N.C., Seamans, J.K.: Dopamine and serotonin interactions in the prefrontal cortex: insights on antipsychotic drugs and their mechanism of action. Pharmacopsychiatry 40(Suppl 1), S27-33 (2007). https://doi.org/10.1055/s-2007-992133
Lu, H., Liu, Q.: Serotonin in the frontal cortex: a potential therapeutic target for neurological disorders. Biochem. Pharmacol. Open Access 6(1), e184 (2017). https://doi.org/10.4172/2167-0501.1000e184
Dohmatob, E., Dumas, G., Bzdok, D.: Dark control: the default mode network as a reinforcement learning agent. Hum. Brain Mapp. 41(12), 3318–3341 (2020). https://doi.org/10.1002/hbm.25019
Conio, B., et al.: Opposite effects of dopamine and serotonin on resting-state networks: review and implications for psychiatric disorders. Mol. Psychiatry 25(1), 82–93 (2020). https://doi.org/10.1038/s41380-019-0406-4
Fransson, P., Marrelec, G.: The precuneus/posterior cingulate cortex plays a pivotal role in the default mode network: evidence from a partial correlation network analysis. Neuroimage 42(3), 1178–1184 (2008). https://doi.org/10.1016/j.neuroimage.2008.05.059
Utevsky, A.V., Smith, D.V., Huettel, S.A.: Precuneus is a functional core of the default-mode network. J. Neurosci. 34(3), 932–940 (2014). https://doi.org/10.1523/JNEUROSCI.4227-13.2014
Baird, B., Castelnovo, A., Gosseries, O., Tononi, G.: Frequent lucid dreaming associated with increased functional connectivity between frontopolar cortex and temporoparietal association areas. Sci. Rep. 8(1), 17798 (2018). https://doi.org/10.1038/s41598-018-36190-w
Graziano, M.S.A.: The temporoparietal junction and awareness. Neurosci. Conscious. 2018(1) (2018). https://doi.org/10.1093/nc/niy005
Hassabis, D., Spreng, R.N., Rusu, A.A., Robbins, C.A., Mar, R.A., Schacter, D.L.: Imagine all the people: how the brain creates and uses personality models to predict behavior. Cereb. Cortex 24(8), 1979–1987 (2014). https://doi.org/10.1093/cercor/bht042
Guterstam, A., Bio, B.J., Wilterson, A.I., Graziano, M.: Temporo-parietal cortex involved in modeling one’s own and others’ attention. eLife 10, e63551 (2021). https://doi.org/10.7554/eLife.63551
Davey, C.G., Harrison, B.J.: The brain’s center of gravity: how the default mode network helps us to understand the self. World Psychiatry 17(3), 278–279 (2018). https://doi.org/10.1002/wps.20553
Fan, F., et al.: Development of the default-mode network during childhood and adolescence: a longitudinal resting-state fMRI study. Neuroimage 226, 117581 (2021). https://doi.org/10.1016/j.neuroimage.2020.117581
Buckner, R.L., DiNicola, L.M.: The brain’s default network: updated anatomy, physiology and evolving insights. Nat. Rev. Neurosci. 20(10), 593–608 (2019). https://doi.org/10.1038/s41583-019-0212-7
Hassabis, D., Maguire, E.A.: The construction system of the brain. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 364(1521), 1263–1271 (2009). https://doi.org/10.1098/rstb.2008.0296
Faul, L., St. Jacques, P.L., DeRosa, J.T., Parikh, N., De Brigard, F.: Differential contribution of anterior and posterior midline regions during mental simulation of counterfactual and perspective shifts in autobiographical memories. NeuroImage 215, 116843 (2020). https://doi.org/10.1016/j.neuroimage.2020.116843
Zhou, Y., Friston, K.J., Zeidman, P., Chen, J., Li, S., Razi, A.: The hierarchical organization of the default, dorsal attention and salience networks in adolescents and young adults. Cereb. Cortex NY. 28(2), 726–737 (2018). https://doi.org/10.1093/cercor/bhx307
Santangelo, A.M., et al.: Insula serotonin 2A receptor binding and gene expression contribute to serotonin transporter polymorphism anxious phenotype in primates. Proc. Natl. Acad. Sci. 116(29), 14761–14768 (2019). https://doi.org/10.1073/pnas.1902087116
Rueter, A.R., Abram, S.V., MacDonald, A.W., Rustichini, A., DeYoung, C.G.: The goal priority network as a neural substrate of conscientiousness. Hum. Brain Mapp. 39(9), 3574–3585 (2018). https://doi.org/10.1002/hbm.24195
Barnett, L., Muthukumaraswamy, S.D., Carhart-Harris, R.L., Seth, A.K.: Decreased directed functional connectivity in the psychedelic state. Neuroimage 209, 116462 (2020). https://doi.org/10.1016/j.neuroimage.2019.116462
Schartner, M.M., Carhart-Harris, R.L., Barrett, A.B., Seth, A.K., Muthukumaraswamy, S.D.: Increased spontaneous MEG signal diversity for psychoactive doses of ketamine, LSD and psilocybin. Sci. Rep. 7, 46421 (2017). https://doi.org/10.1038/srep46421
Aru, J., Suzuki, M., Rutiku, R., Larkum, M.E., Bachmann, T.: Coupling the state and contents of consciousness. Front. Syst. Neurosci. 13, 43 (2019). https://doi.org/10.3389/fnsys.2019.00043
Carhart-Harris, R.L.: The entropic brain - revisited. Neuropharmacology 142, 167–178 (2018). https://doi.org/10.1016/j.neuropharm.2018.03.010
Carhart-Harris, R.L., et al.: The entropic brain: a theory of conscious states informed by neuroimaging research with psychedelic drugs. Front. Hum. Neurosci. 8, 20 (2014)
Carhart-Harris, R.L., Friston, K.J.: REBUS and the anarchic brain: toward a unified model of the brain action of psychedelics. Pharmacol. Rev. 71(3), 316–344 (2019). https://doi.org/10.1124/pr.118.017160
Luppi, A.I., et al.: Connectome harmonic decomposition of human brain dynamics reveals a landscape of consciousness. bioRxiv 199, 127 (2020). https://doi.org/10.1101/2020.08.10.244459
Luppi, A.I., Carhart-Harris, R.L., Roseman, L., Pappas, I., Menon, D.K., Stamatakis, E.A.: LSD alters dynamic integration and segregation in the human brain. NeuroImage 227, 117653 (2021). https://doi.org/10.1016/j.neuroimage.2020.117653
Stanley, K.O., Lehman, J.: Why Greatness Cannot Be Planned: The Myth of the Objective. Springer, Chem (2015). https://doi.org/10.1007/978-3-319-15524-1
Safron, A., DeYoung, C.G.: Chapter 18 - integrating cybernetic big five theory with the free energy principle: a new strategy for modeling personalities as complex systems. In: Wood, D., Read, S.J., Harms, P.D., Slaughter, A. (eds.) Measuring and Modeling Persons and Situations, pp. 617–649. Academic Press, New York (2021). https://doi.org/10.1016/B978-0-12-819200-9.00010-7
Constant, A., Hesp, C., Davey, C.G., Friston, K.J., Badcock, P.B.: Why depressed mood is adaptive: a numerical proof of principle for an evolutionary systems theory of depression. Comput. Psychiatry 5(1), 60−80 (2021). https://doi.org/10.5334/cpsy.70
Erritzoe, D., Smith, J., Fisher, P.M., Carhart-Harris, R., Frokjaer, V.G., Knudsen, G.M.: Recreational use of psychedelics is associated with elevated personality trait openness: exploration of associations with brain serotonin markers. J. Psychopharmacol. Oxf. Engl. 33(9), 1068−1075 (2019). https://doi.org/10.1177/0269881119827891
Girn, M., Mills, C., Roseman, L., Carhart-Harris, R.L., Christoff, K.: Updating the dynamic framework of thought: creativity and psychedelics. Neuroimage 213, 116726 (2020). https://doi.org/10.1016/j.neuroimage.2020.116726
Safron, A.: Strengthened beliefs under psychedelics (SEBUS)? A commentary on ‘REBUS and the anarchic brain: toward a unified model of the brain action of psychedelics.’ PsyArXiv. https://doi.org/10.31234/osf.io/zqh4b. Accessed 30 Nov 2020
Schwartenbeck, P., Passecker, J., Hauser, T.U., FitzGerald, T.H., Kronbichler, M., Friston, K.J.: Computational mechanisms of curiosity and goal-directed exploration. eLife 8, 10 (2019). https://doi.org/10.7554/eLife.41703
ul Haq, R., et al.: Serotonin dependent masking of hippocampal sharp wave ripples. Neuropharmacology 101, 188–203 (2016). https://doi.org/10.1016/j.neuropharm.2015.09.026
Latuske, P., Kornienko, O., Kohler, L., Allen, K.: Hippocampal remapping and its entorhinal origin. Front. Behav. Neurosci. 11, 253 (2018). https://doi.org/10.3389/fnbeh.2017.00253
O’Callaghan, C., Walpola, I.C., Shine, J.M.: Neuromodulation of the mind- wandering brain state: the interaction between neuromodulatory tone, sharp wave-ripples and spontaneous thought. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 376(1817), 20190699 (2021). https://doi.org/10.1098/rstb.2019.0699
Pollan, M.: How to Change Your Mind: The New Science of Psychedelics. Penguin Books Limited, London (2018)
Acknowledgements
We gratefully acknowledge partial funding support from the Waterloo-Huawei Joint Innovation Lab within the project “the Active Inferential Meta-Learning Engine”.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Safron, A., Sheikhbahaee, Z. (2021). Dream to Explore: 5-HT2a as Adaptive Temperature Parameter for Sophisticated Affective Inference. In: Kamp, M., et al. Machine Learning and Principles and Practice of Knowledge Discovery in Databases. ECML PKDD 2021. Communications in Computer and Information Science, vol 1524. Springer, Cham. https://doi.org/10.1007/978-3-030-93736-2_56
Download citation
DOI: https://doi.org/10.1007/978-3-030-93736-2_56
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-93735-5
Online ISBN: 978-3-030-93736-2
eBook Packages: Computer ScienceComputer Science (R0)