Skip to main content

Dream to Explore: 5-HT2a as Adaptive Temperature Parameter for Sophisticated Affective Inference

  • Conference paper
  • First Online:
Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2021)

Abstract

Relative to other neuromodulators, serotonin (5-HT) has received far less attention in machine learning and active inference. We will review prior work interpreting 5-HT1a signaling as an uncertainty parameter with opponency to dopamine. We will then discuss how 5-HT2a receptors may promote more exploratory policy selection by enhancing imaginative planning (as sophisticated affective inference). Finally, we will briefly comment on how qualitatively different effects may be observed across low and high levels of 5-HT2a signaling, where the latter may help agents to change self-adversarial policies and break free of maladaptive absorbing states in POMDPs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Barnes, N.M., et al.: International union of basic and clinical pharmacology. CX. Classification of receptors for 5-hydroxytryptamine; pharmacology and function. Pharmacol. Rev. 73(1), 310–520 (2021). https://doi.org/10.1124/pr.118.015552

    Article  Google Scholar 

  2. Moutkine, I., Collins, E.L., Béchade, C., Maroteaux, L.: Evolutionary considerations on 5-HT2 receptors. Pharmacol. Res. 140, 14–20 (2019). https://doi.org/10.1016/j.phrs.2018.09.014

    Article  Google Scholar 

  3. Zhang, G., Stackman, R.W.: The role of serotonin 5-HT2A receptors in memory and cognition. Front. Pharmacol. 6, 225 (2015). https://doi.org/10.3389/fphar.2015.00225

    Article  Google Scholar 

  4. Johnson, M.W., Hendricks, P.S., Barrett, F.S., Griffiths, R.R.: Classic psychedelics: an integrative review of epidemiology, therapeutics, mystical experience, and brain network function. Pharmacol. Ther. 197, 83–102 (2019). https://doi.org/10.1016/j.pharmthera.2018.11.010

    Article  Google Scholar 

  5. Kraehenmann, R.: Dreams and psychedelics: neurophenomenological comparison and therapeutic implications. Curr. Neuropharmacol. 15(7), 1032–1042 (2017). https://doi.org/10.2174/1573413713666170619092629

    Article  Google Scholar 

  6. Carhart-Harris, R., Nutt, D.: Serotonin and brain function: a tale of two receptors. J. Psychopharmacol. Oxf. Engl. 31(9), 1091–1120 (2017). https://doi.org/10.1177/0269881117725915

    Article  Google Scholar 

  7. Shao, L.-X., et al.: Psilocybin induces rapid and persistent growth of dendritic spines in frontal cortex in vivo. Neuron 109(16), 2535–2544 (2021). https://doi.org/10.1016/j.neuron.2021.06.008

  8. Friston, K.J., FitzGerald, T., Rigoli, F., Schwartenbeck, P., Pezzulo, G.: Active inference: a process theory. Neural Comput. 29(1), 1–49 (2017). https://doi.org/10.1162/NECO_a_00912

    Article  MathSciNet  MATH  Google Scholar 

  9. Friston, K.J.: The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. 11(2), 127–138 (2010). https://doi.org/10.1038/nrn2787

    Article  Google Scholar 

  10. Friston, K.J., Rosch, R., Parr, T., Price, C., Bowman, H.: Deep temporal models and active inference. Neurosci. Biobehav. Rev. 77, 388–402 (2017). https://doi.org/10.1016/j.neubiorev.2017.04.009

    Article  Google Scholar 

  11. Carhart-Harris, R.L., Friston, K.J.: The default-mode, ego-functions and free-energy: a neurobiological account of Freudian ideas. Brain J. Neurol. 133(Pt 4), 1265–1283 (2010). https://doi.org/10.1093/brain/awq010

    Article  Google Scholar 

  12. Safron, A.: The radically embodied conscious cybernetic Bayesian brain: from free energy to free will and back again. Entropy 23(6), 783 (2021). https://doi.org/10.3390/e23060783

  13. Safron, A.: An integrated world modeling theory (IWMT) of consciousness: combining integrated information and global neuronal workspace theories with the free energy principle and active inference framework; toward solving the hard problem and characterizing agentic causation. Front. Artif. Intell. 3, 30 (2020). https://doi.org/10.3389/frai.2020.00030

  14. Friston, K., Da Costa, L., Hafner, D., Hesp, C., Parr, T.: Sophisticated Inference. https://arxiv.org/abs/2006.04120v1. Accessed 18 Jun 2020

  15. Hesp, C., Tschantz, A., Millidge, B., Ramstead, M., Friston, K., Smith, R.: Sophisticated affective inference: simulating anticipatory affective dynamics of imagining future events. In: Active Inference, pp. 179–186. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-64919-7_18

  16. Barron, H.C., Auksztulewicz, R., Friston, K.: Prediction and memory: a predictive coding account. Prog. Neurobiol. 192, 101821 (2020). https://doi.org/10.1016/j.pneurobio.2020.101821

  17. Çatal, O., Verbelen, T., Van de Maele, T., Dhoedt, B., Safron, A.: Robot navigation as hierarchical active inference. Neural Netw. 142, 192–204 (2021). https://doi.org/10.1016/j.neunet.2021.05.010

    Article  Google Scholar 

  18. Johnston, A., McBain, C.J., Fisahn, A.: 5-Hydroxytryptamine1A receptor- activation hyperpolarizes pyramidal cells and suppresses hippocampal gamma oscillations via Kir3 channel activation. J. Physiol. 592(19), 4187–4199 (2014). https://doi.org/10.1113/jphysiol.2014.279083

    Article  Google Scholar 

  19. Mannella, F., Gurney, K., Baldassarre, G.: The nucleus accumbens as a nexus between values and goals in goal-directed behavior: a review and a new hypothesis. Front. Behav. Neurosci. 7, 135 (2013). https://doi.org/10.3389/fnbeh.2013.00135

    Article  Google Scholar 

  20. FitzGerald, T.H.B., Dolan, R.J., Friston, K.J.: Dopamine, reward learning, and active inference. Front. Comput. Neurosci. 9, 136 (2015). https://doi.org/10.3389/fncom.2015.00136

  21. Friston, K.J., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., Dolan, R.J.: The anatomy of choice: dopamine and decision-making. Philos. Trans. R. Soc. B Biol. Sci. 369(1655), 20130481 (2014). https://doi.org/10.1098/rstb.2013.0481

  22. Moran, R.J., et al.: The protective action encoding of serotonin transients in the human brain. Neuropsychopharmacology 43(6), 1425−1435 (2018). https://doi.org/10.1038/npp.2017.304

  23. Grossman, C.D., Bari, B.A., Cohen, J.Y.: Serotonin neurons modulate learning rate through uncertainty. bioRxiv 103, 922 (2020). https://doi.org/10.1101/2020.10.24.353508

  24. Ohmura, Y., et al.: Disruption of model-based decision making by silencing of serotonin neurons in the dorsal raphe nucleus. Curr. Biol. 31(11), 2446–2454 (2021). https://doi.org/10.1016/j.cub.2021.03.048

    Article  Google Scholar 

  25. Boureau, Y.-L., Dayan, P.: Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacology 36(1), 74−97 (2011). https://doi.org/10.1038/npp.2010.151

  26. Daw, N.D., Kakade, S., Dayan, P.: Opponent interactions between serotonin and dopamine. Neural Netw. Off. J. Int. Neural Netw. Soc. 15(4–6), 603–616 (2002). https://doi.org/10.1016/s0893-6080(02)00052-7

  27. Doya, K., Miyazaki, K.W., Miyazaki, K.: Serotonergic modulation of cognitive computations. Curr. Opin. Behav. Sci. 38, 116–123 (2021). https://doi.org/10.1016/j.cobeha.2021.02.003

    Article  Google Scholar 

  28. Bromberg-Martin, E.S., Hikosaka, O., Nakamura, K.: Coding of task reward value in the dorsal raphe nucleus. J. Neurosci. Off. J. Soc. Neurosci. 30(18), 6262–6272 (2010). https://doi.org/10.1523/JNEUROSCI.0015-10.2010

    Article  Google Scholar 

  29. Yagishita, S.: Transient and sustained effects of dopamine and serotonin signaling in motivation-related behavior. Psychiatry Clin. Neurosci. 74(2), 91–98 (2020). https://doi.org/10.1111/pcn.12942

    Article  Google Scholar 

  30. Hayes, S.C.: A Liberated Mind: How to Pivot Toward What Matters. Penguin, London (2019)

    Google Scholar 

  31. Atasoy, S., Deco, G., Kringelbach, M.L.: Playing at the edge of criticality: expanded whole-brain repertoire of connectome-harmonics. In: Tomen, N., Herrmann, J.M., Ernst, U. (eds.) The Functional Role of Critical Dynamics in Neural Systems. SSBN, vol. 11, pp. 27–45. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20965-0_2

    Chapter  Google Scholar 

  32. Davis, A.K., Barrett, F.S., Griffiths, R.R.: Psychological flexibility mediates the relations between acute psychedelic effects and subjective decreases in depression and anxiety. J. Context. Behav. Sci. 15, 39–45 (2020). https://doi.org/10.1016/j.jcbs.2019.11.004

    Article  Google Scholar 

  33. Gerraty, R.T., Davidow, J.Y., Foerde, K., Galvan, A., Bassett, D.S., Shohamy, D.: Dynamic flexibility in striatal-cortical circuits supports reinforcement learning. J. Neurosci. 38(10), 2442–2453 (2018). https://doi.org/10.1523/JNEUROSCI.2084-17.2018

  34. Ha, D., Schmidhuber, J.: World Models (Mar 2018). ArXiv180310122 Cs Stat. https://doi.org/10.5281/zenodo.1207631

  35. Voigt, J.-P., Fink, H.: Serotonin controlling feeding and satiety. Behav. Brain Res. 277, 14–31 (2015). https://doi.org/10.1016/j.bbr.2014.08.065

    Article  Google Scholar 

  36. Hjorth, O.R., et al.: Expression and co-expression of serotonin and dopamine transporters in social anxiety disorder: a multitracer positron emission tomography study. Mol. Psychiatry 26(8), 1–10 (2019). https://doi.org/10.1038/s41380-019-0618-7

    Article  Google Scholar 

  37. Fotopoulou, A., Tsakiris, M.: Mentalizing homeostasis: the social origins of interoceptive inference–replies to commentaries. Neuropsychoanalysis 19(1), 71–76 (2017)

    Article  Google Scholar 

  38. Ciaunica, A., Constant, A., Preissl, H., Fotopoulou, A.: The first prior: from co-embodiment to co-homeostasis in early life. PsyArXiv. https://doi.org/10.31234/osf.io/twubr. Accessed 5 Jan 2021

  39. Pavlova, I.V., Rysakova, M.P.: Effects of administration of serotonin 5- HT1A receptor ligands into the amygdala on the behavior of rats with different manifestations of conditioned reflex fear. Neurosci. Behav. Physiol. 48(3), 267–278 (2018). https://doi.org/10.1007/s11055-018-0560-1

    Article  Google Scholar 

  40. Dayan, P., Huys, Q.J.M.: Serotonin in affective control. Annu. Rev. Neurosci. 32(1), 95–126 (2009). https://doi.org/10.1146/annurev.neuro.051508.135607

    Article  Google Scholar 

  41. Colyn, L., Venzala, E., Marco, S., Perez-Otaño, I., Tordera, R.M.: Chronic social defeat stress induces sustained synaptic structural changes in the prefrontal cortex and amygdala. Behav. Brain Res. 373, 112079 (2019). https://doi.org/10.1016/j.bbr.2019.112079

    Article  Google Scholar 

  42. Badcock, P.B., Davey, C.G., Whittle, S., Allen, N.B., Friston, K.J.: The depressed brain: an evolutionary systems theory. Trends Cogn. Sci. 21(3), 182–194 (2017). https://doi.org/10.1016/j.tics.2017.01.005

    Article  Google Scholar 

  43. Schultz, W.: Neuronal reward and decision signals: from theories to data. Physiol. Rev. 95(3), 853–951 (2015). https://doi.org/10.1152/physrev.00023.2014

    Article  Google Scholar 

  44. Dalley, J.W., Roiser, J.P.: Dopamine, serotonin and impulsivity. Neuroscience 215, 42–58 (2012). https://doi.org/10.1016/j.neuroscience.2012.03.065

    Article  Google Scholar 

  45. Di Pietro, N.C., Seamans, J.K.: Dopamine and serotonin interactions in the prefrontal cortex: insights on antipsychotic drugs and their mechanism of action. Pharmacopsychiatry 40(Suppl 1), S27-33 (2007). https://doi.org/10.1055/s-2007-992133

    Article  Google Scholar 

  46. Lu, H., Liu, Q.: Serotonin in the frontal cortex: a potential therapeutic target for neurological disorders. Biochem. Pharmacol. Open Access 6(1), e184 (2017). https://doi.org/10.4172/2167-0501.1000e184

    Article  Google Scholar 

  47. Dohmatob, E., Dumas, G., Bzdok, D.: Dark control: the default mode network as a reinforcement learning agent. Hum. Brain Mapp. 41(12), 3318–3341 (2020). https://doi.org/10.1002/hbm.25019

    Article  Google Scholar 

  48. Conio, B., et al.: Opposite effects of dopamine and serotonin on resting-state networks: review and implications for psychiatric disorders. Mol. Psychiatry 25(1), 82–93 (2020). https://doi.org/10.1038/s41380-019-0406-4

    Article  Google Scholar 

  49. Fransson, P., Marrelec, G.: The precuneus/posterior cingulate cortex plays a pivotal role in the default mode network: evidence from a partial correlation network analysis. Neuroimage 42(3), 1178–1184 (2008). https://doi.org/10.1016/j.neuroimage.2008.05.059

    Article  Google Scholar 

  50. Utevsky, A.V., Smith, D.V., Huettel, S.A.: Precuneus is a functional core of the default-mode network. J. Neurosci. 34(3), 932–940 (2014). https://doi.org/10.1523/JNEUROSCI.4227-13.2014

    Article  Google Scholar 

  51. Baird, B., Castelnovo, A., Gosseries, O., Tononi, G.: Frequent lucid dreaming associated with increased functional connectivity between frontopolar cortex and temporoparietal association areas. Sci. Rep. 8(1), 17798 (2018). https://doi.org/10.1038/s41598-018-36190-w

    Article  Google Scholar 

  52. Graziano, M.S.A.: The temporoparietal junction and awareness. Neurosci. Conscious. 2018(1) (2018). https://doi.org/10.1093/nc/niy005

  53. Hassabis, D., Spreng, R.N., Rusu, A.A., Robbins, C.A., Mar, R.A., Schacter, D.L.: Imagine all the people: how the brain creates and uses personality models to predict behavior. Cereb. Cortex 24(8), 1979–1987 (2014). https://doi.org/10.1093/cercor/bht042

    Article  Google Scholar 

  54. Guterstam, A., Bio, B.J., Wilterson, A.I., Graziano, M.: Temporo-parietal cortex involved in modeling one’s own and others’ attention. eLife 10, e63551 (2021). https://doi.org/10.7554/eLife.63551

  55. Davey, C.G., Harrison, B.J.: The brain’s center of gravity: how the default mode network helps us to understand the self. World Psychiatry 17(3), 278–279 (2018). https://doi.org/10.1002/wps.20553

    Article  Google Scholar 

  56. Fan, F., et al.: Development of the default-mode network during childhood and adolescence: a longitudinal resting-state fMRI study. Neuroimage 226, 117581 (2021). https://doi.org/10.1016/j.neuroimage.2020.117581

    Article  Google Scholar 

  57. Buckner, R.L., DiNicola, L.M.: The brain’s default network: updated anatomy, physiology and evolving insights. Nat. Rev. Neurosci. 20(10), 593–608 (2019). https://doi.org/10.1038/s41583-019-0212-7

    Article  Google Scholar 

  58. Hassabis, D., Maguire, E.A.: The construction system of the brain. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 364(1521), 1263–1271 (2009). https://doi.org/10.1098/rstb.2008.0296

    Article  Google Scholar 

  59. Faul, L., St. Jacques, P.L., DeRosa, J.T., Parikh, N., De Brigard, F.: Differential contribution of anterior and posterior midline regions during mental simulation of counterfactual and perspective shifts in autobiographical memories. NeuroImage 215, 116843 (2020). https://doi.org/10.1016/j.neuroimage.2020.116843

  60. Zhou, Y., Friston, K.J., Zeidman, P., Chen, J., Li, S., Razi, A.: The hierarchical organization of the default, dorsal attention and salience networks in adolescents and young adults. Cereb. Cortex NY. 28(2), 726–737 (2018). https://doi.org/10.1093/cercor/bhx307

    Article  Google Scholar 

  61. Santangelo, A.M., et al.: Insula serotonin 2A receptor binding and gene expression contribute to serotonin transporter polymorphism anxious phenotype in primates. Proc. Natl. Acad. Sci. 116(29), 14761–14768 (2019). https://doi.org/10.1073/pnas.1902087116

    Article  Google Scholar 

  62. Rueter, A.R., Abram, S.V., MacDonald, A.W., Rustichini, A., DeYoung, C.G.: The goal priority network as a neural substrate of conscientiousness. Hum. Brain Mapp. 39(9), 3574–3585 (2018). https://doi.org/10.1002/hbm.24195

    Article  Google Scholar 

  63. Barnett, L., Muthukumaraswamy, S.D., Carhart-Harris, R.L., Seth, A.K.: Decreased directed functional connectivity in the psychedelic state. Neuroimage 209, 116462 (2020). https://doi.org/10.1016/j.neuroimage.2019.116462

    Article  Google Scholar 

  64. Schartner, M.M., Carhart-Harris, R.L., Barrett, A.B., Seth, A.K., Muthukumaraswamy, S.D.: Increased spontaneous MEG signal diversity for psychoactive doses of ketamine, LSD and psilocybin. Sci. Rep. 7, 46421 (2017). https://doi.org/10.1038/srep46421

    Article  Google Scholar 

  65. Aru, J., Suzuki, M., Rutiku, R., Larkum, M.E., Bachmann, T.: Coupling the state and contents of consciousness. Front. Syst. Neurosci. 13, 43 (2019). https://doi.org/10.3389/fnsys.2019.00043

  66. Carhart-Harris, R.L.: The entropic brain - revisited. Neuropharmacology 142, 167–178 (2018). https://doi.org/10.1016/j.neuropharm.2018.03.010

    Article  Google Scholar 

  67. Carhart-Harris, R.L., et al.: The entropic brain: a theory of conscious states informed by neuroimaging research with psychedelic drugs. Front. Hum. Neurosci. 8, 20 (2014)

    Article  Google Scholar 

  68. Carhart-Harris, R.L., Friston, K.J.: REBUS and the anarchic brain: toward a unified model of the brain action of psychedelics. Pharmacol. Rev. 71(3), 316–344 (2019). https://doi.org/10.1124/pr.118.017160

    Article  Google Scholar 

  69. Luppi, A.I., et al.: Connectome harmonic decomposition of human brain dynamics reveals a landscape of consciousness. bioRxiv 199, 127 (2020). https://doi.org/10.1101/2020.08.10.244459

  70. Luppi, A.I., Carhart-Harris, R.L., Roseman, L., Pappas, I., Menon, D.K., Stamatakis, E.A.: LSD alters dynamic integration and segregation in the human brain. NeuroImage 227, 117653 (2021). https://doi.org/10.1016/j.neuroimage.2020.117653

    Article  Google Scholar 

  71. Stanley, K.O., Lehman, J.: Why Greatness Cannot Be Planned: The Myth of the Objective. Springer, Chem (2015). https://doi.org/10.1007/978-3-319-15524-1

  72. Safron, A., DeYoung, C.G.: Chapter 18 - integrating cybernetic big five theory with the free energy principle: a new strategy for modeling personalities as complex systems. In: Wood, D., Read, S.J., Harms, P.D., Slaughter, A. (eds.) Measuring and Modeling Persons and Situations, pp. 617–649. Academic Press, New York (2021). https://doi.org/10.1016/B978-0-12-819200-9.00010-7

  73. Constant, A., Hesp, C., Davey, C.G., Friston, K.J., Badcock, P.B.: Why depressed mood is adaptive: a numerical proof of principle for an evolutionary systems theory of depression. Comput. Psychiatry 5(1), 60−80 (2021). https://doi.org/10.5334/cpsy.70

  74. Erritzoe, D., Smith, J., Fisher, P.M., Carhart-Harris, R., Frokjaer, V.G., Knudsen, G.M.: Recreational use of psychedelics is associated with elevated personality trait openness: exploration of associations with brain serotonin markers. J. Psychopharmacol. Oxf. Engl. 33(9), 1068−1075 (2019). https://doi.org/10.1177/0269881119827891

  75. Girn, M., Mills, C., Roseman, L., Carhart-Harris, R.L., Christoff, K.: Updating the dynamic framework of thought: creativity and psychedelics. Neuroimage 213, 116726 (2020). https://doi.org/10.1016/j.neuroimage.2020.116726

    Article  Google Scholar 

  76. Safron, A.: Strengthened beliefs under psychedelics (SEBUS)? A commentary on ‘REBUS and the anarchic brain: toward a unified model of the brain action of psychedelics.’ PsyArXiv. https://doi.org/10.31234/osf.io/zqh4b. Accessed 30 Nov 2020

  77. Schwartenbeck, P., Passecker, J., Hauser, T.U., FitzGerald, T.H., Kronbichler, M., Friston, K.J.: Computational mechanisms of curiosity and goal-directed exploration. eLife 8, 10 (2019). https://doi.org/10.7554/eLife.41703

  78. ul Haq, R., et al.: Serotonin dependent masking of hippocampal sharp wave ripples. Neuropharmacology 101, 188–203 (2016). https://doi.org/10.1016/j.neuropharm.2015.09.026

  79. Latuske, P., Kornienko, O., Kohler, L., Allen, K.: Hippocampal remapping and its entorhinal origin. Front. Behav. Neurosci. 11, 253 (2018). https://doi.org/10.3389/fnbeh.2017.00253

    Article  Google Scholar 

  80. O’Callaghan, C., Walpola, I.C., Shine, J.M.: Neuromodulation of the mind- wandering brain state: the interaction between neuromodulatory tone, sharp wave-ripples and spontaneous thought. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 376(1817), 20190699 (2021). https://doi.org/10.1098/rstb.2019.0699

    Article  Google Scholar 

  81. Pollan, M.: How to Change Your Mind: The New Science of Psychedelics. Penguin Books Limited, London (2018)

    Google Scholar 

Download references

Acknowledgements

We gratefully acknowledge partial funding support from the Waterloo-Huawei Joint Innovation Lab within the project “the Active Inferential Meta-Learning Engine”.

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Safron, A., Sheikhbahaee, Z. (2021). Dream to Explore: 5-HT2a as Adaptive Temperature Parameter for Sophisticated Affective Inference. In: Kamp, M., et al. Machine Learning and Principles and Practice of Knowledge Discovery in Databases. ECML PKDD 2021. Communications in Computer and Information Science, vol 1524. Springer, Cham. https://doi.org/10.1007/978-3-030-93736-2_56

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-93736-2_56

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-93735-5

  • Online ISBN: 978-3-030-93736-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics