Skip to main content
SpringerLink
Log in

Transcranial Direct Current Stimulation (tDCS) of the Anterior Prefrontal Cortex (aPFC) Modulates Reinforcement Learning and Decision-Making Under Uncertainty: a Double-Blind Crossover Study

  • Original Article
  • Published:
Journal of Cognitive Enhancement Aims and scope Submit manuscript

Abstract

Reinforcement learning refers to the ability to acquire information from the outcomes of prior choices (i.e. positive and negative) in order to make predictions on the effect of future decision and adapt the behaviour basing on past experiences. The anterior prefrontal cortex (aPFC) is considered to play a key role in the representation of event value, reinforcement learning and decision-making. However, a causal evidence of the involvement of this area in these processes has not been provided yet. The aim of the study was to test the role of the orbitofrontal cortex in feedback processing, reinforcement learning and decision-making under uncertainly. Eighteen healthy individuals underwent three sessions of tDCS over the prefrontal pole (anodal, cathodal, sham) during a probabilistic learning (PL) task. In the PL task, participants were invited to learn the covert probabilistic stimulus-outcome association from positive and negative feedbacks in order to choose the best option. Afterwards, a probabilistic selection (PS) task was delivered to assess decisions based on the stimulus-reward associations acquired in the PL task. During cathodal tDCS, accuracy in the PL task was reduced and participants were less prone to maintain their choice after positive feedback or to change it after a negative one (i.e., win-stay and lose-shift behavior). In addition, anodal tDCS affected the subsequent PS task by reducing the ability to choose the best alternative during hard probabilistic decisions. In conclusion, the present study suggests a causal role of aPFC in feedback trial-by-trial behavioral adaptation and decision-making under uncertainty.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Abbreviations

aPFC:

Anterior prefrontal cortex

vmPFC/mOFC:

Ventromedial prefrontal cortex/medial orbitofrontal cortex

tDCS:

Transcranial direct current stimulation

References

  • Bechara, A., Damasio, A. R., Damasio, H., & Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition, 50, 7–15.

    Article  PubMed  Google Scholar 

  • Bechara, A., Damasio, H., & Damasio, A. R. (2000). Emotion, decision making and the orbitofrontal cortex. Cerebral Cortex, 10, 295–307.

    Article  PubMed  Google Scholar 

  • Boggio, P. S., Sultani, N., Fecteau, S., Merabet, L., Mecca, T., Pascual-Leone, A., Basaglia, A., & Fregni, F. (2008). Prefrontal cortex modulation using transcranial DC stimulation reduces alcohol craving: a double-blind, sham-controlled study. Drug and Alcohol Dependence, 92, 55–60.

    Article  PubMed  Google Scholar 

  • Camille, N., Tsuchida, A., & Fellows, L. K. (2011). Double dissociation of stimulus-value and action-value learning in humans with orbitofrontal or anterior cingulate cortex damage. The Journal of Neuroscience, 31, 15048–15052.

    Article  PubMed  Google Scholar 

  • Chib, V. S., Yun, K., Takahashi, H., & Shimojo, S. (2013). Noninvasive remote activation of the ventral midbrain by transcranial direct current stimulation of prefrontal cortex. Trans Psychiatry, 3, e268.

    Article  Google Scholar 

  • Elliott, R., Newman, J. L., Longe, O. A., & Deakin, J. W. (2003). Differential response patterns in the striatum and orbitofrontal cortex to financial reward in humans: a parametric functional magnetic resonance imaging study. The Journal of Neuroscience, 23, 303–307.

    PubMed  Google Scholar 

  • Elliott, R., Agnew, Z., & Deakin, J. F. (2010). Hedonic and informational functions of the human orbitofrontal cortex. Cerebral Cortex, 20, 198–204.

    Article  PubMed  Google Scholar 

  • Evenden, J. L., & Robbins, T. W. (1983). Dissociable effects of d-amphetamine, chlordiazepoxide and alpha-flupenthixol on choice and rate measures of reinforcement in the rat. Psychopharmacology, 79, 180–186.

    Article  PubMed  Google Scholar 

  • Fecteau, S., Fregni, F., Boggio, P. S., Camprodon, J. A., & Pascual-Leone, A. (2010). Neuromodulation of decision-making in the addictive brain. Substance Use & Misuse, 45, 1766–1786.

    Article  Google Scholar 

  • Fellows, L. K., & Farah, M. J. (2003). Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. Brain, 126, 1830–1837.

    Article  PubMed  Google Scholar 

  • Fellows, L. K., & Farah, M. J. (2007). The role of ventromedial prefrontal cortex in decision making: judgment under uncertainty or judgment per se?. Cerebral Cortex, 17(11), 2669–2674.

  • Ferrucci, R., Bortolomasi, M., Fumagalli, M., Giacopuzzi, M., Mameli, F., Priori, A., Scelzo, E., & Vergari, M. (2013). Transcranial direct current stimulation (tDCS): a treatment for depressive illness. Int Journ Psychiatry Clinic Practic, 15, 22–23.

    Google Scholar 

  • Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and non-medicated parkinsonism. Journal of Cognitive Neuroscience, 17, 51–72.

    Article  PubMed  Google Scholar 

  • Frank, M. J., & Claus, E. D. (2006). Anatomy of decision: striato-orbitofrontal interactions in reinforcement learning, decision making and reversal. Psychological Review, 113, 300–326.

    Article  PubMed  Google Scholar 

  • Frank, M. J., & Kong, L. (2008). Learning to avoid in older age. Psychology and Aging, 23, 392–398.

    Article  PubMed  Google Scholar 

  • Frank, M. J., Seeberger, L. C., & O’Reilly, R. C. (2004). By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science, 306, 1940–1943.

    Article  PubMed  Google Scholar 

  • Fregni, F., Orsati, F., Pedros, W., Fectea, S., Tome, F. A., Nitsch, M. A., Mecca, T., Macedo, E., Pascual-Leon, A., & Boggio, P. S. (2008). Transcranial direct current stimulation of the prefrontal cortex modulates the desire for specific foods. Appetite, 51, 34–41.

    Article  PubMed  Google Scholar 

  • Fumagalli, M., Vergari, M., Pasqualetti, P., Marceglia, S., Mameli, F., Ferrucci, R., Mrakic-Sposta, S., Zago, S., Sartori, G., Pravettoni, G., Barbieri, S., Cappa, S., & Priori, A. (2010). Brain switches utilitarian behavior: does gender make the difference? PloS One, 5(1), e8865.

    Article  PubMed  PubMed Central  Google Scholar 

  • Gozli, D. G., Moskowitz, J. B., & Pratt, J. (2014). Visual attention to features by associative learning. Cognition, 133, 488–501.

    Article  PubMed  Google Scholar 

  • Jacobson, L., Koslowsky, M., & Lavidor, M. (2012). tDCS polarity effects in motor and cognitive domains: a meta-analytical review. Experimental Brain Research, 216, 1–10.

    Article  PubMed  Google Scholar 

  • Jasper, H. H. (1958). The ten–twenty electrode system of the international federation. Electroencephalography and Clinical Neurophysiology, 10, 371–375.

    Google Scholar 

  • Karim, A. A., Schneider, M., Lotze, M., Veit, R., Sauseng, P., Braun, C., & Braibaumer, N. (2010). The truth about lying: inhibition of the anterior refrontal cortex improves deceptive behavior. Cerebral Cortex, 20, 205–213.

    Article  PubMed  Google Scholar 

  • Knutson, B., & Bossaerts, P. (2007). Neural antecedents of financial decisions. The Journal of Neuroscience, 27, 8174–8177.

    Article  PubMed  Google Scholar 

  • Kringelbach, M. L., & Rolls, E. T. (2003). Neural correlates of rapid context-dependent reversal learning in a simple model of human social interaction. NeuroImage, 20, 1371–1383.

    Article  PubMed  Google Scholar 

  • Kringelbach, M. L., & Rolls, E. T. (2004). The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology. Progress in Neurobiology, 72, 341–372.

    Article  PubMed  Google Scholar 

  • Kringelbach, M. L., O’Doherty, J., Rolls, E. T., & Andrews, C. (2003). Activation of the human orbitofrontal cortex to a liquid food stimulus is correlated with its subjective pleasantness. Cerebral Cortex, 13, 1064–1071.

    Article  PubMed  Google Scholar 

  • Kringelback, M. L. (2005). The human orbitofrontal cortex: linking reward to hedonic experience. Nature Reviews. Neuroscience, 6, 691–702.

    Article  Google Scholar 

  • Levasseur-Moreau, J., & Fecteau, S. (2012). Translational application of neuromodulation of decision-making. Brain Stimulation, 5, 77–83.

    Article  PubMed  Google Scholar 

  • Liu, X., Powell, D. K., Wang, H., Gold, B. T., Corbly, C. R., & Joseph, J. E. (2007). Functional dissociation in frontal and striatal areas for processing of positive and negative reward information. The Journal of Neuroscience, 27, 4587–4597.

    Article  PubMed  Google Scholar 

  • Loo, C. K., Alonzo, A., Martin, D., Mitchell, P. B., Galvez, V., & Sachdev, P. (2012). Transcranial direct current stimulation for depression: 3-week, randomised, sham-controlled trial. British J Psychiatry, 20, 52–59.

    Article  Google Scholar 

  • Ly, V., Bergmann, T. O., Gladwi, T. E., Volman, I., Usberti, N., Cools, R., & Roelofs, K. (2016). Reduced affective biasing of instrumental action with tDCS over the prefrontal cortex. Brain Stimulation, 9, 380–387.

    Article  PubMed  PubMed Central  Google Scholar 

  • Manuel, A. L., David, A. W., Bikson, M., & Schinder, A. (2014). Frontal tDCS modulates orbitofrontal reality filtering. Neuroscience, 265, 21–27.

    Article  PubMed  Google Scholar 

  • Marin, M. F., Camprodon, J. A., Dougherty, D. D., & Mohammed, R. M. (2014). Device-based brain stimulation to augment fear extinction: implications for PTSD treatment and beyond. Depression and Anxiety, 31, 269–278.

    Article  PubMed  Google Scholar 

  • Nitsche, M. A., & Paulus, W. (2001). Sustained excitability elevations induced by transcranial DC motor cortex stimulation in humans. Neurology, 57, 1899–1901.

    Article  PubMed  Google Scholar 

  • Nitsche, M. A., Liebetanz, D., Anta, A., Lang, N., Tergau, F., & Paulus, W. (2003). Modulation of cortical excitability by weak direct current stimulation—technical, safety and functional aspects. Supplements to Clinical Neurophysiology, 56, 255–276.

    Article  PubMed  Google Scholar 

  • Noonan, M. P., Walton, M. E., Behrens, T. E., Sallet, J., Buckley, M. J., & Rushworth, M. F. (2010). Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. Proceedings of the National Academy of Sciences of the United States of America, 107, 20547–20552.

    Article  PubMed  PubMed Central  Google Scholar 

  • Noonan, M. P., Kolling, N., Walton, M. E., & Rushworth, M. F. (2012). Re-evaluating the role of the orbitofrontal cortex in reward and reinforcement. The European Journal of Neuroscience, 35, 997–1010.

    Article  PubMed  Google Scholar 

  • O’Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J., & Andrews, C. (2001). Abstract reward and punishment representations in the human orbitofrontal cortex. Nature Neuroscience, 4, 95–102.

    Article  PubMed  Google Scholar 

  • Ott, D. V., Ullsperger, M., Jocham, G., Neumann, J., & Klein, T. A. (2011). Continuous theta-burst stimulation (cTBS) over the lateral prefrontal cortex alters reinforcement learning bias. NeuroImage, 57, 617–623.

    Article  PubMed  Google Scholar 

  • Reato, D., Gasca, D., Datta, A., Bikson, M., Marshall, L., & Parra, L. C. (2013). Transcranial electrical stimulation accelerates human sleep homeostasis. PLoS Computational Biology, 9(2), e1002898. doi:10.1371/journal.pcbi.1002898.

    Article  PubMed  PubMed Central  Google Scholar 

  • Rolls, E. T., & Grabenhorst, F. (2008). The orbitofrontal cortex and beyond: from affect to decision-making. Progress in Neurobiology, 86, 216–244.

    Article  PubMed  Google Scholar 

  • Rolls, E. T., Hornak, J., Wade, D., & McGrath, J. (1994). Emotion-related learning in patients with social and emotional changes associated with frontal lobe damage. Journal of Neurology, Neurosurgery, and Psychiatry, 57, 1518–1524.

    Article  PubMed  PubMed Central  Google Scholar 

  • Rolls, E. T., Grabenhorst, F., & Deco, G. (2010). Choice, difficulty, and confidence in the brain. NeuroImage, 53, 694–706.

    Article  PubMed  Google Scholar 

  • Rothermund, K., Wentura, D., & De Houwer, J. (2015). Retrieval of incidental stimulus-response associations as a source of negative priming. Journal of Experimental Psychology. Learning, Memory, and Cognition, 3, 482–495.

    Google Scholar 

  • Rushworth, M. F., Noonan, M. P., Boorman, E. D., Walton, M. E., & Behrens, T. E. (2011). Frontal cortex and reward-guided learning and decision-making. Neuron, 70, 1054–1069.

    Article  PubMed  Google Scholar 

  • Seymour, B., Daw, N., Dayan, P., Singer, T., & Dolan, R. (2007). Differential encoding of losses and gains in the human striatum. The Journal of Neuroscience, 27, 4826–4831.

    Article  PubMed  PubMed Central  Google Scholar 

  • Small, D. M., Zatorre, R. J., Dagher, A., Evans, A. C., & Jones-Gotman, M. (2001). Changes in brain activity related to eating chocolate: from pleasure to aversion. Brain, 124, 1720–1733.

    Article  PubMed  Google Scholar 

  • Tsuchida, A., Doll, B. D., & Fellows, K. F. (2010). Beyond reversal: a critical role for human orbitofrontal cortex in flexible learning from probabilistic feedback. The Journal of Neuroscience, 30, 16868–16875.

    Article  PubMed  Google Scholar 

  • Turi, Z., Mittner, M., Opitz, A., Popkes, M., Paulus, W., & Antal, A. (2015). Transcranial direct current stimulation over the left prefrontal cortex increases randomness of choice in instrumental learning. Cortex, 63, 145–154.

    Article  PubMed  Google Scholar 

  • Volpato, C., Piccione, F., Cavinato, M., Duzzi, D., Schiff, S., Foscolo, L., & Venneri, A. (2013). Modulation of affective symptoms and resting state activity by brain stimulation in a treatment-resistant case of obsessive-compulsive disorder. Neurocase, 19, 360–370.

    Article  PubMed  Google Scholar 

  • Volpato, C., Schiff, S., Facchini, S., Silvoni, S., Cavinato, M., Piccione, F., Antonini, A., & Birbaumer, N. (2016). Dopaminergic medication modulates learning from feedback and error-related negativity in Parkinson’s disease: a pilot study. Frontiers in Behavioral Neuroscience. doi:10.3389/fnbeh.2016.00205.

  • Wallis, J. D. (2007). Orbitofrontal cortex and its contribution to decision-making. Annual Review of Neuroscience (Palo Alto, CA), 30, 31–56.

    Article  Google Scholar 

  • Walton, M. E., Behrens, T. E., Noonan, M. P., & Rushworth, M. F. (2011). Giving credit where credit is due: orbitofrontal cortex and valuation in an uncertain world. Annals of the New York Academy of Sciences, 1239, 14–24.

    Article  PubMed  Google Scholar 

  • Wheeler, E., & Fellows, L. K. (2008). The human ventromedial frontal lobe is critical for learning from negative feedback. Brain, 131, 1323–1331.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors are grateful to Esmeralda Balistreri, Gloria Imperato, and Daniele Aprile for participants’ recruitment and data collection.

Author information

Authors and Affiliations

  1. Non-invasive Brain Stimulation Unit, Department of Behavioural and Clinical Neurology, Santa Lucia Foundation IRCCS, Rome, Italy

    Elias P. Casula

  2. Department of General Psychology, University of Padova, Padova, Italy

    Elias P. Casula & Patrizia S. Bisiacchi

  3. Department of Human and Social Sciences, University of Bergamo, Bergamo, Italy

    Giulia Testa

  4. Centro Interdipartimentale di Ricerca sulla Modellistica delle Alterazioni Neuropsichiche in Medicina Clinica (CIRMANMEC), University of Padova, Padova, Italy

    Patrizia S. Bisiacchi, Sara Montagnese, Lorenza Caregaro, Piero Amodio & Sami Schiff

  5. Department of Medicine, University of Padova, Padova, Italy

    Sara Montagnese, Piero Amodio & Sami Schiff

Authors
  1. Elias P. Casula
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giulia Testa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patrizia S. Bisiacchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sara Montagnese
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lorenza Caregaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Piero Amodio
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sami Schiff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sami Schiff.

Ethics declarations

All the participants took part in the experiment voluntarily, after providing written informed consent. The University Hospital of Padua Ethical Committee approved the experimental procedure.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Casula, E.P., Testa, G., Bisiacchi, P.S. et al. Transcranial Direct Current Stimulation (tDCS) of the Anterior Prefrontal Cortex (aPFC) Modulates Reinforcement Learning and Decision-Making Under Uncertainty: a Double-Blind Crossover Study. J Cogn Enhanc 1, 318–326 (2017). https://doi.org/10.1007/s41465-017-0030-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41465-017-0030-7

Keywords

Search

Navigation