Abstract
EEG is a direct measure of cortical neuronal activities, making it particularly well-suited to identify latent computations that underlie learning and decision making. This chapter takes a targeted look at how EEG signals and cognitive models of decision making can be linked in a mutually beneficial way. The first section begins with a tutorial on commonly used “linking” approaches, with a focus on sequential sampling models of decision making. This includes a depiction of increasingly sophisticated studies: from condition-level summaries of EEG signals with independently estimated model parameters to regressing trial-level EEG signals with trial-level estimates of model parameters. We then draw attention to joint modeling approaches that assess the bidirectional relationship between EEG signals and parameters of cognitive models. These approaches provide integrated, confirmatory frameworks that propose a common latent source that generates predictions for multiple outputs, such as behavior and neural data. The second section is a review of linking approaches in reinforcement learning models of decision making. This includes a brief history of formal linking approaches between EEG and reinforcement learning, from origins in qualitative comparison of prediction errors and aggregate EEG summaries through to regression of individual-trial data. The chapter concludes with some caveats to current linking approaches and a discussion of potential future directions for advancing the methods of linking EEG signals with cognitive models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
Van Ravenzwaaij et al. (2017) tested multiple joint models with different linking functions. For brevity, we describe just one of those models.
References
Brown, D. R., & Cavanagh, J. F. (2018). Rewarding images do not invoke the reward positivity: They inflate it. International Journal of Psychophysiology, 132, 226–235.
Calhoun, V. D., Liu, J., & Adalı, T. (2009). A review of group ICA for FMRI data and ICA for joint inference of imaging, genetic, and ERP data. NeuroImage, 45(1), S163–S172.
Caplin, A., & Dean, M. (2008). Axiomatic methods, dopamine and reward prediction error. Current Opinion in Neurobiology, 18(2), 197–202.
Cassey, P. J., Gaut, G., Steyvers, M., & Brown, S. D. (2016) A generative joint model for spike trains and saccades during perceptual decision-making. Psychonomic Bulletin & Review, 23(6):1757–1778
Cavanagh, J. F. (2015). Cortical delta activity reflects reward prediction error and related behavioral adjustments, but at different times. NeuroImage, 110, 205–216.
Cavanagh, J. F. (2019). Electrophysiology as a theoretical and methodological hub for the neural sciences. Psychophysiology, 56(2), e13314.
Cavanagh, J. F., Bismark, A. W., Frank, M. J., & Allen, J. J. (2019a). Multiple dissociations between comorbid depression and anxiety on reward and punishment processing: Evidence from computationally informed EEG. Computational Psychiatry, 3, 1–17.
Cavanagh, J. F., Eisenberg, I., Guitart-Masip, M., Huys, Q., & Frank, M. J. (2013). Frontal theta overrides Pavlovian learning biases. Journal of Neuroscience, 33(19), 8541–8548.
Cavanagh, J. F., & Frank, M. J. (2014). Frontal theta as a mechanism for cognitive control. Trends in Cognitive Sciences, 18(8), 414–421.
Cavanagh, J. F., Figueroa, C. M., Cohen, M. X., & Frank, M. J. (2012a). Frontal theta reflects uncertainty and unexpectedness during exploration and exploitation. Cerebral Cortex, 22(11), 2575–2586.
Cavanagh, J. F., Frank, M. J., Klein, T. J., & Allen, J. J. (2010). Frontal theta links prediction errors to behavioral adaptation in reinforcement learning. Neuroimage, 49(4), 3198–3209.
Cavanagh, J. F., Kumar, P., Mueller, A. A., Richardson, S. P., & Mueen, A. (2018). Diminished EEG habituation to novel events effectively classifies Parkinson’s patients. Clinical Neurophysiology, 129(2), 409–418.
Cavanagh, J. F., Rieger, R. E., Wilson, J. K., Gill, D., Fullerton, L., Brandt, E., & Mayer, A. R. (2019b). Joint analysis of frontal theta synchrony and white matter following mild traumatic brain injury. Brain Imaging and Behavior, 14(6), 2210–2223
Cavanagh, J. F., Wiecki, T. V., Cohen, M. X., Figueroa, C. M., Samanta, J., Sherman, S. J., & Frank, M. J. (2011). Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold. Nature Neuroscience, 14, 1462–1467.
Cavanagh, J. F., Zambrano-Vazquez, L., & Allen, J. J. (2012b). Theta lingua franca: A common mid-frontal substrate for action monitoring processes. Psychophysiology, 49(2), 220–238.
Chase, H. W., Swainson, R., Durham, L., Benham, L., & Cools, R. (2011). Feedback-related negativity codes prediction error but not behavioral adjustment during probabilistic reversal learning. Journal of Cognitive Neuroscience, 23(4), 936–946.
Cockburn, J., & Holroyd, C. B. (2018). Feedback information and the reward positivity. International Journal of Psychophysiology, 132, 243–251.
Cohen, M. X. (2014). Analyzing neural time series data: Theory and practice. MIT Press.
Cohen, M. X. (2018). Using spatiotemporal source separation to identify prominent features in multichannel data without sinusoidal filters. European Journal of Neuroscience, 48(7), 2454–2465.
Cohen, M. X., & Donner, T. H. (2013). Midfrontal conflict-related theta-band power reflects neural oscillations that predict behavior. Journal of Neurophysiology, 110(12), 2752–2763.
Cohen, M. X., & Ranganath, C. (2007). Reinforcement learning signals predict future decisions. Journal of Neuroscience, 27(2), 371–378.
Collins, A. G. E., & Frank, M. J. (2012). How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis. European Journal of Neuroscience, 35(7), 1024–1035.
Collins, A. G. E., & Frank, M. J. (2016). Neural signature of hierarchically structured expectations predicts clustering and transfer of rule sets in reinforcement learning. Cognition, 152, 160–169.
Collins, A. G. E., & Frank, M. J. (2018). Within-and across-trial dynamics of human eeg reveal cooperative interplay between reinforcement learning and working memory. Proceedings of the National Academy of Sciences, 115(10), 2502–2507.
Collins, A. G. E., & Koechlin, E. (2012). Reasoning, learning, and creativity: Frontal lobe function and human decision-making. PLoS Biology, 10(3), e1001293.
Correa, C. M., Noorman, S., Jiang, J., Palminteri, S., Cohen, M. X., Lebreton M, & van Gaal S. (2018). How the level of reward awareness changes the computational and electrophysiological signatures of reinforcement learning. Journal of Neuroscience, 38(48), 10338–10348.
Dien, J. (2012). Applying principal components analysis to event-related potentials: A tutorial. Developmental Neuropsychology, 37(6), 497–517.
Dien, J., Khoe, W., & Mangun, G. R. (2007). Evaluation of PCA and ICA of simulated ERPS: Promax vs. Infomax rotations. Human Brain Mapping, 28(8),742–763.
Eppinger, B., Walter, M., & Li, S. C. (2017). Electrophysiological correlates reflect the integration of model-based and model-free decision information. Cognitive, Affective, & Behavioral Neuroscience, 17(2), 406–421.
Fischer, A. G., & Ullsperger, M. (2013). Real and fictive outcomes are processed differently but converge on a common adaptive mechanism. Neuron, 79(6), 1243–1255.
Folstein, J. R., & Van Petten, C. (2008). Influence of cognitive control and mismatch on the n2 component of the ERP: A review. Psychophysiology, 45(1), 152–170.
Forstmann, B. U., Dutilh, G., Brown, S., Neumann, J., Von Cramon, D. Y., Ridderinkhof, K. R., & Wagenmakers, E. J. (2008). Striatum and pre-SMA facilitate decision-making under time pressure. Proceedings of the National Academy of Sciences, 105(45), 17538–17542.
Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: A neurocomputational account of cognitive deficits in medicated and nonmedicated parkinsonism. Journal of Cognitive Neuroscience, 17(1), 51–72.
Fries, P. (2009). Neuronal gamma-band synchronization as a fundamental process in cortical computation. Annual Review of Neuroscience, 32, 209–224.
Friston, K. (2003). Learning and inference in the brain. Neural Networks, 16(9), 1325–1352.
Friston, K. (2005). A theory of cortical responses. Philosophical Transactions of the Royal Society B: Biological Sciences, 360(1456), 815–836.
Friston, K. (2010). The free-energy principle: A unified brain theory? Nature Reviews Neuroscience, 11(2), 127–138.
Hauser, T. U., Iannaccone, R., Stämpfli, P., Drechsler, R., Brandeis, D., Walitza, S., & Brem, S. (2014). The feedback-related negativity (FRN) revisited: New insights into the localization, meaning and network organization. NeuroImage, 84, 159–168.
Holroyd, C. B., & Coles, M. G. (2002). The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109(4), 679.
Holroyd, C. B., Pakzad-Vaezi, K. L., & Krigolson, O. E. (2008). The feedback correct-related positivity: Sensitivity of the event-related brain potential to unexpected positive feedback. Psychophysiology, 45(5), 688–697.
Hunt, L. T., Kolling, N., Soltani, A., Woolrich, M. W., Rushworth, M. F. S., & Behrens, T. E. J. (2012). Mechanisms underlying cortical activity during value-guided choice. Nature Neuroscience, 15(3), 470–476.
Ichikawa, N., Siegle, G. J., Dombrovski, A., & Ohira, H. (2010). Subjective and model-estimated reward prediction: Association with the feedback-related negativity (FRN) and reward prediction error in a reinforcement learning task. International Journal of Psychophysiology, 78(3), 273–283.
Karayanidis, F., Mansfield, E. L., Galloway, K. L., Smith, J. L., Provost, A., & Heathcote, A. J. (2009). Anticipatory reconfiguration elicited by fully and partially informative cues that validly predict a switch in task. Cognitive, Affective, & Behavioral Neuroscience, 9, 202–215.
Luck, S. J. (2014). An introduction to the event-related potential technique. MIT Press.
Ly, A., Boehm, U., Heathcote, A., Turner, B. M., Forstmann, B. U., Marsman, M., & Matzke, D. (2018). A flexible and efficient hierarchical Bayesian approach to the exploration of individual differences in cognitive-model-based neuroscience. In A. A. Moustafa (Ed.), Computational models of brain and behavior. Wiley Blackwell.
Nassar, M. R., Bruckner, R., & Frank, M. J. (2019). Statistical context dictates the relationship between feedback-related EEG signals and learning. Elife, 8, e46975.
Otto, A. R., Skatova, A., Madlon-Kay, S., & Daw, N. D. (2014). Cognitive control predicts use of model-based reinforcement learning. Journal of Cognitive Neuroscience, 27(2), 319–333.
Pedersen, M. L., Frank, M. J., & Biele, G. (2017). The drift diffusion model as the choice rule in reinforcement learning. Psychonomic Bulletin & Review, 24(4), 1234–1251.
Pessoa, L. (2008). On the relationship between emotion and cognition. Nature Reviews Neuroscience, 9(2), 148–158.
Philiastides, M. G., Biele, G., Vavatzanidis, N., Kazzer, P., & Heekeren, H. R. (2010). Temporal dynamics of prediction error processing during reward-based decision making. NeuroImage, 53(1), 221–232.
Proudfit, G. H. (2015). The reward positivity: From basic research on reward to a biomarker for depression. Psychophysiology, 52(4), 449–459.
Provost, A., Johnson, B., Karayanidis, F., Brown, S. D., & Heathcote, A. (2013). Two routes to expertise in mental rotation. Cognitive Science, 37(7), 1321–1342.
Rac-Lubashevsky, R., & Kessler, Y. (2019). Revisiting the relationship between the p3b and working memory updating. Biological Psychology, 148, 107769.
Rescorla, R., & Wagner, A. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. Black, & W. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). Appleton-Century Crofts.
Ribas-Fernandes, J. J., Solway, A., Diuk, C., McGuire, J. T., Barto, A. G., Niv, Y., & Botvinick, M. M. (2011). A neural signature of hierarchical reinforcement learning. Neuron, 71(2), 370–379.
Rouder, J. N., & Haaf, J. M. (2019). A psychometrics of individual differences in experimental tasks. Psychonomic Bulletin & Review, 26(2), 452–467.
Sambrook, T. D., & Goslin, J. (2014). Mediofrontal event-related potentials in response to positive, negative and unsigned prediction errors. Neuropsychologia, 61, 1–10.
Sambrook, T. D., Hardwick, B., Wills, A. J., & Goslin, J. (2018). Model-free and model-based reward prediction errors in EEG. NeuroImage, 178, 162–171.
Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593–1599.
Shackman, A. J., Salomons, T. V., Slagter, H. A., Fox, A. S., Winter, J. J., & Davidson, R. J. (2011). The integration of negative affect, pain and cognitive control in the cingulate cortex. Nature Reviews Neuroscience, 12(3), 154–167.
Siegel, M., Donner, T. H., & Engel, A. K. (2012). Spectral fingerprints of large-scale neuronal interactions. Nature Reviews Neuroscience, 13(2), 121–134.
Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An Introduction. MIT Press.
Talmi, D., Atkinson, R., & El-Deredy, W. (2013). The feedback-related negativity signals salience prediction errors, not reward prediction errors. Journal of Neuroscience, 33(19):8264–8269.
Thorndike, E. (1898). Animal intelligence: An experimental study of the associative processes in animals. The Psychological Review: Monograph Supplements, 2(4), i–109.
Threadgill, A. H., & Gable, P. A. (2018). The sweetness of successful goal pursuit: Approach-motivated pregoal states enhance the reward positivity during goal pursuit. International Journal of Psychophysiology, 132, 277–286.
Turner, B. M., Forstmann, B. U., Love, B. C., Palmeri, T. J., & Van Maanen, L. (2017). Approaches to analysis in model-based cognitive neuroscience. Journal of Mathematical Psychology, 76, 65–79.
Turner, B. M., Forstmann, B. U., & Steyvers, M. (2019). Joint models of neural and behavioral data. Springer.
Van Ravenzwaaij, D., Provost, A., & Brown, S. D. (2017). A confirmatory approach for integrating neural and behavioral data into a single model. Journal of Mathematical Psychology, 76, 131–141.
Wagenmakers, E. J., van der Maas, H. J. L., Dolan, C., & Grasman, R. P. P. P. (2008). Ez does it! Extensions of the EZ-diffusion model. Psychonomic Bulletin & Review, 15, 1229–1235.
Whitton, A. E., Kakani, P., Foti, D., Van’t Veer, A., Haile, A., Crowley, D. J., & Pizzagalli, D. A. (2016) Blunted neural responses to reward in remitted major depression: A high-density event-related potential study. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 1(1), 87–95.
Acknowledgements
Funding This work was supported by: Australian Research Council (ARC) Discovery Early Career Researcher Award (Hawkins, DE170100177); ARC Discovery Project (Hawkins and Brown, DP180103613); National Institutes of Mental Health (Cavanagh, NIMH RO1MH119382-01). The funding sources had no role in the study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Hawkins, G.E., Cavanagh, J.F., Brown, S.D., Steyvers, M. (2024). Cognitive Models as a Tool to Link Decision Behavior with EEG Signals. In: Forstmann, B.U., Turner, B.M. (eds) An Introduction to Model-Based Cognitive Neuroscience. Springer, Cham. https://doi.org/10.1007/978-3-031-45271-0_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-45271-0_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45270-3
Online ISBN: 978-3-031-45271-0
eBook Packages: Behavioral Science and PsychologyBehavioral Science and Psychology (R0)