Advertisement

Combining error-driven models of associative learning with evidence accumulation models of decision-making

  • David K. SewellEmail author
  • Hayley K. Jach
  • Russell J. Boag
  • Christina A. Van Heer
Theoretical Review

Abstract

As people learn a new skill, performance changes along two fundamental dimensions: Responses become progressively faster and more accurate. In cognitive psychology, these facets of improvement have typically been addressed by separate classes of theories. Reductions in response time (RT) have usually been addressed by theories of skill acquisition, whereas increases in accuracy have been explained by associative learning theories. To date, relatively little work has examined how changes in RT relate to changes in response accuracy, and whether these changes can be accounted for quantitatively within a single theoretical framework. The current work examines joint changes in accuracy and RT in a probabilistic category learning task. We report a model-based analysis of changes in the shapes of RT distributions for different category responses at the level of individual stimuli over the course of learning. We show that changes in performance are determined solely by changes in the quality of information entering the decision process. We then develop a new model that combines an associative learning front end with a sequential sampling model of the decision process, showing that the model provides a good account of all aspects of the learning data. We conclude by discussing potential extensions of the model and future directions for theoretical development that are opened up by our findings.

Keywords

Error-driven learning Category learning Response time modeling Diffusion model Categorization 

Notes

Author note

Address correspondence to David K. Sewell at the School of Psychology, The University of Queensland, St. Lucia, QLD 4072, Australia. Electronic mail may be sent to d.sewell@uq.edu.au. This research was supported by an Australian Research Council Discovery Early Career Researcher Award (DE140100772) to David Sewell.

References

  1. Ashby, F. G., Boynton, G., & Lee, W. W. (1994). Categorization response time with multidimensional stimuli. Perception & Psychophysics, 55, 11–27.CrossRefGoogle Scholar
  2. Ashby, F. G., & Maddox, W. T. (1993). Relations between prototype, exemplar, and decision bound models of categorization. Journal of Mathematical Psychology, 37, 372–400.CrossRefGoogle Scholar
  3. Ashby, F. G. & Maddox, W. T. (1994). A response time theory of separability and integrality in speeded classification. Journal of Mathematical Psychology, 38, 423–466.CrossRefGoogle Scholar
  4. Bott, L., Hoffman, A. B., & Murphy, G. L. (2007). Blocking in category learning. Journal of Experimental Psychology: General, 136, 685–699.CrossRefGoogle Scholar
  5. Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision, 10, 433–436.CrossRefGoogle Scholar
  6. Brown, S., & Heathcote, A. (2005). A ballistic model of choice response time. Psychological Review, 112, 117–128.CrossRefGoogle Scholar
  7. Brown, S. D. & Heathcote, A. (2008). The simplest complete model of choice response time: Linear ballistic accumulation. Cognitive Psychology, 57, 153–178.CrossRefGoogle Scholar
  8. Bush, R. R., & Mosteller, F. (1951). A mathematical model for simple learning. Psychological Review, 58, 313–323.CrossRefGoogle Scholar
  9. Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24, 87–185.CrossRefGoogle Scholar
  10. Craig, S., Lewandowsky, S., & Little, D. R. (2011). Error discounting in probabilistic category learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37, 673–687.Google Scholar
  11. Denton, S. E., Kruschke, J. K., & Erickson, M. A. (2008). Rule-based extrapolation: A continuing challenge for exemplar models. Psychonomic Bulletin & Review, 15, 780–786.CrossRefGoogle Scholar
  12. Donkin, C., Brown, S., Heathcote, A., & Wagenmakers, E. J. (2011). Diffusion versus linear ballistic accumulation: Different models but the same conclusions about psychological processes? Psychonomic Bulletin & Review, 18, 61–69.CrossRefGoogle Scholar
  13. Dutilh, G., Krypotos, A. M., & Wagenmakers, E. J. (2011). Task-related vs. stimulus-specific practice: A diffusion model account. Experimental Psychology, 58, 434–442.CrossRefGoogle Scholar
  14. Dutilh, G., Vandekerckhove, J., Tuerlinckx, F., & Wagenmakers, E. J. (2009). A diffusion model decomposition of the practice effect. Psychonomic Bulletin & Review, 16, 1026–1036.CrossRefGoogle Scholar
  15. Edwards, W. (1961). Probability learning in 1000 trials. Journal of Experimental Psychology, 62, 385–394.CrossRefGoogle Scholar
  16. Erickson, M. A., & Kruschke, J. K. (1998). Rules and exemplars in category learning. Journal of Experimental Psychology: General, 127, 107–140.CrossRefGoogle Scholar
  17. Erickson, M. A., & Kruschke, J. K. (2002). Rule-based extrapolation in perceptual categorization. Psychonomic Bulletin & Review, 9, 160–168.CrossRefGoogle Scholar
  18. Estes, W. K. (1950). Toward a statistical theory of learning. Psychological Review, 57, 94–107.CrossRefGoogle Scholar
  19. Fifić, M., Little, D. R., & Nosofsky, R. M. (2010). Logical-rule models of classification response times: A synthesis of mental-architecture, random-walk, and decision-bound approaches. Psychological Review, 117, 309–348.CrossRefGoogle Scholar
  20. Frank, M. J. (2005). Dynamic dopamine modulation in the basal ganglia: A neurocomputational account of cognitive deficits in medicated and nonmedicated parkinsonism. Journal of Cognitive Neuroscience, 17, 51–72.CrossRefGoogle Scholar
  21. Frank, M. J. (2006). Hold your horses: A dynamic computational role for the subthalamic nucleus in decision making. Neural Networks, 19, 1120–1136.CrossRefGoogle Scholar
  22. Frank, M. J., Gagne, C., Nyhus, E., Masters, S., Wiecki, T. V., Cavanagh, J. F., & Badre, D. (2015). fMRI and EEG predictors of dynamic decision parameters during human reinforcement learning. Journal of Neuroscience, 35, 485–494.CrossRefGoogle Scholar
  23. Frank, M. J., Seeberger, L. C., & O’Reilly, R. C. (2004). By carrot or by stick: Cognitive reinforcement learning in parkinsonism. Science, 306, 1940–1943.CrossRefGoogle Scholar
  24. Friedman, D., & Massaro, D. W. (1998). Understanding variability in binary and continuous choice. Psychonomic Bulletin & Review, 5, 370–389.CrossRefGoogle Scholar
  25. Garner, W. R. (1974). The processing of information and structure. Potomac, MD: Erlbaum.Google Scholar
  26. Goodman, N. D., Tenenbaum, J. B., Feldman, J., & Griffiths, T. L. (2008). A rational analysis of rule-based concept learning. Cognitive Science, 32, 108–154.CrossRefGoogle Scholar
  27. Heathcote, A., Brown, S., & Mewhort, D. J. K. (2000). The power law repealed: The case for an exponential law of practice. Psychonomic Bulletin & Review, 7, 185–207.CrossRefGoogle Scholar
  28. Holmes, W. R., Trueblood, J. S., & Heathcote, A. (2016). A new framework for modeling decisions about changing information: The piecewise linear ballistic accumulator model. Cognitive Psychology, 85, 1–29.CrossRefGoogle Scholar
  29. Jamieson, R. K., Crump, M. J. C., & Hannah, S. D. (2012). An instance theory of associative learning. Learning & Behavior, 40, 61–82.CrossRefGoogle Scholar
  30. Kamin, L. J. (1968). “Attention-like” processes in classical conditioning. In M. R. Jones (Ed.), Miami symposium on the prediction of behavior: Aversive stimulation (pp. 9–33). Coral Gables, FL: University of Miami Press.Google Scholar
  31. Kruschke, J. K. (1992). ALCOVE: An exemplar-based connectionist model of category learning. Psychological Review, 99, 22–44.CrossRefGoogle Scholar
  32. Kruschke, J. K. (1996). Dimensional relevance shifts in category learning. Connection Science, 8, 225–247.CrossRefGoogle Scholar
  33. Kruschke, J. K. (2008). Models of categorization. In R. Sun (Ed.), The Cambridge handbook of computational psychology (pp. 267–301). Cambridge, UK: Cambridge University Press.CrossRefGoogle Scholar
  34. Kruschke, J. K., & Johansen, M. K. (1999). A model of probabilistic category learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 1083–1119.Google Scholar
  35. Kurtz, K. J., Levering, K. R., Stanton, R. D., Romero, J., & Morris, S. N. (2013). Human learning of elemental category structure: Revising the classic result of Shepard, Hovland, and Jenkins (1961). Journal of Experimental Psychology: Learning, Memory, and Cognition, 39, 552–572.Google Scholar
  36. Lamberts, K. (1995). Categorization under time pressure. Journal of Experimental Psychology: General, 124, 161–180.CrossRefGoogle Scholar
  37. Lamberts, K. (1998). The time course of categorization. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24, 695–711.Google Scholar
  38. Lamberts, K. (2000). Information-accumulation theory of speeded categorization. Psychological Review, 107, 227–260.CrossRefGoogle Scholar
  39. Le Pelley, M. E. (2004). The role of associative history in models of associative learning: A selective review and a hybrid model. Quarterly Journal of Experimental Psychology, 57B, 193–243.CrossRefGoogle Scholar
  40. Lewandowsky, S. (1995). Base-rate neglect in ALCOVE: A critical reevaluation. Psychological Review, 102, 185–191.CrossRefGoogle Scholar
  41. Little, D. R., Nosofsky, R. M., & Denton, S. E. (2011). Response-time tests of logical-rule models of categorization. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37, 1–27.Google Scholar
  42. Little, D. R., Nosofsky, R. M., Donkin, C., & Denton, S. E. (2013). Logical rules and the classification of integral-dimension stimuli. Journal of Experimental Psychology: Learning, Memory, and Cognition, 39, 801–820.Google Scholar
  43. Little, D. R., Wang, T., & Nosofsky, R. M. (2016). Sequence-sensitive exemplar and decision-bound accounts of speeded-classification performance in a modified Garner-tasks paradigm. Cognitive Psychology, 89, 1–38.CrossRefGoogle Scholar
  44. Liu, C. C., & Watanabe, T. (2012). Accounting for speed-accuracy tradeoff in perceptual learning. Vision Research, 61, 107–114.CrossRefGoogle Scholar
  45. Logan, G. D. (1988). Toward an instance theory of automatization. Psychological Review, 95, 492–527.CrossRefGoogle Scholar
  46. Logan, G. D. (1992). Shapes of reaction-time distributions and shapes of learning curves: A test of the instance theory of automaticity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 883–914.Google Scholar
  47. Logan, G. D. (2002). An instance theory of attention and memory. Psychological Review, 109, 376–400.CrossRefGoogle Scholar
  48. Love, B. C., Medin, D. L., & Gureckis, T. M. (2004). SUSTAIN: A network model of category learning. Psychological Review, 111, 309–332.CrossRefGoogle Scholar
  49. Luce, R. D. (1959). Individual choice behavior. New York, NY: Wiley.Google Scholar
  50. Luce, R. D. (1986). Response times: Their role in inferring elementary mental organization. Oxford, UK: Oxford University Press.Google Scholar
  51. Maddox, W. T., Ashby, F. G., & Gottlob, L. R. (1998). Response time distributions in multidimensional perceptual categorization. Perception & Psychophysics, 60, 620–637.CrossRefGoogle Scholar
  52. Medin, D. L., & Schaffer, M. M. (1978). Context theory of classification learning. Psychological Review, 85, 207–238.CrossRefGoogle Scholar
  53. Moneer, S., Wang, T., & Little, D. R. (2016). The processing architectures of whole-object features: A logical-rules approach. Journal of Experimental Psychology: Human Perception and Performance, 42, 1443–1465.Google Scholar
  54. Newell, A., & Rosenbloom, P. S. (1981). Mechanisms of skill acquisition and the law of practice. In J. R. Anderson (Ed.), Cognitive skills and their acquisition (pp. 1–55). Hillsdale, NJ: Erlbaum.Google Scholar
  55. Nosofsky, R. M. (1986). Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General, 115, 39–57.CrossRefGoogle Scholar
  56. Nosofsky, R. M., & Alfonso-Reese, L. A. (1999). Effects of similarity and practice on speeded classification response times and accuracies: Further tests of an exemplar-retrieval model. Memory & Cognition, 27, 78–93.CrossRefGoogle Scholar
  57. Nosofsky, R. M., Gluck, M. A., Palmeri, T. J., McKinley, S. C., & Gauthier, P. (1994). Comparing models of rule-based classification learning: A replication and extension of Shepard, Hovland, and Jenkins (1961). Memory & Cognition, 22, 352–369.CrossRefGoogle Scholar
  58. Nosofsky, R. M., Kruschke, J. K., & McKinley, S. C. (1992). Combining exemplar-based category representations and connectionist learning rules. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 211–233.Google Scholar
  59. Nosofsky, R. M., & Palmeri, T. J. (1997a). An exemplar-based random walk model of speeded classification. Psychological Review, 104, 266–300.CrossRefGoogle Scholar
  60. Nosofsky, R. M. & Palmeri, T. J. (1997b). Comparing exemplar-retrieval and decision-bound models of speeded perceptual classification. Perception & Psychophysics, 59, 1027–1048.CrossRefGoogle Scholar
  61. Nosofsky, R. M., & Palmeri, T. J. (2015). An exemplar-based random-walk model of categorization and recognition. In J. R. Busemeyer, Z. Wang, J. T. Townsend, & A. Eidels (Eds.), The Oxford handbook of computational and mathematical psychology (pp. 142–164). New York, NY: Oxford University Press.Google Scholar
  62. Nosofsky, R. M., Palmeri, T. J., & McKinley, S. C. (1994). Rule-plus-exception model of classification learning. Psychological Review, 101, 53-79.CrossRefGoogle Scholar
  63. Nosofsky, R. M., & Stanton, R. D. (2005). Speeded classification in a probabilistic category structure: Contrasting exemplar-retrieval, decision-bound, and prototype models. Journal of Experimental Psychology: Human Perception and Performance, 31, 608-629.Google Scholar
  64. Nosofsky, R. M., & Zaki, S. R. (2002). Exemplar and prototype models revisited: Response strategies, selective attention, and stimulus generalization. Journal of Experimental Psychology: Learning, Memory, and Cognition, 28, 924–940.Google Scholar
  65. Palmeri, T. J. (1997). Exemplar similarity and the development of automaticity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 324–354.Google Scholar
  66. Palmeri, T. J. (1999). Theories of automaticity and the power law of practice. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 543–551.Google Scholar
  67. Pedersen, M. L., Frank, M. J., & Biele, G. (2017). The drift diffusion model as the choice rule in reinforcement learning. Psychonomic Bulletin & Review, 24(4), 1234–1251. doi: https://doi.org/10.3758/s13423-016-1199-y CrossRefGoogle Scholar
  68. Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437–442.CrossRefGoogle Scholar
  69. Petrov, A. A., Van Horn, N. M., & Ratcliff, R. (2011). Dissociable perceptual-learning mechanisms revealed by diffusion-model analysis. Psychonomic Bulletin & Review, 18, 490–497.CrossRefGoogle Scholar
  70. Rae, B., Heathcote, A., Donkin, C., Averell, L., & Brown, S. (2014). The hare and the tortoise: Emphasizing speed can change the evidence used to make decisions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40, 1226–1243.Google Scholar
  71. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59–108.CrossRefGoogle Scholar
  72. Ratcliff, R. (2013). Parameter variability and distributional assumptions in the diffusion model. Psychological Review, 120, 281–292.CrossRefGoogle Scholar
  73. Ratcliff, R., & Frank, M. J. (2012). Reinforcement-based decision making in corticostriatal circuits: Mutual constraints by neurocomputational and diffusion models. Neural Computation, 24, 1186–1229.CrossRefGoogle Scholar
  74. Ratcliff, R., & McKoon, G. (2008). The diffusion decision model: Theory and data for two-choice decision tasks. Neural Computation, 20, 873–922.CrossRefGoogle Scholar
  75. Ratcliff, R., & Rouder, J. N. (1998). Modeling response times for two-choice decisions. Psychological Science, 9, 347–356.CrossRefGoogle Scholar
  76. Ratcliff, R., & Smith, P. L. (2004). A comparison of sequential sampling models for two-choice reaction time. Psychological Review, 111, 333–367.CrossRefGoogle Scholar
  77. Ratcliff, R. & Smith, P. L. (2010). Perceptual discrimination in static and dynamic noise: The temporal relation between perceptual encoding and decision making. Journal of Experimental Psychology: General, 139, 70–94.CrossRefGoogle Scholar
  78. Ratcliff, R., Smith, P. L., Brown, S. D., & McKoon, G. (2016). Diffusion decision model: Current issues and history. Trends in Cognitive Sciences, 20, 260–281.CrossRefGoogle Scholar
  79. Ratcliff, R., Thapar, A., & McKoon, G. (2006). Aging, practice, and perceptual tasks: A diffusion model analysis. Psychology and Aging, 21, 353–371.CrossRefGoogle Scholar
  80. Ratcliff, R., Van Zandt, T., & McKoon, G. (1999). Connectionist and diffusion models of reaction time. Psychological Review, 106, 261–300.CrossRefGoogle Scholar
  81. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). New York, NY: Appleton-Century-Crofts.Google Scholar
  82. Sanborn, A. N., Griffiths, T. L., & Navarro, D. J. (2010). Rational approximations to rational models: Alternative algorithms for category learning. Psychological Review, 117, 1144–1167.CrossRefGoogle Scholar
  83. Sewell, D. K., & Lewandowsky, S. (2011). Restructuring partitioned knowledge: The role of recoordination in category learning. Cognitive Psychology, 62, 81–122.CrossRefGoogle Scholar
  84. Sewell, D. K., & Lewandowsky, S. (2012). Attention and working memory capacity: Insights from blocking, highlighting, and knowledge restructuring. Journal of Experimental Psychology: General, 141, 444–469.CrossRefGoogle Scholar
  85. Sewell, D. K., & Smith, P. L. (2016). The psychology and psychobiology of simple decisions: Speeded choice and its neural correlates. In C. Montag & M. Reuter (Eds.) Neuroeconomics (pp. 253–292). Berlin, Germany: Springer.CrossRefGoogle Scholar
  86. Sewell, D. K., Warren, H. A., Rosenblatt, D., Bennett, D., Lyons, M., & Bode, S. (2018). Feedback discounting in probabilistic categorization: Converging evidence from EEG and cognitive modeling. Computational Brain & Behavior, 1, 165–183.CrossRefGoogle Scholar
  87. Shanks, D. R., Tunney, R. J., & McCarthy, J. D. (2002). A re-examination of probability matching and rational choice. Journal of Behavioral Decision Making, 15, 233–250.CrossRefGoogle Scholar
  88. Smith, P. L., & Little, D. R. (2018). Small is beautiful: In defense of the small-N design. Psychonomic Bulletin & Review, 25, 2083–2101.CrossRefGoogle Scholar
  89. Smith, P. L., Ratcliff, R., & Sewell, D. K. (2014). Modeling perceptual discrimination in dynamic noise: Time-changed diffusion and release from inhibition. Journal of Mathematical Psychology, 59, 95–113.CrossRefGoogle Scholar
  90. Smith, P. L., & Vickers, D. (1988). The accumulator model of two-choice discrimination. Journal of Mathematical Psychology, 32, 135–168.CrossRefGoogle Scholar
  91. Swensson, R. G. (1972). The elusive tradeoff: Speed vs accuracy in visual discrimination tasks. Perception & Psychophysics, 12, 16–32.CrossRefGoogle Scholar
  92. Townsend, J. T., & Ashby, F. G. (1983). Stochastic modeling of elementary psychological processes. Cambridge, UK: Cambridge University Press.Google Scholar
  93. Tuerlinckx, F. (2004). The efficient computation of the cumulative distribution and density functions in the diffusion model. Behavior Research Methods, Instruments, & Computers, 36, 702–716.CrossRefGoogle Scholar
  94. Usher, M., & McClelland, J. L. (2001). The time course of perceptual choice: The leaky, competing accumulator model. Psychological Review, 108, 550–592.CrossRefGoogle Scholar

Copyright information

© The Psychonomic Society, Inc. 2019

Authors and Affiliations

  • David K. Sewell
    • 1
    • 2
    Email author
  • Hayley K. Jach
    • 2
  • Russell J. Boag
    • 2
    • 3
  • Christina A. Van Heer
    • 2
  1. 1.School of PsychologyThe University of QueenslandSt. LuciaAustralia
  2. 2.Melbourne School of Psychological SciencesThe University of MelbourneMelbourneAustralia
  3. 3.School of PsychologyThe University of Western AustraliaPerthAustralia

Personalised recommendations