Psychonomic Bulletin & Review

, Volume 21, Issue 2, pp 436–444 | Cite as

Unexpected downshifts in reward magnitude induce variation in human behavior

  • Greg Jensen
  • Patricia D. Stokes
  • Anthea Paterniti
  • Peter D. Balsam
Brief Report


We investigated how changes in outcome magnitude affect behavioral variation in human volunteers. Our participants entered strings of characters using a computer keyboard, receiving feedback (gaining a number of points) for any string at least ten characters long. During a “surprise” phase in which the number of points awarded was changed, participants only increased their behavioral variability when the reward value was downshifted to a lower amount, and only when such a shift was novel. Upshifts in reward did not have a systematic effect on variability.


Human learning Variability 


Author Note

The authors thank Karen Zechowy and Jacqui Rick for their assistance in conducting this experiment. This work was supported by National Institute of Mental Health Grant No. 5R01MH068073, awarded to P.B.


  1. Allan, L. G., Hannah, S. D., Crump, M. J. C., & Siegel, S. (2008). The psychophysics of contingency assessment. Journal of Experimental Psychology. General, 137, 226–243.PubMedCrossRefGoogle Scholar
  2. Amsel, A. (1992). Frustration theory. Cambridge, UK: Cambridge University Press.CrossRefGoogle Scholar
  3. Balsam, P., Deich, J., Ohyama, T., & Stokes, P. D. (1997). Origins of new behavior. In W. O’Donohue (Ed.), Learning and behavior therapy (pp. 403–421). Boston, MA: Allyn & Bacon.Google Scholar
  4. Bowerman, M. (1982). Starting to talk worse: Clues to language acquisition from children’s late speech errors. In S. Strauss (Ed.), U-shaped behavior growth (pp. 101–145). New York, NY: Academic Press.CrossRefGoogle Scholar
  5. Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O. (2010). Dopmaine in motivational control: Rewarding, aversive, and alerting. Neuron, 68, 815–834.PubMedCentralPubMedCrossRefGoogle Scholar
  6. Conover, W. J., & Iman, R. L. (1981). Rank transformations as a bridge between parametric and nonparametric statistics. The American Statistician, 35, 124–129.Google Scholar
  7. Davison, M., & Baum, W. M. (2000). Choice in a variable environment: Every reinforcer counts. Journal of the Experimental Analysis of Behavior, 74, 1–24.PubMedCentralPubMedCrossRefGoogle Scholar
  8. Elsner, B., & Hommel, B. (2004). Contiguity and contingency in action-effect learning. Psychological Research, 68, 138–154.PubMedCrossRefGoogle Scholar
  9. Freidin, E., Cuello, M. I., & Kacelnik, A. (2009). Successive negative contrast in a bird: Starlings’ behaviour after unpredictable negative changes in food quality. Animal Behaviour, 77, 857–865.CrossRefGoogle Scholar
  10. Goldin-Meadows, S., Alibani, M. W., & Church, R. B. (1993). Transitions in concept acquisition: Using the hand to read the mind. Psychological Review, 100, 279–297.CrossRefGoogle Scholar
  11. Gusfield, D. (1997). Algorithms on strings, trees and sequences: Computer science and computational biology. Cambridge, UK: Cambridge University Press.CrossRefGoogle Scholar
  12. Hoeffding, W. (1948). A class of statistics with asymptotically normal distribution. Annals of Mathematical Statistics, 19, 293–325.Google Scholar
  13. Johnson, P. E., Duran, A. S., Hassebrock, F., Moller, J., Prietula, M., Feltovich, P. J., & Swanson, D. B. (1981). Expertise and error in diagnostic reasoning. Cognitive Science, 5, 235–283.CrossRefGoogle Scholar
  14. Kamin, L. J. (1969). Selective associations and conditioning. In N. J. Mackintosh & W. K. Honig (Eds.), Fundamental issues in associative learning (pp. 42–64). Halifax, NS: Dalhousie University Press.Google Scholar
  15. Killeen, P. R. (1994). Frustration: Theory and practice. Psychonomic Bulletin & Review, 1, 323–326. doi: 10.3758/BF03213973 CrossRefGoogle Scholar
  16. Kinloch, J. M., Foster, T. M., & McEwan, J. S. A. (1981). Extinction-induced variability in human behavior. Psychological Record, 59, 347–370.Google Scholar
  17. Lesgold, A., Rubinson, H., Feltovich, P., Klopfer, R. G. D., & Wang, Y. (1988). Expertise is a complex skill: Diagnosing x-ray pictures. In M. T. H. Chi, R. Glaser, & M. J. Farr (Eds.), The nature of expertise (pp. 311–342). Hillsdale, NJ: Erlbaum.Google Scholar
  18. Levenshtein, V. I. (1966). Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics – Doklady, 10, 707–710.Google Scholar
  19. March, J. G. (1991). Exploration and exploitation in organizational learning. Organization Science, 2, 71–87.CrossRefGoogle Scholar
  20. Moore, J. W., Lagnado, D., Deal, D. C., & Haggard, P. (2009). Feelings of control: Contingency determines experience of action. Cognition, 110, 279–283.PubMedCrossRefGoogle Scholar
  21. Neuringer, A. (2002). Operant variability: Evidence, functions, and theory. Psychonomic Bulletin & Review, 9, 672–705.CrossRefGoogle Scholar
  22. Neuringer, A., & Jensen, G. (2010). Operant variability and voluntary action. Psychological Review, 117, 972–993.PubMedCrossRefGoogle Scholar
  23. Neuringer, A., Kornell, N., & Olufs, M. (2001). Stability and variability in extinction. Journal of Experimental Psychology. Animal Behavior Processes, 27, 79–94.PubMedCrossRefGoogle Scholar
  24. Nickerson, R. S. (2002). The production and perception of randomness. Psychological Review, 109, 330–357. doi: 10.1037/0033-295X.109.2.330 PubMedCrossRefGoogle Scholar
  25. Papini, M. R. (2002). Pattern and process in the evolution of learning. Psychological Review, 109, 186–201. doi: 10.1037/0033-295X.109.1.186 PubMedCrossRefGoogle Scholar
  26. Pinheiro, H. P., de Souza Pinheiro, A., & Sen, P. K. (2005). Comparison of genomic sequences using the hamming distance. Journal of Statistical Planning and Inference, 130, 325–339.CrossRefGoogle Scholar
  27. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). New York, NY: Appleton-Century-Crofts.Google Scholar
  28. Roulston, M. (1999). Estimating the errors on measured entropy and mutual information. Physica D: Nonlinear Phenomena, 125, 285–294.CrossRefGoogle Scholar
  29. Schultz, W. (2006). Behavioral theories and the neurophysiology of reward. Annual Review of Psychology, 57, 87–115.PubMedCrossRefGoogle Scholar
  30. Shahan, T. A., & Chase, P. N. (2002). Novelty, stimulus control, and operant variability. Behavior Analyst, 25, 175–190.PubMedCentralPubMedGoogle Scholar
  31. Siegler, R. S. (1995). How does change occur? A microgenetic study of number conservation. Cognitive Psychology, 28, 225–273.PubMedCrossRefGoogle Scholar
  32. Siegler, R. S., & Jenkins, E. (1989). How children discover new strategies. Hillsdale, NJ: Erlbaum.Google Scholar
  33. da Silva Souza, A., Abreu-Rodrigues, J., & Baumann, A. A. (2010). History effects on induced and operant variability. Learning & Behavior, 38, 426–437.Google Scholar
  34. Stahlman, W. D., & Blaisdell, A. P. (2011). The modulation of operant variation by the probability, magnitude, and delay of reinforcement. Learning and Motivation, 42, 221–236.PubMedCentralPubMedCrossRefGoogle Scholar
  35. Stahlman, W. D., Young, M. E., & Blaisdell, A. P. (2010). Response variability in pigeons in a Pavlovian task. Learning & Behavior, 38, 111–118.CrossRefGoogle Scholar
  36. Stokes, P. D. (2001). Variability, constraints, and creativity: Shedding light on Claude Monet. American Psychologist, 56, 355–359.PubMedCrossRefGoogle Scholar
  37. Stokes, P. D., Lai, B., Holtz, D., Rigsbee, E., & Cherrick, D. (2008). Effects of practice on variability, effects of variability on transfer. Journal of Experimental Psychology. Human Perception and Performance, 34, 640–659.PubMedCrossRefGoogle Scholar
  38. Stokes, P. D., Mechner, F., & Balsam, P. (1999). Effects of different acquisition procedures on response variability. Animal Learning & Behavior, 27, 28–41.CrossRefGoogle Scholar
  39. Wagner, A. R., & Brandon, S. (1989). Evolution of a structured connectionist model of Pavlovian conditioning (æsop). In S. B. Klein & R. R. Mowrer (Eds.), Contemporary learning theory: Pavlovian conditioning and the status of traditional learning theory (pp. 149–189). Hillsdale, NJ: Erlbaum.Google Scholar
  40. Wang, D. V., & Tsien, J. Z. (2011). Convergent processing of both positive and negative motivational signals by the VTA dopamine neuronal populations. PLoS One, 6, e17047. doi: 10.1371/journal.pone.0016505 PubMedCentralPubMedCrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2013

Authors and Affiliations

  • Greg Jensen
    • 1
  • Patricia D. Stokes
    • 1
    • 2
  • Anthea Paterniti
    • 2
  • Peter D. Balsam
    • 1
    • 2
  1. 1.Columbia UniversityNew York CityUSA
  2. 2.Barnard CollegeNew York CityUSA

Personalised recommendations