Preference reversal and delayed reinforcement


Pigeons chose, in a two-key discrete-trial procedure, between 2- and 4-sec access to grain, with the larger amount always presented 4 sec later than the smaller. As the delay between the choice and the availability of the smaller reinforcement was varied from .01 to 12 sec, all subjects reversed preference from the small-early to the large-late reinforcement. The values of delay at which preference reversed were approximately consistent with the matching law as adapted for delayed reinforcement.

Reference Notes

  1. 1.

    Ainslie, G. W.Impulse and compulsion: A behavioral economic understanding of involuntary behavior. Unpublished manuscript, 1980.

  2. 2.

    Ainslie, G. W.Correlates of peck-withholding on a discrete trial specious reward schedule. Unpublished manuscript, 1981.


  1. Ainslie, G. W. Impulse control in pigeons.Journal of the Experimental Analysis of Behavior, 1974,21, 485–489.

    Article  PubMed  Google Scholar 

  2. Ainslie, G. Specious reward: A behavioral theory of impulsiveness and impulse control.Psychological Bulletin, 1975,82, 463–496.

    Article  PubMed  Google Scholar 

  3. Baum, W. M., &Rachlin, H. C. Choice as time allocation.Journal of the Experimental Analysis of Behavior, 1969,12, 861–874.

    Article  PubMed  Google Scholar 

  4. Brown, R., &Herrnstein, R. J. Psychology. Boston: Little, Brown, 1975.

    Google Scholar 

  5. Chung, S.-H., &Herrnstein, R. J. Choice and delay of reinforcement.Journal of the Experimental Analysis of Behavior, 1967,10, 67–74.

    Article  PubMed  Google Scholar 

  6. Deluty, M. Z. Self-control and impulsiveness involving aversive events.Journal of Experimental Psychology: Animal Behavior Processes, 1978,4, 250–266.

    Article  Google Scholar 

  7. Fantino, E. Immediate reward followed by extinction vs. later reward without extinction.Psychonomic Science, 1966,6, 233–234.

    Google Scholar 

  8. Herrnstein, R. J. Self control as response strength. In C. M. Bradshaw, E. Szabadi, & C. F. Lowe (Eds.),Quantification of steady-state operant behaviour. Amsterdam: Elsevier/North-Holland Biomedical Press, 1981.

    Google Scholar 

  9. Logan, R. A., &Spanier, D. Relative effect of delay of food and water reward.Journal of Comparative and Physiological Psychology, 1970,72, 102–104.

    Article  Google Scholar 

  10. Mazur, J. E., &Logue, A. W. Choice in a “self-control” paradigm: Effects of a fading procedure.Journal of the Experimental Analysis of Behavior, 1978,30, 11–17.

    Article  PubMed  Google Scholar 

  11. Navarick, D. J., &Fantino, E. Self-control and general models of choice.Journal of Experimental Psychology: Animal Behavior Processes, 1976,2, 75–87.

    Article  Google Scholar 

  12. Prelec, D., &Herrnstein, R. J. Feedback functions for reinforcement: A paradigmatic experiment.Animal Learning & Behavior, 1978,6, 181–186.

    Google Scholar 

  13. Rachlin, H. Introduction to modern behaviorism. San Francisco: Freeman, 1970.

    Google Scholar 

  14. Rachlin, H., &Green, L. Commitment, choice and self-control.Journal of the Experimental Analysis of Behavior, 1972,17, 15–22.

    Article  PubMed  Google Scholar 

  15. Renner, K. E. Delay of reinforcement: A historical review.Psychological Bulletin, 1964,61, 341–361.

    Article  PubMed  Google Scholar 

  16. Solnick, J. V., Kannenberg, C. H., Eckerman, D. A., &Waller, M. B. An experimental analysis of impulsivity and impulse control in humans.Learning and Motivation, 1980,11, 61–77.

    Article  Google Scholar 

  17. Strotz, R. H. Myopia and inconsistency in dynamic utility maximization.Review of Economic Studies, 1956,23, 166–180.

    Google Scholar 

  18. Williams, B. A., &Fantino, E. Effects on choice of reinforcement delay and conditioned reinforcement.Journal of the Experimental Analysis of Behavior, 1978,29, 77–86.

    Article  PubMed  Google Scholar 

Download references

Author information



Corresponding authors

Correspondence to George Ainslie or R. J. Herrnstein.

Additional information

This research was conducted with support from NIMH, Grant MH-15494 to Harvard University.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Ainslie, G., Herrnstein, R.J. Preference reversal and delayed reinforcement. Animal Learning & Behavior 9, 476–482 (1981).

Download citation


  • Preference Reversal
  • Delay Function
  • Discount Function
  • Time Discount
  • Fading Procedure