Skip to main content
SpringerLink
Log in

Conditioned Reinforcement: Experimental and Theoretical Issues

  • Tutorial
  • Published:
The Behavior Analyst Aims and scope Submit manuscript

Abstract

The concept of conditioned reinforcement has received decreased attention in learning textbooks over the past decade, in part because of criticisms of its validity by major behavior theorists and in part because its explanatory function in a variety of different conditioning procedures has become uncertain. Critical data from the major procedures that have been used to investigate the concept (second-order schedules, chain schedules, concurrent chains, observing responses, delay-of-reinforcement procedures) are reviewed, along with the major issues of interpretation. Although the role played by conditioned reinforcement in some procedures remains unresolved, the results taken together leave little doubt that the underlying idea of conditioned value is a critical component of behavior theory that is necessary to explain many different types of data. Other processes (marking, bridging) may also operate to produce effects similar to those of conditioned reinforcement, but these clearly cannot explain the full domain of experimental effects ascribed to conditioned reinforcement and should be regarded as complements to the concept rather than theoretical competitors. Examples of practical and theoretical applications of the concept of conditioned reinforcement are also considered.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  • Amsel, A. (1992). Frustration theory: An analysis of dispositional learning and memory. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Astley, S. L., & Perkins, C. C. (1985). Stimulus duration and conditioned reinforcing value measured by a learning-tests procedure. Animal Learning & Behavior, 13, 18–24.

    Article  Google Scholar 

  • Balsam, P., & Bondy, A. (1983). The negative side effects of reward. Journal of Applied Behavior Analysis, 16, 283–296.

    Article  PubMed  PubMed Central  Google Scholar 

  • Baptista, L. F., & Petrinovich, L. (1984). Social interaction, sensitive phases and the song template hypothesis in the white-crowned sparrow. Animal Behaviour, 32, 172–181.

    Article  Google Scholar 

  • Baptista, L. F., & Petrinovich, L. (1986). Song development in the white-crowned sparrow: Social factors and sex differences. Animal Behaviour, 34, 1359–1371.

    Article  Google Scholar 

  • Bugelski, R. (1938). Extinction with and without sub-goal reinforcement. Journal of Comparative Psychology, 26, 121–134.

    Article  Google Scholar 

  • Case, D. A., Fantino, E., & Wixted, J. (1985). Human observing: Maintained by negative informative stimuli only if correlated with improvement in response efficiency. Journal of the Experimental Analysis of Behavior, 43, 285–300.

    Article  Google Scholar 

  • Catania, A.C. (1992). Learning (3rd ed.). Englewood Cliffs, NJ: Prentice-Hall.

    Google Scholar 

  • Chance, P. (1994). Learning & behavior (3rd ed.). Pacific Grove, CA: Brooks/Cole.

    Google Scholar 

  • Cohen, S. L., Calisto, G., & Lentz, B. E. (1979). Separating the reinforcing and discriminative properties of brief-stimulus presentations in second-order schedules. Journal of the Experimental Analysis of Behavior, 32, 149–156.

    Article  PubMed  PubMed Central  Google Scholar 

  • Cowles, J. T. (1937). Food tokens as incentives for learning by chimpanzees. Comparative Psychology Monographs, 14, 1–96.

    Google Scholar 

  • Cronin, P. B. (1980). Reinstatement of postresponse stimuli prior to reward in delayed-reward discrimination learning by pigeons. Animal Learning & Behavior, 8, 352–358.

    Article  Google Scholar 

  • Daly, H. B. (1985). Observing response acquisition: Preference for unpredictable appetitive rewards obtained under conditions predicted by DMOD. Journal of Experimental Psychology: Animal Behavior Processes, 11, 294–316.

    PubMed  Google Scholar 

  • Daly, H. B. (1989). Preference for unpredictable food rewards occurs with high proportion of reinforced trials or alcohol if rewards are not delayed. Journal of Experimental Psychology: Animal Behavior Processes, 15, 3–13.

    PubMed  Google Scholar 

  • Davison, M. C., & McCarthy, D. (1988). The matching law. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Dinsmoor, J. A. (1983). Observing and conditioned reinforcement. Behavioral and Brain Sciences, 6, 693–728.

    Article  Google Scholar 

  • Dollard, J., & Miller, N. E. (1950). Personality and psychotherapy: An analysis in terms of learning, thinking and culture. New York: McGraw-Hill.

    Google Scholar 

  • Everitt, B. J., Fray, P., Kostarczyk, R., Taylor, S., & Stacy, P. (1987). Studies of instrumental behavior with sexual reinforcement in male rats (Rattus norvegicus): I. Control by brief visual stimuli paired with a receptive female. Journal of Comparative Psychology, 101, 395–406.

    Article  PubMed  Google Scholar 

  • Fantino, E. (1969). Choice and rate of reinforcement. Journal of the Experimental Analysis of Behavior, 12, 723–730.

    Article  PubMed  PubMed Central  Google Scholar 

  • Fantino, E. (1977). Conditioned reinforcement: Choice and information. In W. K. Honig & J. E.R. Staddon (Eds.), Handbook of operant behavior (pp. 313–339). Englewood Cliffs, NJ: Prentice-Hall.

    Google Scholar 

  • Fantino, E., & Dunn, R. (1983). The delay-reduction hypothesis: Extension to three-alternative choice. Journal of Experimental Psychology: Animal Behavior Processes, 9, 132–146.

    Google Scholar 

  • Ferster, C. B. (1972). The experimental analysis of clinical phenomena. Psychological Record, 22, 1–16.

    Google Scholar 

  • Gibbon, J., & Balsam, P. (1981). Spreading association in time. In C. M. Locurto, H. S. Terrace, & J. Gibbon (Eds.), Autoshaping and conditioning theory (pp. 219–253). New York: Academic Press.

    Google Scholar 

  • Gollub, L. R. (1958). The chaining of fixed-interval schedules. Unpublished doctoral dissertation, Harvard University.

  • Gollub, L. R. (1970). Information on conditioned reinforcement: A review of Conditioned Reinforcement edited by D. Hendry. Journal of the Experimental Analysis of Behavior, 14, 361–372.

    Article  PubMed Central  Google Scholar 

  • Gollub, L. (1977). Conditioned reinforcement: Schedule effects. In W. K. Honig & J. E. R. Staddon (Eds.), Handbook of operant behavior (pp. 288–312). Englewood Cliffs, NJ: Prentice-Hall.

    Google Scholar 

  • Grice, G. R. (1948). The relation of secondary reinforcement to delayed reward in visual discrimination learning. Journal of Experimental Psychology, 38, 1–16.

    Article  PubMed  Google Scholar 

  • Herrnstein, R. J., & Loveland, D. H. (1972). Food-avoidance in hungry pigeons and other perplexities. Journal of the Experimental Analysis of Behavior, 18, 369–383.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hoffman, H. S., & Ratner, A. M. (1973). A reinforcement model of imprinting: Implications for socialization in monkeys and man. Psychological Review, 80, 527–546.

    Article  Google Scholar 

  • Honey, R. C., Schachtman, T. R., & Hall, G. (1987). Partial reinforcement in serial autoshaping: The role of attentional and associative factors. Learning and Motivation, 18, 288–300.

    Article  Google Scholar 

  • Hyde, T. S. (1976). The effect of Pavlovian stimuli in the acquisition of a new response. Learning and Motivation, 7, 223–239.

    Article  Google Scholar 

  • Jensen, G. D. (1963). Preference for bar pressing over “freeloading” as a function of number of rewarded presses. Journal of Experimental Psychology, 65, 451–454.

    Article  PubMed  Google Scholar 

  • Katz, J. L. (1979). A comparison of responding maintained under second-order schedules of intramuscular cocaine injection or food presentation in squirrel monkeys. Journal of the Experimental Analysis of Behavior, 32, 419–431.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kazdin, A. E. (1977). The token economy: A review and evaluation. New York: Plenum.

    Book  Google Scholar 

  • Kazdin, A. E. (1985). The token economy. In R. M. Turner & L. M. Ascher (Eds.), Evaluating behavior therapy outcome. New York: Springer.

    Google Scholar 

  • Kelleher, R. T. (1958). Fixed-ratio schedules of conditioned reinforcement with chimpanzees. Journal of the Experimental Analysis of Behavior, 1, 281–289.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kelleher, R. T. (1966). Conditioned reinforcement in second-order schedules. Journal of the Experimental Analysis of Behavior, 9, 475–485.

    Article  PubMed  PubMed Central  Google Scholar 

  • Keller, F. S., & Schoenfeld, W. N. (1950). Principles of psychology. New York: Appleton-Century-Crofts.

    Google Scholar 

  • Killeen, P. (1968). On the measure of reinforcement frequency in the study of preference. Journal of the Experimental Analysis of Behavior, 11, 263–269.

    Article  PubMed  PubMed Central  Google Scholar 

  • Killeen, P. R. (1981). Learning as causal inference. In M. L. Commons & J. A. Nevin (Eds.), Quantitative analyses of behavior: Discriminative properties of reinforcement schedules (pp. 89–112). Cambridge, MA: Ballinger.

    Google Scholar 

  • Kimble, G. A. (1961). Hilgard and Marquis’ conditioning and learning. Englewood Cliffs, NJ: Prentice-Hall.

    Google Scholar 

  • Klein, R. M. (1959). Intermittent primary reinforcement as a parameter of secondary reinforcement. Journal of Experimental Psychology, 58, 423–427.

    Article  PubMed  Google Scholar 

  • Levis, D. J. (1989). The case for a return to a two-factor theory of avoidance: The failure of non-fear interpretations. In S. B. Klein & R. R. Mowrer (Eds.), Contemporary learning theories: Pavlovian conditioning and the status of traditional learning theory (pp. 227–277). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Lieberman, D. A., Davidson, F. H., & Thomas, G. V. (1985). Marking in pigeons: The role of memory in delayed reinforcement. Journal of Experimental Psychology: Animal Behavior Processes, 11, 611–624.

    Google Scholar 

  • Lieberman, D. A., McIntosh, D. C., & Thomas, G. V. (1979). Learning when reward is delayed: A marking hypothesis. Journal of Experimental Psychology: Animal Behavior Processes, 5, 224–242.

    PubMed  Google Scholar 

  • Luck, S., Colgrove, M., & Neuringer, A. (1988). Response sequence learning as a function of primary versus conditioned reinforcement. Animal Learning & Behavior, 16, 8–14.

    Article  Google Scholar 

  • Mann, N. I., Slater, P. J. B., Eales, L. A., & Richards, C. (1991). The influence of visual stimuli on song-tutor choice in the zebra finch, Taeniopygia guttata. Animal Behaviour, 42, 285–293.

    Article  Google Scholar 

  • Martin, G., & Pear, J. (1992). Behavior modification: What it is and how to do it (4th ed.). Englewood Cliffs, NJ: Prentice-Hall.

    Google Scholar 

  • Mazur, J. E. (1986). Choice between single and multiple delayed reinforcers. Journal of the Experimental Analysis of Behavior, 46, 67–77.

    Article  PubMed  PubMed Central  Google Scholar 

  • Mazur, J. E. (1993). Predicting the strength of a conditioned reinforcer: Effects of delay and uncertainty. Current Directions in Psychological Science, 2, 70–74.

    Article  Google Scholar 

  • Mowrer, O. H. (1950). Learning theory and personality dynamics. New York: The Ronald Press.

    Google Scholar 

  • Mowrer, O. H. (1960). Learning theory and the symbolic processes. New York: Wiley.

    Book  Google Scholar 

  • Neuringer, A. J. (1969). Animals respond for food in the presence of free food. Science, 166, 399–401.

    Article  PubMed  Google Scholar 

  • Neuringer, A., & Nueringer, M. (1974). Learning by following a food source. Science, 184, 1005–1008.

    Article  PubMed  Google Scholar 

  • Nevin, J. A. (1973). Conditioned reinforcement. In J. A. Nevin & G. S. Reynolds (Eds.), The study of behavior (pp. 154–198). Glenview, IL: Scott-Foresman.

    Google Scholar 

  • Osborne, S. R. (1977). The free food (contrafree-loading) phenomenon: A review and analysis. Animal Learning & Behavior, 5, 221–235.

    Article  Google Scholar 

  • Pavlov, I. P. (1927). Conditioned reflexes (G. V. Anrep, Trans.). London: Oxford University Press.

    Google Scholar 

  • Preston, R. A., & Fantino, E. (1991). Conditioned reinforcement value and choice. Journal of the Experimental Analysis of Behavior, 55, 155–175.

    Article  PubMed  PubMed Central  Google Scholar 

  • Rachlin, H. (1976). Behavior and learning. San Francisco: Freeman.

    Google Scholar 

  • Rachlin, H. C., Battalio, R., Kagel, J., & Green, L. (1981). Maximization theory in behavioral psychology. Behavioral and Brain Sciences, 4, 371–417.

    Article  Google Scholar 

  • Rashotte, M. E., Marshall, B. S., & O’Connell, J. M. (1981). Signaling functions of the second-order CS: Partial reinforcement during second-order conditioning of the pigeon’s keypeck. Animal Learning & Behavior, 9, 253–260.

    Article  Google Scholar 

  • Rescorla, R. A. (1982). Effect of a stimulus intervening between CS and US in autoshaping. Journal of Experimental Psychology: Animal Behavior Processes, 8, 131–141.

    PubMed  Google Scholar 

  • Richards, R. W. (1981). A comparison of signaled and unsignaled delay of reinforcement. Journal of the Experimental Analysis of Behavior, 35, 145–152.

    Article  PubMed  PubMed Central  Google Scholar 

  • Royalty, P., Williams, B. A., & Fantino, E. (1987). Effects of delayed conditioned reinforcement in chain schedules. Journal of the Experimental Analysis of Behavior, 47, 41–56.

    Article  PubMed  PubMed Central  Google Scholar 

  • Schaal, D. W., & Branch, M. N. (1988). Responding of pigeons under variable-interval schedules of unsignaled, briefly signaled, and completely signaled delays to reinforcement. Journal of the Experimental Analysis of Behavior, 50, 33–54.

    Article  PubMed  PubMed Central  Google Scholar 

  • Schaal, D. W., & Branch, M. N. (1990). Responding of pigeons under variable-interval schedules of signaled-delayed reinforcement: Effects of delay-signal duration. Journal of the Experimental Analysis of Behavior, 53, 103–121.

    Article  PubMed  PubMed Central  Google Scholar 

  • Schuster, R. H. (1969). A functional analysis of conditioned reinforcement. In D. Hendry (Ed.), Conditioned reinforcement (pp. 192–234). Homewood, IL: Dorsey Press.

    Google Scholar 

  • Shull, R. L., Mellon, R. C., & Sharp, J. A. (1990). Delay and number of food reinforcers: Effect on choice and latencies. Journal of the Experimental Analysis of Behavior, 53, 235–246.

    Article  PubMed  PubMed Central  Google Scholar 

  • Shull, R. L., Spear, D. J., & Bryson, A. E. (1981). Delay or rate of food delivery as a determiner of response rate. Journal of the Experimental Analysis of Behavior, 35, 129–143.

    Article  PubMed  PubMed Central  Google Scholar 

  • Skinner, B. F. (1938). The behavior of organisms. New York: Appleton-Century-Crofts.

    Google Scholar 

  • Skinner, B. F. (1951). How to teach animals. Scientific American, 185, 26–29.

    Article  Google Scholar 

  • Skinner, B. F. (1953). Science and human behavior. New York: Free Press.

    Google Scholar 

  • Staddon, J. E. R. (1983). Adaptive learning and behavior. Cambridge: Cambridge University Press.

    Google Scholar 

  • Staddon, J. E. R., & Ettinger, R. H. (1989). Learning: An introduction to the principles of adaptive behavior. San Diego: Harcourt Brace Jovanovich.

    Google Scholar 

  • Stubbs, A. (1971). Second-order schedules and the problem of conditioned reinforcement. Journal of the Experimental Analysis of Behavior, 16, 289–313.

    Article  PubMed  PubMed Central  Google Scholar 

  • Thomas, G. V., Robertson, D., & Cunniffe, G. (1989). Facilitation of responding in a filled-delay trace autoshaping procedure: An occasion-setting hypothesis. Learning and Motivation, 20, 225–241.

    Article  Google Scholar 

  • Wike, E. L. (1966). Secondary reinforcement: Selected experiments. New York: Harper & Row.

    Google Scholar 

  • Williams, B. A. (1976). The effects of unsignalled delayed reinforcement. Journal of the Experimental Analysis of Behavior, 26, 441–449.

    Article  PubMed  PubMed Central  Google Scholar 

  • Williams, B. A. (1988). Reinforcement, choice, and response strength. In R. C. Atkinson, R. J. Herrnstein, G. Lindzey, & R. D. Luce (Eds.), Stevens’ handbook of experimental psychology (2nd ed., pp. 167–244). New York: Wiley.

    Google Scholar 

  • Williams, B. A. (1989). Partial reinforcement effects on discrimination learning. Animal Learning & Behavior, 17, 418–432.

    Article  Google Scholar 

  • Williams, B. A. (1991). Marking and bridging versus conditioned reinforcement. Animal Learning & Behavior, 19, 264–269.

    Article  Google Scholar 

  • Williams, B. A., & Dunn, R. (1991a). Preference for conditioned reinforcement. Journal of the Experimental Analysis of Behavior, 55, 37–46.

    Article  PubMed  PubMed Central  Google Scholar 

  • Williams, B. A., & Dunn, R. (1991b). Substitutability between conditioned and primary reinforcers in discrimination acquisition. Journal of the Experimental Analysis of Behavior, 55, 21–35.

    Article  PubMed  PubMed Central  Google Scholar 

  • Williams, B. A., & Royalty, P. (1990). Conditioned reinforcement versus time to reinforcement in chain schedules. Journal of the Experimental Analysis of Behavior, 53, 381–393.

    Article  PubMed  PubMed Central  Google Scholar 

  • Wolfe, J. B. (1936). Effectiveness of token rewards for chimpanzees. Comparative Psychology Monographs, 12, 1–72, No. 60.

    Google Scholar 

  • Wyckoff, L. B., Jr. (1952). The role of observing responses in discrimination learning. Psychological Review, 59, 431–442.

    Article  PubMed  Google Scholar 

  • Zimmerman, J., Hanford, P. V., & Brown, W. (1967). Effects of conditioned reinforcement frequency in an intermittent free-feeding situation. Journal of the Experimental Analysis of Behavior, 10, 331–340.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Psychology Department, UCSD, La Jolla, CA, 92093-0109, USA

    Ben A. Williams

Authors
  1. Ben A. Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Preparation of this paper was supported by grants from the National Science Foundation.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Williams, B.A. Conditioned Reinforcement: Experimental and Theoretical Issues. BEHAV ANALYST 17, 261–285 (1994). https://doi.org/10.1007/BF03392675

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF03392675

Key words

Search

Navigation