Abstract
The concept of conditioned reinforcement has received decreased attention in learning textbooks over the past decade, in part because of criticisms of its validity by major behavior theorists and in part because its explanatory function in a variety of different conditioning procedures has become uncertain. Critical data from the major procedures that have been used to investigate the concept (second-order schedules, chain schedules, concurrent chains, observing responses, delay-of-reinforcement procedures) are reviewed, along with the major issues of interpretation. Although the role played by conditioned reinforcement in some procedures remains unresolved, the results taken together leave little doubt that the underlying idea of conditioned value is a critical component of behavior theory that is necessary to explain many different types of data. Other processes (marking, bridging) may also operate to produce effects similar to those of conditioned reinforcement, but these clearly cannot explain the full domain of experimental effects ascribed to conditioned reinforcement and should be regarded as complements to the concept rather than theoretical competitors. Examples of practical and theoretical applications of the concept of conditioned reinforcement are also considered.
References
Amsel, A. (1992). Frustration theory: An analysis of dispositional learning and memory. Cambridge: Cambridge University Press.
Astley, S. L., & Perkins, C. C. (1985). Stimulus duration and conditioned reinforcing value measured by a learning-tests procedure. Animal Learning & Behavior, 13, 18–24.
Balsam, P., & Bondy, A. (1983). The negative side effects of reward. Journal of Applied Behavior Analysis, 16, 283–296.
Baptista, L. F., & Petrinovich, L. (1984). Social interaction, sensitive phases and the song template hypothesis in the white-crowned sparrow. Animal Behaviour, 32, 172–181.
Baptista, L. F., & Petrinovich, L. (1986). Song development in the white-crowned sparrow: Social factors and sex differences. Animal Behaviour, 34, 1359–1371.
Bugelski, R. (1938). Extinction with and without sub-goal reinforcement. Journal of Comparative Psychology, 26, 121–134.
Case, D. A., Fantino, E., & Wixted, J. (1985). Human observing: Maintained by negative informative stimuli only if correlated with improvement in response efficiency. Journal of the Experimental Analysis of Behavior, 43, 285–300.
Catania, A.C. (1992). Learning (3rd ed.). Englewood Cliffs, NJ: Prentice-Hall.
Chance, P. (1994). Learning & behavior (3rd ed.). Pacific Grove, CA: Brooks/Cole.
Cohen, S. L., Calisto, G., & Lentz, B. E. (1979). Separating the reinforcing and discriminative properties of brief-stimulus presentations in second-order schedules. Journal of the Experimental Analysis of Behavior, 32, 149–156.
Cowles, J. T. (1937). Food tokens as incentives for learning by chimpanzees. Comparative Psychology Monographs, 14, 1–96.
Cronin, P. B. (1980). Reinstatement of postresponse stimuli prior to reward in delayed-reward discrimination learning by pigeons. Animal Learning & Behavior, 8, 352–358.
Daly, H. B. (1985). Observing response acquisition: Preference for unpredictable appetitive rewards obtained under conditions predicted by DMOD. Journal of Experimental Psychology: Animal Behavior Processes, 11, 294–316.
Daly, H. B. (1989). Preference for unpredictable food rewards occurs with high proportion of reinforced trials or alcohol if rewards are not delayed. Journal of Experimental Psychology: Animal Behavior Processes, 15, 3–13.
Davison, M. C., & McCarthy, D. (1988). The matching law. Hillsdale, NJ: Erlbaum.
Dinsmoor, J. A. (1983). Observing and conditioned reinforcement. Behavioral and Brain Sciences, 6, 693–728.
Dollard, J., & Miller, N. E. (1950). Personality and psychotherapy: An analysis in terms of learning, thinking and culture. New York: McGraw-Hill.
Everitt, B. J., Fray, P., Kostarczyk, R., Taylor, S., & Stacy, P. (1987). Studies of instrumental behavior with sexual reinforcement in male rats (Rattus norvegicus): I. Control by brief visual stimuli paired with a receptive female. Journal of Comparative Psychology, 101, 395–406.
Fantino, E. (1969). Choice and rate of reinforcement. Journal of the Experimental Analysis of Behavior, 12, 723–730.
Fantino, E. (1977). Conditioned reinforcement: Choice and information. In W. K. Honig & J. E.R. Staddon (Eds.), Handbook of operant behavior (pp. 313–339). Englewood Cliffs, NJ: Prentice-Hall.
Fantino, E., & Dunn, R. (1983). The delay-reduction hypothesis: Extension to three-alternative choice. Journal of Experimental Psychology: Animal Behavior Processes, 9, 132–146.
Ferster, C. B. (1972). The experimental analysis of clinical phenomena. Psychological Record, 22, 1–16.
Gibbon, J., & Balsam, P. (1981). Spreading association in time. In C. M. Locurto, H. S. Terrace, & J. Gibbon (Eds.), Autoshaping and conditioning theory (pp. 219–253). New York: Academic Press.
Gollub, L. R. (1958). The chaining of fixed-interval schedules. Unpublished doctoral dissertation, Harvard University.
Gollub, L. R. (1970). Information on conditioned reinforcement: A review of Conditioned Reinforcement edited by D. Hendry. Journal of the Experimental Analysis of Behavior, 14, 361–372.
Gollub, L. (1977). Conditioned reinforcement: Schedule effects. In W. K. Honig & J. E. R. Staddon (Eds.), Handbook of operant behavior (pp. 288–312). Englewood Cliffs, NJ: Prentice-Hall.
Grice, G. R. (1948). The relation of secondary reinforcement to delayed reward in visual discrimination learning. Journal of Experimental Psychology, 38, 1–16.
Herrnstein, R. J., & Loveland, D. H. (1972). Food-avoidance in hungry pigeons and other perplexities. Journal of the Experimental Analysis of Behavior, 18, 369–383.
Hoffman, H. S., & Ratner, A. M. (1973). A reinforcement model of imprinting: Implications for socialization in monkeys and man. Psychological Review, 80, 527–546.
Honey, R. C., Schachtman, T. R., & Hall, G. (1987). Partial reinforcement in serial autoshaping: The role of attentional and associative factors. Learning and Motivation, 18, 288–300.
Hyde, T. S. (1976). The effect of Pavlovian stimuli in the acquisition of a new response. Learning and Motivation, 7, 223–239.
Jensen, G. D. (1963). Preference for bar pressing over “freeloading” as a function of number of rewarded presses. Journal of Experimental Psychology, 65, 451–454.
Katz, J. L. (1979). A comparison of responding maintained under second-order schedules of intramuscular cocaine injection or food presentation in squirrel monkeys. Journal of the Experimental Analysis of Behavior, 32, 419–431.
Kazdin, A. E. (1977). The token economy: A review and evaluation. New York: Plenum.
Kazdin, A. E. (1985). The token economy. In R. M. Turner & L. M. Ascher (Eds.), Evaluating behavior therapy outcome. New York: Springer.
Kelleher, R. T. (1958). Fixed-ratio schedules of conditioned reinforcement with chimpanzees. Journal of the Experimental Analysis of Behavior, 1, 281–289.
Kelleher, R. T. (1966). Conditioned reinforcement in second-order schedules. Journal of the Experimental Analysis of Behavior, 9, 475–485.
Keller, F. S., & Schoenfeld, W. N. (1950). Principles of psychology. New York: Appleton-Century-Crofts.
Killeen, P. (1968). On the measure of reinforcement frequency in the study of preference. Journal of the Experimental Analysis of Behavior, 11, 263–269.
Killeen, P. R. (1981). Learning as causal inference. In M. L. Commons & J. A. Nevin (Eds.), Quantitative analyses of behavior: Discriminative properties of reinforcement schedules (pp. 89–112). Cambridge, MA: Ballinger.
Kimble, G. A. (1961). Hilgard and Marquis’ conditioning and learning. Englewood Cliffs, NJ: Prentice-Hall.
Klein, R. M. (1959). Intermittent primary reinforcement as a parameter of secondary reinforcement. Journal of Experimental Psychology, 58, 423–427.
Levis, D. J. (1989). The case for a return to a two-factor theory of avoidance: The failure of non-fear interpretations. In S. B. Klein & R. R. Mowrer (Eds.), Contemporary learning theories: Pavlovian conditioning and the status of traditional learning theory (pp. 227–277). Hillsdale, NJ: Erlbaum.
Lieberman, D. A., Davidson, F. H., & Thomas, G. V. (1985). Marking in pigeons: The role of memory in delayed reinforcement. Journal of Experimental Psychology: Animal Behavior Processes, 11, 611–624.
Lieberman, D. A., McIntosh, D. C., & Thomas, G. V. (1979). Learning when reward is delayed: A marking hypothesis. Journal of Experimental Psychology: Animal Behavior Processes, 5, 224–242.
Luck, S., Colgrove, M., & Neuringer, A. (1988). Response sequence learning as a function of primary versus conditioned reinforcement. Animal Learning & Behavior, 16, 8–14.
Mann, N. I., Slater, P. J. B., Eales, L. A., & Richards, C. (1991). The influence of visual stimuli on song-tutor choice in the zebra finch, Taeniopygia guttata. Animal Behaviour, 42, 285–293.
Martin, G., & Pear, J. (1992). Behavior modification: What it is and how to do it (4th ed.). Englewood Cliffs, NJ: Prentice-Hall.
Mazur, J. E. (1986). Choice between single and multiple delayed reinforcers. Journal of the Experimental Analysis of Behavior, 46, 67–77.
Mazur, J. E. (1993). Predicting the strength of a conditioned reinforcer: Effects of delay and uncertainty. Current Directions in Psychological Science, 2, 70–74.
Mowrer, O. H. (1950). Learning theory and personality dynamics. New York: The Ronald Press.
Mowrer, O. H. (1960). Learning theory and the symbolic processes. New York: Wiley.
Neuringer, A. J. (1969). Animals respond for food in the presence of free food. Science, 166, 399–401.
Neuringer, A., & Nueringer, M. (1974). Learning by following a food source. Science, 184, 1005–1008.
Nevin, J. A. (1973). Conditioned reinforcement. In J. A. Nevin & G. S. Reynolds (Eds.), The study of behavior (pp. 154–198). Glenview, IL: Scott-Foresman.
Osborne, S. R. (1977). The free food (contrafree-loading) phenomenon: A review and analysis. Animal Learning & Behavior, 5, 221–235.
Pavlov, I. P. (1927). Conditioned reflexes (G. V. Anrep, Trans.). London: Oxford University Press.
Preston, R. A., & Fantino, E. (1991). Conditioned reinforcement value and choice. Journal of the Experimental Analysis of Behavior, 55, 155–175.
Rachlin, H. (1976). Behavior and learning. San Francisco: Freeman.
Rachlin, H. C., Battalio, R., Kagel, J., & Green, L. (1981). Maximization theory in behavioral psychology. Behavioral and Brain Sciences, 4, 371–417.
Rashotte, M. E., Marshall, B. S., & O’Connell, J. M. (1981). Signaling functions of the second-order CS: Partial reinforcement during second-order conditioning of the pigeon’s keypeck. Animal Learning & Behavior, 9, 253–260.
Rescorla, R. A. (1982). Effect of a stimulus intervening between CS and US in autoshaping. Journal of Experimental Psychology: Animal Behavior Processes, 8, 131–141.
Richards, R. W. (1981). A comparison of signaled and unsignaled delay of reinforcement. Journal of the Experimental Analysis of Behavior, 35, 145–152.
Royalty, P., Williams, B. A., & Fantino, E. (1987). Effects of delayed conditioned reinforcement in chain schedules. Journal of the Experimental Analysis of Behavior, 47, 41–56.
Schaal, D. W., & Branch, M. N. (1988). Responding of pigeons under variable-interval schedules of unsignaled, briefly signaled, and completely signaled delays to reinforcement. Journal of the Experimental Analysis of Behavior, 50, 33–54.
Schaal, D. W., & Branch, M. N. (1990). Responding of pigeons under variable-interval schedules of signaled-delayed reinforcement: Effects of delay-signal duration. Journal of the Experimental Analysis of Behavior, 53, 103–121.
Schuster, R. H. (1969). A functional analysis of conditioned reinforcement. In D. Hendry (Ed.), Conditioned reinforcement (pp. 192–234). Homewood, IL: Dorsey Press.
Shull, R. L., Mellon, R. C., & Sharp, J. A. (1990). Delay and number of food reinforcers: Effect on choice and latencies. Journal of the Experimental Analysis of Behavior, 53, 235–246.
Shull, R. L., Spear, D. J., & Bryson, A. E. (1981). Delay or rate of food delivery as a determiner of response rate. Journal of the Experimental Analysis of Behavior, 35, 129–143.
Skinner, B. F. (1938). The behavior of organisms. New York: Appleton-Century-Crofts.
Skinner, B. F. (1951). How to teach animals. Scientific American, 185, 26–29.
Skinner, B. F. (1953). Science and human behavior. New York: Free Press.
Staddon, J. E. R. (1983). Adaptive learning and behavior. Cambridge: Cambridge University Press.
Staddon, J. E. R., & Ettinger, R. H. (1989). Learning: An introduction to the principles of adaptive behavior. San Diego: Harcourt Brace Jovanovich.
Stubbs, A. (1971). Second-order schedules and the problem of conditioned reinforcement. Journal of the Experimental Analysis of Behavior, 16, 289–313.
Thomas, G. V., Robertson, D., & Cunniffe, G. (1989). Facilitation of responding in a filled-delay trace autoshaping procedure: An occasion-setting hypothesis. Learning and Motivation, 20, 225–241.
Wike, E. L. (1966). Secondary reinforcement: Selected experiments. New York: Harper & Row.
Williams, B. A. (1976). The effects of unsignalled delayed reinforcement. Journal of the Experimental Analysis of Behavior, 26, 441–449.
Williams, B. A. (1988). Reinforcement, choice, and response strength. In R. C. Atkinson, R. J. Herrnstein, G. Lindzey, & R. D. Luce (Eds.), Stevens’ handbook of experimental psychology (2nd ed., pp. 167–244). New York: Wiley.
Williams, B. A. (1989). Partial reinforcement effects on discrimination learning. Animal Learning & Behavior, 17, 418–432.
Williams, B. A. (1991). Marking and bridging versus conditioned reinforcement. Animal Learning & Behavior, 19, 264–269.
Williams, B. A., & Dunn, R. (1991a). Preference for conditioned reinforcement. Journal of the Experimental Analysis of Behavior, 55, 37–46.
Williams, B. A., & Dunn, R. (1991b). Substitutability between conditioned and primary reinforcers in discrimination acquisition. Journal of the Experimental Analysis of Behavior, 55, 21–35.
Williams, B. A., & Royalty, P. (1990). Conditioned reinforcement versus time to reinforcement in chain schedules. Journal of the Experimental Analysis of Behavior, 53, 381–393.
Wolfe, J. B. (1936). Effectiveness of token rewards for chimpanzees. Comparative Psychology Monographs, 12, 1–72, No. 60.
Wyckoff, L. B., Jr. (1952). The role of observing responses in discrimination learning. Psychological Review, 59, 431–442.
Zimmerman, J., Hanford, P. V., & Brown, W. (1967). Effects of conditioned reinforcement frequency in an intermittent free-feeding situation. Journal of the Experimental Analysis of Behavior, 10, 331–340.
Author information
Authors and Affiliations
Additional information
Preparation of this paper was supported by grants from the National Science Foundation.
Rights and permissions
About this article
Cite this article
Williams, B.A. Conditioned Reinforcement: Experimental and Theoretical Issues. BEHAV ANALYST 17, 261–285 (1994). https://doi.org/10.1007/BF03392675
Published:
Issue Date:
DOI: https://doi.org/10.1007/BF03392675