Advertisement

The Behavior Analyst

, Volume 17, Issue 2, pp 261–285 | Cite as

Conditioned Reinforcement: Experimental and Theoretical Issues

  • Ben A. Williams
Tutorial

Abstract

The concept of conditioned reinforcement has received decreased attention in learning textbooks over the past decade, in part because of criticisms of its validity by major behavior theorists and in part because its explanatory function in a variety of different conditioning procedures has become uncertain. Critical data from the major procedures that have been used to investigate the concept (second-order schedules, chain schedules, concurrent chains, observing responses, delay-of-reinforcement procedures) are reviewed, along with the major issues of interpretation. Although the role played by conditioned reinforcement in some procedures remains unresolved, the results taken together leave little doubt that the underlying idea of conditioned value is a critical component of behavior theory that is necessary to explain many different types of data. Other processes (marking, bridging) may also operate to produce effects similar to those of conditioned reinforcement, but these clearly cannot explain the full domain of experimental effects ascribed to conditioned reinforcement and should be regarded as complements to the concept rather than theoretical competitors. Examples of practical and theoretical applications of the concept of conditioned reinforcement are also considered.

Key words

conditioned reinforcement behavior theory observing behavior chain schedules delay of reinforcement concurrent chains 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Amsel, A. (1992). Frustration theory: An analysis of dispositional learning and memory. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  2. Astley, S. L., & Perkins, C. C. (1985). Stimulus duration and conditioned reinforcing value measured by a learning-tests procedure. Animal Learning & Behavior, 13, 18–24.CrossRefGoogle Scholar
  3. Balsam, P., & Bondy, A. (1983). The negative side effects of reward. Journal of Applied Behavior Analysis, 16, 283–296.CrossRefPubMedPubMedCentralGoogle Scholar
  4. Baptista, L. F., & Petrinovich, L. (1984). Social interaction, sensitive phases and the song template hypothesis in the white-crowned sparrow. Animal Behaviour, 32, 172–181.CrossRefGoogle Scholar
  5. Baptista, L. F., & Petrinovich, L. (1986). Song development in the white-crowned sparrow: Social factors and sex differences. Animal Behaviour, 34, 1359–1371.CrossRefGoogle Scholar
  6. Bugelski, R. (1938). Extinction with and without sub-goal reinforcement. Journal of Comparative Psychology, 26, 121–134.CrossRefGoogle Scholar
  7. Case, D. A., Fantino, E., & Wixted, J. (1985). Human observing: Maintained by negative informative stimuli only if correlated with improvement in response efficiency. Journal of the Experimental Analysis of Behavior, 43, 285–300.CrossRefGoogle Scholar
  8. Catania, A.C. (1992). Learning (3rd ed.). Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
  9. Chance, P. (1994). Learning & behavior (3rd ed.). Pacific Grove, CA: Brooks/Cole.Google Scholar
  10. Cohen, S. L., Calisto, G., & Lentz, B. E. (1979). Separating the reinforcing and discriminative properties of brief-stimulus presentations in second-order schedules. Journal of the Experimental Analysis of Behavior, 32, 149–156.CrossRefPubMedPubMedCentralGoogle Scholar
  11. Cowles, J. T. (1937). Food tokens as incentives for learning by chimpanzees. Comparative Psychology Monographs, 14, 1–96.Google Scholar
  12. Cronin, P. B. (1980). Reinstatement of postresponse stimuli prior to reward in delayed-reward discrimination learning by pigeons. Animal Learning & Behavior, 8, 352–358.CrossRefGoogle Scholar
  13. Daly, H. B. (1985). Observing response acquisition: Preference for unpredictable appetitive rewards obtained under conditions predicted by DMOD. Journal of Experimental Psychology: Animal Behavior Processes, 11, 294–316.PubMedGoogle Scholar
  14. Daly, H. B. (1989). Preference for unpredictable food rewards occurs with high proportion of reinforced trials or alcohol if rewards are not delayed. Journal of Experimental Psychology: Animal Behavior Processes, 15, 3–13.PubMedGoogle Scholar
  15. Davison, M. C., & McCarthy, D. (1988). The matching law. Hillsdale, NJ: Erlbaum.Google Scholar
  16. Dinsmoor, J. A. (1983). Observing and conditioned reinforcement. Behavioral and Brain Sciences, 6, 693–728.CrossRefGoogle Scholar
  17. Dollard, J., & Miller, N. E. (1950). Personality and psychotherapy: An analysis in terms of learning, thinking and culture. New York: McGraw-Hill.Google Scholar
  18. Everitt, B. J., Fray, P., Kostarczyk, R., Taylor, S., & Stacy, P. (1987). Studies of instrumental behavior with sexual reinforcement in male rats (Rattus norvegicus): I. Control by brief visual stimuli paired with a receptive female. Journal of Comparative Psychology, 101, 395–406.CrossRefPubMedGoogle Scholar
  19. Fantino, E. (1969). Choice and rate of reinforcement. Journal of the Experimental Analysis of Behavior, 12, 723–730.CrossRefPubMedPubMedCentralGoogle Scholar
  20. Fantino, E. (1977). Conditioned reinforcement: Choice and information. In W. K. Honig & J. E.R. Staddon (Eds.), Handbook of operant behavior (pp. 313–339). Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
  21. Fantino, E., & Dunn, R. (1983). The delay-reduction hypothesis: Extension to three-alternative choice. Journal of Experimental Psychology: Animal Behavior Processes, 9, 132–146.Google Scholar
  22. Ferster, C. B. (1972). The experimental analysis of clinical phenomena. Psychological Record, 22, 1–16.Google Scholar
  23. Gibbon, J., & Balsam, P. (1981). Spreading association in time. In C. M. Locurto, H. S. Terrace, & J. Gibbon (Eds.), Autoshaping and conditioning theory (pp. 219–253). New York: Academic Press.Google Scholar
  24. Gollub, L. R. (1958). The chaining of fixed-interval schedules. Unpublished doctoral dissertation, Harvard University.Google Scholar
  25. Gollub, L. R. (1970). Information on conditioned reinforcement: A review of Conditioned Reinforcement edited by D. Hendry. Journal of the Experimental Analysis of Behavior, 14, 361–372.CrossRefPubMedCentralGoogle Scholar
  26. Gollub, L. (1977). Conditioned reinforcement: Schedule effects. In W. K. Honig & J. E. R. Staddon (Eds.), Handbook of operant behavior (pp. 288–312). Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
  27. Grice, G. R. (1948). The relation of secondary reinforcement to delayed reward in visual discrimination learning. Journal of Experimental Psychology, 38, 1–16.CrossRefPubMedGoogle Scholar
  28. Herrnstein, R. J., & Loveland, D. H. (1972). Food-avoidance in hungry pigeons and other perplexities. Journal of the Experimental Analysis of Behavior, 18, 369–383.CrossRefPubMedPubMedCentralGoogle Scholar
  29. Hoffman, H. S., & Ratner, A. M. (1973). A reinforcement model of imprinting: Implications for socialization in monkeys and man. Psychological Review, 80, 527–546.CrossRefGoogle Scholar
  30. Honey, R. C., Schachtman, T. R., & Hall, G. (1987). Partial reinforcement in serial autoshaping: The role of attentional and associative factors. Learning and Motivation, 18, 288–300.CrossRefGoogle Scholar
  31. Hyde, T. S. (1976). The effect of Pavlovian stimuli in the acquisition of a new response. Learning and Motivation, 7, 223–239.CrossRefGoogle Scholar
  32. Jensen, G. D. (1963). Preference for bar pressing over “freeloading” as a function of number of rewarded presses. Journal of Experimental Psychology, 65, 451–454.CrossRefPubMedGoogle Scholar
  33. Katz, J. L. (1979). A comparison of responding maintained under second-order schedules of intramuscular cocaine injection or food presentation in squirrel monkeys. Journal of the Experimental Analysis of Behavior, 32, 419–431.CrossRefPubMedPubMedCentralGoogle Scholar
  34. Kazdin, A. E. (1977). The token economy: A review and evaluation. New York: Plenum.CrossRefGoogle Scholar
  35. Kazdin, A. E. (1985). The token economy. In R. M. Turner & L. M. Ascher (Eds.), Evaluating behavior therapy outcome. New York: Springer.Google Scholar
  36. Kelleher, R. T. (1958). Fixed-ratio schedules of conditioned reinforcement with chimpanzees. Journal of the Experimental Analysis of Behavior, 1, 281–289.CrossRefPubMedPubMedCentralGoogle Scholar
  37. Kelleher, R. T. (1966). Conditioned reinforcement in second-order schedules. Journal of the Experimental Analysis of Behavior, 9, 475–485.CrossRefPubMedPubMedCentralGoogle Scholar
  38. Keller, F. S., & Schoenfeld, W. N. (1950). Principles of psychology. New York: Appleton-Century-Crofts.Google Scholar
  39. Killeen, P. (1968). On the measure of reinforcement frequency in the study of preference. Journal of the Experimental Analysis of Behavior, 11, 263–269.CrossRefPubMedPubMedCentralGoogle Scholar
  40. Killeen, P. R. (1981). Learning as causal inference. In M. L. Commons & J. A. Nevin (Eds.), Quantitative analyses of behavior: Discriminative properties of reinforcement schedules (pp. 89–112). Cambridge, MA: Ballinger.Google Scholar
  41. Kimble, G. A. (1961). Hilgard and Marquis’ conditioning and learning. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
  42. Klein, R. M. (1959). Intermittent primary reinforcement as a parameter of secondary reinforcement. Journal of Experimental Psychology, 58, 423–427.CrossRefPubMedGoogle Scholar
  43. Levis, D. J. (1989). The case for a return to a two-factor theory of avoidance: The failure of non-fear interpretations. In S. B. Klein & R. R. Mowrer (Eds.), Contemporary learning theories: Pavlovian conditioning and the status of traditional learning theory (pp. 227–277). Hillsdale, NJ: Erlbaum.Google Scholar
  44. Lieberman, D. A., Davidson, F. H., & Thomas, G. V. (1985). Marking in pigeons: The role of memory in delayed reinforcement. Journal of Experimental Psychology: Animal Behavior Processes, 11, 611–624.Google Scholar
  45. Lieberman, D. A., McIntosh, D. C., & Thomas, G. V. (1979). Learning when reward is delayed: A marking hypothesis. Journal of Experimental Psychology: Animal Behavior Processes, 5, 224–242.PubMedGoogle Scholar
  46. Luck, S., Colgrove, M., & Neuringer, A. (1988). Response sequence learning as a function of primary versus conditioned reinforcement. Animal Learning & Behavior, 16, 8–14.CrossRefGoogle Scholar
  47. Mann, N. I., Slater, P. J. B., Eales, L. A., & Richards, C. (1991). The influence of visual stimuli on song-tutor choice in the zebra finch, Taeniopygia guttata. Animal Behaviour, 42, 285–293.CrossRefGoogle Scholar
  48. Martin, G., & Pear, J. (1992). Behavior modification: What it is and how to do it (4th ed.). Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
  49. Mazur, J. E. (1986). Choice between single and multiple delayed reinforcers. Journal of the Experimental Analysis of Behavior, 46, 67–77.CrossRefPubMedPubMedCentralGoogle Scholar
  50. Mazur, J. E. (1993). Predicting the strength of a conditioned reinforcer: Effects of delay and uncertainty. Current Directions in Psychological Science, 2, 70–74.CrossRefGoogle Scholar
  51. Mowrer, O. H. (1950). Learning theory and personality dynamics. New York: The Ronald Press.Google Scholar
  52. Mowrer, O. H. (1960). Learning theory and the symbolic processes. New York: Wiley.CrossRefGoogle Scholar
  53. Neuringer, A. J. (1969). Animals respond for food in the presence of free food. Science, 166, 399–401.CrossRefPubMedGoogle Scholar
  54. Neuringer, A., & Nueringer, M. (1974). Learning by following a food source. Science, 184, 1005–1008.CrossRefPubMedGoogle Scholar
  55. Nevin, J. A. (1973). Conditioned reinforcement. In J. A. Nevin & G. S. Reynolds (Eds.), The study of behavior (pp. 154–198). Glenview, IL: Scott-Foresman.Google Scholar
  56. Osborne, S. R. (1977). The free food (contrafree-loading) phenomenon: A review and analysis. Animal Learning & Behavior, 5, 221–235.CrossRefGoogle Scholar
  57. Pavlov, I. P. (1927). Conditioned reflexes (G. V. Anrep, Trans.). London: Oxford University Press.Google Scholar
  58. Preston, R. A., & Fantino, E. (1991). Conditioned reinforcement value and choice. Journal of the Experimental Analysis of Behavior, 55, 155–175.CrossRefPubMedPubMedCentralGoogle Scholar
  59. Rachlin, H. (1976). Behavior and learning. San Francisco: Freeman.Google Scholar
  60. Rachlin, H. C., Battalio, R., Kagel, J., & Green, L. (1981). Maximization theory in behavioral psychology. Behavioral and Brain Sciences, 4, 371–417.CrossRefGoogle Scholar
  61. Rashotte, M. E., Marshall, B. S., & O’Connell, J. M. (1981). Signaling functions of the second-order CS: Partial reinforcement during second-order conditioning of the pigeon’s keypeck. Animal Learning & Behavior, 9, 253–260.CrossRefGoogle Scholar
  62. Rescorla, R. A. (1982). Effect of a stimulus intervening between CS and US in autoshaping. Journal of Experimental Psychology: Animal Behavior Processes, 8, 131–141.PubMedGoogle Scholar
  63. Richards, R. W. (1981). A comparison of signaled and unsignaled delay of reinforcement. Journal of the Experimental Analysis of Behavior, 35, 145–152.CrossRefPubMedPubMedCentralGoogle Scholar
  64. Royalty, P., Williams, B. A., & Fantino, E. (1987). Effects of delayed conditioned reinforcement in chain schedules. Journal of the Experimental Analysis of Behavior, 47, 41–56.CrossRefPubMedPubMedCentralGoogle Scholar
  65. Schaal, D. W., & Branch, M. N. (1988). Responding of pigeons under variable-interval schedules of unsignaled, briefly signaled, and completely signaled delays to reinforcement. Journal of the Experimental Analysis of Behavior, 50, 33–54.CrossRefPubMedPubMedCentralGoogle Scholar
  66. Schaal, D. W., & Branch, M. N. (1990). Responding of pigeons under variable-interval schedules of signaled-delayed reinforcement: Effects of delay-signal duration. Journal of the Experimental Analysis of Behavior, 53, 103–121.CrossRefPubMedPubMedCentralGoogle Scholar
  67. Schuster, R. H. (1969). A functional analysis of conditioned reinforcement. In D. Hendry (Ed.), Conditioned reinforcement (pp. 192–234). Homewood, IL: Dorsey Press.Google Scholar
  68. Shull, R. L., Mellon, R. C., & Sharp, J. A. (1990). Delay and number of food reinforcers: Effect on choice and latencies. Journal of the Experimental Analysis of Behavior, 53, 235–246.CrossRefPubMedPubMedCentralGoogle Scholar
  69. Shull, R. L., Spear, D. J., & Bryson, A. E. (1981). Delay or rate of food delivery as a determiner of response rate. Journal of the Experimental Analysis of Behavior, 35, 129–143.CrossRefPubMedPubMedCentralGoogle Scholar
  70. Skinner, B. F. (1938). The behavior of organisms. New York: Appleton-Century-Crofts.Google Scholar
  71. Skinner, B. F. (1951). How to teach animals. Scientific American, 185, 26–29.CrossRefGoogle Scholar
  72. Skinner, B. F. (1953). Science and human behavior. New York: Free Press.Google Scholar
  73. Staddon, J. E. R. (1983). Adaptive learning and behavior. Cambridge: Cambridge University Press.Google Scholar
  74. Staddon, J. E. R., & Ettinger, R. H. (1989). Learning: An introduction to the principles of adaptive behavior. San Diego: Harcourt Brace Jovanovich.Google Scholar
  75. Stubbs, A. (1971). Second-order schedules and the problem of conditioned reinforcement. Journal of the Experimental Analysis of Behavior, 16, 289–313.CrossRefPubMedPubMedCentralGoogle Scholar
  76. Thomas, G. V., Robertson, D., & Cunniffe, G. (1989). Facilitation of responding in a filled-delay trace autoshaping procedure: An occasion-setting hypothesis. Learning and Motivation, 20, 225–241.CrossRefGoogle Scholar
  77. Wike, E. L. (1966). Secondary reinforcement: Selected experiments. New York: Harper & Row.Google Scholar
  78. Williams, B. A. (1976). The effects of unsignalled delayed reinforcement. Journal of the Experimental Analysis of Behavior, 26, 441–449.CrossRefPubMedPubMedCentralGoogle Scholar
  79. Williams, B. A. (1988). Reinforcement, choice, and response strength. In R. C. Atkinson, R. J. Herrnstein, G. Lindzey, & R. D. Luce (Eds.), Stevens’ handbook of experimental psychology (2nd ed., pp. 167–244). New York: Wiley.Google Scholar
  80. Williams, B. A. (1989). Partial reinforcement effects on discrimination learning. Animal Learning & Behavior, 17, 418–432.CrossRefGoogle Scholar
  81. Williams, B. A. (1991). Marking and bridging versus conditioned reinforcement. Animal Learning & Behavior, 19, 264–269.CrossRefGoogle Scholar
  82. Williams, B. A., & Dunn, R. (1991a). Preference for conditioned reinforcement. Journal of the Experimental Analysis of Behavior, 55, 37–46.CrossRefPubMedPubMedCentralGoogle Scholar
  83. Williams, B. A., & Dunn, R. (1991b). Substitutability between conditioned and primary reinforcers in discrimination acquisition. Journal of the Experimental Analysis of Behavior, 55, 21–35.CrossRefPubMedPubMedCentralGoogle Scholar
  84. Williams, B. A., & Royalty, P. (1990). Conditioned reinforcement versus time to reinforcement in chain schedules. Journal of the Experimental Analysis of Behavior, 53, 381–393.CrossRefPubMedPubMedCentralGoogle Scholar
  85. Wolfe, J. B. (1936). Effectiveness of token rewards for chimpanzees. Comparative Psychology Monographs, 12, 1–72, No. 60.Google Scholar
  86. Wyckoff, L. B., Jr. (1952). The role of observing responses in discrimination learning. Psychological Review, 59, 431–442.CrossRefPubMedGoogle Scholar
  87. Zimmerman, J., Hanford, P. V., & Brown, W. (1967). Effects of conditioned reinforcement frequency in an intermittent free-feeding situation. Journal of the Experimental Analysis of Behavior, 10, 331–340.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Association for Behavior Analysis International 1994

Authors and Affiliations

  • Ben A. Williams
    • 1
  1. 1.Psychology DepartmentUCSDLa JollaUSA

Personalised recommendations