Animal Learning & Behavior

, Volume 22, Issue 1, pp 1–18 | Cite as

Motivational control of goal-directed action

  • Anthony Dickinson
  • Bernard Balleine
Article
  • 2.1k Downloads

Abstract

The control of goal-directed, instrumental actions by primary motivational states, such as hunger and thirst, is mediated by two processes. The first is engaged by the Pavlovian association between contextual or discriminative stimuli and the outcome or reinforcer presented during instrumental training. Such stimuli exert a motivational influence on instrumental performance that depends upon the relevance of the associated outcome to the current motivational state of the agent. Moreover, the motivational effects of these stimuli operate in the absence of prior experience with the outcome under the relevant motivational state. The second, instrumental, process is mediated by knowledge of the contingency between the action and its outcome and controls the value assigned to this outcome. In contrast to the Pavlovian process, motivational states do not influence the instrumental process directly; rather, the agent has to learn about the value of an outcome in a given motivational state by exposure to it while in that state. This incentive learning is similar in certain respects to the acquisition of “cathexes” envisaged by Tolman (1949a, 1949b).

References

  1. Adams, C. D., &Dickinson, A. (1981). Instrumental responding following reinforcer devaluation.Quarterly Journal of Experimental Psychology,33B, 109–122.Google Scholar
  2. Asratyan, E. A. (1974). Conditioned reflex theory and motivational behavior.Acta Neurobiologiae Experimentalis,34, 15–31.PubMedGoogle Scholar
  3. Baeyens, F., Eelen, P., van den Berg, H., &Crombez, G. (1990). Flavor-flavor and color-flavor conditioning in humans.Learning & Motivation,21, 434–455.CrossRefGoogle Scholar
  4. Balleine, B. (1992). Instrumental performance following a shift in primary motivation depends upon incentive learning.Journal of Experimental Psychology: Animal Behavior Processes,18, 236–250.PubMedCrossRefGoogle Scholar
  5. Balleine, B., &Dickinson, A. (1991). Instrumental performance following reinforcer devaluation depends upon incentive learning.Quarterly Journal of Experimental Psychology,43B, 279–296.Google Scholar
  6. Balleine, B., &Dickinson, A. (1992). Signalling and incentive processes in instrumental reinforcer devaluation.Quarterly Journal of Experimental Psychology,45B, 285–301.Google Scholar
  7. Berridge, K. C. (1991). Modulation of taste affect by hunger, caloric satiety, and sensory-specific satiety in the rat.Appetite,16, 103–120.PubMedCrossRefGoogle Scholar
  8. Berridge, K. C., Flynn, F. W., Schulkin, J., &Grill, H. J. (1984). Sodium depletion enhances salt palatability in rats.Behavioral Neuroscience 98, 652–660.PubMedCrossRefGoogle Scholar
  9. Berridge, K. C., Grill, H. J., &Norgren, R. (1981). Relation of consummatory responses and preabsorptive insulin release to palatability and learned taste aversions.Journal of Comparative & Physiological Psychology,95, 363–382.CrossRefGoogle Scholar
  10. Bindra, D. (1972). A unified account of classical conditioning and operant training. In A. H. Black & W. F. Prokasy (Eds.),Classical conditioning II: Current research and theory (pp. 453–482). New York: Appleton-Century-Crofts.Google Scholar
  11. Bindra, D. (1978). How adaptive behavior is produced: A perceptual motivational alternative to response-reinforcement.Behavior & Brain Sciences,1, 41–52.CrossRefGoogle Scholar
  12. Bolles, R. C. (1972). Reinforcement, expectancy, and learning.Psychological Review,95, 394–409.CrossRefGoogle Scholar
  13. Bolles, R. C. (1975).Theory of motivation. New York: Harper & Row.Google Scholar
  14. Bolles, R. C., Holtz, R., Dunn, T., &Hill, W. (1980). Comparison of stimulus learning and response learning in a punishment situation.Learning & Motivation,11, 78–96.CrossRefGoogle Scholar
  15. Cabanac, M. (1971). Physiological role of pleasure.Science,173, 1103–1107.PubMedCrossRefGoogle Scholar
  16. Campbell, D. H., Capaldi, E. D., Sheffer, J. D., &Bradford, J. P. (1988). An examination of the relationship between expectancy learning and preference conditioning.Learning & Motivation,19, 162–182.CrossRefGoogle Scholar
  17. Capaldi, E. D., Davidson, T. L., &Myers, D. E. (1981). Resistance to satiation: Reinforcing effects of food and eating under satiation.Learning & Motivation,12, 171–195.CrossRefGoogle Scholar
  18. Capaldi, E. D., &Myers, D. E. (1982). Taste preferences as a function of food deprivation during original taste exposure.Animal Learning & Behavior,10, 211–219.Google Scholar
  19. Capaldi, E. D., Myers, D. E., Campbell, D. H., &Sheffer, J. D. (1983). Conditioned flavor preferences based on hunger level during original flavor exposure.Animal Learning & Behavior,11, 107–115.Google Scholar
  20. Capaldi, E. D., Sheffer, J., &Owens, J. (1991). Food deprivation and conditioned flavor preferences based on sweetened and unsweetened foods.Animal Learning & Behavior,19, 361–368.CrossRefGoogle Scholar
  21. Capaldi, E. D., Viveiros, D. M., &Davidson, T. L. (1981). Deprivation stimulus intensity and incentive factors in the control of instrumental performance.Journal of Experimental Psychology: Animal Behavior Processes,7, 140–149.CrossRefGoogle Scholar
  22. Colwill, R. C., &Rescorla, R. A. (1985). Postconditioning devaluation of a reinforcer affects instrumental responding.Journal of Experimental Psychology: Animal Behavior Processes,11, 120–132.CrossRefGoogle Scholar
  23. Colwill, R. C., &Rescorla, R. A., (1986). Associative structures in instrumental learning. In G. H. Bower (Ed.),The psychology of learning and motivation (Vol. 20, pp. 55–104). New York: Academic Press.Google Scholar
  24. Crocetti, C. P. (1962). Drive level and response strength in the barpressing apparatus.Psychological Reports,10, 563–575.CrossRefGoogle Scholar
  25. Davidson, T. L. (1987). Learning about deprivational intensity stimuli.Behavioral Neuroscience,101, 198–208.PubMedCrossRefGoogle Scholar
  26. Davidson, T. L., Flynn, F. W., &Jarrard, L. E. (1992). Potency of food deprivation intensity cues as discriminative stimuli.Journal of Experimental Psychology: Animal Behavior Processes,18, 174–181.PubMedCrossRefGoogle Scholar
  27. Dickinson, A. (1980).Contemporary animal learning theory. Cambridge: Cambridge University Press.Google Scholar
  28. Dickinson, A. (1986). Re-examination of the role of the instrumental contingency in the sodium-appetite irrelevant incentive effect.Quarterly Journal of Experimental Psychology,38B, 161–172.Google Scholar
  29. Dickinson, A. (1989). Expectancy theory in animal conditioning. In S. B. Klein & R. R. Mowrer (Eds.),Contemporary learning theories: Pavlovian conditioning and the status of traditional learning theory (pp. 279–308). Hillsdale, NJ: Erlbaum.Google Scholar
  30. Dickinson, A., &Balleine, B. (1990). Motivational control of instrumental performance following a shift from hunger to thirst.Quarterly Journal of Experimental Psychology,42B, 413–431.Google Scholar
  31. Dickinson, A., &Dawson, G. R. (1987a). Pavlovian processes in the motivational control of instrumental performance.Quarterly Journal of Experimental Psychology,38B, 201–213.Google Scholar
  32. Dickinson, A., &Dawson, G. R. (1987b). The role of the instrumental contingency in the motivational control of performance.Quarterly Journal of Experimental Psychology,39B, 77–93.Google Scholar
  33. Dickinson, A., &Dawson, G. R. (1988). Motivational control of instrumental performance: The role of prior experience of the reinforcer.Quarterly Journal of Experimental Psychology,40B, 113–134.Google Scholar
  34. Dickinson, A., &Dawson, G. R. (1989). Incentive learning and the motivational control of instrumental performance.Quarterly Journal of Experimental Psychology,41B, 99–112.Google Scholar
  35. Dickinson, A., &Nicholas, D. J. (1983). Irrelevant incentive learning during instrumental conditioning: The role of drive-reinforcer and response-reinforcer relationships.Quarterly Journal of Experimental Psychology,35B, 249–263.Google Scholar
  36. Eisman, E., Theios, J., &Linton, M. (1961). Habit strength as a function of drive in a bar pressing situation.Psychological Reports,9, 583–590.CrossRefGoogle Scholar
  37. Estes, W. K. (1943). Discriminative conditioning: 1. A discriminative property of conditioned anticipation.Journal of Experimental Psychology,32, 150–155.CrossRefGoogle Scholar
  38. Everitt, B. J., &Stacey, P. (1987). Studies of instrumental behavior with sexual reinforcement in male rats (Rattus norvegicus): 2. Effects of preoptic area lesions, castration, and testosterone.Journal of Comparative Psychology,101, 407–419.PubMedCrossRefGoogle Scholar
  39. Fitzsimons, J. T. (1961). Drinking by nephrectomized rats injected with various substances.Journal of Physiology,155, 563–579.PubMedGoogle Scholar
  40. Garcia, J. (1989). Food for Tolman: Cognition and cathexis in concert. In T. Archer & L.-G. Nilsson (Eds.),Aversion, avoidance and anxiety (pp. 45–85). Hillsdale, NJ: Erlbaum.Google Scholar
  41. Gormezano, I., &Tait, R. W. (1976). The Pavlovian analysis of instrumental conditioning.Pavlovian Journal of Biological Sciences,11, 37–55.Google Scholar
  42. Grice, G. R., &Davis, J. D. (1957). Effect of irrelevant thirst motivation on a response learned with food reward.Journal of Experimental Psychology,53, 347–352.PubMedCrossRefGoogle Scholar
  43. Grill, H. J., &Berridge, K. C. (1985). Taste reactivity as a measure of the neural control of palatability. In J. M. Sprague & A. N. Epstein (Eds.),Progress in psychobiology and physiological psychology (Vol. 11, pp. 1–61). Orlando, FL: Academic Press.Google Scholar
  44. Grindley, G. C. (1932). The formation of a simple habit in guinea pigs.British Journal of Psychology,23, 127–147.Google Scholar
  45. Heathers, G. L., &Arakelian, P. (1941). The relationship between strength of drive and rate of extinction of a bar-pressing reaction in the rat.Journal of General Psychology,24, 243–258.Google Scholar
  46. Hendersen, R. W., &Graham, J. (1979). Avoidance of heat by rats: Effects of thermal context on the rapidity of extinction.Learning & Motivation,10, 351–363.CrossRefGoogle Scholar
  47. Hershberger, W. A. (1986). An approach through the looking-glass.Animal Learning & Behavior,14, 443–451.Google Scholar
  48. Horenstein, B. R. (1951). Performance of conditioned responses as a function of strength of hunger drive.Journal of Comparative & Physiological Psychology,44, 210–224.CrossRefGoogle Scholar
  49. Hovancik, J. R. (1978). The effect of deprivation level during noncontingent pairings and instrumental learning on subsequent instrumental performance.Learning & Motivation,9, 1–15.CrossRefGoogle Scholar
  50. Hull, C. L. (1943).Principles of behavior. New York: Appleton-Century-Crofts.Google Scholar
  51. Irwin, F. W. (1971).Intentional behavior and motivation: A cognitive theory. Philadelphia: Lippincott.Google Scholar
  52. Jenkins, H. M., &Harrison, R. H. (1960). The effects of discrimination training on auditory generalization.Journal of Experimental Psychology,59, 246–253.PubMedCrossRefGoogle Scholar
  53. King, D. L. (1979).Conditioning: An image approach. New York: Gardner.Google Scholar
  54. Koch, S., &Daniel, W. J. (1945). The effect of satiation on behavior mediated by a habit of maximum strength.Journal of Experimental Psychology,35, 167–187.CrossRefGoogle Scholar
  55. Konorski, J. (1967).Integrative activity of the brain: An interdisciplinary approach. Chicago: University of Chicago Press.Google Scholar
  56. Krieckhaus, E. E., &Wolf, G. (1968). Acquisition of sodium by rats: Interaction of innate mechanisms and latent learning.Journal of Comparative & Physiological Psychology,65, 197–201.CrossRefGoogle Scholar
  57. Lopez, M., Balleine, B., &Dickinson, A. (1992a). Incentive learning and the motivational control of instrumental performance by thirst.Animal Learning & Behavior,20, 322–328.CrossRefGoogle Scholar
  58. Lopez, M., Balleine, B., &Dickinson, A. (1992b). Incentive learning following reinforcer devaluation is not conditional upon motivational state during re-exposure.Quarterly Journal of Experimental Psychology,45B, 265–284.Google Scholar
  59. Meachum, C. L. (1990). The role of response-contingent incentives in lithium chloride-mediated suppression of an operant response.Quarterly Journal of Experimental Psychology,42B, 175–195.Google Scholar
  60. Miller, N. E. (1963). Some reflections on the law of effect produce a new alternative to drive reduction. In M. R. Jones (Ed.),Nebraska symposium on motivation (pp. 65–112). Lincoln: University of Nebraska Press.Google Scholar
  61. Miller, S., &Konorski, J. (1969). On a particular form of conditioned reflex.Journal of the Experimental Analysis of Behavior,12, 187–189. [Translation by B. F. Skinner of: Sur une forme particulière des reflex conditionnels.Les Compte Rendus des Seances de la Société Polonaise de Biologie, 1928,49, 1155–1157].PubMedCrossRefGoogle Scholar
  62. Morgan, M. J. (1974). Resistance to satiation.Animal Behaviour,22, 449–466.CrossRefGoogle Scholar
  63. Morrison, S. D., Mackay, C., Hurlbrink, E., Wier, J. K., Nick, M. S., &Miller, F. K. (1967). The water exchange and polyuria of rats deprived of food.Quarterly Journal of Experimental Physiology,52, 51–67.Google Scholar
  64. Mowrer, O. H. (1960).Learning theory and behavior. New York: Wiley.CrossRefGoogle Scholar
  65. Pavlov, I. P. (1927).Conditioned reflexes. Oxford: Oxford University Press.Google Scholar
  66. Pavlov, I. P. (1932). The reply of a physiologist to a psychologist.Psychological Review,39, 91–127.CrossRefGoogle Scholar
  67. Pelchat, M. L., Grill, H. J., Rozin, P., &Jacobs, J. (1983). Quality of acquired responses to taste byRattus norvegicus depends upon type of associated discomfort.Journal of Comparative & Physiological Psychology,97, 140–153.Google Scholar
  68. Perin, C. T. (1942). Behavior potentiality as a joint function of the amount of training and the degree of hunger at the time of extinction.Journal of Experimental Psychology,30, 93–113.CrossRefGoogle Scholar
  69. Peterson, D., &McHose, J. H. (1980). The residual deprivation effect in instrumental conditioning as a function of deprivation level at the time of exposure to the reinforcer.Animal Learning & Behavior,8, 642–646.CrossRefGoogle Scholar
  70. Rescorla, R. A. (1992). Depression of an instrumental response by a single devaluation of its outcome.Quarterly Journal of Experimental Psychology,44B, 123–136.Google Scholar
  71. Rescorla, R. A., &Solomon, R. L. (1967). Two-process learning theory: Relationship between Pavlovian conditioning and instrumental learning.Psychological Review,74, 151–182.PubMedCrossRefGoogle Scholar
  72. Revusky, S. H. (1967). Hunger level during food consumption: Effects on subsequent preference.Psychonomic Science,7, 109–110.Google Scholar
  73. Revusky, S. H. (1968). Effects of thirst level during consumption of flavored water on subsequent preference.Journal of Comparative & Physiological Psychology,66, 777–779.CrossRefGoogle Scholar
  74. Rolls, B. J., &McFarland, D. J. (1973). Hydration releases inhibition of feeding produced by intracranial angiotensin.Physiology & Behavior,11, 881–884.CrossRefGoogle Scholar
  75. Rozin, P., &Fallon, A. E. (1987). A perspective on disgust.Psychological Review,94, 23–41.PubMedCrossRefGoogle Scholar
  76. Sackett, R. S. (1939). The effect of strength of drive at the time of extinction upon resistance to extinction in rats.Journal of Comparative Psychology,27, 411–431.CrossRefGoogle Scholar
  77. Saltzman, I., &Koch, S. (1948). The effect of low intensities of hunger on the behavior mediated by a habit of maximum strength.Journal of Experimental Psychology,38, 347–370.PubMedCrossRefGoogle Scholar
  78. Sheffield, F. D. (1965). Relation between classical conditioning and instrumental learning. In W. F. Prokasy (Ed.),Classical conditioning: A symposium (pp. 302–322). New York: Appleton-Century-Crofts.Google Scholar
  79. Skinner, B. F. (1932). On the rate of formation of a conditioned reflex.Journal of General Psychology,7, 274–285.CrossRefGoogle Scholar
  80. Skinner, B. F. (1936). Conditioning and extinction and their relation to drive.Journal of General Psychology,14, 296–317.CrossRefGoogle Scholar
  81. Spence, K. W. (1956). Behavior theory and conditioning. New Haven, CT: Yale University Press.CrossRefGoogle Scholar
  82. Sutton, R. S., &Barto, A. G. (1981). An adaptive network that constructs and uses an internal model of its world.Cognition & Brain Theory,4, 217–246.Google Scholar
  83. Thorndike, E. L. (1911).Animal intelligence: Experimental studies. New York: Macmillan.Google Scholar
  84. Toates, F. M. (1978). A physiological control theory of hunger-thirst interaction. In D. A. Booth (Ed.),Hunger model (pp. 347–374). London: Academic Press.Google Scholar
  85. Tolman, E. C. (1949a). The nature and function of wants.Psychological Review,56, 357–369.PubMedCrossRefGoogle Scholar
  86. Tolman, E. C. (1949b). There is more than one kind of learning.Psychological Review,56, 144–155.PubMedCrossRefGoogle Scholar
  87. Tolman, E. C. (1959). Principles of purposive behavior. In S. Koch (Ed.),Psychology: A study of a science (Vol. 2, pp. 92–157). New York: McGraw-Hill.Google Scholar
  88. Trapold, M. A., &Overmier, J. B. (1972). The second learning process in instrumental learning. In A. H. Black & W. F. Prokasy (Eds),Classical conditioning II: Current research and theory (pp. 427–452). New York: Appleton-Century-Crofts.Google Scholar
  89. Van Hemel, P. E., &Myers, J. S. (1970). Control of food-motivated instrumental behavior in water-deprived rats by prior water and saline drinking.Learning & Motivation,1, 86–94.CrossRefGoogle Scholar
  90. Verplanck, W. S., &Hayes, J. R. (1953). Eating and drinking as a function of maintenance schedule.Journal of Comparative & Physiological Psychology,46, 327–333,CrossRefGoogle Scholar
  91. Webb, W. B. (1955). Drive stimuli as cues.Psychological Reports,1, 287–298.CrossRefGoogle Scholar
  92. Wetzel, R. J. (1959). The effect of experience with a taste reward.Journal of Comparative A. Physiological Psychology,52, 267–271.CrossRefGoogle Scholar
  93. Woods, S. C., Vasselli, J. R., &Milan, K. M. (1977). Iron appetite and latent learning in rats.Physiology & Behavior,19, 623–626.CrossRefGoogle Scholar
  94. Young, P. T. (1961).Motivation and emotion. New York: Wiley.Google Scholar

Copyright information

© Psychonomic Society, Inc. 1994

Authors and Affiliations

  • Anthony Dickinson
    • 1
  • Bernard Balleine
    • 1
  1. 1.University of CambridgeDepartment of Experimental PsychologyCB2 3EBUK

Personalised recommendations