Skip to main content

Motivational control of goal-directed action

Abstract

The control of goal-directed, instrumental actions by primary motivational states, such as hunger and thirst, is mediated by two processes. The first is engaged by the Pavlovian association between contextual or discriminative stimuli and the outcome or reinforcer presented during instrumental training. Such stimuli exert a motivational influence on instrumental performance that depends upon the relevance of the associated outcome to the current motivational state of the agent. Moreover, the motivational effects of these stimuli operate in the absence of prior experience with the outcome under the relevant motivational state. The second, instrumental, process is mediated by knowledge of the contingency between the action and its outcome and controls the value assigned to this outcome. In contrast to the Pavlovian process, motivational states do not influence the instrumental process directly; rather, the agent has to learn about the value of an outcome in a given motivational state by exposure to it while in that state. This incentive learning is similar in certain respects to the acquisition of “cathexes” envisaged by Tolman (1949a, 1949b).

References

  1. Adams, C. D., &Dickinson, A. (1981). Instrumental responding following reinforcer devaluation.Quarterly Journal of Experimental Psychology,33B, 109–122.

    Google Scholar 

  2. Asratyan, E. A. (1974). Conditioned reflex theory and motivational behavior.Acta Neurobiologiae Experimentalis,34, 15–31.

    PubMed  Google Scholar 

  3. Baeyens, F., Eelen, P., van den Berg, H., &Crombez, G. (1990). Flavor-flavor and color-flavor conditioning in humans.Learning & Motivation,21, 434–455.

    Article  Google Scholar 

  4. Balleine, B. (1992). Instrumental performance following a shift in primary motivation depends upon incentive learning.Journal of Experimental Psychology: Animal Behavior Processes,18, 236–250.

    PubMed  Article  Google Scholar 

  5. Balleine, B., &Dickinson, A. (1991). Instrumental performance following reinforcer devaluation depends upon incentive learning.Quarterly Journal of Experimental Psychology,43B, 279–296.

    Google Scholar 

  6. Balleine, B., &Dickinson, A. (1992). Signalling and incentive processes in instrumental reinforcer devaluation.Quarterly Journal of Experimental Psychology,45B, 285–301.

    Google Scholar 

  7. Berridge, K. C. (1991). Modulation of taste affect by hunger, caloric satiety, and sensory-specific satiety in the rat.Appetite,16, 103–120.

    PubMed  Article  Google Scholar 

  8. Berridge, K. C., Flynn, F. W., Schulkin, J., &Grill, H. J. (1984). Sodium depletion enhances salt palatability in rats.Behavioral Neuroscience 98, 652–660.

    PubMed  Article  Google Scholar 

  9. Berridge, K. C., Grill, H. J., &Norgren, R. (1981). Relation of consummatory responses and preabsorptive insulin release to palatability and learned taste aversions.Journal of Comparative & Physiological Psychology,95, 363–382.

    Article  Google Scholar 

  10. Bindra, D. (1972). A unified account of classical conditioning and operant training. In A. H. Black & W. F. Prokasy (Eds.),Classical conditioning II: Current research and theory (pp. 453–482). New York: Appleton-Century-Crofts.

    Google Scholar 

  11. Bindra, D. (1978). How adaptive behavior is produced: A perceptual motivational alternative to response-reinforcement.Behavior & Brain Sciences,1, 41–52.

    Article  Google Scholar 

  12. Bolles, R. C. (1972). Reinforcement, expectancy, and learning.Psychological Review,95, 394–409.

    Article  Google Scholar 

  13. Bolles, R. C. (1975).Theory of motivation. New York: Harper & Row.

    Google Scholar 

  14. Bolles, R. C., Holtz, R., Dunn, T., &Hill, W. (1980). Comparison of stimulus learning and response learning in a punishment situation.Learning & Motivation,11, 78–96.

    Article  Google Scholar 

  15. Cabanac, M. (1971). Physiological role of pleasure.Science,173, 1103–1107.

    PubMed  Article  Google Scholar 

  16. Campbell, D. H., Capaldi, E. D., Sheffer, J. D., &Bradford, J. P. (1988). An examination of the relationship between expectancy learning and preference conditioning.Learning & Motivation,19, 162–182.

    Article  Google Scholar 

  17. Capaldi, E. D., Davidson, T. L., &Myers, D. E. (1981). Resistance to satiation: Reinforcing effects of food and eating under satiation.Learning & Motivation,12, 171–195.

    Article  Google Scholar 

  18. Capaldi, E. D., &Myers, D. E. (1982). Taste preferences as a function of food deprivation during original taste exposure.Animal Learning & Behavior,10, 211–219.

    Google Scholar 

  19. Capaldi, E. D., Myers, D. E., Campbell, D. H., &Sheffer, J. D. (1983). Conditioned flavor preferences based on hunger level during original flavor exposure.Animal Learning & Behavior,11, 107–115.

    Google Scholar 

  20. Capaldi, E. D., Sheffer, J., &Owens, J. (1991). Food deprivation and conditioned flavor preferences based on sweetened and unsweetened foods.Animal Learning & Behavior,19, 361–368.

    Article  Google Scholar 

  21. Capaldi, E. D., Viveiros, D. M., &Davidson, T. L. (1981). Deprivation stimulus intensity and incentive factors in the control of instrumental performance.Journal of Experimental Psychology: Animal Behavior Processes,7, 140–149.

    Article  Google Scholar 

  22. Colwill, R. C., &Rescorla, R. A. (1985). Postconditioning devaluation of a reinforcer affects instrumental responding.Journal of Experimental Psychology: Animal Behavior Processes,11, 120–132.

    Article  Google Scholar 

  23. Colwill, R. C., &Rescorla, R. A., (1986). Associative structures in instrumental learning. In G. H. Bower (Ed.),The psychology of learning and motivation (Vol. 20, pp. 55–104). New York: Academic Press.

    Google Scholar 

  24. Crocetti, C. P. (1962). Drive level and response strength in the barpressing apparatus.Psychological Reports,10, 563–575.

    Article  Google Scholar 

  25. Davidson, T. L. (1987). Learning about deprivational intensity stimuli.Behavioral Neuroscience,101, 198–208.

    PubMed  Article  Google Scholar 

  26. Davidson, T. L., Flynn, F. W., &Jarrard, L. E. (1992). Potency of food deprivation intensity cues as discriminative stimuli.Journal of Experimental Psychology: Animal Behavior Processes,18, 174–181.

    PubMed  Article  Google Scholar 

  27. Dickinson, A. (1980).Contemporary animal learning theory. Cambridge: Cambridge University Press.

    Google Scholar 

  28. Dickinson, A. (1986). Re-examination of the role of the instrumental contingency in the sodium-appetite irrelevant incentive effect.Quarterly Journal of Experimental Psychology,38B, 161–172.

    Google Scholar 

  29. Dickinson, A. (1989). Expectancy theory in animal conditioning. In S. B. Klein & R. R. Mowrer (Eds.),Contemporary learning theories: Pavlovian conditioning and the status of traditional learning theory (pp. 279–308). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  30. Dickinson, A., &Balleine, B. (1990). Motivational control of instrumental performance following a shift from hunger to thirst.Quarterly Journal of Experimental Psychology,42B, 413–431.

    Google Scholar 

  31. Dickinson, A., &Dawson, G. R. (1987a). Pavlovian processes in the motivational control of instrumental performance.Quarterly Journal of Experimental Psychology,38B, 201–213.

    Google Scholar 

  32. Dickinson, A., &Dawson, G. R. (1987b). The role of the instrumental contingency in the motivational control of performance.Quarterly Journal of Experimental Psychology,39B, 77–93.

    Google Scholar 

  33. Dickinson, A., &Dawson, G. R. (1988). Motivational control of instrumental performance: The role of prior experience of the reinforcer.Quarterly Journal of Experimental Psychology,40B, 113–134.

    Google Scholar 

  34. Dickinson, A., &Dawson, G. R. (1989). Incentive learning and the motivational control of instrumental performance.Quarterly Journal of Experimental Psychology,41B, 99–112.

    Google Scholar 

  35. Dickinson, A., &Nicholas, D. J. (1983). Irrelevant incentive learning during instrumental conditioning: The role of drive-reinforcer and response-reinforcer relationships.Quarterly Journal of Experimental Psychology,35B, 249–263.

    Google Scholar 

  36. Eisman, E., Theios, J., &Linton, M. (1961). Habit strength as a function of drive in a bar pressing situation.Psychological Reports,9, 583–590.

    Article  Google Scholar 

  37. Estes, W. K. (1943). Discriminative conditioning: 1. A discriminative property of conditioned anticipation.Journal of Experimental Psychology,32, 150–155.

    Article  Google Scholar 

  38. Everitt, B. J., &Stacey, P. (1987). Studies of instrumental behavior with sexual reinforcement in male rats (Rattus norvegicus): 2. Effects of preoptic area lesions, castration, and testosterone.Journal of Comparative Psychology,101, 407–419.

    PubMed  Article  Google Scholar 

  39. Fitzsimons, J. T. (1961). Drinking by nephrectomized rats injected with various substances.Journal of Physiology,155, 563–579.

    PubMed  Google Scholar 

  40. Garcia, J. (1989). Food for Tolman: Cognition and cathexis in concert. In T. Archer & L.-G. Nilsson (Eds.),Aversion, avoidance and anxiety (pp. 45–85). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  41. Gormezano, I., &Tait, R. W. (1976). The Pavlovian analysis of instrumental conditioning.Pavlovian Journal of Biological Sciences,11, 37–55.

    Google Scholar 

  42. Grice, G. R., &Davis, J. D. (1957). Effect of irrelevant thirst motivation on a response learned with food reward.Journal of Experimental Psychology,53, 347–352.

    PubMed  Article  Google Scholar 

  43. Grill, H. J., &Berridge, K. C. (1985). Taste reactivity as a measure of the neural control of palatability. In J. M. Sprague & A. N. Epstein (Eds.),Progress in psychobiology and physiological psychology (Vol. 11, pp. 1–61). Orlando, FL: Academic Press.

    Google Scholar 

  44. Grindley, G. C. (1932). The formation of a simple habit in guinea pigs.British Journal of Psychology,23, 127–147.

    Google Scholar 

  45. Heathers, G. L., &Arakelian, P. (1941). The relationship between strength of drive and rate of extinction of a bar-pressing reaction in the rat.Journal of General Psychology,24, 243–258.

    Google Scholar 

  46. Hendersen, R. W., &Graham, J. (1979). Avoidance of heat by rats: Effects of thermal context on the rapidity of extinction.Learning & Motivation,10, 351–363.

    Article  Google Scholar 

  47. Hershberger, W. A. (1986). An approach through the looking-glass.Animal Learning & Behavior,14, 443–451.

    Google Scholar 

  48. Horenstein, B. R. (1951). Performance of conditioned responses as a function of strength of hunger drive.Journal of Comparative & Physiological Psychology,44, 210–224.

    Article  Google Scholar 

  49. Hovancik, J. R. (1978). The effect of deprivation level during noncontingent pairings and instrumental learning on subsequent instrumental performance.Learning & Motivation,9, 1–15.

    Article  Google Scholar 

  50. Hull, C. L. (1943).Principles of behavior. New York: Appleton-Century-Crofts.

    Google Scholar 

  51. Irwin, F. W. (1971).Intentional behavior and motivation: A cognitive theory. Philadelphia: Lippincott.

    Google Scholar 

  52. Jenkins, H. M., &Harrison, R. H. (1960). The effects of discrimination training on auditory generalization.Journal of Experimental Psychology,59, 246–253.

    PubMed  Article  Google Scholar 

  53. King, D. L. (1979).Conditioning: An image approach. New York: Gardner.

    Google Scholar 

  54. Koch, S., &Daniel, W. J. (1945). The effect of satiation on behavior mediated by a habit of maximum strength.Journal of Experimental Psychology,35, 167–187.

    Article  Google Scholar 

  55. Konorski, J. (1967).Integrative activity of the brain: An interdisciplinary approach. Chicago: University of Chicago Press.

    Google Scholar 

  56. Krieckhaus, E. E., &Wolf, G. (1968). Acquisition of sodium by rats: Interaction of innate mechanisms and latent learning.Journal of Comparative & Physiological Psychology,65, 197–201.

    Article  Google Scholar 

  57. Lopez, M., Balleine, B., &Dickinson, A. (1992a). Incentive learning and the motivational control of instrumental performance by thirst.Animal Learning & Behavior,20, 322–328.

    Article  Google Scholar 

  58. Lopez, M., Balleine, B., &Dickinson, A. (1992b). Incentive learning following reinforcer devaluation is not conditional upon motivational state during re-exposure.Quarterly Journal of Experimental Psychology,45B, 265–284.

    Google Scholar 

  59. Meachum, C. L. (1990). The role of response-contingent incentives in lithium chloride-mediated suppression of an operant response.Quarterly Journal of Experimental Psychology,42B, 175–195.

    Google Scholar 

  60. Miller, N. E. (1963). Some reflections on the law of effect produce a new alternative to drive reduction. In M. R. Jones (Ed.),Nebraska symposium on motivation (pp. 65–112). Lincoln: University of Nebraska Press.

    Google Scholar 

  61. Miller, S., &Konorski, J. (1969). On a particular form of conditioned reflex.Journal of the Experimental Analysis of Behavior,12, 187–189. [Translation by B. F. Skinner of: Sur une forme particulière des reflex conditionnels.Les Compte Rendus des Seances de la Société Polonaise de Biologie, 1928,49, 1155–1157].

    PubMed  Article  Google Scholar 

  62. Morgan, M. J. (1974). Resistance to satiation.Animal Behaviour,22, 449–466.

    Article  Google Scholar 

  63. Morrison, S. D., Mackay, C., Hurlbrink, E., Wier, J. K., Nick, M. S., &Miller, F. K. (1967). The water exchange and polyuria of rats deprived of food.Quarterly Journal of Experimental Physiology,52, 51–67.

    Google Scholar 

  64. Mowrer, O. H. (1960).Learning theory and behavior. New York: Wiley.

    Book  Google Scholar 

  65. Pavlov, I. P. (1927).Conditioned reflexes. Oxford: Oxford University Press.

    Google Scholar 

  66. Pavlov, I. P. (1932). The reply of a physiologist to a psychologist.Psychological Review,39, 91–127.

    Article  Google Scholar 

  67. Pelchat, M. L., Grill, H. J., Rozin, P., &Jacobs, J. (1983). Quality of acquired responses to taste byRattus norvegicus depends upon type of associated discomfort.Journal of Comparative & Physiological Psychology,97, 140–153.

    Google Scholar 

  68. Perin, C. T. (1942). Behavior potentiality as a joint function of the amount of training and the degree of hunger at the time of extinction.Journal of Experimental Psychology,30, 93–113.

    Article  Google Scholar 

  69. Peterson, D., &McHose, J. H. (1980). The residual deprivation effect in instrumental conditioning as a function of deprivation level at the time of exposure to the reinforcer.Animal Learning & Behavior,8, 642–646.

    Article  Google Scholar 

  70. Rescorla, R. A. (1992). Depression of an instrumental response by a single devaluation of its outcome.Quarterly Journal of Experimental Psychology,44B, 123–136.

    Google Scholar 

  71. Rescorla, R. A., &Solomon, R. L. (1967). Two-process learning theory: Relationship between Pavlovian conditioning and instrumental learning.Psychological Review,74, 151–182.

    PubMed  Article  Google Scholar 

  72. Revusky, S. H. (1967). Hunger level during food consumption: Effects on subsequent preference.Psychonomic Science,7, 109–110.

    Google Scholar 

  73. Revusky, S. H. (1968). Effects of thirst level during consumption of flavored water on subsequent preference.Journal of Comparative & Physiological Psychology,66, 777–779.

    Article  Google Scholar 

  74. Rolls, B. J., &McFarland, D. J. (1973). Hydration releases inhibition of feeding produced by intracranial angiotensin.Physiology & Behavior,11, 881–884.

    Article  Google Scholar 

  75. Rozin, P., &Fallon, A. E. (1987). A perspective on disgust.Psychological Review,94, 23–41.

    PubMed  Article  Google Scholar 

  76. Sackett, R. S. (1939). The effect of strength of drive at the time of extinction upon resistance to extinction in rats.Journal of Comparative Psychology,27, 411–431.

    Article  Google Scholar 

  77. Saltzman, I., &Koch, S. (1948). The effect of low intensities of hunger on the behavior mediated by a habit of maximum strength.Journal of Experimental Psychology,38, 347–370.

    PubMed  Article  Google Scholar 

  78. Sheffield, F. D. (1965). Relation between classical conditioning and instrumental learning. In W. F. Prokasy (Ed.),Classical conditioning: A symposium (pp. 302–322). New York: Appleton-Century-Crofts.

    Google Scholar 

  79. Skinner, B. F. (1932). On the rate of formation of a conditioned reflex.Journal of General Psychology,7, 274–285.

    Article  Google Scholar 

  80. Skinner, B. F. (1936). Conditioning and extinction and their relation to drive.Journal of General Psychology,14, 296–317.

    Article  Google Scholar 

  81. Spence, K. W. (1956). Behavior theory and conditioning. New Haven, CT: Yale University Press.

    Book  Google Scholar 

  82. Sutton, R. S., &Barto, A. G. (1981). An adaptive network that constructs and uses an internal model of its world.Cognition & Brain Theory,4, 217–246.

    Google Scholar 

  83. Thorndike, E. L. (1911).Animal intelligence: Experimental studies. New York: Macmillan.

    Google Scholar 

  84. Toates, F. M. (1978). A physiological control theory of hunger-thirst interaction. In D. A. Booth (Ed.),Hunger model (pp. 347–374). London: Academic Press.

    Google Scholar 

  85. Tolman, E. C. (1949a). The nature and function of wants.Psychological Review,56, 357–369.

    PubMed  Article  Google Scholar 

  86. Tolman, E. C. (1949b). There is more than one kind of learning.Psychological Review,56, 144–155.

    PubMed  Article  Google Scholar 

  87. Tolman, E. C. (1959). Principles of purposive behavior. In S. Koch (Ed.),Psychology: A study of a science (Vol. 2, pp. 92–157). New York: McGraw-Hill.

    Google Scholar 

  88. Trapold, M. A., &Overmier, J. B. (1972). The second learning process in instrumental learning. In A. H. Black & W. F. Prokasy (Eds),Classical conditioning II: Current research and theory (pp. 427–452). New York: Appleton-Century-Crofts.

    Google Scholar 

  89. Van Hemel, P. E., &Myers, J. S. (1970). Control of food-motivated instrumental behavior in water-deprived rats by prior water and saline drinking.Learning & Motivation,1, 86–94.

    Article  Google Scholar 

  90. Verplanck, W. S., &Hayes, J. R. (1953). Eating and drinking as a function of maintenance schedule.Journal of Comparative & Physiological Psychology,46, 327–333,

    Article  Google Scholar 

  91. Webb, W. B. (1955). Drive stimuli as cues.Psychological Reports,1, 287–298.

    Article  Google Scholar 

  92. Wetzel, R. J. (1959). The effect of experience with a taste reward.Journal of Comparative A. Physiological Psychology,52, 267–271.

    Article  Google Scholar 

  93. Woods, S. C., Vasselli, J. R., &Milan, K. M. (1977). Iron appetite and latent learning in rats.Physiology & Behavior,19, 623–626.

    Article  Google Scholar 

  94. Young, P. T. (1961).Motivation and emotion. New York: Wiley.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Anthony Dickinson.

Additional information

This research was supported by the S.E.R.C. and the Commonwealth Scholarship Commission. We thank A. Watt and R. A. Boakes for their comments on a draft of the manuscript.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Dickinson, A., Balleine, B. Motivational control of goal-directed action. Animal Learning & Behavior 22, 1–18 (1994). https://doi.org/10.3758/BF03199951

Download citation

Keywords

  • Motivational State
  • Incentive Learning
  • Instrumental Performance
  • Concurrent Training
  • Instrumental Training