Is It Better to Forget? Stimulus-Response, Prediction, and the Weight of Past Experience in a Fast-Paced Bargaining Task

  • Faison P. Gibson


Decision makers in dynamic environments such as air traffic control, firefighting, and call center operations adapt in real-time using outcome feedback. Understanding this adaptation is important for influencing and improving the decisions made. Recently, stimulus-response (S-R) learning models have been proposed as explanations for decision makers' adaptation. S-R models hypothesize that decision makers choose an action option based on their anticipation of its success. Decision makers learn by accumulating evidence over action options and combining that evidence with prior expectations. This study examines a standard S-R model and a simple variation of this model, in which past experience may receive an extremely low weight, as explanations for decision makers' adaptation in an evolving Internet-based bargaining environment. In Experiment 1, decision makers are taught to predict behavior in a bargaining task that follows rules that may be the opposite of, congruent to, or unrelated to a second task in which they must choose the deal terms they will offer. Both models provide a good account of the prediction task. However, only the second model, in which decision makers heavily discount all but the most recent past experience, provides a good account of subsequent behavior in the second task. To test whether Experiment 1 artificially related choice behavior and prediction, a second experiment examines both models' predictions concerning the effects of bargaining experience on subsequent prediction. In this study, decision models where long-term experience plays a dominating role do not appear to provide adequate explanations of decision makers' adaptation to their opponent's changing response behavior.

dynamic decision making game theory stimuls-response reinforcement learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Anderson, J.R. (1993), Rules of the Mind. Lawrence Erlbaum Associates, Hillsdale, NJ.Google Scholar
  2. Barto, A.G., R.S. Sutton and C.J.C.H. Watkins (1989), “Learning and Sequential Decision Making,” Tech. Rep. No. COINS 89-95, Amherst: University of Massachusetts.Google Scholar
  3. Brehmer, B. (1990), “Strategies in Real-Time, Dynamic Decision Making,” in R. Hogarth (Ed.) Insights from Decision Making. University of Chicago Press, Chicago.Google Scholar
  4. Brehmer, B. (1992), “Dynamic Decision Making: Human Control of Complex Systems,” Acta Psychologica, 81, 211–241.Google Scholar
  5. Brehmer, B. (1995), “Feedback Delays in Complex Dynamic Decision Tasks,” in P. Frensch and J. Funke (Eds.) Complex Problem Solving: The European Perspective. Lawrence Erlbaum Associates, Hillsdale, NJ.Google Scholar
  6. Carley, K.M. and J.-S. Lee (1998), “Dynamic Organizations: Organizational Adaptation in a Changing Environment,” in J. Baum (Ed.) Advances in StrategicManagement: Disciplinary Roots of StrategicManagement Research. JAI Press, London, Vol. 15, pp. 269–297.Google Scholar
  7. Dienes, Z. and R. Fahey (1995), “Role of Specific Instances in Controlling a Dynamic System,” Journal of Experimental Psychology: Learning, Memory, and Cognition, 21(4), 1–15.Google Scholar
  8. Dougherty, M.R.P., C.F. Gettys and E.E. Ogden (1999), “MINERVA-DM:AMemory Process Model for Judgments of Likelihood,” PsyRev, 106(1), 180–209.Google Scholar
  9. Erev, I. (1998), “Signal Detection by Human Observers:ACutoff Reinforcement Learning Model of Categorization Decisions under Uncertainty,” Psychological Review, 105(2), 280–298.Google Scholar
  10. Erev, I., Y. Bereby-Meyer and A.E. Roth (1999), “The Effect of Adding a Constant to All Payoffs: Experimental Investigation, and Implications for Reinforcement Learning Models,” Journal of Economic Behavior and Organization, 39, 111–128.Google Scholar
  11. Erev, I. and A.E. Roth (1998), “Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria,” American Economic Review, 88(4), 848–881.Google Scholar
  12. Fudenberg, D. and D.K. Levine (1998), The Theory of Learning in Games. The MIT Press, Cambridge, MA.Google Scholar
  13. Gibson, F.P. (2000), “Feedback Delays: How can Decision Makers Learn Not to Buy a New Car Every Time the Garage is Empty?” Organizational Behavior and Human Decision Processes, 83(1), 141–166.Google Scholar
  14. Gibson, F.P. and M. Fichman (1998), “Fear Appeals: What HappensWhen the Credit Collector Calls?” Unpublished Manuscript, in revision.Google Scholar
  15. Gibson, F.P., M. Fichman and D.C. Plaut (1996), “Learning in Dynamic Decision Tasks—Credit Collections at Citicorp,” Technical Report, Citicorp Behavioral Sciences Research Council.Google Scholar
  16. Gibson, F.P., M. Fichman and D.C. Plaut (1997), “Learning in Dynamic Decision Tasks: Computational Model and Empirical Evidence,” Organizational Behavior and Human Decision Processes, 71(1), 1–35.Google Scholar
  17. Joslyn, S. and E. Hunt (1998), “Evaluating Individual Differences in Response to Time-Pressure Situations,” Journal of Experimental Psychology: Applied, 4(1), 16–43.Google Scholar
  18. Kanfer, R. and P. Ackerman (1989), “Motivation and Cognitive Abilities: An Integrative/Aptitude-Treatment Interaction Approach to Skill Acquisition,” Journal of Applied Psychology, 74(4), 657–690.Google Scholar
  19. Klein, G.A., J. Orasanu, R. Calderwood and C.E. Zsambok (Eds.) (1993), Decision Making in Action: Models and Methods. Ablex Publishing Corporation, Norwood, NJ.Google Scholar
  20. Kleinmuntz, D.N. (1993), “Information Processing and Misperceptions of the Implications of Feedback in Dynamics Decision Making,” System Dynamic Reviews, 9(3), 223–237.Google Scholar
  21. LeDoux, J. (1996), The Emotional Brain.Simon & Schuster, New York.Google Scholar
  22. Levinthal, D. (1997), “Adaptation on Rugged Landscapes,” Management Science, 43, 934–950.Google Scholar
  23. Logan, G.D. (1988), “Toward an Instance Theory of Automatization,” Psychological Review, 95(4), 492–527.Google Scholar
  24. Logan, G.D. (1990), “Repetition Priming and Automaticity: Common Underlying Mechanisms?” Cognitive Psychology, 22, 1–35.Google Scholar
  25. Logan, G.D. (1992), “Shapes of Reaction-Time Distributions and Shapes of Learning Curves: A Test of the Instance Theory of Automaticity,” Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(5), 883–914.Google Scholar
  26. McClelland, J.L., B.L. McNaughton and R.C. O'Reilly (1995), “Why There are Complementary Learning Systems in the Hippocampus and Neocortex: Insights from the Successes and Failures of Connectionist Models of Learning and Memory,” Psychological Review.Google Scholar
  27. Naylor, J.C. and R.D. Clark (1968), “Intuitive Inference Strategies in Interval Learning Tasks as a Function of Validity Magnitude and Sign,” Organizational Behavior and Human Performance, 3, 378–399.Google Scholar
  28. Paich, M. and J.D. Sterman (1993), “Boom, Bust, and Failures to Learn in Experimental Markets,” Management Science, 39(12), 1439–1458.Google Scholar
  29. Rafaeli, A. and R.I. Sutton (1991), “Emotional Contrast Strategies as Means of Social Influence: Lessons from Criminal Interrogators and Bill Collectors,” Academy of Management Journal, 34(4), 749–775.Google Scholar
  30. Raiffa, H. (1982), The Art and Science of Negotiation. Harvard University Press, Cambridge, MA.Google Scholar
  31. Rapoport, A., I. Erev, E.V. Abraham and D.E. Olson (1997), “Randomization and Adaptive Learning in a Simplified Poker Game,” Organizational Behavior and Human Decision Processes, 69(1), 31–49.Google Scholar
  32. Roth, A.E. and I. Erev (1995), “Learning in Extensive-Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Term,” Games and Economic Behavior, 8, 164–212.Google Scholar
  33. Stanley, W.B., R.C. Mathews, R.R. Buss and S. Kotler-Cope (1989), “Insight without Awareness: On the Interaction of Verbalization, Instruction, and Practice in a Simulated Process Control Task,” Quarterly Journal of Experimental Psychology, 41A(3), 553–577.Google Scholar
  34. Sterman, J.D. (1989), “Misperceptions of Feedback in Dynamic Decision Making,” Organizational Behavior and Human Decision Processes, 43, 301–335.Google Scholar
  35. Sterman, J.D. (1994), “Learning in and about Complex Systems,” System Dynamics Review, 10(2/3), 291–330.Google Scholar
  36. Sutton, R.I. (1991), “Maintaining Norms about Expressed Emotions: The Case of Bill Collectors,” Administrative Science Quarterly, 36, 245–268.Google Scholar
  37. Sutton, R.S. (1988), “Learning to Predict by the Method of Temporal Differences,” Machine Learning, 3, 9–44.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Faison P. Gibson
    • 1
  1. 1.University of Michigan Business SchoolAnn ArborUSA

Personalised recommendations