Minds and Machines

, Volume 26, Issue 1–2, pp 125–148 | Cite as

Rational Task Analysis: A Methodology to Benchmark Bounded Rationality

Article

Abstract

How can we study bounded rationality? We answer this question by proposing rational task analysis (RTA)—a systematic approach that prevents experimental researchers from drawing premature conclusions regarding the (ir-)rationality of agents. RTA is a methodology and perspective that is anchored in the notion of bounded rationality and aids in the unbiased interpretation of results and the design of more conclusive experimental paradigms. RTA focuses on concrete tasks as the primary interface between agents and environments and requires explicating essential task elements, specifying rational norms, and bracketing the range of possible performance, before contrasting various benchmarks with actual performance. After describing RTA’s core components we illustrate its use in three case studies that examine human memory updating, multitasking behavior, and melioration. We discuss RTA’s characteristic elements and limitations by comparing it to related approaches. We conclude that RTA provides a useful tool to render the study of bounded rationality more transparent and less prone to theoretical confusion.

Keywords

Bounded rationality Benchmarking Optimality Task environment Rational analysis Ecological rationality 

References

  1. Akerlof, G. A., & Shiller, R. J. (2010). Animal spirits: How human psychology drives the economy, and why it matters for global capitalism. Princeton, NJ: Princeton University Press.Google Scholar
  2. Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  3. Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., & Qin, Y. (2004). An integrated theory of the mind. Psychological Review, 111(4), 1036–1060.CrossRefGoogle Scholar
  4. Anderson, J. R., & Milson, R. (1989). Human memory: An adaptive perspective. Psychological Review, 96(4), 703–719.CrossRefGoogle Scholar
  5. Anderson, J. R., & Schooler, L. J. (1991). Reflections of the environment in memory. Psychological Science, 2(6), 396–408.CrossRefGoogle Scholar
  6. Ariely, D. (2008). Predictably irrational: The hidden forces that shape our decisions. New York, NY: Harper Collins.Google Scholar
  7. Birnbaum, M. H. (1983). Base rates in Bayesian inference: Signal detection analysis of the cab problem. The American Journal of Psychology, 96(1), 85–94.Google Scholar
  8. Brunswik, E. (1943). Organismic achievement and environmental probability. Psychological Review, 50(3), 255–272.CrossRefGoogle Scholar
  9. Brunswik, E. (1955). Representative design and probabilistic theory in a functional psychology. Psychological Review, 62(3), 193–217.CrossRefGoogle Scholar
  10. Brunswik, E. (1956). Perception and the representative design of psychological experiments. Berkeley, CA: University of California Press.Google Scholar
  11. Burns, B. D. (2004). Heuristics as beliefs and as behaviors: The adaptiveness of the “hot hand”. Cognitive Psychology, 48(3), 295–331.CrossRefGoogle Scholar
  12. Burns, K. (2001). TRACS: A tool for research on adaptive cognitive strategies: The game of confidence and consequence. http://www.tracsgame.com. Accessed 1 May 2015.
  13. Burns, K. (2002). On straight TRACS: A baseline bias from mental models. In Proceedings of the 24th Annual Meeting of the Cognitive Science Society (pp. 154–159). Lawrence Erlbaum, Hillsdale, NJ.Google Scholar
  14. Dawkins, R. (1999). The extended phenotype: The long reach of the gene, revised edn. Oxford, UK: Oxford University Press.Google Scholar
  15. Dhami, M. K., Hertwig, R., & Hoffrage, U. (2004). The role of representative design in an ecological approach to cognition. Psychological Bulletin, 130(6), 959–988.CrossRefGoogle Scholar
  16. Ferguson, N. (2008). The ascent of money: A financial history of the world. London, UK: Penguin.Google Scholar
  17. Fiedler, K., & Juslin, P. (Eds.). (2006). Information sampling and adaptive cognition. New York, NY: Cambridge University Press.Google Scholar
  18. Fu, W. T., & Gray, W. D. (2004). Resolving the paradox of the active user: Stable suboptimal performance in interactive tasks. Cognitive Science, 28(6), 901–935.CrossRefGoogle Scholar
  19. Gigerenzer, G. (1991). How to make cognitive illusions disappear: Beyond heuristics and biases. European Review of Social Psychology, 2(1), 83–115.CrossRefGoogle Scholar
  20. Gigerenzer, G. (1996). On narrow norms and vague heuristics: A reply to Kahneman and Tversky. Psychological Review, 103, 592–596.CrossRefGoogle Scholar
  21. Gigerenzer, G. (2004). The irrationality paradox. Behavioral and Brain Sciences, 27(3), 336–338.CrossRefGoogle Scholar
  22. Gigerenzer, G., & Brighton, H. (2009). Homo heuristicus: Why biased minds make better inferences. Topics in Cognitive Science, 1(1), 107–143.CrossRefGoogle Scholar
  23. Gigerenzer, G., Hertwig, R., & Pachur, T. (Eds.). (2011). Heuristics: The foundations of adaptive behavior. New York, NY: Oxford University Press.Google Scholar
  24. Gigerenzer, G., Todd, P. M., & the ABC Research Group (1999). Simple heuristics that make us smart. New York, NY: Oxford University Press.Google Scholar
  25. Gray, W. D., & Boehm-Davis, D. A. (2000). Milliseconds matter: An introduction to microstrategies and to their use in describing and predicting interactive behavior. Journal of Experimental Psychology: Applied, 6(4), 322–335.Google Scholar
  26. Gray, W. D., Neth, H., & Schoelles, M. J. (2006). The functional task environment. In A. F. Kramer, D. A. Wiegman, & A. Kirlik (Eds.), Attention: From theory to practice (pp. 100–118). New York, NY: Oxford University Press.CrossRefGoogle Scholar
  27. Hammond, K. R., & Stewart, T. R. (2001). The essential Brunswik: Beginnings, explications, applications. New York, NY: Oxford University Press.Google Scholar
  28. Herrnstein, R. J. (1981). Self-control as response strength. In C. M. Bradshaw, E. Szabadi, & C. F. Lowe (Eds.), Recent developments in the quantification of steady-state operant behavior (pp. 3–20). Amsterdam, NL: Elsevier.Google Scholar
  29. Herrnstein, R. J. (1982). Melioration as behavioral dynamism. Quantitative Analyses of Behavior, 2, 433–458.Google Scholar
  30. Herrnstein, R. J. (1990). Behavior, reinforcement and utility. Psychological Science, 1(4), 217–224.CrossRefGoogle Scholar
  31. Herrnstein, R. J. (1990). Rational choice theory. American Psychologist, 45(3), 356–367.CrossRefGoogle Scholar
  32. Herrnstein, R. J. (1991). Experiments on stable suboptimality in individual behavior. The American Economic Review, 81(2), 360–364.Google Scholar
  33. Herrnstein, R. J., & Prelec, D. (1992). A theory of addiction. Choice over time, 331–360.Google Scholar
  34. Herrnstein, R. J., & Vaughan, W, Jr. (1980). Melioration and behavioral allocation. In J. E. R. Staddon (Ed.), Limits to action: The allocation of individual behavior (pp. 143–176). New York, NY: Academic Press.CrossRefGoogle Scholar
  35. Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004). Decisions from experience and the effect of rare events in risky choice. Psychological Science, 15(8), 534–539.CrossRefGoogle Scholar
  36. Hertwig, R., & Erev, I. (2009). The description-experience gap in risky choice. Trends in Cognitive Sciences, 13(12), 517–523.CrossRefGoogle Scholar
  37. Hertwig, R., Hoffrage, U., & the ABC Research Group (2013). Simple heuristics in a social world. New York, NY: Oxford University Press.Google Scholar
  38. Heyman, G. M., & Dunn, B. (2002). Decision biases and persistent illicit drug use: An experimental study of distributed choice and addiction. Drug and Alcohol Dependence, 67(2), 193–203.CrossRefGoogle Scholar
  39. Howes, A., Lewis, R. L., & Vera, A. (2009). Rational adaptation under task and processing constraints: Implications for testing theories of cognition and action. Psychological Review, 116(4), 717.CrossRefGoogle Scholar
  40. Kahneman, D. (2003). Maps of bounded rationality: Psychology for behavioral economics. American Economic Review, 93(5), 1449–1475.CrossRefGoogle Scholar
  41. Kahneman, D., Slovic, P., & Tversky, A. (1982). Judgment under uncertainty: Heuristics and biases. Cambridge, UK: Cambridge University Press.CrossRefGoogle Scholar
  42. Kahneman, D., & Tversky, A. (1996). On the reality of cognitive illusions. Psychological Review, 103(3), 582–591.CrossRefGoogle Scholar
  43. Kieras, D. E., & Meyer, D. E. (2000). The role of cognitive task analysis in the application of predictive models of human performance. In J. M. C. Schraagen, S. E. Chipman, & V. L. Shalin (Eds.), Cognitive task analysis (pp. 237–260). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  44. Knight, F. H. (1921). Risk, uncertainty and profit. Chicago, IL: University of Chicago Press.Google Scholar
  45. Koehler, J. J. (1996). The base rate fallacy reconsidered: Descriptive, normative, and methodological challenges. Behavioral and Brain Sciences, 19(01), 1–17.CrossRefGoogle Scholar
  46. Krueger, J. I., & Funder, D. C. (2004). Towards a balanced social psychology: Causes, consequences, and cures for the problem-seeking approach to social behavior and cognition. Behavioral and Brain Sciences, 27(3), 313–327.Google Scholar
  47. Lewis, R. L., Howes, A., & Singh, S. (2014). Computational rationality: Linking mechanism and behavior through bounded utility maximization. Topics in Cognitive Science, 6(2), 279–311.CrossRefGoogle Scholar
  48. Lopes, L. L. (1991). The rhetoric of irrationality. Theory and Psychology, 1(1), 65–82.CrossRefGoogle Scholar
  49. Meyer, D. E., & Kieras, D. E. (1997). A computational theory of executive cognitive processes and multiple-task performance. I: Basic mechanisms. Psychological Review, 104(1), 3–65.CrossRefGoogle Scholar
  50. Neth, H., Carlson, R. A., Gray, W. D., Kirlik, A., Kirsh, D., & Payne, S. J. (2007). Immediate interactive behavior: A symposium on embodied and embedded cognition. In D. S. McNamara & J. G. Trafton (Eds.), Proceedings of the 29th Annual Meeting of the Cognitive Science Society (pp. 33–34). Cognitive Science Society, Austin, TX.Google Scholar
  51. Neth, H., & Gigerenzer, G. (2015). Heuristics: Tools for an uncertain world. In R. Scott & S. Kosslyn (Eds.), Emerging trends in the social and behavioral sciences. New York, NY: Wiley Online Library.Google Scholar
  52. Neth, H., Khemlani, S. S., & Gray, W. D. (2008). Feedback design for the control of a dynamic multitasking system: Dissociating outcome feedback from control feedback. Human Factors, 50(4), 643–651.CrossRefGoogle Scholar
  53. Neth, H., Khemlani, S. S., Oppermann, B., & Gray, W. D. (2006). Juggling multiple tasks: A rational analysis of multitasking in a synthetic task environment. In Proceedings of the Human Factors and Ergonomics Society, vol. 50, (pp. 1142–1146). Sage, San Francisco, CA.Google Scholar
  54. Neth, H., Sims, C. R., & Gray, W. D. (2005). Melioration despite more information: The role of feedback frequency in stable suboptimal performance. In Proceedings of the Human Factors and Ergonomics Society, vol. 49, (pp. 357–361). Sage, Orlanco, FL.Google Scholar
  55. Neth, H., Sims, C. R., & Gray, W. D. (2006). Melioration dominates maximization: Stable suboptimal performance despite global feedback. In R. Sun & N. Miyake (Eds.), Proceedings of the 28th Annual Meeting of the Cognitive Science Society (pp. 627–632). Lawrence Erlbaum, Hillsdale, NJ.Google Scholar
  56. Neth, H., Sims, C. R., Veksler, V. D., & Gray, W. D. (2004). You can’t play straight TRACS and win: Memory updates in a dynamic task environment. In K. D. Forbus, D. Gentner & T. Regier (Eds.), Proceedings of the 26th Annual Meeting of the Cognitive Science Society (pp. 1017–1022). Lawrence Erlbaum, Hillsdale, NJ.Google Scholar
  57. Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press.Google Scholar
  58. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
  59. Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241–301.CrossRefGoogle Scholar
  60. Norman, D. A., & Bobrow, D. G. (1975). On data-limited and resource-limited processes. Cognitive Psychology, 7(1), 44–64.CrossRefGoogle Scholar
  61. Oaksford, M., & Chater, N. (2007). Bayesian rationality: The probabilistic approach to human reasoning. New York, NY: Oxford University Press.CrossRefGoogle Scholar
  62. Pleskac, T. J., & Hertwig, R. (2014). Ecologically rational choice and the structure of the environment. Journal of Experimental Psychology: General, 143(5), 2000–2019.CrossRefGoogle Scholar
  63. Rachlin, H., & Laibson, D. I. (Eds.). (1997). The matching law: Papers on psychology and economics by Richard Herrnstein. New York, NY: Russell Sage Foundation.Google Scholar
  64. Ross, L., & Nisbett, R. E. (1991). The person and the situation: Perspectives of social psychology. New York, NY: McGraw-Hill.Google Scholar
  65. Samuels, R., Stich, S., & Bishop, M. (2002). Ending the rationality wars: How to make disputes about human rationality disappear. In R. Elio (Ed.), Common sense, reasoning and rationality (pp. 236–268). New York, NY: Oxford University Press.CrossRefGoogle Scholar
  66. Sanborn, A. N., Griffiths, T. L., & Navarro, D. J. (2010). Rational approximations to rational models: Alternative algorithms for category learning. Psychological Review, 117(4), 1144–1167.CrossRefGoogle Scholar
  67. Scriven, M. (1991). The methodology of evaluation. In A. A. Bellack & H. M. Kliebard (Eds.), Curriculum and evaluation (pp. 334–371). Berkeley, CA: McCutchan.Google Scholar
  68. Shakeri, S. (2003). A mathematical modeling framework for scheduling and managing multiple concurrent tasks. Ph.D. thesis, Oregon State University, Corvallis, OR.Google Scholar
  69. Shakeri, S., & Funk, K. (2007). A comparison of human and near-optimal task management behavior. Human Factors, 49(3), 400–416.CrossRefGoogle Scholar
  70. Simon, H. A. (1955). A behavioral model of rational choice. The Quarterly Journal of Economics, 69(1), 99–118.CrossRefGoogle Scholar
  71. Simon, H. A. (1956). Rational choice and the structure of the environment. Psychological Review, 63(2), 129–138.CrossRefGoogle Scholar
  72. Simon, H. A. (1990). Invariants of human behavior. Annual Reviews in Psychology, 41(1), 1–20.CrossRefGoogle Scholar
  73. Simon, H. A. (1991). Cognitive architectures and rational analysis: Comment. In K. VanLehn (Ed.), Architectures for intelligence: The 22nd Carnegie Mellon Symposium on Cognition (pp. 25–39). Lawrence Erlbaum, Hillsdale, NJ.Google Scholar
  74. Simon, H. A. (1996). The Sciences of the artificial (3rd ed.). Cambridge, MA: The MIT Press.Google Scholar
  75. Sims, C. R., Neth, H., Jacobs, R. A., & Gray, W. D. (2013). Melioration as rational choice: Sequential decision making in uncertain environments. Psychological Review, 120(1), 139–154. doi:10.1037/a0030850.CrossRefGoogle Scholar
  76. Stewart, N., Chater, N., & Brown, G. D. A. (2006). Decision by sampling. Cognitive Psychology, 53(1), 1–26.CrossRefGoogle Scholar
  77. Thaler, R. H. (1994). Quasi rational economics. New York, NY: Russell Sage Foundation.Google Scholar
  78. Todd, P. M., & Gigerenzer, G. (2001). Shepard’s mirrors or Simon’s scissors? Behavioral and Brain Sciences, 24(04), 704–705.CrossRefGoogle Scholar
  79. Todd, P. M., Gigerenzer, G., & the ABC Research Group (2012). Ecological rationality: Intelligence in the world. New York, NY: Oxford University Press.Google Scholar
  80. Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131.CrossRefGoogle Scholar
  81. Vaughan, W, Jr. (1981). Melioration, matching, and maximization. Journal of the Experimental Analysis of Behavior, 36(2), 141–149.MathSciNetCrossRefGoogle Scholar
  82. Venturino, M. (1997). Interference and information organization in keeping track of continually changing information. Human Factors, 39(4), 532–539.Google Scholar
  83. Wilson, J. Q., & Herrnstein, R. J. (1985). Crime and human nature. New York, NY: Simon & Schuster.Google Scholar
  84. Yechiam, E., Erev, I., Yehene, V., & Gopher, D. (2003). Melioration and the transition from touch-typing training to everyday use. Human Factors, 45(4), 671–684.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  • Hansjörg Neth
    • 1
    • 2
  • Chris R. Sims
    • 3
  • Wayne D. Gray
    • 4
  1. 1.Center for Adaptive Behavior and Cognition (ABC)Max Planck Institute for Human DevelopmentBerlinGermany
  2. 2.Social Psychology and Decision SciencesUniversity of KonstanzKonstanzGermany
  3. 3.Drexel UniversityPhiladelphiaUSA
  4. 4.Rensselaer Polytechnic InstituteTroyUSA

Personalised recommendations