Skip to main content
SpringerLink
Log in

Understanding decision processes in guessing games: a protocol analysis approach

  • Original Paper
  • Published:
Journal of the Economic Science Association Aims and scope Submit manuscript

Abstract

Protocol analysis, in the form of concurrent verbal ‘thinking aloud’ reports, is a method of collecting and analyzing data about cognitive processes. This approach can help economists in evaluating competing theories of behavior and in categorizing heterogeneity of thinking patterns. As a proof of concept, I tested this method in the context of a guessing game. I found that concurrent think aloud protocols can inform us about individual’s thought processes without affecting decisions. The method allowed me to identify game theoretic thinking and heterogeneous approaches to unravelling the guessing game. The think aloud protocol is inexpensive and scalable, and it is a useful tool for identifying empirical regularities regarding decision processes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. Confusion happens when the environment the researcher has setup to observe behaviors is not the same environment perceived by the subjects.

  2. There are many important contributions on communication in experimental settings. Here I narrowed my attention to works that more explicitly used verbal reports to trace decision processes. See Brandts et al. (2019) for a review of the communication literature in experiments.

  3. A criticism of TAP is that participants may feel watched and/or judged by the experimenter when expressing their thoughts. The additional incentive to elicit thought processes could have accentuated this problem. However, after having run pilot experiments, I realized that subjects at my university needed incentives to write. The instructions are included in the supplemental materials.

  4. CRT − 16.161 (5.852); z_errWM 9.598 (4.550); z_Imotiv − 6.617(5.818). Robust SE in parentheses.

  5. Disagreements had to do mostly with D > 2 and L > 2 classifications. That said, the method is not devoid of error, but I devised the majority voting rule to reduce error in classifying subjects.

  6. This particular individual’s second round choice was 70!

  7. See Penczynski (2018) for an exploration of the use of machine learning techniques for analyzing communication content from experiments.

References

  • Agranov, M., Caplin, A., & Tergiman, C. (2015). Naive play and the process of choice in guessing games. Journal of the Economic Science Association, 1(2), 146–157.

    Article  Google Scholar 

  • Bosch-Domenech, A., Montalvo, J. G., Nagel, R., & Satorra, A. (2002). One, two, (three), infinity: Newspaper and lab beauty-contest experiments. American Economic Review, 92(5), 1687–1701.

    Article  Google Scholar 

  • Brandts, J., Cooper, D., & Rott, C. (2019). Communication in laboratory experiments. In A. Schram & A. Ule (Eds.), Handbook of research methods and applications in experimental economics. Cheltenham: Edward Elgar Publishing Ltd.

    Google Scholar 

  • Brown, T. C. (2005). Loss aversion without the endowment effect, and other explanations for the WTA–WTP disparity. Journal of Economic Behavior & Organization, 57(3), 367–379.

    Article  Google Scholar 

  • Bühren, C., Frank, B., Krabel, S., & Werner, A. (2012). Decision-making in competitive framings—Strategic behavior of chess players in mini-ultimatum game chess puzzles. Economics Letters, 115(3), 356–358.

    Article  Google Scholar 

  • Burchardi, K. B., & Penczynski, S. (2014). Out of your mind: Eliciting individual reasoning in one shot games. Games and Economic Behavior, 84, 39–57.

    Article  Google Scholar 

  • Cacioppo, J. T., Petty, R. E., & Feng Kao, C. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 48(3), 306–307.

    Article  Google Scholar 

  • Camerer, C. F., Ho, T. H., & Chong, J. K. (2004). A cognitive hierarchy model of games. The Quarterly Journal of Economics, 119(3), 861–898.

    Article  Google Scholar 

  • Chou, E., McConnell, M., Nagel, R., & Plott, C. R. (2009). The control of game form recognition in experiments: Understanding dominant strategy failures in a simple two person “guessing” game. Experimental Economics, 12(2), 159–179.

    Article  Google Scholar 

  • Cooper, D. J., & Kagel, J. (2005). Are two heads better than one? Team versus individual play in signaling games. American Economic Review, 95(3), 477–509.

    Article  Google Scholar 

  • Cooper, D. J., & Kagel, J. (2016). A failure to communicate: An experimental investigation of the effects of advice on strategic play. European Economic Review, 82, 24–45.

    Article  Google Scholar 

  • Costa-Gomes, M. A., & Crawford, V. P. (2006). Cognition and behavior in two-person guessing games: An experimental study. American Economic Review, 96(5), 1737–1768.

    Article  Google Scholar 

  • Duffy, J., & Nagel, R. (1997). On the robustness of behavior in experimental beauty-contest games. Economic Journal, 107, 1684–1700.

    Article  Google Scholar 

  • Earl, E., Friesen, L., & Shadforth, C. (2019). Elusive optima: A process tracing analysis of procedural rationality in mobile phone connection plan choices. Journal of Economic Behavior & Organization, 161, 303–322. https://doi.org/10.1016/j.jebo.2019.03.021.

    Article  Google Scholar 

  • Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87(3), 215–251.

    Article  Google Scholar 

  • Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.

    Google Scholar 

  • Fox, M. C., Ericsson, K. A., & Best, R. (2011). Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods. Psychological Bulletin, 137(2), 316–344. https://doi.org/10.1037/a0021663.

    Article  Google Scholar 

  • Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic perspectives, 19(4), 25–42.

    Article  Google Scholar 

  • Grosskopf, B., & Nagel, R. (2008). The two-person beauty contest. Games and Economic Behavior, 62(1), 93–99.

    Article  Google Scholar 

  • Highhouse, S. (1994). A verbal protocol analysis of choice under ambiguity. Journal of Economic Psychology, 15(4), 621–635.

    Article  Google Scholar 

  • Ho, T.-H., Camerer, C. F., & Weigelt, K. (1998). Iterated dominance and iterated best response in experimental ‘p-beauty contests’. American Economic Review, 88, 947–969.

    Google Scholar 

  • Johnson, P. E., Durán, A. S., Hassebrock, F., Moller, J., Prietula, M., Feltovich, P. J., et al. (1981). Expertise and error in diagnostic reasoning. Cognitive Science, 5(3), 235–283.

    Article  Google Scholar 

  • Larkin, J., McDermott, J., Simon, D. P., & Simon, H. A. (1980). Expert and novice performance in solving physics problems. Science, 208(4450), 1335–1342.

    Article  Google Scholar 

  • Nagel, R. (1995). Experimental results on interactive competitive guessing. The American Economic Review, 85, 1313–1326.

    Google Scholar 

  • Nisbett, R. E., & Wilson, T. (1977). Telling more than we can know: Verbal reports and mental processes. Psychological Review, 84, 231–259.

    Article  Google Scholar 

  • Penczynski, S. P. (2018). Using machine learning for communication classification. Experimental Economics. https://doi.org/10.1007/s10683-018-09600-z.

    Article  Google Scholar 

  • Ranyard, R., Hinkley, L., Williamson, J., & Mchugh, S. (2006). The role of mental accounting in consumer credit decision processes. Journal of Economic Psychology, 27, 571–588.

    Article  Google Scholar 

  • Ranyard, R., & Svenson, O. (2011). Verbal data and decision process analysis. In M. Schulte-Mecklenbeck, A. Kühberger, & R. Ranyard (Eds.), A handbook of process tracing methods for decision research: A critical review and user’s guide (pp. 115–130). New York: Taylor & Francis.

    Google Scholar 

  • Russo, E. J., Johnson, E. J., & Stephens, D. L. (1989). The validity of verbal protocols. Memory and Cognition, 17, 759–769.

    Article  Google Scholar 

  • Ryan, M., Watson, V., & Entwistle, V. (2009). Rationalising the ‘irrational’: A think aloud study of discrete choice experiment responses. Health Economics, 18(3), 321–336.

    Article  Google Scholar 

  • Schkade, D., & Payne, J. (1994). How People respond to contingent valuation questions: A verbal protocol analysis of willingness to pay for an environmental regulation. Journal of Environmental Economics and Management, 26, 88–109.

    Article  Google Scholar 

  • Smith, R. D. (2007). The relationship between reliability and size of willingness-to-pay values: A qualitative insight. Health Economics, 16(2), 211–216.

    Article  Google Scholar 

  • Stahl, D. O. (1996). Boundedly Rational rule learning in a guessing game. Games and Economic Behavior, 16(2), 303–330.

    Article  Google Scholar 

  • Stahl, D. O., & Wilson, P. W. (1995). On players’ models of other players: Theory and experimental evidence. Games and Economic Behavior, 10(1), 218–254.

    Article  Google Scholar 

  • Turner, M., & Engle, R. W. (1989). Is working memory capacity task dependent? Journal of Memory and Language, 28(2), 127–154.

    Article  Google Scholar 

  • Unsworth, N., Heitz, R. P., Schrock, J. C., & Engel, R. (2005). An automated version of the operation span task. Behavior Research Methods, 37(4), 498–505.

    Article  Google Scholar 

  • Weber, R. A. (2003). ‘Learning’ with no feedback in a competitive guessing game. Games and Economic Behavior, 44(1), 134–144.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Claremont Graduate University, Claremont, USA

    C. Mónica Capra

Authors
  1. C. Mónica Capra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to C. Mónica Capra.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 2244 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Capra, C.M. Understanding decision processes in guessing games: a protocol analysis approach. J Econ Sci Assoc 5, 123–135 (2019). https://doi.org/10.1007/s40881-019-00074-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40881-019-00074-0

Keywords

JEL Classification

Search

Navigation