Experimental Economics

, Volume 14, Issue 1, pp 1–14 | Cite as

Classification of natural language messages using a coordination game

  • Daniel HouserEmail author
  • Erte Xiao


The role of natural language communication in economic exchange has been the focus of substantial experimental analysis. Recently, scholars have taken the important step of investigating whether certain types of communication (e.g., promises) might affect decisions differently than other types of communication. This requires classifying natural language messages. Unfortunately, no broadly-accepted method is available for this purpose. We here describe a coordination game for classification of natural language messages. The game is similar in spirit to the “ESP” game that has proven successful for the classification of tens of millions of internet images. We compare our approach to self-classification as well as to classifications based on a standard content analysis. We argue that our classification game has advantages over those alternative approaches, and that these advantages might stem from the salient rewards earned by our game’s participants.


Classification game Natural language Messages Coordination game Experimental methods 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

10683_2010_9254_MOESM1_ESM.doc (53 kb)
(doc 53.1 KB)


  1. Ben-Ner, A., Putterman, L., & Ren, T. (2007). Lavish returns on cheap talk: non-binding communication in a trust experiment. Mimeo. Google Scholar
  2. Brandts, J., & Cooper, D. (2007). It’s what you say, not what you pay: an experimental study of manager-employee relationships in overcoming coordination failure. Journal of the European Economic Association, 5(6), 1223–1268. CrossRefGoogle Scholar
  3. Camerer, C. (2003). Behavioral game theory. Princeton: Princeton University Press. Google Scholar
  4. Cason, T., & Mui, V.-L. (2007). Communication and coordination in the laboratory collective resistance game. Experimental Economics, 10, 251–267. CrossRefGoogle Scholar
  5. Cason, T., & Mui, V.-L. (2010). Coordinating resistance through communication and repeated interaction. Working paper, Purdue University. Google Scholar
  6. Charness, G., & Dufwenberg, M. (2006). Promises and partnership. Econometrica, 74(6), 1579–1601. CrossRefGoogle Scholar
  7. Charness, G., & Dufwenberg, M. (2010). Bare promises. Economics Letters, 107, 281–283. CrossRefGoogle Scholar
  8. Cooper, D., & Kagel, J. (2005). Are two heads better than one? Team versus individual play in signaling games. American Economic Review, 95(3), 477–509. CrossRefGoogle Scholar
  9. Corazzini, L., Kubez, S., Maréchal, M.A., & Nicolóx, A. (2009). Elections and deceptions: theory and experimental evidence. Working paper. Google Scholar
  10. Crawford, V. (1997). Theory and experiment in the analysis of strategic interactions. In D. Kreps & K. Wallis (Eds.), Advances in economics and econometrics: theory and applications. Seventh world congress (Vol. I). Cambridge: Cambridge University Press. Google Scholar
  11. Dawes, R. M., McTavish, J., & Shaklee, H. (1977). Behavior communication and assumptions about other people’s behavior in a commons dilemma situation. Personality and Social Psychology, 35(1), 1–11. CrossRefGoogle Scholar
  12. Ellingsen, T., & Johannesson, M. (2007). Anticipated verbal feedback induces altruistic behavior. Evolution and Human Behavior. doi: 10.1016/j.evolhumbehav.2007.11.001.
  13. Engle-Warnick, J., & Soroka, S.N. (2009). Harnessing the power of focal points to measure social agreement. Working paper. Google Scholar
  14. Epstein, J. (2007). Generative social science. Princeton: Princeton University Press. Google Scholar
  15. Feiler, L., & Camerer, C. (2009). Code creation in endogenous merger experiments. Economic Inquiry, 48(2), 337–352. CrossRefGoogle Scholar
  16. Fouraker, L., & Sigel, S. (1963). Bargaining behavior. New York: McGraw-Hill. Google Scholar
  17. Hennig-Schmidt, H., Li, Z.-Y., & Yang, C. (2008). Why people reject advantageous offers—non-monotone strategies in ultimatum bargaining. Journal of Economic Behavior and Organization, 65, 373–384. CrossRefGoogle Scholar
  18. Holt, C. (1995). Industrial organization: a survey of laboratory research. In J. Kagel & A. E. Roth (Eds.), The handbook of experimental economics. Princeton: Princeton University Press. Google Scholar
  19. Houser, D., & Wooders, J. (2006). Reputation in auctions: theory, and evidence from eBay. Journal of Economics and Management Science. doi: 10.1111/j.1530-9134.2006.00103.x.
  20. Houser, D., Keane, M., & McCabe, K. (2004). Behavior in a dynamic decision problem: an analysis of experimental evidence using a Bayesian type classification algorithm. Econometrica, 72(3), 781–822. CrossRefGoogle Scholar
  21. Issac, R. M., & Walker, J. (1988). Communication and free riding behavior: the voluntary contributions mechanism. Economic Inquiry 26. doi: 10.1111/j.1465-7295.1988.tb01519.x.
  22. Kimbrough, E., Smith, V., & Wilson, B. (2008). Historical property rights, sociality, and the emergence of impersonal exchange in long distance trade. American Economic Review. doi: 10.1257/aer.98.3.1009.
  23. Krippendorff, K. (2004). Content analysis: an introduction to its methodology (2nd edn.). Thousand Oaks: Sage. Google Scholar
  24. Luce, R. D., & Raiffa, H. (1957). Games and decisions: introduction and critical survey. New York: Wiley. Google Scholar
  25. Ledyard, J. O. (1995). Public goods: a survey of experimental research. In J. Kagel & A. E. Roth (Eds.), The handbook of experimental economics. Princeton: Princeton University Press. Google Scholar
  26. Ochs, J. (1995). Coordination problems. In J. Kagel & A. E. Roth (Eds.), The handbook of experimental economics. Princeton: Princeton University Press. Google Scholar
  27. Schotter, A., & Sopher, B. (2007). Advice and behavior in intergenerational ultimatum games: an experimental approach. Games and Economic Behavior, 58, 365–393. CrossRefGoogle Scholar
  28. Smith, V. (1962). An experimental study of competitive market behavior. Journal of Political Economy, 70(2), 111–137. CrossRefGoogle Scholar
  29. Sutter, M., & Strassmair, C. (2009). Communication, cooperation and collusion in team tournaments—an experimental study. Games and Economic Behavior, 66, 506–525. CrossRefGoogle Scholar
  30. von Ahn, L. (2005). Games with purpose. PhD Dissertation, Carnegie Mellon University. Google Scholar
  31. von Neumann, J., & Morgenstern, O. (1944). Theory of games and economic behavior. Princeton: Princeton University Press. Google Scholar
  32. Xiao, E., & Houser, D. (2005). Emotion expression in human punishment behavior. Proceedings of the National Academy of Sciences, 102(20), 7398–7401. CrossRefGoogle Scholar
  33. Xiao, E., & Houser, D. (2007). Emotion expression and fairness in economic exchange. Manuscript, George Mason University. Google Scholar

Copyright information

© Economic Science Association 2010

Authors and Affiliations

  1. 1.George Mason UniversityFairfaxUSA
  2. 2.Carnegie Mellon UniversityPittsburghUSA

Personalised recommendations