Self-selection and variations in the laboratory measurement of other-regarding preferences across subject pools: evidence from one college student and two adult samples

Abstract

We measure the other-regarding behavior in samples from three related populations in the upper Midwest of the United States: college students, non-student adults from the community surrounding the college, and adult trainee truckers in a residential training program. The use of typical experimental economics recruitment procedures made the first two groups substantially self-selected. Because the context reduced the opportunity cost of participating dramatically, 91 % of the adult trainees solicited participated, leaving little scope for self-selection in this sample. We find no differences in the elicited other-regarding preferences between the self-selected adults and the adult trainees, suggesting that selection is unlikely to bias inferences about the prevalence of other-regarding preferences among non-student adult subjects. Our data also reject the more specific hypothesis that approval-seeking subjects are the ones most likely to select into experiments. Finally, we observe a large difference between self-selected college students and self-selected adults: the students appear considerably less pro-social.

This is a preview of subscription content, log in to check access.

Fig. 1

Notes

  1. 1.

    Specifically, subjects were informed that the data was going to the University and not the firm (their new employer), and the role of the University’s Institutional Review Board (IRB) in enforcing the promise of individual confidentiality was explained.

  2. 2.

    With the cooperation of the training school, the study was run on Saturdays that came in the middle of a two-week residential basic training program. Lunch was provided and the buses to and from the trainee’s lodgings arrived at an early hour and left at the end of the afternoon. Only a half day of training activity was scheduled, so trainees were split into two groups and in the morning one did training while the other took part in the study, with the reverse in the afternoon. Those not participating in the study with their group did not have extra training available and had to spend the time in a break room.

  3. 3.

    In Sect. 3.1 we assess the potential implications for our results of the fact that 9.2 % of trainee drivers did not take part. Here we would like to remark that our strategy of running an experiment with trainee truckers as a method to gather data on a relatively non-self-selected sample is similar to the use of classroom experiments with college students by related studies that address the selection issue (e.g., Cleave et al. 2011 or Eckel and Grossman 2000). Running an experiment which is not announced in advance during the course of a regularly scheduled class is meant to minimize the potential for selection, although participation remains voluntary, and a non-random selection of students may be absent, as well. Recruiting an adult sample to voluntary participation is in general likely to make it harder, not easier, to achieve a sample with low self-selection relative to student samples. Students may be more prone to comply with requests made by a relevant authority figure (the professor/experimenter). Even so there is normally still some self-selection (for example, 2 % of potential subjects declined to participate in the classroom experiments of Cleave et al. 2011). There is also a potential cost that is not likely to be as high with adults: student participants in classroom experiments, exposed to an authority figure as the experimenter, may have an increased potential for experimenter demand effects.

  4. 4.

    The full set of activities thus took four and a half hours. The computerized tasks were programmed and implemented with the software z-Tree (Fischbacher 2007).

  5. 5.

    The fixed payments were doubled for Self-Selected Non-Students because on average they faced relatively higher opportunity costs to participation, since they had to come to campus from the surrounding town.

  6. 6.

    Thus, a technically correct label for this game is a sequential and strategic form of the prisoner’s dilemma.

  7. 7.

    Before each decision screen subjects were also asked to predict the behavior of the other participants in the room, and received additional earnings for correct answers, which is why the highest earnings were $16.00 (see Burks et al. 2008).

  8. 8.

    The Unlikely Virtues Scale developed by Patrick et al. (2002), actually consists of 14 items. Due to a programming error, one item was not included in the questionnaire administered to participants in the experiment.

  9. 9.

    Although the subject pools were not intended to be representative of the corresponding population, we report for comparison a summary of socio-demographic characteristics of the population of Morris (residence of two of the three subject pools) for the period 2005–2009: Age (median): 30.3 years; Female: 54.9 %; Non-White or Hispanic: 8.3 %; Years of Education Completed for the population aged 25 or above (mean): 12.9 years; Marital Status of the population aged 15 or above: 33 % married (source: 2005–2009 American Community Survey 5-Year Estimates).

  10. 10.

    In Appendix C we follow an alternative approach to address our research questions, and directly compare the amounts transferred by second-movers across subject pools instead of using these amounts to classify subjects in different ‘preference types’ (we thank an anonymous referee for suggesting this alternative approach). The results of this alternative approach are qualitatively equivalent to those reported in Sect. 3.

  11. 11.

    The experiment also delivers data on unconditional cooperation decisions by subjects in the role of first-mover. Compared to decisions as second-mover, it is more difficult to infer other-regarding motives from first-movers’ choices since these may also reflect considerations about the profitability of cooperating, false-consensus effects, etc. (see, e.g., Gächter et al. forthcoming 2012). For this reason, in the main text we focus on decisions in the role of second-mover, and only briefly discuss here first-mover’s behavior. In the role of first-mover, 74 % of Self-Selected Non-Students, chose to transfer $5 to the second-mover. This is significantly more than the fraction of Self-Selected Students, choosing to do so (55 %, χ 2(1)=6.93, p=0.008). The share of Non-Self-Selected Trainee Truckers sending $5 is 67 %, which is not significantly different from that of Self-Selected Non-Students (χ 2(1)=1.38, p=0.239). Further analysis of the first mover behavior of Non-Self-Selected Trainee Truckers may be found in Burks et al. (2009b).

  12. 12.

    In order to have a well-defined classification of subjects’ cooperativeness one needs to observe their behavior in both subgames. Observing second-movers’ behavior in only one subgame may not be sufficient. For example, observing a second-mover who sends $0 when the first-mover sends $0 does not reveal whether she is a ‘conditional cooperator’ who defects when the first-mover defects, or whether she is instead motivated by material payoff maximization. The use of the strategy method solves this problem by allowing us to observe how a second-mover responds to both possible decisions of the first-mover.

  13. 13.

    See, e.g., Camerer and Fehr (2006); Fehr and Gächter (2000); Fischbacher and Gächter (2010).

  14. 14.

    Formally, if x $0 is the amount that a subject returns when the first-mover sends $0 and x $5 is the amount returned when the first-mover sends $5, we compute the distance of the subject’s decisions from the Free Rider type as \(D_{FR}=\sqrt{(x_{\$0}-0)^{2}+(x_{\$5}-0)^{2}}\), from the Conditional Cooperator type as \(D_{CC}=\sqrt{(x_{\$0}-0)^{2}+(x_{\$ 5}-5)^{2}}\), and from the Unconditional Cooperator type as \(D_{UC}=\sqrt{(x_{\$0}-5)^{2}+(x_{\$5}-5)^{2}}\).

  15. 15.

    All participants classified as Others cannot be classified because they are equally distant from a Free Rider and an Unconditional Cooperator. In the remainder of this sub-section we will focus on the three major cooperation types and ignore the 25 subjects classified as Others.

  16. 16.

    The multinomial logit model relies on the assumption known as the ‘independence of irrelevant alternatives’ (IIA) whereby introducing or removing any category type from our classification should have the same proportional impact on the probability of the other categories. We tested the IIA assumption using the two tests presented by Long and Freese (2006), the Hausman test and the Small-Hsiao test. The results show no evidence that the IIA assumption has been violated (these tests results are available from the authors upon request).

  17. 17.

    For 29 Non-Self-Selected Trainee Truckers there are missing data for some of the items composing the Unlikely Virtues Scale. To compute a score for these subjects we impute the neutral midpoint of the scale for those items whose answers are missing. Results do not change if we conduct the analysis excluding these 29 subjects.

  18. 18.

    A two-sided Mann-Whitney-U-test shows that the difference is not statistically significant (p=0.102).

  19. 19.

    Although it is standard to simply sum the responses to summarize the Unlikely Virtues Scale, we also conducted a factor analysis. The analysis resulted in one eigenvalue above one and using the resulting factor scores we find similar results: two-sided Mann-Whitney-U-tests reveal that socially desirable responding is somewhat more prevalent among Non-Self-Selected Trainee Truckers than Self-Selected Non-Students (p=0.094), and more prevalent among Self-Selected Non-Students than Self-Selected Students (p=0.000).

  20. 20.

    The regression also shows that approval-seeking is positively correlated with age (p=0.002) and with the dummy variable for Non-White or Hispanic subjects (p=0.000). Full regressions results are available upon request.

  21. 21.

    Another interesting question is whether there is a link between the need for social approval and other-regarding preferences. To address this question we re-ran the multinomial logit regression reported in Table 3 (Model II) adding the Unlikely Virtues Scale scores to the list of explanatory variables. We find that, if anything, social desirability slightly increases (by about 3 %) the odds of being classified as a Free Rider rather than a Conditional Cooperator, and the effect is significant at the 10 % level. None of the other comparisons is statistically significant.

  22. 22.

    Another strand of the literature has examined the implications of selection for the elicitation of risk preferences, see, e.g., Harrison et al. (2009); von Gaudecker et al. (2011).

  23. 23.

    Interestingly, offers and acceptance rates of students and non-students do not differ if the non-student sample is restricted to a sub-group of participants in the same age groups as students.

  24. 24.

    However, differences in contributions vanish once participants’ socio-economic characteristics are accounted for.

  25. 25.

    Belot et al. (2010) also compare the choices of students and non-students in other games where other-regarding preferences may be relevant (a dictator game and a trust game). They find that non-students are more other-regarding than students in these games as well.

  26. 26.

    Also related is Cardenas (2005) who conducts common pool resources game experiments with students and villagers in Colombia, and finds that villagers are more cooperative than students.

  27. 27.

    Burks et al. (2009a) label Unconditional Cooperators “Altruists” and Free Riders “Egoists.”

  28. 28.

    Where the populations of advanced industrial societies fall in the full range of behavior typical of humans as a species is an open question that our data do not address; see, for example, the discussion in Henrich et al. (2010).

References

  1. Bellemare, C., & Kroger, S. (2007). On representative social capital. European Economic Review, 51(1), 183–202. http://www.elsevier.com/wps/find/journaldescription.cws_home/505541/description#description.

    Article  Google Scholar 

  2. Belot, M., Duch, R., & Miller, L. (2010). Who should be called to the lab? A comprehensive comparison of students and non-students in classic experimental games. University of Oxford, Nuffield College Discussion Papers (2010-001).

  3. Burks, S., Carpenter, J., Götte, L., Monaco, K., Porter, K., & Rustichini, A. (2008). Using behavioral economic field experiments at a firm: the context and design of the truckers and turnover project. In S. Bender, J. Lane, K. Shaw, F. Andersson & T. von Wachter (Eds.), The analysis of firms and employees: quantitative and qualitative approaches (pp. 45–106). Chicago: NBER and University of Chicago.

    Google Scholar 

  4. Burks, S., Carpenter, J., & Goette, L. (2009a). Performance pay and worker cooperation: evidence from an artefactual field experiment. Journal of Economic Behavior and Organization, 70(3), 458–469. http://www.elsevier.com/wps/find/journaldescription.cws_home/505559/description#description.

    Article  Google Scholar 

  5. Burks, S., Carpenter, J., Götte, L., & Rustichini, A. (2009b). Cognitive skills affect economic preferences, social awareness, and job attachment. Proceedings of the National Academy of Science (USA), 106(19), 7745–7750.

    Article  Google Scholar 

  6. Camerer, C. F. (2003). Roundtable series in behaviorial economics. Behavioral game theory: experiments in strategic interaction. Princeton: Princeton University Press.

    Google Scholar 

  7. Camerer, C. F., & Fehr, E. (2006). When does “economic man” dominate social behavior? Science, 311(5757), 47–52. doi:10.1126/science.1110600.

    Article  Google Scholar 

  8. Cardenas, J. C. (2005). Groups, commons and regulations: experiments with villagers and students in Colombia. In B. Agarwal & A. Vercelli (Eds.), Psychology, rationality and economic behaviour: challenging standard assumptions. Basingstoke: Palgrave Macmillan.

    Google Scholar 

  9. Carpenter, J., & Seki, E. (2011). Do social preferences increase productivity? Field experimental evidence from Fishermen in Toyama Bay. Economic Inquiry, 49(2), 612–630. http://www.blackwell-synergy.com/loi/ecin/.

    Article  Google Scholar 

  10. Carpenter, J., Burks, S., & Verhoogen, E. (2005). Comparing students to workers: the effects of social framing on behavior in distribution games. Research in Experimental Economics, 10, 261–290.

    Article  Google Scholar 

  11. Carpenter, J., Connolly, C., & Myers, C. (2008). Altruistic behavior in a representative dictator experiment. Experimental Economics, 11(3), 282–298. http://www.springerlink.com/link.asp?id=102888.

    Article  Google Scholar 

  12. Cleave, B., Nikiforakis, N., & Slonim, R. (2011). Is there selection bias in laboratory experiments? Institute for the Study of Labor Discussion Papers (#5488), 1-34.

  13. Eckel, C., & Grossman, P. (2000). Volunteers and pseudo-volunteers: the effect of recruitment method in dictator experiments. Experimental Economics, 3(2), 107–120. http://www.springerlink.com/link.asp?id=102888.

    Google Scholar 

  14. Falk, A., Meier, S., & Zehnder, C. (2012, forthcoming). Do lab experiments misrepresent social preferences? The case of self-selected student samples. Journal of the European Economic Association.

  15. Fehr, E., & Gächter, S. (2000). Cooperation and punishment in public goods experiments. American Economic Review, 90(4), 980–994.

    Article  Google Scholar 

  16. Fehr, E., & List, J. (2004). The hidden costs and returns of incentives–trust and trustworthiness among CEOs. Journal of the European Economic Association, 2(5), 743–771. http://www.mitpressjournals.org/loi/jeea.

    Article  Google Scholar 

  17. Fehr, E., & Schmidt, K. (2006). The economics of fairness, reciprocity and altruism—experimental evidence and new theories. In S. Kolm & J. M. Ythier (Eds.), Handbook of the economics of giving, reciprocity and altruism (Vol. 1, pp. 615–691). Amsterdam: Elsevier.

    Google Scholar 

  18. Fischbacher, U. (2007). Z-tree: Zurich toolbox for ready-made economic experiments. Experimental Economics, 10(2), 171–178. http://www.springerlink.com/link.asp?id=102888.

    Article  Google Scholar 

  19. Fischbacher, U., & Gächter, S. (2010). Social preferences, beliefs, and the dynamics of free riding in public goods experiments. American Economic Review, 100(1), 541–556. http://www.aeaweb.org/aer/.

    Article  Google Scholar 

  20. Fischbacher, U., Gächter, S., & Fehr, E. (2001). Are people conditionally cooperative? Evidence from a public goods experiment. Economics Letters, 71(3), 397–404. http://www.elsevier.com/wps/find/journaldescription.cws_home/505574/description#description.

    Article  Google Scholar 

  21. Gächter, S., Herrmann, B., & Thoni, C. (2004). Trust, voluntary cooperation, and socio-economic background: survey and experimental evidence. Journal of Economic Behavior and Organization, 55(4), 505–531. http://www.elsevier.com/wps/find/journaldescription.cws_home/505559/description#description.

    Article  Google Scholar 

  22. Gächter, S., Nosenzo, D., Renner, E., & Sefton, M. (2012, forthcoming). Who makes a good leader? Cooperativeness, optimism and leading-by-example. Economic Inquiry. doi:10.1111/j.1465-7295.2010.00295.x.

  23. Güth, W., Schmidt, C., & Sutter, M. (2007). Bargaining outside the lab—a newspaper experiment of a three-person ultimatum game. Economic Journal, 117(518), 449–469. http://www.blackwellpublishing.com/journal.asp?ref=0013-0133.

    Article  Google Scholar 

  24. Harrison, G., Lau, M., & Rutstrom, E. (2009). Risk attitudes, randomization to treatment, and self-selection into experiments. Journal of Economic Behavior and Organization, 70(3), 498–507. http://www.elsevier.com/wps/find/journaldescription.cws_home/505559/description#description.

    Article  Google Scholar 

  25. Henrich, J., Heine, S., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2/3), 1–75. doi:10.1017/S0140525X0999152X.

    Google Scholar 

  26. Herrmann, B., & Thoni, C. (2009). Measuring conditional cooperation: a replication study in Russia. Experimental Economics, 12(1), 87–92. http://www.springerlink.com/link.asp?id=102888.

    Article  Google Scholar 

  27. Hoffman, M., & Morgan, J. (2011). Who’s naughty? Who’s nice? Social preferences in online industries. University of California, Berkeley (unpublished manuscript).

  28. Kocher, M., Cherry, T., Kroll, S., Netzer, R., & Sutter, M. (2008). Conditional cooperation on three continents. Economics Letters, 101(3), 175–178. http://www.elsevier.com/wps/find/journaldescription.cws_home/505574/description#description.

    Article  Google Scholar 

  29. Levitt, S., & List, J. (2007). What do laboratory experiments measuring social preferences reveal about the real world? Journal of Economic Perspectives, 21(2), 153–174. http://www.aeaweb.org/jep/.

    Article  Google Scholar 

  30. List, J. (2004). Young, selfish and male: field evidence of social preferences. The Economic Journal, 114, 121–149.

    Article  Google Scholar 

  31. Long, S. J., & Freese, J. (2006). Regression models for categorical dependent variables using Stata (2nd ed.). College Station: Stata Press.

    Google Scholar 

  32. Patrick, C., Curtin, J., & Tellegen, A. (2002). Development and validation of a brief form of the multidimensional personality questionnaire. Psychological Assessment, 14(2), 150–164.

    Article  Google Scholar 

  33. Rosenthal, R., & Rosnow, R. (1969). The volunteer subject. In R. Rosenthal & R. L. Rosnow (Eds.), Artifact in behavioral research (pp. 61–118). New York: Academic Press.

    Google Scholar 

  34. Rustagi, D., Engel, S., & Kosfeld, M. (2010). Conditional cooperation and costly monitoring explain success in forest commons management. Science, 330(6006), 961–965. doi:10.1126/science.1193649.

    Article  Google Scholar 

  35. Stoop, J., Noussair, C. N., & van Soest, D. (2009). From the lab to the field: cooperation among fishermen. Munich Personal RePEc Archive (#28924).

  36. von Gaudecker, H.-M., van Soest, A., & Wengström, E. (2011). Experts in experiments: how selection matters for estimated distributions of risk preferences. Institute for the Study of Labor Discussion Papers (#5575).

  37. Voors, M., Bulte, E., Kontoleon, A., List, J., & Turley, T. (2011). Using artefactual field experiments to learn about the incentives for sustainable forest use in developing economies. American Economic Review, 101(3), 329–333. http://www.aeaweb.org/aer/.

    Article  Google Scholar 

  38. Zizzo, D. (2010). Experimenter demand effects in economic experiments. Experimental Economics, 13(1), 75–98.

    Article  Google Scholar 

Download references

Acknowledgements

We thank the editor and two anonymous referees for useful comments. We received helpful comments from Simon Gächter, John Galbraith, Herbert Gintis, Nikos Nikiforakis and participants at the 2011 International Meeting of the Economic Science Association in Chicago (IL). The Truckers and Turnover Project acknowledges financial and in-kind support from the cooperating firm, and financial support from the MacArthur Foundation’s Research Network on the Nature and Origin of Norms and Preferences, the Sloan Foundation’s Industry Studies Program, the Trucking Industry Program at Georgia Institute of Technology, the University of Nottingham, and the University of Minnesota, Morris. Götte acknowledges support from the Federal Reserve Bank of Boston, and Nosenzo from the Leverhulme Trust (ECF/2010/0636). The views expressed are those of the authors, and do not necessarily reflect those of the supporting entities.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Daniele Nosenzo.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(PDF 1.3 MB)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Anderson, J., Burks, S.V., Carpenter, J. et al. Self-selection and variations in the laboratory measurement of other-regarding preferences across subject pools: evidence from one college student and two adult samples. Exp Econ 16, 170–189 (2013). https://doi.org/10.1007/s10683-012-9327-7

Download citation

Keywords

  • Methodology
  • Selection bias
  • Laboratory experiment
  • Field experiment
  • Other-regarding behavior
  • Social preferences
  • Prisoner’s dilemma
  • Truckload
  • Trucker

JEL Classification

  • C90
  • D03