Turking overtime: how participant characteristics and behavior vary over time and day on Amazon Mechanical Turk
- 431 Downloads
Online experiments allow researchers to collect datasets at times not typical of laboratory studies. We recruit 2336 participants from Amazon Mechanical Turk to examine if participant characteristics and behaviors differ depending on whether the experiment is conducted during the day versus night, and on weekdays versus weekends. Participants make incentivized decisions involving prosociality, punishment, and discounting, and complete a demographic and personality survey. We find no time or day differences in behavior, but do find that participants at nights and on weekends are less experienced with online studies; on weekends are less reflective; and at night are less conscientious and more neurotic. These results are largely robust to finer-grained measures of time and day. We also find that those who participated earlier in the course of the study are more experienced, reflective, and agreeable, but less charitable than later participants.
KeywordsCooperation Honesty Decision-making Time of day MTurk Self-control
JEL ClassificationC80 C90
The authors gratefully acknowledge funding from the Templeton World Charity Foundation (Grant No. TWCF0209), the Defense Advanced Research Projects Agency NGS2 program (Grant No. D17AC00005), and the National Institutions of Health (Grant No. P30-AG034420). They also thank Becky Fortgang, the Editor, and two anonymous reviewers for their valuable feedback, and SJ Language Services for copyediting.
- Casey, L. S., Chandler, J., Levine, A. S., Proctor, A., & Strolovitch, D. Z. (2016). Intertemporal differences among MTurk worker demographics. https://osf.io/preprints/psyarxiv/8352x.
- Deetlefs, J., Chylinski, M., & Ortmann, A. (2015). MTurk ‘Unscrubbed’: Exploring the good, the ‘super’, and the unreliable on Amazon’s mechanical turk. http://ssrn.com/abstract=2654056.
- Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on amazon mechanical turk. Judgment and Decision Making, 5(5), 411–419.Google Scholar