Skip to main content

Table 1. Experiment results - summary of experiment results

From: Please Stay vs Let’s Play: Social Pressure Incentives in Paid Collaborative Crowdsourcing

Experiment results
  Low threshold High threshold
Traditional Collaborative Traditional Collaborative Soc’l pressure
Total workers 402 365 514 499 508
Total tags 21,538 48,171 27,652 108,950 158,716
Unique images tagged 200 200 2,196 2,200 2,200
Inter-annotator agreement 29.44 % 34.55% 14.26 % 25.82 % 29.35%
ESP tags agreement 41.26 % 25.39 % 43.96 % 37.94 % 40.11 %
Avg images tagged/person 26.68 (SD = 38.21) 9.77 (SD = 13.23) 26.75 (SD = 42.07) 25.05 (SD = 17.92) 29.00 (SD = 28.30)
Avg tags/person 53.57 131.97 53.80 218.34 312.43
Avg new tags/person 2.78 (1,117/402) 8.69 (3,172/365) 1.80 (925/514) 11.83 (5,903/499) 16.21 (8,236/508)