The effect of economic incentives and cooperation messages on user participation in crowdsourced public transport technologies

Abstract

Transport data is crucial for transport planning and operations. Collecting high-quality data has long been challenging due to the difficulty of achieving adequate spatiotemporal coverage within a representative sample. The increasingly integrated use of Information and Communication technologies in transport systems offers an opportunity to collect data using non-traditional methods. Crowdsourcing applications are an example where a community of users shares information about their travel experience. However, crowdsourcing applications depend on a critical mass of users providing feedback. We conducted a large-scale field experiment to examine the effect of economic incentives (a lottery for free trips) and cooperation messages (asking users to help the community) to encourage users to share reports about bus stop conditions using a crowdsourcing app. We found that offering an economic incentive increased the participation rate almost three times compared to a control group, which did not receive any message. This positive effect lasted for several weeks but decreased over time, especially for users who had not made reports prior to the experiment. This incentive also increased the number of reports shared by users. Using a cooperation message, with or without the economic incentive, also increased the participation rate compared to the control group, but adding a cooperation message decreased the effect of a standalone economic incentive.

Appendices

Appendix 1: Notifications received by users on their phones

Messages sent to users. From left to right: Economic incentive, Cooperation message and Both conditions. Users had to press the notification (at the top of the figure) in order to read the full message.¹⁵

Details of the notifications and messages sent to users, translated to English:

	Economic incentive	Cooperation message	Both
Notification	Participate for CLP$10,000 in bip! credits!	Help other commuters!	Help and participate for CLP$10,000 in bip! credits!
	Campaign: Win by reporting your bus stop	Campaign: Help by reporting your bus stop	Campaign: Help and win by reporting your bus stop
Condition-specific message	Participate in this campaign between [dates] by sending reports about a bus stop, and you will be participating in a draw for three bip! credits of CLP$10,000	By participating in this campaign between [dates] by sending reports about a bus stop, you will be helping to improve other peoples’ trips, as well as the public transport system	Participate in this campaign between [dates] by sending reports about a bus stop, and you will be helping to improve other peoples’ trips and the public transport system. You will also be participating in a draw for three bip! charges of CLP$10,000
Common message	Remember that to report a bus stop near your location, you must first click on the bus stop on the map, and then on the yellow button that will appear at the bottom of the screen. Thanks for being part of our community

Appendix 2: Sample size

To identify the sample size required for each treatment, we perform a statistical power analysis based on the results of a pilot. This pilot showed that the cooperation message had the smallest effect compared to the control (0.8 p.p. from 1.1%). Therefore, we required approximately 6000 individuals in each of these experimental conditions to have a 95% statistical power (we added a few hundred people because some phones may have changed or not be working). The rest of the sample was evenly distributed to detect differences between the economic incentive and both conditions, and to be able to split the economic incentive condition into two additional conditions for people who opened the economic incentive notification. For the latter, the message section in the app either repeated the text from the notification (i.e., offering an economic incentive) or also included the text from the cooperation message. We expected the difference to be small, if detectable, because few people may use the message section in the app. Therefore, the final sample was 11,164 for each of these three groups (the Both condition and the two inside the economic incentive one). Consistently, Table 4 shows no significant difference between the texts in the message section for the economic incentive condition (p > 0.2 for all models).

Table 4 Estimation of the effect of each experimental condition on participation rate using a logit model

Appendix 3: Estimation of the effect of each experimental condition on participation rate using a linear probability model

	All	Previous contributors	Previous Lurckers	All with interactions
Economic incentive	0.039***	0.079***	0.031***	0.024***
	(0.003)	(0.010)	(0.003)	(0.003)
	$<0.001$	$<0.001$	$<0.001$	$<0.001$
Cooperation message	0.010**	0.035**	0.004	-0.002
	(0.003)	(0.013)	(0.003)	(0.004)
	0.005	0.005	0.169	0.538
Both	0.027***	0.062***	0.019***	0.013***
	(0.003)	(0.011)	(0.003)	(0.003)
	$<0.001$	$<0.001$	$<0.001$	$<0.001$
Contributors-economic incentive				0.085***
				(0.003)
				$<0.001$
Contributors-cooperation message				0.068***
				(0.006)
				$<0.001$
Contributors-both				0.080***
				(0.005)
				$<0.001$
Constant	0.013***	0.044***	0.007**	0.013***
	(0.002)	(0.009)	(0.002)	(0.002)
	$<0.001$	$<0.001$	0.002	$<0.001$
p values for pairwise comparisons
Economic incentive versus cooperation message	$<0.001$	$<0.001$	$<0.001$	$<0.001$
Economic incentive versus both	$<0.001$	0.043	$<0.001$	$<0.001$
Cooperation message versus both	$<0.001$	0.016	$<0.001$	$<0.001$
R$^2$	0.006	0.008	0.006	0.027
Observations	46,516	8136	38,380	46,516

1. $^+p<0.10$, *$p<0.05$, **$p<0.01$, ***$p<0.001$
2. All columns show standard errors between parentheses and p values in italics

Appendix 4: Estimation of the effect of each experimental condition on the level of user contribution using different models and different analyses

Negative binomial
	Without exclusion	With Poisson	With bus reports
Economic incentive	1.231**	0.235*	0.322
	(0.408)	(0.112)	(0.430)
	0.003	0.036	0.453
Cooperation message	0.517$^+$	0.088	0.685
	(0.289)	(0.139)	(0.693)
	0.074	0.523	0.323
Both	1.526*	0.234*	0.226
	(0.590)	(0.119)	(0.639)
	0.010	0.048	0.724
Constant	$-$ 2.43***	0.752***	0.108
	(0.231)	(0.108)	(1.039)
	$<0.001$	$<0.001$	0.916
Zero-inflated
Economic incentive	$-$ 18.1***	$-$ 1.33***	$-$ 0.28
	(0.348)	(0.118)	(0.106)
	$<0.001$	$<0.001$	0.239
Cooperation message	$-$ 0.63$^+$	$-$ 0.50***	0.106
	(0.371)	(0.143)	(0.164)
	0.089	$<0.001$	0.739
Both	$-$ 1.21$^+$	$-$ 1.05***	0.164
	(0.656)	(0.124)	(3.695)
	0.065	$<0.001$	0.588
Constant	0.345	4.196***	3.695
	(0.317)	(0.113)	(1.030)
	0.277	$<0.001$	$<0.001$
Ln alpha	3.965***		2.454*
	(0.125)		(0.037)
	$<0.001$		0.037
alpha	52.751		11.641
	(6.637)		(13.72)
Log-likelihood	$-$ 11865.09	$-$ 11057.26	$-$ 2431.907
Observations	46,516	46,438	46,516

1. $^+p<0.10$, *$p<0.05$, **$p<0.01$, ***$p<0.001$
2. All columns show standard errors between parentheses and p values in italics

Appendix 5: Estimation of the effect of each experimental condition on the level of user contribution using a Zero-inflated negative binomial model

See Tables 5, 6 and 7.

Table 5 Estimation of effects of each experimental condition on the level of all users’ contributions over time

Table 6 Estimation of effects of each experimental condition on the level of Previous Contributors’ contributions over time

Table 7 Estimation of effects of each experimental condition on the level of previous lurkers’ contributions over time