A Rewarding Framework for Crowdsourcing to Increase Privacy Awareness

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12840)


Digital applications typically describe their privacy policy in lengthy and vague documents (called PrPs), but these are rarely read by users, who remain unaware of privacy risks associated with the use of these digital applications. Thus, users need to become more aware of digital applications’ policies and, thus, more confident about their choices. To raise privacy awareness, we implemented the CAP-A portal, a crowdsourcing platform which aggregates knowledge as extracted from PrP documents and motivates users in performing privacy-related tasks. The Rewarding Framework is one of the most critical components of the platform. It enhances user motivation and engagement by combining features from existing successful rewarding theories. In this work, we describe this Rewarding Framework, and show how it supports users to increase their privacy knowledge level by engaging them to perform privacy-related tasks, such as annotating PrP documents in a crowdsourcing environment. The proposed Rewarding Framework was validated by pilots ran in the frame of the European project CAP-A and by a user evaluation focused on its impact in terms of engagement and raising privacy awareness. The results show that the Rewarding Framework improves engagement and motivation, and increases users’ privacy awareness.


Data privacy Privacy awareness Privacy policies GDPR Crowdsourcing Rewarding Collective intelligence 



This work has been supported by the CAP-A project which has received funding from the European Union’s Horizon 2020 research and innovation programme under the NGI_TRUST grant agreement no 825618. The described research activities were also funded by Ghent University, imec, Flanders Innovation & Entrepreneurship (VLAIO). Ruben Verborgh is a postdoctoral fellow of the Research Foundation – Flanders (FWO).


  1. 1.
    Amini, S., Lin, J., Hong, J.I., Lindqvist, J., Zhang, J.: Mobile application evaluation using automation and crowdsourcing (2018).
  2. 2.
    Amini, S.: Analyzing mobile app privacy using computation and crowdsourcing. Ph.D. thesis, Carnegie Mellon University (2014)Google Scholar
  3. 3.
    Antikainen, M.J., Vaataja, H.K.: Rewarding in open innovation communities-how to motivate members. Int. J. Entrepre. Innov. Manag. 11(4), 440–456 (2010)Google Scholar
  4. 4.
    Antón, A.I., Earp, J.B., He, Q., Stufflebeam, W., Bolchini, D., Jensen, C.: Financial privacy policies and the need for standardization. IEEE Secur. Privacy 2(2), 36–45 (2004)CrossRefGoogle Scholar
  5. 5.
    Bergmann, M.: Testing privacy awareness. In: IFIP Summer School on the Future of Identity in the Information Society, pp. 237–253 (2008)Google Scholar
  6. 6.
    Bergram, K., Bezençon, V., Maingot, P., Gjerlufsen, T., Holzer, A.: Digital nudges for privacy awareness: from consent to informed consent? In: ECIS (2020)Google Scholar
  7. 7.
    Bhatia, J., Breaux, T.D., Schaub, F.: Mining privacy goals from privacy policies using hybridized task recomposition. ACM TOSEM 25(3), 1–24 (2016)Google Scholar
  8. 8.
    Chrysakis, I.: CAP-A rewarding framework evaluation - list of questions (2020).
  9. 9.
    Chrysakis, I.: Introduction to CAP-A portal & rewarding evaluation scenario (2020).
  10. 10.
    Chrysakis, I., Flouris, G., Patkos, T., Dimou, A., Verborgh, R.: REWARD: ontology for reward schemes. In: Harth, A., et al. (eds.) ESWC 2020. LNCS, vol. 12124, pp. 56–60. Springer, Cham (2020). Scholar
  11. 11.
    Chrysakis, I., et al.: CAP-A: a suite of tools for data privacy evaluation of mobile applications. In: JURIX (2020)Google Scholar
  12. 12.
    Chrysakis, I., et al.: Evaluating the data privacy of mobile applications through crowdsourcing. In: JURIX (2020)Google Scholar
  13. 13.
    Craig, T., Ludloff, M.E.: Privacy and Big Data: The Players, Regulators, and Stakeholders. O’Reilly Media, Inc., Sebastopol (2011)Google Scholar
  14. 14.
    Diamantopoulou, V., Androutsopoulou, A., Gritzalis, S., Charalabidis, Y.: An assessment of privacy preservation in crowdsourcing approaches: towards GDPR compliance. In: RCIS, pp. 1–9. IEEE (2018)Google Scholar
  15. 15.
    Fatima, R., Yasin, A., Liu, L., Wang, J., Afzal, W., Yasin, A.: Sharing information online rationally: An observation of user privacy concerns and awareness using serious game. J. Inf. Secur. Appl. 48, 102351 (2019)Google Scholar
  16. 16.
    Finnerty, A., Kucherbaev, P., Tranquillini, S., Convertino, G.: Keep it simple: reward and task design in crowdsourcing. In: CHItaly (2013)Google Scholar
  17. 17.
    Flouris, G., et al.: Towards a collective awareness platform for privacy concerns and expectations. In: ODBASE (2018)Google Scholar
  18. 18.
    Grobbink, E., Peach, K.: Combining crowds and machines (2020),
  19. 19.
    Hatamian, M., Kitkowska, A., Korunovska, J., Kirrane, S.: ‘It’s shocking!’: analysing the impact and reactions to the A3: android apps behaviour analyser. In: DBSec, pp. 198–215 (2018)Google Scholar
  20. 20.
    Kani-Zabihi, E., Helmhout, M.: Increasing service users’ privacy awareness by introducing on-line interactive privacy features. In: Laud, P. (ed.) NordSec 2011. LNCS, vol. 7161, pp. 131–148. Springer, Heidelberg (2012). Scholar
  21. 21.
    Kavaliova, M., Virjee, F., Maehle, N., Kleppe, I.A.: Crowdsourcing innovation and product development: gamification as a motivational driver. Cogent Bus. Manag. 3(1), 1128132 (2016)Google Scholar
  22. 22.
    Kokolakis, S.: Privacy attitudes and privacy behaviour: a review of current research on the privacy paradox phenomenon. Comput. Secur. 64, 122–134 (2017)Google Scholar
  23. 23.
    Krishnamurthy, S., Ou, S., Tripathi, A.K.: Acceptance of monetary rewards in open source software development. Res. Policy 43(4), 632–644 (2014)Google Scholar
  24. 24.
    Lee, T.Y., et al.: Experiments on motivational feedback for crowdsourced workers. In: \(7^{th}\) AAAI Conference on Weblogs and Social Media (2013)Google Scholar
  25. 25.
    Liang, F., Yu, W., An, D., Yang, Q., Fu, X., Zhao, W.: A survey on big data market: pricing, trading and protection. IEEE Access 6, 15132–15154 (2018)Google Scholar
  26. 26.
    McCall, M., Voorhees, C.: The drivers of loyalty program success: an organizing framework and research agenda. Cornell Hosp. Q. 51(1), 35–52 (2010)CrossRefGoogle Scholar
  27. 27.
    McGonigal, J.: Reality Is Broken: Why Games Make Us Better and Bow They Can Change the World, Penguin, New York (2011)Google Scholar
  28. 28.
    Morschheuser, B., Hamari, J., Koivisto, J.: Gamification in crowdsourcing: a review. In: \(49^{th}\) Hawaii International Conference on System Sciences (2016)Google Scholar
  29. 29.
    Morschheuser, B., Hamari, J., Koivisto, J., Maedche, A.: Gamified crowdsourcing: conceptualization, literature review, and future agenda. Int. J. Hum. Comput. Stud. 106, 26–43 (2017)CrossRefGoogle Scholar
  30. 30.
    Nejad, N.M., Scerri, S., Lehmann, J.: KnIGHT: mapping privacy policies to GDPR. In: European Knowledge Acquisition Workshop, pp. 258–272 (2018)Google Scholar
  31. 31.
    Newman, G., Wiggins, A., Crall, A., Graham, E., Newman, S., Crowston, K.: The future of citizen science: emerging technologies and shifting paradigms. Front. Ecol. Environ. 10(6), 298–304 (2012)Google Scholar
  32. 32.
    Oltramari, A., Piraviperumal, D., Schaub, F., Wilson, S., Cherivirala, S., Norton, T.B., Russell, N.C., Story, P., Reidenberg, J., Sadeh, N.: Privonto: a semantic framework for the analysis of privacy policies. Seman. Web 9(2), 185–203 (2018)CrossRefGoogle Scholar
  33. 33.
    Pötzsch, S.: Privacy awareness: a means to solve the privacy paradox? In: Matyáš, V., Fischer-Hübner, S., Cvrček, D., Švenda, P. (eds.) Privacy and Identity 2008. IAICT, vol. 298, pp. 226–236. Springer, Heidelberg (2009). Scholar
  34. 34.
    Samaha, S.A., Palmatier, R.W., Dant, R.P.: Poisoning relationships: perceived unfairness in channels of distribution. J. Market. 75(3), 99–117 (2011)CrossRefGoogle Scholar
  35. 35.
    Scekic, O., Truong, H.L., Dustdar, S.: Incentives and rewarding in social computing. Commun. ACM 56(6), 72–82 (2013)CrossRefGoogle Scholar
  36. 36.
    Solove, D.J.: The myth of the privacy paradox. Available at SSRN (2020)Google Scholar
  37. 37.
    Vroom, V.H.: Work and Motivation. Wiley, New York (1964)Google Scholar
  38. 38.
    Wilson, S., et al.: Crowdsourcing annotations for websites’ privacy policies: can it really work? In: Proceedings of WWW-16, pp. 133–143 (2016)Google Scholar
  39. 39.
    Xu, H., Dinev, T., Smith, H.J., Hart, P.J.: Examining the formation of individual’s privacy concerns: toward an integrative view. In: ICIS (2008)Google Scholar
  40. 40.
    Yang, D., Xue, G., Fang, X., Tang, J.: Crowdsourcing to smartphones: incentive mechanism design for mobile phone sensing. In: MobiCom (2012)Google Scholar
  41. 41.
    Zichermann, G., Cunningham, C.: Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps. O’Reilly Media, Inc., Sebastopol (2011)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2021

Authors and Affiliations

  1. 1.FORTH, Institute of Computer ScienceHeraklionGreece
  2. 2.FORTH, PRAXI NetworkHeraklionGreece
  3. 3.IDLab, Department of Electronics and Information SystemsUgent, imecGhentBelgium
  4. 4.imec-SMIT, Vrije Universiteit BrusselBrusselBelgium

Personalised recommendations