Skip to main content

Reputation modelling in Citizen Science for environmental acoustic data analysis


Citizen Science projects are initiatives in which members of the general public participate in scientific research projects and perform or manage research-related tasks such as data collection and/or data annotation. Citizen Science is technologically possible and scientifically significant. However, as the gathered information is from the crowd, the data reliability is always hard to manage. Data reliability issues are significant to the domain of Citizen Science due to the quantity and diversity of people and devices involved. Participants may submit low quality, misleading, inaccurate, or even malicious data. Therefore, finding a way to improve the data reliability has become an urgent demand. This study aims to investigate techniques to enhance the reliability of data contributed by general citizens in scientific research projects especially for acoustic sensing projects. In particular, we propose to design a reputation framework to enhance data reliability and also investigate some critical elements that should be aware of during developing and designing new reputation systems.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7


  • Abdulmonem A, Hunter J (2010) Enhancing the quality and trust of citizen science data. In: Proceedings of the 6th IEEE international conference on e-science, pp 81–88

  • Abdul-Rahman A, Hailes S (2000) Supporting trust in virtual communities. In: Proceedings of the 33rd annual Hawaii international conference on system sciences, pp 1–9

  • Acevedo MA, Villanueva-Rivera LJ (2006) Using automated digital recording systems as effective tools for the monitoring of birds and amphibians. Wildl Soc Bull 34(1):211–214

    Article  Google Scholar 

  • Acevedo MA, Corrada-Bravo CJ, Corrada-Bravo H, Villanueva-Rivera LJ, Aide TM (2009) Automated classification of bird and amphibian calls using machine learning: a comparison of methods. Ecol Inform 4(4):206–214

    Article  Google Scholar 

  • Blaze M, Feigenbaum J, Lacy J (1996) Decentralized trust management. In: Proceedings of the 17th IEEE symposium on security and privacy, pp 164–173

  • Blaze M, Ioannidis J, Keromytis AD (2003) Experience with the keynote trust management system: applications and future directions. In: Proceedings of the 1st international conference on trust management, pp 284–300

  • Bonney R, Cooper CB, Dickinson J, Kelling S, Phillips T, Rosenberg KV et al (2009) Citizen science: a developing tool for expanding science knowledge and scientific literacy. Bioscience 59(11):977–984

    Article  Google Scholar 

  • Brogan C, Smith J (2009) Trust agents: using the web to build influence, improve reputation, and earn trust. Wiley, New York

  • Burke JA, Estrin D, Hansen M, Parker A, Ramanathan N, Reddy S et al (2006) Participatory Sensing. In: Proceedings of the sensor web workshop, ACM SenSys, pp 117–134

  • Chandola V, Banerjee A, Kumar V (2009) Anomaly detection: a survey. ACM Comput Surv 41(3):1–58

    Article  Google Scholar 

  • Chen L, Nayak R (2012) Leveraging the network information for evaluating answer quality in a collaborative question answering portal. Soc Netw Anal Min 2(3):197–215

    Google Scholar 

  • Cooper CB, Dickinson J, Phillips T, Bonney R (2007) Citizen Science as a tool for conservation in residential ecosystems. Ecol Soc 12(2):11

    Google Scholar 

  • Cooper S, Khatib F, Treuille A, Barbero J, Lee J, Beenen M, Players F (2010) Predicting protein structures with a multiplayer online game. Nature 466(7307):756–760

    Article  Google Scholar 

  • Cuff D, Hansen M, Kang J (2008) Urban sensing: out of the woods. Comm ACM 51(3):24–33

    Article  Google Scholar 

  • Davis JG (2011) From Crowdsourcing to Crowdservicing. IEEE Internet Comput 15(3):92–94

    Article  Google Scholar 

  • Delaney D, Sperling C, Adams C, Leung B (2008) Marine invasive species: validation of citizen science and implications for national monitoring networks. Biol Invasions 10(1):117–128

    Article  Google Scholar 

  • Dickinson J, Zuckerberg B, Bonter DN (2010) Citizen science as an ecological research tool: challenges and benefits. Annu Rev Ecol Evol Syst 41(1):149–172

    Article  Google Scholar 

  • Dong Y, Kanhere S, Chou C, Bulusu N (2008) Automatic collection of fuel prices from a network of mobile cameras. In: Nikoletseas S, Chlebus B, Johnson D, Krishnamachari B (eds) Distributed computing in sensor systems, vol 5067. Springer, Berlin, pp 140–156

  • Estrin DL (2010) Participatory Sensing: applications and architecture. In: Proceedings of the 8th international conference on mobile systems, applications, and services, ACM Mobisys’10, pp 3–4

  • Etalle S, Hartog JD, Marsh S (2007) Trust and punishment. In: Proceedings of the 1st international conference on autonomic computing and communication systems, pp 1–6

  • Gage SH, Napoletano BM, Cooper MC (2001) Assessment of ecosystem biodiversity by acoustic diversity indices. J Acoust Soc Am 109(5):2430

    Google Scholar 

  • Galaxy Zoo (2010) The story so far. Retrieved from;

  • Galloway AWE, Tudor MT, Haegen WMV (2006) The reliability of Citizen Science: a case study of Oregon White Oak Stand surveys. Wildl Soc Bull 34(5):1425–1429

    Google Scholar 

  • Golbeck J (2005) Computing and applying trust in Web-based social networks. Thesis for the degree of doctor of philosophy, University of Maryland. Retrieved from

  • Golbeck J (2009) Trust and nuanced profile similarity in online social networks. ACM Trans Web 3(4):1–33

    Article  Google Scholar 

  • Goldman J, Shilton K, Burke J, Estrin D, Hansen M, Ramanathan N, Reddy S et al (2009) Participatory Sensing: a citizen-powered approach to illuminating the patterns that shape our world. Office 14(5). Retrieved from

  • Horlick-Jones T (1997) Citizen Science: a study of people, expertise and sustainable development. Sci Technol Hum Values 22(4):525–527

    Article  Google Scholar 

  • Houser D, Wooders J (2006) Reputation in auctions: theory, and evidence from eBay. J Econ Manag Strategy 15(2):353–369

    Article  Google Scholar 

  • Howie J (2005) Secure your wireless network. Windows IT Security 5(11):4

    Google Scholar 

  • Hsueh P-Y, Melville P, Sindhwani V (2009) Data quality from crowdsourcing: a study of annotation selection criteria. In: Proceedings of the NAACL HLT 2009 workshop on active learning for natural language processing, pp 27–35

  • Huang KL, Kanhere SS, Hu W (2010) Preserving privacy in Participatory Sensing systems. Comput Commun 33(11):1266–1280

    Article  Google Scholar 

  • Ibrahim E, Md Noor N, Mehad S (2007) Seeing is not believing but interpreting, inducing trust through institutional symbolism: a conceptual framework for online trust building in a web mediated information environment. In: Smith M, Salvendy G (eds) Human interface and the management of information. interacting in information environments, vol 4558. Springer, Berlin, pp 64–73

  • IUCN (2011) Numbers of threatened species by major groups of organisms (1996–2011). IUCN Red List version 2011. Retrieved from

  • Jain AK, Duin RPW, Jianchang M (2000) Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intel 22(1):4–37

    Article  Google Scholar 

  • Josang A, Keser C, Dimitrakos T (2005) Can we manage trust? In: Herrmann P, Issarny V, Shiu S (eds) Trust management, vol 3477, Springer, Berlin, pp 93–107

  • Lane ND, Eisenman SB, Musolesi M, Miluzzo E, Campbell AT (2008) Urban sensing systems: opportunistic or participatory? In: Proceedings of the 9th workshop on mobile computing systems and applications, ACM, pp 11–16

  • Lenders V, Koukoumidis E, Zhang P, Martonosi M (2008) Location-based trust for mobile user-generated content: Applications, challenges and implementations. In: Proceedings of the 9th workshop on mobile computing systems and applications, ACM, pp 60–64

  • Lintott CJ, Schawinski K, Slosar A, Land K, Bamford S, Thomas D, Murray P (2008) Galaxy Zoo: morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey. Mon Notices R Astron Soc 389(3):1179–1189

    Article  Google Scholar 

  • Liu L, Shi W (2010) Trust and reputation management. IEEE Internet Comput 14(5):10–13

    Article  Google Scholar 

  • Mohan P, Padmanabhan VN, Ramjee R (2008) Nericell: rich monitoring of road and traffic conditions using mobile smartphones. In: Proceedings of the 6th ACM conference on embedded network sensor systems, ACM, pp 323–336

  • Mun M, Reddy S, Shilton K, Yau N, Burke J, Estrin D et al (2009) PEIR, the personal environmental impact report, as a platform for Participatory Sensing systems research. In: Proceedings of the 7th international conference on mobile systems, applications, and services, ACM, pp 55–68

  • Norman DA (2009) The way I see it: compliance and tolerance. Interactions 16(3):61–65

    Article  Google Scholar 

  • Nov O, Arazy O, Anderson D (2011) Dusting for science: motivation and participation of digital citizen science volunteers. In: Proceedings of the 2011 iConference, ACM, pp 68–74

  • Patcha A, Park J-M (2007) An overview of anomaly detection techniques: existing solutions and latest technological trends. Comput Netw 51(12):3448–3470

    Article  Google Scholar 

  • Paxton M, Benford S (2009) Experiences of Participatory Sensing in the wild. In: Proceedings of the 11th international conference on ubiquitous computing, ACM, pp 265–274

  • Penman TD, Lemckert FL, Mahony MJ (2005) A cost–benefit analysis of automated call recorders. Appl Herpetol 2(4):389–400

    Article  Google Scholar 

  • Porter J, Arzberger P, Braun H-W, Bryant P, Gage S, Hansen T et al (2005) Wireless sensor networks for ecology. Bioscience 55(7):561–572

    Article  Google Scholar 

  • Reddy S, Shilton K, Burke J, Estrin D, Hansen M, Srivastava MB (2008) Evaluating participation and performance in Participatory Sensing. Paper presented at the international workshop on urban, community, and social applications of networked sensing systems, Raleigh, North Carolina, USA, pp 7–11

  • Reddy S, Samanta V, Burke J, Estrin D, Hansen M, Srivastava M (2009) MobiSense—mobile network services for coordinated Participatory Sensing. Paper presented at the international symposium on autonomous decentralized systems, pp 1–6

  • Reddy S, Estrin D, Srivastava M (2010) Recruitment framework for Participatory Sensing data collections. Paper presented at the 8th international conference on pervasive computing, Helsinki, Finland, pp 138–155

  • Roman R, Fernandez-Gago MC, Lopez J (2007) Featuring trust and reputation management systems for constrained hardware devices. In: Proceedings of the 1st international conference on autonomic computing and communication systems, pp 1–6

  • Romer K, Mattern F (2004) The design space of wireless sensor networks. IEEE Wirel Commun 11(6):54–61

    Article  Google Scholar 

  • Ruohomaa S, Kutvonen L (2005) Trust management survey. In: Proceedings of the 3rd international workshop on trust management, pp 77–92

  • Ruohomaa S, Kutvonen L, Koutrouli E (2007) Reputation management survey. Paper presented at the second international conference on availability, reliability and security, pp 103–111

  • Shilton K, Burke JA, Estrin D, Hansen M, Srivastava M (2008) Participatory privacy in urban sensing. In: Proceedings of the international workshop on mobile device and urban sensing, St. Louis, Missouri, pp 1–7

  • Srinivasan A, Teitelbaum J, Liang H, Wu J, Cardei M (2006) Reputation and trust-based systems for ad hoc and sensor networks, algorithms and protocols for wireless, mobile ad hoc networks. Wiley, New York

  • Thau D, Morris RA, White S (2009) Contemporary challenges in ambient data integration for biodiversity informatics. In: Proceedings of the confederated international workshops and posters on the move to meaningful internet systems: ADI, CAMS, EI2N, ISDE, IWSSA, MONET, OnToContent, ODIS, ORM, OTM Academy, SWWS, SEMELS, Beyond SAWSDL, and COMBEK, pp 59–68

  • Underwood A (1994) On beyond BACI: sampling designs that might reliably detect environmental disturbances. Ecol Appl 4(1):3–15

    MathSciNet  Article  Google Scholar 

  • Vromen A (2007) Australian young people’s participatory practices and Internet use. Inf Comm Soc 10(1):48–68

    Article  Google Scholar 

  • Wang X, Zhang F (2008) A new trust model based on social characteristic and reputation mechanism for the semantic web. Paper presented at the first international workshop on knowledge discovery and data mining, pp 414–417

  • Wimmer J, Towsey M, Planitz B, Roe P, Williamson I (2010) Scaling acoustic data analysis through collaboration and automation. In: Proceedings of the 6th IEEE international conference on e-science, pp 308–315

  • Yang Y, Sun YL, Kay S, Yang Q (2009) Defending online reputation systems against collaborative unfair raters through signal modeling and trust. In: Proceedings of the 2009 ACM symposium on applied computing, pp 1308–1315

  • Yu J, Wong W-K, Hutchinson R (2010) Modeling experts and novices in citizen science data for species distribution modeling. Paper presented at the IEEE international conference on data mining, ICDM 2010, pp 1157–1162

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to HaoFan Yang.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Yang, H., Zhang, J. & Roe, P. Reputation modelling in Citizen Science for environmental acoustic data analysis. Soc. Netw. Anal. Min. 3, 419–435 (2013).

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:


  • Citizen Science
  • Reputation management
  • Decision-making
  • Community informatics