Citizen Science: An Information Quality Research Frontier

The rapid proliferation of online content producing and sharing technologies resulted in an explosion of user-generated content (UGC), which now extends to scientific data. Citizen science, in which ordinary people contribute information for scientific research, epitomizes UGC. Citizen science projects are typically open to everyone, engage diverse audiences, and challenge ordinary people to produce data of highest quality to be usable in science. This also makes citizen science a very exciting area to study both traditional and innovative approaches to information quality management. With this paper we position citizen science as a leading information quality research frontier. We also show how citizen science opens a unique opportunity for the information systems community to contribute to a broad range of disciplines in natural and social sciences and humanities.


Introduction
Increased availability of digital resources to support critical decisions makes information quality (IQ) a paramount concern for organizations and society.Recognizing its social importance, information systems (IS) research considers IQ to be a core interest resulting in a long and rich research tradition (Batini et al. 2015;DeLone and McLean 1992;Madnick et al. 2009;Petter et al. 2013;Wahyudi et al. 2018;Wang and Strong 1996).
While IQ remains a prominent subject in IS, much of foundational research on IQ has focused on traditional uses of data in corporate environments and for-profit organizations.In these settings information production was relatively transparent and controlled, typically created by known organizational actors (e.g., database operators) for well-defined purposes (Burton-Jones and Volkoff 2017; Wang 1998;Zuboff 1988).The growth of networks, including the Internet, fuels studies that investigate IQ in inter-organizational, trans-national projects and on the web (Alonso 2015;Batini and Scannapieca 2006;Calero et al. 2008;Heath and Bizer 2011;Wells et al. 2011;Zhu and Wu 2011).One such area of research deals with information created by ordinary people in the course of their daily lives, collectively known as user-generated content, or UGC (Doan et al. 2011;Levina and Arriaga 2014;Nov et al. 2014;Susarla et al. 2012).UGC encompasses such diverse and now ubiquitous forms as social media and social networks (e.g., Facebook, Twitter, Instagram, YouTube), online reviews (e.g., Yelp, TripAdvisor), blogs, chat rooms, discussion boards, comments (e.g., on news stories, products in an ecommerce catalog), knowledge production (e.g., Wikipedia, Wiktionary, Urban Dictionary), content sharing (e.g., Reddit, Digg), amateur news (e.g., Huffington Post), collaborative maps (e.g., OpenStreetMap), and others.
T h e g r ow i n g s oc i et al i m p o r t an c e o f U G C i s underscored by calls for its addition to economic calculations for national gross domestic product (Brynjolfsson and McAfee 2014).UGC gives organizations a unique view into the lives of ordinary people; more and more organizations consult UGC when designing new products, providing commercial services, shaping public policy, or conducting scientific research.Explicit open calls to members of the general public for specific information underlie the booming practice of crowdsourcing (Deng et al. 2016;Ghezzi et al. 2017;Love and Hirschheim 2017;Lukyanenko and Parsons 2018).To offer crowd services at massive scale, platforms like Amazon's Mechanical Turk and CrowdFlower.comallow companies to tap into thousands of paid crowd workers (Chittilappilly et al. 2016;Deng et al. 2016;Garcia-Molina et al. 2016;Li et al. 2016;Stewart et al. 2015).Organizations also create their own platforms, such as the widely popular Zooniverse (www.zooniverse.org),where scientists can post data processing tasks (e.g., image annotation, classification, transcription) to over a million engaged volunteers (Simpson et al. 2014).Data collection and processing through crowdsourcing is different from many other forms of UGC, as it focuses participation on specific, often standardized, information needs and may require specific background, interests, and abilities.
User-generated content stands to rapidly expand the scope of IQ research as it presents novel challenges and opens many new questions (Li et al. 2016;Lukyanenko and Parsons 2015b).Unlike information produced by organizational employees, suppliers, and contractors with close ties to the organization, UGC is created by Bcasualĉ ontent contributors with varying levels of subject matter expertise, reasons for producing content, backgrounds, and worldviews (Daugherty et al. 2008;Levina and Arriaga 2014;Lukyanenko et al. 2014b;Susarla et al. 2012).In some such projects, content producers remain largely anonymous, making it challenging to determine contribution quality based on traditional indicators like known credentials.Physical, and often cultural and emotional distance online further complicates the process of understanding and interpreting data (Deng et al. 2016;Gray et al. 2016).Lack of context of information creation, an important factor for appropriate use of information (Lee 2003;Shankaranarayanan and Blake 2017;Strong et al. 1997), can significantly complicate using UGC.
Many novel research questions emerge in response to the rise of UGC and crowdsourcing.For example, since UGC is typically produced voluntarily (as opposed to relying on extrinsic organizational rewards), new issues related to motivation emerge (Daugherty et al. 2008;Nov et al. 2014).
Similarly, novel forms of information creation such as collaborative production (e.g., Wikipedia) or information co-creation, call for explanation and understanding of the mechanisms of quality control in these settings (Arazy et al. 2011(Arazy et al. , 2017;;Liu and Ram 2011;Zwass 2010).Challenges involved in collecting, storing, and using UGC motivate innovations in database, data visualization, data integration and analytic solutions (Bonney et al. 2014a(Bonney et al. , 2014b;;Daniel et al. 2018;Hochachka et al. 2012;Stevens et al. 2014).
Illustrating the potential of UGC to make significant contributions to the broader field of IQ, we focus here on one of its prominent types: technology-supported citizen science.Broadly, citizen science refers to participation of ordinary people in scientific research (Bonney et al. 2014b;Show 2015).The term describes a range of approaches to community-based science and public involvement in research (Newman et al. 2010(Newman et al. , p. 1852)).Although many labels exist to describe this phenomena and its more specific forms, citizen science has become the most popular (Kullenberg and Kasperowski 2016).
While citizen science has many interesting dimensions (e.g., scientific, social, educational, economic) (Burgess et al. 2017;Louv et al. 2012), we focus on its technological aspects.From the point of view of IS, citizen science involves an unusual type of technology when compared with traditional corporate IS, such as ERP or supply chain systems, but also differs to some extent from such technologies as social media (more on these differences below).In citizen science, data is created or processed by ordinary people, often (but not always) with no formal education in the subject matter to be used by scientists or domain experts who typically require information of highest quality.
A major aspect of online citizen science is the open and frequently anonymous participation, due primarily to large scales of participation, further complicating information quality activities.At the same time, as citizen science is increasingly part of modern scientific practice, it has attracted considerable attention from researchers from natural and social sciences, medicine, and engineering.For example, citizen science research, including aspects dealing with IQ is on the rise in ecology, biology, physics, chemistry, astronomy, geology, anthropology, and politics, as well as humanities (Fortson et al. 2011;Haklay 2013;Hand 2010;Show 2015;Stevens et al. 2014).Few other applications of IQ have received such intense interdisciplinary attention.Finally, although typically conducted in public sector organizations (NGOs, academia, government), aspects of citizen science have potential for translation to corporate research and development as well.Recent studies in information systems argue for the importance of conducting research in citizen science due to its implications beyond its immediate context (Levy and Germonprez 2017;Lukyanenko et al. 2016b).
Importantly, in dealing with multiple challenges of citizen science IQ, interdisciplinary researchers investigate novel theoretical concepts, and deploy and evaluate innovative IQ solutions that could potentially advance broader IQ thinking and practice.Motivated by this potential, we propose research on citizen science as a promising IQ research frontier.To better understand the potential of this domain, we review the IQ challenges in citizen science and relate them to traditional IQ concepts and approaches.We then offer directions for future research in citizen science and IS focusing on IQ and conclude by identifying the value that citizen science research can bring to strengthening connections between the information systems discipline and other fields in sciences and humanities.

Situating Citizen Science in the Broader UGC Landscape
Although citizen science is an established topic in natural sciences (below, we briefly discuss the history of citizen science), it is a relatively recent area of research in the information systems discipline, including a stream focused on information quality, and in related fields such as computing.Before discussing specific issues that relate to the quality of data in citizen science, we position citizen science in the context of other related phenomena.Table 1 provides definitions of some of related concepts (note: the list is not exhaustive, and is meant to show major categories to elaborate the niche which citizen science occupies from an IS perspective).
Citizen science largely relies on UGC, as volunteers are routinely engaged in collection, processing or analysis of data.However, citizen science brings an important scientific dimension that general UGC lacks.The scientific dimension heightens the importance of high information quality.Also, as citizen science projects tap into personal interest in public goods, successful projects can easily attract many engaged participants.
Many successful citizen science projects allow for high social interactivity among volunteers.As Silvertown (2010) noted in a Nature commentary, a solution to species identification by non-experts may lie in Bsocial networking^, whereby users are allowed to debate a particular case and among each other and vote up or down candidate species categories, precisely the mechanism now encoded in the increasingly popular iNaturalist platform (inaturalist.org)for biodiversity data contributions.Many scientific discoveries, such as that of Hanny's Voorwerp (see Table 2) on the Zooniverse platform, originated through the discussions on the forums provided to citizen scientists.However, high user interactivity may not be suitable for some citizen science projects (for reasons we discuss below), or can be simply unnecessary for the fulfilment of specific project objectives.Finally, the essence of social media is facilitation of social exchange.Issues of scientific discovery, high information quality, pursuit of objective truth, adherence to protocols and standards, and environmental conservation are very much on the fringes for most social media projects.As Maddah et al. (2018) argue, social media emphasizes personal experiences (or episodic information), rather than general or semantic information (the typical focus of citizen science).
Citizen science projects frequently Bcrowdsource^data collection or other activities.In this case, citizen science utilizes crowdsourcing as a work organization strategy.However, crowdsourcing first and foremost is a business model and as Levy and Germonprez (2017, p. 29) noted, Bcrowdsourcing is not rooted in citizens' intervening in the scientific process^.Indeed, much of crowdsourcing has a prominent monetary dimension that is typically missing in citizen science.For example, turning to crowds for project funding or product developmentcrowdfundingspecifically pursues monetary goals (Choy and Schlagwein 2016;Zhou et al. 2018).Likewise, Amazon's Mechanical Turk and CrowdFlower.comserve as intermediaries between companies and online workers (Chittilappilly et al. 2016;  (Abbasi et al. 2018, Kane et al. 2014, Kapoor et al. 2018, Miranda et al. 2016) Social networks, chats, message boards, forums, photo and video sharing apps and platforms Crowdsourcing A model of labour in which an organization turns to crowds (i.e., ill-defined members of the general public) to obtain desired data, goods, or services (Brabham 2013, Ghezzi et al. 2017, McAfee and Brynjolfsson 2017, Poblet et al. 2017, Zhao and Zhu 2014) Many business, public and even individual initiatives, such as crowdfunding, ideation, contest, citizen science, crisis management or online employment platforms

Collective intelligence
Knowledge that emerges as a result of collaboration between many individuals engaged in the same task (Awal and Bharadwaj 2017, Bonabeau 2009, Malone et al. 2010, Woolley et al. 2010) Problem-solving, decision-making, collaborative action in a variety of domains Deng et al. 2016;Garcia-Molina et al. 2016;Li et al. 2016;Stewart et al. 2015;Zhao and Zhu 2014).The commercial nature of many crowdsourcing initiatives creates a different information quality management environment compared to typical citizen science projects.In addition, many crowdsourcing projects are not interested in testing scientific theories or discovering ground truths, but rather engage online users in routine tasks that add business value.
Crowdsourcing projects such as those on Amazon Mechanical Turk can frequently utilize redundancy or scale in the crowds to remove data outliers or obtain response distributions (Ipeirotis and Gabrilovich 2014;Tang et al. 2015;Zhao and Zhu 2014).This objective may be applicable to some crowdsourcing projects (e.g., mapping distribution of species in an area), but would not be aligned with the goals of making scientific discoveries, promoting environmental conservation, or raising awareness of scientific issues (Wiggins and Crowston 2011).Finally, crowdsourcing projects typically end when the desired data or services are obtained from the crowds; in contrast, citizen science projects are commonly long-term and involve continuous engagement with the contributors, including feedback, debriefing, and sharing results with interested participants (Newman et al. 2012;Rotman et al. 2012).
Citizen science also creates fertile grounds to leverage collective intelligence.Citizen science projects sometimes involve collaborative decision-making processes, creating opportunities to discover solutions or produce results superior to those which would have been created if volunteers or scientists worked individually (Malone et al. 2010).Citizen science can especially benefit from collective intelligence in projects that require group tasks, problem solving, and decisionmaking.For example, InnoCentive is an online platform which posts difficult research problems, such as how to synthesize certain chemical compounds, for crowds to solve.Best solutions can receive substantial monetary rewards, sometimes worth tens or hundreds of thousands of US dollars (Malone et al. 2010).However, citizen science activities such as environmental monitoring and conservation do not directly depend on collective intelligence for success.Recreational scuba diver along the coast of Tarragona, Spain (Azzurro et al. 2013) *Note: Typically, citizens are the first to see the phenomena and alert the scientific community via the citizen science IS.However, scientists then commonly need to conduct rigorous testing or experimentation to confirm the discovery and publish it.Often, but not always, citizens are invited to coauthor publications, such as the paper co-authored by Mardon Erbland, a citizen scientist, who discovered a new distribution of mosquito native to Asia in Eastern Canada (Fielden et al. 2015) As we show below, the differences between citizen science and other related fields translate into important nuances for information management and information quality improvement strategies.This makes it important to study information quality in citizen science in its own right.At the same time, the shared properties between citizen science and crowdsourcing, collective intelligence, and other forms of user generated content, mean that progress in information quality research on citizen science may also be leveraged to tackle challenges and problems related to information quality in related domains.
Finally, like citizen science, information quality research on crowdsourcing, user generated content, collective intelligence, social media and other emerging media forms, is a relatively recent subject in the information systems discipline (Kapoor et al. 2018;Lukyanenko and Parsons 2015b;Palacios et al. 2016;Tilly et al. 2016).Thus, focusing on citizen science a societally important form of public-professional collaborationcan spearhead broader development of information quality research and continue to expand its scope and impact beyond the traditional research in the context of corporate organizational settings.
3 Why Study Information Quality in Citizen Science?
As we showed in the previous section, citizen science presents a unique study opportunity for information quality research community due to its distinctive forms of information production.In this section we focus on several important characteristics of citizen science that not only distinguish it from other types of UGC, but more importantly, make technologysupported citizen science particularly fertile for advancing IQ research.

Relevance to Science and Society
In recent decades, the IS research community has begun to explore relevant topics outside of traditional organizational settings (Levy and Germonprez 2017;Winter et al. 2014).Calls are increasing to conduct research in information systems that is explicitly concerned for large social groups and society (Goes 2014;Melville 2010;Seidel et al. 2013).Citizen science is uniquely positioned to establish IS as a discipline concerned with social progress and social improvement.
Studying IQ in citizen science stands to benefit sciences broadly and has potential for major social improvements (Burgess et al. 2017;McKinley et al. 2016).Much hope is vested in citizen science, which is most strongly adopted in sciences tackling humanity's Bevil quintet^of climate change, overexploitation, invasive species, land use change, and pollution (Theobald et al. 2015).As Light and Miskelly (2014, p. 14) put it, B[t]he urgency of environmental issues draws our attention to the management of finite resources, and the potential of digital tools to help us work with them effectively.T hese are conversations that IS can ill afford to ignore in the Twenty-First Century.
Citizen science is also a booming hobby for online participants worldwide and a growing trend in science.In biodiversity research alone, it is estimated that more than two million people are engaged in major citizen science projects, contributing up to $2.5 billion of in-kind value annually (Theobald et al. 2015).As of 2009, the public was already providing 86% of biological survey data in Europe (Schmeller et al. 2009).Constantly pushing the boundaries of what is possible, scientists are engaging ordinary people in an ever increasing range of tasks.Volunteers may be asked to observe plants and animals (Hochachka et al. 2012), transcribe historical texts (Eveleigh et al. 2013), map cultural and geographic objects (Haklay and Weber 2008), identify natural resources to preserve and protect from development and exploitation (Vitos et al. 2013), design new drugs (Khatib et al. 2011), search for extraterrestrial intelligence (Korpela 2012), catalog galaxies (Fortson et al. 2011), describe oceans floors and document marine life (Pattengill-Semmens and Semmens 2003), track butterflies (Davis and Howard 2005), and report on active wildfires (Goodchild and Glennon 2010), among many other tasks.Citizen science can take volunteers into wilderness areas, as in the case of some biological monitoring projects, but are just as likely to occur in kitchens or backyards (some projects require mailing specimens, such as sourdough starters or soil samples), can be conducted entirely online (such as peering at photographs taken by the Hubble telescope to classify galaxies or scanned specimens to transcribe museum collections), or require little direct involvement, like in volunteer computing projects such as SETI@Home's search for extraterrestrial intelligence and a wide variety of other research projects supported by the BOINC distributed computing architecture (Hand 2010;Louv et al. 2012).
Ingenuous application of online technologies, coupled with enthusiasm and creativity of ordinary people, have already resulted in a number of high-profile scientific discoveries.Table 1 provides several examples of discoveries originating in citizen science projects.
Notably, citizen science information quality is already a well-explored subject in sciences (discussed further below).It builds on the long tradition of involving humans in data collection that invariably involved external research participants, field research settings, noisy and unreliable sources, and investments into innovative data collection infrastructure.Some date the origins of citizen science to Victorian-era Britain (or earlier) and online citizen science was in fact one of the early adopters of content producing technologies known as Web 2.0 (Bauer et al. 2000;Bonney et al. 2009;Brossard et al. 2005;Goodchild 2007;Louv et al. 2012;Osborn et al. 2005;Wiersma 2010).By bringing IS expertise to citizen science, IS researchers can contribute to advancing an established and socially valuable body of long-standing cumulative research.This work has potential to advance practice as a value multiplier.

Relentless Pursuit of Quality
Information quality is a core concern in science, and therefore also in citizen science.As Prestopnik and Crowston (2012b, p. 9) conclude from observing a real-world citizen science project development: B[scientists'] primary goal is that [citizen science projects] should produce large amounts of very high quality data.Virtually all other considerations are secondary( emphasis added).Similarly, data quality is B[o]ne of the highest priorities^for the Atlas of Living Australia, which maps biodiversity of the continent based on citizen-supplied data, as well as many other sources, including government agencies (Belbin 2011).Of the challenges that face the practice of citizen science, information quality is consistently considered paramount (Cohn 2008;Kosmala et al. 2016;Lewandowski and Specht 2015;Sheppard and Terveen 2011;Wiggins et al. 2011).
The persistent and often lopsided focus on IQ in citizen science contrasts with many other types of UGC (e.g., social networking, micro-task markets, blogs), where information quality competes with other priorities for attention, such as profit, enjoyment of participants, ability to advertise products, disseminating content, attracting followers, co-creating products, and solving practical problems (Ipeirotis et al. 2010;Levina and Arriaga 2014;Sorokin and Forsyth 2008;Susarla et al. 2012;Wattal et al. 2010;Zwass 2010).In contrast, in the context of science, these considerations are present but typically draw less attention because they are less core to the success of scientific research and are generally considered to be operational issues.Empirical evidence lies at the heart of the scientific method, making information quality a critical and central citizen science theme.Yet, this persistent interest in IQ is what makes citizen science an exciting setting for better understanding and advancing the general information quality research tradition.
As with any complex social phenomena, citizen science is more than just data collection and information quality.It also includes considerations of the engagement of volunteers in doing research; increasing public trust in the scientific enterprise; co-creation of scientific endeavors; engagement of the public in policies and debates arising as a result of science; building communities; involving people in addressing climate change and environmental sustainability; fostering education, literacy, and awareness of science among the general population (Bonney et al. 2009;Irwin 1995;Irwin and Michael 2003;Levy and Germonprez 2017;Sieber 2006).While these are important to the phenomenon and practice of citizen science, conducting research in information quality in citizen science holds undisputed value among scientists, and IS researchers are particularly well positioned to conduct research affecting the core issue in this emerging field.

A Multitude of Wicked Challenges
IS researchers are frequently challenged to embark on risky research that tackles difficult and ill-defined, wicked problems (Hevner et al. 2004).Due to its unique nature, citizen science has presented a number of information quality research problems that are either unreported in traditional organizational settings or familiar issues with notable contextual considerations.
One of the interesting aspects of citizen science is its typically democratic and open nature (Irwin 1995).Citizen science is fundamentally about empowering ordinary people in advancing research endeavors.Consequently, it is against the spirit of citizen science to discriminate against participants based on lack of scientific training or low literacy (ESCA 2015).Citizen science also puts an important premium on unique local knowledge.Yet it is precisely these characteristics that may endanger information quality (Gura 2013): BYou don't necessarily know who is on the other end of a data point … It could be a retired botany professor reporting on wildflowers or a pure amateur with an untrained eye.As a result, it is difficult to guarantee the quality of the data.Î n many emerging crowdsourcing environments, such as micro-task markets (Garcia-Molina et al. 2016;Lukyanenko and Parsons 2018;Ogunseye et al. 2017;Palacios et al. 2016), one of the ways to deal with information quality is to restrict participation to those members of the community deemed competent to produce high-quality information (Allahbakhsh et al. 2013;Chittilappilly et al. 2016;Ogunseye et al. 2017).While this is technically also possible to do in citizen science, it both runs against the spirit of openness and inclusion and can jeopardize the recruitment and retention of contributors that is key to success in projects that rely on voluntary labor.Yet, as noted earlier, high-quality consistent data is needed to make strong inferences and conclusions.Thus, scientists face a dilemma of optimizing information quality while keeping participation levels high enough to meet research needs and remaining sensitive to all relevant stakeholders.The challenge therefore becomes to preserve the openness of citizen science, allowing anybody interested to participate while delivering information of sufficient quality for scientific research.
Another consequence of open and often anonymous participation is that citizens often have diverse backgrounds and hold different views from each other.Sometimes the differences between scientists and contributors as well as the nature of the information are truly Bextreme,^as in the case of a conservation project in the Congo rainforest basin where data collectors include non-literate seminomadic hunter-gatherers (Pygmies), local farmers, loggers, and foresters (Rowland 2012;Stevens et al. 2014;Vitos et al. 2013).Lukyanenko et al. (2014aLukyanenko et al. ( , 2014b) ) show that when views of volunteer contributors differ from those of professional experts (scientists) and one another, quality of contributions may suffer.
For the reasons above, quality remains a wicked problem.Indeed, concerns about IQ are so ingrained that scientists continue to face challenges in sharing findings based on citizen data with the scientific community even when they can conclusively demonstrate that data quality is high (Law et al. 2017;Lewandowski and Specht 2015;Riesch and Potter 2013).It is this untamed nature that makes citizen science information quality research interesting and productive.
4 Where Is IS in the Citizen Science Landscape?
Despite the growing societal value of citizen science and many opportunities for IS researchers, IS continues to lag behind such disciplines as biology and education in working with citizen science as a context for research.To assess the extent of the gap and also estimate the general volume of citizen science research, we conducted a review of citizen science literature and present the results below.
Figure 1 shows the growth of research on and with citizen science across disciplines and time.To generate this analysis, we used exact-match querying for the phrase Bcitizen scienceô ccurring in the title, abstract, or keywords of journal articles and conference proceedings indexed in Scopus (3002), Web of Science (2302), IEEE eXplore (130), and ACM Digital Library (204).This is a conservative method for identifying the extant literature on citizen science, as not all relevant publications use this currently popular terminology, but our initial query results included 5806 items.The results were manually de-duplicated by using title name and DOI for reference, since there was extensive cross-indexing in several sources.We also removed editorial introductions to journal issues, dissertations, workshop papers, and posters to leave only full papers.Abstracts were then examined for papers with titles that were not obviously relevant to citizen science to evaluate the extent of focus on citizen science as a phenomenon or method.Papers mentioning citizen science only in passing, e.g., as an example of a broader trend (as opposed to a primary focus of the work) were eliminated, as were a few papers for which DOIs did not resolve during verification, bringing the total corpus to 1633.
The resulting papers were categorized to discipline and sub-discipline based on publication venue using the Classification of Instructional Programs (CIP) taxonomy by the United States Department of Education, National Center for Education Statistics.1Although any such classification scheme is imperfect and the CIP is pending revision in 2020, nearly all sub-specialties were located within the document and could be associated with a relevant discipline.Since some publication venues are non-disciplinary, e.g., Public Library of Science, papers published in these journals were categorized to discipline based on title and keywords, where available, or by checking the abstract if necessary.To avoid underestimating the volume of IS research and be more conservative, all papers published in the Hawai'i International Conference on Systems Science (HICSS) were classified as BIS^although work originating in Computing (Computer Science) and other disciplines is also often presented in this venue.For the sake of effective summary, several fields with just one or two publications were folded into Bnearest neighbor^disciplines; these included Mathematics, Leisure studies, Linguistics, Philosophy, Psychology, and Business.Sub-disciplines (or primary themes, where disciplinary specialties were not present) were assigned to 1545 papers (95%) as several of the disciplinary categories are very broad, e.g., biology.
As Fig. 1 clearly shows, there has been substantial growth in annual publications on the topic of citizen science over the last 30 years.Even in our highly conservative data set, for the fields with more than two publications (over all time), IS leads only Agriculture, Architecture (i.e., Urban Planning), Humanities, and Legal scholarship.The closely related fields of Biology and Natural Resources Conservation together yielded 819 papers, approximately 50% of the total, and Ecology was the leading sub-discipline with 257 papers.By comparison, Computing had 260 (primarily empirical) papers while IS had just 17, and notably, the papers in IS originated with a very small handful of authors and several are nonempirical in focus.
It is further notable that the range of disciplinary specialties represented within Computing was not entirely out of line with the interests for IS scholars; Fig. 2 shows the subdisciplinary topic areas for Computing publications, drawn primarily from the specific publication venues, e.g., ACM conferences on pervasive and ubiquitous computing (Ubicomp) or human-computer interaction (Design and Users).Of these, topics of interest for IS included Information Quality, Users, Ubiquitous Computing, Crowdsourcing, Social Computing, and Design.Among IS publications, the primary themes included Design (7), Information Quality (3), Social Computing (2), Crowdsourcing (1), Games (1), Open Source Software (1), Users (1), and a general perspective article (Levy and Germonprez 2017).Across the entire data set, the topic of Information Quality appeared 49 times, including 29 Conservation papers where it was a more common primary focus than species groups or habitats, and 15 papers in Computing, although Information Quality was in fact a dominant theme in far more publications than was captured by this analysis.
As we can see, developing and applying information systems theories to improve praxis of citizen science offers an as yet unexplored opportunity for IS researchers to have impact beyond its traditional boundaries.Through citizen science, IS researchers can begin to address an important call to expand the frontiers of information systems research beyond traditional core subject matter and establish itself in the broader constellation of sciences (Beath et al. 2013) as well as doing a major service for the public good.

Information Quality Research Opportunities in Citizen Science
While information quality in citizen science is already an active area of research in sciences, it has only recently captured the attention of IS researchers (Lukyanenko et al. 2014b;Nov et al. 2014;Robinson and Imran 2015;Sheppard et al. 2014;Wiggins and Crowston 2011;Wiggins and He 2016).Interdisciplinary researchers focused on both traditional approaches to information quality, as well as implementing novel, untried methods and techniques have studied citizen science.The findings generated in this context can offer interesting and original insights with promise to advance the general understanding of theoretical and practical nature of IQ.Below, we illustrate this potential by examining major trends in citizen science information quality research.

Training and Evaluating Learning and Performance
Information quality research in organizational contexts studies many factors to improve IQ, including total data production monitoring and control, adherence to best practices in IS development (e.g., requirements elicitation, participative design, conceptual, database and interface design), training of data creators (e.g., database clerks), clarity of data entry instructions, close monitoring of the data production process, providing timely quality feedback and continuous improvement of information production processes (Ballou et al. 2003;Batini and Scannapieca 2006;Lee et al. 2002;Levitin and Redman 1995;Redman 1996;Strong et al. 1997;Wang et al. 1995aWang et al. , 1995b)).Many of these strategies have been applied and evaluated in citizen science projects (Bonney et al. 2009;Kosmala et al. 2016;Lewandowski and Specht 2015;Newman et al. 2003;Wiggins et al. 2011).Surveys of citizen science projects show that multiple quality assurance and quality control strategies are usually implemented (Wiggins and Crowston 2014).Thus, citizen science is already a partial extension of the IQ tradition in traditional settings.
Much like data entry in organizations, some citizen science projects require testing or certification in a certain skill to participate (Crall et al. 2011;Lewandowski and Specht 2015).Often projects naturally attract or are specifically marketed to members of the general public with domain expertise, such as avid birders (e.g., Cohn 2008).Insisting on high domain expertise or prequalifying tests of expertise can severely curtail participation in many projects and runs contrary to the open spirit of citizen science (Lukyanenko et al. 2014b;Wiggins et al. 2011).Alternative approaches, such as embedded assessments that evaluate learning or skill through the process of participation as opposed to using a quiz or other explicit instrument, are currently a hot topic of research in evaluation related to citizen science (Becker- Klein et al. 2016;Nelson et al. 2017;Ogunseye and Parsons 2016).A typical embedded assessment task for an online content classification project injects a handful of images into the workflow that have been pre-classified by experts to evaluate contributors' agreement with experts, producing individual-level performance measures.
Another traditional measure to improve IQ is training for data creators.Training has been widely applied and studied in the citizen science context.Anecdotal and direct evidence of the positive impact of training on accuracy is reported (Delaney et al. 2008;Newman et al. 2003;Wiggins et al. 2011).Citizen science studies explore theoretical boundaries on how training is designed and delivered, identifying limitations in projects with many users, globally distributed participants, ill-defined tasks, minimal resources for training, or large and complex domains (e.g., entirety of natural history).Furthermore, requiring extensive training may discourage participation, can lead to hypothesis guessing, and may result in other biases (Lukyanenko et al. 2014b;Ogunseye et al. 2017).For example, participants who know the objective of the project may overinflate or exaggerate information (Galloway et al. 2006;Miller et al. 2012).Some have explored the use of tasks which specifically do not require training (Eveleigh et al. 2014;Lukyanenko et al. 2019), while others have investigated the possibility of attracting diverse crowds so that training induced biases are mitigated due to the diversity of the participants (Ogunseye et al. 2017;Ogunseye and Parsons 2016).All these are novel ideas for traditional information quality research.As training is prevalent in corporate settings, findings from citizen science may suggest new approaches for this context as well.

Optimizing Tradeoffs
In citizen science, as in broader IQ research, information quality is considered a multidimensional construct composed of such dimensions as accuracy, timeliness, completeness (Arazy and Kopak 2011;Ballou and Pazer 1995;Girres and Touya 2010;Haklay and Weber 2008;Lee et al. 2002;Nov et al. 2014;Rana et al. 2015;Sicari et al. 2016).One of the findings of traditional IQ research are the trade-offs between dimensions (Batini and Scannapieca 2006;Pipino et al. 2002;Scannapieco et al. 2005).For example, previous research in corporate settings proposed trade-offs between consistency and completeness (Ballou and Pazer 2003), and accuracy and timeliness (Ballou and Pazer 1995).These findings have been corroborated in the citizen science context as well.
For example, research established a common trade-off in citizen science between accuracy and data completeness (Parsons et al. 2011).Specifically, to maximize accuracy, projects may be tempted to create restrictive data collection protocols and limit participation to the most knowledgeable contributors.While this strategy leads to increased accuracy of classification (e.g., identification of species), it also limits the amount of data (or completeness) such projects may otherwise achieve.A pressing challenge in citizen science is finding ways to mitigate such trade-offs, allowing many people to participate and provide information in an unconstrained manner, while ensuring that the information provided is adequate for scientists to use in research.
A novel trade-off between consistency and discovery appears to be further inherent to the citizen science context.Consistency is an important dimension of citizen science quality as it promotes reliability and comparability of results, often requisite for scientific data analysis.For example, in the River Watch Project (www.iwinst.org),Bit is critical that the data is collected in exactly 'the same way, every time' so that any differences that are found are real and not due to inconsistent procedures.The goal is to minimize 'variables' as much as possible, so that data can be meaningfully compared.( Sheppard and Terveen 2011, p. 33).Researchers in citizen science have applied analysis techniques such as confusion matrices, finding that contributors who are consistently wrong can be just as valuable as those who are consistently right for certain tasks, because consistent responses can be corrected to make the data useful (Simpson et al. 2013).
Such standardization, of course, may preclude some projects from attaining another important scientific objective, discovery, where variability is a potential requisite (Lukyanenko, Parsons andWiersma 2016, Nielsen 2011).Recent work in IS began to examine whether using innovative instance-based approaches to data collection and storage can eliminate trade-offs that appeared to be inevitable in citizen science (Parsons et al. 2011).An instance-based approach to storage and collection focuses on capturing unique attributes of individual objects of interest (e.g., birds, plants, geological and geographic forms, events), rather than their similarity.This information that is inherently variable and sparse can then be made consistent through the application of high performance data mining algorithms and inferential rules.Recent experiments demonstrate the advantages of this approach over traditional methods of data collection and storage in increasing simultaneously accuracy, completeness and discovery (Lukyanenko et al. 2014b, a), however the ability of this approach to promote consistency is unknown, suggesting a need for additional research in this direction (Lukyanenko, Parsons, Wiersma, et al. 2016).Other analytical techniques have also been applied, such as using effort information to standardize data (Bird et al. 2014), or using modeling (Sousa et al. 2018;Strien et al. 2013), applying different analytic approaches to solving some of the problems in participant variability, which also may have utility for IS research.Considering this work, there is a clear need for further research on dimensions and the nature of IQ tradeoffs in citizen science.
As illustrated by the examples above, a good deal of citizen science research is quite similar to the typical work information systems scholars have conducted in other contexts.This makes the findings from citizen science potentially transferable to broader areas of IQ and allows it to contribute by corroborating the existing theories and implementing proposed approaches.Yet another exciting aspect of citizen science is a growing number of ideas and strategies that are original and unique, promising novel insights and opportunities for the broader information quality research tradition.We focus the remainder of this section on select contributions that advance or challenge established theoretical notions and offer innovative practical solutions.

Human Factors in IQ
An important contribution of citizen science context is better understanding of individual human factors' impact on IQ.In traditional organizational settings, common training, background, instructions, and organizational culture and norms resulted in a relatively homogenous views of people in the same roles.Similarly, data creators in corporate environments are compelled to create high quality data as part of the general disposition to perform work well, despite known individual differences, e.g., in virtual teams (Watson-Manheim et al. 2002).Consequently, research on individual factors in IQ has been scarce.In a recent Binformation quality research challenge paper^, Alonso (2015) laments: B[m]ost of the data quality frameworks … superficially [touch] the quality of work that is produced by workers.New labeling tasks that rely heavily on crowdsourcing need to pay attention to the different social aspects that can influence the overall quality of such labels.Payments, incentives, reputation, hiring, training, and work evaluation are some of those aspects^(p.2).
By contrast to employees in corporate settings, the voluntary nature of content creation in citizen science means we must assume variation in backgrounds, motivation, knowledge, literacy skills, and other abilities, which motivates increased attention to the impact of individual factors on IQ.Many studies have demonstrated the importance and positive impact of motivation on quantity of contributions, noting the particular importance of intrinsic motivation in citizen science (Flanagin and Metzger 2008;Nov et al. 2011;Prestopnik et al. 2013).Nov et al. (2014) report positive impact of collective motives (contribution to collective goals) and rewards, such as increased reputation among project members and beyond or making new friends, on accuracy of contributions.
The nature of citizen science data collection also introduces biases in data that are generally assumed not to be present when information production occurs in controlled settings.These include self-selection biases, predisposition of volunteers to notice charismatic, unusual, easy to report events and objects, and spatial and temporal biases (Janssens and Kraft 2012), all of which are also known problems in professional science (Kosmala et al. 2016).Citizen science researchers have documented, investigated, and proposed solutions to detect and mitigate these biases when analyzing data and drawing conclusions (Crall et al. 2011;Delaney et al. 2008;Girres and Touya 2010;Wiersma 2010).These ideas could significantly enrich IS quality research, where individual variation in information production has attracted attention only recently (Burton-Jones and Volkoff 2017), and assumptions of bias, or lack thereof, may merit re-examination.

Hardware Factors
Data collection in traditional organizational settings was typically conducted in a well-controlled environment.Among other things it meant that in designing data collection process, IS specialists decided the hardware available in the process of design.In citizen science, controlling which devices volunteers use is much more problematic (He and Wiggins 2017).Citizen science research stands to enrich information quality literature by examining the impact of hardware and devices as well as the role of physical settings for doing data entry (Eveleigh et al. 2014;Stevens et al. 2014;Wiggins and He 2016), factors that have not received much attention in traditional IQ research.Among other things, research in citizen science has underscored the importance of designing for multiple devices and creating processes which are simple, flexible, easy to use, and work on devices with memory and Internet connectivity constraints (e.g., mobile apps or miniaturized systems) (Cohn 2008;Higgins et al. 2016;Kim et al. 2013;Newman et al. 2010;Prestopnik and Crowston 2012a).Another interesting idea, rarely investigated in corporate settings, is to allow users to explore, experiment, tinker, and even Bfail^when working with the systems as a strategy for identifying ways to develop more robust and usable tools (Newman et al. 2010).
A general lesson emerging from this research is that the physical environment and hardware characteristics have a measurable impact on information qualitya novel consideration for traditional information systems research, which routinely abstracts away implementation details when dealing with representational issues (Jabbari et al. 2018;Jabbari Sabegh et al. 2017;Wand and Weber 1995).This insight may be increasingly relevant in Bbring your own deviceâ nd app-driven organizational settings (French et al. 2014;Hopkins et al. 2017;Jordan 2017;Weeger et al. 2015).

Social Factors
Another original contribution of citizen science is the exploration of social factors in content creation.Social networking and interactivity between participants is a common design feature of citizen science IS (Haklay and Weber 2008;Silvertown 2010).Yet, unlike general social networking projects (such as Facebook, Twitter, Youtube), in citizen science these features are implemented and investigated with explicit information quality objectives, making this a particularly interesting context for understanding the implications of sociality on information production (Nov et al. 2014;Silvertown 2010).Silvertown (2010) suggested that social networking could aid novices in species identification through the process of peer identification, which Wiggins and He (2016) examined in a study of community-based data validation, where social practices and device types used to collect information impacted information quality.Social networking is generally expected to improve information accuracy, foster peer-based learning and co-creation where participants are able to both generate and evaluate content, develop new knowledge, and refine existing skills, all of which can lead to higher quality UGC (Lukyanenko 2014;Silvertown 2010;Tan et al. 2010).Contributors remain aware of the social context in which observations are being made: the study by Nov et al. (2011) found the impact of social motives on quantity of observations.
As the same time, overreliance on social media in the context of citizen science may impose scientific limitations and negative impacts on quality.High user interactivity may lead to group think or hypothesis guessing.As scientists frequently conduct online experiments in the context of citizen science (e.g., Lukyanenko et al. 2014a), high social interactivity may engender treatment spillover, when participants in one condition may communicate with the participants in another (e.g., treatment or control) condition.Social media also brings additional challenges of managing a less structured user engagement, which can be fraught with cyberbullying, trolling or other malignant user behavior prevalent on social media platforms (Kapoor et al. 2018), and may undermine citizen science projects.
As social networking has only recently been incorporated into corporate information production (Andriole 2010), there is scarce work on IQ implications of these platforms, and findings from citizen science may offer useful insights to social networking-related IQ research in corporate settings.
Consider malfeasance, a persistent concern in open online settings (Law et al. 2017), where employing social networking has shown promise for data sabotage detection (Delort et al. 2011;Wiggins and He 2016).Research has also considered features of systems that make them more robust to malfeasance (Wiggins and He 2016), as anecdotal evidence has suggested extremely low levels of malicious behavior in citizen science.These lessons may be applicable in more traditional settings to detect data sabotage by disgruntled employees.Research on social factors in citizen science can also contribute to the emerging area of online digital experimentation conducted for marketing or product development purposes in commercial contexts (e.g., Crook et al. 2009;Davenport and Harris 2017;Kohavi et al. 2013).

Re-Examining IS Research Assumptions
Citizen science sheds new light on and invites reconsideration of several long-standing assumptions of IS research.Some researchers argue that novel challenges of understanding and improving quality in citizen science warrant extending prevailing definitions of IQ and suggest rethinking traditional methods of improvement (Lukyanenko and Parsons 2015b).The organizational IQ paradigm focused on the data consumers and has defined quality as fitness of data to their needs (Madnick et al. 2009;Wang and Strong 1996).This view remains central to much of citizen science, as scientists routinely approach citizen science the same way they approach scientific research: they formulate predefined hypotheses, set goals for the projects, and then use the data provided by the volunteers to corroborate their predictions and assumptions (Lukyanenko and Parsons 2018;Wiersma 2010).This strategy is also sound as it reduces noise and variation in the data and makes the resulting data much easier to interpret and analyze.
Notwithstanding the value of approaching citizen science from the traditional fitness for use perspective, it may have limitations for some projects.A new definition of IQ has been proposed defining it as Bthe extent to which stored information represents the phenomena of interest to data consumers and project sponsors, as perceived by information contributors ( Lukyanenko et al. 2014b, p. 671).This definition avoids the traditional Bfitness for use^definition (Madnick et al. 2009) since contributors may struggle to provide data that satisfies data consumer needs and may not even be fully aware of them.Thus, holding data contributors to data consumer standards may curtail their ability to provide high quality content as defined by data consumers, or suggest a need to refine instructions, procedures, or expectations.Guided by this definition, a series of laboratory and fields experiments demonstrated that accuracy and completeness of citizen science data can indeed be improved by relaxing the requirements to comply with data consumer needs (Lukyanenko et al. 2014a(Lukyanenko et al. , 2014b;;Lukyanenko et al. 2019), which is a standard project design strategy for achieving data quality targets in citizen science.Further, for projects focusing on new, emerging phenomena, it may be challenging to anticipate the optimal structure of citizen science data due to the different needs of diverse data consumers, so traditional solutions for storage premised on a priori structures (e.g., relational databases) may be inadequate in this setting (Sheppard et al. 2014).The new definition, however, remains controversial: the original definition was clearly useful for project leaders, and the data resulting from putting the views of data contributors first may be sparse and difficult to work with or even intractable for scientific requirements.Nonetheless, the advantages for some contexts suggest that the traditional fitness for use view may not fully capture the diverse and changing IQ landscape.These findings further provide a novel connection between conceptual modeling grammars (i.e., rules for building conceptual models and constructs that can be used to do so, see Gemino and Wand 2004, Mylopoulos 1998, Wand and Weber 2002, Woo 2011), database design approaches, and IQ, which are traditionally considered quite different research domains.The citizen science context revealed an exciting new possibility for combining research strands within IS.
Another controversial idea born out of citizen science is the notion that expertise may be harmful to IQ (Ogunseye and Parsons 2016).Contrary to the prevailing common sense assumption, Ogunseye and Parsons argue that it is the essence of expertise to be focused, and potentially ignore what is deemed irrelevant based on prior knowledge.Citizen science is a setting where discoveries are possible, and thus, it is important to have an open mind (Lukyanenko et al. 2016a).Crowdsourcing strategies where added value is premised on diverse contributions, e.g., open innovation (Brabham 2013;Howe 2008), are a prime example of this principle.
An active stream of research has studied design antecedents of quality in citizen science: many novel IS design solutions are rapidly tested with information quality being the most common outcome variable evaluated.Due to the importance of participants' motivation in creating large quantities of accurate information, design antecedents focused on motivation have received the most attention, including design features that support promotion of collective goals, ability to interact and make friends, rewards, and game-like features (Elevant 2013;Eveleigh et al. 2013;Nov et al. 2011;Prestopnik et al. 2013;Prestopnik and Crowston 2011).It can also offer a setting for research on participatory design (Bowser et al. 2017;Lukyanenko et al. 2016b;Preece et al. 2016), another major IS topic (Bodker 1996;Bratteteig and Wagner 2014;Garrity 2001;Klein and Hirschheim 2001;Kyng 1995).
Growing evidence demonstrates benefits of gamification for data quantity and discovery.For example, Foldit is a highly successful project that turns a complex and esoteric task of designing proteins into an interactive game resulting in important discoveries in healthcare and biology (Belden et al. 2015;Khatib et al. 2011;Khoury et al. 2014).Defying the notoriously mysterious nature of quantum theory, the Quantum Game Jam project (scienceathome.org)effectively engages ordinary people in such tasks as solving the Schrödinger equation.As gamification is quite novel in IS (e.g., Amir and Ralph 2014;Richards et al. 2014) citizen science can pave way for wider adoption of gamification in other settings.At the same time, gamification has potential to subvert IQ by incentivizing Bgaming the system^and irritating or demotivating some of the most prolific and reliable contributors (Bowser et al. 2014), suggesting both that the benefits of gamification are audience specific, and that additional work is needed on this topic.
The quality challenges in citizen science also fuel the search for innovative design solutions, such as applying artificial intelligence techniques like machine learning.Artificial intelligence appears fruitful where knowledge and abilities of volunteers are deemed limited (Bonney et al. 2014a(Bonney et al. , 2014b;;Hochachka et al. 2012).Typically, however, machine learning is applied post hocafter data is capturedto detect trends or label items based on data provided (Hochachka et al. 2012;Tremblay et al. 2010).Citizen science is one of the pioneers of artificial intelligence used to augment and improve quality at the point of data creation.For example, the Merlin App aids people with bird species identification based on uploaded photographs and other basic information, using data from eBird contributors to predict likely species matches.The decision is made in real time and allows people to confirm or reject the automatic recommendation; the machine learning results can also be used to dynamically generate more effective data collection forms.The same project uses post hoc data mining to sift through millions of bird sightings to identify outliers, automatically flag erroneous species identifications, and detect potential vandalism.The substantial progress in the application of artificial intelligence and high-performance computing as well as growing evidence of effective adaptation of successful project designs led to recent Nature (Show 2015) and Science (Bonney et al. 2014a(Bonney et al. , 2014b) ) articles making the cautious conclusion that the problem of data quality in citizen science is Bbecoming relatively simple to address^ (Show 2015, p. 265) for some projects.Such a conclusion, of course, is based on limited indicators of quality based on accuracy of identification (e.g., of biological species) which ignores many other potentially relevant dimensions (Lukyanenko et al. 2016a).At the same time, such signals from the leaders in the citizen science community are quite telling of the significant progress the field made in a very short period of time, which also stands ready to advance the broader field of IS.
Finally, a notable contribution of the citizen science context is investigating IQ in real settings.As visitors of websites can be easily converted into research subjects by assigning them to different control and treatment conditions (Lukyanenko and Parsons 2014), many citizen science experiments can be conducted in field settings (Delaney et al. 2008;Lewandowski and Specht 2015).Despite challenges in controlling for confounds, field experiments offer several advantages over laboratory experiments or surveys.Using a real setting allows tracking real user behavior (as opposed to behavioral intentions).It also allows for research with the population of interest rather than surrogates (e.g., students).Conducting research in a real setting also increases external validity compared to similar studies conducted in laboratory environments.As field experimentation is generally scarce in IS (Siau and Rossi 2011; Zhang and Li 2005), citizen science field experimental evidence can be used to corroborate and compare findings from more prevalent surveys and laboratory data (e.g., Jackson et al. 2016).The examples of research in citizen science provided above are not meant to be comprehensive and exhaustive.Instead, they illustrate the potential of this research domain for advancing knowledge and practice in IS related to IQ.As discussed earlier, citizen science information quality research actively investigates traditional aspects of information quality, novel concepts, methods for conducting research, and approaches to IS development and IQ management.Currently rooted in specific scientific disciplines, citizen science studies have much to offer to the traditional information quality work in IS.At the same time, as information quality is one of the core subject areas for the information systems research, citizen science promises unique opportunities for cross-pollination of theories and methods, and for exporting foundational knowledge from IS to sciences where practitioners actively seek collaborators but may be unaware of IS research community.In the next section, we further consider the role citizen science information quality research may play in connecting IS scholars with the broader scientific community.

Directions for Future Research
Citizen science is becoming a prolific area of research in sciences credited with an increasing number of scientific breakthroughs and discoveries, and indeed, its own journal -Citizen Science: Theory and Practice (Bonney et al. 2016).At the same time, citizen science presents a wicked theoretical and design challenge (Hevner et al. 2004) of ensuring that the data provided by ordinary people is of the highest quality possible for rigorous science.Extensive evidence has emerged to support the effectiveness of enlisting ordinary people in this endeavor, so these wicked challenges are part of what makes citizen science such a promising venue for information quality research.More and more interdisciplinary teams from sciences and humanities continue to devise innovative and ingenious methods for harnessing the power of the people.One can hardly find another example of the convergence of so many different disciplinary perspectives and ideas focused on ensuring high-quality data.This paper demonstrates the strong potential of citizen science to advance IQ and offer unique insights to the information systems community, identifying ways that IS researchers can contribute to this sociallyimportant movement.
Based on our review of research on citizen science IQ, several themes emerged: & Citizen science is being actively pursued by scientists from different disciplines, who are rarely familiar with IS; & Many of the disciplines engaged in citizen science view IQ issues from their own perspectives, using their own vocabularies, with limited incentive to contribute beyond their own areas of research (e.g., to IS literature); and & Much of the research on citizen science is published in non-IS and non-IQ journals, making it likely to be overlooked by IQ scholars.
These observations suggest that citizen science can be an important area of focus for future research that increases contact and collaboration between researchers in IS and other scientific disciplines.Below, we identify several directions in information quality in citizen science to support the assimilation of citizen science research conducted in sciences into IS.
Research Direction # 1 Survey of the state of the art in citizen science.As shown in this paper, there is a wealth of theoretical and practical knowledge that is being continuously generated in non-IS domains.Much of knowledge on IQ in citizen science comes from the broader scientific community, including biology, physics, geography, anthropology, heath care, and engineering.Their interdisciplinary approaches can provide inspiration for new IQ work in IS.We suggest IS researchers should continuously monitor citizen science literature (currently almost entirely outside IS) to find relevant results and solutions.We call on the due diligence of both researchers in IS and the reviewers of such work, as a literature review on citizen science is fundamentally incomplete if it includes IS publications alone.
To guide such surveys of the state of the art, researchers in IS can take advantage of the many reviews of citizen science and IQ published in domain journals in sciences and humanities.Indeed, such reviews have already commenced, but thus far they focus on the citizen science itself (e.g., Prestopnik et al. 2013;Wiggins et al. 2011) or the respective scientific disciplines in which citizen science is considered a research method (e.g., Lewandowski and Specht 2015).Publications Indeed, some studies already demonstrate the potential of citizen science as a source of insight for data production in commercial settings.Ogunseye et al. (2017) triangulated the context of citizen science with product reviews on Amazon.com and showed that the same information quality patterns (related to decline in contribution diversity over time) hold in both environments.These findings support our contention that citizen science can be a fruitful source of insights for information quality issues in corporate and other settings.
Broadly, citizen science also presents an opportunity for IS to share foundational concepts, principles, and design knowledge that can support excellence and advancement in a broad range of scientific disciplines.This would have a variety of benefits for IS, including amplifying its relevance to practice.
As citizen science grapples with particularly daunting IQ challenges, it pioneers theoretical and practical approaches that can improve IQ in other domains.For example, in an analogy to citizen science, the role of telemedicine health IS is connecting patients (domain non-experts, familiar with own conditions) and medical professionals (domain experts) (Hailey et al. 2002).At the same time, since most of citizen science IQ research occurs in sciences, these researchers may be unfamiliar with the IS community, creating opportunity for IS research to bridge this gap by leveraging the knowledge of IQ to benefit other domains.
We thus encourage researchers to: & Conduct cross-disciplinary literature reviews which identify common trends related to IQ (e.g., citizen science and participatory design, citizen science and open source software, citizen science and social media).& Consider cross-disciplinary research questions such as: How does the effect of training differ in tasks that involve semantic (i.e., general, shared, as in typical citizen science) vs. episodic (i.e., personal, typical to social media) knowledge?How does the design of online training systems impact task performance?& Leverage similarities in underlying technology to better understand the impact of designs on IQ.For example, addressing the question of how mobile environments (due to their limited screen space, but also novel affordances related to sensors and mobility) could impede or promote user engagement in citizen science vs social networking settings.
Research Direction # 5 General framework on IQ that considers citizen science.Existing frameworks for understanding an improving information quality have focused on corporate data production (Blake and Shankaranarayanan 2015;Madnick et al. 2009;Wang et al. 1995a).To better integrate citizen science into the broader IQ tradition in IS, existing frameworks need to be extended to include UGC and citizen science.This is important as citizen science does not exhaust the IQ landscape of emerging technologies nor vice versa, so their inclusion should lead to more robust frameworks.Indeed, some IQ topics have received relatively less attention in citizen science compared to other UGC contexts and other areas.For example, much of work that relates to the impact of content moderation and editing appears to originate in the context of online forums and wikis (Arazy et al. 2011;Delort et al. 2011;Kane and Ransbotham 2016;Liu and Ram 2011).Wikipedia is the prototypical case for work on collaborative content production and its impact on IQ (Arazy et al. 2011(Arazy et al. , 2016;;Arazy and Kopak 2011;Liu and Ram 2011).Applying data mining to automatically parse through data to make UGC more usable by consumers tends to concentrate on social media and social networking data (Abbasi et al. 2016;Bifet and Frank 2010;Susarla et al. 2012).Other open collaboration contexts include online reviews (Pavlou and Dimoka 2006), Linked Open Data (Heath and Bizer 2011), micro-tasks and crowdsourcing markets (Deng et al. 2016;Ipeirotis et al. 2010), and blogs (Chau and Xu 2012;Wattal et al. 2010), all somewhat different from citizen science and promising their own unique insights.Thus, it is important to provide a comprehensive assessment of the entire space of UGC.Early work in this direction has been undertaken by Tilley et al. (2016), but considering the growth and diversity of UGC, more work of this kind is needed.The diversity of UGC landscape calls for the development of general framework for understanding IQ in UGC that would also position citizen science relative to other forms of information production.
Accordingly, we call on future studies to: & Create a general IQ framework which would account for citizen science and other emerging technologies.& Integrate the framework on emerging technologies with traditional IQ to provide a general and unified view of IQ management.
As follows from the illustration of some recent theoretical and practical contributions, along with many fertile areas for future work discussed above, citizen science is an area of exciting opportunities for intellectually stimulating and socially impactful research.As IQ is arguably one of the core problems of the information systems discipline, quality becomes a common thread that can connect IS with many other disciplines in sciences and humanities.As astronomers, biologists, physicists, anthropologists, and geographers constantly grapple with the daunting task of ensuring that the data provided by ordinary people meets their high standards, information systems research with its long IQ tradition, explicit IQ focus, and proven IQ solutions has the potential to become the foundation on which the citizen science architecture of the future is built.
In closing, although we focused on the opportunities in conducting information quality research on citizen science in IS, we reiterate the general call made by Levy and Germonprez (2017) for information systems researchers to adopt citizen science as a rich, promising, and socially important research context.For example, one traditional expertise area of information systems researchers is adoption of new technologies (Dwivedi et al. 2015;Larsen and Bong 2016;Sidorova et al. 2008;Venkatesh et al. 2007).Adoption issues are fraught with challenges in citizen science, as information quality concerns place real, or at times, psychological (Lewandowski and Specht 2015) barriers for scientists or policymakers interested in engaging with the public and using citizen-generated data in research or public policy decisions.Conducting research in citizen science also heeds the call within the IS discipline to conduct research that promotes or supports environmental sustainability through innovative information technologies (Goes 2014;Melville 2010;Seidel et al. 2013).We hope that this paper adds to this momentum and impels more work in IS to explore this new research frontier.
Open Access This article is distributed under the terms of the Creative Comm ons Attribution 4.0 International License (http:// creativecommons.org/licenses/by/4.0/),which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Fig. 1
Fig. 1 Publications on citizen science by discipline over time

Table 1
Examples of concepts closely related to citizen science

Table 2
Examples of recent discoveries made by citizen scientists Table3summarizes the exciting new contributions of information quality research and citizen science to the broader information quality discipline.

Table 3
(Endicott et al. 2017;Larsen and Bong 2016)ce researchInformation Quality Research Dimensions IQ-enhancing methods employed in citizen science contextKosmala et al. (2016)focus on IQ in citizen science, but target domain research audiences (e.g., ecologists).More work is needed that surveys the ongoing research in the sciences from the perspective of the information quality tradition in IS, with the aim of assimilating these ideas and findings for the benefit of IS.Who uses the data generated by citizen science and how well does it achieve Bfitness for purpose^for these data users?How does this compare to other instances of data production and use in IS studies?&Find analogs of research problems studied in IS that also occur in citizen science.Once found, consider triangulating studies in IS context (e.g., on product reviews) with studies in citizen science context (e.g., descriptions of planets or galaxies based on digital imagery).This will not only strengthen the generalizability of specific studies but also move the field of IS and citizen science closer.&Conductingsystematic literature reviews of citizen science from IS perspective, including focused reviews on relevant sub-topics to improve the utility and depth of the analysis.This would include translation of disciplinary concepts into IS vocabulary via a nomological network.&Ascitizen science is a rapidly moving field, such reviews should be repeated frequently.Here, a promising strategy is utilizing the novel machine-learning approaches for automatic literature review and knowledge integration currently under active development in IS(Endicott et al. 2017;Larsen and Bong 2016).Indeed, citizen science's vastness and rapid growth is easily Ba big data of IQ research^and can motivate further development of automatic literature review and integration methods in IS.How can designs encourage freedom of expression, promoting novel perspectives and discoveries while ensuring that the data remains consistent and usable for science?& How can designs mitigate participation and reporting biases (e.g., overrepresentation of charismatic species) and deliver more complete and representative data?& How can citizen science be augmented with AI and hybrid intelligence to make it more effective?