This article of the journal Gruppe. Interaktion. Organisation (GIO) deals with the question how work and organizational psychology can contribute to a better understanding of work design in crowdwork. Over the last decade, crowdsourcing (CS) has gained much momentum and attention, yet people who use CS as an additional or exclusive source of income are experiencing less consideration overall. Therefore, we define the term crowdwork (CW), and delimit it from related concepts, e.g., CS and gig economy. We then address how work and organizational psychology theory can contribute to the research of CW, with a focus on work design, and where new approaches are necessary. We give an overview of current research in this field, and derive suggestions and recommendations for both further research approaches and also practical application of work design in CW.
Dieser Beitrag der Zeitschrift Gruppe. Interaktion. Organisation. (GIO) befasst sich mit der Frage, wie die Arbeits- und Organisationspsychologie zu einem besseren Verständnis der Arbeitsgestaltung im Crowdwork beitragen kann. In den letzten Jahren hat das Thema Crowdsourcing (CS) viel Beachtung gefunden. Personen, die CS als zusätzliche oder einzige Einnahmequelle nutzen, haben jedoch als eigenständige und neue Erwerbsgruppe weniger Beachtung gefunden. Für diese Form der Arbeit definieren wir den Begriff Crowdwork (CW) und grenzen ihn von benachbarten Begriffen, bspw. CS und Gig Economy, ab. Wir skizzieren anschließend, wie Theorien der Arbeits- und Organisationspsychologie zur gegenwärtigen Forschung mit einem Fokus auf die Arbeitsplatzgestaltung beitragen können und wo neue Ansätze notwendig sind. Wir geben einen Überblick über den aktuellen Forschungsstand und leiten daraus Empfehlungen für die praktische Umsetzung und zukünftige Forschung ab.
“Where have all the workers gone?” asked Bergman and Jean in 2016 in their publication on sample characteristics in industrial and organizational (I-O) psychology. They argued that in I‑O psychology literature, the salaried, core, managerial, professional, and executive employees are overrepresented as samples, whereas wage earners, laborers, firstline personnel, freelancers, contract workers, and other workers are underrepresented and less targeted as samples. One of those underrepresented samples are people performing crowdwork (CW). The term CW encompasses digital work that is outsourced by a person or company to an open, anonymous crowd in the web, which is only contracted and paid for this single and specific work. This work is often limited to a simple single task that can be solved within seconds, but can also be a complex job that needs many hours, or even days, to complete. It is assumed that the total value of the CW market will quintuple from 2016 to 2020 (Huws et al. 2016), with a gross services revenue up to $25 billion (Kuek et al. 2015), making CW an important labor market. Nevertheless, 9.2% of the European, and 6.9% of the German, workforce is still active, or has already performed, some kind of CW in the past (Serfling and Serfling 2019). Following Bergman and Jean’s (2016) recommendation, CW is a rapidly growing workforce so its employees should be investigated by I‑O psychology research more extensively. In this article, we focus on work design for those employees. That means we address how the work can be arranged to be satisfying and motivating for the crowdworker, but also of high-quality performance from the employer’s point of view.
We will start with a brief discourse of different forms of CW and why the diverse spectrum of the definition makes it difficult to compare different CW tasks or generalize findings of research. A more precise definition of the concept is necessary to a) allow for more focused research and interpretable effects, b) help organizations which want to outsource tasks to design successful tasks for a crowd. Furthermore, we will illustrate current CW related research in the field of work design, and point out how classical and new theories in I‑O psychology can enrich and enhance our understanding of CW. Additionally, we derive recommendations and give suggestions for further research in I‑O psychology, as well as practical application in the field of work design.
Definition and categorization
In this article, we will focus on CW that is sometimes used interchangeably with other terms, such as crowdsourcing (CS) and gig economy (GE). However, there is a hierarchical structure within the comprehensiveness of these three terms (Fig. 1). Following the definition of De Stefano (2015), the most comprehensive term is GE, which encompasses all jobs and tasks that are not presented on a full-time employment basis, but as single tasks allocated via the web. These jobs and tasks, which should be solved, are mostly presented on specific online platforms as intermediators. People who are registered on these platforms are called crowd. This leads in most cases to an interplay between three stakeholders in CW work environment instead of a two sided employer-employee relation: a) The employer/crowdsourcer, who seeks to solve a specific job or task, b) the crowd which is hired for solving a single task, and is therefore a short contracted employee and c) the platform, which advertises and distributes the task and moderates the market (Fig. 1). The jobs and tasks performed in GE can be web-based goods like Apps, but also physical, such as services, which need specialized equipment or a specialized location. One such example would be the transportation service Uber, where an independent contractor transports passengers, instead of classical taxi companies.
More specific is the term CS. It was introduced by Howe (2006) in the Wired magazine as a portmanteau word of crowd and outsourcing. It underlines that tasks, problems, and competitions were outsourced and addressed to an anonymous crowd in the web to generate a wide range of ideas or solutions. In contrast to GE, the content of the job or task in CS is limited to web-based work services. As this first definition, the term itself has gained more attention compared to GE, and found its way into science when the first researchers realized that the wisdom of the crowd emerged new forms of work. During the rise in awareness and attention, the definition itself became diffuse (Brabham 2009; Kleemann et al. 2008). Estellés-Arolas and González-Ladrón-de-Guevara (2012) developed an integrated definition for CS by reviewing 32 separate definitions in the literature:
Crowdsourcing is a type of participative online activity in which an individual, an institution, a non-profit organization, or company proposes to a group of individuals of varying knowledge, heterogeneity, and number, via a flexible open call, the voluntary undertaking of a task. The undertaking of the task, of variable complexity and modularity, and in which the crowd should participate bringing their work, money, knowledge and/or experience, always entails mutual benefit. The user will receive the satisfaction of a given type of need, be it economic, social recognition, self-esteem, or the development of individual skills, while the crowdsourcer will obtain and utilize to their advantage what the user has brought to the venture, whose form will depend on the type of activity undertaken. (p. 197)
This definition seems cumbersome for practical use because it conjoins many aspects of CS, but also underlines a vast field that is subsumed by the term (e.g., paid and unpaid tasks), in comparison to GE limited to digital goods only. These tasks can vary in duration for completion, content, or qualification needs.
CW, however, as the most specific term, can be limited to those requests in CS, where financial compensation or countervailing benefit for the worker is provided. This can be paid to all (successful) participants, or just to the best or winner of a competition. CW is a more specified form of CS, although both terms CW and CS are part of the so-called GE. Whereas most research does not differentiate in more detail, and only adds the label paid to CS (Kaufmann et al. 2011; Schulze et al. 2011), we think CW should be an independent group within CS, and the most specific term for three reasons: First, Estellés-Arolas and González-Ladrón-de-Guevara’s (2012) definition also encompasses money that is provided by the crowd (donation or investment, e.g., Kickstarter), and the term paid CS gives no clear direction of compensation, whereas CW does because it suggests that compensation is received for the work conducted. Second, for all involved stakeholders, money is important in CW (e.g., Kaufmann et al. 2011), but not necessarily in CS, where it may be a charitable project or pastime activity. And third, there is evidence that CW task design follows similar mechanisms as classical work design. We will further elaborate on this in the following chapters of the article.
Current research in CW is unstructured, segmented, and caused by the aforementioned blurred terminological use of GE, CS, and CW. This lack of clarity currently results in the application of different definitions, resulting in mixed research findings. This issue threatens practitioners, as well as bedevils the transfer of knowledge from research. There are many platforms offering different solutions and opportunities for addressing a crowd (e.g., MTurk, clickworker or 99designs (for details, see Table 1)). Thus, there is much controversy about different providers and platforms regarding whether their concept should be labeled as CW or GE. Uber and Lyft—taxi platforms—might be a perfect example. Some collate the platforms to CS (Prassl and Risak 2015), whereas others see them as work-on-demand via apps and GE (De Stefano 2016) because it is not a mere online activity. However, the service changed the entire sector of transportation in some countries, and, for example, has effects on health and well-being of the drivers (Skok and Baker 2018). We recommend relying on the definition of Estellés-Arolas and González-Ladrón-de-Guevara (2012) because their approach incorporates the most influential articles with a systematical approach and definitions, and is most sufficient with our differentiation of CW within CS. By this definition, taxi services are not CW because they are not a web-based work but localized, and require a car. Thus, new organizations of taxi services should be considered as GE, but not as CS and CW.
Even if we focus on CW only, this market offers a broad range of possible tasks and work that can be performed. The spectrum can range from small tasks with time to completion durations of a few seconds, such as tagging pictures called click work (mostly prominent on MTurk, e.g., an image is divided into multiple squares and all squares with a traffic light have to be selected) to tasks that resemble traditional full employment (e.g., inventing new product ideas or writing a book on a given topic). There are also differences in how the compensation is arranged. Most smaller tasks rely their payment on a first-come, first-serve arrangement for fast fulfillment. Other tasks are arranged as a competition, where the compensation is only offered for the best solutions. Therefore, a broad definition of the concept CW might not hold the necessary precision needed to describe subgroups.
Some researchers have addressed this problem of a broad market by providing subdivisions for different types of tasks in CW. Leimeister et al. (2016) identified microtask-, marketplace-, design-, testing-, and innovation-platforms, depending on the tasks they offer, as did Boudreau and Lakhani (2013), who discriminated between microtasks and innovation tasks. Although these articles are helpful for sorting different CW platforms, they all undertook their subdivisions via a bottom-up approach. Therefore, the subdivisions might seem logical, but it is not proven whether they are sustained from a practical perspective: Neither the boundaries between subdivisions (e.g., difference of microtasks and marketplace tasks) nor the homogeneity within the subdivisions (e.g., microtasks on platform A vs. platform B) have been investigated yet. We argue that more replications on similar platforms are necessary to demonstrate the generalizability of findings, models, and assumptions as well as limitations and boundaries.
In conclusion, even though the terms CW, CS, and GE are not consistently used, a clear classification is possible. We subsumed their hierarchical order in Fig. 1. Additionally, for all three terms, we give a definition and examples for typical platforms with short descriptions in Table 1. By contrast, the subdivisions and categorizations within CW tasks (e.g., microtask, marketplace) are insufficient, and should be used carefully.
The role of I-O psychology in the explanation of CW
To date, current research on CW is explorative, unstructured, and application-centric for specific platforms because it is still a newer form of work arrangement. But it is also limited in its topics because it is mainly researched by information systems and computer science researchers. This is because CW and CS arose from technical advances, and were first researched by those disciplines (Pedersen et al. 2013; Zhao and Zhu 2014). These disciplines and researchers identify the biggest issues of CW in the communication between crowdworker and platform, the platform architecture and legal questions, such as whether to impose a tax on CW tasks. Those issues are also mainly addressed from a platform or organizational point of view (Buettner 2015), therefore referring to needs and desires of these two stakeholder. Other fields, such as human resource management, so far have only a subordinate role in the research discourse of CW. As Zhao and Zhu (2014) outlined in their review, many articles which contribute to CW lack a sufficient theoretical basis and the use of established models.
Therefore, the crowdworker and their perspective is less targeted in current research. The mechanisms and consequences of doing CW for the crowdworker are not fully understood yet. This leads to different opinions on the status of CW as a form of work. Some researchers criticize exploitation (Ettlinger 2016), and a missing delimitation of work and leisure time (Postigo 2003). Other findings suggest that crowdworkers themselves do not feel overburdened (Busarovs 2013), or do not perceive CW as an additional job at all, when they have a regular contracted job as well (Kuhn 2016). Especially the latter emphasizes that CW can have varying effects in different target groups. Still, we have rarely found this distinction in research.
Target groups that have been researched so far include the chronically ill, and special income groups (e.g., students and pensioners). CW can offer job opportunities when it provides access to the labor market for the chronically ill (Hara and Bigham 2017; Zyskowski et al. 2015), and all those who seek a side job, including students, employees, and pensioners. Thus CW should be understood as an incremental labor market, where only 10–40% consider themselves as a serious crowdworker (Brawley and Pury 2016). Understanding CW participants and general mechanisms of CW is crucial for a well-founded understanding of CW in general (Kuhn 2016). I‑O psychology, with its experience and development in the research of the nature of work, can enrich the current CW research. There are plenty of well-established psychological models and theories which could be applied to CW because they could explain mechanisms that are not sufficiently answered today (e.g., how tasks should be designed to produce a high-quality outcome). Still, we have to demonstrate whether these models are suitable for this new form of work or whether we have to develop new models and theories (e.g., Brawley 2017). We think that established models might be suitable but have to be developed and adjusted which we will discuss and outline in the next chapter.
Additionally, I‑O psychology can enrich the organizational perspective as well. CW is a perfect example for alternative and digital work arrangements. Spreitzer et al. (2017) argued that these new work claims three forms of flexibility in a) employment relationship, b) scheduling, and c) the place where it is accomplished. CW meets those criteria because the employment relations are maximally flexibly, limited to one task only, offer no boundaries in scheduling, and as web-based work can be accomplished wherever access to the web exists. We think CW should be more heeded in I‑O psychology. It can help to understand the changing of jobs and work arrangements, but also delivers possible theoretical assumptions that should be tested on CW. However, I‑O psychology can help to shape the design of work in CW as well. With over one hundred years of work design research, there is a huge base of knowledge, theory, and implications (see Parker et al. 2017), which could be adopted to CW. Initial attempts and findings regarding classical work design in CW have been already done, and will be presented in the next chapter.
While crowdworker as a target group and relevant population are less investigated in general psychology and I‑O psychology literature (status May 2019), crowdsourced data however—as participant recruitment pools—are well established and common in those domains (Behrend et al. 2011; Cheung et al. 2017; Paolacci and Chandler 2014), with well elaborated guidelines for the use as participants (Straub et al. 2016). Therefore, we think I‑O psychology should investigate CW as a new form of work, and its impact on the people performing it. The analysis of work design is a suitable starting point.
Work design in CW: past and future
In the following chapter, we will outline an overview of work design in CW from an I‑O psychology perspective. We will focus on some of the most prominent theories in work design, as well as findings for those theories in CW research. This shall not be a comprehensive overview, but a proposal on how CW can be understood, and on what are the most striking unanswered questions, challenges, and opportunities. First, we will introduce the context in which CW is performed, and how CW is embedded in the macro structure of the work environment. Then, we will inspect the content of the work that is performed in CW, with a focus on the single task on a micro level. In the end, we will merge those two subdivisions to draw a conclusion and point out what makes CW unique for I‑O psychology, and also why it matches aspects of typical work forms as well.
Context in which CW is performed—The makro level
To understand CW as a form of work, we have argued in earlier chapters that it is an incremental labor market, where motivation to participate differs between main or side jobs to mere leisure activities, and where the work has to be designed to fulfill these requirements. Logically, one of the very first and, to date, most researched questions in CW is the reason why people participate in CW. In past research, many articles have dealt with the question of motivation, either by conducting interviews (Brewer et al. 2016), experiments (Rogstadius et al. 2011), surveys (Kaufmann et al. 2011), or meta-analyses (Spindeldreher and Schlagwein 2016), or by reviewing the literature (Hossain 2012). Altogether, payment, enjoyment, challenges, and varieties of tasks were consistently reported as the most common motivational aspects. Most of the research distinguishes between extrinsic motivations, which are addressed by any kind of reward or gain through the performance (e.g., money, higher rank on platform), and the intrinsic motivation which arose from the performance itself (e.g., enjoyment), but ends with this dichotomous subdivision, or introduces its own assumptions (Kaufmann et al. 2011). Nevertheless, one of the most famous theories in psychology—not limited to work, but general behavior—might enlarge our understanding of the motivation in CW: the Self-Determination Theory (SDT; Deci and Ryan 1985). This theory offers two amenities because it divides motivation into more differentiated categories beyond extrinsic and intrinsic, and also postulates that three psychological needs have to be satisfied to lead to intrinsic motivation (Deci et al. 2017). Those needs are autonomy in the decision of action, competence in those actions, and relatedness to others—so being part of social environment (for details see Ryan and Deci 2000). Indeed, there is evidence that job satisfaction in CW is related to the fulfilling of those needs (Brawley 2017). Still, it has to be investigated how CW can actively be shaped to design work that matches these needs for a broad range of crowdworker, and therefore the SDT might be of importance in CW work design.
Beside motivational aspects we have to consider what CW in general offers to and demands of the crowdworkers as well. We outlined in the previous chapter that there is a controversial debate of whether there are any risks and demands in CW. There are prominent psychosocial work design theories, such as the Job-demand-control model (JDC; Karasek 1979), and the Job demands-resources model (JD‑R; Demerouti et al. 2001), addressing this scope. Karasek (1979) divided the work into the demands that cause strain and job control, which has a counterpart because more control, e.g. more autonomy, diminishes strain (e.g., Klonek & Parker 2018). The JD‑R enriches these assumptions by offering a broader scope for the counterparts, by introducing job resources (e.g., self-efficacy, optimism, and feedback; Demerouti et al. 2001). Especially the potential of resources is widely neglected in CW research, when it is seen as a whole job as resources might be easily changed (e.g. switching to a task where a higher self-efficacy is expected, switching to a task of a different requester). However, the positive impact of resources, such as self-efficacy or optimism, on job satisfaction and work engagement has been identified for classical work arrangements (Xanthopoulou et al. 2009), yet it still has to be tested for CW as well. Investigating the JD‑R would enrich our understanding of CW by spotting that the work output relies not only on the top-down of work design, but also of the resources (and job demands which can be changed by the crowdworker by switching the task) of each single crowdworker—still, we know little about specific resources and demands in CW. Additionally, we have to think of CW itself as a resource or demand, influencing the main job or work schedule.
Content of work in CW—The micro level
From those broader theories, we have to take a closer look at the work itself, and at the tasks in particular. In contrast to classical work characteristics, tasks are independent from the whole job because the platform provides the environment, but the tasks are designed and advertised mostly by a requester who is independent from the platform. On the micro level of tasks, Schulze et al. (2011) identified fourteen favored task characteristics that are consistently reported on the platform MTurk (e.g., short task description, background information, a high reputation of a crowdsourcer, i.e., employer, or the multiple availability of the same task). This approach is useful for recording specific characteristics for a platform, but limits it also in the same way: The multiple availability of the same task might work for MTurk, but will not be applicable for other platforms. For example, on platforms where creative work is performed, the multiple availability is not possible because usually a new logo or design just has to be designed once. Additionally, some characteristics which are identified are not comparable to other work forms, e.g. anonymity on platforms. Therefore, more general characteristics of tasks are necessary to compare within CW, but also with other work forms, to understand its mechanisms. The Job characteristics model (JCM; Hackman and Oldham 1975) has been preliminarily investigated in the context of CW (Brawley and Pury 2016). It postulates the existence of core job characteristics, such as skill variety, task identity, task significance, autonomy, and feedback, which leads to the experience of meaningfulness, responsibility, and knowledge at work, which results in satisfaction, and also higher motivation and effectiveness (for details, see Hackman and Oldham 1975). There is evidence that autonomy, skill variety, and task identity in CW are related to satisfaction in single tasks (Kaufmann et al. 2011). Nevertheless, the strongest evidence arises from meaningfulness of a task as a predictor for satisfaction and output quality in single tasks (Brawley and Pury 2016; Chandler and Kapelner 2013). This meaningfulness of tasks is in strong contrast to the legend and self-description of MTurk, which defines their crowd as artificial artificial intelligence (MTurk 2016). This means that crowdworkers solve very simple tasks, such as marking traffic lights on pictures, which humans can still perform better than artificial intelligence, without any background of meaningfulness or further information. On the one hand, Models and theories, such as the JCM on the one hand, are helpful for understanding CW better, and to prove that this work depends on more complex characteristics. On the other hand, CW points out where classical theories, such as the JCM, reach their boundaries because these theories are designed for the description of whole jobs rather than a single task. Whereas meaningfulness plays its role on the micro level, too, task knowledge and responsibility—the other two psychological states of the JCM—could not be demonstrated yet. Nevertheless it might be caused by methodological limitations because the only article on this topic used a very brief questionnaire which does not cover all aspects as the authors discuss in their article (Brawley and Pury 2016).
The interplay of task and job—Uniqueness in CW
From those two levels—micro and macro—one of the most challenging features of CW is revealed: the independence from the whole job of platform work and single tasks. In classical work arrangements, the design of work is regulated by contracts and regulations. This mechanism of control is lost in CW due to a lack of definition of the work form. Nevertheless, it seems obvious that single tasks influence the job, and vice versa. For that reason, both practitioner and researcher in I‑O psychology have to rethink work design. Whereas past models, such as JCM or JD‑R, have been designed to measure the job as a whole object, in CW we face a new work form, where it is advisable to investigate both the job and the single task level. Also, the role of the (crowd)worker has changed. Of course, today we are aware that every worker does not moderate or mediate work design and work output, but can also shape the work design itself under the condition that they have the right to change, the necessary degrees of freedom and qualification. This redesign of work by the employees is called job crafting (Wrzesniewski and Dutton 2001). However, this autonomous modification of aspects of a job for a better fit of one’s own needs, abilities, and preferences in job crafting (Berg et al. 2008) is on a different level in CW. Not only the modification of an existing job, but the creation of completely new workplaces and jobs as main or side occupation introduces an even more extensive definition, which is better addressed by the term of career crafting. All in all, in CW we have a more complex two-level structure of job and task with a crowdworker, whose role in work design is of much higher impact.
From this perspective of crafting, we think there is an opportunity for how I‑O psychology can enhance current research of work design methodically. Until this day, there have been no complex and longitudinal studies that have investigated how the work is accomplished, which would allow for cause-and-effect interpretations, e.g., of the interplay between CW and other aspects of daily life, such as other jobs, leisure activities, and behavior outside the platform. There is a huge amount of techniques in psychology, such as diary-studies, but also reconstruction methods of the day, such as the Day-reconstruction method (DRM; Kahneman et al. 2004), or specific events within a time period, such as the Event-reconstruction method (ERM; Grube et al. 2008)—which are less biased than mere surveys in their investigation of this interplay (Grube et al. 2008). Longitudinal research provides better insight into how the work is composed by the crowdworker on his own. A detailed investigation is not just a matter of importance for CW, but for the whole of I‑O psychology. In the past, jobs have consisted of different tasks, too, but as the jobs and job crafting activities were limited by contractual work arrangements, it has not been necessary to divide between those two levels. Now we have to do so because the whole job, and the tasks, have become more independent. This not only holds true for CW, but also for all flexible work arrangements which meet the same criteria.
CW as a typical form of work
Whereas it seems that CW is unique in its mechanisms, the whole job of CW is a typical modern work form as well. The JCM (Hackman and Oldham 1975), which explains work design as we outlined before, has been advanced and refined for today’s work forms. New characteristics have been added to the JCM, such as specialization, problem solving, and ergonomics (Humphrey et al. 2007). They can also be identified in CW as well. Specialization and problem solving are crucial to some CW tasks, especially in the domains where creativity or solutions for specific problems are desired. Ergonomics, in contrast, are not crucial, but differ in shape in CW. Its great strength, working everywhere and at any time, can cause the same problems (e.g., neck pain, Korhonen et al. 2003) as every workplace does when it does not fit to the person, but in CW, crowdworkers acccount for the configuration on their own. The advancement of the JCM by Morgeson and Humphrey (2006) divides the work design into the three domains task, social, and contextual. Even this classification has experienced further development by Xie et al. (2019). They argue that some work characteristics are hybrids because they refer to more than one of the three aforementioned domains. These hybrid work characteristics are boundarylessness, multitasking, demand for constant learning, and non-work-related interruptions. They should be investigated for a more thorough description of the nature of CW as well. In CW, boundarylessness and multitasking experience no limitations because most platforms in CW do not have fixed working hours or limitations in the number of performed tasks at the same time. It addresses the demand for constant learning in some design and innovation competitions (Leimeister et al. 2009), and challenges the non-work-related interruptions in particular, because CW is performed outside an office. A fast adaptation of these work design features to CW can be a key for a better understanding of the nature of this new work form. Therefore, we must address how CW is arranged.
All in all, on the one hand CW is unique in its interplay between job and tasks, and the complex role of crowdworkers in designing their own workplace. On the other hand, CW also has aspects of other (more classical) work forms, which fits perfect into today’s advances in work design. We think a more extensive research of CW in I‑O psychology, with a more zoomed out perspective on job and task level regarding well established theories and longitudinal measurements, will advance current research for good reason: CW may be a small market compared to the global economy, but it provides—as part of the GE—an outlook for future work arrangements. We think that all vivid and application-oriented sciences should adapt to new markets by finding out what is unique to the phenomenon, and what follows well-known mechanisms described by established theories.
Future research guideline
We suggest that I‑O psychology can contribute substantially to the current research in work design in CW by expanding and developing it mainly in two topics. First, by offering a broader framework for a better understanding of the nature of work that is inherent in CW and its unique interplay of the job and task level. This can be accomplished by a more comprehensive focus using established theories, such as SDT (Deci and Ryan 1985), JCM (Hackman and Oldham 1975), JDC (Karasek 1979), and JD‑R (Demerouti et al. 2001), as well as referring to newer approaches and extensions (e.g., Xie et al. 2019). This is a win-win situation for CW as a new work form and for the advances in its work design theory. Although well-established I‑O psychology models are a fruitful pathway to a better understanding of CW, the results give suggestions for further theory development in psychology. The CW market might be a window to future directions in the evolution of work. Therefore, it offers some idiosyncrasies in its arrangements. A flexible workplace might be of less novelty, but the way of commitment and contracting limited to single tasks, even more than telework and home office, as well as a blurred line between work and leisure activity, challenges current approaches. Whereas work becomes segmented into single tasks or gigs, the design of this work has to match this level, too. We think the first question to answer in CW work design is: Can we measure work design on the level of single tasks even in microtasks (i.e., very short tasks)? Another question that derives from the first question is which of those task characteristics are relevant?
We then have to look at the level of the whole CW job of a single person as well, together with other jobs and the leisure activity of that person, as most crowdworkers perform CW as a hobby (Giard et al. 2019). We think these challenges can be met by incorporating adjoining topics and disciplines, such as differential psychology and its vast body of research regarding leisure activities. Nevertheless, most research is more of a snapshot. More longitudinal designs are necessary to fill the gap between anonymous user profiles on platforms, and the people performing CW. If we look at the level of single tasks, the daily CW job of a person is a composition of many tasks. We have to investigate how single tasks influence the perception of the whole job design in CW. The third question that we postulate is: How does the perception of single tasks of a job (in CW) with the perception of the whole job interplay in the domain of job design? Are there additive effects, multiplier or even decreasing effects and how do the subjective perceptions differ from objective measurements (e.g. Rau 2010)?
This can be accomplished by using experience sampling methods, such as DRM or ERM. We need the application of more multilevel longitudinal research designs for a deeper understanding of the nature of work in CW, with all its facets measuring both the levels of single tasks and the whole job. Thus, I‑O psychology adequately addresses a huge development in the way future work might be designed and organized.
In this article, we illustrated the current status in work design for CW, with a focus on the research perspective. We identified many topics and unanswered research questions. Nevertheless, there are first implications and insights into the topic of CW that can be adopted to practical use to address both groups: the individual who performs the CW, and the organization (i.e., requester) which wants to make use of CW.
First, whenever the term CW, GE, or paid CS is used, it must always be clarified as to what specific work is meant. All terms are some kind of umbrella term for a multifaceted web-based work forms but addressing different types of them. In a first step, it should be identified which kind of work shall be arranged as an open call to an anonymous digital crowd of potential employees. Does the work consist more of highly granulated small tasks, or more complex ones, like design, innovation, and idea development? Second, a decision needs to be made whether the work will be arranged in a first-come-first-serve arrangement for fast fulfillment, or as a competition, where the compensation is only offered for the best solutions. On the basis of this assumption, organizations should pick the platform that addresses the most appropriate crowd for the desired work.
Then the work has to be designed in a manner to produce the best outcome. Still, payment is most crucial for good performance in CW; however, quality, workers satisfaction, and well-being are boosted when competence, autonomy, and relatedness are experienced and meaningfulness can be found in the work. We encourage addressing these needs of the crowdworker as well. Although it seems alluring to offer any work to the crowd, the downsides are low quality and dissatisfaction (Iaffaldano and Muchinsky 1985). There are still other practical questions outside the scope of this review of CW work design that practitioners could address. For example, the reputation of a requester is of importance for crowdworkers (Schulze et al. 2011). It is on behalf of the platform and requester in this market, without any commitment and bonds of legal contracts to keep high performers. And what about the legacy of the full-time crowdworker in Germany? It is an illusion, as most crowdworkers use CW at most as an incremental job, or a mere leisure activity. Therefore, CW must be an attractive offer in the way the work is designed. Whereas the actual CW research lacks a comprehensive overview of a specific work design, it is mostly advisable to access more common models, such as JCM or JD‑R. Thus, we recommend designing tasks with regards to classical work design approaches, and by the help of I‑O psychologists.
MTurk (2016). Celebrating 11 years of Artificial, Artificial Intelligence. Retrieved from. https://blog.mturk.com/celebrating-11-years-of-artificial-artificial-intelligence-e94ec6a56b0b. Accessed: 28 Nov 2019.
Behrend, T. S., Sharek, D. J., Meade, A. W., & Wiebe, E. N. (2011). The viability of crowdsourcing for survey research. Behavior Research Methods, 43(3), 800–813.
Berg, J. M., Dutton, J. E., & Wrzesniewski, A. (2008). What is job crafting and why does it matter? Retrieved from. https://positiveorgs.bus.umich.edu/wp-content/uploads/What-is-Job-Crafting-and-Why-Does-it-Matter1.pdf. Accessed: 23 Nov 2019.
Bergman, M. E., & Jean, V. A. (2016). Where have all the “workers” gone? A critical analysis of the unrepresentativeness of our samples relative to the labor market in the industrial–organizational psychology literature. Industrial and Organizational Psychology, 9(1), 84–113.
Boudreau, K. J., & Lakhani, K. R. (2013). Using the crowd as an innovation partner. Harvard Business Review, 91(4), 60–69.
Brabham, D. C. (2009). Crowdsourcing the public participation process for planning projects. Planning Theory, 8(3), 242–262.
Brawley, A. M. (2017). The big, gig picture: We can’t assume the same constructs matter. Industrial and Organizational Psychology, 10(4), 687–696.
Brawley, A. M., & Pury, C. L. S. (2016). Work experiences on MTurk: Job satisfaction, turnover, and information sharing. Computers in Human Behavior, 54, 531–546.
Brewer, R., Morris, M. R., & Piper, A. M. (2016). “Why would anybody do this?”: Older adults’ understanding of and experiences with crowd work. In CHI ’16 Proceedings of the 2016 CHI Conference on Human Factors in. Computing Systems, 2246–2257. ACM Press.
Buettner, R. (2015). A systematic literature review of crowdsourcing research from a human resource management perspective. In HICSS ’15 Proceedings of the 2015 48th Hawaii International Conference on System. Sciences, 4609–4618. IEEE Computer Society.
Busarovs, A. (2013). Ethical aspects of crowdsourcing, or is it a modern form of exploitation. International Journal of Economics & Business Administration, 1(1), 3–14.
Chandler, D., & Kapelner, A. (2013). Breaking monotony with meaning: Motivation in crowdsourcing markets. Journal of Economic Behavior & Organization, 90, 123–133.
Cheung, J. H., Burns, D. K., Sinclair, R. R., & Sliter, M. (2017). Amazon Mechanical Turk in organizational psychology: An evaluation and practical recommendations. Journal of Business and Psychology, 32(4), 347–361.
Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior. New York: Springer.
Deci, E. L., Olafsen, A. H., & Ryan, R. M. (2017). Self-determination theory in work organizations: The state of a science. Annual Review of Organizational Psychology and Organizational Behavior, 4(1), 19–43.
Demerouti, E., Bakker, A. B., Nachreiner, F., & Schaufeli, W. B. (2001). The job demands-resources model of burnout. Journal of Applied Psychology, 86(3), 499–512.
Estellés-Arolas, E., & González-Ladrón-de-Guevara, F. (2012). Towards an integrated crowdsourcing definition. Journal of Information Science, 38(2), 189–200.
Ettlinger, N. (2016). The governance of crowdsourcing: Rationalities of the new exploitation. Environment and Planning A: Economy and Space, 48(11), 2162–2180.
Giard, N., Hemsen, P., Hesse, M., Löken, N., Nouri, Z., Reddehase, J., & Schulte, J. (2019). Interdisziplinäre Befragung von Crowdworkern. Technical Report. https://doi.org/10.4119/unibi/2936990.
Grube, A., Schroer, J., Hentzschel, C., & Hertel, G. (2008). The event reconstruction method: An efficient measure of experience-based job satisfaction. Journal of Occupational and Organizational Psychology, 81(4), 669–689.
Hackman, J. R., & Oldham, G. R. (1975). Development of the Job Diagnostic Survey. Journal of Applied Psychology, 60(2), 159–170.
Hara, K., & Bigham, J. P. (2017). Introducing people with ASD to crowd work. In ASSETS ’17 Proceedings of the 19th International ACM SIGACCESS Conference on Computers and. Accessibility, 42–51. ACM Press.
Hossain, M. (2012). Users’ motivation to participate in online crowdsourcing platforms. In R. S. R. Kasim (Ed.), International Conference on Innovation, Management and Technology Research (ICIMTR), 2012 (pp. 310–315). Piscataway, NJ: IEEE.
Howe, J. (2006). The rise of crowdsourcing. Wired magazine, 14(6), 1–4.
Humphrey, S. E., Nahrgang, J. D., & Morgeson, F. P. (2007). Integrating motivational, social, and contextual work design features: A meta-analytic summary and theoretical extension of the work design literature. Journal of Applied Psychology, 92(5), 1332–1356.
Huws, U., Spencer, N. H., & Joyce, S. (2016). Crowd work in Europe. Preliminary results from a survey in the UK, Sweden, Germany, Austria and the Netherlands. Commissioned report, Foundation for European Progressive Studies, 1–51.
Iaffaldano, M. T., & Muchinsky, P. M. (1985). Job satisfaction and job performance: A meta-analysis. Psychological Bulletin, 97(2), 251–273.
Kahneman, D., Krueger, A. B., Schkade, D. A., Schwarz, N., & Stone, A. A. (2004). A survey method for characterizing daily life experience: The day reconstruction method. Science, 306(5702), 1776–1780.
Karasek, R. A. (1979). Job demands, job decision latitude, and mental strain: Implications for job redesign. Administrative Science Quarterly, 24(2), 285–308.
Kaufmann, N., Schulze, T., & Veit, D. (2011). (2011). More than fun and money. Worker motivation in crowdsourcing—A study on Mechanical Turk. AMCIS. Proceedings, Vol. 11 (pp. 1–11).
Kleemann, F., Voß, G. G., & Rieder, K. (2008). Un(der)paid innovators. Science, Technology & Innovation Studies, 4(1), 5–26.
Klonek, F. E., & Parker, S. (2018). Teil-autonome Arbeitsteams oder tayloristische Produktionslinie? Welche Erkenntnisgewinne bieten simulationsbasierte Work Design Trainings. Gruppe. Interaktion. Organisation. Zeitschrift für Angewandte Organisationspsychologie (GIO), 49(2), 167–175.
Korhonen, T., Ketola, R., Toivonen, R., Luukkonen, R., Häkkänen, M., & Viikari-Juntura, E. (2003). Work related and individual predictors for incident neck pain among office employees working with video display units. Occupational and Environmental Medicine, 60(7), 475–482.
Kuek, S. C., Paradi-Guilford, C., Fayomi, T., Imaizumi, S., Ipeirotis, P., Pina, P., & Singh, M. (2015). The global opportunity in online outsourcing. Washington, DC: World Bank Group, openknowledge.worldbank.org.
Kuhn, K. M. (2016). The rise of the “gig economy” and implications for understanding work and workers. Industrial and Organizational Psychology, 9(1), 157–162.
Leimeister, J. M., Huber, M., Bretschneider, U., & Krcmar, H. (2009). Leveraging crowdsourcing: Activation-supporting components for IT-based ideas competition. Journal of Management Information Systems, 26(1), 197–224.
Leimeister, J. M., Zogaj, S., Durward, D., & Blohm, I. (2016). Systematisierung und Analyse von Crowd-Sourcing-Anbietern und Crowd-Work-Projekten. No., Vol. 324. Studie der: Hans-Böckler-Stiftung.
Morgeson, F. P., & Humphrey, S. E. (2006). The Work Design Questionnaire (WDQ): Developing and validating a comprehensive measure for assessing job design and the nature of work. Journal of Applied Psychology, 91(6), 1321–1339.
Paolacci, G., & Chandler, J. (2014). Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23(3), 184–188.
Parker, S. K., Morgeson, F. P., & Johns, G. (2017). One hundred years of work design research: Looking back and looking forward. The Journal of Applied Psychology, 102(3), 403–420.
Pedersen, J., Kocsis, D., Tripathi, A., Tarrell, A., Weerakoon, A., Tahmasbi, N., … Vreede, G.-J. de (2013). Conceptual foundations of crowdsourcing: A review of IS research. In HICSS ’13 Proceedings of the 2013 46th Hawaii International Conference on System Sciences (pp. 579–588). IEEE Computer Society.
Postigo, H. (2003). From pong to planet quake: Post-industrial transitions from leisure to work. Information, Communication & Society, 6(4), 593–607.
Prassl, J., & Risak, M. (2015). Uber, taskrabbit, and co.: Platforms as employers-rethinking the legal analysis of crowdwork. Comparative Labor Law & Policy. Journal, 37, 619–652.
Rau, R. (2010). Befragung oder Beobachtung oder beides gemeinsam? – Welchen Instrumenten ist der Vorzug bei Untersuchungen zur psychischen Belastung und Beanspruchung zu geben? Zentralblatt für Arbeitsmedizin, Arbeitsschutz und Ergonomie, 60(9), 294–301.
Rogstadius, J., Kostakos, V., Kittur, A., Smus, B., Laredo, J., & Vukovic, M. (2011). An assessment of intrinsic and extrinsic motivation on task performance in crowdsourcing markets. Fifth International AAAI Conference on Weblogs and Social. Media, 2011. https://doi.org/10.13140/RG.2.2.19170.94401
Ryan, & Deci (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67.
Schulze, T., Seedorfer, S., Geiger, D., Kaufmann, N., & Schader, M. (2011). Exploring task properties in crowdsourcing-an empirical study on Mechanical. ECIS. https://dblp.uni-trier.de/db/conf/ecis/ecis2011.html
Serfling, O., & Serfling, O. (2019). Crowdworking Monitor No. 2. Technical Report. https://doi.org/10.13140/RG.2.2.36135.91044
Skok, W., & Baker, S. (2018). Evaluating the impact of Uber on London’s taxi service: A strategic review. Knowledge and Process Management, 25(4), 232–246.
Spindeldreher, K., & Schlagwein, D. (2016). What drives the crowd? A meta-analysis of the motivation of participants in crowdsourcing. In PACIS 2016 Proceedings (p. 119).
Spreitzer, G. M., Cameron, L., & Garrett, L. (2017). Alternative work arrangements: Two images of the new world of work. Annual Review of Organizational Psychology and Organizational. Behavior, Vol. 4 (pp. 473–499).
De Stefano, V. (2015). The rise of the just-in-time workforce: On-demand work, crowdwork, and labor protection in the gig-economy. Comparative Labor Law & Policy. Journal, 37, 471–504.
De Stefano, V. (2016). Introduction: Crowdsourcing, the gig-economy and the law. Comparative Labor Law & Policy. Journal, 37(3), 461–470.
Straub, T., Hawlitschek, F., & Weinhardt, C. (2016). Towards a guideline for conducting economic experiments on Amazon’s Mechanical Turk. Proceedings of the Second KSS Research Workshop Karlsruhe, Germany, February 2016. pp. 158–166. https://doi.org/10.5445/IR/1000076317
Wrzesniewski, A., & Dutton, J. E. (2001). Crafting a job: Revisioning employees as active crafters of their work. The Academy of Management Review, 26(2), 179–201.
Xanthopoulou, D., Bakker, A. B., Demerouti, E., & Schaufeli, W. B. (2009). Work engagement and financial returns: A diary study on the role of job and personal resources. Journal of Occupational and Organizational Psychology, 82(1), 183–200.
Xie, J. L., Elangovan, A. R., Hu, J., & Hrabluik, C. (2019). Charting new terrain in work design: A study of hybrid work characteristics. Applied Psychology, 68(3), 479–512.
Zhao, Y., & Zhu, Q. (2014). Evaluation on crowdsourcing research: Current status and future direction. Information Systems Frontiers, 16(3), 417–434.
Zyskowski, K., Morris, M. R., Bigham, J. P., Gray, M. L., & Kane, S. K. (2015). Accessible crowdwork?: Understanding the value in and challenge of microtask employment for people with disabilities. In CSCW ’15: Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (pp. 1682–1693). New York, NY: ACM.
This research was funded by the Ministry of Culture and Science of the German State of North Rhine-Westphalia within the research programme “Digital Future”. Open Access funding provided by Projekt DEAL.
About this article
Cite this article
Schulte, J., Schlicher, K.D. & Maier, G.W. Working everywhere and every time?—Chances and risks in crowdworking and crowdsourcing work design. Gr Interakt Org 51, 59–69 (2020). https://doi.org/10.1007/s11612-020-00503-3
- Work design