Introduction

The success of online labor market (OLM) platforms over the last decade can be attributed to their ability to match buyers of electronically deliverable services with workers, potentially connecting geographically dispersed freelancers and employers. Although the lack of geographical boundaries is one of the biggest advantages of these web-based freelance marketplaces [7], it also poses a challenge. The reason is that OLMs differ from traditional labor markets in the assessment of job applicants. As this assessment takes place fully online, employers and freelancers face a problem of information asymmetry [4, 47]: freelancers can communicate their education and work experience via their online profiles, but information about the quality of their work or their trustworthiness is more difficult to demonstrate [5]. These frictions are exacerbated by the often large geographical and cultural distances between employers and freelancers, which can make it even more difficult to assess the suitability of applicants.

Trying to alleviate this problem, the existing OLM platforms have implemented various reputation systems that aim at aggregating information about freelancers’ talents and previous conduct on the platform [22, 57]. At first glance, the use of reputation systems in an OLM context appears attractive, as such systems may make it easier for employers to assess the abilities and trustworthiness of potential employees. As such, reputation systems are often seen as an indispensable feature that enables business transactions and hiring online that transcend geographical boundaries. They potentially reduce the level of uncertainty that employers face and thereby enhance the efficiency of the online recruitment process.

However, earlier research on reputation systems (in both online and offline contexts) has shown that such systems have a tendency to create winner-takes-all dynamics, in which differences in candidates’ reputations become disconnected from objective differences in ability and quality [19, 25, 41]. Indeed, emerging evidence suggests that many of the existing freelancing platforms are extremely unequal in terms of employees’ earnings [56]. The reason is that many of the existing platforms create market conditions that offer better access to work and higher remuneration to workers who are already in socially better socio-economic positions, thereby reproducing existing inequalities among workers. Considering the enormous scale of OLMs (over the last 5 years, the number of Americans who have freelanced in the past year has risen to 56.7 million, [54]), this can have significant consequences for social inequality and influence the well-being of millions of workers worldwide. So far, comparatively little research has been done to disentangle the twofold effects that reputation systems have on the hiring process in OLMs: on the one hand, they are enablers of online transactions, but, on the other hand, they may also give rise to highly segmented outcomes and reproduce existing inequalities.

In this paper, we seek to address this lacuna by exploring what role the level of information asymmetry in the hiring process may play in the emergence of inequality in OLM platforms. More specifically, we suggest that the level of information asymmetry that characterizes different OLM platforms is one of the main determinants of how much inequality reputation systems will create. Previous research has shown that even in idealized labor markets, information asymmetry can lead to emergence of inequality [53]. We investigate theoretically whether and to what extent commonly used reputation systems may affect the fairness of an OLM platform’s economic outcomes. For this, we use an agent-based computational model, in which we represent key features of OLMs in computer code. We validate the assumptions of our model with empirical web-scraped data from a popular platform and submit this code to systematic computational simulation experiments. In these experiments, we explore how jobs and earnings are redistributed on the platform between workers with various levels of reputation under different contextual conditions of information asymmetry. This approach makes it possible to deal with the complex, self-reinforcing social dynamics that often emerge in reputation systems, and which tend to be difficult to address with verbal reasoning alone [27]. Such dynamics arise from the fact that reputation systems make hiring decisions both interdependent and path dependent: the hiring decisions that some employers have made at one point in time will affect workers’ reputation, thereby affecting hiring decisions by other employers in the future [21, 38]. Results show that information asymmetry can influence the extent to which the reputation systems lead to inequality on OLM freelancing platforms. Contrary to our initial expectations, simulations show that diminishing information asymmetry brings greater inequality in outcomes. We show that reputation systems can potentially reproduce inequalities present in offline labor markets and produce unfair outcomes that disproportionately favor already successful applicants.

In what follows, we first discuss the theoretical background of our work. Second, we present the computational model, providing both mathematical definitions and verbal explanations for each of its different elements, and we empirically validate our model assumptions. Third, we present the results which we obtained from conducting systematic computational simulations with the model and discuss their implications for our understanding of OLMs. We end with conclusions and implications for future research.

Theoretical background

Trust and information asymmetry in online labor markets

Business transactions, be it goods or services, depend on mutual trust between the contracting parties. The reason is that not all aspects of a good or service can be evaluated unambiguously before the purchase and sometimes critical flaws show only after a considerable amount of time [10]. Lack of expertise on the side of the buyer may even make it impossible to evaluate the quality of a good or service before the purchase. This problem is commonly referred to as the problem of information asymmetry [4, 17] and arises in any transaction between two parties. In the case of conducting business online, issues of geographic and cultural distance and a lack of legal protections against fraud across states and international borders can further exacerbate the problem [21, 34]. This does not only concern the question of whether the business partner will deliver at all, but also concerns the agreed upon quality of a service or good.

There are large differences in how different platforms in the gig economy are structured. Some of the platforms offer location- or app-based work (transportation or delivery; e.g., UBER or Deliveroo), or microtasking (e.g., Amazon MTurk), where the tasks are distributed to larger crowds (see [7]). Our focus is on OLMs [30]—freelance marketplaces, where digital labor is distributed via a web-based platform and allocated to an individual freelancer, rather than to a crowd of workers (examples include Upwork, Freelancer, or Fiverr). In these, recruitment is typically initiated by employers who seek a certain service, by opening a new project on the platform. The project serves as a call for bids, in which the employer specifies a price range and expected duration of the project, together with a short description of the required service. Hiring on OLMs can also occur directly without a call for bids, yet in most cases, it takes place via an auction (specifically buyer-determined reverse auctions). As the project becomes public, freelancers have a chance to apply for the job by submitting their bids. This entails stating a price for which they are willing to perform the service and the expected time to finalize the task. The employer can choose to close the auction at any time and choose the winning freelancer from the available bids. After the employer has accepted one of the bids, the work on the project commences. Payments for work can be made either during the project, based on the number of hours worked, or at the end as a lump sum when the project is delivered. Once the freelancer delivers the project, both parties are asked to leave public feedback about their exchange.

Evaluation and selection of applicants by employers are typically based on very limited information about the bidders [21]. Freelancers can easily provide credentials on education or work experience via their profiles, but providing information on less directly observable characteristics—such as quality, trustworthiness, or motivation—is more difficult online [5]. Evaluating the work quality of a freelancer is based on uncertain and unreliable signals found in online profiles.

The extent to which this creates challenges for employers depends on the nature of the tasks that they seek to have completed. Existing OLM platforms feature a large diversity of tasks and vary in the skills that are requested [7], ranging from simple tasks, such as “click-work” or typing, to highly complex tasks, such as training machine learning algorithms [14]. Even more, for some tasks, it is easier to evaluate the quality of the delivered service than for others. For example, it is easier to assess the visual appeal of a graphical design or a newly developed website, than the quality of a translation of a foreign text. Depending on the task at hand, it can be difficult for freelancers to persuade the employer of their capabilities, as it remains a challenge to transmit the information about one’s qualities through the online space to motivate a business exchange.

Reputation systems as a potential solution

Reputation systems present a potential solution to the dilemma of information asymmetry that employers face on OLM platforms. In such systems, the reputation of a given freelancer is an algorithmically generated score that carries information about the worker’s activities and past conduct on the platform. It is based on evaluations from previous employers and can take the form of a star-rating or any other arbitrary scale. Since the reputation score is public and accessible by future employers, it provides an immediate evaluation of employees’ quality of work and motivates them to provide the best service possible. As a result, reputation systems are a crucial feature of OLMs and the reputation score represents a powerful signal during the hiring process [48].

OLM platforms feature a diverse pool of projects [7] and employers are likely to adjust their expectations regarding price and reputation depending on the project they are currently advertising [11]. Especially for projects highly dependent on the quality of the service (complex tasks, e.g. software development, data science, or marketing; see [49]), the reputation of the freelancer becomes a fundamental signal in hiring, as it is assumed to reflect their ability to deliver high-quality work [32]. Simple repetitive tasks with low potential for quality variation (e.g., typing, clerical, or data entry work), by contrast, do not require extensive talent or education and the reputation score will thus be of lower importance. These tasks are expected to be performed as cheaply as possible, and therefore, the price is the deciding factor for choosing the freelancer. In practice, hiring criteria will combine both signals—price and reputation—when choosing a freelancer and the decision is based on a deliberation between the two.

The extent to which employers can weigh the signal of reputation and price depends on the amount of information that is available to the employer during the hiring process [50]. When asymmetry is low, employers have enough signals about the freelancer. The decision to hire a freelancer will thus be more equally based on the required price along with the signaled quality via the reputation score. In a situation with high information asymmetry, however, employers must rely on the most relevant signal for the advertised project: in case of simple projects, selection is guided predominantly by price; for complex projects, selection is guided predominantly by the reputation score.

The (unintended) consequences of reputation systems

Reputation systems have been previously equated with the “invisible hand” that rewards good producers while punishing poor ones [24, p. 53]. Indeed, research shows that reputation systems can offset social biases among a large heterogeneous and independent population of users at online marketplaces [1, 12]. However, processes of reputation formation can also have unintended consequences that might be socially and individually undesirable [27]. For example, Kas et al. [33] have shown that since reputation scores and reviews at online marketplaces are obtained only after completed interactions, initial differences between users in the probability to be selected as transaction partner are crucial. The reason is that early differentials in success are likely to multiply as chances of obtaining more projects in the future increase (see [8, 51, 57]) and result in a significant cumulative advantage over time [18, 42]. Such dynamics are likely to produce core-periphery divides between freelancers that are likely to grow over time.

The problem is particularly severe for newcomers to OLM platforms, who—by definition—have no algorithmically generated reputation. Hence, the only way in which they can convey information about their abilities is by self-portrayals in their profile [13], personal branding [55], educational and employment credentials [21], and digital skill certificates [34]. These signals of quality are, however, easy to fake and, therefore, are often seen as not trustworthy by employers. Reputation scores, by contrast, are often considered a form of a costly signal that is more difficult to fake [58], given that accruing high reputation scores requires large amounts of time and energy, as well as the delivery of high-quality work [23]. The existence of several competing OLM platforms further exacerbates the perceived cost of the reputation score, as reputation accrued on one platform is not transferable to other platforms [35].

Empirical research shows that in anonymous online markets with a reputation system, buyers will trust and pay sellers with a good reputation more [15]. The work of Pallais [44] illustrates this with experimental evidence: Freelancers with no prior work experience who received the experimental treatment—being hired for a project and obtaining positive feedback—almost tripled their relative income in comparison to the control group (not hired for a project). One of the coping strategies that freelancers use to accrue an initial reputation is selling labor at a substantially lower cost to attract the first clients [22]. Furthermore, empirical research has shown that poor reputation leads to lower bid amounts in auctions on the side of workers, to compensate for the perceived disadvantage due to a low reputation score [15]. This finding is in line with the previous studies (e.g., [16, 36, 45]) and highlights the fact that participants in OLMs have direct financial incentives to invest in a good reputation.

Establishing an initial reputation on a platform is harder for some workers than others. Freelancers from less developed countries are disadvantaged relative to those from developed countries in terms of their likelihood of being hired [3]. Furthermore, building up reputation is contingent on spending a significant amount of time on unpaid work such as looking for tasks, taking qualification tests, and researching clients. The International Labour Organization (ILO) estimates that “for every hour of paid work, [freelancers at OLMs] spent 20 min performing unpaid work” [7, p. 4]. Such requirements segment the workforce into freelancers who can afford undertaking a period of low or unpaid work and those who are dependent on earning a steady income to make ends meet—a group of workers who are already in an insecure position in regular labor markets [52]. This presents an important entry barrier for workers with precarious backgrounds or from vulnerable geographies (e.g., the global South), who might not be able to make such investment in terms of time and resources for building up their reputation [6]. As a consequence, OLMs potentially reproduce inequalities present in offline labor markets and produce unfair outcomes that disproportionately favor already successful applicants. The importance of initial differences for obtaining a first project to build reputation, and the penalization of low (or no) reputation by lower payments, introduces an important feedback loop that helps successful workers and further disadvantages workers from vulnerable social and economic circumstances.

The literature underlines the vast differences in the experiences of workers with OLMs. On the one hand, evidence of personal success of workers from the global South and low-income countries shows the possibility of talented workers to realize their potential on the global scale and earn wages far above the level of their home country [39]. On the other hand, testimonies of pay below the livable minimum, high work pressures due to easy replaceability of workers, and a lack of workers’ voice highlight structural problems in how work is allocated at online platforms (see [9, 39]). As Graham et al. demonstrate, “there are important and tangible benefits for a range of workers, [but] there are also a range of risks and costs that unduly affect the livelihoods of digital workers” [26, p. 135]. It is, therefore, crucial to uncover the potential underlying mechanisms that segment the OLMs into winners and losers. Aside from the ongoing pressures for policy response from national and international actors (see [31]), platform reputation and skills are crucial individual resources in the platform economy and a direct source of income, security, and marketplace bargaining power [56]. Despite its global presence, the setup of OLM platforms has the potential to preserve already existing disparities and social divides in access to labor.

Modelling OLM complexity

As the foregoing discussion highlights, reputation systems may yield efficiency benefits for employers in OLM contexts, because they can reduce problems that arise from information asymmetries. Yet, they might also create new inefficiencies, by disconnecting the allocation of tasks to workers from objective quality differences. So far, it is poorly understood how likely it is that such inefficiencies emerge, and which specific characteristics of existing reputation systems might make this emergence more likely. One reason for this is that the self-reinforcing dynamics that reputation systems can create are often difficult to grasp [27].

To address this issue, in this paper, we develop an agent-based computational model of reputation systems in OLMs, that we inform with web-scraped data from an undisclosed OLM platform (see more on data collection in the “Ethics and data availability”). With this approach, we explicitly model the decision processes that underlie the behavior of both employers and freelancers as a dynamic social system. This approach is fitting, because for freelancers, the likelihood of succeeding in auctions is dependent on their previous success and the reputation of other employees in the pool of applicants. This path dependence is difficult to explore with empirical data, since different dynamics may lead to the same outcome. Systematic simulations allow us to re-run the experiment and explore the outcomes of potential alternative worlds, not only in terms of their results but also the variability of the results across different assumptions about individual behavior. Moreover, some of the important aspects of the social processes of interest are not directly observable—such as the extent of information asymmetry at a specific OLM platform. Social simulation offers a possibility to explore large range of implications of different assumptions and their outcomes to establish bounds on the potential effect that reputation systems can have on freelancers’ success under different conditions of information asymmetry.

Model

The model is a stylized representation of an OLM, and, therefore, does not aim to explicitly model any specific platform. However, as it is set up, we argue that our model subsumes the most prominent OLM platforms (e.g., Freelancer, Upwork, Fiverr, etc.) and our experiments yield results that are directly applicable to the context of these platforms. Hence, the results that our model generates can be considered as hypotheses, that can be assessed with appropriate data from actual OLM platforms, as we describe in more detail below. The code for the model, figures, and analyses can be obtained from the corresponding author’s GitHub online repositoryFootnote 1, together with a more detailed model description.

Modelling the recruiting at OLMs

The model contains two types of agents: employers (j; from \(1, \dots , N_j\)) and employees (i; from \(1, \dots , N_i\); also referred to as freelancers or workers), and jobs (k; from \(1, \dots , N_k\); also referred to as projects). The aim of employers is to find the most fitting employees for their projects. The aim of employees is to find as many projects to work on as possible. The ratio of employers and employees stays constant during a simulation run and we carried out a sensitivity test to verify that the results are not dependent on this assumption. The model proceeds in discrete time steps and closely mirrors progression in recruitment on OLM platforms. In each time step, a set of actions take place, which are shown in Fig. 1.

Fig. 1
figure 1

Model flow diagram

Full size image

Each employee can be described by a set of characteristics, namely their true talent score (\(t_i\); unobserved by the employer), the number of previously finished jobs (\(o_i\)), and their earnings obtained on the platform (\(e_i\)). Employees’ talent (\(t_i\)) is initialized as coming from a uniform distribution between 1 and 100. To make the context of the OLM realistic, we assume that the platform has been established for some time already. This means that there is a mix of employees in terms of the jobs that they have received in the past and the reputations that they have developed based on this. To model the distribution of the number of jobs that employees have received before, the number of previously received jobs comes from a random exponential distribution (with \(\lambda = 1/2\); rounded down to the nearest integer) where the majority of users received small number of jobs and a minority of users received a large number of jobs (see [22]). Earnings are initially set to zero, to track how much a given freelancer was able to earn since the start of the simulation. Employers differ in their heuristic by which they evaluate employees: selection weight (\(S_j\), defined below). Jobs differ in complexity (\(z_k\)) and price (\(p_k\); positively correlated with \(z_k\) as more complex jobs are likely to be more expensive; [2]) and are randomly assigned to employers.

Employee’s talent and the number of finished jobs are aggregated into a reputation score (\(r_i\)). Inspired by the insights from previous research [21, 57], the reputation score is calculated at the end of each time step as:

$$\begin{aligned} r_i = q[(e^{t_{i,\text{std}} + o_{i,\text{std}}}) / (1 + e^{t_{i,\text{std}} + o_{i,\text{std}}})], \end{aligned}$$
(1)

where q is a normalizing constant to scale the result between \(1-100\), and \(t_i\) and \(o_i\) are standardized by subtracting the respective mean and dividing them by the respective standard deviation in the employee population, yielding \(t_{i,\text{std}}\) and \(o_{i,\text{std}}\). In short, a freelancer’s reputation is proportional to their (latent) talent (representing their ability to provide high-quality service) and the number of finished projects.

Employees’ values on \(t_i\), \(o_i\), and \(e_i\) are assigned when they enter the simulation (as discussed below), followed by calculating the initial reputation \(r_i\). Their reputation then changes over the course of the simulation as a result of receiving new jobs. Table 1 provides an overview of all agent variables and Table 2 provides an overview of all other model parameters.

Table 1 Overview of all agent variables and their initial values (distributions)
Full size table

The model proceeds as follows: in each iteration, (1) one employer creates a job and employees have an opportunity to apply. Each job receives an application from at least two and up to ten randomly chosen (uniform distribution) employees (\(a_k\), from \(1, \dots , A_k \le 10\)). (2) Employees submit their bids (\(b_{ik}\); bid of employee i on job k)—a price for which they are willing to perform the advertised task (see details in “Employees’ bidding strategy”). (3) After all bids are submitted, the employer evaluates each employee’s bid and their respective reputation and constructs a selection score for each bidding employee, mirroring the best match based on the available information (see details in “Employers’ selection and information asymmetry”). (4) The employee with the highest selection score is selected and awarded the job. After finishing the transaction, the reputation (\(r_i\)) of the winning employee is recalculated and the process starts over until a prespecified number of jobs (\(N_k\)) is finished.

Table 2 Overview of model parameters and ranges for experimental values)
Full size table

Employees’ bidding strategy

Our model distinguishes between the two possible auction formats that platforms typically employ—sealed vs. open bid auction (see [29]). The sealed bid auction format does not permit employees to see each other’s bids, and hence, they decide about their bid only using the information about the job which they are bidding on. In the sealed bid auction, employees submit bids to a project (step 2 in Fig. 1) as:

$$\begin{aligned} b_{ik} = p_k + \epsilon _{ik} \quad \text {where } \epsilon _{ik} \sim N(\mu = 0, \sigma ^2 = 10). \end{aligned}$$
(2)

Employee’s bid on project \(b_{ik}\) consists of the project’s price \(p_k\) and a random noise \(\epsilon _{ik}\), representing variation around this price given unobserved employee characteristics. The employee does not need to separately take into account the project’s complexity, as this is already encoded in its price.

Under the open bid auction format, employees bidding on a project are free to observe the bids previously submitted by other applying workers. As a result, we assume that employees take this additional information into consideration when deciding about their own bid (see [40, 59]). The first bidding employee applies the heuristic used with sealed bid auctions (Eq. 2), since they are the first to bid and there is no other information to adjust to. Subsequent bidders will use the additional information provided by previous bidders:

$$\begin{aligned} b_{ik} = 1 / n \sum _{a_k = 1}^{A_k} b_{a_k} + \gamma _i (r_i - 1/n \sum _{a_k =1}^{A_k} r_{a_k}) \quad \text{where } \gamma _{i} \sim N(\mu = 1, \sigma ^2 = 0.1). \end{aligned}$$
(3)

Thus, applicants take the average bid of all previous bidders as a starting point (first term in Eq. 3) and adjust this value by taking the difference between their own reputation and the average reputation of previous bidders (second term in Eq. 3). The reputation adjustment is weighted by \(\gamma _i\) which represents freelancer’s responsiveness to such information. The intuition behind this heuristic is that applicants with lower than average reputation will try to compete by compensating their reputation with a lower price, whereas applicants with high reputation will ask a higher price due to their verified status on the platform. There is a minimum and maximum bid limit on each project, set by the interval \([0.75 p_k; 1.25 p_k]\). After winning a project, the employee receives the bid amount (\(b_{ik}\)) to their earnings (\(e_i\)).

Employers’ selection and information asymmetry

An employer’s decision to select the winning employee depends on the job’s complexity. For more complex jobs, employers require workers with better reputation, as the quality of the product is crucial and reputation systems serve as a useful signal. For simpler jobs, employers are not as interested in reputation as they are interested in the price for which the employee is willing to perform the task. Accordingly, employers combine these two factors—bid and reputation—and weigh each input according to the complexity of the task when selecting from the available workers. The precise way in which the information about reputation and bids translates into decision strategies, however, depends on the amount of information asymmetry (\(\tau\)) that exists on the platform. This is shown in Fig. 2a, where the level of information asymmetry changes the slope of a logistic function that determines the weighting of information by the employer. Under low information asymmetry (represented by low values of \(\tau\)), the employer relies equally on both, reputation and bid, as there is abundant information to use in a freelancer’s evaluation. On the contrary, under high information asymmetry (represented by high values of \(\tau\)), the employer relies on the most relevant information at hand for the project: for simple jobs, this is the height of a worker’s bid, but for complex jobs, this is the worker’s reputation. The coefficients depicted in Fig. 2 are calculated as:

$$\begin{aligned} S_j = e^{\tau z_{k, cent}} / (1 + e^{\tau z_{k, cent}}), \end{aligned}$$
(4)

and depend on two inputs: \(\tau\) which is the information asymmetry parameter and \(z_{k, cent}\) which is the value of job complexity (centered around 50). For example (see Fig. 2b), if an employer offers a job with complexity of 75 (and information asymmetry parameter \(\tau = 0.061\)), approximately 82% of the decision is based on an employee’s reputation and 18% on their bid.

Fig. 2
figure 2

Employers’ selection strategies under different levels of information asymmetry (\(\tau\)): a shows how mixing of information varies across different levels of information asymmetry, b shows an example of how employers weigh the importance of bid and reputation information

Full size image

Each applying employee is evaluated based on an individual selection score, which uses \(S_j\) as a weight to combine information about the employee’s reputation (\(r_i\)) and bid (\(b_{ik, \text{std}}\)). The selection score for employee i for project k is calculated as:

$$\begin{aligned} SelectionScore_{ik} = S_j r_i + (1 - S_j)(1 - b_{ik, \text{std}}). \end{aligned}$$
(5)

The selection score summarizes an employer’s selection heuristic to choose the employee, given their reputation and bid, for a specific project, given its complexity. In the final step, the employee with the highest selection score is selected as a winner.

Experimental set-up and outcome measures

The number of employees and employers is fixed at 30 and 120, respectively. Each simulation run consists of 200 jobs being auctioned and assigned. The experimental parameters are the information asymmetry and auction type: the information asymmetry parameter \(\tau\) is manipulated between 0.001 and 0.4 in increments of size 0.005 (outside this interval, the function barely changes its slope), while auction type is varied between open and sealed in a full factorial design with 160 experimental conditions. All results are based on averages obtained from 50 independent simulation runs per condition, totaling 8000 simulation runs.

We investigate two main measures: First, we focus on employee’s earnings (\(e_i\)) at the end of each simulation run to evaluate their economic success. Second, we look at the number of received jobs (\(o_i\)). As some workers already have an initial number of jobs at the start of a given simulation run, the value of interest is the number of jobs received since the start of the simulation, which is given by \(o_{i, final} = o_i - o_{i, initial}\).

For ease of presenting the results, each worker is categorized into one of three reputation groups. If their initial reputation is below 33, they are categorized as low, between 33 and 67, they are categorized as medium, and above 67, as high.

We measured the inequality in employee’s monetary outcomes (\(K_{e}\)) as the ratio of average earnings of the high reputation group over those of the low reputation group (\(\bar{e_{\text{high}}}\) and \(\bar{e_{\text{low}}}\), respectively). Similarly, we measure inequality in the average number of jobs received (\(K_{o}\)) as the ratio of the number of jobs (\(o_{i, \text{final}}\)) that the high reputation group received over that of the low reputation group (\(\bar{o_{\text{high}}}\) and \(\bar{o_{\text{low}}}\), respectively):

$$\begin{aligned} K_e= & {} \frac{\bar{e_{\text{high}}}}{\bar{e_{\text{low}}}}, \end{aligned}$$
(6)
$$\begin{aligned} K_o= & {} \frac{\bar{o_{\text{high}}}}{\bar{o_{\text{low}}}}. \end{aligned}$$
(7)

High K values show large distances between the reputation groups, and thereby indicate large disparities in access to work and earnings of workers with different reputations. For example, \(K_e = 2\) implies that freelancers with a high reputation earned twice as much per job as freelancers with a low reputation.

Finally, to gauge the extent of a core-periphery divide, we measure the percentage of workers who did not obtain any projects during a simulation run. We analyze their average reputation to see whether a freelancer’s ability to thrive is related to their reputation. If the average reputation of workers who did not obtain any projects is low, we conclude that low reputation workers (and in particular newcomers) face barriers in access to work due to the reputation system.

Model validation data

Validation with empirical data provides some evidence as to whether the model represents the phenomenon that we aim to model (see [43]). The model formulates two agent-level rules that we validate with two web-scraped datasets. Data were collected from an undisclosed OLM platform (see “Ethics and data availability”), during 2 days in November 2018. The first dataset, focusing on employee decisions about bids, contains data on 3434 bids from 192 randomly selected projects across all job categories. The second dataset, focusing on employer selection strategies, contains 879 bids from 55 awarded projects labelled as “typing job” and 575 bids from 52 awarded projects labelled “Python job”, totaling 1454 observations.

Results

Our results are split into three sections: first, we start by validating our assumption about employer and employee behavior with web-scraped data. Second, we focus on the average earnings and the number of jobs that the artificial employees received across simulation runs, followed by a quantification of the inequality that emerged under various conditions of information asymmetry and auction type. Finally, we investigate the access to work by looking at how many and which freelancers have not received work across simulation runs.

Validating model assumptions

The first rule states that employees make decisions about bids based on available local information and adjust their bids to increase the chances of obtaining projects (see Eq. 2 and Eq. 3). Workers with high reputation ask a higher price for their work, while workers with lower reputation compensate their reputation with a lower bid. Bidding behavior is, therefore, correlated with reputation within each auction. In line with this assumption, the empirical data show a significant positive (Pearson) correlation coefficient between reputation and bids. One standard deviation increase in reputation is associated with approximately 0.1530 standard deviation increase in bids (CI95: 0.1361–0.1699, p value < 0.001).

Table 3 Binary logistic regression model of the probability of winning a project, based on standardized bid and project-centered reputation. Point estimates are in log odds ratios and standard errors in parentheses. Based on the web-scraped OLM data
Full size table

The second agent-level rule states that employers adjust their selection strategy based on information asymmetry of the platform and advertised project complexity. Although we cannot directly measure information asymmetry of a platform, we can compare job categories that are evidently different in their complexity. For simple jobs, we use the category of typing jobs, as these jobs are highly repetitive, cognitively undemanding, and with low variation in quality. For complex jobs, we chose jobs containing use of the Python programming language, as programming is relatively varied from case-to-case, demanding, and quality of the product is highly dependent on freelancer’s quality. The validation is facilitated by a comparison of coefficients in logistic regression models predicting the winner of the auction for each job category shown in Tables 3 and 4.

Table 4 Binary logistic regression model of the probability of winning a project, based on standardized bid and project-centered reputation. Point estimates are in log odds ratios and standard errors in parentheses. Based on the web-scraped OLM data
Full size table

In agreement with Eq. 4, empirical data show that the probability of winning a project decreases with higher bids; although there is a difference in estimated effects, this negative relationship does not significantly differ between the two job categories. Reputation, on the other hand, increases the probability of winning a project. Moreover, the estimated effects are significantly different between job categories; the probability of winning is less dependent on reputation for simple projects and more dependent on reputation for complex projects—in other words, reputation matters more for complex jobs.

These findings suggest that our assumptions about employee selection are plausible and the heuristics represented in the social simulation can be considered credible representation of the investigated phenomenon.

Earnings and job access

Success on OLM platforms can be defined by the volume of jobs and earnings that a freelancer is able to obtain. Figure 3 plots the average earnings and average number of received jobs by level of information asymmetry on the platform along with our inequality measure K, which summarizes the differences in the average pay per job and number of received jobs between high and low reputation freelancers (see Eqs. 6 and 7). The results in Fig. 3 show that recruiting in the simulated OLMs leads to outcomes that considerably favor the high reputation freelancers: regardless of the auction format and level of information asymmetry, workers with high reputation systematically receive higher remuneration and more projects to work on. Success in receiving jobs is dependent on reputation; however, the initial success is likely to further inflate worker’s reputation in a positively self-reinforcing loop, thereby improving their chances of receiving more work in the future.

Fig. 3
figure 3

a Earnings and number of jobs received under different levels of information asymmetry and auction types (open vs. sealed). Different shades refer to low (<33), medium (33–67), or high (>67) reputation (on a 1–100 scale). b Measure of inequality summarizing the distance between high and low reputation workers under different auction types (open vs. sealed) (see Eq. 5)

Full size image

According to the simulation model, information asymmetry influences the distribution of earnings and jobs on OLM platforms. Lower information asymmetry—i.e. the more information employers have—accentuates the differences in success between low and high reputation workers both in terms of earnings and number of jobs received. The results in panel (b) of Fig. 3 show that under low information asymmetry, employees with high reputation can, on average, obtain as many as ten times the number of jobs as the low reputation group. In addition, high reputation employees are on average likely to gain as much as eight-to-nine times the earnings as members of the low reputation group. On the contrary, the downward sloping lines in panel (b) show that higher levels of information asymmetry moderate the extent of inequality of OLM outcomes.

In open bid auctions, with increasing information asymmetry, the distance between high and low reputation workers decreases faster than in sealed bid auctions. The ratio of earnings between high and low reputation groups in open bid auctions becomes as low as 2.5, compared to sealed bid auctions, where it remains as high as 5. The number of received jobs shows a similar pattern; under open bid auctions, high reputation employees acquire approximately 1.5 times as many jobs as low reputation employees. Sealed bid auctions gain a lower yet substantial inequality reduction to slightly more than 3 times as many jobs for high rather than low reputation employees.

These results offer support for the notion that the distributions of earnings and jobs at OLM platforms are influenced by the recruiting systems based on algorithmically generated reputation scores. Although a reduction in information asymmetry is commonly considered desirable in business exchanges, it does not necessarily apply to OLM platforms. Lower levels of information asymmetry at OLM platforms can sway employers to start biasing towards freelancers with the highest reputation score, blocking out access to work for other freelancers.

Emerging barriers at OLMs

At the end of each simulation run, several freelancers were left without receiving any work (regardless of the number of past jobs). In Fig. 4, we plot the size of this group (% of employees) and their average reputation. If jobs were awarded at random, each freelancer would receive on average 1.67 jobs. However, the level of information asymmetry and the reputation system directly influence the proportion of workers who received work during a simulation run. Under low information asymmetry, regardless of the auction type, 37–41% of workers did not obtain any project to work on. Increasing the information asymmetry, this number diminishes rapidly and then levels off at around 20% for open bid auctions and 28% for sealed bid auctions. Together with the results from Fig. 3, it becomes clear that the core of the workforce that repeatedly obtains jobs is relatively small and is concentrated mainly in the ranks of high reputation workers.

Fig. 4
figure 4

Proportion of workers who have not received any work in the course of the simulation run and their mean reputation under open and sealed bid auction format

Full size image

The average reputation of workers who fail to obtain work over a simulation run also changes with information asymmetry. Low information asymmetry leads to a situation in which those who struggle to obtain work are mostly workers with low reputation (hence also newcomers). With increasing information asymmetry, we observe a diverging trend between open and sealed bid auctions. Under open bid auctions, the average reputation of freelancers without work converges to the median reputation of all workers (shown as the dashed line). The probability to obtain work, therefore, becomes less dependent on freelancers’ reputation and more equal among all workers. On the contrary, under sealed bid auctions, the average reputation of freelancers without work remains relatively stable with varying levels of information asymmetry. This shows that redistribution under sealed bid auctions is less dependent on information asymmetry, but freelancers’ success is inherently more dependent on their reputation score.

Overall, open auctions seem to generate fairer outcomes than sealed auctions for the accessibility to work, not only for established workers, but also for newcomers. Open bid auctions maximize the redistribution of work to the largest proportion of the workforce and democratize access to work away from algorithmically generated reputation scores. Sealed bid auctions are much less accessible as they redistribute work to a smaller number of workers with high reputation. To conclude, based on the simulation results, low level of information asymmetry does not offer optimal results when evaluated through its ability to impartially redistribute work among freelancers.

Discussion

Our simulation results demonstrate that reputation systems can play a crucial role in determining the outcomes of OLM platforms. We explored the extent to which information asymmetry may play a role in shaping these outcomes. As we show, information asymmetry influences the extent to which the reputation system may lead to inequality between freelancers. Diminishing information asymmetry on OLM platforms might bring about greater inequality, while higher information asymmetry generates more equal outcomes. This finding is essential, as platforms often go to great lengths to diminish information asymmetry by any means possible—e.g., platform-facilitated tests, premium user badges, or mandatory fields in extensive profiles. As the simulation shows, such measures might, contrary to their intention, be a cause for higher inequality on the platform. As a result, these measures indirectly shift the burden of the transaction from employers to freelancers, as they segment the online workforce into core and periphery—those whose success is driven by their high platform reputation and those who are left to compete for poorly paid projects or fail to obtain projects altogether.

As the reputation score heavily influences freelancer’s future success on the platform, the setup and fine-tuning of the reputation system can be considered a crucial feature of OLM platforms. Reputation scores are an imperfect measure of workers’ skills and capability [9, 13, 21], and therefore, it is problematic that these scores, as a means of algorithmic control, can account for such a large part of a worker’s success. On the one hand, reputation systems are necessary for overcoming the social dilemma of coordinating work between distant and anonymous parties, serving as a signal for employers to make selection decisions among many workers online. On the other hand, however, we show that reputation systems are a source of inequality that can strongly influence the distribution of jobs in OLMs.

The success of freelancers on OLM platforms hinges on the ability to invest in building a reputation—be it working for very low pay or performing jobs that are below worker’s actual skill level [21]. Since reputation scores and reviews are obtained only after completing some work on a given platform, initial differences between users play a major role in freelancers’ success. Workers with strong academic credentials, from developed countries, and previous work experience are likely to have a higher probability of obtaining their first job, even without a reputation score [3, 6]. The existence of numerous online markets further exacerbates this problem, as reputation accrued on one platform is not transferable to the others [35]. As a result, workers have to repeatedly bear the cost of generating a good reputation to access various online markets. Such differentials are likely to snowball into a Matthew effect [42] where the already successful get richer, leaving only little opportunity for success to others. OLM platforms may thus reproduce the existing social and economic disparities from the offline world.

Our simulation findings highlight the potential source of the risks emerging in the literature, such as freelance workers undertaking extremely long working hours for relatively low pay, which often results in overwork, sleep deprivation, and exhaustion [56]. Such behavior may likely stem from the necessity to receive positive reviews to be competitive for better paying projects. Especially in a context where it is easy to dispose of one worker and hire another—which is likely the situation on OLMs—workers involuntarily become strongly entrepreneurial under pressure to protect their interests and settle for substantially lower pay [28, 46]. As such, digital platforms are enablers of virtual exploitation of geographical differences in skills and labor costs [37] and thus mirror the existing global inequalities. Although OLMs were initially considered “’borderless’ virtual business platforms” [20] that were to globally equalize access to labor, our simulation results along with previous research show that recruiting at OLMs based on algorithmically generated reputation can give rise to new and previously unknown barriers in access to labor that need to be scrutinized further.

Conclusions

In this paper, we have introduced a novel tool—agent-based modeling—that can be used to explore many issues occurring on OLM platforms, either prior to or in concert with the use of traditional methodologies and new data sources. We have demonstrated this process through a systematic simulation experiment that shows how such modeling can provide further insight and new ideas for investigation and thus widen the focus of the ongoing research.

We have demonstrated that commonly used reputation systems can create segmented core-periphery outcomes and shown that worker’s ability to obtain jobs on OLM platforms is closely tied to their reputation score. The extent of inequality produced at an OLM depends on the level of information asymmetry at the platform. Information asymmetry is, therefore, demonstrated as one of the factors that explain the twofold nature of reputation systems: on the one hand, reputation systems help overcoming the information barriers in hiring online; while on the other hand, they channel resources to freelancers with the highest reputation score. Diminishing information asymmetry, although widely considered a desirable aim, can in its extreme create exceptionally unequal outcomes, where high reputation workers obtain disproportionately large part of the resources allocated through the OLM platforms.