In early 2010, the publication of a correspondence testing study for Germany (Kaas and Manger, 2012) triggered a lively public debate about discrimination in the hiring decisions of German firms. The study finds that applicants with a Turkish-sounding name are on average 14 percentage points less likely to receive an invitation for a job interview than applicants with a German-sounding name who are otherwise similar. In small- and medium-sized firms, this difference is even larger and amounts to 24 percentage points. Against this background and inspired by experiments in other European countries, the Federal Anti-Discrimination Agency initiated a field experiment with anonymous job applications in Germany to investigate their potential in combating hiring discrimination.
3.1. Project setup and evaluation design
The German experiment officially commenced in November 2010 and lasted for twelve months in each of the participating organizations (Krause et al., 2012b). In total, eight organizations voluntarily joined the experiment. Among those organizations were four international companies, one medium-sized firm, and three public administrations. These organizations committed themselves to review anonymous job applications in specific departments for different types of jobs. The vacant jobs included apprentices, students, specialists as well as management positions. The characteristics that were made anonymous include the applicant’s name and contact details, gender, nationality, date and place of birth, disability, marital status and the applicant’s picture. Additionally, any information about professional experience should only indicate the duration of employment spells and not their actual start date or end date. 2 Importantly, applications were treated anonymously only in the first stage the recruitment process, i.e., when recruiters decide about which candidates to invite for a job interview. However, as soon as the interview invitations had been sent out, the candidates’ identity was revealed.
The goal of this project was twofold. First, a qualitative evaluation should shed light on the practicability of anonymous job applications. For this purpose, structured interviews with the respective recruiters and a survey among the applicants were conducted. The practicability of four different methods was examined which are in principal suited to make sensitive information anonymous:
standardized application forms in which sensitive information is not included;
refinements of existing online application forms such that sensitive information is disabled;
copying applicant’s non-sensitive information into another document;
blackening sensitive information in the original application documents.
Results on practicability are reported in Section 3.2.
As a second goal of the experiment, the effects of anonymous job applications on interview invitation probabilities should be empirically investigated. The necessary data on applicants was collected by the participating organizations during the project. To empirically investigate the effects of anonymous job applications on interview invitation probabilities, we conduct a two-step analysis. In both steps, we apply linear probability models.
First, using data on anonymous job applications only, we analyze whether interview invitation probabilities systematically differ by gender and migration background when these characteristics are unknown to the recruiters. Our hypothesis is that, if the anonymization is effective, the applicants’ anonymous characteristics cannot determine the interview invitation probability. Such influence could only be present in two cases. One possibility is that recruiters are able to deduce minority group membership status from other characteristics. Another possibility is that minority group membership status is correlated with other characteristics such as, for example, educational attainment or professional experience.
Second, to detect possible effects of introducing anonymous job applications, we compare the situation of anonymous job applications with a counterfactual situation, i.e., a comparable situation with standard job applications. Theoretically, three different effects are possible if we assume that anonymous job applications eliminate any differential treatment of minority groups. First, anonymous job applications may cause the interview invitation probabilities of minority groups to increase and to equalize with those of majority groups. In this situation, minority groups are discriminated against with standard applications. Second, if discrimination is not present with standard applications, anonymous job applications have no effect on interview invitation probabilities. Third, if instruments such as affirmative action are applied through which minority groups actually benefit from higher interview invitation probabilities with standard applications, the introduction of anonymous job applications may cause interview invitation probabilities of minority groups to decrease. Given that the hiring processes are very heterogeneous (sometimes even within the same organization) with respect to the type of vacancy, the method that is used to make sensitive information anonymous and the number of applicants, we conduct our analysis separately for each hiring process. Results on the effects of anonymous job applications are for one example recruitment process reported in Section 3.3, and for all processes in Section 3.4.
An overview of all hiring processes is given in Table 1. For nine processes are data on the anonymous job applications available. Additionally, the organizations provided data on control groups for seven out of nine processes. The number of applications range from 27 to over 800 per hiring process which provides some indication for the heterogeneity between the different processes. Copying non-sensitive information into another document is used only once as method of anonymization, whereas blackening is the most frequent method. Certain descriptive statistics of processes 1, 4, 8 and 9 stand out. 93 percent of the applicants in process 1 are female, whereas this applies to only to 2 percent of the applicants in process 4. Migrants usually make up at least 17 percent of the applicants. However, in process 8 this number is only 3 percent. Finally, the callback rate in process 9 is 93 percent, which is unusually high. The organization explains this number with a surprisingly high incidence of refusals by the candidates, a generally rather less qualified cohort of applicants and a slow progression of the recruitment process. Most control groups come either from recruitment processes of similar vacancies in the past (e.g., from the preceding year) or from hiring processes for similar vacancies, which take place simultaneously to the anonymous hiring process. Process 7 is an exception in this regard, as in this organization the applicants were reviewed in the standard way without anonymization which led to interview invitations. These same applications were subsequently anonymously reviewed by other recruiters, which are similar in demographic characteristics and experience to the first recruiter team according to the organization. This second review represents the control group for this process, although it did not lead to any interview invitation, but was entirely fictitious. This situation is difficult, given that the recruiters knew about their fictitious review, and needs to be taken into account in the empirical analysis.
3.2. Results on practicability
The results of the qualitative evaluation can be summarized as follows. 3 Both, recruiters and applicants do not report major practical problems in implementing anonymous job applications. In particular, the use of a standardized application form appears as a very efficient method—at least, once such a form is developed. As a non-negligible side effect, a standardized application form increases comparability between the applicants. Organizations which used this method even consider continuing with anonymous job applications after the experiment. In contrast, in particular the method of blackening the applications is a very time-consuming and error-prone technique. Irrespective of the implementation method, most recruiters appreciate being forced to reconsider their own recruitment practices as well as the stronger focus on qualifications and skills that results with anonymous job applications. Recruiters regard particularly the removal of the applicant’s picture as a positive development, since pictures often lead to misleading first impressions in the reviewing process. Moreover, when hiring anonymously, the organizations can present themselves as open-minded employers that aim at objective selection processes. Several recruiters rate this as a benefit for their corporate image.
All applicants who applied via an anonymous standardized application form were asked to answer a short survey about their experiences with this application method. As only the candidates who were reviewed with this method actively take notice of the anonymous job applications, this survey population displays a selection out of all applicants that were part of the experiment. About 41 percent of the applicants self-evaluate their chances to receive an interview invitation as higher with anonymous job applications and about 33 percent rate their chances to be equal between anonymous and non-anonymous hiring methods. About 48 percent of the respondents generally prefer anonymous job applications over standard ones, whereas 31 percent prefer the opposite. Moreover, a majority of 44 percent do not rate the time spent for the application process as being different between anonymous and standard applications, whereas 31 percent claim to need more time for standard applications. In addition, more than half of the applicants (54 percent) declare that the potential to present themselves is higher with anonymous job applications or that it is at least not different from that with standard applications.
3.3. Quantitative effects: example recruitment process
We first describe the effects of anonymous job applications for one specific recruitment process in detail (see Table 1, process 8). We select this process as an example because of a relatively large sample size and because data on two different control groups are available. We have information about 809 applicants whose applications were anonymously reviewed during the experiment. About half of these applicants are female, about 3 percent have a migration background and their average age is 20 years. The applicants have on average less than one year of work experience, report less than one previous employer, and completed two internships. About one third receives an interview invitation as the outcome of the first stage in this hiring process.
Table 2 shows the analysis of the determinants of the interview invitation probability with anonymous job applications in this example. In this first step, we only consider those candidates whose applications were anonymously reviewed. Both potentially disadvantaged groups of female applicants and applicants with a migration background have a slightly higher probability to be invited for interview compared to men and natives, respectively. 4 However, these differences are not statistically significant in column (1). When controlling for additional characteristics of the applicants such as age, educational attainment and number of internships in column (2), the coefficient of the female dummy drops and becomes virtually zero. The coefficient of the migrant dummy slightly increases, but remains statistically insignificant. In this example, it thus appears that female applicants and applicants with a migration background do not face systematically different interview invitation probabilities when their applications are anonymously reviewed.
In the second part of our empirical analysis we analyze the effects of introducing anonymous job applications. To compare the situation of anonymous job applications with the counterfactual situation of standard applications, data for two potential control groups is available in the example of this specific recruitment process. The first potential control group consists of applicants who applied for the same vacancy in the previous year using standard applications. This group would represent an appropriate control group if no other changes had occurred between the two years besides the fact that applications were anonymously reviewed in 2011. The group of applicants in 2010 consists of 1,357 individuals and is therefore by about 500 applicants larger than in 2011 when applications were anonymously reviewed. However, there are no significant differences in the shares of female or migrant applicants, or in the applicants’ average age. In both groups, equal shares of applicants have completed the general qualification for university entrance or the qualification for technical college entrance. Only the number of internships is significantly different as candidates in 2010 report on average only one internship.
Table 3 displays the results of the comparison between the two recruitment processes in 2010 and 2011. Results in the first row of columns (1) and (2) indicate that the interview invitation probability is about 6 to 8 percentage points higher for applicants who are anonymously treated. However, this difference might be related to the different number of applicants. Fewer individuals applied in 2011 when applications were anonymously reviewed, which could result in a higher overall share of interview invitations. Results in columns (1) and (2) moreover show that female applicants are significantly more likely to be invited for an interview—irrespective of whether they belong to the treatment or control group. Results in column (3) include, in addition to the treatment dummy and the dummy variables for the two demographic groups, two interaction terms between these variables. The coefficient on the interaction term between women and anonymous job applications indicates that female applicants are about 9 percentage points less likely to receive an interview invitation when their applications are anonymously reviewed. To estimate the causal effect of anonymous job applications on the interview invitation probability for female applicants compared to female applicants who were recruited with standard applications, we calculate the sum of the treatment dummy and the interaction term. This overall effect of anonymous job applications for female applicants is not statistically significant, and hence we do not find a systematic effect of anonymous job applications for this group. Applicants with a migration background are more likely to receive an interview invitation with anonymous job applications as indicated by the coefficient on the interaction term for migrants. Although this difference is not statistically significant, there is a positive significant overall effect of anonymous job applications for applicants with a migration background. With about 30 percentage points this effect is moreover quite substantial. Therefore, the comparison with this control group reveals no effect of anonymous job applications for women, but migrants benefit from higher interview invitation rates when applications are anonymously reviewed.
The second potential control group consists of applicants which are part of the same cohort of applicants as the treatment group, with the only difference that their applications were not anonymously reviewed. The treatment group was made artificially smaller since the organization started to anonymously review applications only after the recruitment process had already started. This first part of the cohort of applicants in 2011 thus constitutes a second potential control group. It includes 129 individuals and is therefore substantially smaller than the treatment group. However, applicants’ characteristics are similar in both groups and do not significantly differ. The only exception is the share of individuals with a migration background, which is 7 percent in the control group and thus exceeds the respective share in the treatment group by about 4 percentage points.
Table 4 displays the results of the comparison in interview invitation rates between these two groups. Columns (1) and (2) show that interview invitation probabilities do not significantly differ by treatment status, gender and migration background. When additionally including interaction terms in column (3), we find that female applicants are more likely to receive an interview invitation with standard applications. Female applicants’ advantage, however, disappears with anonymous job applications. Our analysis moreover reveals a negative overall effect of anonymous job applications for women. More precisely, the chances of female applicants to be invited for an interview decrease with anonymous job applications by about 14 percentage points when compared to standard applications. In contrast to the results from the analysis with the first control group, we do not detect any significant effects of anonymous job applications for migrants. This could, however, be due to the smaller sample size of this control group.
The effects of anonymous job applications in this organization can therefore be summarized as follows. First, female applicants and individuals with a migration background have the same chances to receive an interview invitation with anonymous job applications. Second, the effects of introducing anonymous job applications differ for female applicants and applicants with a migration background. The latter group seems to benefit from anonymous job applications. The interview invitation probability of migrants is significantly higher when their applications are anonymously reviewed, at least based on the comparison with the previous year’s recruitment process. This result could be interpreted as the first type of a causal effect, namely that previously existing discrimination is eliminated. However, our results indicate the opposite effect for female applicants. At least when we use the second potential control group in this example, women are significantly less likely to receive an interview invitation with anonymous job applications than with standard applications. This could indicate that previous affirmative action to promote the chances of female applicants is not possible anymore when the applicant’s gender is unknown. The effects of anonymous job applications therefore depend by and large on the initial situation.
3.4. Quantitative effects: overview
Including the example of the previously discussed specific recruitment process in one organization, data are available for a total of nine different recruitment processes that were part of the German experiment. For most of these processes, data on potential control groups are also available. Tables 5 and 6 display the results for these recruitment processes obtained from the same two-step empirical analysis as in case of the previously discussed specific recruitment process.
Accordingly, Table 5 displays the analysis of interview invitation probabilities only for those applicants that were anonymously treated in these recruitment processes. Next to dummy variables indicating gender and migration background, the regressions include additional control variables depending on data availability. Assuming that sensitive information was effectively removed from the applications and that we control for relevant qualifications and skills in our regressions, we expect no significant differences in the interview invitation probabilities for minority groups. And indeed, this is the case for nearly all recruitment procedures. Exceptions are three procedures, where female applicants have higher chances to receive an interview invitation and one case where migrants are less likely to be invited for an interview. There are, however, plausible explanations for these results. For example, the share of female applicants in recruitment process 1 is about 93 percent and therefore any results regarding the applicants’ gender should be interpreted with caution. The positive effect for female applicants in procedure 3 could be due to omitted variables, i.e., variables that the recruiters observe, but we do not have information about—such as grades in high school diplomas. These grades might be on average higher for females than for males in Germany (see, e.g., BMFSFJ, 2004). A similar argument holds for the negative effect for migrants, since recruiters might have information about the qualification of the applicants that we do not have, which can lead to a seemingly lower interview invitation probability.
Table 6 displays the results of the effects of introducing anonymous job applications. The data on potential control groups stem from either past or simultaneous recruitment processes for vacancies that are comparable to the vacancies for which the organizations anonymously recruited. The results of the empirical analysis can be categorized into the previously discussed three different effects: a) elimination of discrimination, b) no effect because no discrimination was present initially, and c) elimination of affirmative action. For instance, the results for recruitment processes 5 and 6 may be viewed as examples for the elimination of discrimination against female applicants. Interestingly, the applicants’ average work experience in these recruitment processes is between 6 and 8 years, which could be a crucial period in a women’s typical working life. Recruiters might anticipate a possible desire to have children, which could have a negative effect on their invitation probability if gender is known. Process 8a may also be viewed as an example for this type of effect of anonymous job applications, but in this case for individuals with a migration background. In contrast, the recruitment processes 1 and 7 may be viewed as examples for the second type of effects (“no effect because no discrimination was present initially”). Finally, the recruitment processes 2 and 8b may be viewed as examples for the last category of causal effects, i.e., a situation in which affirmative action is no longer possible. This seems to be the case for migrant applicants in the former recruitment process and for female applicants in the latter recruitment process. Special cases in our analysis are the recruitment processes 9a and 9b since in both cases, the interview invitation probability with anonymous job applications amounts to 93 percent. Hence, virtually all applicants are invited which makes the interpretation of any effects basically impossible.
We find for most recruitment processes statistically significant differences in the interview invitation probabilities between anonymous and standard applications for all applicants. However, the direction of the effect is not consistent. The interview invitation probability is higher when applicants are anonymously reviewed in some cases, and it is lower in other cases. These differences are unexpected and it is not clear why such differences should exist. Possible explanations include a certain lack of familiarity with the new method, which should disappear after some time. Other factors that may serve as explanations include a different number of applicants or a different number of vacant jobs that are to be filled in the recruitment processes underlying our treatment and control groups.
Altogether, we find that once applicants are anonymously reviewed, the interview invitation probability is in general not influenced by gender and by migration background. This confirms our initial hypothesis that when information about minority group membership is unknown, minority applicants cannot face systematically different interview invitation probabilities. It furthermore appears that the introduction of anonymous job applications can lead to a reduction of discrimination—if discrimination is present in the initial situation. Anonymous job application can also have no effects if no discrimination is present initially, and they can stop measures such as affirmative action that may have been present before. In any case, the effects of anonymous job applications depend on the initial situation. 5
There are, however, certain limitations to the analysis of the German experiment that should be kept in mind when interpreting the empirical results. First, the participating organizations voluntarily joined the experiment and appear as a positive and non-representative selection of German firms and administrations. Most of them had already applied measures such as affirmative action to promote minority groups before joining the experiment. Second, it was not possible to design the experiment as a truly randomized experiment. Hence, our evaluation relies on non-experimental evidence from a comparison with potential control groups. Third, data on these potential control groups were of relatively poor quality and doubts remain whether they adequately approximate the counterfactual situation. Fourth, we restrict our analysis on the effects of anonymous job applications on the interview invitation probabilities for female applicants and applicants with a migration background. Given the number of observations in the experiment, it is not possible to investigate effects on job offer rates, and also not if effects result in terms of other characteristics employers may discriminate against (e.g., age). Due to these limitations our results should be interpreted with caution and should be viewed as indications of possible effects.
Nevertheless, the German experiment shows that anonymous job applications can be practically implemented without excessive costs. Additionally, they can lead to equal opportunities for minority groups of applicants. Because the participating organizations are a positive selection of German firms, it seems moreover plausible to assume that the effects of anonymous job applications would be even larger in a representative sample of German firms.