Documenting the extent to which discrimination exists, why it occurs, and how it effects individual life chances is a crucial but difficult task. It is crucial because the magnitude of discrimination, at least to a certain extent, defines its salience as a political issue. It is difficult because no method of measurement is without flaws. Indeed, decades of research in sociology, economics, and social psychology have dealt with questions of discrimination, using a wide range of methodological approaches, and providing strong evidence that discrimination occurs. However, no single method is able to grasp the full picture. Different methods provide insights into different aspects of the discrimination complex, suggesting that they are complementary approaches rather than competing.

This chapter reviews the strengths and weaknesses of the most commonly used methods of measurement in the field of discrimination research. Taking as its point of departure how we can assess the extent to which discrimination occurs, the chapter reviews quantitative and qualitative analyses of experiences, attitudes, legal complaints, and residual gaps, as well as different forms of experimental designs. A key point in the chapter is to show that although all of these methods shed light on discrimination, they are useful for answering somewhat different questions. Consequently, careful consideration of the range of methods available is necessary for matching one’s research question with the appropriate research design.

4.1 Experiences of Discrimination

The perhaps most intuitive approach to studying discrimination is to ask members of underprivileged groups whether they have experienced differential treatment based on their personal characteristics, which in the context of this book means their ethnic, racial, or religious background. Such studies are conducted in many national contexts, typically by including questions about discrimination in survey questionnaires, such as in the French Trajectories and Origins survey (Beauchemin et al. 2018) or the Norwegian Living conditions among immigrants’ survey (Statistics Norway 2017). Questions about experiences of discrimination are also included in several comparative surveys, at the EU level most notably in the European Union Minorities and Discrimination Surveys (EU-MIDIS), conducted in 2008 and 2016. Additionally, discrimination is covered in the European Social Survey (ESS), but in the ESS, respondents are asked whether they believe that they belong to a group that is discriminated against in the country of residence, rather than if they have experienced discrimination themselves. Of course, asking respondents about individual experiences or their experience of being a member of a discriminated group do not measure the same phenomenon. For example, it is possible to consider oneself a member of a discriminated minority group, such as Muslims in Europe, while never having had any personal experiences of differential treatment. Indeed, there is a tendency in the literature that the levels of perceived group discrimination are higher than the level of personal experiences (e.g., Skrobanek 2009).

Several Eurobarometer surveys also include questions about discrimination. Here, respondents are asked whether they think that discrimination against specific groups are widespread in their own country, whether they have personal experiences of discrimination, and whether they have witnessed discrimination as a third party. Since these questions clearly measure different aspects of discrimination, it should come as no surprise that the results vary strongly depending on the question posed. For example, the Eurobarometer report Discrimination in the EU in 2015 (European Commission 2015) shows that while, at the aggregate level, 64% of the respondents believe that discrimination against ethnic minorities is widespread in their own country, only 3% of the respondents had personally experienced discrimination. Among the ethnic minorities in the sample, however, 30% had personal experiences of discrimination.

Besides large-scale surveys, experiences of discrimination may also be studied by conducting ethnographic work or in-depth interviews among potential target groups. The advantage of such qualitative approaches, compared to surveys, is that the researcher gets the opportunity to dig more deeply into the forms, locations, and consequences of discrimination. Many qualitative studies show that discrimination can take quite subtle forms, which may be difficult to capture by standardized survey questionnaires. Additionally, qualitative research can provide important glimpses into how experiences of discrimination shape future action, for example by investigating what strategies individuals develop to avoid discrimination (e.g., Kang et al. 2016; Lamont et al. 2016; see also Chap. 6). Though qualitative studies cannot produce results that are generalizable to a broader population, they are invaluable in providing the researcher with rich data that increases our understanding of the discrimination complex.

The great advantage of studying experiences of discrimination is that such data documents important aspects of the living conditions of individuals and groups in society. Large-scale surveys can shed light on the prevalence of experiences of discrimination and whether such experiences vary by conditions such as place of residence, level of education, and type of work. Survey data also allows for comparing variations of discrimination between different minority groups and how discrimination on the basis of ethnicity, race, or religion intersects with discrimination based on gender, age, health status, or sexual orientations – what we referred to as intersectional discrimination in Chap. 2. When using longitudinal survey designs, it is also possible to investigate the long-term effects of discrimination on, for example, the level of well-being, mental health, feelings of belonging to majority society, job search strategies, as well as key integration outcomes such as employment and income. Qualitative studies, on the other hand, can provide a deeper understanding of the forms of discrimination involved, what reactions such experiences create, and what kind of strategies individuals develop to avoid future discrimination.

Yet, a major problem of experience-based studies, especially concerning surveys, is the inavailability of high-quality data. Potential target groups are often small and typically underrepresented in population-wide surveys, leading to biased measures of discrimination. Even with high-quality data, however, there remain uncertainties concerning the measure of discrimination provided. Whether individuals perceive an action or situation as discriminatory is largely subjective. Moreover, perceptions may depend on individuals’ consciousness of their exposure to unfair treatment. Individuals might interpret the same situation differently, according to their expectations, their sensitivity and frames of reference, and of course their previous experiences. Furthermore, in selection processes such as job recruitment, the decision-making is not observed directly by the applicant, making it hard to detect whether a rejection is due to discrimination or based on legitimate criteria. Hence, studies of experiences of discrimination can result in both over- and under-estimation of the actual extent of discrimination.

4.2 Attitudinal Studies

Another important line of discrimination research deals with the opposite source of the phenomenon, by considering attitudes toward immigrants and ethnic minorities. Questions about the views of minority groups, perceptions of how the integration or diversity policies work and whether all groups should be offered equal opportunities in society, are part of many population-wide surveys. Such surveys provide useful insights into general attitudes in society, how attitudes differ from country to country, and – through repeated measurement – whether attitudinal changes occur over time.

Studies of attitudes toward immigration are regularly conducted at both the national level and the EU level. One out of many examples is a report based on rounds 1 and 7 of the ESS (Heath and Richards 2016), which compares attitudes among representative samples of the populations in 21 European countries. The report finds that attitudes toward immigration have gradually become more positive over time. Yet there are large differences between countries; the Scandinavian populations display the most positive attitudes while inhabitants in the Czech Republic and Hungary are the most negative. The report also shows a clear hierarchy of minority groups: Jewish people are more welcome than Muslims, who again are more welcome than Roma. Furthermore, highly educated migrants are preferred to low-educated migrants, and low-educated migrants from European countries are preferred to those from outside Europe. Although such numbers do not shed direct light on discrimination patterns, both cross-country differences and the existence of group hierarchies provide useful insights into prevailing sentiments that may shape access to opportunities for minority groups. A recent Swedish study of housing discrimination (Carlsson and Eriksson 2017), for example, shows that landlords are more likely to discriminate in regions where people are more negative toward ethnic minorities, suggesting that reported attitudes expressed in surveys indeed might be a useful predictor of instances of ethnic discrimination.

Of course, it is also possible to measure discrimination more directly, for instance in employment, by conducting surveys or in-depth interviews with employers and asking concrete questions about their hiring practices. A range of studies conducted in both the US and Europe show that employers can be surprisingly outspoken when it comes both to their perceptions about minority groups and in accounting for their considerations in recruitment processes. In a seminal study among Los Angeles employers’ attitudes toward African American and Latino low-educated workers, for example, Johanna Shih (2002) found that a central concern of employers is control at the workplace. The employers in Shih’s study consequently sought workers whom they perceived as manageable and pliable. As perceptions of this kind are not only based on individual merit or employers’ assessments of single applicants, but also vary along categorical lines such as race and gender, studies such as this show how stereotypes at the group level might affect the employment prospects of minority workers.

To be sure, a range of similar studies has been conducted in the European context, not least in the field of low-skilled employment and studies of immigrant niches in the lower tiers of European labor markets. Employers in such labor markets tend to have limited information about individual applicants and therefore often use categorical characteristics such as gender, age, ethnicity, nationality, immigration status or race as a proxy for skills (Friberg and Midtbøen 2018; Moss and Tilly 2001). Importantly, though these processes might be especially salient in low-wage labor markets, they are not limited to them, and stereotypical assessments of specific immigrant groups might affect both the employment prospects of other groups as well as of later generations. Indeed, as Midtbøen (2014) found in a qualitative study among Norwegian employers, stereotypes associated with immigrants seem to be inferred from ethnically distinct names, and negative experiences are regularly generalized between ethnic groups and across generations. The implications of such dynamics for children of immigrants are potentially severe: Instead of experiencing equal access to the labor market, they encounter attitudes and stereotypes attached to their parents’ generation, making their domestic educational qualifications and linguistic fluency “invisible” in the eyes of employers.

Clearly, studies that directly examine gatekeepers’ attitudes are a valuable source of knowledge about discriminatory practices. However, it is not easy to establish a clear relationship between attitudes and actions. As mentioned in Chap. 3, LaPiere (1934) found, in a classic experiment, that hotel receptionists in the US in practice were more indifferent to racial minorities than they said they would be when prompted with direct questions. However, recent studies have shown that the opposite might be equally true. In a seminal study, Pager and Quillian (2005) explored the relationship between American employers’ actions and attitudes by matching data from an experimental audit study with a telephone survey among the same employers. The authors found that although the employers in the survey claimed that they would not discriminate against African American job applicants, the experiment showed large racial disparities in chances of landing a job. This suggests that interviews among potential perpetrators of discrimination leaves open the question of the reliability of the accounts gathered by the researcher.

Furthermore, important discussions in the current field of discrimination research, as discussed in Chap. 3, is whether discrimination occurs deliberately or unconsciously and whether discriminatory practices can be mediated by rules and procedures at the organizational level, such as standardized applications and transparent decision-making processes. Although one can capture a bit of the conscious motivations behind gatekeepers’ actions through surveys and in-depth interviews, such accounts are not necessarily reliable indicators of the actual level of discrimination.

4.3 Studies of Legal Complaints

A different source of knowledge about discrimination is formal complaints put forward to courts or public bodies such as the Equal Employment Opportunity Commission (EEOC) in the US, the Employment Tribunal in Britain, or the Antidiscrimination Tribunal in Norway. In many countries, official records documenting claims of discrimination and the legal treatment of the complaints are accessible to researchers through an application. These records provide an interesting glimpse into the types of discrimination that are claimed, how the volume and content of claims change over time, and how antidiscrimination policies are enforced in specific contexts.

Studies of legal claims are most frequently conducted in the US context, and this body of work clearly demonstrates that such claims represent an interesting entry to studies of discrimination. For example, in the book The Face of Discrimination: How Race and Gender Impact Work and Home Lives (2007), Vincent J. Roscigno uses narrative data from employment and housing claims submitted to the Ohio Civil Rights Commission. Roscigno finds that the highest number of claims come from the low-wage service sector and that firing discrimination is the most important claim in the private sector, whereas, in the public sector, discrimination in hiring, promotion, and firing are evenly distributed. Looking specifically into race and gender differences, Roscigno also shows that while white women are more likely to report discrimination due to pregnancy, black women tend to report more frequently instances of racial discrimination than discrimination related to their experiences as women. Altogether, the book builds on more than 14,000 verified discrimination cases as well as qualitative analyses of about 850 of the same cases, including in-depth studies of how employers and plaintiffs narrate their positions in cases with disputed outcomes.

As this book demonstrates, legal cases may provide insight into both the concrete management of the discrimination legislation and the different parties‘reasoning. Legal cases typically also offer detailed descriptions of a range of different situations, and they include the legal assessment made in each case. When a large amount of cases is available, it is also possible to look in depth into the intersections of race, class, and family statuses, as well as comparing similarities and differences between the public and the private sector.

Despite their merits, studies of legal complaints have some major drawbacks. Most importantly, few incidents of discrimination actually end up in the legal system. This is especially the case in national settings with an underdeveloped (or even non-existent) public grievance system. Furthermore, putting forward a legal claim requires time and resources, and that victims of discrimination believe that they would find reparation with legal action. Discrimination cases frequently fail to be successful in the legal system, because firm evidence is hard to provide. In this context of uncertainty, victims might not see the benefits of putting a claim in justice. Finally, discriminatory actions and decisions are often hidden from the ones affected by it, suggesting that most discriminatory acts go under the legal radar. Consequently, though studies of this kind represent an important source of knowledge about the nature of discrimination, legal reports are less useful as indicators of the overall extent of discrimination occurring in a specific national context.

4.4 Studies of Residual Gaps

As discrimination is part of, but not the sole driver of, creating and maintaining ethnic inequalities, a key question in much social science research is the actual role that discrimination plays in shaping access to opportunities. How much, say, of the unemployment rates that exist between the native and the foreign-born population in a country can be explained by human capital factors such as differences in the level of education and language proficiency, and how much is due to discrimination in hiring processes?

To answer such questions, discrimination is often measured indirectly as the unequal access to positions or resources – such as jobs, wages, housing, selective education tracks – by statistical analyses of large data sets. In these types of studies, the focus is not on the experiences that individuals or groups have with discrimination or on the attitudes of the dominant group. Rather, the researcher takes as the point of departure the mean distribution of groups on a specific dependent variable, such as wages, unemployment, or occupational attainment, and then controls for relevant, non-discriminatory factors that could explain the observed group differences, such as school performance, level of education, and work experience. The residual gap remaining between groups in a given outcome is usually referred to as “ethnic penalties”; that is, the disadvantages facing ethnic minorities compared to majority peers after controlling for (most) productivity-relevant factors.

A vast body of work builds on this “residual method.” One influential example is Unequal Chances: Ethnic Minorities in Western Labour Markets (2007), a comparative volume edited by Anthony Heath and Sin-Yi Cheung. This book compares patterns of unemployment and occupational attainment for a range of different ethnic groups of both the first and second generation in altogether 13 countries, including Austria, Britain, Canada, France, Germany, the Netherlands, Sweden, and the US. The book demonstrates that in all countries examined, non-European minority groups face ethnic penalties in accessing the labor market and that these disadvantages are transferred across generations despite the educational progress achieved by children of immigrants. Further, the book shows that there is a considerable cross-national variation in the magnitude and scope of ethnic penalties. In some contexts, such as in Britain and Sweden, disadvantages appear to be reserved to the labor market entrance, while in others, such as in Germany and Austria, ethnic penalties are also present later in the employment relationship, suggesting a pattern of cumulative disadvantage in labor market trajectories in some context but not in others.

Statistical analyses of group differentials, such as in Unequal Chances, are of utmost importance in providing large-scale pictures of ethnic inequalities, as well as in differentiating between relevant factors explaining gaps in a given outcome. However, it is important to have in mind that ethnic penalties are not equivalent to ethnic discrimination. Indeed, because the role of discrimination in studies using the residual method is not examined directly, but rather is left as part of the unexplained residual, there is always uncertainty regarding the existence of unobserved factors that might explain the remaining difference between the groups, such as ethnic differentials in access to relevant social networks. Some studies attempt to isolate the effect of social networks and thus come closer to a “clean” measure of discrimination, but the direct role of discrimination in explaining ethnic differentials in labor market outcomes remains nevertheless unresolved in studies of this type.

4.5 Experimental Studies

The limitations of traditional methods in assessing the direct role of discrimination in access to opportunities in employment and housing have paved the way for the increasing use of experimental approaches. Indeed, the strength of experimental approaches to studies of discrimination is the ability to isolate causal effects; that is, the direct effect of a racial appearance or a minority-ethnic sounding name on, for example, the chances of landing a job. In a randomized, controlled experiment, subjects are randomly assigned to clearly defined “treatment” and “control” conditions in order to control for every other factor potentially influencing the outcome of interest. As such, experimental studies, when conducted carefully, are able to examine the role of discrimination directly.

Experimental approaches to discrimination come in different forms. One much-debated method is the so-called “Implicit Association test” (IAT), in which participants in quick succession are presented pictures of different categories of people (women and men, elderly and young, white and black) and asked to connect these pictures to positively and negatively charged properties (see Chap. 3). The idea is to investigate whether individuals more quickly associate stereotypical (often negative) characteristics to traditional “out-groups” than to “in-groups” (e.g., Greenwald et al. 1998).

Another approach is survey experiments or so-called vignettes. A typical example is studies where respondents are asked to assess whether they would hire a particular person or what they would offer to the person in pay. In such studies, the formal qualifications of the fictitious person in question are held constant, but respondents are randomly given persons with different names or different racial appearance, to measure the effect of that isolated variable on the respondents’ decision (e.g., Pedulla 2014). Another version of this method is to include an experimental element as part of ordinary survey questionnaires, for example, to investigate whether respondents vary in tolerance when confronted with different groups. Toril Aalberg et al. (2012), for example, conducted a survey experiment to examine whether the willingness to admit individuals as legal immigrants depends on their attributes. Using an experimental design in the Norwegian context, specific attributes of immigrants were manipulated, making them appear more or less likely to make an economic contribution and more or less likely to assimilate into Norwegian culture. The authors found that the decision to admit individuals were influenced by the immigrant’s economic background, in which Norwegians were especially supportive of highly skilled immigrants, but also that immigrants with an Afrocentric appearance were more likely to be rejected by men, but accepted by women.

The most direct measure of discrimination, however, is provided by field experiments. Field experiments of discrimination can be divided into two main categories or techniques: Audit studies and correspondence test studies. In audit studies, pairs of individuals who differ in racial markers but are carefully matched in relevant productivity characteristics and trained to act similarly, apply for real-world jobs or housing vacancies by showing up in person (e.g., Pager 2003). In correspondence test studies, matched pairs of résumés and cover letters differing in the names of the applicants (signaling different race or ethnicity) are sent in response to job openings or to housing offers (e.g., Bertrand and Mullainathan 2004). In both types of studies, the effect of race or ethnicity on opportunities is directly measured. Because all factors other than race or ethnicity are isolated and the résumés are randomly assigned to the test persons, well-conducted field experiments provide convincing estimates of the incidence of discrimination in specific markets.

More than 100 field experiments of ethnic and racial discrimination in employment have been conducted all over the world, but predominantly in North America and Western Europe. Results have varied across countries, but not one single study has concluded that discrimination is not a relevant factor in shaping access to employment for a variety of racial and ethnic minority groups. In several countries, minority applicants have to apply to twice as many applications to get job interview offers compared to equally qualified majority peers. However, there is an interesting variation across national contexts (e.g. Di Stasio et al. 2019). A recent meta-analysis of field experiments by Eva Zschirnt and Didier Ruedin (2016), for example, shows that discrimination levels are lower in German-speaking countries than in other countries, probably reflecting the high amount of information required to apply for jobs in these contexts. Another meta-analysis, conducted by Lincoln Quillian et al. (2019), compares the countries where most field experiments have been conducted, demonstrating that the level of racial discrimination in the US is significantly lower than the discrimination against ethnic minorities in France.

Recently, the meta-analysis technique has also been used to investigate trends in discrimination over time in single countries. In the US, where most field experiments have been conducted, Quillian et al. (2017) find that there has been no change in the level of discrimination against African Americans over the past 25 years, suggesting a distressing persistence of discrimination patters. The same pattern is documented in Britain, where a recent meta-analysis of all field experiments conducted between 1967 and 2017 found no reduction in the level of discrimination against black Caribbean and Asian minorities over a fifty-year time span (Heath and Di Stasio 2019).

Importantly, these overall negative effects of racial and ethnic minority background on employment opportunities conceal important variations in the results of single field experiments and countries. One such dimension is whether different minority groups constitute an “ethnic hierarchy” in which some groups (e.g., white immigrant-origin groups) are systematically preferred over “visible” or racialized minorities of non-European origin. Many studies do indeed point to the existence of such hierarchies and, in those cases, applicants with backgrounds from North Africa and the Middle East tend to be most severely disadvantaged. In a few other studies, by contrast, no ethnic hierarchy is identified (e.g., McGinnity and Lunn 2011). Still, when taking all studies together, the level of discrimination against white immigrants and their descendants are significantly lower than the discrimination against racially visible minority groups (Quillian et al. 2019).

The obvious advantage of experimental approaches over non-experimental studies is the researcher’s extensive control over the variables in play. By isolating an “ethnic variable,” as in field experiments, or manipulating the link between names and specific characteristics, as in survey experiments, it is possible to draw causal inferences about the effect of ethnic background on, say, wage setting or callbacks for a job interview. The disadvantage of laboratory and survey experiments is external validity: Because the research is conducted in artificial settings, it is difficult to assess whether the results obtained may be generalized to the real world. Field experiments, by contrast, allow researchers to retain the ability to draw causal inferences while staging the research in real-world settings like hiring processes ensures that conclusions are relevant to actual social contexts. Nevertheless, even field experiments face limitations. Although these studies have convincingly documented the fact that discrimination occurs, this research tradition has been less productive in explaining the processes by which race and ethnicity become factors of importance in employers’ decision-making (Pager et al. 2009; Midtbøen 2015). This means that a field experiment can demonstrate the causal effect of a foreign name on employment prospects, but unless it is complemented with other methods it cannot shed much light on the mechanisms leading to discriminatory practices.

4.6 Conclusion

This chapter has briefly reviewed the most commonly used methods and approaches in research on discrimination. The key take-away message is that the suitability of methods depend on the question posed: A focus on people’s experiences highlights central aspects of everyday life, studies of potential discriminators can provide insights into the way individuals in power positions make their decisions, and studies of residual gaps are of indisputable importance in providing large-scale pictures of ethnic inequalities in key outcomes such as unemployment, occupational attainment, education, housing, or health. To assess the direct role of discrimination in shaping groups’ access to opportunities in the labor or housing market, however, field experiments are considered the “gold standard.” As each approach to the study of discrimination nevertheless suffers from certain limitations, the more widespread use of research designs that combine different methods in single studies would be much welcome.