A New Approach to the Study of Tolerance: Conceptualizing and Measuring Acceptance, Respect, and Appreciation of Difference

Previous empirical research on tolerance suffers from a number of shortcomings, the most serious being the conceptual and operational conflation of (in)tolerance and prejudice. We design research to remedy this. First, we contribute to the literature by advancing research that distinguishes analytically between the two phenomena. We conceptualize tolerance as a value orientation towards difference. This definition—which is abstract and does not capture attitudes towards specific out-groups, ideas, or behaviors—allows for the analysis of tolerance within and between societies. Second, we improve the measurement of tolerance by developing survey items that are consistent with this conceptualization. We administer two surveys, one national (Sweden) and one cross-national (Australia, Denmark, Great Britain, Sweden, and the United States). Results from structural equation models show that tolerance is best understood as a three-dimensional concept, which includes acceptance of, respect for, and appreciation of difference. Analyses show that measures of tolerance have metric invariance across countries, and additional tests demonstrate convergent and discriminant validity. We also assess tolerance’s relationship to prejudice and find that only an appreciation of difference has the potential to reduce prejudice. We conclude that it is not only possible to measure tolerance in a way that is distinct from prejudice but also necessary if we are to understand the causes and consequences of tolerance.


Introduction
Tolerance is generally understood as a necessary component of a functioning democracy and stable world order. Indeed, the Preamble of the United Nations Charter (UN 1945) declares the intention of its member states "to practice tolerance and live together in peace with one another as good neighbours." Later, the United Nations Educational, Scientific and Cultural Organization (UNESCO 1995) clarified the meaning of tolerance. According 1 3 to Article 1.1., " [t]olerance is respect, acceptance and appreciation of the rich diversity of our world's cultures, our forms of expression and ways of being human…Tolerance is harmony in difference." Tolerance is often invoked as something to which individuals and societies should aspire, especially given diversity, in all its forms, is increasingly a feature of contemporary democracies. When tensions arise, some leaders call for a "greater tolerance" of particular groups or encourage general efforts to become "a more tolerant society." For example, in 2004, then Secretary-General of the UN, Kofi Annan said, "Tolerance, inter-cultural dialogue and respect for diversity are more essential than ever in a world where peoples are becoming more and more closely interconnected" (United Nations 2004). According to UNESCO Director-General Audrey Azoulay, "Tolerance is an act of humanity, which we must nurture and enact each in our own lives every day, to rejoice in the diversity that makes us strong and the values that bring us together" (UNESCO 1996). Yet, what does this mean in practice? That those who hold prejudicial attitudes should fight against their dislike of particular out-groups? That everyone should respect others' values or attitudes even when they are contrary to their own? That society should always value or embrace diversity? Leaders rarely give answers to these questions. Unfortunately, science does not provide much guidance either.
Over 40 years ago, Ferrar (1976:63) proclaimed, "The concept of tolerance is in a state of disarray." According to Ferrar, tolerance has multiple dimensions, but the empirically oriented literature primarily emphasizes one: negative attitudes towards out-groups. She argues that when scholars rely on indicators of prejudice towards social groups or discrimination in their analyses of tolerance, they imply that "tolerance and its opposite are sufficiently described by reference to categoric prejudgments of minority groups and their members" (p. 67). We take this argument one step further and contend that incorporating prejudice into the meaning or measurement of tolerance makes the concept of tolerance analytically indistinguishable from prejudice, confusing what tolerance is and how it differs from dislike, disapproval, or disgust with specific out-groups. Despite a great deal of empirical research on tolerance over the past 40 years, some of which includes overt efforts to clarify the concept, this disarray persists. We claim that the central problem continues to be the conflation-explicit or implicit-of prejudice and (in)tolerance, either in conceptualization or operationalization.
Despite problems in the scientific literature, it is generally accepted that tolerance is something necessary for democracies. As Kuklinski et al. (1991:3) note: "Few aspects of political life so directly and immediately touch upon the daily lives of common citizens as does their tolerance toward each other." 1 To answer some of the pressing, if not existential questions facing multiethnic, democratic societies today, we need a clearer understanding of tolerance-what it is and what it isn't. And, before we can begin to assess its impact on various aspects of social, economic, and political life, we need better tools to measure it.
In the sections that follow, we begin with a review of previous empirical research on tolerance. Then, based on scholarship on toleration, we advance a conceptualization of tolerance that is abstract as well as analytically separate from other concepts. Specifically, we define tolerance as a value orientation towards difference. Next, we develop new measures to operationalize three aspects of tolerance. Importantly, these measures do not include references to specific social or political out-groups or particular types of attitudes or behaviors. Instead, the items capture acceptance of, respect for, and appreciation of difference in the abstract. We administer a survey twice-first using a random sample of the Swedish population and second using an online format in Australia, Denmark, Sweden, United Kingdom, and the United States. After validating our measures empirically, we demonstrate their relationship to prejudice and other variables. We conclude with a discussion of our results, their contributions, limitations, as well as practical implications.

Previous Approaches to the Study of Tolerance
In general, two broad conceptualizations of tolerance exist. The first approach understands tolerance as a permissive attitude towards a disliked out-group. Thus, this conceptualization begins with the notion that in order to be tolerant one first has to be prejudiced. Previous research from this tradition incorporates the dislike of out-groups into the measurement of tolerance. We critique this approach on both theoretical and methodological grounds. The second approach defines tolerance as a positive response to diversity itself. This conceptualization is analytically distinct from prejudice and lays the foundation for superior operationalization. However, previous studies that begin with this definition have nevertheless relied on measures of prejudice in their analyses, which means our understanding of tolerance remains limited. Thus, our critique of this approach is primarily methodological. In the sections that follow, we examine these approaches to tolerance in greater detail and discuss their theoretical and empirical implications.

Tolerance as Phenomenon Dependent on Prejudice
The first conceptualization of tolerance can be summarized as: Person X is tolerant if Person X dislikes Person Y doing Z. Person X has the means to prevent Person Y from doing Z, but Person X refrains from doing so. Therefore, in order to tolerate someone or something, one first needs to experience disapproval or dislike, and then despite these negative sentiments exhibit permissiveness or acceptance. Tolerance in this sense implies "forbearance" or the readiness to "put up with" with what one dislikes (Rapp and Freitag 2015;Robinson et al. 2001;Sullivan et al. 1979;Verkuyten and Slooter 2007).
To "put up with" in political terms translates into allowing the expression of objectionable ideas (Sullivan et al. 1979), or more specifically, to extend social rights related to political participation and freedom of speech to groups one dislikes or disagrees with (Mondak and Sanders 2005;Rapp 2017). The "objection criteria" is at the core of this conceptualization, as "… one cannot tolerate ideas of which one approves (Gibson 2006, p. 22)." Tolerance, in this sense, is a sequential or twofold concept (Rapp and Freitag 2015), where the crux of the matter is the initial position of like or dislike.
This understanding of tolerance is theoretically problematic for two reasons. First, by this definition, the existence of tolerance depends on the existence of prejudice. People who are not prejudiced are incapable of being tolerant let alone becoming more tolerant. Moreover, we can only gauge if society has become more tolerant by knowing if a society has become less prejudiced. Second, this definition excludes reactions to the mere existence of out-groups. In theory, an individual must have the capacity to prevent what is disliked in order to demonstrate tolerance. Because the presence of racial and ethnic outgroups is likely beyond any one person's control, it becomes theoretically impossible to be tolerant of this type of diversity. Beyond these theoretical shortcomings, we also argue that this understanding of tolerance necessarily leads to the empirical conflation of tolerance and prejudice.
Many empirical studies of tolerance begin with the assumption that particular groups are widely disliked or, at the very least, viewed with skepticism (Bobo and Licari 1989;van Doorn 2016;Gibson and Bingham 1982;Gibson 1998). An important example is Stouffer's (1955) seminal work on tolerating non-conformity (e.g., socialism and atheism) in the United States. In his study, examples of tolerance include the willingness to extend rights such as freedom of speech to these "non-conformist" groups. Verkuyten and Slooter (2007) study tolerance of Muslim beliefs and practices among Dutch teenagers. They motivate their choice of out-group with reference to the general status of Islam in Dutch society. The main issue with this "unpopular groups" strategy is that it is impossible to distinguish empirically between people who support rights for groups they dislike and people who support rights because they are positively disposed towards the group in question (Sullivan et al. 1979). Sullivan et al. (1979) introduce the "least-liked" approach in part to avoid contaminating the measurement of tolerance with respondents' attitudes towards specific groups. As they put it: "If we had merely asked all respondents whether communists should be allowed to hold public office, their responses would depend not only on their levels of tolerance, but also on their feelings toward communists" (p. 785). To establish initial dislike, Sullivan et al. (1979) measure respondents' attitudes about various groups in society. After identifying a disliked, or least liked, group, the respondents report preferences regarding these group members' participation in political and civic activities. Adopting the same strategy, Rapp (2017) first examines respondents' attitudes towards groups that are ethnically, religiously, or culturally diverse from them. Anti-immigrant attitudes constitute the rejection component. She then restricts her sample only to those respondents who are prejudiced, because theoretically, they are the only ones who can be tolerant.
We argue that neither strategy truly captures tolerance, because in both prejudice remains fundamental to the measurement of tolerance. 2 Thus, regardless of whether dislike is assumed, as in the unpopular group strategy, or measured, as in the least liked approach, empirical findings actually reflect respondents' attitudes towards an out-group.

3
In summary, this first approach to the study of tolerance treats prejudice as a prerequisite for tolerance. 3 If dislike of an out-group is a precondition for tolerance, this means that in theory one cannot be tolerant without having been prejudiced at some earlier point in time. Conceptually there is a great deal of overlap between prejudice and tolerance, which inevitably extends to the measurement of tolerance (e.g., Kuklinski et al. 1991;Davis 1995;Gibson 1998;Verkuyten and Slooter 2007;Rapp and Ackermann 2016).

Tolerance as a Phenomenon Distinct from Prejudice
A second approach to analyzing tolerance does not begin with dislike of groups and instead focuses on subjective reactions to the existence of diverse values, behaviors, and lifestyles. Kirchner et al. (2011:205) define tolerance as "the willingness to tolerate or accept persons or certain groups as well as their underlying values and behavior by means of a co-existence (even if they are completely different from one's own)." Norris (2002:158) defines tolerance as "the willingness to live and let live, to tolerate diverse lifestyles and political perspectives." Dunn et al. (2009:284) define tolerance "as a non-negative general orientation toward groups outside of one's own." Some scholars make explicit that tolerance does not require prejudice. For example, Allport (1958:398) points out while tolerance may mean putting up with something or someone one dislikes, such as a headache or a neighbor, "the term also has a more rugged meaning. We say that an individual who is on friendly terms with all sorts of people is a tolerant person. He makes no distinction of race, color, or creed. He not only endures but, in general, approves his fellow men." By providing two examples of tolerance-one where the subject dislikes what he tolerates and one where he likes what he tolerates-Allport (1958) demonstrates not only that dislike is not fundamental to tolerance but also that it is unnecessary for tolerance. Chong (1994:26) also argues that, based on this conception, is it is possible to tolerate things that we like. Thus, tolerance may be either to endure something or to show esteem for something. 3 Tolerance is also recurrent in research on prejudice, especially in analyses of attitudes towards immigrants and ethnic minorities. Here the use of tolerance is not necessarily theoretical, and intolerance and prejudice are generally regarded as equivalents. For example, Togeby (1998) uses tolerance interchangeably with broadminded views and (absence of) ethnocentrism, making an empirical distinction between positive attitudes towards immigrants coming to the country (prejudice) and positive attitudes towards immigrants already living in the country (tolerance). Other prejudice scholars conceive of tolerance constituting positive attitudes toward immigrants as well as by an abstract ideological belief in and endorsement of equality ( Van Zalk et al. 2014;Miklikowska 2016). Hainmueller and Hiscox (2007), who study tolerance as a mediator of the education effect on immigration attitudes, operationalize tolerance by an "… array of different measures of individuals' values and beliefs" (p. 429). The item most explicitly tied to tolerance captures views on laws against promoting racial or ethnic hatred, with more positive attitudes indicating greater tolerance.
Others researchers, however, explicitly describe tolerance as the absence of prejudice. Dunn and Singh (2011) define intolerance as "a negative general orientation toward groups outside of one's own" (p. 319). The degree of tolerance is derived from the respondents' willingness to accept as their neighbors social groups such as immigrants, drug users, homosexuals, or Jews. Evans (2002) focuses exclusively on racial prejudice and negative attitudes towards homosexuality, interpreting the absence of such attitudes as an expression of "focused tolerance." For others, the equating of tolerance with positive out-group attitudes appears to come down to semantics. "Tolerance," then, is not defined or operationalized, but only used to summarize positions on different indicators of prejudice (Crepaz and Damron 2009;Craig and Richeson 2014;Rustenbach 2010).
As they do not begin with disliking a group of people, these definitions are more analytically distinct from prejudice. They emphasize reactions to diversity without specifically identifying marginalized social groups or indicating that individuals' behaviors, values, or lifestyles are anything other than different from one's own. However, some of these conceptualizations of tolerance incorporate the word tolerate or intolerance into its own definition. Moreover, some definitions equate acceptance with tolerance while others treat acceptance and tolerance as two different things. We maintain that while these definitions are more analytically distinct from the previous approach, they still lack conceptual clarity.
Nevertheless, previous research from this tradition has failed to operationalize tolerance in a manner consistent with its own definition. Put simply, these studies also incorporate prejudice into their measurement of tolerance. Persell et al. (2001) rely on five questions to measure tolerance. Two ask about attitudes towards homosexuals and three refer to African-Americans. Both Dunn et al. (2009) and Kirchner et al. (2011) use a long battery of measures found in the World Values Survey (WVS) to capture respondents' willingness to have individuals from specific social groups as neighbors. These groups span from people of a different race to heavy drinkers to people with a criminal record. Kirchner et al. (2011) argue that by focusing on a multitude of groups, they are able to distinguish between individuals who tolerate only one "objectionable" group from those who tolerate many or all. While this approach does improve upon studies that analyze attitudes towards a few, specific groups, it still measures attitudes towards out-groups. In fact, according to Norris (2002:158), this WVS scale "taps many of the most common types of narrow-mindedness and bigotry" (also cited in Kirchner et al. 2011:205), put simply, prejudice.
Our critique of this strand of research is methodological. By incorporating prejudice into the measurement of tolerance, these previous studies do not analyze attitudes about the existence of diversity nor do they investigate an "orientation toward groups outside of one's own" (Dunn et al. 2009:284). Instead, they measure a willingness to accept specific groups as neighbors, which certainly speaks to how respondents feel about these groups and not diversity in general. Measuring attitudes towards a multitude of groups does not change this; these indices only tell us the extent to which one is prejudiced-in other words, if one is prejudiced towards one, two, or many, but always a subsample of out-groups. In summary, this conceptualization defines tolerance as a phenomenon distinct from prejudice and emphasizes reactions to diversity in all forms. However, previous research from this tradition has not measured tolerance in a way that is consistent with that definition.

Other Concerns: Abstraction and Multidimensionality
We have argued that incorporating elements of prejudice into the meaning and measurement of tolerance has rendered intolerance and prejudice conceptually and empirically indistinguishable. Yet there are other limitations to these two approaches that stem from a lack of abstraction. First, using attitudes about specific values, behaviors, lifestyles or social groups as indicators of tolerance makes it difficult to study tolerance longitudinally. The status of particular social groups changes over time due to a number of factors, including societal prejudice. Norms about what is acceptable to do, say, or believe also change. Linking tolerance to something specific means we can only measure whether positive or negative attitudes towards a specific entity have changed over time.
Second, by focusing on attitudes towards particular social or political groups, previous research has often conflated conservative beliefs with intolerance and liberal attitudes with tolerance. Ferrar (1976:75-76) identifies this problem as originating with Stouffer's index (1955), which connects tolerance to permissive attitudes regarding civil liberties of groups associated with the political left but not political right. It is, of course, theoretically possible to have a principled commitment to diversity while simultaneously holding conservative beliefs about political issues, just as it is also possible to oppose specific types of diversity and take liberal political stances. Although more recent studies incorporate attitudes towards a greater number of social groups (Dunn et al. 2009;Kirchner et al. 2011), this bias still exists.
Third, scholars that focus on attitudes towards groups not only conflate prejudice with tolerance but also disregard people's ability to support diversity in the abstract. Sniderman et al. (1989:27) call this outright dismissal of principled tolerance a deeply cynical and pessimistic view of "the willingness of the average citizen to embrace, disinterestedly and consistently, a foundational value of democratic politics-tolerance." We contend that at the very least this is an empirical question worthy of investigation. Without measures of tolerance in the abstract, we simply do not know.
Finally, most previous empirical research neglects the multidimensionality of tolerance, although theoretical research on tolerance emphasizes this. Thinking about tolerance as an attitude towards diversity, Walzer (1997) argues that there are five types of tolerance that vary from resigned acceptance to aesthetic endorsement. Similarly, Forst (2013) claims that there are four types of tolerance, which range from acceptance to appreciation. Persell et al. (2001:208) contend that complete tolerance would entail recognition and acceptance while a lesser version would be "an unwillingness to openly express intolerance." Despite definitions that include a number of ways that tolerance can be expressed, subsequent empirical analyses treat tolerance as a unidimensional concept (e.g., Kirchner et al. 2011).
In summary, previous studies of tolerance suffer from one or more of the following three main problems: (1) conceptual overlap of tolerance and prejudice; (2) operational overlap of tolerance and prejudice; and (3) a lack of abstraction in the conceptualization and operationalization of tolerance. Moreover, previous empirical research has, for the most part, ignored the multidimensionality of tolerance, something emphasized in theoretical work. Therefore, in the next section, we advance a conception of tolerance and develop new measures of tolerance consistent with our definition. To avoid the pitfalls of previous approaches, we do not identify particular social groups, behaviors, or values in our indicators. Further, our measures are politically and temporally neutral.

Tolerance as an Orientation Towards Difference
We advance a new conception of the phenomenon in question and define tolerance as a value orientation towards difference. The fundamental question is not whether one puts up with something disliked but how one responds to the existence of diversity itself. This definition is abstract and analytically distinct from other concepts. 4 Our focus is on subjective reactions to difference; thus, this conceptualization does not require dislike of or identification of potentially objectionable groups, ideas, or behaviors. In practice, this definition is consistent with the approach to tolerance that does incorporate forbearance into its definition.
Our definition-a value orientation towards difference-is consistent with Walzer's (1997) understanding of tolerance as an attitude or state of mind. This conception of tolerance is also consistent with previous accounts that do not see prejudice as a prerequisite for tolerance (e.g., Allport 1958;Chong 1994;Walzer 1997) and suggest that multiple expressions of tolerance are possible (e.g., Persell et al. 2001;Rapp 2017). To identify different possible expressions of tolerance, we adapt Forst's (2013) four dimensions. We choose these not because they differ substantively from others but instead because Forst draws distinctions among different types of tolerance in a manner that lends itself well to the empirical investigation of the phenomenon.
According to Forst (2017), there are four understandings of tolerance and each may be present in a society at the same time. The first and second are related in that they both understand tolerance as a permissive relationship between different groups. In this expression of tolerance, groups do not interfere with each other or their practices but instead accept their existence. The difference between the first and second conception is the structure of society. In the first conception, the groups have unequal power. There is a clear majority that tolerates a minority group. In the second version, the groups have roughly equal power. Because we do not want to distinguish theoretically between societies with different social structures and systems of stratification, we combine these two. Thus, we identify the most basic expression of tolerance as an acceptance of difference.
Forst maintains that tolerance may also be respect for diversity or esteem for diversity. In Forst's third conception of tolerance, individuals show respect for diversity by viewing disparate groups as morally and politically equal even though they may differ fundamentally in beliefs, practices, and lifestyles. In his fourth conception, tolerance is esteem or appreciation for diversity. According to Forst, esteem is a more demanding reaction to diversity than respect. This version of tolerance means viewing others' beliefs, practices, or lifestyles as something valuable and worthy of ethical esteem even though they are different from one's own. Thus, we call the second and third expressions of tolerance respect for difference and appreciation of difference.
One can think of different aspects of tolerance as points on a continuum (Walzer 1997). One can also think of these expressions as hierarchical, where individuals who appreciate diversity are also likely to respect and accept diversity, yet acceptance of diversity does not necessarily mean one appreciates it. Our conception of tolerance has clear advantages in terms of measurement. By identifying three distinct aspects of tolerance, we can measure different expressions of tolerance instead of measuring the number of groups one dislikes. Indeed, because we do not identify particular social groups, behaviors, or values in our indicators, our measures are politically and temporally neutral. In the next section, we describe our tolerance measures as well as our survey design and data.

Samples and Surveys
We measure tolerance in two different samples. The first is a random sample of the Swedish population (aged 18 years and older). We administered the survey via the national postal service in spring of 2016. We sent the survey once without reminders. Our response rate is 27.6%, which generated a sample of 1107 individuals. The respondents are comparable to the general population in Sweden in regards to gender (49.7% women) and average age (50.9 years). People with higher education are slightly underrepresented in our sample. In 2016, 27% of the Swedish population had three or more years of tertiary education, while only 19% of our respondents above 25 years have at least 3 years of tertiary education. 5 11.1% of the sample is foreign-born, which is lower than the 17.9% of the total population born abroad in 2016, but not surprising given the survey was administered in Swedish.
In addition to questions about tolerance, our survey includes a number of questions associated with prejudice, such as attitudes towards immigrants and homosexuals. We include these so that we can assess whether our items capture something distinct from prejudice. These additional items come from established cross-national surveys and have been validated in previous empirical research. We also ask respondents about their political preferences and voting behavior. As previously mentioned, the survey also includes demographic questions.
To investigate if of our measures of tolerance have the same meaning in other countries, we rely on a second dataset, a cross-national sample of individuals from Australia, Denmark, Sweden, the United Kingdom, and the United States. Country selection is one of convenience, as we were invited to add our tolerance items to an existing Qualtrics websurvey 6 in each of the five countries. Our resulting dataset includes only questions about demographic background and tolerance. In total, there are 6300 respondents with equal numbers from each of the countries. To minimize potential carry-over effects for each item measuring tolerance, we randomized the item order for every respondent. Previous studies comparing population data and survey data provided by self-selected panels such as Qualtrics indicate that these samples are fairly representative (Heen et al. 2014). We report descriptives for the cross-national sample by country in "Appendix 1" section.

Measures of Tolerance
To develop our measures, we ran two pilot studies in 2014 and 2015 with self-selected samples. The first pilot study was online and the web address widely advertised. The second pilot study was a paper survey administered to university students. We included 15-20 tolerance items in each. We used these studies to get feedback about the wording of questions and run preliminary analyses. Preliminary results indicated that 9 items produced a good model fit in a SEM analysis, but given we did not use representative samples in the pilot studies, we still included 17 items in the final Swedish survey.
We administered our nationally representative paper and online surveys in 2016. Results from analyses of these samples corroborated preliminary results from the pilot studies. Therefore, we kept a total of 9 items, which as a result of further analyses became 8. Ultimately, we only retained items that adequately load on their respective factor (acceptance, respect, and appreciation) and do not have high cross-loadings on the other two factors. Some have argued that unidirectional scales risk acquiescence bias (e.g., McClendon 1991; Billiet and McClendon 2000), where differences among items are underestimated, producing a seemingly coherent scale. However, other research suggests this concern is overstated (Rammstedt and Farmer 2013). Research on bidirectional items reveal that this approach also has limitations, as items worded both positively and negatively may damage response accuracy (Schriesheim and Hill 1981). With these risks in mind, we settled on positively worded items and randomized the items in the cross-national survey. Table 1 reports the final selection of items we use to capture each of the three dimensions of tolerance. Descriptive statistics for each measure are found in Table 2. Responses vary from 'completely disagree' to 'completely agree' on a five-point Likert scale. Scores for each item indicate that on average respondents articulate moderate to fairly high levels of tolerance. Most items display modest skewness, which is expected given the mean values. However, this should be of little concern given the robustness of maximum likelihood estimators (e.g., Reinartz et al. 2009), normal distribution of the data, and that all variables are modestly skewed in the same direction. We report descriptive statistics for our cross-national sample in "Appendix 2" section.
We rely on these two datasets for a number of different analyses. First, we use our Swedish sample to test a three-factor model of tolerance. To do this we rely on a structural equation model (SEM). Next, we use our second sample and multi-group confirmatory factor analysis to validate our results cross-nationally. In our next set of analyses, we regress latent constructs of attitudes towards out-groups on tolerance. Last, we provide a demographic portrait of tolerance by examining levels of tolerance among different demographic groups in Sweden.

Factorial Structure of Tolerance
To test our proposed three-factor model of tolerance we conduct a confirmatory factor analysis. In Fig. 1, we present standard overall model fit statistics, including Chi squared test, comparative fit index (CFI), Tucker-Lewis index (TLI), and the root mean squared error of approximation (RMSEA). CFI, TLI, and RMSEA all indicate a good model fit. The Chi squared for the model fit is significant, which is expected given the large sample size (N = 1077). In order to improve the model further, we specify a correlation between item Ac1 and Ac2 (CFI: 0.987, RMSEA: 0.042), which seems appropriate given the linguistic proximity of the items in question. However, we select the most parsimonious model since its model fit is clearly acceptable. 7 Considering our three expressions of tolerance are correlated, it is possible that a onefactor model actually describes the data better or at least as well as the three-factor model. Footnote 7 (continued) whereas 0.5 is moderate. They also claim that the average variance extracted (AVE) should be 0.5, which is the AVE for the first factor. So, even though items Ac3 and Ap1 contribute somewhat less to the latent factors we retain them in the model. However, results indicate that this is not the case (CFI: 0.85, RMSEA: 0.13). We also ran analyses using a third item for respect ("It bothers me that some people have different traditions and lifestyles") but its inclusion leads to slightly worse fit in the Swedish sample (CFI: 0.958, RMSEA: 0.068) and poor fit in the cross-national sample. Thus, we choose the most parsimonious 8-item model.
We only have two items measuring respect, which one could argue violates the common assumption that one needs three manifest items for a latent construct. However, this is not obvious as the contention exists that one item may suffice if the constructs are theoretically well defined (e.g., Bergkvist and Rossiter 2007;Hayduk and Littvay 2012). We also ran models where we set the covariance between the two respect items to zero (CFI = 0.975 RMSEA = 0.057) as well further restricting the model to include equal loadings. These alternative model specifications reduce the model fit marginally (CFI = 0.972 RMSEA = 0.058) but still indicate an acceptable fitting model. Additionally, we ran models with only two items for each dimension of tolerance, by removing Ac3 and Ap1 in a nonrestricted model (CFI = 0.995 RMSEA = 0.034). Finally, we restricted this model, assigning the factor loadings to be pairwise equal while setting error term correlation between the pairwise variable to zero, producing a somewhat worse but still acceptable fit (CFI = 0.979 RMSEA = 0.063). Such models also yield standardized loadings between 0.68 and 0.85. This suggests (net further tests) that these six theoretically motivated variables may be used to measure tolerance in situations where it is pragmatic or necessary to have fewer items in a survey.
To assess if the model holds in different contexts, we use multi-group confirmatory factor analysis to test for measurement invariance across five countries (Australia, Denmark, Sweden, United Kingdom, and United States). These countries also represent three different languages (Danish, English, and Swedish). We report results from this analysis in Table 3. According to Chen (2007), when N > 300 we should expect, foremost, CFI to decrease less than 0.01 between models. Results from the configural and the metric models demonstrate this. In fact, CFI is only reduced by 0.012 when comparing the metric and the scalar model, indicating that our invariance test almost reaches the threshold for scalar invariance. We choose to be conservative and only acknowledge metric invariance. Changes in RMSEA and SRMR are also within acceptable boundaries (Chen 2007).
Based on this analysis, we conclude that our respondents from Australia, Denmark, Sweden, the United Kingdom, and the United States understand tolerance similarly. Furthermore, we find support for metric invariance, meaning that the factor structure of the measure is equivalent across groups. This indicates that participants attribute the same meaning to the three latent constructs regardless of country. Thus, it is possible to study associations between three dimensions of tolerance and other individual-level variables across countries in the future. We conclude that the cross-national sample provides evidence that our measurement of tolerance works in three large countries in the Englishspeaking world as well as in two Northern European countries. The aim of this research is to advance a new way of operationalizing tolerance; thus, our efforts center on identifying commonalities across countries and not to explain differences between countries.

The Relationships Among Tolerance, Prejudice, and Other Attitudes
Results from our confirmatory factor analysis show that our measures of tolerance are internally consistent. Our next aim is to evaluate if these measures also have good convergent validity. We assess this by analyzing the bivariate relationship between tolerance and various measures of prejudice. We rely on measures commonly used in analyses of prejudice and already validated in previous research (e.g., Bohman and Hjerm 2016;Glick and Fiske 1996;Pettigrew and Meertens 1995). These variables capture prejudice, anti-gay sentiment, anti-immigration sentiment, and sexism. We expect the three aspects of tolerance to be associated with prejudice but to vary in the strength of those relationships. We also examine discriminant validity by analyzing the relationship between tolerance and attitudes towards welfare state support, as we have no theoretical reason to expect tolerance and welfare state attitudes to be related. These measures are also commonly used and empirically validated in previous research (e.g., Roosma et al. 2013;Eger and Breznau 2017). Table 4 describes the items included in these analyses. Table 5 reports bivariate correlations among our three latent constructs of tolerance and these attitudes. Coefficients reveal a clear pattern: each aspect of tolerance is negatively correlated with prejudice, specifically negative attitudes towards immigrants, immigration, women, and homosexuals. As we move from acceptance of diversity to respect and appreciation, the size of the correlations increases. This indicates that people who express an appreciation for diversity are less prejudiced than those who only accept diversity. As expected, there are weak correlations among all aspects of tolerance and welfare state support.
Given the different expressions of tolerance are correlated, we also examine multivariate relationships to isolate their respective effects and to provide a clearer picture of the relationship between tolerance and prejudice. As Table 6 shows, when regressing the dependent variables on all three aspects of tolerance, only appreciation for diversity remains negatively associated with attitudes towards out-groups. These results show three important things. First, it is only appreciation of diversity-but not acceptance or respect-that helps explain prejudice. Thus, the bivariate relationships (reported in Table 5) between acceptance of diversity and prejudice as well as between respect for diversity and prejudice are driven by those who express all three types of tolerance. Second, these multivariate relationships demonstrate that it is possible to express some degree of tolerance regardless of whether one likes or dislikes racial and ethnic out-groups. Prejudice is not a prerequisite for acceptance of or respect for diversity. Third, it is possible to measure tolerance in a way that is distinct from prejudice towards specific out-groups.
We also note a relationship between acceptance of diversity and sexism. When controlling for different expressions of tolerance, this weak relationship becomes positive. As expected, we find no relationship between tolerance and welfare state support.
Although the three aspects of tolerance are correlated, additional analyses lead us to conclude that these results are not due to multicollinearity. First, we regress all outcome variables on factor scores to produce variance inflation factors (VIF), which indicate how much of the increased variance of a regression coefficient is due to collinearity. The VIF Table 4 Attitudinal variables Concept Source Items Classic prejudice European Social Survey How much would you mind or not mind if an immigrant who is of a different race or ethnic group than most people in Sweden: Was appointed your boss? How much would you mind or not mind if an immigrant who is of a different race or ethnic group than most people in Sweden: Married a close relative of yours? How much would you mind or not mind if an immigrant who is of a different race or ethnic group than most people in Sweden: Became your neighbor?
Anti-immigration European Social Survey Increase/decrease immigration: people with same race and ethnicity? Increase/decrease immigration: people with different race and ethnicity? Increase/decrease immigration: people from poorer countries in Europe? Increase/decrease immigration: people from poor countries outside of Europe?
Sexism Ambivalent Sexism Inventory (Glick and Fiske 1996) Women are too easily offended When women lose fairly, they claim discrimination Most women interpret innocent remarks as sexism Anti-gay Pew Research Center Do you think more gay and lesbian couples raising children is a good or bad thing for society?
Welfare state attitudes European Social Survey It is the government's responsibility to provide a job for anyone who needs it It is the government's responsibility to provide health care It is the government's responsibility to provide standard of living for the old It is the government's responsibility to provide a standard of living for the unemployed It is the government's responsibility to provide child care services for working parents It is the government's responsibility to provide paid leave from work to care for sick family is approximately 1.5 or lower for all cases, indicating low levels of multicollinearity. Second, we compare the models reported in Table 6 with models where we set all (as well as combinations of) outcome variables on tolerance to be equal (see Marsh et al. 2004). This enables a test of Chi squared difference between models. In no case is the more restricted model better than the less restricted model (i.e., more freely estimated parameters), suggesting no multicollinearity issues.

Predicting Tolerance in Sweden
To provide a descriptive overview of tolerance in Sweden, we regress a number of demographic variables on these three tolerance constructs. For ease of interpretation, we use manifest tolerance scores instead of factor scores. Values to range from 1 to 5, with 5 indicating the highest level of each aspect of tolerance. In Table 7, we report the relationships among tolerance and sex, age group, nativity, education level, civil status, subjective income and the Big Five personality traits. Results indicate no association between sex, nativity, or subjective income. Younger people express greater acceptance of, respect for, and appreciation for difference than those over 65 years old. Married and cohabitating partners articulate less acceptance than those who are single, but there are no differences in terms of respect and appreciation. Education matters for respect and appreciation but not for acceptance. In terms of personality, results show relationships among agreeableness and openness and all three dimensions of tolerance. Conscientiousness and neuroticism are weakly associated with one dimension. Extraversion is unrelated to tolerance. Comparing these results to findings from the literature on prejudice, important differences emerge. Research on prejudice tends to show that women are less prejudiced than men; the elderly are more prejudiced than those who are younger; income is negatively associated with prejudice; and immigrants are much less prejudiced (towards other immigrants). Moreover, research consistently demonstrates that higher education is inversely related to prejudice. The results reported in Table 7 deviate from this pattern in that there is no sex difference nor any differences due to income or nativity. This provides further evidence that, although tolerance is related to prejudice, it is a distinct phenomenon. The relationships we find among tolerance and the Big Five personality traits are consistent with some studies of prejudice (e.g., Ekehammar and Akrami 2003) and inconsistent with others (Rapp and Freitag 2015).

Conclusion
In this article, we advance the study of tolerance by designing research to overcome both the theoretical and empirical conflation of prejudice and tolerance. There are two main theoretical approaches to tolerance. In the first, dislike of an out-group is a prerequisite for tolerance, meaning that one cannot be tolerant without having first been prejudiced. The implication of this conceptualization is that intolerance is also an indicator of prejudice, making it impossible to analytically-let alone empirically-separate the two constructs. According to the second theoretical tradition, tolerance is a phenomenon distinct from prejudice. Nevertheless, previous empirical research from this tradition incorporates prejudice into the measurement of tolerance by using questions that gauge attitudes towards specific out-groups. Our goal was to overcome these limitations by developing a theoretically driven, multidimensional conception of tolerance that can also be operationalized and measured in a way that is distinct from prejudice or any other concept. Therefore, we began with a definition of tolerance that is analytically distinct from prejudice. We define tolerance as a value orientation towards difference. Based on previous theoretical work, we identified three expressions of tolerance: acceptance of, respect for, and appreciation of diversity. Next, we developed measures consistent with this conceptualization. Specifically, we designed survey items that capture reactions to diversity itself instead of attitudes towards specific out-groups, thereby also acknowledging people's capacity for abstract thought. Our measures are temporally and politically neutral, which are essential for analyses over time and geography. With these efforts, we overcome additional limitations of previous research.
We administered two surveys, first in a single country (Sweden) and then cross-nationally in five countries (Australia, Denmark, Sweden, the United Kingdom, and the United States). Results from confirmatory factor analysis demonstrate that the three-factor model has good fit. Based on results from multi-group confirmatory factor analysis, we conclude that respondents across countries understand tolerance similarly. This means that these items can be used to examine tolerance, including its determinants or consequences, both within and across countries. Further, we demonstrated convergent validity by examining the relationship among tolerance and various measures of prejudice. We also found discriminant validity in relation to welfare attitudes. Relationships among types of tolerance and demographic variables lend credence to our claim that, although tolerance is correlated with prejudice, it is a distinct phenomenon that can, and should, be operationalized as such.
Our results suggest that only an appreciation of difference has the potential to reduce prejudice, but we do not know how tolerance is related to other individual-level or societallevel outcomes. Thus, we do not argue that individuals and societies should strive to appreciate all forms of difference. Future research should examine the extent to which these aspects of tolerance affect behavior-political and mundane. Research should also study the societal-level consequences of different aspects of tolerance.
Despite our contribution, we must acknowledge several limitations to our study. First, our data collection was limited to WEIRD countries (i.e., western, educated, industrialized, rich, and democratic) (Henrich et al. 2010); hence, future studies should assess whether our measures of tolerance are valid within other institutional and cultural contexts. Second, our empirical studies rely on cross-sectional data. Therefore, we do not know how stable our measures of tolerance are over time. Third, we have not examined our items in relation to earlier attempts at measuring tolerance, including political tolerance. This was beyond the scope of this paper, but something that future research should address.
Finally, we want to reiterate that our goal is not to make moral judgments about tolerance. The purpose of this research was to develop new measures that are consistent with an abstract, analytically distinct conceptualization of tolerance. Thus, we do not claim that tolerance is something inherently good or bad. The consequences of tolerance-different expressions and levels-remain empirical questions. Balint (2010) explains that "[e]ven if it is found empirically that learning about and respecting each other's differences is useful for achieving and maintaining a tolerant society, crucially this does not give carte blanche to have this approach to difference maximized. It must be used in the minimal possible way to achieve its ends" (p. 137). Balint's critique is consistent with Popper's (1945) "paradox of tolerance." Popper argues that tolerance of everything may actually lead to the disappearance of tolerance itself. Tolerating ideas or groups that infringe on others' freedoms and civil rights may undermine their existence (both the groups and freedoms). Davis (1995) illustrates this point in his analysis of tolerance of the Ku Klux Klan (KKK) among African-Americans.
As stated at the outset of this research, we believe there is an important practical reason to clarify the meaning and measurement of tolerance, as it is often invoked as something important for individuals and societies to develop and demonstrate. In the face of increasing diversity across contemporary democracies, calls for "greater tolerance" of particular social groups has become commonplace. Yet without a clear understanding of tolerance, these imperatives are hollow. How much acceptance of, respect for, or appreciation of difference is necessary to reduce discrimination, violence, or other social problems that may undermine the functioning of democratic societies? We do not claim to have these answers, but by developing tools to study tolerance, this research moves us in the direction of being able to address these types of questions. Indeed, the analytical and methodological approach developed in this article makes this type of empirical research possible.