1 Introduction

The rise of radical right-wing populist parties (RRP) has been the subject of much discussion and debate among scholars. One can divide this debate into supply-side research, which emphasises factors such as party ideology and policy strategies, and demand-side research, which argues that considerations such as economic and attitudinal factors are key to understanding support. The existing demand-side literature mostly relies on direct self-report measures of these variables, which may be prone to two biases in particular: social desirability and avoidance of cognitive dissonance.

It has been shown that citizens may hide or mitigate socially stigmatised opinions, known as social desirability bias (Chung and Monroe 2003; Janus 2010). To circumvent this issue, researchers have developed methods to assess attitudes in a more “automatic, unconsciously activated” (Bos et al. 2018, p. 70) way, labelled as implicit attitudes (Greenwald et al. 1998; Payne et al. 2005). Indeed, techniques such as the Implicit Association Test (IAT) have shown that the relationship between anti-immigration attitudes (AIA) and RRP support may be biased at both ends, meaning that both variables may suffer from underreporting (Bos et al. 2018; Tutić and Grehl 2021; Maier et al. 2022). In parallel, research on political ideologies and polarisation has long considered cognitive dissonance as an influence on citizens’ reported attitudes (Festinger 1957; Harmon-Jones and Harmon-Jones 2007). More recent research suggests that implicit attitudes may also circumvent this phenomenon (Breithaupt et al. 2020).

The current study aims to bridge the gaps between these different strands of literature. Drawing upon their findings and theoretical arguments, grounded hypotheses were formulated to capture the potential influence of social desirability bias and cognitive dissonance avoidance on the association between AIA and RRP support. The preregisteredFootnote 1 research design of this study relies on implicit and explicit ratings of the independent variable (AIA) and the dependent variable (RRP sympathy), enabling distinct assessments of the proposed hypotheses. The test case for this study is the German party Alternative for Germany (AfD).

This study advances existing knowledge in several ways. First, it adds empirical evidence to the relationship between AIA and support for an RRP, making a unique contribution by assessing both variables explicitly and implicitly. This approach allows estimation of the effect size in all possible combinations of these variables, enabling the testing of specific hypotheses regarding social desirability bias and cognitive dissonance avoidance. Second, this study advances theoretical understanding by integrating two previously separate strands of research in a combined setting and analysis. By drawing on theoretical and empirical knowledge from both strands, it establishes a solid theoretical foundation for the empirical analyses. Third, at the conceptual level, it uses a stratified quota sample, which allows for better generalisation than common convenience samples in social psychology (Gawronski and Strack 2004; Friese et al. 2007).

2 Biases in the Association Between Anti-immigration Attitudes and Sympathy for the AfD

2.1 The AfD—A Radical Right-wing Populist Party

Radical right-wing populist parties have entered most European parliaments. In the last decade, previously exceptional cases such as Sweden (Rydgren and van der Meiden 2019) and Germany (Arzheimer and Berning 2019) have also seen the successful establishment of such parties. Although this family of parties has attracted considerable scholarly attention (Golder 2016; Mudde 2019), the lack of a universally accepted definition has led to a wide range of interpretations. The often-cited definition by Mudde (2007) emphasises the element of “identity” in politics, whereas more recent studies seek to identify several relatively well-defined criteria, which may include, for example, whether “the electorate sees the party as far right” (Donovan 2020). Alternatively, some researchers split the term, defining “populist” and “radical/extreme right” separately and then identifying parties that meet both criteria (Rooduijn et al. 2019). Despite this lack of an accepted definition, there is “a near consensus on which parties should be included in the party family” (Rydgren 2007), which has allowed scholars to identify common explanatory factors for their success.

These factors can be grouped into contextual (Knigge 1998; Han 2016; Amengay and Stockemer 2019), supply-side (Werkmann and Gherghina 2018; Heinze 2018), and demand-side factors. Commonly considered individual demand-side predictors include, but are not limited to, sociodemographic variables, such as gender and education (Spierings and Zaslove 2017; Stockemer et al. 2018); attitudes towards politics and the political system, such as dissatisfaction with and distrust of politics and polities (Bowler et al. 2017; Rooduijn 2018; Muis et al. 2021); populist attitudes (van Hauwaert and van Kessel 2018; Hawkins et al. 2020); nationalist or authoritarian attitudes (Dunn 2015; Lubbers and Coenders 2017); and individual economic situations/attitudes (Golder 2016; Rama Caamaño and Cordero 2018; Savage 2023). Among these factors, AIA stand out due to their frequent consideration and consistently strong results (Rydgren 2008; Oesch 2008; Lubbers and Coenders 2017; Spierings and Zaslove 2017; Arzheimer 2018; van Hauwaert and van Kessel 2018; Rama Caamaño and Cordero 2018; Rooduijn 2018; Stockemer et al. 2018; Muis et al. 2021; Savage 2023).

Many of these common predictors also reliably predict support for the party examined in this study, the AfD. Men, the less educated, the politically disaffected, and citizens from eastern Germany are more likely to express support for the AfD (Klein et al. 2018; Arzheimer and Berning 2019; Weisskircher 2020; Peshty et al. 2021). There is considerable scholarly debate regarding the relationship between economic disadvantage and support for the AfD, with some arguing that education and AIA mediate this association (Klein et al. 2018; Lengfeld 2018; Lux 2018; Rippl and Seipel 2018; Schröder 2018; Tutić and von Hermanni 2018). However, scholars mostly agree that sociocultural issues, especially AIA, are crucial for understanding support for this party (Goerres et al. 2018; Steiner and Landwehr 2018; Hansen and Olsen 2019; Franzmann et al. 2020; Wurthmann et al. 2021). The remainder of this study will focus on this explanatory factor in particular.

Although today’s AfD is almost unanimously seen as an RRP party and shares core characteristics with other parties in this family, this was not always the case. When it was founded only a decade ago as a more or less single-issue party focused on the Euro and European politics, it was regarded more as an ordo-liberal, conservative challenger party with populist elements (Grimm 2015; Franzmann 2016; Schmitt-Beck 2017). However, even in its formative years, its electorate held the typical AIA (Schwarzbözl and Fatke 2016) and was seen by some as an RRP or “functional equivalent” (Arzheimer 2015; Berbuir et al. 2015; Lewandowsky et al. 2016). In the course of the so-called refugee crisis and the change of party leadership, the party completely transformed into a radical right-wing populist party (Jankowski and Lewandowsky 2018; Arzheimer and Berning 2019; Atzpodien 2022).

The unique setting of German politics and the history of the party make it a perfect test case for the present study. First, the German setting is particularly interesting for the study of social desirability bias—one of the two biases that this paper seeks to investigate. Due to the history of a fascist regime and the Holocaust, openly racist opinions were heavily and promptly sanctioned in public debates. This contributed to both a long-standing German exceptionalism with regard to the presence of radical right parties in national politics and social desirability biases with regard to AIA the main independent variable of this study (Art 2006; Berbuir et al. 2015). Second, compared to other European RRPs, the AfD is a relatively young party, founded only a decade ago, and was not generally considered racist or anti-immigration per se until the so-called refugee crisis (Arzheimer and Berning 2019). This distinguishes it from parties in culturally or historically similar neighbouring countries, which have a much longer tradition of anti-immigrant and partly racist positions, such as the Lega (Nord) in Italy (Richardson and Colombo 2013), the SVP in Switzerland or the FPÖ in Austria (McGann and Kitschelt 2005). Particularly as this study attempts to use implicit—“automatic, unconsciously activated” (Bos et al. 2018, p. 70), this temporal distinction is advantageous as it allows to examine associations with the AfD that are based solely on the party’s recent years, rather than decades-old associations or prominent historical figures.

2.2 Issues with the Use of Self-reports in Studying the Effect of AIA On AfD Sympathy

Although extensive literature has revealed a clear pattern regarding the impact of AIA on support for the AfD, these findings are mostly based on direct self-reports. These direct measures are prone to bias. Participants may—consciously or unconsciously—deviate from their “true” opinion on an issue, which ultimately distorts the analysis of such variables and the relationships between them. In this study, two biases are discussed that may have particularly distorted results in the past: social desirability bias and cognitive dissonance avoidance.

Social desirability bias, which means that respondents may hide—or at least mitigate—their opinions if they are perceived as “undesirable”, may have biased previous, fully explicit results, on the relationship between AIA and AfD sympathy. Issues of migration have become highly controversial and emotional in recent years, especially since the European refugee crisis, so respondents may have learned to take softer positions in this context, even if their “true” opinion is more pronounced, in order to avoid confrontation. Previous research suggests, that direct measurements indeed underestimate the extent of AIA among citizens (Janus 2010; An 2015; Creighton et al. 2015; Bazo Vienrich and Creighton 2018; Maier et al. 2022) and particularly among sympathisers of right-wing parties (Carmines and Nassar 2021). With respect to the research topic of this study, social desirability may actually lead to flawed direct measures at both ends of the relationship. Similar to open AIA, radical right parties face considerable opposition in Germany, as in many other European countries. Involvement or sympathy for such parties can cause serious problems with one’s social environment. Therefore, social desirability bias may also lead to an underestimation of reported support for the AfD (Bos et al. 2018; Gschwend et al. 2018).

The possibility of underestimating anti-immigrant attitudes and sympathy for the AfD in direct measures introduces a potential bias in the distribution and levels of these variables, potentially impacting their relationship. Specifically, the influence of AIA on AfD sympathy may have been underestimated if individuals with simultaneous attachment to both opted for lower ratings on the relevant scales, despite holding more extreme views towards both. Thus, the correlation and ultimately the effects between these variables may be lower than they would be without social desirability bias. Consequently, the following hypotheses are formulated, capturing the anticipation that the underestimation of the relationship between these variables is a result of social desirability bias in the explicit measurement of one or both of them.

H1a)

Using explicit measures of both concepts underestimates the relation of AIA to AfD sympathy because of social desirability bias in the measurement of AIA.

H1b)

Using explicit measures of both concepts underestimates the relation of AIA to AfD sympathy because of social desirability bias in the measurement of AfD sympathy.

Another factor, albeit less prominent, that may introduce bias in drawing conclusions is respondents’ tendency to avoid cognitive dissonance. According to Festinger (1957), individuals often strive to avoid holding contradictory opinions. When confronted with conflicting attitudes towards two objects, people may attempt to resolve the contradiction by adjusting their evaluation of one of the objects to harmonise their ideology.Footnote 2 This phenomenon has been extensively documented across various domains (for a summary, see Harmon-Jones and Harmon-Jones 2007) and is vital in recent publications on, e.g., technology adoption (Marikyan et al. 2023), leadership–employee relations (Mesdaghinia et al. 2019), and eating habits and health (Ong et al. 2017; Rothgerber and Rosenfeld 2021; Stiglbauer et al. 2019). Regarding political attitudes, researchers have shown that individuals tend to “stick to their vote” (Mullainathan and Washington 2009) after an election, adjusting their attitudes (Vecchione et al. 2013) and even perceptions of economic performance (Sorace and Hobolt 2021) to avoid cognitive dissonance.

The presence of cognitive dissonance avoidance is also likely to be significant in this study. Given the highly polarised discourse surrounding migration, individuals may feel compelled to maintain a high level of consistency in their political stance, leading them to align their attitudes with the appropriate political parties,Footnote 3 even if their true ideology is less coherent (Mullainathan and Washington 2009; Levendusky 2010; Vecchione et al. 2013). This alignment could potentially inflate the correlation between the two concepts, resulting in an overestimation of the relationship between AIA and AfD sympathy when assessed through direct self-report measures. Consequently, the following hypothesis is formulated:

H2)

Using explicit measures of both concepts overestimates the relation of AIA to AfD sympathy because of avoidance of cognitive dissonance by the respondents (leading them to give answers that fit well together).

2.3 Implicit Attitudes as a Possible Way to Detect Such Biases

It can be inferred from the preceding discussion that the use of explicit attitudes is not ideal for evaluating citizens’ attitudes and their interrelationships. Social psychology has proposed alternative indirect measures to assess so-called implicit attitudes (Greenwald et al. 1998). The aim of this paper is to compare the assessment of attitudes using both direct and indirect measures, with particular focus on examining the associations between them. By doing so, this empirical study contributes to the ongoing discourse on predictors of sympathy for RRPs and the current debate on implicit attitudes within political sociology (Tutić and Grehl 2021; Maier et al. 2022). To this end, it is crucial that terms are clearly defined and used consistently. This study adheres to the definition of implicitFootnote 4 attitudes as a measurement of an “automatic, unconsciously activated component” (Bos et al. 2018, p. 70) of attitudes, drawing upon the dual-process model (Smith and DeCoster 2000; Greenwald et al. 2009). Stated differently, “what differs [between explicit and implicit attitudes] is just that there are factors that influence explicit (verbal) responses that don’t influence implicit (non-verbal) ones, and vice versa” (Carruthers 2018, p. 70).

Despite the documented advantages of implicit attitudes in the field of demand-side predictors, particularly in assessing socially undesirable views such as AIA in voting preferences (Friese et al. 2007; Arcuri et al. 2008; Roccato and Zogmaister 2010; Ditonto et al. 2013), the method is still used relatively rarely. In the German research context, however, there has been a recent resurgence of interest in this topic, exemplified by the works of Tutić and Grehl (2021) and Maier et al. (2022). The former discovered relatively weak correlations between explicit and implicit racism, and no significant impact of implicit attitudes on explicit AfD voting intentions. Likewise, in the study by Maier et al. (2022), the explanatory power of implicit AIA, based on Muslim/Islam primes, in predicting AfD sympathy was found to be quite limited. The literature on assessing implicit attitudes towards an RRP is even scarcer. As of now, the study by Bos et al. (2018) is perhaps the only publication that examines implicit attitudes toward a specific party, namely the Dutch PVV.

Although these recent findings concerning the explanatory power of implicit attitudes in predicting voting intentions are somewhat discouraging (Tutić and Grehl 2021; Maier et al. 2022), they converge on the notion that implicit attitudes have the potential to mitigate or at least minimize the influence of social desirability. Furthermore, previous research suggests that implicit attitudes may also circumvent the issue of cognitive dissonance avoidance. While explicit attitudes can be manipulated by inducing cognitive dissonance, implicit attitudes appear to remain unaffected (Breithaupt et al. 2020; Gawronski and Strack 2004). A compelling example of this is the study by Swanson et al. (2001), in which stigmatized individuals (smokers) tended to report highly consistent views about their stigmatised behaviour, despite having rather inconsistent implicit attitudes. The same may be applicable to the examination of support for the AfD, potentially leading to an overestimation of the true relationship between AIA and sympathy for this party.

These facets of implicit attitudes form the basis of this study’s research design. Assuming that social desirability bias and avoidance of cognitive dissonance are factors that “influence explicit (verbal) responses [but] don’t influence implicit (non-verbal) ones, and vice versa” (Carruthers 2018, p. 70), their differences in empirical explanatory power are expected to provide a measure of the extent of these factors. The subsequent section outlines the approach taken to quantify and assess the significance of these differences.

2.4 Analytic Strategy to Test for Significant Differences Between Implicit and Explicit Relationships

The primary focus of this study is to examine whether prior findings on the association between AIA and sympathy for the AfD have been influenced by social desirability bias and avoidance of cognitive dissonance. In order to accomplish this, it was investigated whether there were significant variations in the impact of AIA on AfD sympathy when different combinations of direct and indirect measures were employed in the analyses. Table 1 presents the specific combinations used in the study, with groups A to D representing distinct combinations of measures. The dataset contains both implicit and explicit measures of both variables for each respondent, which have been transformed into a long format. Each respondent is thus represented by four entries, corresponding to the different combinations of explicit and implicit measurements. The implicit and explicit measures of AfD sympathy and AIA have been collapsed into one column each. The group variable (A, B, C, or D) is used to differentiate the combinations, indicating whether none, one, or both variables were measured implicitly for a particular entry in the dataset.

Table 1 Illustration of combinations of implicit and explicit variables and the associated group label
Full size table

The experimental design enables the testing of the aforementioned hypotheses. A preregistered analytical and inferential strategy was employed as follows: Hypothesis 1a, which assumes a bias resulting from social desirability bias in AIA, will be supported if the effect of AIA is significantly greater when AIA is measured implicitly and AfD preferences are measured explicitly (B) compared to when both variables are measured explicitly (A). Hypothesis 1b, which posits a social desirability bias in AfD sympathy, will be supported if the effect of AIA is significantly larger when AfD sympathy is measured implicitly and AIA explicitly (C) than when both are measured explicitly (A). Hypothesis 2, which captures the expectation of avoidance of cognitive dissonance bias, will be supported if the effect of AIA on AfD sympathy is significantly reduced when both variables are measured implicitly (D) compared to when both are measured explicitly (A).

3 Methods

3.1 Description of the Survey, Data Collection, and Cleaning Process

Data collection for this study was carried out via an online access panel provider (mingle, a Bilendi platform). Invitation and access to the survey followed stratification based on census data for sex and age.Footnote 5 The panel provider was instructed to invite only participants who were permanent residents of Germany and had proficient command of the German language. Individuals with severe visual impairments, including partial or total blindness, were excluded from participating in the survey.Footnote 6

Within the survey, participants were initially requested to provide information regarding their sociodemographic characteristics. Subsequently, they were presented with anti-immigration items, questions related to party sympathies, and two Single-Category Implicit Association Tests (SC-IATs). These SC-IATs are adaptations of the original Implicit Association Test (IAT) developed to measure implicit attitudes (Greenwald et al. 1998). Although the IAT is considered “the most frequently used and most carefully tested technique” (Arcuri et al. 2008) in that regard, it has also encountered criticism (see Jost 2019 for a review and response). The SC-IAT reduces the target categories from two to one (Karpinski and Steinman 2006). Instead of having two poles within one category (e.g., black and white faces), it evaluates implicit positive or negative attitudes towards a single category (e.g., white faces). In brief, participants are presented with visual or verbal primes from three categories: positive, negative, and the target category. They are then required to swiftly sort these primes into the appropriate category by pressing keys on a keyboard. The target category shares a key either with the “positive” or the “negative” category. If an individual performs better (worse) when the target category is associated with the negative category, it is assumed that they have a more negative (positive) implicit attitude towards the target concept. The questionnaire concluded with an open feedback section and a short debriefing on the SC-IATs. By employing a stratified sample and incorporating the use of SC-IATs, the aim is to leverage the internal validity of an experimental design while enhancingFootnote 7 the generalizability of the study’s findings.

The field phase of data collection commenced with the primary collection period from 20 to 24 April 2022, followed by a supplementary phase from 23 to 25 May 2022. The purpose of the second phase was to address gaps in the intended quotations caused by unevenly distributed error rates observed in the SC-IATs.Footnote 8 A total of 547 individuals completed the questionnaire after both rounds, but approximately 33% had to be excluded due to missing data. A listwise deletion approach was employed to handle missing data, encompassing both the questionnaire responses and the results of the SC-IATs. This decision was preregistered and implemented to reduce potential issues arising from the interaction between responsiveness in the questionnaire and performance in the tests. Within the SC-IATs, participants were excluded based on the routine outlined by Karpinski and Steinman (2006), whereby individuals with an error rate exceeding 20% were not included in the analysis. Following these selection criteria, a final sample of 369 individuals remained for analyses.

This final sample slightly diverges from general population characteristics.Footnote 9 Most notably, the ratio of individuals reporting a higher education degree is 29.5%, which is considerably higher than the approximately 23% in the general population aged 18 years and older (Statistisches Bundesamt 2022a). Men in their 50s and women in their 60s are slightly underrepresented in the sample due to their particularly high failure rates of over 40% among all cases collected, which could not completely be compensated for by oversampling these groups. However, in general, the sample matches age and sex of the general German public relatively closely. In the final sample, women make up about 51% and 61.5% were employed, compared to about 65% in the general population aged 18 years and older (Statistisches Bundesamt 2022b).

3.2 Measurements

The study measures four main variables of interest: explicit and implicit AIA (independent variable) as well as explicit and implicit RRP preferences (dependent variable). This section briefly describes these measurements of this survey. For more details, please refer to the Online Appendix.

3.2.1 Explicit Measurements

To assess RRP preferences, this study relies on a “sympathy scale” (“In general, what do you think of the following parties?”) instead of the traditional voting question (Gschwend et al. 2018). Given that citizens’ voting intentions are influenced by external factors (e.g., thresholds for parliamentary entry, coalition considerations; see van der Eijk et al. 2006), sympathy scales are more suited to capture tendencies towards a specific party. Wording was adopted from the German Longitudinal Election Study (GLES) Short-term Campaign Panel (GLES 2019).

The direct measurement of AIA is composed of four well-established items. These items aimed to capture respondents’ opinions on the economic, cultural, and crime rate impacts of immigration in Germany, as well as their stance on whether immigration to Germany should be facilitated. The wording for the first three items was adapted from the Comparative Study of Electoral Systems (CSES 2021), and the wording for the fourth item was derived from the GLES Short-term Campaign Panel (GLES 2019). An equally weighted index of the four items was computed. For this index of explicit AIA, Cronbach’s alpha shows high reliability (0.90).

Furthermore, participants were asked to report their age, gender, level of education, and employment status within the survey. These variables constitute common covariates in the related literature and are used accordingly.

3.2.2 Implicit Measurements

The core elements of data collection for this study were two SC-IATs, one focusing on indirect attitudes towards migration and the other on attitudes towards the AfD. Verbal primes were used for all categories, drawing on previous research. Positive and negative terms were derived from the German version of the Project Implicit website (Nosek et al. 2007). The terms for “migration” were informed by Johann and Thomas (2018) and are mostly synonyms or subgroups of the term “migrants.” Note that, unlike other studies in this field (Tutić and Grehl 2021; Maier et al. 2022), the primes for this study were limited to synonyms of “migration” and avoid any direct association with Islam or Muslims, such as Islamic names or images of women wearing headscarves. While acknowledging that the majority of migrants in recent years have come from Islamic countries and that this fact has a prominent role in public debates and strategies of RRPs, these are nevertheless distinct concepts and should be studied that way. This is crucial to ascertain the driving factors behind the observed effects, namely opposition to immigration/immigrants, opposition to Islam or Muslims, or opposition to Muslim immigrants. This differentiation also facilitates the applicability of methodology and findings to various contexts, including Christian immigrants or Islamic host countries. However, it remains possible that respondents’ associations between Islam and immigration could still exert an influence on the results. The terms used to describe the AfD were developed from scratch, encompassing the party’s full name, prominent and leading figures, and the slogan employed during the most recent election, which took place approximately 6 months prior to the data collection phase.

The methodology closely adhered to the guidelines set forth by Karpinski and Steinmann (2006). Each of the two tests comprised four blocks of trials, during which participants were required to swiftly categorise items from the “positive” and “negative” categories, as well as from a target category, by pressing the corresponding keys on their keyboard. The first test focused on attitudes towards migration, and the second test targeted attitudes towards the AfD. Following established procedures (Karpinski and Steinman 2006), D‑scores were derived from the collected reaction times. These D‑scores serve as the implicit measures of attitudes, with higher values indicating higher AIA or sympathy for the AfD.

4 Results

This section consists of two components. The first part is a descriptive discussion of the results obtained from the SC-IATs. Subsequently, a preregistered multivariate analysis was conducted to test the hypotheses. This study adheres to the most recent standards of open science. All analyses presented in this article were preregistered, and comprehensive robustness checks address important limitations of the obtained results. The dataset and all recoding, analyses, and visualisations are openly accessible, facilitating easy replication of all findings.Footnote 10

4.1 Descriptive Results

4.1.1 Descriptive Results of Main Variables

Before turning to the multivariate model, the collected data are described, with specific emphasis on the SC-IAT data. Descriptive results of the SC-IATs are reported alongside the corresponding explicit measures. All variables were z‑standardised to a mean of zero and a standard deviation of 1.

The AIA measure shows relatively similar distributions for both measures (Fig. 1). The traditional self-report measure shows an overrepresentation of extreme values and a higher concentration around the mean, whereas the implicit measure partially compensates for these characteristics. However, when the measurement of AfD sympathy is examined, notable differences between the two methods emerge. The explicit measurement demonstrates a highly skewed shape with a prominent peak at the lowest value on the scale, indicating that nearly three-quarters of all valid respondents reported having no sympathy at all for the AfD. In contrast, the implicit attitudes display an almost perfect Gaussian distribution.

Fig. 1
figure 1

Density plots of explicit and implicit anti-immigration attitudes (a) and Alternative for Germany (AfD) sympathy (b). All variables were z‑standardised. All graphs are based on n = 369 cases

Full size image

The skewed sympathy distribution for the AfD is not entirely unexpected, as surveys often find an overwhelming majority of respondents expressing no sympathy towards the party.Footnote 11 However, the discrepancy between the explicit and implicit assessments raises questions regarding the validity of implicit attitudes and their implementation in this study or the justification of explicit assessments of AfD sympathy. The widely used method of SC-IATs was implemented in this study with great attention, adhering closely to the established guidelines and using face-valid items. If one agrees with the general assumption that implicit measures reflect a distinct yet valid assessment of individuals’ attitudes, there is little reason to question the general validity of the present findings. This suggests that common direct measures of AfD party sympathy may indeed be biased. However, the correlations between implicit and explicit measures are remarkably similar for both variables at 0.37 (p < 0.01) and 0.38 (p < 0.01), which is in line with previous research (Greenwald et al. 2015; Bos et al. 2018), providing evidence that the extreme aversion towards the AfD expressed in direct measures may overestimate its true extent, but the general tendency of citizens is also reflected in the implicit measure. Consequently, both variables were deemed comparable and used in subsequent multivariate analyses.

4.1.2 Universality of the SC-IAT

Due to unevenly distributed failure rates in the SC-IATs,Footnote 12 it was suspected that variations in these results might be related to sociodemographic variables. Figure 2 plots the means for AfD sympathy and AIA by method of measurement as well as age and education level. To keep the graphs comparable, the standardised versions of all variables were used; therefore, the overall mean is always zero, and the overall standard deviation is always 1.

Fig. 2
figure 2

Descriptive statistics for anti-immigration attitudes and Alternative for Germany (AfD) sympathy split by means of collection (implicit vs. explicit) and age groups or education

Full size image

Regarding the education variable, the overall pattern and differences between the various education levels remain relatively similar regardless of the method of collection. However, respondents with the lowest level of education move from the far left of the graph (indicating relatively low explicit AfD sympathy and AIA) to the far right in both graphs. It is important to note that the number of respondents in this educational group is small. Nevertheless, this shift may suggest that social desirability is more pronounced within this educational category, leading individuals to report low levels on both variables while holding more right-leaning implicit attitudes.

The age variable also exhibits some differences between explicit and implicit attitudes in this respect, although the pattern is less clear. For AIA, the youngest age group has low levels when asked explicitly, which become even lower when assessed implicitly, relative to other age cohorts. On the other end of the scale, individuals in their 60s shift from the lowest to the highest level compared to others when the implicit measure is used. Regarding sympathy for the AfD, the youngest age group switches from the most unfavourable to the most sympathetic position in implicit attitudes. All other groups remain in almost identical positions between the left and right panels, with older respondents having slightly less sympathy for the AfD.

Regarding gender and employment status, the SC-IAT shows slightly lower implicit and explicit AIA among male respondents than in female respondents. However, this and other differences are unlikely to be statistically significant.Footnote 13

These additional insights should not be overinterpreted because most 95% confidence intervals overlap to a great extent. Although this does not per se rule out statistically significant differences between these groups, it does give confidence that the SC-IATs are generally applicable to all societal groups within the variables investigated.

4.2 Multivariate Results

To adequately test the hypotheses, this study uses linear regression analyses. Note that the terms “effect” and “predictor” in relation to the coefficients from such regressions are used, as such analyses require the specification of independent and dependent variables. However, this should not be interpreted as implying causality. This analysis is subject to the same caveats as other nonexperimental research designs based on survey data, and findings should be interpreted as solely correlational. In all models, AIA is the independent variable, while AfD sympathy is the dependent variable. Additionally, all models include covariates, namely age, gender, level of education, and self-placement on the political left–right spectrum.

As a first step, separate ordinary least square models are estimated for the four possible combinations of measures (Table 1), employing a multigroup design in which each of the four combinations (A, B, C, D) is treated as a separate group. This approach allows an initial understanding of possible differences in the effect of AIA across the different operationalisations. In a second step, the statistical significance of these differences in effect sizes is estimated. This is achieved by using group dummy variables for the measurement combination as part of interaction terms with the AIA variable, testing their effect on AfD sympathy (with A serving as the reference group). This straightforward approach allows for precise point estimates of the difference between the “traditional” operationalisation A and each alternative option, including confidence intervals for these estimates. Central inferential criteria are the p-values of two-tailed tests within the regression analyses, and null hypotheses are rejected below the high threshold of p < 0.01. Standardised regression coefficients are reported. All analyses were conducted as they were preregistered.

4.2.1 Multigroup Analysis

The first conducted analysis is a multigroup analysis (Table 2), treating each of the four possible combinations of implicit and explicit dependent and independent variables as a distinct group of data, although using the same cases. Within each group, all variables were z‑standardised, which also sets the intercept of each model to zero by default. Model A reflects the “traditional” approach, relying solely on direct questions from the questionnaire. In this model, the effect of AIA on sympathy for the AfD is strongest, indicated by a standardised beta of 0.55 (0.05). The following two models use implicit measures for either AfD (model B) or AIA (model C), while using explicit measures for the other main variable. In both models, the effect size is more than halved compared to model A. In model D, in which both variables are implicit measures, the effect is close to zero and not statistically significant. Relatedly, the explanatory power of the models, expressed as R2, is much higher in model A than in the other groups.

Table 2 Multigroup regression analysis. Standardised (std) beta coefficients are reported, with standard errors in brackets. Italics indicate p < 0.1; p-values in boldface indicate significance below p < 0.01. Intercepts have been deliberately omitted from the table because the prior z‑standardisation of all variables sets them to zero within each group. The dependent variable is always AfD sympathy, either explicit (ex) or implicit (im), depending on the group
Full size table

Sociodemographic variables have quite diverse levels of impact on respondents’ evaluation of the AfD. Gender and current employment have no significant effect in any model. Interestingly, the effect of a person’s age is significantly and strongly negatively associated with preferences for the AfD in all models except model B (AfD implicit/AIA explicit). Conversely, the education variable reaches levels of statistical significance in all models except model A (AfD explicit/AIA explicit). These patterns may reflect the limited interaction effects between the mode of data collection and the sociodemographic variables discussed in the descriptive results.

So far, the groupwise analysis has revealed notable differences between the various model/measure combinations. Regarding the study’s hypotheses, the results seem to falsify the expectations of social desirability bias, as the effects of models involving implicit attitudes are significantly lower compared to the explicit–explicit model A, but they support the notion of avoidance of cognitive dissonance. However, it has not yet been assessed whether these differences are statistically significant. The next analytical step is to do precisely that. As described in the analytical strategy above, a multivariate regression was conducted using interaction terms between the groups and AIA.

4.2.2 Combined Analysis

Table 3 presents the results of a combined analysis. All variables were z‑standardised within each group, resulting in a default intercept and group dummies’ effects of zero by default. The sociodemographic variables now capture an overall effect across all combinations of measures. The AIA variable is included, both as it is and as an interaction term with each group dummy. The overall base effect thus reflects the standardised effect for group A, and the interaction terms capture the difference from A. These effect sizes are identical to the differences between each group and model A in the groupwise model above. However, modelling these as interaction effects allows a statistically sound test of the significance of these differences.

Table 3 Combined regression model of all groups. Intercepts and group dummy variables (although included) were deliberately omitted from the table because the prior z‑standardisation of all variables sets them to zero. The dependent variable is AfD sympathy
Full size table

The results confirm the previously described patterns: Although AIA is strongly related to sympathy for the AfD in group A, which uses only explicit measures, all interaction effects are negative, meaning that the positive effect of AIA is reduced in all groups compared to A. These differences are statistically significant (p < 0.001), meaning that whenever one or both of the main variables are assessed implicitly, the relationship between AIA and sympathy for the AfD is significantly lower compared to an all-explicit measure. Note that this analysis does not allow testing for significant differences between models that use at least one implicit measure. As no specific theoretical implications for such differences were specified, these tests were not conducted. The following section discusses the results in light of the theoretical assumptions set out at the beginning of this paper.

5 Discussion

How do these findings relate to the initial, preregistered hypotheses? The discussion focuses on two major theoretical issues that could bias explicit measures: social desirability and cognitive dissonance avoidance. Strategies were designed to test the presence of these phenomena. Regarding hypotheses 1A and 1B, it was stated that a stronger effect in the explicit–explicit model compared to the mixed models would confirm the presence of social desirability bias in the explicit measurement of the main concepts. The results regarding AIA indicate a significantly stronger effect of explicitly assessed AIA on explicit AfD sympathy compared to implicitly measured attitudes (comparing group A to B). This finding implies falsification of hypothesis 1A, which proposed a stronger effect in the partly implicit model due to socially desirable underreporting in explicit AIA. Similarly, when comparing explicit and implicit measures of AfD sympathy, the effect is stronger when the concept is assessed explicitly rather than implicitly (compare A with C). Hence, hypothesis 1B, which postulated a stronger effect for the implicit measurement due to socially desirable underreporting of explicit AfD sympathy, is also rejected.

Hypothesis H2 predicted that the relationship between AfD sympathy and AIA would be more pronounced in the explicit model compared to the implicit model, due to respondents’ tendency to report a coherent ideology, i.e., to report high sympathy for the AfD if they also report high AIA, and vice versa to avoid cognitive dissonance. Results support this hypothesis, as the relationship between sympathy for the AfD and AIA is significantly lower when implicit measures are used compared to when explicit statements are used (comparing A to D). In fact, not only is the effect reduced, but the relationship is essentially zero in the all-implicit model.

Taken together, these findings suggest that social desirability bias appears to be less of an issue, both in the direct measurement of sympathy for the AfD and in AIA. This conclusion depends on the premise that implicit attitudes can circumvent the issue of social desirability and that a comparison between a partially implicit and a fully explicit model can thus estimate the extent of social desirability in the respective variable.

Conversely, the results suggest that avoidance of cognitive dissonance appears to be a real threat to valid results in the context of this study. When asked directly, respondents reported much more consistent ideologies (high sympathy for the AfD along with high AIA and vice versa) compared to indirectly measured attitudes. This finding is based on the assumption that implicit attitudes may circumvent the phenomenon of cognitive dissonance avoidance, and a comparison between an all-implicit and an all-explicit model using the same variables can thus reveal its extent.

6 Robustness Checks

Although the above results appear to provide strong (counter)evidence for the hypotheses of this study, these findings are subject to the typical caveats of correlational studies, and the implications should be treated with caution. This section therefore discusses the robustness of these results. These tests were not preregistered.

6.1 Selective Covariates Bias—Specification Curve

Researchers have considerable freedom in the specification of their models. The set of covariates included in the model, or the coding of their main variables, can have a significant impact on the results. One way to address this issue and test for the influence of such “researcher’s degrees of freedom” is to perform what is known as a multiverse analysis. In this analysis, the researcher determines all possible combinations of different crossroads during the formulation of the model, runs the analysis for all of them, and reports all of them to transparently show how specific choices affect the results. As shown above, the type of measurement (implicit/explicit) can interact with sociodemographic variables. Do these interactions bias the results in Table 3? This was tested using a multiverse approach (Masur and Scharkow 2020). Figure 3 presents the specification curve, which illustrates the effect of AIA on sympathy for the AfD across all possible sets of covariates.

Fig. 3
figure 3

Multiverse analysis for the results obtained in Table 3. a The point estimate for the effect of anti-immigration attitudes on Alternative for Germany sympathy for each model configuration. b Information on which covariates were included in the model (model) and which group (see Table 1) was analysed (subsets)

Full size image

Although the point estimates are ordered by effect size, they strictly resemble the order of the groups, with no overlap between them. This suggests that the choice of covariates has no relevant effect on the differences between the groups in terms of the effect of AIA on AfD sympathy. In all cases, group A has significantly higher estimates compared to any model of the other groups, while the estimates of group D are always statistically insignificant.

6.2 Omitted Variable Bias—Sensitivity Analysis

The combined analysis revealed statistically significant differences for the effect of AIA on AfD sympathy depending on the collection method. However, the analysis of observational data always suffers from a potential omitted variable bias. This means that an unobserved variable could make an effect insignificant once it is included in the model formula. However, sensitivity analysis allows estimation of the strength of such an unobserved variable that would be required to change the conclusion drawn from the observed variables.

Table 4 presents the results of the sensitivity analysis conducted for the three interaction effects as shown in Table 3. The first row concerns the statistically significant difference between groups A and B. The analysis reveals that an unobserved confounder would need to account for 13.5% of the total variance of both the interaction effect and AfD sympathy to bring the interaction effect exactly to zero, while 8.9% would be sufficient to reduce the interaction effect to a nonsignificant level (α = 0.05). In an extreme scenario in which an unobserved confounder would explain all the remaining variance in the dependent variable, AfD sympathy, this confounder would also need to explain 2.1% of the variance in the interaction term in order to set it to zero. The other rows of the table can be interpreted accordingly for the interaction effects of groups C and D.

Table 4 Results of sensitivity analyses testing the interaction effects of AIA with group dummies (see Table 1). The table lists the tested estimate (see also Table 3), its standard error (SE), the t‑value, and sensitivity measures (Cinelli et al. 2020)
Full size table

These values are challenging to evaluate because the likelihood that such confounders exist cannot be estimated. However, they can be put in context by comparing them with multiples of the variables present in the model. Figure 4 displays the remaining effect size (top row) and the t‑value (bottom row) for all interaction effects (columns), assuming the presence of a confounder ten times as strong as the education variable in the model. This 10× education variable would then explain 13.5% of the residual variance of AfD sympathy and 5.5% of the first interaction effect, reducing it to an effect size of −0.149 (red diamond in the upper left window; see also Table 4) and a t-value of −2.22 (red diamond in the lower left window; see also Table 4). Therefore, such a confounder would not render the effect nonsignificant (α = 0.05) or reduce it to zero. The same applies to the other two interaction effects (middle and right columns in Fig. 4).

Fig. 4
figure 4

Sensitivity plots for the interaction terms between anti-immigration attitudes (AIA) and group dummies (see Table 1). The top row shows the value of the actual coefficient (“Unadjusted” black triangle) and compares it to its hypothetical value (“10×edu” red diamond) if there were an unobserved confounder in the model that was ten times as strongly correlated with the interaction effect and the dependent variable as the education variable. The bottom row shows the corresponding t‑values for the same scenario. Crossing the dashed red lines would indicate a zero effect (top row) and a nonsignificant effect (bottom row)

Full size image

Given the strong negative influence of education on sympathy for the AfD in the models of this study, which is also in line with findings in the related literature, it appears rather unlikely that there is a confounder more than ten times as influential. However, the possibility that such a confounder exists cannot be completely dismissed.

7 Conclusion

In this study, the aim was to reconsider the relationship between AIA and sympathy for RRPs by combining direct and indirect measures. In doing so, the present study attempted to fill a gap in previous research. Despite the ability of implicit attitudes to circumvent cognitive biases in explicit attitudes, researchers have only used implicit attitudes at one end of their models (Bos et al. 2018; Maier et al. 2022). The research design of this study included them at both ends: AIA as the independent variable and sympathy for RRPs as the dependent variable are measured as explicit and implicit attitudes. This allows assessment of the strength of the relationship depending on the type of measurement used to collect these variables. The study adhered to open science principles, with the entire project being preregistered, including hypotheses, survey details, and analyses, prior to data collection.

To assess implicit attitudes, the study employed two SC-IATs, one with primes for the AfD, the most relevant RRP party in Germany, and another with synonyms for migrants. These tests were complemented by an explicit survey on the same constructs. In order to enhance the generalisability of the findings compared to using convenience samples (Gawronski and Strack 2004; Friese et al. 2007), a stratified sample based on age and gender was collected.

First, and consistent with previous research, the results provide clear evidence that implicit political attitudes can influence explicitly reported attitudes (Friese et al. 2007; Bos et al. 2018). Regression analyses reveal significant effects for implicit AIA in predicting explicit AfD sympathy, as well as for explicit AIA predicting implicit AfD sympathy. Further findings align with previous studies by indicating that partially implicit models (using one implicit and one explicit variable) yield significantly lower effect sizes than the fully explicit model (Tutić and Grehl 2021; Maier et al. 2022). The conclusion drawn from these insights is that social desirability may actually be less of an issue in distorting the relationship, as measures that avoid this issue tend to result in lower effect sizes than those that account for it.

Second, going beyond previous research, the relationship between these variables was examined in an all-implicit model. It was found that implicit AIA had no impact on implicit AfD sympathy. This outcome supports the expectation of a cognitive dissonance bias in the relationship between these two variables. When giving explicit answers, respondents may match answers (high AIA, high AfD sympathy, and vice versa) in order to project a highly consistent political ideology. However, the findings indicate that their implicit beliefs are less coherent. Consequently, the results suggest that the avoidance of cognitive dissonance may threaten valid results in attitudinal political science research, in the sense that attitudes that appear to be “good matches” may be artificially overreported by participants in such surveys.

Robustness checks provide evidence that these results are resilient to changes in covariates, and sensitivity analyses suggest that unmeasured confounders are unlikely to overturn them. Nevertheless, it is important to stress the limitations of this study. First, although this study achieved a stratified sample by matching age and sex to population statistics, and the sample also approximates the general distributions of education and employment status, it does not claim to use a representative sample of the German population. Therefore, this study underpins the necessity already described by other scholars to include implicit attitudes in permanent, representative panels (Tutić and Grehl 2021). Second, the selection of primes for this study’s SC-IATs was based on sound theoretical considerations, deliberately limiting them to “migration,” as such avoiding any direct association with Islam. However, public images of terms such as “refugee” and “asylum seeker” are likely to be associated with the Islamic religion. This is difficult to disentangle in designs like the one in this study, and results may therefore be influenced by this bias. Nevertheless, the use of such formally neutral terms allows for the best approximation of a distinct, religion-neutral, implicit anti-immigration attitude. Third, this study cannot claim to predict vote choice for the AfD or electoral potential. Although there is evidence that the use of a sympathy scale rather than reported voting intentions is better suited to assessing citizens’ attitudes towards parties, there is still a significant gap between attitudes and actions (Fishbein and Ajzen 1975) that deserves further research, especially with regard to implicit attitudes (Greenwald et al. 2009).

Taken together, the results of this study point to the relevance of implicit attitudes in shaping party preferences. Compared to the broader debate on demand-side versus supply-side factors, the field of unconscious explanations for the rise and support of RRPs has received relatively little attention. Scholars have started to develop a research agenda in this regard in recent years (Bos et al. 2018; Tutić and Grehl 2021; Maier et al. 2022), and this study attempted to contribute to this strand. However, there is a need for further exploration of the combined explanatory power of implicit and explicit attitudes, the implications of their differences, and the contextual factors that influence their impact on citizens’ preferences and actions. Future research in this domain is crucial for a comprehensive understanding of people’s support for right-wing populism.