In the current study, we revisit the ‘diploma democracy’ argument that was put forward by several scholars attempting to chart and explain recent trends in levels of political trust in the Netherlands. Scholars endorsing the ‘diploma democracy’ argument stress that politically less sophisticated citizens become increasingly alienated from politics through the global processes that the negative effects hypothesis postulates, and that they display substantially lower levels of political trust when compared to their politically sophisticated counterparts. However, these studies often lack robust statistical methods to control and account for measurement errors and also for possibly different latent factor models. Therefore, we start our analysis with testing across-group and over-time measurement equivalence of the political trust construct among high and low political sophistication groups. Then we run multi-group confirmatory factor analyses and auto-regressive simplex models to obtain and compare mean political trust scores of these two political sophistication groups.
For the first part of the study, we utilize European Social Survey data from 2004 to 2012 for the Netherlands. The data used for the second part of the study is obtained from Panel Component of European Social Survey, a developmental project aiming at facilitating biannual cross-sectional ESS survey and funded by the Netherlands Organization for Scientific Research (NWO). The panel study started with ESS Round-5 in October 2010 with the participation of 1829 respondents recruited through probability sampling of households and representative of the Dutch population over the age 16. After completing the ESS survey, all respondents were asked to take part in the panel study and 1501 of those respondents agreed to take part in the panel study.Footnote 4 The subsequent waves of the study were conducted between May 2011 and January 2013 with 8 month intervals in-between. The last wave of the panel study completed with the participation of 647 respondents. The response rates for the panel study were 0.60 for the initial ESS study and 0.72, 0.70 and 0.86 for the subsequent waves, respectively.
For the current study, we used list-wise deletion, which yielded total samples of 1831, 1841, 1745, 1785, 1812 respondents for five consecutive waves of ESS (2004, 2006, 2008, 2010, 2012), respectively, and a total sample of 543 respondents in the panel study. While the mean age was 49.2, 48.6, 49.0, 50.0, and 51.1, the average years of formal schooling is recorded as 12.3, 13.3, 13.3, 13.4, and 13.6 in the final samples of subsequent waves of ESS, respectively. In the panel study, the final sample had a highly even gender distribution of 273 male (50.3 %) and 270 female (49.7 %) respondents, with a mean age of 51.6 (SD = 16.24) and with an average of 13.9 (SD = 4.07) years of formal schooling.
We measure political sophistication in two categories (1 = high; 2 = low). We assess political sophistication on the basis of educational attainment and level of interest in politicsFootnote 5 (Table 1). Although there is no causal relation, there is a strong correlation between education, political interest and political sophistication (Althaus 2003; Highton 2009; Luskin 1990). The high political sophistication group consists of those respondents holding a university or university of applied sciences degree, who also expressed interest in politics (very interested or quite interested), at the time of the first interview. By the same token, the low political sophistication group is composed of those respondents without a college or vocational degree and/or those expressed little or no interest in politics.
Consequently, in the consecutive waves of the ESS study, approximately 30 % of the respondents are grouped into the high political sophistication group (28.9 % in 2004; 30.6 % in 2006; 33.4 % in 2008; 30.2 % in 2010; 29.6 % in 2012). Similarly, in the panel study, the high political sophistication group consisted of 193 respondents (35.5 %), while 350 of those respondents are categorized into the low political sophistication group (64.5 %).Footnote 6 As Table 2 illustrates, high and low political sophistication groups display different demographic characteristics. The high political sophistication groups contain a significantly higher proportion of males when compared to the low political sophistication groups. Age and average years of formal schooling differences between high and low political sophistication groups were also found to be statistically significant in all datasets we employed in the current study. The mean age of the low political sophistication groups was on average 5 years higher when compared to their high sophistication group counterparts, whereas average years of formal schooling was observed to be higher among the high sophistication groups.
We further test the criterion validity of our political sophistication variable through inspection of its relation to political participation. Politically sophisticated citizens are expected to display higher levels of political involvement compared to their less sophisticated counterparts. In line with the expectations, we observe different levels of political involvement among high and low political sophistication groups. In all datasets, high political sophistication groups were found to display higher levels of engagement in conventional and unconventional political participation compared to their less sophisticated counterparts; they have a higher turn-out rate in the last elections, they more often sign petitions and work in organizations and associations.
To operationalize political trust we follow Dalton’s definition of key elements of representative democracy (Dalton 2004, p. 37), and we measure political trust by expressed levels of trust in three representative institutions, namely; national parliament, politicians, and political parties (0 = no trust at all; 10 = complete trust). We expect these three indicators to load on a single factor as earlier theoretical and empirical research illustrated that trust in representative institutions forms a one-dimensional constructFootnote 7 (Hooghe and Marien 2013; Marien and Hooghe 2011; Newton and Zmerli 2011; Quintelier and Hooghe 2011; Zmerli 2006). Furthermore, we expect that political trust measured in the form of confidence in the political institutions displays a one-dimensional structure regardless of respondents’ level of political sophistication (Hooghe 2011).
Measurement Model and Measurement Invariance Test
The comparison of means of multiple-item constructs across groups and/or across time is problematic unless the measurement invariance of these constructs can be established through statistical testing (Beuckelaer and Swinnen 2011; Davidov and Coromina 2013; Steenkamp and Baumgartner 1998; Van Deth 2009). While various statistical techniques have been developed to test for measurement equivalence such as item response theory and latent class analysis, the most frequently used approach to test measurement equivalence is the multi-group confirmatory factor analysis model (Jöreskog 1971; Bollen 1989; Steenkamp and Baumgartner 1998; for a review of statistical tests for detecting measurement invariance, see also Braun and Johnson 2010).
To establish the extent of measurement invariance through MGCFA models three hierarchical levels of measurement invariance need to be tested, namely; configural invariance, metric invariance and scalar invariance (Fig. 1).
These models are nested, and the test for invariance is conducted by comparing global model fit indices. Configural invariance refers to the model where only the factorial structures are invariant across groups whereas all the parameters for the model are freely estimated in each group. Configural invariance is the least restrictive level of invariance and it constitutes the baseline model for more restrictive tests.
Metric invariance requires factor loadings of indicator variables to be equivalent across groups. If this restriction, applied to the configural model does not significantly deteriorate the overall model fit, then metric invariance holds, and the relationships between latent constructs and other constructs can be meaningfully interpreted across groups. Metric invariance constitutes a more restrictive level of invariance when compared to configural invariance, and it allows for comparisons of regression coefficients across groups. However, a meaningful comparison of latent construct means requires an even more restrictive equivalence, namely scalar invariance. In the scalar invariance model, not only the factor loadings of indicators are restricted to equality but also all the indicator intercepts are required to be invariant across groups. This is called full scalar invariance. When full scalar invariance does not hold, latent means can still be compared if partial scalar invariance can be attained. Partial scalar invariance requires factor loadings and indicator intercepts of at least two indicators of each latent construct to be invariant across groups.
For evaluating the model fit of each invariance model we employ several goodness-of-fit indices, namely; root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR), comparative fit index (CFI), and Tucker-Lewis index (TLI). We consider RMSEA values lower than 0.05 and SRMR values lower than 0.09, together with CFI and TLI values higher than 0.90 as an indication of acceptable fit for the models. In order to evaluate whether the restrictions introduced at each level of invariance test decrease the model fit within acceptable limits, we adopt a bottom-up strategy where we start with the least restrictive model and proceed with more restrictive models. We use Chi square (χ2) test statistics and CFI values for evaluating the decrease in the model fit as we move from the configural to metric and scalar invariance models. We consider Chi square test p-values greater than 0.05, and a change in CFI values less than or equal to 0.01 as an indication that the null hypothesis of invariance should not be rejected (Cheung and Rensvold 2002). Furthermore, we examine the modification indices (MI) and expected parameter change (EPC) for detecting possible model misspecifications.
Multi-Group Latent Mean Comparison
For estimating the mean levels of political trust among high and low political sophistication groups across five time periods (2004, 2006, 2008, 2010, 2012) with ESS data, we utilize single factor MGCFA model. In the panel study, on the other hand, for identifying the mean structure of the latent political trust construct over time for high and low sophistication groups we utilize an autoregressive (simplex) model. Figure 2 illustrates the path diagram of an autoregressive model for a four-wave panel design. In the figure, the observed variables are labelled as Parl
and they represent trust in parliament, trust in politicians and trust in political parties at ith wave of the panel study, respectively. In the model, Parl
are modelled to be indicators of latent political trust construct at time i. The parameters denoted by λij and ɛij refer, respectively, to the factor loading and error term of the jth indicator at time i. To put it differently, while lambda parameters indicate the degree to which each trust indicator represents the underlying political trust construct, error terms illustrate the extent to which the observed variables are affected by random measurement errors.
The model assumes a lag-1 or Markovian process, meaning that the distribution of political trust construct at time t is dependent only on its distribution at time t − 1, and not directly dependent on the earlier distributions. The parameters β21, β32 and β43 can be interpreted as stability coefficients of the political trust construct over time. The variances of political trust construct are denoted by ζ1, ζ2, and ζ3, and they represent the extent that political trust changes over time.
For the current study, we preferred the marker variable method (Little et al. 2006) for estimating latent political trust means for low and high sophistication groups. The latent mean estimates obtained through the marker-variable method preserve the metric of the marker indicator, and hence provide us with a latent mean estimate on the same scale as the marker variable. Since we utilize marker political trust indicators measured on an 11-point scale, the estimates for latent political trust means will also be adopting the same scale.
We start our analysis by testing the measurement equivalence of political trust construct across high and low political sophistication groups between the years 2004 and 2012 by utilizing five rounds of the ESS dataset for the Netherlands. As we mentioned earlier, we employ MGCFA analysis starting with the least restrictive invariance model and gradually moving towards the more strict invariance models. Table 3 summarizes the global fit indices for MGCFA models for different levels of measurement invariance. We do not provide global fit statistics for the configural invariance in the table below, as the model for a single latent construct with three indicators is just identified and fit indices cannot be obtained.
The global model fit indices for the metric invariance model reported in the first row of Table 3 indicates a good fit of the model with RMSEA and SRMR values below 0.05, as well as, CFI and TLI values above 0.90. Inspection of MI and EPC statistics also did not reveal any significant model misspecifications. Based on the model results, we conclude that factor loadings of the political trust indicators are equivalent across high and low political sophistication groups at five time points. Next, we proceed with testing scalar measurement invariance. The model fit indices for scalar invariance model indicate that introducing equality constraints on indicator intercepts resulted in a significant reduction in model fit. As we moved from the metric invariance model to the scalar invariance model, the increase in Chi square was highly significant (Δχ2(18) = 203.849, p < .000). In a similar vein, CFI and TLI values also decreased more than .01 points and the RMSEA value for the invariance model fell below the acceptable limit of 0.05. Thus we reject the full scalar invariance model. In order to assess partial scalar invariance, we released the equality constraints on six indicator intercepts through inspection of MI and EPC values for model misspecifications. Although the increase in Chi square relative to the metric invariance model was still significant (Δχ2(12) = 41.849, p < .000), RMSEA values improved significantly and the decrease in CFI fell within acceptable limits for the partial scalar invariance model. Furthermore, MI and EPC values did not suggest any considerable model misspecifications. Therefore, partial scalar invariance for the high and low political sophistication groups across five time points between 2004 and 2012 is supported. This means that latent political trust means can be meaningfully compared across high and low political sophistication groups at each measurement point.
Having established partial scalar invariance, we proceed with estimating the latent political trust means for high and low political sophistication groups. Table 4 presents political trust mean estimates from the CFA models for each political sophistication group at each measurement point. As mentioned earlier, the latent political trust means estimated by the model adopts the response scale of the indicator variables, and hence they should be interpreted as mean scores of a latent scale ranging from 0 to 10.
Comparison of latent mean estimates for high and low political sophistication groups at each time point reveals that politically more sophisticated citizens consistently display higher levels of political trust than their less sophisticated counterparts at each of the five measurement points (Table 4). More specifically, the model implies that political trust means for high political sophistication groups range between 5.21 and 5.77, whereas latent mean estimates for the low political sophistication group fluctuate between 4.41 and 4.98 levels. However, closer inspection of the latent mean differences between high and low political sophistication groups reveals that differences are indeed minor. Comparison of high and low political sophistication groups on mean trust levels between 2004 and 2012 shows that the differences between group means are on average 0.79 points, and the gap between political trust levels of high and low political sophistication groups never exceeds one point on an eleven point scale.Footnote 8
Furthermore, we compare latent and composite score estimates for the two sophistication groups across five consecutive waves of the ESS. Table 5 reports the mean estimates obtained through the composite score models for the identical groups and time points utilized for estimating latent political trust means. The comparison of composite score means with previously presented latent means (Table 4), suggests that the composite score models yield slightly different results. Composite score models tend to overestimate political trust means in both high and low political sophistication groups, yet to differing extents. While the composite score models estimated political trust means on average 0.05 points higher than CFA models for the low sophistication groups, the overestimation bias was 0.12 points for the high sophistication groups. Consequently, the differences between high and low political sophistication groups tend to be greater when political trust means are obtained by utilizing composite score models.
In the second part of this study, we investigate the mean trust levels among high and low political sophistication groups by utilizing panel data with four waves. As discussed earlier, comparison of latent construct means can only be meaningful if the structure of the latent factor is similar in each group and at each time point of measurement. Therefore, we again start our analysis by testing the measurement equivalence of political trust constructs among high and low sophistication groups across four waves of measurement. To this end, we conduct a multi-group confirmatory factor analysis with eight groups to test for across sophistication groups and for over-time invariance of political trust construct.
Table 6 summarizes measurement invariance test results with four wave panel data.Footnote 9 The model fit indices for the metric invariance model are high, with an RMSEA value of 0.000, as well as CFI and TLI values above 0.99. Hence, metric invariance of the political trust construct is supported. In a similar vein, the full scalar invariance model fits the data reasonably well with an RMSEA value below 0.05 and SRMR value below 0.09. Although the Chi square increase in comparison with the metric invariance model is significant (Δχ2(14) = 28.782, p < 0.000), the increase in CFI value (ΔCFI = 0.004) is well below the cut-off criteria of 0.01. Further inspection of MI and EPC statistics also does not suggest any considerable model misspecifications, and hence the full scalar invariance model was supported in the panel study.
Our findings of the measurement invariance test allow meaningful comparisons of political trust means and regression coefficients between the two sophistication groups across four waves of panel study. Hence, we proceed with examination of the latent political trust means through a two group auto-regressive (simplex) model. Table 7 presents the latent means for political trust among high and low sophistication groups obtained through the auto-regressive model (denoted by κ) as well as means computed through the composite score model (denoted by μ). Inspection of the mean estimates again reveals that politically more sophisticated citizens display slightly higher levels of political trust when compared to less sophisticated citizens. The gap between trust levels of the two sophistication groups is however only minor. The panel study shows that, on average, politically more sophisticated citizens display 0.7 points higher levels of trust on a political trust latent trait measured on an 11-point scale, which amounts to a difference of 7 % points.Footnote 10
Comparison of the latent and composite score mean estimates illustrates that the composite score model yields slightly higher levels of political trust in both sophistication groups at each measurement point. However, in the panel study where full scalar invariance is established the overestimation bias applies to high and low sophistication groups alike. In other words, when full scalar invariance holds across groups and over-time, the differences between mean political trust levels among high and low sophistication groups are observed not to be different than those estimated through composite score model. In the previous analysis of five rounds of ESS data where only partial scalar invariance was present, the bias was found to be different for the high- and low-sophistication groups, and hence the differences between sophistication groups were slightly exaggerated when composite score model was used. This in turn suggests that composite score model yields similar results as the latent means model only when the full scalar invariance condition has met.