Introduction

Recent public and scholarly concerns about the democratic engagement of youngsters read that they would be less supportive of liberal democracy and less likely to participate politically (e.g. Foa & Mounck, 2016; Print, 2007). Moreover, large differences in democratic engagement exist between youngsters from various societal backgrounds, mirroring social inequalities (Schulz et al., 2018). In response to these concerns many governments intensify their efforts to stimulate democratic engagement and reduce inequalities between advantaged and disadvantaged adolescents. One of the instruments that governments employ is on-site citizenship education (or subsidizing independent organizations that provide on-site citizenship education). On-site citizenship programs assume many forms, but usually include a visit to a government institution, interaction with government officials and/or a reenactment of government institutions.

The aim of on-site citizenship education is twofold. First, it is supposed to increase the level of democratic engagement. I define democratic engagement broadly, encompassing political knowledge, political interest, political efficacy, political participation and support for representative democracy. Second, on-site citizenship education is assumed to decrease differences between advantaged and disadvantaged adolescents.

However, it remains unclear whether on-site citizenship education actually boosts democratic engagement, as is assumed by providers of on-site citizenship programs. The citizenship education literature has predominantly focused on the effects of formal instruction, classroom climate and voluntary activities such as participation in debate clubs (for reviews see Campbell, 2019; Geboers et al., 2013, for studies that compare the different types of citizenship education see Dassonneville et al., 2012; Hoskins et al., 2017; Neundorf et al., 2016). Compulsory on-site citizenship programs, by contrast, received marginal attention. There are a few studies that examine the effect of specific on-site citizenship education activities, such as visits to local government offices, prisons, police stations or hospitals (Finkel & Ernst, 2005) or visits to parliament or city hall (Dassonneville et al., 2012), but the effects of other types of activities, such as interacting with government officials, reenacting government institutions, visiting an interactive democracy museum, or participating in quizzes and games, have not yet been studied.

Despite this lack of scholarly attention, on-site citizenship programs might be an effective way of citizenship education, because they combine experience-based education with entertainment (Rapeepisarn et al., 2006). Moreover, unlike voluntary citizenship activities (Hoskins & Janmaat, 2019), on-site citizenship programs are not troubled by selection effects within schools: not only those students that are already highly engaged participate in the program. The first question of this paper therefore is: To what extent is on-site citizenship education successful in stimulating democratic engagement?

Second, it is not clear how on-site citizenship education affects pre-existing inequalities based on gender, educational level, socioeconomic status or migration background. Youngsters from disadvantaged backgrounds may particularly benefit from on-site citizenship education, because it may be their primary source of getting into contact with the political arena. By contrast, adolescents with an advantaged background already have more options to become democratically engaged at home. This way, on-site citizenship education may compensate differences in democratic engagement. However, there are also theoretical reasons to believe that they actually reproduce or even accelerate pre-existing inequalities, for instance when on-site citizenship programs are better suited to politically sophisticated adolescents. The second question therefore reads: do on-site citizenship programs compensate, reproduce or accelerate pre-existing inequalities in democratic engagement between advantaged and disadvantaged adolescents?

Instead of a cross-sectional or longitudinal research design, I employ a quasi-experimental panel design. In order to ensure that potential effects are truly attributable to the on-site citizenship program, I take great effort in ruling out other potential explanations. First, I minimize selection effects at the school, class and student level. Since it is likely that schools that choose to include on-site citizenship programs in their curriculum attach more importance to citizenship education and therefore already differ from schools that do not choose to do so, I have control groups within the same school. I compare students in classes that participate in the main on-site citizenship program of the Netherlands early in the school year (treatment group) to students in classes that did not (yet) participate in the program but would do so later in the school year (control group) by means of Difference-In-Difference (DiD) estimation. Second, I test the robustness of effects by using a Solomon four group design, which accounts for sensitizing by the pre-test and time period (Solomon, 1949). Third, since I carried out four waves among the same students in the school year 2017–2018, it is possible to assess whether the potential effects of participation in on-site citizenship programs last over time. In total, this study has 2340 observations, nested in 585 students, in thirty classes and in three schools.

Theory

On-Site Citizenship Education

Citizenship education refers to efforts aimed at developing attitudes and behaviors that contribute to active, informed and engaged participation in a pluralistic democracy (Galston, 2001; Lawy & Biesta, 2006). It can be divided into four categories: formal instruction, open classroom climate, voluntary activities, and compulsory activities (e.g. Campbell, 2019; Dassonneville et al., 2012; Geboers et al., 2013). Formal instruction refers to citizenship related courses within the classroom context, such as civics or social studies. The open classroom climate refers to the atmosphere in and organization of the classroom, exemplified by active participation, student-driven learning, and interaction. Voluntary activities encompass student council membership, mock elections, or participation in a debating team. Voluntary activities are subjected to selection effects: particularly youngsters that are already politically engaged will participate in voluntary activities (Glanville, 1999; Hoskins & Janmaat, 2019).

On-site citizenship education falls within the fourth category, i.e. compulsory activities that usually take place outside the classroom context. In most Western European countries, governments provide on-site citizenship education (e.g. the UK Parliament Education and Engagement Service in the United Kingdom) or subsidize independent organizations to offer on-site citizenship programs (e.g. the organization ProDemos—House of Democracy and Rule of Law in the Netherlands). On-site citizenship programs usually include a visit to a government institution (such as the House of Representatives, country hall, court, prison, police station), interaction with government officials (e.g. Members of Parliament, city councilors, judges, police officers) or the reenactment of government institutions (e.g. debating and voting on a law, reenacting a lawsuit). It may also cover visiting an (interactive) democracy museum, or participating in informative quizzes and games about democracy and politics.

The aim of (on-site) citizenship education is twofold. First, it aims to enhance democratic engagement, that is, to transfer knowledge about democracy and its institutions, raise political interest and efficacy, stimulate political participation, and increase democratic values (e.g. Hoge, 2002; Langton & Jennings, 1968). Second, it aims to assist particularly adolescents from politically disadvantaged backgrounds to be democratically involved at the same level as those from politically advantaged backgrounds (e.g. Campbell, 2008; Neundorf et al., 2016).

However, it remains unclear whether on-site citizenship education actually contributes to stimulating democratic engagement and compensates for inequalities, as is assumed by both governments and civic education organizations. Remarkably, on-site citizenship education has remained understudied empirically.

Boosting Democratic Engagement

On-site citizenship programs, such as the ones mentioned above, generally aim to influence democratic engagement rather than civic engagement (such as volunteering, see Henderson et al., 2007) or political engagement (such as voting, see Keating & Janmaat, 2016). Yet, there is no consensus in the literature on how to define or measure democratic engagement among adults (Ekman & Amnå, 2012; Keating & Janmaat, 2016), let alone among youngsters. Given that they are not yet in the position to formally participate in the democratic process due to their lack of voting rights,Footnote 1 many items that are regularly asked among adults are irrelevant for youngsters (Deimel et al., 2020).

The categorization of Ekman and Amnå (2012) is useful, as it does not only consider manifest political behavior, but also more latent, pre-political forms of engagement. This is a fruitful strategy, especially for research among youngsters, since there are many attitudes that are of great importance for future democratic engagement. Therefore, I define democratic engagement broadly, encompassing political knowledge, political interest, (internal and external) political efficacy and (intended) political participation (anticipated voting intention). Since on-site citizenship education defines the stimulation of democratic engagement as a core aim, I also include support for representative democracy.

There are multiple reasons to believe that on-site citizenship education effectively contributes to democratic engagement. First, the organizers of on-site citizenship programs design the content of the program in such a way that it boosts democratic engagement.Footnote 2 Second, the specific set-up of the programs is conducive to effective learning. On-site citizenship education is based on the idea of experience learning: seeing and experiencing the political arena in real life renders politics less abstract and is assumed to leave a longer lasting impression than reading from a text book (Dassonneville et al., 2012). On-site citizenship education also combines education and entertainment, which is a very effective way of learning (Rapeepisarn et al., 2006). The mere location of on-site citizenship programs outside the school creates an entertaining atmosphere. Third, youngsters generally participate in on-site citizenship education as part of their formal school curriculum. On-site citizenship education is therefore not troubled by selection effects in the way that voluntary activities are.

Yet, the high expectations of the effects of on-site citizenship education on democratic engagement need qualification. First of all, on-site citizenship programs generally only take one day, which is not very long compared to other forms of citizenship education. Their potential to boost democratic engagement is therefore limited. Second, it is likely that the aspects of democratic engagement listed above are not affected to the same extent by on-site citizenship education. Some aspects of democratic engagement, such as political knowledge, are more easily changed than others, such a political values, or participatory intent (Finkel & Ernst, 2005; Prior, 2010; Van Ingen & Van der Meer, 2016). In fact, the aspects of democratic engagement studied in this paper (i.e. political knowledge, political interest, political efficacy, intended political participation, and support for representative democracy) are to some extent causally related (Delli Carpini & Keeter, 1996; Hooghe and Dassonneville 2011; Van Ingen & Van der Meer, 2016). In line with this argument Finkel and Ernst (2005) find that visits to government institutions increase political knowledge, but do not affect political values. Similarly, Dassonneville et al., (2012) find no effect of government visits on political values and behavior, such as political interest, political efficacy, political trust and political participation. On the basis of these findings, I expect that on-site citizenship education affects political knowledge first, political attitudes second, and political behavior last. This causal order is reflected in the first hypothesis, which sorts the aspects of democratic engagement on the basis of expected effect sizes.

Hypothesis 1

On-site citizenship education has a positive effect on the (a) political knowledge, (b) political interest, political efficacy, support for democracy, and (c) voting intention of youngsters.

Reducing Political Inequalities

Youngsters are not blank slates when they participate in on-site citizenship education. The variation in political attitudes and behaviors that is observed among adults is mirrored among youngsters (Schulz et al., 2018). Already at a young age political knowledge, political interest, political efficacy, (intended) political participation, and support for democracy differ by gender, educational level, socioeconomic status, and migration background (e.g. Abendschön & Tausendpfund, 2017; Geboers et al., 2015; Schulz et al., 2018). These differences are particularly problematic when they reflect structural inequalities in society. Especially citizens with lower education, lower socioeconomic status, migration background, and women belong to minoritized groups. In many countries, they gained suffrage later, and are still in an underrepresented position in the political arena today (Bovens and Wille 2011; Schakel & Hakhverdian, 2018).

The literature on the effects of citizenship education finds that well-designed citizenship education in general compensates inequalities (e.g. Campbell, 2008, 2019; Campbell & Niemi, 2016; Deimel et al., 2020; Gainous & Martens, 2012; Hoskins et al., 2017; Langton & Jennings, 1968; Neundorf et al., 2016; Robinson, 2019). Compensation occurs when citizenship education has a greater impact on adolescents from disadvantaged backgrounds than on adolescents from advantaged backgrounds (Campbell, 2008; Neundorf et al., 2016), thereby narrowing the gap between the two groups. The main mechanism underlying compensation is the catch-up effect: adolescents with disadvantaged backgrounds gain more from citizenship education, because they lacked opportunities to develop democratic engagement at home.

Yet, the literature also points towards other ways in which citizenship education might affect pre-existing inequalities. Acceleration takes place when adolescents from advantaged backgrounds benefit more than youngsters from disadvantaged backgrounds from on-site citizenship programs, while reproduction occurs when on-site citizenship education does not have a differential effect on adolescents from advantaged and disadvantaged backgrounds. Support for these differential effects in the literature is mixed. While Neundorf et al., (2016) and Deimel et al., (2020) find no evidence of acceleration, there is at least one study that does show an accelerating pattern: Hooghe and Dassonneville (2011) observe that students that know more about politics learn more from group projects in class. Deimel et al., (2020) also investigate whether reproduction takes place, finding that students’ socioeconomic background does not affect the positive relation between an open classroom climate and intended voting.

On-site citizenship education aims to compensate inequalities in democratic engagement between advantaged and disadvantaged groups by adjusting programs to specific groups, instead of a one-size fits all approach. For example, students with a lower educational level participate in a different program than those with a higher educational level. However, whether on-site citizenship education indeed succeeds in compensating inequalities and preventing acceleration in democratic engagement is an empirical question. All in all, the second hypothesis is:

Hypothesis 2

On-site citizenship education compensates structural inequalities in the democratic engagement of youngsters.

Design and Methods

Case Selection

The Netherlands has assigned the legal task to develop citizenship competences to schools. Yet, the definition of this task is procedural rather than substantive. Schools can define citizenship to match the (pedagogical or religious) character of their school. Consequently, there is a large variety in the ways that schools organize citizenship education. Yet, many secondary schools of different backgrounds participate in the main Dutch on-site citizenship program. This program is provided by ProDemos—House of Democracy and Rule of Law (an organization that is subsidized to a large extent by the Dutch government), with the explicit aim to stimulate democratic engagement and compensate inequalities in democratic engagement. The on-site citizenship program of ProDemos is representative of many other such programs in scope and activities. The program takes one day and adolescents generally participate in the program as part of their formal school curriculum. In 2018, more than 99.000 adolescents participated in the program (ProDemos, 2018). Part of the program is located in and around parliament: here, students visit the House of Representatives and/or Senate, interact with a Member of Parliament (when possible) and do a politics-related digitally-assisted scavenger hunt.Footnote 3 The second main part of the education program takes place in an interactive democracy museum near parliament that is designed specifically for the on-site citizenship program. Here, students participate in interactive instruction (e.g. reenactment of parliament, debate and discussion, quizzes, escape room).

Research Design

In the literature on the effect of citizenship education, most studies are based on observational data (Campbell, 2019). Be it cross-sectional or longitudinal, usually they rely on questions on past exposure to citizenship education, without a pre-test measuring democratic engagement before this exposure (e.g. Dassonneville et al., 2012; Finkel & Ernst, 2005). Moreover, due to the lack of random assignment they have to make strong assumptions about the comparability of students in their study. To enhance my causal claims, I employ a quasi-experimental panel design. I selected all schools that participate in the main Dutch on-site citizenship program with a random selection of classes at the beginning of the school year (treatment group) and with a random selection of classes at the end of the year (control group). To test whether students who took part in the program were affected differently from those that did not (yet) do so, this study relies on Difference-In-Difference (DiD) estimators. This method compares the differences between the pre- and post-test in the treatment group to the differences between the pre- and post-test in the control group.

In order to ensure that potential effects are truly attributable to the on-site citizenship program, I take great effort in ruling out alternative explanations. First, I minimize selection effects at the school, class and student level. Because schools that choose to include on-site citizenship programs in their curriculum likely differ from schools that do not choose to do so, I rely on control groups within the same school. I eliminate selection at the class level by only including schools that participate in on-site citizenship programs with all their classes. Selection effects at the student level are minimized by selecting schools in which participation in the on-site citizenship program is not voluntary, but part of the school’s formal curriculum.

Second, to rule out that any patterns are artefacts of pre-test sensitizing or period effects, I extended the data collection to a Solomon four-group design (Solomon, 1949). A pre-test may influence participants’ sensitivity to the treatment, for example by causing participants to contemplate a specific issue. I therefore add a control group that does not fill in a pre-test. Participants may also be subject to intervening influences at the time of the post-test. For example, if there is an election campaign going on at the time of the posttest, one can wrongly attribute an observed rise in political interest to the intervention. I therefore include a control group that takes part at the time of the post-test only.

Finally, I assess whether the possible effects of participation in on-site citizenship programs last over time by adding an additional wave. In total, I carried out four waves among the same students over the course of the school year 2017–2018. The control groups did not participate in the on-site citizenship program during the time of the study, but only later after T4. The full design is presented in Table 1.Footnote 4

Table 1 Solomon research design
Full size table

Data

In the school year 2017–2018 three schools met the demands that I defined above and were hence selected for this study. In the Netherlands there are three main educational tracks: pre-vocational education (preparing for vocational education), senior general education (preparing for university of applied sciences) and pre-academic education (preparing for university). In this study, two schools offer pre-vocational education and participated with all third grades (equal to ninth grade in the U.S. system; 15 years old). One school combines senior general and pre-academic education and participated with all fourth grades (equal to tenth grade in the U.S. system; 16 years old). The schools are comparable in the sense that they are all relatively large public schools (600–1500 students) based in urban areas.

Within these three schools, I randomly allocated the total amount of thirty classes to the treatment group or one of the control groups, while taking into account the planning of ProDemos.Footnote 5 In total 585 students (x̄T4 = 14.95 years, s.d. = 0.76) received a number of paper-and-pencil surveys during class, ranging from one to four times.Footnote 6 The survey took approximately 30 min. The first wave took place 2 to 31 (x̄ = 7) days before participating in the on-site citizenship program, the second wave 15 to 48 (x̄ = 30) days after participation, the third wave 81 to 157 (x̄ = 129) days after participation and the fourth wave 112 to 249 (x̄ = 173) days after participation. This resulted in 2340 observations, nested in 585 students, in thirty classes and in three schools.

Measurement

Dependent Variables

The choice for the dependent variables is driven by the range of outcomes that most on-site citizenship education programs aim to affect, that is, democratic engagement. As argued in the theory section, I define democratic engagement broadly, encompassing political knowledge, political interest, (internal and external) political efficacy and (intended) political participation (voting intention). Since on-site citizenship education aims to stimulate democratic engagement, and not solely political engagement, I also include support for representative democracy. The descriptive statistics for the dependent variables are presented in Table 2.

Table 2 Descriptive statistics dependent variables
Full size table

Political knowledge was measured by students’ sum score of four multiple-choice questions (designed for this study): “Do you know how many seats there are in the Houses of Parliament?”, “Do you know how many seats there are in the Senate?”, “Do you know who can propose laws?” and “Do you know what is contained in Article 1 of the Constitution?” For each good answer, students could earn 1 point. Since there were 5 good answers,Footnote 7 the maximum number of points is 5. ‘I don’t know’ answers get 0 points.

Political interest was measured using the single item employed by the European Social Survey. Students were asked how interested they were in politics on a scale from 0 (not at all interested) to 4 (very interested).

Political efficacy, the feeling that one can influence politics, has both an internal and external component (Balch, 1974). Internal political efficacy refers to confidence in one’s own competence to understand and effectively participate in politics. It was measured by agreement with two items on a 5-point Likert scale: “I don’t understand much about politics” (reverse coded, so that a higher score refers to a greater feeling of understanding) and “I understand more about politics than most people my age” (Cronbach's α = 0.64).Footnote 8External political efficacy refers to beliefs about the governments’ responsiveness to citizens’ needs and demands. It was measured by means of a three-item scale: “I think politicians don’t care much about what people like me think”, “People like me have no influence on what happens in politics” and “Politicians are not interested in my opinion” (Cronbach's α = 0.76). In all cases items were measured on a 5-point scale and reverse coded, so that a higher score refers to a greater feeling of external political efficacy.

Voting intention was measured by a single item: “Do you intent to vote when you are 18 years old?” (1 = yes, 0 = no). As discussed in the theory section, given that youngsters are not yet allowed to vote, it is not possible to assess their actual voting behavior. Voting intention is a good and frequently used alternative to measure formal political participation among this age group (Deimel et al., 2020; Glasford, 2008).

Support for democracy was measured by three items: “How important is it for you to live in a democracy? (from 0 (not important at all) to 4 (very important)), “I think democracy is the best way to govern a country” (from 0 (totally disagree) to 4 (totally agree)) and “I think that important political decisions should be made by elected politicians ((from 0 (totally disagree) to 4 (totally agree)) (Cronbach's α: 0.69).

Independent Variables

To test the second hypothesis whether on-site citizenship education has any differential effects, I include the main sources of structural inequalities in society. That is, gender (ref: boy), educational level (pre-vocational education—senior general and pre-academic secondary education), socioeconomic status as indicated by the highest education of students’ parents (did not attend education, did not finish primary education, did not finish secondary education, did finish secondary education, did vocational training, did higher professional or university education) and the number of books at home (no books, 1–10, 11–50, 51–100, 101–200, 201–500, more than 500 books),Footnote 9 and migration background (no migration background, first generation migrant, one parent born abroad, both parents born abroad).

Control Variables

As control variables, I include age of the respondent (in years) and dummies for the school they are in.

Results

Overall Effects

The first hypothesis states that on-site citizenship education has a positive effect on democratic engagement. This hypothesis is tested by means of Difference-in-Difference (DiD) estimation, which compares differences in the pre- and post-test of the treatment group to differences in the pre- and post-test of the control group. The results are presented in Table 3.

Table 3 Summary of difference-in-difference estimates for all components of democratic engagement
Full size table

What stands out from Table 3 is the high number of non-effects. The DiD estimators of political interest, internal and external political efficacy, voting intention and support for democracy are not only insignificant, they are also substantially small. This implies that the main Dutch on-site citizenship program does not have an effect on political interest, internal and external political efficacy, voting intention and support for democracy.

However, the DiD estimator of political knowledge is significant, indicating that there are differences in the growth trajectories of the students that did and did not yet participate in the on-site citizenship programs. Therefore, this finding is scrutinized further.

First, Fig. 1 shows that students that participated in the on-site citizenship program (treatment group) experience a strong increase in their political knowledge between time 1 (before participation) and time 2 (after participation) (b = 0.856, p = 0.000), while the political knowledge of students that did not participate in the program (control group) remains the same (b = 0.053, p = 0.367). Students that participated in the on-site citizenship program have a 0.809 higher increase in their political knowledge score compared to students that did not participate. This means that students almost score one point higher on the knowledge scale due to their participation in the on-site citizenship program.

Fig. 1
figure 1

Visual representation of political knowledge among all experimental groups

Full size image

To rule out that this effect is caused by pre-test sensitization, the treatment and control group are compared to a group of students that did participate in the on-site citizenship program, but did not receive a pre-test (pre-test group). The increase in political knowledge is not caused by taking the pre-test, when the difference between the pre-test sensitization control group and treatment group is not significant, but the difference with the ‘regular’ control group is significant. In line with expectation, the results (see Table 4) show no significant difference between participants with and without a pre-test at T2; yet they reveal significant differences between participants without a pre-test and non-participants. This suggests that the increase in political knowledge is not affected by taking the pre-test.

Table 4 Political knowledge—Marginal effects of the treatment group compared to the pre-test sensitization control group
Full size table

In addition, the effect of participation in the on-site citizenship program lasts over time, given that the knowledge score of the treatment group is still higher at time 3 (b = 0.822, p = 0.000) and 4 (b = 0.806, p = 0.000) than before their participation.Footnote 10 However, although the political knowledge of the control group does not increase from T1 to T2 (b = 0.053, p = 0.367), it does increase from T2 to T3 (b = 0.683, p = 0.000) and remains at the same level after that (T3-T4: b = -0.045, p = 0.381). Also, the DiD estimators of T1-T3 (b = -0.077, p = 0.350) and T1-T4 (b = -0.119, p = 0.268) are not significant, implying that there is no difference in the growth trajectories of the treatment and control group between T1 and T3/T4. This suggests that students’ political knowledge increases over the course of a schoolyear anyway, but that students that participate in the on-site citizenship program simply gain political knowledge earlier.

Differential Effects

The second hypothesis is that on-site citizenship education compensates structural inequalities in youngsters’ democratic engagement. The first step is to examine what kind of inequalities in democratic engagement are observable among youngsters. The descriptive statistics of democratic engagement among the various groups are presented in Table 5. Some differences are significant. Girls have less internal political efficacy and are more supportive of democracy than boys. Moreover, higher educated studentsFootnote 11 have more political knowledge, political interest, internal political efficacy and support for democracy than lower educated students. Furthermore, students with higher educated parentsFootnote 12 have more internal political efficacy than those with lower educated parents and students with more books at homeFootnote 13 are more politically interested than those with less books at home. Lastly, those with a migration background have less external political efficacy, and are less likely to vote than those without a migration background.

Table 5 Descriptive statistics at T1 (s.e. between brackets)
Full size table

In a next step, I investigate how various subgroups are affected by the on-site program by means of three-way interactions. Since my primary interest are the outcomes rather than the underlying pathways, the three-way interactions (Table 6) and resulting differential effects (Table 7) are modeled in separate models and without covariates.

Table 6 DID interaction without covariates (s.e. between brackets)
Full size table
Table 7 Summary of the differential effects, based on Tables 5 and 6
Full size table

Again, the large number of non-effects stands out. For all aspects of democratic engagement, there is not a single significant three-way interaction with regard to the number of books at home and migration background. This means that the pre-existing inequalities based on the number of books at home (i.e. political interest) and migration background (i.e. external political efficacy and voting intention) are reproduced by on-site citizenship education. Of course, there is also reproduction in the aspects of democratic engagement in which there were no inequalities beforehand. There are, however, some significant three-way interactions for gender, educational level and parental education.

Gender

There is no differential effect on the aspects of democratic engagement in which there were no gender based inequalities at time 1 (i.e. political knowledge, political interest, internal political efficacyFootnote 14 and voting intention). However, given the significant three-way interactions, the on-site citizenship program does have a differential effect on boys’ and girls’ support for democracy and external political efficacy. The results point to a slight acceleration and creation of inequalities due to the on-site citizenship program. For the exact pattern, I turn to differences in the marginal effects.

First, participation in the on-site citizenship program accelerates differences based on gender with regard to support for democracy (see Fig. 2). In the control group (dy/dx = 0.183, p = 0.080) and the treatment group (dy/dx = 0.176, p = 0.063), there is a marginal significant difference between boys and girls in support for democracy at time 1. At time 2, the difference in the control group disappears (dy/dx = -0.030, p = 0.421), while remaining in the treatment group (dy/dx = 0.286, p = 0.011). This implies that participation in the on-site citizenship program accelerates the difference in support for democracy.

Fig. 2
figure 2

Differential effects of support for democracy based on gender

Full size image

Second, participation in the on-site citizenship program creates differences based on gender with regard to external political efficacy (see Fig. 3). There are no differences in external political efficacy between boys and girls in the control group at time 1 (dy/dx = -0.142, p = 0.165) and time 2 (dy/dx = 0.150, p = 0.192). Also in the treatment group there are no differences between boys and girls before their participation in the on-site citizenship program (dy/dx = 0.005, p = 0.486). However, after their participation, the external political efficacy of girls seems to be marginally higher than that of boys (dy/dx = 0.192, p = 0.085), thereby creating an inequality that was not there before.

Fig. 3
figure 3

Differential effects of external political efficacy based on gender

Full size image

Educational Level

There is no differential effect on the aspects of democratic engagement in which there were no inequalities based on educational level before participation in the on-site citizenship program (i.e. external political efficacy and voting intention). Since participation in the on-site citizenship program has no differential effect on the pre-existing inequalities in political interest and support for democracy with regard to educational level, these inequalities are reproduced. In addition, given the significant three-way interactions, participation in the on-site citizenship program compensates inequalities based on educational level with regard to internal political efficacy, but also accelerates inequalities based on educational level with regard to political knowledge. Again, I turn to the differences in the marginal effects for the exact pattern.

First, in the control group there are no differences with regard to internal political efficacy between higher and lower educated students at time 1 (dy/dx = 0.199, p = 0.138) and time 2 (dy/dx = 0.282, p = 0.072) (see Fig. 4). In the treatment group, higher educated students have more internal political efficacy than lower educated students before they participate in the on-site citizenship program (dy/dx = 0.634, p = 0.000). However, lower educated students show a steeper growth curve, which decreases the difference between lower and higher educated students after participation (dy/dx = 0.349, p = 0.010) and leads to a compensation effect.

Fig. 4
figure 4

Differential effects of internal political efficacy based on educational level

Full size image

With regard to political knowledge, there are no differences between higher and lower educated students in the control group, both at T1 (dy/dx = 0.129, p = 0.291) and T2 (dy/dx = -0.090, p = 0.363) (see Fig. 5). In the treatment group, higher educated students are already more politically knowledgeable than lower educated students before they participate in the on-site citizenship program (dy/dx = 0.851, p = 0.000), but this difference increases after their participation (dy/dx = 1.411, p = 0.000). Higher educated students show a steeper growth curve than lower educated students, suggesting an acceleration effect.

Fig. 5
figure 5

Differential effects of political knowledge based on educational level

Full size image

Parental Education

There is no differential effect on the aspects of democratic engagement in which there were no inequalities based on parents’ educational level at time 1 (i.e. political knowledge, political interest, external political efficacy, voting intention and support for democracy). However, given the significant three-way interaction, the on-site citizenship program does have a differential effect on the internal political efficacy of youngsters with lower and higher educated parents. The results point to an acceleration of inequalities due to the on-site citizenship program. Again, I turn to differences in the marginal effects for the exact pattern.

First, in the control group there are no substantial differences with regard to internal political efficacy between students with higher and lower educated parents at time 1 (dy/dx = 0.280, p = 0.050) and time 2 (dy/dx = -0.126, p = 0.257) (see Fig. 6). In the treatment group, students with higher educated parents have more internal political efficacy than those with lower educated parents before they participate in the on-site citizenship program (dy/dx = 0.559, p = 0.000). Students with higher educated parents also show a steeper growth curve, which increases the difference between students with lower and higher educated parents after participation (dy/dx = 0.632, p = 0.000) and leads to an acceleration effect.

Fig. 6
figure 6

Differential effects of internal political efficacy based on parental education

Full size image

Conclusion and Discussion

While most studies in the citizenship education literature focus on formal instruction, an open classroom climate or voluntary activities (Geboers et al., 2013), this study broadens the literature by focusing on on-site citizenship education. Governments hold high hopes for on-site citizenship education, which they expect to benefit democratic engagement and compensate inequalities between advantaged and disadvantaged adolescents.

To isolate any causal effects I employed a quasi-experimental panel design, comparing students in classes that took part in the main Dutch on-site citizenship program early in the school year to students in classes of the very same school that would take part later that year. The treatment group was compared to a non-treatment group using Difference-In-Difference estimation; the robustness of effects was tested for the risk of sensitizing by the pre-test and period.

The main finding is rather sobering: the main Dutch on-site citizenship education has an at best limited impact on the democratic engagement of youngsters. In isolation, it does not stimulate political interest, internal and external political efficacy, voting intention, or support for democracy. The on-site citizenship program does, however, have a robust positive effect on political knowledge. Students that participate in the program obtain a significantly higher level of political knowledge compared to those that did not yet participate. This knowledge boost lasts over the period of a school year, even though the levels of knowledge of students that did not yet participate in the on-site citizenship program increases as well during the course of the school year. This implies that on-site citizenship education may play a role in (partly) replacing, or at least adding to, the knowledge transfer in the school’s formal curriculum.

The fact that on-site citizenship education affects political knowledge, but fails to impact on political values or behavior fits the broader literature on political socialization. After all, it has been documented that it is easier to affect political knowledge than political attitudes or behaviors, since the latter do not change easily or greatly (e.g. Prior, 2010; Green et al., 2011). Apparently, the short-term and one-off nature of on-site citizenship programs is not enough to affect political values and behavior. Therefore, future research may focus on the effect of on-site citizenship programs when students participate in longer lasting on-site citizenship programs, and on the way the program is embedded in the school curriculum.

With regard to the effect of on-site citizenship education on pre-existing inequalities, the analyses show that there are not many inequalities in the first place. Those that exist are not very large. Moreover, on-site citizenship education generally does not have a differential effect on advantaged and disadvantaged adolescents. This means that the pre-existing inequalities based on the number of books at home (i.e. political interest) and migration background (i.e. external political efficacy and voting intention) are usually reproduced. When I find differential effects, I see evidence for both compensation and acceleration. On-site citizenship education compensates the gap in internal efficacy between higher and lower educated adolescents, as the lower educated show a steeper growth curve than the higher educated. Their increase in efficacy is, however, not paralleled by a similar increase in their political knowledge. Instead, the gap in political knowledge between educational groups broadens, as the higher educated show a steeper growth curve than the lower educated. The latter finding underlines the importance of preparing for the on-site citizenship program and embedding it in the school’s curriculum: those who already know more about politics beforehand, learn more during the on-site citizenship program.

Tentatively, I even identified that citizenship programs may create differences that did not exist beforehand: After participation in the on-site citizenship program, the external political efficacy of girls increased, while it remained the same for boys. This might be due to the fact that the providers of on-site citizenship programs pay particular attention to the progresses that have been made in the political representation of women. As a result, girls may develop a stronger feeling that the political system is responsive to females’ needs and demands.

This study is based on the on-site citizenship program of one particular organization in one particular country. Nevertheless, it is likely that similar results are found in other countries. The on-site citizenship program of ProDemos is quite similar to that of related institutes in other (West) European countries in terms of scope and activities. Yet, future comparative research is needed to determine whether the results depend on the specific content and didactical methods of the program, and the specific context in which they are situated.

Another interesting question is whether these results can be transferred to other age groups. Generally, students that participate in the on-site citizenship program do so as part of the broader citizenship education they receive in school. Since most students in the Netherlands receive citizenship education in 9th (vocational track; 15 years old) or 10th (general or pre-academic track; 16 years old) grade, I focused on students in these years. As a result, I was unable to examine the effectiveness of on-site citizenship education in other age groups that participate in the program. Since on-site citizenship programs are usually adapted to specific age groups, matching the respective developmental phase that the youngsters are in, it is probable that my conclusions also hold for other age groups. However, this remains an empirical question.

Altogether, this study shows that inequalities in democratic engagement are of a structural nature and are not easily remedied by citizenship education. The high expectations for on-site citizenship education need to be tempered, as in most cases they are not able to close the gaps in democratic engagement. Yet, although on-site citizenship programs cannot change political attitudes and behaviors in a single day, it is quite promising that a one-day on-site citizenship program is able to increase political knowledge and compensate for internal political efficacy. Moreover, on-site citizenship education may also be important for other outcomes. For many students, it is their first encounter with the political arena, and quite often, it changes their (narrow) idea of what politics is. In addition, it can be a small, but important experience that may counter—and potentially even challenge—the (negative or apathetic) view some students are socialized with by the family. In that sense on-site citizenship education definitely has added value on top of other forms of citizenship education.