The Public Purposes of Private Education: a Civic Outcomes Meta-Analysis

Abstract

Since Plato and Aristotle, political theorists have discussed the important role of education in forming democratic citizens. They disagree, however, over whether public or private schools are more effective at nurturing citizenship. We conduct a statistical meta-analysis to identify the average association between private schooling and measures of four central civic outcomes: political tolerance, political participation, civic knowledge and skills, and voluntarism and social capital. Our search identifies 13,301 initial target studies, ultimately yielding 531 effects from 57 qualified studies drawing from 40 different databases. Using Robust Variance Estimation, we determine that, on average, private schooling boosts any civic outcome by 0.055 standard deviations over public schooling. Religious private schooling, particularly, is strongly associated with positive civic outcomes. The evidence is especially strong that private schooling is correlated with higher levels of political tolerance and political knowledge and skills. We discuss heterogeneities, robustness checks, and implications.

The Public Purposes of Private Education: a Civic Outcomes Meta-Analysis

In May of 2023, the National Assessment Governing Board released the latest civics test scores for US students, based on 2022 testing. The results were disappointing. Only 22% of eighth graders were judged to be proficient in civics. The Chair of the Governing Board, Beverly Perdue, stated: “The students who took these tests are in high school today and will soon enter college and the workforce without the knowledge and skills they need to fully participate in civic life and our democracy” (National Assessment Governing Board, 2023). A recent survey of Americans age 18–24 brought more bad news, as only 4 percent of them answered four standard civics questions correctly and only 48% of them responded that they planned to vote in the 2024 election (Institute for Citizens & Scholars, n.d.). Rajiv Vinnakota, president of the organization that sponsored the poll, declared “We urgently need to do more to civically prepare, activate, and support young adults” (Institute for Citizens & Scholars, n.d.).

The disappointing 2022 NAEP civics results, and the concern they provoked, are neither new nor limited to the USA. The average score of the 300-point US test has been restricted to a narrow range of 150–154 over the past 24 years, though slight increases in 2014 and 2018 were eliminated by the post-pandemic drop (National Center for Education Statistics, n.d.). Hoskins et al. (2008, p. 386), in their study of civic outcomes across Europe, warn that "the need to find possible responses for enhancing Active Citizenship has become increasingly pertinent." Persistently disappointing civics scores in democracies around the world motivated UNESCO to launch a framework for global citizenship education in 2015 (De Vries, 2023). Cohen et al., (2021, p. 229) states, “The worrisome state of American democracy calls for a systematic examination of civic education and civic capacity…. Schools can play an important role in increasing civic engagement.”

Since Aristotle and Plato, political theorists have discussed which type of school is most effective at forming democratic citizens. Contemporary theorists such as Amy Gutmann (1987, pp. 65–70) and Sarah Stitzlein (2023[2017]) argue that government-operated public schools are more effective than private schools at promoting such civic values as political tolerance, political participation, civic knowledge and skills, and voluntarism and social capital (i.e., community engagement). They claim that the very fact that public schools are government controlled and open to all students gives them an inherent advantage over private schools in inculcating the civic outcomes central to citizenship in our constitutional republic (Gutmann, 1987, pp. 65–70, 97). Stitzlein (2023[2017], p. 1) asserts that “The health of our democracy in the United States depends directly on our public schools.”

Supporters of private schooling counter such claims. As voluntaristic community institutions responsive to parents, private schools do as well or better than government-run public schools in promoting civic outcomes (Berkowitz, 1996, p. 447; Brandl, 2010, pp. 31–32). Moreover, they argue, private schooling empowers parents and signals to students the importance of their own agency, rooted in their parents’ value system, a message that is central to raising active and confident citizens capable of self-government (Rowe, 2022, p. 46; Stewart & Wolf, 2014, pp. 111–121). Wolfe (2003, p. 10) observes that “never-ending debates take place in…education because we have always made schools so central to the question of the kind of society to which we aspire.”

This dispute is essentially empirical. However, theorists have seldom subjected their claims to systematic empirical tests, despite the availability of numerous quantitative studies that have investigated school sector effects on a variety of civic values. We provide such a test here.

We conduct a statistical meta-analysis to identify the average association between private schooling and 10 specific measures of four categories of civic outcomes. Meta-analyses are especially valuable when an important scientific or policy controversy persists despite the presence of many individual quantitative studies, drawing from different limited samples, at least some of which reach conflicting conclusions (Ringquist, 2013, pp. 2–4). Such is the case regarding whether public or private schooling is associated with better civic outcomes. Based on a search yielding 13,301 target studies, we identify 531 empirical findings from 57 qualified studies drawing from 40 different databases. Using Robust Variance Estimation (RVE) regression and meta-regression, we determine that the average reported effect of private schooling on any of the civic outcomes is an increase of 0.055 standard deviations that is statistically significant with greater than 99% confidence. We subject our overall finding to a variety of robustness tests, all of which it passes. The evidence is especially strong that private schooling is correlated with higher levels of political tolerance and political knowledge and skills. Religious private schooling is strongly associated with positive civic outcomes. Claims that private schooling imperils democracy are inconsistent with this empirical evidence.

Our paper proceeds as follows. We first define civic values and draw from the psychology and political science literatures to describe the competing theories of supporters of public and private schools regarding their effects on civic outcomes. Second, we describe the empirical methods used for study identification and search, data extraction, coding, analytical procedures, and robustness checks. Third, we discuss results for the overall sample, various subgroups, meta-regression, and checks for publication and reporting bias. Lastly, we discuss the implications of our findings.

Defining Civic Values

Democracies rely on a common set of values to provide social cohesion and enable self-government (Macedo, 2000, p. 151). These civic values convey the responsibilities of citizenship and constitute the foundation of social and political interactions between individuals and organizations. In this meta-analysis, we leverage a large body of research to analytically determine whether public or private schooling provides an advantage in the development of civic values. Scholars have articulated various conceptions of the civic values required to support a democracy (e.g., Coleman & Hoffer, 1987, pp. 212–213; Fleming, 2014, pp. 58–60; Ravitch & Viteritti, 2001, pp. 3–7). Campbell (2008, p. 489) defines four categories of civic education: “community service, civic skills, political knowledge, and political tolerance.” We build on Campbell’s categories in this study, expanding and redefining them to capture a broader set of civic values assessed in the education policy literature, as recommended by Barrett (2018). We also provide motivation for our theoretical expectations regarding public versus private school advantages in civic-value development. We define four categories of civic outcomes: Political Tolerance, Political Participation, Civic Knowledge and Skills, and Voluntarism and Social Capital (i.e., Community Engagement). The remainder of this section outlines each of these categories, defining how each contributes to civic education.

Political Tolerance

Many prior studies on education and civic values examine political tolerance (e.g., Campbell, 2008; Wolf et al., 2001a). This civics category captures an individual’s willingness to respect the rights and opinions of people who are different from them—encompassing political, religious, and racial tolerance (Campbell, 2008, pp. 491–492). Political tolerance typically is measured by first asking a respondent to identify their least-liked political group and then asking them a series of questions regarding what rights they would extend to a member of that disliked group (Sullivan et al., 1982, pp. 60–63). We embrace this definition as our conceptual understanding of political tolerance; however, recent developments in the political and research literature warrant an expansion of the category to encompass a more holistic set of outcomes. We therefore include additional indicators such as attitudes about equality, human dignity, respect for individual worth, personal integrity, and measures of antisemitism in our definition of political tolerance (Greene & Kingsbury, 2017; Hanif et al., 2020; Shafiq & Myers, 2014). We believe these measures capture an expansive yet coherent understanding of political tolerance.

Political Participation

Political participation is a behavioral measure of a person’s willingness to engage in the activities of self-government. Specific indicators of political participation can involve something as basic as voting in the previous election, or more involved political actions such as contacting a political representative or participating in a political rally (Nie & Hillygus, 2001, p. 33). Most data on political participation come from self-reports on surveys and voter registration databases.

Civic Knowledge and Skills

Civic knowledge and skills are a set of understandings and abilities widely viewed as conducive to self-government. Specific indicators include scores on factual quizzes regarding the country’s constitutional system (e.g., Ten Dam et al., 2020), such as its separate branches of government, federalism, and how a bill becomes a law. Other indicators include self-reports regarding a person’s ability to write a persuasive letter or give a public speech. Civic knowledge also includes “understanding of and commitment to democratic norms” such as accepting the results from fair elections even if they do not favor one’s preferred candidate (Viteritti, 2001, p. 181).

Voluntarism and Social Capital

Voluntarism and social capital are measures of one’s involvement in activities that benefit the broader community and the depth of one’s community attachments. As Cobb (1992, p. 7) writes, “We learn about public responsibility by becoming involved in community activities.” Voluntarism typically is measured by asking, “Did you volunteer more than an hour of your time, without pay, in any service activity in the past month?” or “How many hours of volunteer service did you provide in the past year?” Drawing heavily from Bourdieu’s (1987, p. 122) conceptualization, social capital tends to be measured in scales based on indicators of shared social norms, networks, and cooperation. Charitable giving also is included in this category, supported by research indicating that individuals prefer to make financial donations rather than participate in community endeavors outside their home (Dronkers, 2004, p. 294).

Private Schooling and Civic Outcomes in Theory

While civic values can be inculcated in various settings, schools are a cornerstone of civic education in America (Chan et al., 2014). Many education traditionalists contend that common public schools are best situated to create civic-minded citizens (e.g., Cubberley, 1920, pp. 722–723; Gutmann, 1987; Mann, 1848; Stitzlein, 2023[2017], p. 7; Wolfe, 2003). Other scholars, however, argue that private schools are better at cultivating community and modeling the democratic values that create an informed and engaged public (e.g., Cobb, 1992; Cremin, 1980; Glenn, 1988; Moe, 2000). Here, we review the theories of action grounded in the pro-public and the pro-private perspectives regarding the inculcation of civic values and describe why we hypothesize that private schooling will be positively associated with civic outcomes.

The liberal political tradition, from John Locke to contemporary thinkers, has emphasized the importance of training young people to become active and autonomous choosing selves. Free societies will be preserved only if succeeding generations understand their duties of self-government and are equipped to use a sufficient level of sound moral judgment in exercising them (Allen, 2023, p. 207). They should have the freedom and self-confidence to choose the kind of life they wish to live and to resist oppression and excessive conformity. Where liberal political theorists differ is regarding who should direct and control the formation of autonomous, civic-oriented, young adults: the state or parents (e.g., Brighouse, 2000, pp. 32–33; Godwin & Kemerer, 2002, p. 91; Gutmann, 1987, p. 70)? The difference is one of means not ends, control more so than content. Private schooling is central to the dispute.

The Pro-Public School of Thought

Some liberal theorists assert that government control of K-12 schooling helps ensure that it is delivered to students in ways that emphasize the public purposes of education, namely the preparation of democratic citizens (Cubberley, 1920, pp. 722–723; Mann, 1848; Stitzlein, 2023[2017], p. 7). Private schooling is presumed to be privatizing and inordinately focused on developing the practical knowledge and skills of students to prepare them to profit, personally, from their human capital (Abrams, 2023, p. 88; Gutmann, 1987; Wolfe, 2003, p. 6; Stitzlein, 2023[2017], p. 13). Gutmann (1987, p. 70), for example, states “…public, not private, schooling is … the primary means by which citizens can morally educate future citizens.” Public control equates to the fostering of public aims while private control equates to the fostering of private aims, they argue.

Second, as public institutions dedicated to public ends, public schools are theorized to model the operation of democratic institutions and processes better than private schools (Stitzlein, 2023[2017], p. 15). School boards govern public schools, whose members the people freely elect, and operate according to procedures that encourage public input (Dewey, 1916, p. 65). In contrast, private organizations and secretive procedures govern private schools, which do not model democratic norms to students or parents, it is claimed.

Third, private schooling is theorized to decrease civic outcomes by allowing parents and students easy “exit” options from public schools. If parents cannot easily remove their child from a public school, they have a stronger incentive to exercise “voice” in ways that generate constructive changes in public schools (Hirschman, 1970, pp. 109–110). If a parent does not like something about their child’s public school, they can and should change it, public school advocates say.

Fourth, public schools traditionally have been viewed as “common” schools that serve as a melting pot to foster shared public values among students from families with diverse national, religious, and ideological commitments (Levinson & Levinson, 2003, p. 104; Mann, 1848; Torney-Purta et al., 2007). While the “melting pot” function of public schooling was considered especially vital during the nineteenth and early twentieth centuries, at the peak of immigration to the USA (e.g., Cubberley, 1909, p. 15), the USA and many other countries around the globe continue to lean on public schooling to instill a unified set of cultural norms in citizens (Cremin, 1980, p. 181; Wolfe, 2003, p. 4). Abrams (2023, p. 91) claims that private school choice theorists such as John Stuart Mill and Milton Friedman “did not foresee significant problems with government funding of privately run schools, with segregation, moral hazard, and erosion of civic engagement chief among them.” Although the unifying power of public schools may have been over-sold and the dividing power of private schools similarly exaggerated (Callan, 1997, pp. 163–166), nevertheless, Callan (1997, p. 178) states “there is a presumptive case for common education in common schools.”

Finally, public schools are assumed to be more effective at teaching civics because they have greater potential to draw upon a consistent set of curricular materials taught by certified teachers, though they may not always avail themselves of that opportunity (Hirsch, 2009). Torney-Purta (2002) stresses that civic content taught in public schools is a significant factor in promoting student civic outcomes. Public schools can promote civic values, but do they outperform private schools in that vital responsibility?

The Pro-Private School of Thought

Defenders of the positive effects of private schooling on civic outcomes argue instead that public schools tend to be products of the democratic political system and not necessarily pillars of it (Moe, 2000, pp. 127–145). Public control does not prevent rent-seeking behavior by actors in the public school system to advance their own private interests (Cremin, 1980, pp. 174–177; Cobb, 1992, p. 210; Moe, 2011, pp. 1–25). Extended school closures during the pandemic are a recent example of public school personnel placing their own self-interest above the public interest (Dasgupta, 2022, pp. 19–20). Private schools are more responsive to parent preferences (Cobb, 1992, p. 226). A prominent survey indicates that private school parents, like their public school counterparts, demand that schools prepare their children for democratic citizenship (e.g., Zeehandelaar & Winkler, 2013, pp. 6, 32). Since private school parents, as customers, are expected to have greater leverage over their child’s school than are public school parents (e.g., Chubb & Moe, 1990, pp. 30–35; Stewart & Wolf, 2014, pp. 123–124), private schools will outperform public schools in promoting civic outcomes, these scholars predict.

In practice, public school governance is not necessarily the attractive model of democratic processes and values that many people imagine it to be (Meyer, 2004, pp. 34–37). For example, the race profiles of voters in school board elections differ significantly from the demographics of the students who attend the schools governed by those boards (Kogan, Lavertu & Peskowitz, 2021, p. 1083). Top-down public-school governance necessitates winners and losers among parental preferences for instructional material, pedagogical approaches, and extra-curricular offerings (Cobb, 1992, p. 3). Terry Moe (2000, p. 130) argues, “It is the [electoral] winners who will control the schools, and the winners’ preferences that will set educational policy and structure.” Under this vision, promoting civic values demands more than our current educational “winner-take-all” politics because minority rights and tolerance of divergent views are core democratic values. As Cobb (1992, p. 3) writes, “Government schools have become the antithesis of democracy because they represent mass society instead of local community.” This consideration crystallized in the USA in 2021 when the National School Boards Association was forced to apologize for asking the Department of Justice to investigate parent protestors as “domestic terrorists” (Rokita et al., 2021).

Restricting public school “exit” options may have a negative, and not a positive, effect on parental “voice” in education (Barry, 1974, pp. 86–93). As Hirschmann (1986, p. 89) himself later admitted, parental voice has no force if there is no alternative to the public school system. It may be precisely when students are a legitimate flight risk, due to the availability of options such as private schooling, that public school officials tend to be most attentive to the voices of parents. Moreover, the very act of choosing a private school for your child, and the obligations that come with that privilege, have the potential to empower traditionally disenfranchised populations of parents (Stewart & Wolf, 2014, pp. 111–121).

The image of public schools in the USA as benevolent, inclusive, “common” schools has, in actuality, been a myth (Cremin, 1980, pp. 166–168; Glenn, 1988; Godwin & Kemerer, 2002, p. 66). Access to public school districts and individual public schools is determined by neighborhood enrollment bodies, a form of educational “redlining” in the eyes of some observers (DeRoche, 2020). Public schools have largely served the interests of powerful elites (Brighouse, 2000, pp. 32–33) and sought to impose a particular set of political, ideological, and religious values on disadvantaged minority groups (Macedo, 2000, pp. 110–130). Private schooling, in contrast, is highly pluralistic, allowing parents to choose the specific conception of the good life to be inculcated in their child. As Berner (2017, p. 5) states, “because education entails moral commitment, it should honor the beliefs of the nation’s families.” The pluralistic nature of private schooling may make it an environment more conducive to instilling civic values in students (Godwin & Kemerer, 2002, p. 97).

Private schools could be more effective than public schools at teaching civics. Zabolotna & Pidhaietska (2021) claim that the culture and values in a child’s environment are more important in shaping civic engagement than their school’s social science curriculum. Private schools tend to be “strong-culture” educational organizations and strong-culture schools tend to be better than weak-culture schools at instilling values in students (Akerlof & Kranton, 2002, p. 1170; Coleman & Hoffer, 1987). Van Goethem et al. (2014) argue that community service and reflection practices common in private schools likely mediate the enhancement of civic dispositions in students, perhaps due to their fostering of a “we-mentality” (Huening, 2022; see also Albanesi et al., 2007; Flanagan et al., 2007). The development of interpersonal trust is vital to subsequent civic engagement (Sullivan & Transue, 1999) and value-rich private schools might better foster interpersonal trust than supposedly value-neutral public schools. Other scholars (e.g., Liem & Chua, 2013) claim that values beliefs associated with civic outcomes are more powerful forces than expectancy of achievement in motivating young people to realize key civic outcomes. In related work, Youniss & Yates (1999) argue that the very act of volunteering fosters a mature moral and civic identity that persists into adulthood, an important early finding subsequently replicated by Lakin & Mahoney (2006) and Zarrett et al. (2021). If values and action trump curricular content, and private schools are more value-rich educational environments that induce civic activity, then we might expect private schools to outperform public schools in inculcating civic values.

Because many contemporary political topics are controversial, public schools might shy away from considering them, denying their students the experience of debating contested issues (Higgins & Abowitz, 2011, p. 378). Berner (2017, p. 20) questions the supposed public-school advantage in encouraging the open-mindedness central to citizenship preparation: “Indeed the uniform public school system was designed precisely to limit conflict between competing ideals”. In contrast, many private schools encourage students to confront controversial, value-laden issues (Berner, 2017, p. 22). Even Amy Gutmann (1987, p. 65), a liberal theorist asserting strong normative claims in opposition to private schooling, admits that private schools likely are more effective at teaching civics because public schools “have instituted some of the most intellectually deadening methods of teaching American history and civics that one might imagine”.

Both pro-public school and pro-private school theorists and advocates articulate coherent theories regarding the superior performance of their preferred school type in promoting civic outcomes. Still, the claims in favor of private schooling enhancing civic values strike us as more persuasive. Thus, our primary research hypotheses for this empirical meta-analysis will test the proposition that private schooling is positively associated with civic outcomes.

Characteristics of private schools and the students who attend them might mediate or moderate the effects of private schooling on civic outcomes. Our specific hypotheses regarding mediation and moderation are that:

• Religious private schools will have civic effects distinct from those of secular private schools (Larson et al., 2006; Macedo, 2000, p. 130; Vieno et al., 2007).

• Private schooling will demonstrate larger effects on civic outcomes in developing countries compared to developed ones, an outcome observed in a prior meta-analysis of the achievement effects of private school vouchers (Shakeel et al., 2021, p. 531), possibly because both public and private schools are less institutionalized in developing countries (Card, 2001, p. 1127).

• The civic effects of private schooling will be smaller in higher-quality studies which more completely control for selection bias (Figlio & Stone, 1997; Showalter & Mullet, 2017, pp. 221–227) than in lower quality studies which do not, since students from advantaged backgrounds may be attracted to private schools specifically due to the better civics instruction those schools deliver, a selection issue that we discuss further in the “Method” section.

• Those who receive a higher “dose” of private schooling through experiencing a larger portion of their K-12 experience in the private sector will evidence larger civic effects of private schooling than those who receive a smaller dose (Schochet, 2013, p. 325).

• Private schooling received in the middle and high school grades, when civics is an element of the curriculum (e.g., Adeyemi, 2017; Cook et al., 2020), will be associated with larger civic effects than private schooling received in the elementary grades, when it is not.

• Older studies of private schooling and civic outcomes will report larger civic effects than newer studies due to the forces of institutional isomorphism that have driven schools in the private and public sectors to more closely resemble each other (Burke, 2016, pp. 62–72).

• Private schooling received as part of a private school choice program will be associated with larger civic effects over the course of the study than private schooling received without the use of a private school choice program, since switching from a public to a private school likely presents a sharper contrast in schooling environments than does being educated in a specific sector one’s entire life (Howell et al., 2002, pp. 90–113).

We explore these potential mediators and moderators of private schooling and civic values as a supplement to our primary hypothesis tests.

Method

We employ RVE based on hierarchical effects with small sample corrections (Tipton, 2015) to estimate the overall and subgroup meta-analytic effects. In addition, we conduct RVE meta-regression to identify study moderators. Checks for publication and reporting bias are carried out through Egger’s test (Egger et al., 1997), fail-safe N calculations (Orwin, 1983; Rosenberg, 2005), and nonparametric trim-and-fill analysis of the full sample (Duval & Tweedie, 2000a, 2000b).

Identification and Selection of Studies

We pre-registered our study at the Open Science Framework on February 14, 2022 (Shakeel et al., 2024). Our pre-registration specified our confirmatory hypotheses and defined the variables, databases, citation and experts search, inclusion and exclusion criteria, and empirical strategy.

Inclusion and Exclusion Criteria

Our meta-analysis includes all relevant English-language quantitative studies relating to private and government-run public schooling and their effects on generating political tolerance, political participation, civic knowledge and skills, and voluntarism and social capital in parents and students. To ensure that the meta-analysis is comprehensive, up-to-date, and informed by as much evidence as possible, we do not restrict the study sample based on timing, geography, publication type, study quality, and research methods. Selection bias is a major concern when comparing outcomes from private and public schools (e.g., Figlio & Stone, 1997), so much so that at least one meta-analysis we know of that examines private school voucher effects limits its sample to 21 random assignment experiments (Shakeel et al., 2021). Only seven of the 40 databases that inform our meta-analysis were generated using either experimental or quasi-experimental designs. Thus, were we to exclude observational studies from our sample, our meta-analysis would be underpowered and lack external validity. To test whether the internal validity of our study is threatened by the inclusion of observational studies, we code for variables such as research and analytical methodology, data structure and scope, self-selection, and study quality that are typically associated with selection bias. We present results across these subgroups, as well as include them in a meta-regression framework to understand their role in explaining variation in our findings.

Studies were excluded if they did not clearly describe their comparison group as government-run public schooling. This comparison group excludes public–private partnerships such as charter schools in the USA and academies in the UK, homeschools, hybrid-homeschools, and virtual schools. Studies were also excluded if they only examined outcomes outside the scope of our study such as non-cognitive skills, socio-emotional learning, and criminal behavior.

Search Strategy

Our systematic search was conducted on four databases: ProQuest Central, ERIC, Academic Search Complete, and PsycINFO. We used the following keywords in our searches:

[private school* OR parochial school* OR catholic school* OR independent school* OR Christian school* OR school choice OR school voucher* OR education* voucher* OR education* choice] AND [civic* OR democracy OR vote* OR voting OR voter registration OR volunteer* OR tolerance OR political knowledge OR political participation OR political skills OR social capital OR civic engagement OR civic knowledge OR citizenship education OR citizen participation OR democratic knowledge OR civic values].

On ProQuest Central, we accepted the following document types: books, conference papers, dissertations, government publications, reports, and scholarly journals. On ERIC, Academic Search Complete, and PsycINFO, we accepted all document types that each database indexes. Parallel to the databases search, we conducted a citation and expert search. We carried out a citation search from the retrieved studies, which included the latest review we knew of on private schooling and civic outcomes (Wolf, 2020). We consulted a robust set of US and international scholars in the field for suggestions regarding additional studies. The set of scholars included David Campbell, Jan DeGroof, Charles Glenn, Ignasi Grau, Karthik Muralidharan, and David Sikkink.

Search Results

The databases search produced 13,248 target articles. Thereafter, we identified and removed duplicate studies. First, we eliminated studies with clearly irrelevant titles using the third-party software Rayyan. Second, two researchers reviewed each abstract for eligibility for inclusion, blind to the other researcher’s determination. If the initial reviewers disagreed, a third researcher broke the tie. Each researcher documented their justification for excluding an article (Table S1, online only). Our intercoder agreement Kappa was 0.47 for the abstract review stage, prior to resolution by the third coder (Hallgren, 2012) (Table S2, online only). Third, we conducted a full study review of each target that survived the abstract review, using the same approach of two independent reviewers and a third to break ties. In our full-study review stage, our intercoder agreement Kappa was 0.77. In parallel to the 13,248 target articles produced by the databases search, two team members assessed the list of 53 studies obtained through the citation (34) and experts (19) search. Seventeen studies were judged to be ineligible. From the remaining 36 eligible studies, 16 overlapped with the studies obtained from the databases search. The review of targets generated by the citation and experts search produced a perfect intercoder agreement Kappa of 1. The final sample of 57 studies includes 21 studies uniquely identified through the databases search + 16 jointly identified studies + 12 studies uniquely generated by the citation search + 8 studies uniquely generated by experts. Figure 1 describes the PRISMA flow diagram (Page et al., 2021) of the stages of identification, screening, and inclusion of studies in the meta-analysis.

Fig. 1

Note. Sourced from Page et al. (2021). Databases search conducted on February 17, 2022; other methods search conducted from February 27 to May 31 of 2022

PRISMA flow diagram for stages of study search. 

Coding of Studies

We used a predesigned spreadsheet in Microsoft Excel to code study details. Each of the five coders for this phase was randomly assigned to 34 or 35 studies from which to extract data. Each coder served as the primary or secondary coder for 22 or 23 studies and an adjudicator for 11 or 12 other studies. Thus, there were three coders assigned to each study, two to conduct independent data extractions and one to adjudicate. The two independent coders first compared their coding sheets, then reconciled any differences, with the adjudicator resolving any disputes that the coders did not resolve themselves (see online Appendix for the data extraction guide).

The coded details included the name of the study’s database, study years, authors, location, publication type, sample, type of civic outcome, research design, analytic method, treatment, data structure, geographical scope, whether effects were explicitly reported or had to be calculated, effect year, demographics, grades, dosage, study quality, school religious identity, details of civic outcome measure, any study notes, the type of effect size, mean scores, differences in means, and standard deviation (treatment and control), standard error, t statistics, p values, and 95% confidence intervals. The index for study quality was constructed from Conn (2017, pp. 867–870) and derived from the Cochrane risk of bias framework (Higgins et al., 2011) and the GRADE system for method quality (Higgins & Green, 2011). Each study’s score on the index fell between a range of 1 to 6 across the domains of study quality: clarity of intervention or experiment description, presentation, balance (treatment and control) for experiments, overlap for matching design, attrition description, and quality of one and two-stage least squares for instrumental variables (see Conn, 2017, pp. 867–870 for more details). We took an average of scores across domains and standardized the index. This vast amount of data allowed us to build upon our pre-registration plan without compromising our basic pre-registered hypotheses. Our findings are robust to reversing the few study enhancements we implemented post-registration. For example, we preferred unbiased Hedge’s g over biased Cohen’s d, and carried out nonparametric trim-and-fill analysis of the full sample in addition to Egger’s test and fail-safe N calculations (see online Appendix for the list of deviations from pre-registration). Our coding resulted in 531 effect sizes.

Studies Included in Meta-Analysis

Our search identified 57 studies which represent 40 unique databases (Table 1). The oldest data in the study were drawn from 1982 (Coleman & Hoffer, 1987; Willms, 1985). The most recent data in the sample were from 2020 (Hanif et al., 2020). Twenty-one studies were peer-reviewed journal articles. Study locations included Australia (1), Canada (1), Chile (1), Chile, Colombia, and Mexico (1), Italy (1), most European countries (1), Pakistan (3), Sweden (1), the USA (46), and the UK (2). Thirty-one studies had nationally representative samples.

Table 1 Description of database and studies included in the meta-analysis

Table S3 (online only) shows that studies varied in their research designs and analytical strategy. Most studies employed observational designs while some employed experimental and quasi-experimental designs. Studies used continuous regression, probit, logit, and ordinal regression estimation techniques. Five studies examined the effects of receiving an offer of a private school choice scholarship or voucher, whereas the others examined the effects of self-selected private school attendance. Twelve studies used panel data while the rest used (pooled) cross-sectional data. The grade ranges of study participants varied vastly, and many studies included some or all high school years. Study samples consisted of students only (42), parents only (8), and the combination of parents and students (7).

Twenty-four studies were sufficiently detailed so that coders were able to copy and paste all effect sizes and standard errors directly into the spreadsheet, whereas three studies contained sufficient details to do that for some effects, and 30 studies lacked one or more details necessary to readily copy and paste the estimates into our coding sheet. We were able to make reasonable assumptions necessary to calculate the effect size and standard error from the information in the studies that did not provide those details explicitly (see study notes in online Appendix).

The private school sectors represented by the studies varied in religious identity (Catholic, Protestant, other religious, and secular). Islamic schools, called madrasas, were included in the “other religious” category (e.g., Hanif et al., 2020). Some studies examined the civic effects of Catholic, Protestant, and secular private schools compared to public schools (e.g., Hill & Den Dulk, 2013). Seventeen studies provided no information on the religious identity of private schools.

The number of effects coded for each of the 10 specific civic outcomes varied. We coded 81 effects on political tolerance, 87 effects on voting, 30 effects on voter registration, 14 effects on political participation, 21 effects on civic engagement, 27 effects on civic knowledge and skills, 20 effects on political knowledge, 10 effects on political skills, 210 effects on voluntarism, and 31 effects on social capital (Table S4, online only). The operational definitions of each civic outcome varied across some of the studies (Tables S5, S6, S7, S8, S9, S10, S11, S12, S13, S14, online only).

Background and Moderator Variables

We coded 12 variables that could suggest effect heterogeneity: sample (parent vs. student), civic outcome (voluntarism vs. any other type), research design (experimental/quasi-experimental, observational with control variables, observational without control variables), analytical strategy (mean/proportional difference, binary/ordinal regression, continuous regression), private schooling (choice vs. self-selected), data structure (panel vs. (pooled) cross-sectional), representative (national vs. regional, state, and local), scope (international vs. USA), reporting bias (effects calculated vs. reported explicitly in a study), timing (effect year ≥ 1998 vs. prior to 1998), publication (peer reviewed journal article vs. other categories), and study quality.

These 12 variables are likely to moderate the civic effects of private schooling. For example, private schooling may affect students differently from their parents. Voluntarism is the most common civic outcome in our database, as it is the subject of 210 of the 531 effects. Voluntarism is an explicit civic goal in the private school sector, so private schooling effects on voluntarism may differ from those on other civic outcomes. Private school attendees often come from higher socio-economic status in the absence of choice programs, plus different research methods and data structures may moderate the effects we observe. Hence, we included the variables for research design, analytical strategy, choice vs. self-selected, data structure, representative, scope, publication, and study quality. We coded for reporting bias because it would allow us to test for whether our calculation of effects using information inside and outside the study leads to different outcomes than our simple extraction of author-reported effects from studies where effects and standard errors are explicated.

We first started with a binary timing coding of the year 2000, since we expected older studies to report larger effects than younger ones, due to greater attention to correcting for self-selection in more recent studies. As we show later, our expectation proved right, and with more testing we found 1998 to be the best binary indicator variable for timing, as publication bias is limited to studies before 1998. We also coded for the year of the data that informed each effect estimate and, from that information, derived the data age. Data age is based on the earliest year of data for a civic outcome within a study. Both the effect year and data age variables vary within and between databases, while the other moderators vary only between them. We centered these two variables to distinguish between the within database cluster and between database cluster effects of these covariates (Fisher & Tipton, 2015, pp. 11–13; Tanner-Smith & Tipton, 2014, p. 19). These variables defined subgroups in the main analysis and moderators in the meta-regression.

Effect Size Choice and Calculations

We chose Hedges’ g and its standard error (Hedges & Olkin, 1985, p. 81) as the common metric to allow comparability across studies. Hedge’s g provides unbiased estimates of the difference between the outcomes for treatment and control groups, especially when the sample size is small. The Online Appendix describes the formula used to derive our chosen metric (Altman & Bland, 2011; Higgins & Green, 2011; Ringquist, 2013, pp. 116–117). We carried out the meta-analysis in STATA 17 using the robumeta package with hierarchical effects, and metafunnel, metabias, and meta trimfill for assessing publication and reporting bias. Furthermore, we conducted publication and reporting bias checks in R studio with the fsn command in the metafor package.

Selection of Empirical Models

We use RVE regression for our confirmatory analysis. We employ both RVE regression and meta-regression for our exploratory analysis of subgroup effects and effect moderation.

RVE

Table 1 shows that the 531 effects obtained from 57 studies are derived from 40 distinct databases. Thus, the dependencies among effects are hierarchical. Such dependencies could be among parent and student samples, different civic outcomes, studies using the same database, or impacts across different years due to individual students contributing to multiple effect sizes. The RVE method is similar to that of a cluster robust standard error approach. It models dependencies, drawing from patterns in the actual data, when the within-study covariance is unknown (Hedges et al., 2010). RVE yields consistent estimates of the underlying population parameters, and the results are consistent even under non-normality conditions (Fisher & Tipton, 2015, p. 15). We employ RVE based on hierarchical effects using small-sample corrections as our meta-analytic model (Tipton, 2015). RVE typically uses inverse variance weights (Hedges et al., 2010, p. 48) defined as:

$${\omega }_{ij}=\frac{1}{({\nu }_{ij}+{\tau }^{2}+{\omega }^{2})}$$

for effect size i in study j where ${\nu }_{ij}$ is the within-study sampling variance, ${\tau }^{2}$ is the estimate of the between-cluster variance, and ${\omega }^{2}$ is the estimate of the within-cluster between-study variance.

Meta-regression

We incorporate the known study features as moderators in an RVE meta-regression model with hierarchical effects with small sample corrections. We use three variants of timing: effect year ≥ 1998 vs. before 1998, the centered variables for effect year, and the centered variables for data age. In two specifications, we include interaction models between peer-reviewed journal article and effect year ≥ 1998, and voluntarism and effect year ≥ 1998.

Results

The distribution of the estimates of the effects of private schooling on civic outcomes, based on 90% confidence intervals, yields 168 positive, 320 null, and 43 negative findings (Fig. 2). This varied distribution suggests that an overall positive effect may be observed when RVE is used to cluster the effects within databases. We next present the results for an overall effect and various subgroup effects defined by background and moderator variables. In Table 2, the pooled effect size (ES) is shown in standard deviation (SD) units along with its standard error (SE). ${N}_{s}$ represents the number of databases at which the effects are clustered and ${N}_{E}$ represents the number of effect estimates. As Tipton (2015) suggests, when the degrees of freedom are below 4, the results are merely suggestive, as the p values are not reliable.

Fig. 2

Distribution of positive, null, and negative estimates. Note: Based off 90% confidence intervals

Table 2 Meta-analytic impacts based on various samples

RVE Impacts from the Main Regression Model

Our hypotheses for our confirmatory analysis, specified in our pre-registration, are that private schooling will be associated with higher levels of civic outcomes overall as well as with higher levels of political tolerance, political participation, political knowledge and skills, and voluntarism and social capital, specifically. The top panel consisting of the first five rows in Table 2 presents the results for our confirmatory analysis. The overall effect of private schooling on civic outcomes is 0.055 SD, statistically significant at the 99% confidence interval (CI). When divided across our four general types of civic outcomes, the effect is positive for political tolerance (0.120 SD, 99% CI), positively signed but null for political participation (0.033 SD), positive for political knowledge and skills (0.121 SD, 99% CI), and positive for voluntarism and social capital (0.043 SD, 90% CI). Thus, four of our five research hypotheses are confirmed by the RVE meta-analysis results. Private schooling is associated with positive and statistically significant civic outcomes overall and for each of our general types of outcomes except political participation.

The private schooling association is positive for all 10 of our specific measures of civic outcomes and statistically significant for six of them. Private school attendance is significantly correlated with higher levels of political tolerance (0.120 SD, 99% CI), political participation (0.180 SD, 90% CI), civic engagement (0.161 SD, 99% CI), civic knowledge/skills (0.061 SD, 95% CI), political knowledge (0.160 SD, 99% CI), and social capital (0.088 SD, 90% CI). The RVE estimates of the association between private schooling and voting, voter registration, political skills and voluntarism all are positively signed but statistically null. The null findings for voting, voter registration, and political skills should be interpreted with caution, as the RVE degrees of freedom for those effect estimates are less than 4. The null effect on voluntarism does not appear to be a power issue but could be a measurement issue related to definitional heterogeneity. Voluntarism contained the most varied operational definitions among the 10 civic outcomes in our study (Tables S5, S6, S7, S8, S9, S10, S11, S12, S13, S14, online only). Such variation in the operational definitions of voluntarism across the studies should introduce measurement error, thus inflating the standard errors and biasing the significance test toward a null conclusion. At a minimum, it does not appear that private schooling harms any of 10 specific civic outcomes in comparison to government run schooling. Yet, we need to be more certain by looking at the effects by other study characteristics.

The effect is larger on parents (0.107 SD) than on students (0.046 SD). When distinguished by research design, it appears that the positive effect of private schooling is largest in studies with observational research designs with control variables (0.086 SD). The average effect from the studies with more plausibly causal experimental/quasi-experimental designs is positive 0.019 SD, but null. This null finding itself is questionable, as the modest number of experimental/quasi-experimental studies in the sample yield less than four degrees of freedom. Studies with observational design without controls also yield a null effect. As we show below, research design does not significantly moderate the private schooling effect in our meta-regressions.

The effect for self-selected private schooling is positive (0.073 SD) and statistically significant (99% CI), but the accompanying null effect for studies of private school choice programs is highly uncertain (df < 4). We find positive and statistically significant effects for studies identified as (pooled) cross-sectional (0.070 SD), nationally representative (0.065 SD), and in the USA (0.057 SD). Reporting bias does not seem to be a concern, as the effects across the calculated and reported samples are nearly identical (0.056 SD and 0.055 SD).

Timing of the effects seems concerning, with effects in studies published before 1998 hugely positive (0.254 SD), whereas those from studies published since 1998 are only modestly positive (0.036 SD). These effects may indicate publication bias for effects prior to 1998, which we confirm later. We do not find clear subgroup effects by gender, ethnicity, disadvantaged status, grades, or dosage categories. Effects drawn from peer reviewed journal articles are only modestly positive (0.027 SD) while those from all other types of publications are largely positive (0.114 SD). Dividing the study quality index by terciles shows positive effects from low (0.056 SD) and medium quality (0.071 SD) studies, but a null effect from high quality studies (0.039) that itself is highly uncertain (df < 4).

Religious schooling seems to play a positive role in shaping civic outcomes. We find positive effects across various definitions of religious private schooling. A combined effect of any religious private schooling is positive (0.076 SD) and statistically significant (95% CI) in comparison to a null effect for secular schooling (0.015 SD) which itself is uncertain (df < 4).

Robustness Checks

We conduct two robustness checks for the overall estimate of the private schooling effect. First, we use ${N}_{E}$ (the number of effect size estimates) instead of inverse variance weights as a weighting factor. Second, we pool the effects at the study level instead of at the database level. Both checks yield overall effects of private schooling that are positive and significant at the 99% CI level (0.124 SD and 0.055 SD).

So far, we found suggestive but not confirmatory evidence for several background variables as potential study moderators. The null findings could be due to a small number of effects within a given category or due to a limited number of databases in which those effects are clustered, thereby leading to merely suggestive results. For example, we have 10 effects clustered in 2 databases for the effect of private schooling on political skills, 169 effects clustered in 7 databases from experimental/quasi-experimental research designs, 84 effects clustered in 5 databases for choice programs, and 127 effects clustered in 12 databases for panel data. The 151 effects for the disadvantaged category are clustered in 10 databases. This unbalanced nature of our data limits our ability to generate confirmatory takeaways from our RVE subgroup analysis. Next, we use meta-regression to further investigate potential study moderators.

Meta-regression Results

Table 3 presents results from meta-regressions where the private schooling effect on civic outcomes is the dependent variable and study moderators are the independent variables. Continuous regression is the omitted baseline analytic method category. Overall patterns show null effects for most potential study moderators. These null findings suggest that variation in research and analytical methodology, data structure and scope, self-selection, and study quality typically suspected of biasing private schooling effects do not moderate the civic effects of private schooling observed in this meta-analysis. In contrast to these null findings, it is puzzling that peer-reviewed journal articles yield much smaller effects in comparison to other publication types, even while controlling for research design and study quality, and the relationship is maintained across all models. Only 31.9% of the effects before 1998 came from peer-reviewed journal articles. This number jumps to 66.8% for 1998 and beyond. It is possible that early studies of the question were rushed out as policy reports or working papers, and set the agenda for the field, which then proceeded to establish a counter-narrative that was attractive to journal editors and reviewers as a critique of that early work. Unfortunately, we do not have data to shed light on the reason for this puzzling finding. Our results suggest that the findings of the effects of private schooling on civic outcomes reported in peer-reviewed journals may be negatively biased, while those reported in non-peer-reviewed publications may be positively biased.

Table 3 Meta-regression

Timing bears a negative and statistically significant relationship on the private schooling effect estimated by most models, even when the timing variables are centered and jointly included in a model (column 10). Only one estimation shows a lack of timing effect when data age loses statistical significance (column 7). Similarly, voluntarism maintains a negative and statistically significant relationship on the private schooling effect estimated in most models, indicating a larger effect for non-voluntarism civic outcomes in comparison to voluntarism. When peer-reviewed publication and voluntarism are interacted with effect year ≥ 1998, we find a null relationship on the interaction (columns 13 and 16). Moreover, the voluntarism coefficient loses statistical significance in column 16.

The null effect on voluntarism does not appear to be a power issue but could be a measurement issue related to definitional heterogeneity. Voluntarism contained the most varied operational definitions among the 10 civic outcomes in our study (Tables S5, S6, S7, S8, S9, S10, S11, S12, S13, S14, online only). Such variation in the operational definitions of voluntarism across the studies should introduce measurement error, thus inflating the standard errors and biasing the significance test toward a null conclusion.

Other unreported interaction models and coding formats of study moderators do not yield any different takeaways, and in some cases the RVE matrix does not converge when we further break out independent variables, such as further differentiating among research designs. Given that the df falls below four for differently coded variables in such models, and we do not observe any departures from the reported findings, we do not report those additional results.

For RVE meta-regressions, goodness of fit is determined by this equation:

$${{\text{R}}}^{2}=100*\left[1-\left({\tau }^{2}/{\tau }_{c}^{2}\right)\right]$$

where ${\tau }^{2}$ is the portion of the variance explained by a simple model with the focal predictor (private schooling) as the sole explanatory variable and ${\tau }_{c}^{2}$ is the portion of the variance explained by a complete model with the focal predictor and all moderator variables. In our case, ${\tau }^{2}$ = 0.0084 and ${\tau }_{c}^{2}$ = 0.0169, which yields an R² of 50.29%. Half of the explained variance in our meta-regression is due to the association between private schooling and civic outcomes and the other half is explained by factors that moderate that correlation.

Religious Schooling and Political Tolerance

Although we lack the granularity in our data to test definitively for religious schooling’s moderating effects on all civic outcomes, we are able to provide a strict test of religious schooling’s effect on political tolerance, which is a key concern in the literature on religious schooling (Freedom from Religion Foundation, n.d.; Levinson & Levinson, 2003, p. 105; Macedo, 2000, pp. 162–165). Table 4 shows the results of an RVE regression of effect size on political tolerance, any religious schooling, and their interaction for the subsample of studies that provide information on the religious identification of the private schools. If the outcome is political tolerance, the average private schooling effect is 0.170 SD larger than if it is any of the other nine civic outcomes, a difference that is statistically significant (95% CI). If the type of private school is religious, the average effect on civic outcomes is 0.069 SD larger than if the type of school is secular, also a statistically significant difference (95% CI). As the interaction term demonstrates, however, if the outcome is political tolerance and the private schooling type is religious, the average effect of private schooling is 0.167 SD lower, also a statistically significant difference (95% CI). The total effect of religious schooling on political tolerance in this subsample of findings is thus 0.003 SD, with the general tendency of private schooling to boost political tolerance almost exactly canceled out by the countervailing tendency of religious schooling not to boost political tolerance. The straightforward interpretation of these results is that religious schooling is not likely to generate political intolerance or political tolerance in comparison to government-run public schooling. If religious schooling has no negative effect on political tolerance, the outcome which is the toughest test for faith-based schools, it is difficult to imagine it has negative effects on any other civic outcome.

Table 4 Religious schooling and political tolerance

Publication and Reporting Bias

We made efforts through unrestricted database searches and expansive citation and expert searches to reduce the possibility of publication bias. We coded for a variable that allowed us to rule out reporting bias as a problem. However, if the studies included in the dataset are a biased sample of all potentially relevant studies, then the calculated meta-analytic effect will also be biased (Borenstein et al., 2009, p. 277). Earlier results motivate us to test for the presence of publication bias for effect year < 1998. In Table 5, we present the results of various checks for publication and reporting bias. The Egger’s Test (Egger et al., 1997) is simply a weighted regression of each finding’s effect size on its standard error (SE). Weights represent the precision of the finding, measured as the inverse of the SE. The null hypothesis is a zero slope, which provides a formal test for small-study effects. The Egger’s Test yields a statistically significant and positive coefficient for our overall sample (0.500 SD significant at 99% CI), and a breakout by effect year before and since 1998 shows that the publication bias is coming from the sample of earlier studies (1.391 SD significant at 99% CI for year < 1998 but only 0.107 SD and null for year ≥ 1998). These results are also observed graphically in funnel plots where fewer studies with large SEs are present on the left side of the average pooled effect size in comparison to the right side for the full sample and with an effect year before 1998 (Figs. 3, 4 and 5). Furthermore, we conduct fail-safe N checks based on the Orwin (1983) and Rosenberg (2005) analytical approaches for publication bias, and a stricter test for publication and reporting bias in the portion of the study sample where findings were explicitly reported and did not need to be calculated. Results show that we would need to add 531 null findings for the full sample and 343 null findings for the “effects explicitly reported” subsample to cut the average private schooling effect in half. An infinite number of null findings would need to be added to bring the effect to zero, and 61,325 null findings added to the full sample and 27,853 to the subsample of explicitly reported effects to reduce the combined significance level of the meta-analytic estimate to a target alpha level of 0.05. Our bias checks reveal that we would have to have missed hundreds or thousands of findings, and they all would have had to have been null, in order for our empirical results to be driven entirely by sample selection or reporting bias. Such conditions seem highly implausible, lending credibility to our findings.

Table 5 Publication and reporting bias

Fig. 3

Note. This funnel plot represents the Egger’s test of the full sample (Egger et al., 1997). Egger’s test, bias coefficient: ES = 0.500***, SE = 0.134. ***p < 0.01, **p < 0.05, *p

Funnel plot to assess publication bias in the meta-analysis.  < 0.1

Fig. 4

Note. This funnel plot represents the Egger’s test for effects before 1998 (Egger et al., 1997). Egger’s test, bias coefficient: ES = 1.391***, SE = 0.226. ***p < 0.01, **p < 0.05, *p < 0.1

Funnel plot to assess publication bias in the meta-analysis for effects before 1998.

Fig. 5

Note. This funnel plot represents the Egger’s test for effects since 1998 (Egger et al., 1997). Egger’s test, bias coefficient: ES = 0.107, SE = 0.158. ***p < 0.01, **p < 0.05, *p < 0.1

Funnel plot to assess publication bias in the meta-analysis for effects since 1998.

We also conducted a nonparametric trim-and-fill analysis of the full sample and the subsample with explicitly reported findings (Duval & Tweedie, 2000a, 2000b). This approach estimates the number of findings potentially missing from the dataset due to publication and reporting bias and imputes effect size values for them. It then recalculates the overall effect using both the observed and imputed point estimates. Results indicate that with missing studies imputed on the left side of the graphs (Figs. 6 and 7), the overall effect remains positive, 0.034 SD overall and 0.038 SD for the subsample, and statistically significant at the 99% CI overall and the 95% CI for the subsample. Hence, publication and reporting bias do not appear to be a concern for this meta-analysis, though we do detect and at times control for an upward bias in the findings from studies with an effect year before 1998.

Fig. 6

Note. This funnel plot represents the nonparametric trim-and-fill analysis of the full sample (Duval & Tweedie, 2000a, 2000b). Observed sample: ES = 0.067***, SE = 0.008, N = 531. Observed + Imputed sample: ES = 0.034***, SE = 0.010, N = 615. ***p < 0.01, **p < 0.05, *p < 0.1

Contour enhanced funnel plot to assess publication bias in the meta-analysis.

Fig. 7

Note. This funnel plot represents the nonparametric trim-and-fill analysis of the sample for which studies reported effects (Duval & Tweedie, 2000a, 2000b). Observed sample: ES = 0.075***, SE = 0.011, N = 343. Observed + Imputed sample: ES = 0.038***, SE = 0.016, N = 397. ***p < 0.01, **p < 0.05, *p < 0.1

Contour enhanced funnel plot to assess publication and reporting bias in the meta-analysis.

Heterogeneity Analyses and Limitations

The meta-regression measure of fit suggests that about half of the variation explained in our analysis is due to our moderator variables. Although we tried to code for as many background and study moderators as possible, lack of data across all 531 effects for many variables limits us from testing their confirmatory effects as study moderators. Not all study differences are observable. Although the analysis results reported in Table 2 show considerable heterogeneity across subgroups, most variables do not appear to moderate the effects reported in Table 3. Information on variables such as gender, ethnicity, disadvantaged status, grades, dosage, and school religious identity were available only from a limited number of studies. We only include these variables as the basis for categorical subgroups in the main analysis. Many other factors that may moderate the effects of private schooling could not be coded. For example, we are only able to provide suggestive evidence for the effect of religious vs. secular schooling, as information could not be coded for all 531 effects to be included in a meta-regression. Information was not available on variation in regulations for private schooling, school’s age, and single sex vs. coed-schools.

The lack of standardized measures of civic outcomes that the studies use (Tables S5, S6, S7, S8, S9, S10, S11, S12, S13, S14, online only) prohibits us from assessing moderating effects based on the scope and content of the civics knowledge tests. Nor do we have information on the test rigor and reliability for those civic outcome measures. We are unable to convert the effect sizes in this meta-analysis against a known benchmark for civics related interventions. We could not conduct a cost-effectiveness study because the costs across schooling types were not available. Lack of information also prohibits us from explaining why findings from peer-reviewed journal articles generate downward bias on private schooling’s overall civic effects. Our study does not cover outcomes such as private schooling’s effect on crime, drug and substance use, teenage pregnancy, and anti-social behavior. Lastly, the public–private distinction in school sectors is not universal. While we are confident that the studies retrieved through our search allow a reasonable comparison of public–private sectors, our results suggest that educational pluralism with religious options best promotes civic outcomes.

Discussion

This meta-analysis provides an up-to-date, methodologically sophisticated, and comprehensive overview of the effect of private schooling in comparison to public schooling on 10 civic outcomes nested within the four general categories of political tolerance, political participation, civic knowledge and skills, and voluntarism and social capital. We pre-registered our meta-analysis research plan to avoid the possibility of p-hacking and other unscientific approaches to policy analysis. Our literature search yielded 13,301 potential studies from which 57 were determined to be eligible for inclusion. These 57 studies reported 531 civic effects of private schooling clustered in 40 databases. The overall results indicate positive effects of private schooling on civic outcomes of 0.055 SD that are statistically significant at the 99% CI. Private schooling also demonstrates a statistically significant positive effect on three of our four general types of civic outcomes: political tolerance, political knowledge and skills, and voluntarism and social capital.

We observe significant heterogeneity in the effect of private schooling on civic outcomes across several subgroups in our RVE analysis. Meta-regression, however, shows only voluntarism, timing, and peer-reviewed journal articles are significant study moderators. We identify publication bias in studies with effect year < 1998 which generate an upward bias in the effect estimate from that portion of the sample. Yet, through a variety of tests, we show that publication and reporting bias do not drive the overall findings in this meta-analysis.

Our study yields important theoretical takeaways about the civic role of private schooling and religious schools. Writing about the private school choice movement, political theorist Stephen Macedo (2003, pp. 57–58) states:

Many [parents] are desperate to get a decent education for their children, and they want help to pay for a religious education…because those institutions are thought to provide a superior education, a safer environment, and a more coherent moral framework that will better promote the public virtues and values needed by young adults and future citizens nowadays.

The findings of our meta-analysis are consistent with Macedo’s claims. Religious schooling appears to increase civic outcomes in comparison to its secular counterpart, whether public or private. Even for the outcome of political tolerance, which, arguably, is the toughest test for religious schooling effects on civics, our analysis suggests that the average effect of religious schooling is, at worst, null. Moreover, our evidence indicates that the Catholic school civic advantage acknowledged by some scholars (Coleman & Hoffer, 1987, pp. 212–213; Macedo, 2000, pp. 234–235; Prud’homme, 2022) appears to be a more general religious school civic advantage. The data do not support the theoretical concerns of harmful civic effects from private schooling (Dewey, 1916, p. 85; Gutmann, 1987; Shober, 2012, pp. 192–194; Stitzlein, 2023[2017], p. 14). Based on the empirical evidence, private schooling does not threaten democracy. Educational pluralism seems to be a boon, and not a bane, for civic outcomes (Berner, 2017, pp. 73–103; McCluskey, 2020, pp. 18–19).

Our findings yield a key insight for public policy as well as an important qualification. Religious private schools stand out for their positive association with desirable civic outcomes. The theoretical literature reviewed earlier suggests that may be due to the values-infused culture that many religious schools maintain as a distinctive aspect of their identity. As private school choice programs continue to expand in the USA and globally, our findings suggest efforts to regulate away the distinctiveness of religious private schools, in the name of advancing civic values and behaviors (e.g., Venegoni & Ferrero, 2004), might have the opposite effect.

Our findings are based on the existing empirical record of private and public school performance regarding inculcating civic values in young people. As a more expansive and diverse population of students gain access to private schooling, and as pressure mounts for public schools to improve their civics instruction, the private (mostly religious) school advantage in promoting civic outcomes could disappear or even flip to a public school advantage. Only three data bases in our study allowed us to examine private schooling civic effects by student demographic characteristics, and both the effects and their variation were modest, leaving us with little guidance on how sensitive our results might be to changes in the private school student populations. Still, as stock-market analysts regularly say, “past performance is no guarantee of future returns.”

Implications for Future Research

More studies of private schooling effects on civic outcomes should employ experimental or quasi-experimental designs. The findings included in our meta-analysis from such plausibly causal studies indicate that the average effect of private schooling on civic outcomes may be positive, but the effect is uncertain (df<4), and not statistically significant, even at a minimally acceptable confidence level. More findings from causal studies would allow us to determine if that initial finding is a true null or a Type II (false negative) error. Future research should examine the mechanisms by which private, especially religious, schools boost the civic outcomes of students and parents. Researchers should investigate the role that educational pluralism and religious instruction might play in increasing civic outcomes. Scholars also should explore why the private schooling effect on voluntarism is smaller than the effects on other civic outcomes, especially since the religious service activities prompted by exposure to religious schooling do not appear to be adequately captured in most measures of voluntarism employed in previous studies.

The school-age population in the USA experienced a marked decline in civic knowledge recently (Finn, 2023). Policy makers might consider whether an expansion of private or religious schooling could be an effective policy lever to increase civic outcomes. We also encourage greater standardization of civic outcome measures for the purpose of program evaluation, and for the construction of a benchmark for civics related effect sizes (Institute for Citizens & Scholars, 2023). We hope that our study motivates these and many other investigations of the vital question of which types of schools best advance the strength and resilience of democracies. As Wolfe (2003, p. 10) states, private schooling and school choice will continue to be the subject of fierce debate “because we have always made schools so central to the question of the kind of society to which we aspire.”