The five studies discussed in this section were successful in RER’s peer-review and editorial process. We selected them as a handful of varied examples to highlight how authors of articles published in RER “story” their findings in consequential and compelling ways. These published works guide us scientifically and persuasively through the literature reviewed. At the same time, the authors acted as an “intelligent provider” (Petticrew and Roberts 2006, p. 272, citing Davies 2004) of information by inhabiting the review with the concerns of particular people in particular places. The problems of study are teased out in complex ways, using multifocal perspectives grounded in theory, history, or geography. Two had an international scope and three were restricted to studies conducted in settings in the U.S. Whether crossing national boundaries or focused on the U.S. only, each review engaged variations in the places where the focal policies and practices were carried out. Further, all of the reviews we discuss in this section story their analyses with variation in the characteristics of learners and in the educational practices and policies being examined through the review.
4.1 Theoretical Propositions as Multifocal Lenses: Storying Reviews with Ideas
Østby et al. (2019) of “Does Education Lead to Pacification: A Systematic Review of Quantitative Studies on Education and Political Violence” capture the attention of the non-specialist RER reader by citing Steven Pinker’s acclaimed book The Better Angels of Our Nature. They highlight Pinker’s metaphor characterizing education as an “escalator of reason,” an escalator that has the power to act globally as a “pacifying" force (p. 46). Noting wide acceptance of the idea that societies with a higher level of education will experience lower levels of political violence and armed conflict, the authors quickly shake this assumption. Recent studies have shown, for example, that terrorists and genocide perpetrators have had higher than average levels of education relative to others in their societies. Further, the story of the relationship between increases in educational attainments and political violence in a society unfolds in a more complicated manner when factors such as initial baselines of education in the population, gender disparities in access to elementary and secondary schools, and inequalities among socio-economic groups are taken into account.
Østby et al. (2019) organize their review of 42 quantitative studies of education and political violence around theoretical propositions that add complexity to the notion that education is a pacifying force. From an economic perspective, there are several reasons why education should lead to a decrease in political violence and social unrest. Those with more education typically have higher earnings and may be deterred from engaging in social unrest because they may lose their jobs, a consideration of less consequence to the unemployed or those with marginal labor force status. Alternatively, a political explanation for a positive impact can be found in the fact that those who are more highly educated are more greatly exposed to and culturally inculcated through the curriculum sanctioned by the government, which may be dominated by nationalistic historical narratives.
In contrast, a sociological explanation based in theories of relative deprivation points in the opposite direction, as the sociologist attends to inequality among socio-economic groups. Groups that lack political power and have historically been oppressed or disenfranchised may become more likely to engage in violent political action as they gain in educational attainment yet continue to lag behind dominant social groups. When it comes to the study of the relationship between education and political violence, Østby et al. (2019) show that it is insufficient to characterize a country in terms of the educational attainment of a population without also considering governmental influence in the curriculum, political oppression, and educational inequality.
As Østby et al. (2019) discuss contrasting theoretical propositions for positive and negative associations between education and violence, the reader quickly buys into the premise that the authors’ study of this “complex, multi-faceted, and multidirectional” phenomenon is highly consequential (p. 47). More nuanced understandings clearly hold the potential to inform the manner and degree of governmental and philanthropic investments in education in developed and developing countries around the globe.
Similarly, García and Saavedra (2017), in their examination of the impacts of “conditional cash transfer programs,” utilize human capital and household decision making theories to introduce readers to a very precise, yet varied, set of hypotheses that they subsequently use to structure the reporting of their results. These economic hypotheses postulate the potential effects of governmental programs that provide cash rewards to households or individuals to encourage them to who respond to policy incentives in desired ways. They highlight that the direction and strength of effects depend on a range of household inputs such as parental education, sources of income (e.g., formal and informal labor force participation), time use among household members (adults and children), and community characteristics. As other researchers have before them, these authors meta-analyze impact estimates from studies meeting their threshold methodological quality criteria for making causal claims. Their review synthesizes 94 studies of 47 conditional cash transfer (CCT) programs carried out in 31 countries (p. 929, 934). Their work builds on and extends the findings of prior meta-analyses that produced CCT impact estimates by also examining questions of cost-effectiveness.
In García and Saavedra’s (2017) study, the examination of effects comprises seven outcomes: “primary school enrollment, primary school attendance, primary school dropout, secondary school attendance, secondary school dropout, and school completion” (p. 933). The authors demonstrate that variations in program characteristics delineated in their review correspond to variations in program effectiveness, both in terms of these various effects and of economic investments in the intervention. An important finding of this study (among many others) is that “all else constant, primary enrollment impact estimates are greater in CCT programs that complement cash transfers with supply-side interventions such as school grants” (p. 923). The finding is consequential to future policy design because less than 10% of the CCT programs studied had a design component that attempted to incentivize changes in schooling practices at the same time they were providing incentives for greater household investments in education.
4.2 Engaging Interactions: Storying Reviews with People, Policies, and Practices
The capacity to model, measure, and attend to dynamic interactions among governmental policies, educational institutions or settings, and the behavior of individuals is a hallmark of the quality of this small set of exemplar RER articles. Each of the studies we reviewed in this chapter attend to differences among students in their experiences of schools and educational interventions with varying characteristics. For some this involves differences in national contexts and for others differences among demographic groups.
Welsh and Little (2018), for example, motivate their comprehensive review through synthesis of a large body of research that raises concerns about racial inequities in the administration of disciplinary procedures in elementary and secondary schools in the U.S. Prior studies had shown that Black boys in U.S. schools were more likely than girls and peers with other racial characteristics to receive out-of-school suspensions and other forms of sanctions that diminished students opportunities to learn or exposed them to involvement in the criminal justice system. The authors engage readers in the “complexity of the underlying drivers of discipline disparites” (p. 754) by showing that the phenomenon of the unequal administration of discipline cannot be fully accounted for by behavioral differences among students of different racial and gender characteristics.
By incorporating a synthesis of studies that delineate the problems of inequitable disciplinary treatment alongside a synthesis of what is known about programmatic interventions intended to improve school climate and safety, Welsh and Little make a unique contribution to the extant literature. Winnowing down from an initial universe of over 1300 studies yielded through their broad search criteria, they focus our attention on 183 peer-reviewed empirical studies published between 1990 and 2017 (p. 754). Like García and Saavedra (2017), these authors use critical appraisal of the methodological characteristics of the empirical literature they review to place the findings of some studies in the foreground of their analysis and others in the background. Pointing out that many earlier studies used two-level statistical models (e.g. individual and classroom level), they make the case for bringing the findings of multi-level models that incorporate variables measuring student-, classroom-, school-, and neighborhood-level effects to the foreground. Multi-level modeling allows the complexities of interactions among students, teachers, and schools that are enacting particular policies and practices to emerge.
Teasing out the contributors to disciplinary disparities among racial, gender, and income groups, and also highlighting studies that show unequal treatment of students with learning disabilities and lesbian, bisexual, trans*, and queer-identified youth, Welsh and Little (2018) conclude that race “trumps other student characteristics in explaining discipline disparities” (p. 757). This finding contextualizes their deeper examination of factors such as the racial and gender “match” of teachers and students, especially in public schools where the predominantly White, female teaching force includes very few Black male teachers. Evidence suggests that perceptions, biases, and judgments of teachers and other school personnel (e.g. administrators, security officers) matter in important ways that are not fully addressed by programmatic interventions that have mainly focused on moderating students’ behavior. The interventions examined in this review, therefore, run the gamut from those that seek to instill students with greater social and emotional control to those that attempt to establish “restorative justice” procedures (p. 778).
Ultimately Welsh and Little (2018) conclude that “cultural mismatches play a key role in explaining the discipline disparities” but “there is no ‘smoking gun’ or evidence of bias and discrimination on the part of teachers and school leaders” (p. 780). By presenting a highly nuanced portrayal of the complexities of interactions in schools, Welsh and Little create a compelling foundation for the next generation of research. Their conclusion explicates the challenges to modeling causal effects and highlights the power of interdisciplinary theories. They synthesized literature from different fields of study including education, social work, and criminal justice to expand our understanding of the interactions of students and authorities who judge the nature of disciplinary infractions and determine sanctions. Their insights lend credence to their arguments that future analyses should be informed by integrative theories that enable awareness of local school contexts and neighborhood settings.
The importance of engaging differences in student characteristics and the settings in which students go to school or college also emerges strongly in Bjorklund’s (2018) study of “undocumented” students enrolled or seeking to enroll in higher education in the United States, where the term undocumented refers to immigrants whose presence in the country is not protected by any legal status such as citizenship, permanent resident, or temporary worker. This study makes a contribution by synthesizing 81 studies, the bulk of which were peer-reviewed journal articles published between 2001 and 2016, while attending to differences in the national origins; racial and ethnic characteristics; language use; and generational status of individuals with unauthorized standing in the U.S. Generational status contrasts adult immigrants with child immigrants, who are referred to as the 1.5 generation and “DACA” students, the latter term deriving from a failed federal legislative attempt, the Deferred Action for Childhood Arrivals (DACA), to allow the children of immigrants who were brought to the country by their parents to have social membership rights such as the right to work and receive college financial aid from governmental sources. DACA also sought to establish a pathway to citizenship for unauthorized residents.
Using the word “undocumented” carries political freight in a highly charged social context in which others, with opposing political views, use terms such as “illegal aliens” (Bjorklund 2018, p. 631). Bjorklund acknowledges that he is politically situated and that his review has a political point of view by titling his study a “critical review.” Rather than claiming a lack of bias with respect to the treatment of undocumented students, the author positions himself within the literature with a clear purpose of generating findings that will inform policy makers and practitioners who would like to support the success of undocumented students. Bjorklund then stories his findings through a review of relevant judicial cases, changes in and attempted changes to federal law, and variations in state laws and policies, the latter of which are highly salient in the U.S., where education is primarily governed at the state level. These accounts are more accurately described as purposeful relative to the goals of the review, rather than unbiased. Nevertheless, in describing historical facts and the specifics of policy design, the author’s account proves trustworthy to readers in the sense that these details are transparently referenced with respect to documented legislative actions, proposed and implemented federal and state policies and judicial case law, including Supreme Court rulings.
The extent to which individual legislatures in the 50 U.S. states allow undocumented students to access state benefits (such as reduced college tuition charges for state residents) emerges as an important aspect of this review. Geography matters, too, in the consideration of student characteristics and the design of institutional practices and policies to meet the varied needs of undocumented college students. Some states, cities, and rural areas have a larger proportion of unauthorized immigrants from border countries such as Mexico and countries in Central and South America (which figure prominently in the narratives of those opposing state and federal policies that would provide higher education benefits to undocumented college students), whereas other regions have a larger proportion of immigrants from Asia and Europe.
In addition to reporting salient themes and appraising studies for their intellectual merit, authors of systematic reviews help translate a research purpose for intended audiences and offer a charge for the future. Crisp et al. (2015) accomplish this precisely in their review of literature on undergraduate Latina/o students and factors associated with their academic success. The authors firmly establish the significance of their review, using trend data and statistics showing the growth of the Latina/o population in the U.S. broadly to demonstrate the timeliness of their topic. This growth, the authors note, has also resulted in increases in college enrollment for Latina/o students across the wide variety of postsecondary institutions in the U.S, but institutional policies and practices have not kept up in response to this demographic change. Appreciating the within-group differences of Latina/o students, Crisp and colleagues also acknowledge the varied experiences of Mexican, Peruvian, Colombian, and Salvadoran college students. Such distinctions and clarifications help frame their review within the full context of the topic for the reader.
Crisp, Taggart, and Nora’s (2015) methodological decisions for their systematic review are also clearly informed by the authors’ positionalities and commitment to “be inclusive of a broad range of research perspectives and paradigms” (p. 253). They employ a broad set of search terms and inclusion criteria so as to fully capture the diversity that exists among Latina/o students’ college experiences. For instance, the authors operationalize the conceptualization and measurement of ‘academic success outcomes’ broadly. This yields a wider range of studies—employing quantitative, qualitative, and mixed methodological approaches—for inclusion.
Consistent with this approach, prior to describing the methodological steps taken in their review, the authors present a “prereview note.” The purpose of this section was to offer additional context about larger and overlapping structural, cultural, and economic conditions influencing Latina/o students broadly. For instance, the authors discuss how “social phenomena such as racism and language stigmas” impact the educational experiences of Latina/o students. They also acknowledge cultural mismatch between students’ home culture and school/classroom culture, which “has been linked to academic difficulties among Latina/o students” (Crisp et al. 2015, p. 251). These are just several examples of how the authors help contextualize the topic for readers, especially for those not familiar with the topic or with larger issues impacting Latina/o groups. This is also necessary context for a reader to make sense of the major findings presented in a later section.
Finally, we appreciate the way that Crisp et al. (2015) also make their intended audience of educational researchers clear. They spend the balance of their review, after reporting findings, making connections among various strands of the research they have reviewed, their goal being to “put scholars on a more direct path to developing implications for policy and practice.” They direct their charge, specifically, to call on “the attention of equity-minded scholars” (p. 263). As these authors illustrate, knowing your audience allows you to story your systematic review in ways that directly speak to the intended benefactor(s).