In this section we argue that there are several ways in which Evidential Pluralism can be fruitfully applied to the social sciences. Most obviously, the move from EBM to EBM+ warrants an analogous move from present-day evidence-based policy (EBP) to EBP+, a new approach to policy appraisal which takes evidence of mechanisms more seriously (Section 2.1). Of course, causal claims in the social sciences are not limited to claims about the effectiveness of proposed policy interventions—they also include claims about the causes and effects of societal, economic, legal, geographical, linguistic and psychological phenomena, for example. These claims are at the heart of what might be called ‘basic social science research’. We argue that Evidential Pluralism can be usefully applied to basic social science research, in addition to policy appraisal, because it sheds new light on the evidential relationships involved in establishing causation (Section 2.2). The question then arises as to how Evidential Pluralism relates to mixed methods research in the social sciences. We argue in Section 2.3 that there are important differences between the conceptual distinctions underlying Evidential Pluralism and those prevalent in mixed methods research, but that Evidential Pluralism can nevertheless be thought of as providing new, metaphysics-free, philosophical foundations for certain kinds of mixed methods research.
From evidence-based medicine to evidence-based policy
In the 1990s, the methods of EBM quickly spread to the evaluation of social interventions, leading to what is now known as evidence-based policy (EBP). The Cochrane Collaboration, which promotes EBM, was set up in 1993, while the Campbell Collaboration, which promotes EBP along similar lines, was created in 1999.
In the UK, for example, the primary organ of EBP is the government-led ‘What Works Network’, which includes the National Institute of Health and Care Excellence (NICE) as well as eight other centres tasked with evaluating social interventions. The What Works Network is built around the use of association studies (in particular, RCTs) as the evidence on which to base an evaluation (What Works, 2018a, p. 4) and membership of the network is restricted to centres which share the ranking of evidence promulgated by present-day EBM/EBP (Cabinet Office, 2018, p. 3). Allied to the What Works Network is the UK Government Trials Advice Panel, which was set up in 2015 to promote the use of RCTs in public policy decision making (What Works, 2018b). These structures ensure that the monistic methods of present-day EBM/EBP are entrenched at the heart of policy making in the UK.
The situation in the UK is just one instance of a global phenomenon. In the US, the dominant approach to EBP is also modelled on EBM, with a focus on RCTs (Baron, 2018). In addition, the United Nations actively promotes a global vision of EBP based on statistical association studies—see, e.g., United Nations (2013). EBP and EBM continue to develop hand-in-hand: e.g., the Cochrane and Campbell Collaborations share methods at events such as the ‘Global Evidence Summit’, held in 2017 in Cape Town and 2023 in Prague.
Thus, EBP is modelled on EBM, which, as we noted in Section 1, underestimates the importance of mechanistic studies. Given this, there is arguably a need for what we will call ‘EBP+’, i.e., an analogue of EBM+, but applied to policy evaluation. As with EBM+, the aim of EBP+ is to provide methods for systematically assessing mechanistic studies and integrating these assessments with those of association studies in order to determine the status of a causal claim. The need for EBP+ arises because Evidential Pluralism applies equally to medicine and policy making. If Fig. 1 captures the key evidential relationships when evaluating a causal claim in medicine, then it does so too in the social sciences, in particular when evaluating a social intervention. Moreover, EBP+ is also required for successful extrapolation: in both medicine and the social sciences, it is only by considering mechanisms of action that one can decide whether a causal relationship discovered in a study context can be extrapolated to a target context of application (see, e.g., Steel, 2008; Wilde & Parkkinen, 2019).Footnote 5 Thus EBP+ would proceed very much along the same lines as EBM+. The evaluation methods common to both are set out in detail by Parkkinen et al. (2018).
One might wonder whether there is some systematic difference between the social sciences and the biomedical sciences that undermines the applicability of Evidential Pluralism to the social sciences. However, one cannot draw a sharp distinction between the evaluation of causal claims in medicine and those in the social sciences. Indeed, such claims often overlap: health policy interventions are interventions in both medicine and social policy; hence the inclusion of NICE in the What Works Network. Insofar as one can generalise, the main methodological difference between the biomedical sciences and the social sciences is that in the social sciences it can be harder to isolate an experiment from contextual factors that might influence its results and that can thwart replication. In many cases it can also be harder to properly randomise individuals to social policy interventions, to construct a placebo intervention for the control group, and to ensure adherence to the social policy interventions being tested. In addition, it can be harder to successfully extrapolate claim about the effectiveness of a social intervention from one setting to another than to extrapolate a claim about the effectiveness of a medical intervention. This is because social settings can vary much more widely in their social mechanisms than do human bodies in their pathophysiological mechanisms. All these considerations favour a shift towards Evidential Pluralism over the current reliance on RCTs.
Some EBP practitioners have indeed begun to question the current focus on RCTs. For example, Yamey and Feachem (2011) observe that,
while the RCT is rightly hailed as the ‘pinnacle’ of evidence-based medicine, in the global public health community, there is growing recognition that new research designs are desperately needed to help evaluate ‘real world’ programmes. Such designs would, we believe, also help to illuminate the implementation ‘black box’. (Yamey & Feachem, 2011, p. 98.)
Moreover, one of the What Works centres has begun to recognise the importance of mechanisms. The What Works Centre for Crime Reduction has developed the ‘EMMIE’ framework for systematic reviews of evidence: Effect size, Mechanism, Moderator, Implementation and Economics are all components of an evaluation (Johnson et al., 2015; Tilley, 2016; Thornton et al., 2019). From the point of view of Evidential Pluralism, considering mechanisms is an important step in the right direction. However, EMMIE is based not on Evidential Pluralism but on the realist evaluation approach of Pawson and Tilley (1997), and it will be instructive to consider how their approach differs from one based on Evidential Pluralism.
While mechanisms are important to both realist evaluation and Evidential Pluralism, there are three key philosophical differences between these two approaches.
First, the realist evaluation approach of Pawson and Tilley (1997) makes a firm commitment to scientific realism: specifically, a causal powers metaphysics of causation (Pawson & Tilley, 1997, pp. 33, 56). This realism is accompanied by a rejection of Humean and Kantian metaphysics, which hold that causal relationships are a device we employ to structure the world, and which do not posit causal powers or causal necessitation ‘out there’ in the world. Evidential Pluralism, in contrast, is a purely epistemological thesis that makes no specific metaphysical claims. It is compatible with an anti-realist account of causation which analyses causal claims in terms of rational beliefs, for example (Williamson, 2021b, Appendix).
The second philosophical difference between the realist evaluation of Pawson and Tilley (1997) and Evidential Pluralism is that, on account of its metaphysical commitment, their approach involves a rejection of the experimental methodology that underpins RCTs and certain other kinds of association study. Evidential Pluralism, in contrast, takes experimental methods to have the potential to provide good evidence, relevant to the assessment of a causal claim. If Evidential Pluralism is right, one shouldn’t reject these methods—rather, one should augment them, by considering mechanistic studies alongside association studies. The aim of EBP+ would be to improve, rather than overturn, present-day EBP.
Third, realist evaluation proceeds from the premise that there is no logic of evaluation (Pawson & Tilley, 1997, xiii) while Evidential Pluralism takes there to be a logic of evaluation, portrayed by Fig. 1. According to this logic of evaluation, causation is established by establishing correlation and mechanism, which in turn requires assessment of the confirmation channels C1,C2,M1,M2,M3, i.e., the assessment of any relevant association studies and mechanistic studies. Parkkinen et al. (2018) show in the context of medicine that this logic of evaluation can be broken down into a series of practical steps.
We should note that realist evaluation has been developed in a number of different directions since 1997—see Jagosh et al. (2016), for example, for some pointers. In particular, not all proponents of realist evaluation now reject the experimental methodology and RCTs. For example, Bonell et al. (2012) argue for the use of RCTs in a way that is sensitive to the concerns of realist evaluation. Moreover, the development of EMMIE can be considered to be a move towards a logic of evaluation.
Thornton et al. (2019), although proponents of EMMIE, identify some limitations of realist evaluation as implemented in the EMMIE approach. In practice, EMMIE exclusively scrutinises systematic reviews, which almost always consider association studies rather than mechanistic studies, so mechanistic evidence tends to appear rather scant and hence to be rated as weak in EMMIE evaluations. From the point of view of Evidential Pluralism, it is not enough to consider systematic reviews of association studies—it is essential to articulate specific mechanism hypotheses and to search the literature for evidence relevant to those hypotheses. The International Agency for Research on Cancer provides an example of good practice here (Williamson, 2021d): each carcinogenicity evaluation has a dedicated subgroup responsible for systematically assessing mechanistic studies.
Furthermore, since an EMMIE evaluation has five components, it is not obvious how these five aspects should combine to give an overall assessment. This opens the door to subjective judgements of relative importance to influence the overall assessment. Thus, Thornton et al. (2019) worry that realist reviews may not be replicable. This is less of a concern for Evidential Pluralism, which only has two strands to integrate, namely evidence of correlation and evidence of mechanisms (Williamson, 2021d). Parkkinen et al. (2018, Chapter 7) offer a systematic way of integrating these two strands in order to come to an overall assessment.
One final point which is worth noting with respect to EMMIE is that ‘Mechanism’ is graded on a scale from 0 to 4 but only grade 4 requires concrete evidence of mechanism: grades 1 to 3 merely require some story or theory about what the mechanism might be (Thornton et al., 2019, Figure 2). This may stem from the important role of theory in realist evaluation (Pawson and Tilley, 1997, p. 59). In contrast, Evidential Pluralism is concerned with evidence, not theory. In the social sciences, it is often very easy to conjecture a mechanism hypothesis, and a story of a mechanism that is not backed up by evidence has no confirmatory value, for the Evidential Pluralist.
Although Evidential Pluralism differs from realist evaluation in important ways, the two approaches do share some key claims: most notably, that EBP needs to move beyond EBM’s monistic focus on association studies, and that mechanisms should play a prominent role. Evidential Pluralism can be thought of providing some motivation for these claims that is not tied to realism.
There are other methodologies that fit naturally with Evidential Pluralism. For example, process tracing (Mahoney, 2012), contribution analysis (Mayne, 2001), and sociomarkers (Ghiara & Russo, 2019) have all been put forward as ways of developing and evaluating specific mechanism hypotheses and might thus be invoked to help assess channel M1 in Fig. 1. Moreover, as we shall see shortly, Evidential Pluralism coheres well with a form of mixed methods research. First, though, we turn to the use of Evidential Pluralism in areas of the social sciences other than policy evaluation.
Evidential Pluralism in basic social science research
In medicine, claims about the effects of interventions are not the only causal claims of interest. Claims about the effects of pathogens, chemical exposures and lifestyle factors, for instance, are also central to medicine. More generally, basic medical research is concerned largely with claims about the causal components of mechanisms for health or disease. Likewise, the interests of the social sciences extend well beyond claims about the effects of interventions. The causal components of the mechanisms relevant to the various social sciences are the bread and butter of basic social science research. Figure 1 and Evidential Pluralism apply equally to these causal claims as to claims about interventions. While we do not suggest that social scientists conceptualise their research methodology in terms of Evidential Pluralism, good social science research tends to take both association studies and mechanistic studies into account, where available. (For some examples of mechanistic studies in basic social science research, of varying study designs, see Table 2.) Framing basic social science research in terms of Evidential Pluralism can help researchers to understand the core confirmatory relationships between items of evidence and to reach an overall assessment of the credibility of the causal claim of interest.Footnote 6
Of course, Evidential Pluralism is not the only approach to emphasise the importance of mechanisms to the social sciences. We have already seen that mechanisms are central to realist evaluation, and a mechanism-based approach has also been developed and defended as a central part of analytic sociology (Hedström and Swedberg, 1998; Demeulenaere, 2011). However, Evidential Pluralism differs from this latter mechanism-based approach, just as it differs from realist evaluation. Firstly, the two approaches address different questions: the mechanism-based approach emphasises the role of mechanism-based explanations in the social sciences, whereas Evidential Pluralism concerns the types of evidence needed to establish a causal claim. For this reason, the mechanism-based approach focusses primarily on mechanisms, while Evidential Pluralism is a dualist approach, treating evidence of correlation and evidence of mechanisms on a par. Second, as in the case of realist evaluation, the mechanism-based approach makes metaphysical commitments that are not made by Evidential Pluralism. The mechanism-based approach typically assumes that causal relationships can be analysed or characterised in terms of mechanisms, while according to Evidential Pluralism, mechanisms are merely an important indicator of causality. Moreover, as Hedström and Ylikoski (2010, p. 64) argue, ‘underlying the mechanism-based approach is a commitment to realism and an opposition to any form of instrumentalism.’ Evidential Pluralism makes no explicit metaphysical claims.
As an example of the compatibility of basic social science research with Evidential Pluralism, consider Donohue and Levitt’s study on legalised abortion and crime rates. Donohue and Levitt (2001) argue that the legalisation of abortion in the 1970s was a cause of the decline in the crime rates in the 1990s in the United States. In order to establish this causal claim, they provide evidence of mechanisms as well as evidence of correlation. They find two mechanisms of action. The first is that legalising abortion reduces crime through smaller cohort sizes. The smaller cohort that results from legalised abortion means that when that cohort reaches the late teens or early twenties, there are fewer young males in their highest-crime years, and thus less crime.Footnote 7 The second mechanism stems from the fact that abortion has a disproportionate effect on the birth of those who are most at risk of engaging in criminal behaviour. Teenagers, unmarried women, and the economically disadvantaged are all substantially more likely to seek abortions (Levine et al., 1999). Recent studies have found children born to these mothers to be at higher risk of committing crime in adolescence (Comanor & Philipps, 2002). Thus, the two mechanisms form a mechanism complex linking the legalisation of abortion in the early 1970s to the drops in crime in the early 1990s.
Donohue and Levitt look for evidence of correlation by focusing on the variations of national time series of crime and abortion, of differential crime patterns between states which legalised abortion early and other states, and of state abortion rates and the state crime rates. They show that the legalisation of abortion was associated with a subsequent drop in crime. All of violent crime, property crime, and murder have fallen steadily since 1991, roughly the time the first cohort born would hit its criminal prime. Additionally, the five states that legalised abortion in 1970 saw drops in crime before the other 45 states and Washington DC, which legalised abortion in 1973. Also, higher rates of abortion in a state in the 1970s and early 1980s are strongly linked to lower crime over the period from 1985 to 1997. Moreover, the observed correlation also holds conditional on various potential confounders, such as such as the level of incarceration, the number of police, and measures of the state’s economic well-being (the unemployment rate, income per capita, and poverty rate). It is shown that there is no relationship between abortion rates in the mid-1970s and crime changes between 1972 and 1985, when the cohort directly affected by abortion legislation would have been very young. Almost all of the abortion-related crime decrease can be attributed to reductions in crime among cohorts born after abortion legalisation. In contrast, there is little change in crime among older cohorts, who were not affected by abortion legalisation. The correlation is further supported by the more recent study of Donohue and Levitt (2019).
It is clear that Donohue and Levitt’s justification of the causal claim about legalised abortion and crime rates accords well with Evidential Pluralism. Not only do they look for a conditional correlation, but they also seek relevant mechanisms and they show that these mechanisms can account for the extent of the observed correlation. What is more, Donohue and Levitt’s criticisms of the alternative causal explanations are also compatible with the epistemological picture provided by Evidential Pluralism. For example, the reason that Donohue and Levitt dismiss factors such as the increasing use of incarceration and the rise in police numbers as the causes of the drop in crime rates is that these trends fail to exhibit an appropriate conditional correlation.
Donohue and Levitt’s study has sparked debate and controversy in the literature. Both their evidence of correlation and their evidence of mechanisms have been disputed (Joyce, 2004; Lott & Whitley, 2007; Chamlin et al., 2008; Foote & Goetz, 2008; Dills et al., 2010). Chamlin et al. (2008), for example, argue that there is no evidence that the legalisation of abortion led to a decline in the birth rate for teenage or unmarried women: i.e., they are sceptical of the evidence of mechanisms. Moreover, Lott and Whitley (2007) question whether Donohue and Levitt provide the complete mechanism complex from abortion to crime. As they point out, ‘abortion can eliminate unwanted children and can benefit many women, but it can also make other women who are unable to bring themselves to have an abortion worse-off and more likely to have out-of-wedlock births’ (Lott & Whitley, 2007, p. 324). Joyce (2004) challenges both correlation and mechanism. In response, Donohue and Levitt (2019) defend their causal claim with updated evidence, which is relevant to both correlation and mechanism. That both sides of this debate focus on evidence of correlation and evidence of mechanism provides some support for the view that Evidential Pluralism captures the structure of causal inference in the social sciences. Whether or not Donohue and Levitt are correct, the debate indicates that good social science research needs to consider both association studies and mechanistic studies when assessing causal claims.
New foundations for mixed methods research
Mixed methods research is now widespread in the social sciences and such research also considers a variety of evidence when assessing causal claims. It is important not to conflate Evidential Pluralism with mixed methods research: there are substantial differences between the two, as we shall explain. Nevertheless, we shall suggest that Evidential Pluralism can provide new foundations for those variants of mixed methods research that seek to establish causal claims.
Mixed methods research is usually defined as a methodology or a methodological orientation employing both qualitative and quantitative data, methods, or designs:
With mixed methods research, researchers combine elements of qualitative and quantitative research approaches (e.g., use of qualitative and quantitative viewpoints, data collection, analysis, inference techniques) for the purposes of breadth and depth of understanding, and for mutual corroboration (Johnson et al., 2007, p. 123).
[W]e defined mixed-method designs as those that include at least one quantitative method (designed to collect numbers) and one qualitative method (designed to collect words), where neither type of method is inherently linked to any particular inquiry paradigm (Greene et al., 1989, p. 256).
In mixed methods, the researcher (i) collects and analyses both qualitative and quantitative data rigorously in response to research questions and hypotheses; (ii) integrates (or mixes or combines) the two forms of data and their results; (iii) organises these procedures into specific research designs that provide the logic and procedures for conducting the study; and (iv) frames these procedures within theory and philosophy (Creswell & Plano Clark, 2018, p. 5).
Evidential Pluralism, on the other hand, does not appeal to the qualitative / quantitative distinction. Instead it features two other distinctions: (i) the distinction between evidence of the existence of a conditional correlation and evidence of the existence of an appropriate mechanism complex, and (ii) the distinction between association studies and mechanistic studies. It is true that association studies usually use quantitative methods or designs, while mechanistic studies often employ qualitative methods or designs. But this is not always the case. Mechanistic studies can have quantitative elements: e.g., when investigating mechanisms linking legalised abortion and crime rates, Donohue and Levitt (2001) appeal to various quantitative data and methods. Moreover, association studies can include qualitative elements: e.g., when examining the association between economic inequality and democratic transitions, Haggard & Kaufman (2016, pp. 102–141) employ qualitative methods such as process tracing.
It is also important to note that Evidential Pluralism is purely an account of the epistemology of causation, while mixed methods research is a methodology that sometimes invokes metaphysical presuppositions. In the social sciences, a variety of metaphysical stances are invoked to motivate the use of certain types of qualitative or quantitative study: for example, positivism, postpositivism, constructivism, interpretivism, and, as we have seen, critical realism. These metaphysical stances tend to be mutually incompatible, which creates a tension in mixed methods research. Accordingly, the options for the mixed methods researcher are to live with this tension, or to reject any appeal to ‘inquiry paradigms’—see the above quote of Greene et al. (1989, p. 256). Either way, mixed methods research apparently lacks coherent philosophical foundations.
A typical response to this dilemma is to appeal to pragmatism. This response is rooted in American pragmatism, especially the works of John Dewey, Charles Sanders Peirce and Richard Rorty (Cherryholmes, 1992; Johnson & Onwuegbuzie, 2004; Feilzer, 2010; Tebes, 2012), and it highlights the instrumental role of theories in inquiry (Johnson & Onwuegbuzie, 2004; Johnson & Gray, 2010; Morgan, 2014). According to the pragmatist response, both the mind-independent physical world and the constructed social and psychological world exist, and social reality is a product of both; social scientific research is value-oriented; and the aim of social scientific research is to solve problems. The claim is that social scientists do not have to choose between postpositivism and constructivism / interpretivism (Johnson & Onwuegbuzie, 2004; Teddlie & Tashakkori, 2009; Creswell & Plano Clark, 2018). They are free to choose the methods that best meet their needs and purposes and they can employ both quantitative and qualitative methods in their research (Johnson & Onwuegbuzie, 2004; Greene, 2006; Teddlie & Tashakkori, 2009; Creswell & Plano Clark, 2018).
Unfortunately, the pragmatist response provides rather weak philosophical foundations for mixed methods research: it motivates the consideration of mixed methods as one of several possible options, but does not offer any reason to think that mixed methods might be better than a single method on its own. As Greene and Hall (2010, p. 138) observe, ‘Whatever works; whatever can best engage and usefully inform the important practical problem at hand’ is the best methodology, from the pragmatist perspective. And Tashakkori and Teddlie (1998, p. 24) note, ‘Decisions regarding the use of either qualitative or quantitative methods (or both) depend upon the research question.’ In short, the pragmatist position merely justifies the inclusion of mixed methods in a portfolio of possible research designs—it does not provide normative grounds for using mixed methods. Therefore, the pragmatist position provides at best ‘an attractive philosophical partner for mixed methods research,’ as Johnson and Onwuegbuzie (2004, p. 14) put it.
Evidential Pluralism can help by providing normative grounds for using mixed methods. Evidential Pluralism is well motivated as a theory of the epistemology of causality, and it in turn motivates the use of both qualitative and quantitative methods. Quantitative methods employed by association studies can confirm both the existence of an appropriate correlation as well as the existence of an appropriate mechanism (channels C1 and C2 in Fig. 1). Quantitative methods can also feature in mechanistic studies that investigate particular links or features of the mechanism of action (channel M1). Other mechanistic studies can use qualitative methods to test and explore specific mechanism hypotheses (also M1). It is precisely because one should consider both association and mechanistic studies that one should pay attention to both quantitative and qualitative methods.
In addition, Evidential Pluralism can provide guidance on how to integrate quantitative and qualitative methods, which is often a challenge for the mixed methods researcher. Again, Fig. 1 provides the structure of the integration task. One point at which qualitative and quantitative methods need to be integrated is in the assessment of specific mechanism hypotheses: the question is the extent to which key features of relevant mechanisms are confirmed by qualitative and quantitative methods. Another point of integration is in the assessment of the general mechanistic claim that there exists a mechanism complex that explains instances of the putative effect in terms of instances of the putative cause and that can account for the extent of the observed correlation. As we see in Fig. 1, one needs to consider quantitative data from association studies at this stage, as well as the statuses of the specific mechanism hypotheses. Then there is the assessment of the claim that the putative cause and effect are correlated, conditional on any potential confounding variables suggested by background evidence or theory. At this stage, quantitative data will usually be most relevant, although qualitative methods may also have an influence through channel M3. Finally, the status of the causal claim depends on the statuses of the correlation claim and the general mechanistic claim, and it is at this point that all the qualitative and quantitative data are integrated. Parkkinen et al. (2018, § 7.1) provide some guidance on all these points of integration.
We see then that Evidential Pluralism can help to justify the use of mixed methods in causal analysis and to structure the data integration task. It is in this sense that Evidential Pluralism can help to provide coherent philosophical foundations for mixed methods research.Footnote 8 These foundations can be viewed as complementary to those provided by other approaches, such as pragmatism, or can act in a standalone capacity.
Weinstein’s study of insurgent violence is a good example of the use of mixed methods research to establish correlation and mechanism. Weinstein (2007) proposes a theory to explain differences between the ways in which rebel groups employ violence. He argues that resources and financing are one key causal factor of the strategies of violence: ‘rebel groups that emerge in environments rich in natural resources or with the external support of an outside patron tend to commit high levels of indiscriminate violence; movements that arise in resource-poor contexts perpetrate far fewer abuses and employ violence selectively and strategically’ (Weinstein, 2007, p. 7). Weinstein’s theory not only predicts the correlation between the initial endowment to which rebel leaders have access and their use of violence, but also identifies some mechanisms linking them. For instance, Weinstein argues that resources shape the membership profile of a rebel group, which in turn affects its internal organisation and strategies it uses in war. In order to support his theory, Weinstein integrates qualitative interview-based studies of the rebel groups and community-level social histories with statistical analysis of original newspaper datasets on patterns of violence in four case studies of rebel groups in Mozambique, Peru, and Uganda. The quantitative data provides solid evidence of correlation that supports his theory at the national level, while the qualitative data offers more nuanced mechanistic evidence at the subnational level. Note that Weinstein employs mixed methods both to test the general causal claim and also to assess specific mechanistic hypotheses.Footnote 9
Another example is Ivankova and Stick’s study of PhD students’ persistence. Ivankova and Stick (2007) investigate factors that contribute to students’ persistence in a remote-learning doctoral programme. In order to identify factors, they use a two-phase study, starting with a quantitative approach and followed by a qualitative approach. The roles of the two approaches are clearly stated as follows:
In this study, the quantitative data helped identify a potential predictive power of selected external and internal factors on the distributed doctoral students’ persistence and purposefully select the informants for the second phase. Then, a qualitative multiple case study approach was used to explain why certain external and internal factors, tested in the first phase, were significant predictors of students’ persistence in the program. Thus, the quantitative data and results provided a general picture of the research problem, while the qualitative data and its analysis refined and explained those statistical results by exploring the participants’ views regarding their persistence in more depth. (Ivankova & Stick, 2007, p. 97.)
Interestingly Ivankova and Stick largely avoid causal terminology—perhaps influenced by the legacy of Yule and other early social scientists. They explicitly classify their study design as a ‘sequential explanatory mixed methods design’, i.e., as first identifying factors associated with persistence and then finding explanations of these associations. But it is apparent that their task is really causal: they use their results to make a series of recommendations for how to improve persistence in such programmes, and this move would only be warranted if the factors they have identified are causes—rather than merely correlates—of persistence.
From the point of view of Evidential Pluralism, there is no need for any reticence here with regard to causal claims. This is because Ivankova and Stick have done what they need to do to confirm causality. They used a quantitative association study to identify factors correlated with persistence in the programme and then used a qualitative mechanistic study to provide evidence that there are mechanisms that invoke these factors to explain persistence (or drop-out). Taken together, these studies provide some good evidence for causality.Footnote 10 In situations such as this, an appeal to Evidential Pluralism might give researchers the confidence to draw causal conclusions.
In sum, Evidential Pluralism can provide coherent philosophical foundations for mixed methods research as applied to causal inquiry. It can also provide guidance on how to integrate these quantitative and qualitative methods.