1 Introduction

In the debate on QCA solutions types in Qualitative Comparative Analysis (QCA), only little attention has been given thus far to solution type strategies in multimethod research. This article aims at identifying the most appropriate QCA solution type when engaging in a multimethod design that includes QCA and in-depth process-tracing (PT), thereby combining cross-case claims (via QCA) with within-case causal relationships (via PT). We substantiate our arguments by drawing on a recently completed evaluation study that was commissioned by the Flemish ESF Agency in Belgium (Álamos-Concha et al. 2021). In this ESF evaluation study, we combined QCA with in-depth PT to assess the effectiveness of soft skills training in diverse organizations. Earlier research had shown that plenty of employees fail to adequately transfer the skills that they had learned in the training to the work environment. Therefore, combining QCA with methods to probe mechanistic causation enabled us to uncover under which combination of conditions (QCA) a training programme would lead to successful training transfer, and how (PT) this happened or ‘what worked’ in the successful cases (see Pattyn et al. 2020; Álamos-Concha et al. 2021).

In this themed issue, Haesebrouck and Thomann (2021) call attention to being explicit about one’s approaches to causality, which is even more important when engaging in a multimethod design, that is, a design in which QCA is but one part of the research. We conceive QCA as deterministic and relying on causes as difference-makers. Besides, we consider PT in its mechanistic variant, with its mechanistic understanding of causality (Baumgartner 2020). Scholars usually resort to such in-depth case-based research after QCA in an attempt to unravel the causal mechanism between a given (combination of) condition(s) and the outcome of interest, and to make causal inferences about how the mechanism actually operates in real world cases. We follow this sequence and anticipate on the mechanistic approach to causality when performing QCA. To be clear, in this article, a causal mechanism is understood as a system of interacting parts within which the activities of actors transfer causal forces from causes to outcomes (Machamer 2004; Machamer, Darden and Craver 2000; Waldner 2012; Beach and Pedersen 2016, 2020). Such a system approach comes along with in-depth process-tracing, that requires to detail the dynamic, productive elements of the causal process that links the parts together (Beach and Pedersen 2019, 246). We highlight that such a holistic approach fundamentally differs from a rather minimalist understanding of process-tracing, where these elements are merely depicted as causal arrows (Beach and Pedersen 2019). It is particularly this holistic, productive account of process-tracing that leads us to consider certain QCA solution types as more in alignment with the underlying mechanistic causal claim being made when engaging in a QCA-PT design.

Existing scholarship on QCA solution selection tends to favor either the enhanced intermediate solution (Schneider and Wagemann 2012, 278; Dusa 2019, 24) or the parsimonious one (Baumgartner 2015). However, both solution types can be problematic when adopting underlying assumptions of case-based designs and in-depth tracing of mechanisms as systems (Beach and Pedersen 2016; Goertz and Mahoney 2012; Waldner 2012). In this article, we shed light on three important and linked challenges to consider when selecting the most appropriate QCA solution in multimethod designs: omitted conditions, mechanistic heterogeneity, and generalization.

With Haesebrouck and Thomann (2021), we share the understanding that when working with mechanistic theory, QCA can solely find potential causes to be studied in a suitable population of cases for cautious generalization (Beach and Kass 2020, 9). From a mechanistic perspective, a process causal theory differs from counterfactual assumptions on what could have occurred in the absence of a condition (Beach 2018), since its primary aim is to infer whether a certain causal mechanism works as theorized in a particular case or within a group of (relatively) homogeneous cases (Beach and Pedersen 2016, 2019). For this reason, when anticipating the mechanistic approach to causality when performing QCA, the researcher must distinguish between causal conditions that can “produce something” and contextual conditions “that can be expected to affect the functioning of the mechanism” (Beach and Pedersen 2019, 254). In the same direction, the QCA solution type to proceed from for PT should guarantee that these causes and contextual conditions are indeed present at the level of the solution as a whole or in specific solution terms. Adding to this, one should be wary of causal heterogeneity at the level of mechanisms (i.e. “mechanistic heterogeneity”; Beach and Pedersen 2016, 2019). In addition, one should be vigilant for omitted conditions which can affect the correct functioning of a given mechanism (Falleti and Lynch 2009; Beach and Pedersen 2019; Khosrowi 2019). All such (interrelated) considerations are important to take into account when choosing the most appropriate solution type in a multimethod design. As we will argue, the conservative solution type is most appropriate to address the above-mentioned challenges of mechanistic heterogeneity, omitted conditions and generalization, at least when working with a system-understanding of causal mechanisms (aka in-depth tracing of mechanisms).

In the remainder of the article, we proceed as following. We first describe the current developments in scholarship about QCA solution types, and take stock of the key distinguishing features and merits of each of the solution types, which have thus far been mainly approached from the specific perspective of QCA (or CNA), thereby focusing more on one method than on the interaction between methods. This leads us to call attention to the above-mentioned three challenges that can emerge when engaging in a multimethod study that relies on a QCA-in-depth PT design. We discuss each of these intertwined challenges. Against these theoretical backbones, we subsequently introduce the above mentioned real-world QCA-PT evaluation study, which helps us to empirically illustrate why the conservative solution is most appropriate to mitigate these three challenges. We conclude the article by summarizing our key arguments and highlight the implications for other multimethod research designs comprising QCA.

2 The quest for the most appropriate solution type

2.1 Current state of the quest

The whole debate around solution types, as it has unfolded in the last few years and as discussed extensively by Haesebrouck and Thomann (2021), by other contributions to this themed issue, and by ourselves below, is actually rooted in issues that span beyond one’s approach to causality when using QCA and related methods, and also beyond the development of CNA. It derives from the foundations of QCA itself, as a research approach. Taking a few steps back, it is obvious that QCA, as conceived by Charles Ragin in his successive foundational pieces, is itself a sort of hybrid, integrative or ‘mixed’ research approach (Rihoux, Álamos-Concha and Lobe 2021), grounded both in holistic case-based research—which prioritizes complexity—and in analytical-formal thinking—which prioritizes parsimony.

From this perspective, some distinct arguments may be issued in favor of each one of the three possible QCA solution types: intermediate, parsimonious, but also complex (or ‘conservative’) solutions which are now frequently overlooked in the debates. Let us first consider the conservative solutions, i.e. the longest or ‘descriptive’ solutions not exploiting any logical remainders, even though such solutions are by now largely out of fashion. And yet: the interest of such solutions is at least threefold. First, they enable one to perform a first, case- and/or theory-informed interpretation after the QCA minimization procedure (Rihoux and De Meur 2009, 58). Such an interpretation should obviously be extra cautious in terms of ‘causal’ statements, even though in principle one should have formulated, upstream, some causal statements of some form linking each condition (and some combinations of conditions, ideally) to the outcome. Second, and perhaps more importantly, conservative QCA solutions shed light on those conditions that are not comprised in the solution terms. In other words: they do provide a first, albeit modest, step towards parsimony, namely by excluding some segments of theory through initial logical minimalization. Third and not least, such solutions may be manually rewritten in order to factor out some conditions that seem to play a more central role (Rihoux and De Meur 2009, 58–59; Rihoux and Lobe 2009). Note that these factored out conditions will frequently also be present in the intermediate and parsimonious solutions.

From a logical perspective, and from the perspective of QCA’s development as an approach and a set of techniques, parsimonious solutions may be considered as the opposite option to the conservative one, privileging parsimony at all cost, i.e. the shortest possible solutions. We shall not discuss in detail the nuts and bolts and implications of such a solution type, as they are amply discussed in this issue and in several reference pieces (e.g. Schneider and Wagemann 2012; Rihoux and De Meur 2009, 59 ff). Apart from the issue of causal statements that may (or may not) be derived from such solutions, one core difficulty lies, naturally, in the tricky issue of contradictory simplifying assumptions that may be produced when exploiting all logically useful (from a minimization perspective) logical remainders.

As for the intermediate solutions: these were initially developed in order to avoid some caveats of the parsimonious solutions, as argued by Ragin and Sonnett (2005) in their seminal article. In a nutshell, the whole idea is to restrict the use of logical remainders to those that are ‘plausible’ or ‘easy’, in particular from a theoretical perspective (see a detailed discussion in Schneider and Wagemann 2012). This can be seen as a sort of middle way between the conservative and parsimonious solutions, and also as a sort of cautious, theory-informed way to come closer to identifying the ‘causal core’ (as labelled by Fiss (2011)) conditions. Note also that, at least in intermediate-N designs and when one has gained a sufficient level of ‘intimacy’ with the cases, the handpicking of the easy logical remainders can (and should) be further improved by case-based knowledge (Rihoux and Lobe 2009).

At this stage of expansion of QCA empirical applications, it is obvious that intermediate solutions have taken the lead (Rihoux et al. 2013), probably also because the protocol and good practices for obtaining such solutions have been quite well consolidated and made accessible via textbooks and software functionalities. Our argument would nonetheless be that there is something to learn from each one of the three solution types. From a pragmatic perspective as well, and as QCA is best conducted in an iterative manner (Rihoux and Lobe 2009; Rihoux 2020), a general recommendation of ours would be to produce all three types of solutions, even if one specific solution type is eventually privileged for the final QCA analysis and further interpretation.

2.2 The quest in a multimethod context

The preceding section discussed solution types from the perspective of QCA as a single, stand-alone method. Indeed QCA has been, thus far, most frequently used in monomethod designs, at least in article-length publications (Rihoux et al. 2013). Yet, in the last few years especially, some quite refined designs inserting QCA in a mixed- or multimethod design have been developed and implemented (Rihoux, Álamos-Concha and Lobe 2021). We define here mixed methods research as research that integrates at least one qualitative and at least one quantitative data collection and/or data analysis method (Johnson, Onwuegbuzie and Turner 2007), and multimethod research as a broader term that includes any use of more than one type of method, including, for example, two qualitative methods or two quantitative methods.

Within the framework of this article, we are focusing specifically on QCA in multimethod research designs (cross-case-analysis-first designs followed by within-case analysis), and more precisely on a design in which QCA, as a data analysis method, is sequenced with in -depth PTFootnote 1—but following a mechanistic approach to causality when performing QCA—the logic of the sequence is quite straightforward: Technically, QCA is used instrumentally as a method to identify the core combinations of conditions that lead to the outcome of interest in a given set of selected cases, thereby identifying cross-case patterns; then PT is conducted on typical cases, i.e. those positive cases that share the same combination of conditions, the presence of the outcome and also similar features (contextual conditions). This enables one to provide evidence (if applicable) of a causal relationship between a given combination of conditions and the outcome, and to understand how that combination of conditions produces the outcome by tracing the causal mechanisms (as systems) linking them together under certain contextual conditions.

Established literature on this particular sequence supported us in taking this stance (Beach 2018; Beach and Pedersen 2016, 2019; Beach and Rohlfing 2018; Goertz 2017; Williams and Gemperle 2017). While also the inverse sequence has merits, a QCA first study enables to run a cross-case analysis to identify whether there is indeed evidence for a cross-case relationship. These results can in turn guide a follow-up PT in which one can focus on specific sufficient terms and identify all the parts of the mechanisms and their manifestation in typical cases (Beach and Rohlfing 2018: 16; Schneider and Rohlfing, 2013). By engaging in this sequence, one can gain cross-case knowledge of the population of cases, which helps to better understand the combinations of conditions in which a given mechanism can be operative (Beach and Pedersen 2019, 6). Also, it makes PT more robust by empirically identifying mechanisms in a more holistic and robust way and by identifying the boundaries of mechanisms (Bennett 2010, 209, King Keohane and Verba 1994, 86).

As can be derived from the above, in this article, we rely on theory-focused, theory-testing variant of in-depth PT, which enables the testing of QCA-informed conjectures about plausible mechanism(s) operating between the cause(s) and the outcome (Beach and Pedersen 2019, 245). Theory-testing in-depth PT, as its label suggests, is geared towards “testing whether a hypothesized causal mechanism exists in a positive case or set of positive cases by exploring whether the predicted evidence of a hypothesized causal mechanism exists in reality”Footnote 2 (Beach 2017, 18). Consistent with the approach to mechanisms as systems, and in line with realistic evaluation (Pawson and Tilley 1997), we conceive them as consisting of “entities that engage in activities. Entities are factors (actors, organizations, or structures) engaging in activities, whereas the activities are the producers of change or what transmits causal forces or powers through a mechanism” (Beach and Pedersen 2019, 38). We return to this later in the article when illustrating how we empirically approached this in practice.

In terms of the main operations in the PT part of the protocol, the main steps consist of (1) selecting a typical case that is particularly relevant for the purpose of generalization, (2) conceptualizing the hypothesized causal mechanism between the configuration of conditions and the outcome, (3) implementing the PT empirical tests for each part of a causal mechanism, that is, the operationalization of the conceptualized causal mechanism (via fingerprints, priors, theoretical uncertainty and uniqueness), (4) collecting empirical data based on the empirical predictions (fingerprints) for each part of the mechanism, as observations, and finally (5) assessing whether these observations may be considered as evidence (i.e. reliability, independence of sources) of the presence of each part of the causal mechanism and of the causal mechanism as a whole (empirical certainty and uniqueness) (See more details in Beach and Pedersen 2019).

Such a stepwise unpacking of causal mechanisms connecting configurations with the outcome leads to finding that parts worked as theorized, but others did not. There are two possible types of results when choosing typical cases: (1) No evidence on the within-case relationship in the case selected, implying that the analysis can be terminated with negative causal inference, or (2) evidence on the within-case relationship is available, which confirms that the theorized mechanism is indeed at play (the analysis can then be terminated with positive causal inference). (See more details in Beach and Rohlfing 2018, 16).

Up to the best of our knowledge, no systematic review of such QCA-PT published applications has been conducted thus far, and full-fledged analyses of this type are still scarce.Footnote 3 Therefore no standards exist (yet) as of which QCA solution type to choose when QCA is to be sequenced with PT–even if one also adopts a mechanistic approach to causality in the QCA part of the research. Sequencing QCA and in-depth PT actually raises some thorny challenges if one takes causality seriously (Beach and Pedersen 2016, 2019; Beach 2017; Schneider and Rohlfing, 2013; Rohlfing and Schneider 2013), as discussed extensively here below.

2.3 Potential challenges common to a QCA/PT setting

Of the challenges possible, we will focus here on three linked issues that may lead to flawed causal inferences (see Fig. 1): the issue of omitted conditions for disentangling the causal chain, the problem of mechanistic heterogeneity and the issue of generalization.

Fig. 1
figure 1

Source Own elaboration

Challenges in a QCA-PT multimethod design

Full size image

2.3.1 Omitted contextual conditions

In QCA-in-depth PT designs one should distinguish between causal and contextual conditions. While both types of conditions are causally relevant in QCA, they must be distinguished when combining QCA with in-depth PT. In PT, causal conditions are ‘triggers’ of processes that have a productive relationship with the outcome (Beach and Pedersen 2016, 89), whereas contextual conditions are passive and do not trigger a mechanism. Instead, they merely enable the functioning of the causal mechanism to produce the outcome (Beach and Pedersen 2016, 89). This distinction is especially made by scholars interested in the study of causal mechanisms (Beach and Pedersen 2016, 2019). For instance, firm size can have a role in a given process but it does itself not trigger a mechanism. We thus conceive it as a passive contextual condition, rather than a causal one. In QCA this distinction is seldom made, with the exception of the two-step QCA variant in which one distinguishes remote and proximate conditions (but various approaches to the conceptualization of remote and proximate exist) (Schneider and Wagemann, 2006; Schneider 2019), and some recent contributions on QCA-PT multimethod designs (see Pattyn et al 2020).

To be sure, contextual conditions have a central place in a system understanding of causal mechanisms, because they “determine whether a causal relationship functions as theorized, be it a CO relationship or a mechanism in between linking C and O” (“Beach and Pedersen 2016, 89). They play a key role in the integrity and functioning of the mechanism, because one may expect that a given mechanism can only operate as theorized under certain contextual conditions in typical cases. Thus, in the absence of certain contextual conditions, the outcome could be prevented from occurring, or change how it works.

When performing in-depth PT after getting a truth table analysis we start with a QCA model that is reduced in terms of the number of conditions kept for the minimization procedure. This is of course, part of the QCA core aim: parsimony, as mentioned above. However, the exclusion of conditions can be detrimental to the functioning of a causal mechanism. Going further, if we identify a process that works within a certain context in which we expect it to occur, the omission of such contextual conditions can affect the causal process that plays out across different cases and that seems to be homogeneous at the cross-case level (Bunge 1997; Falleti and Lynch 2009; Gerring 2010; Goertz and Mahoney 2009). Contextual differences therefore need to be taken seriously because they might change the causal dynamics between the conditions and the outcome.

2.3.2 Mechanistic heterogeneity

The omission of contextual conditions and the possible contextual differences across cases can lead us to the problem of mechanistic heterogeneity. At the cross-case level, with QCA, cases are expected to be heterogeneous in terms of potential equifinality, although we assume the existence of causal homogeneity in the set of cases sharing the same configuration or term. However: when omitting contexts, such contextual differences may result in differences at the level of mechanisms. Mechanistic heterogeneity is understood here as the presence of the same cause(s) linking the same outcome in different cases via different mechanisms and different contexts (Beach and Pedersen 2019, 54; Beach and Pedersen 2016, 41; Schneider and Rohlfing 2016, 555). Yet, we expect some degree of mechanistic homogeneity when studying the process linking causes and outcome, which means that the same expected mechanism works in all cases that share the same (combination of) causal condition(s), outcome, and contextual conditions. Only by avoiding mechanistic heterogeneity across similar cases will we be able to make causal inferences about how a certain process took place in certain group of cases.

2.3.3 Challenges in generalization

As mechanistic heterogeneity can be produced by differences in contexts, we may theorize our causal mechanisms by specifying the context within which the given mechanism is expected to operate (Beach and Pedersen 2019, 54). This, of course, limits the scope of generalization, but the researcher may group the cases according to the causes being present within a common context. In other words: causal inferences can only be made in groups of cases sharing same causes, outcome, context, and mechanism. Of course, there may be a situation in which a causal mechanism follows a different pathway at some stage of the process, because a given contextual condition that was omitted in the QCA model (unknown condition) or in the minimization (known condition) seems to play a relevant role in the process. This particular situation could also enable generalization if more than a single case shares such a pathway. The important thing here is that the causal mechanism does not necessarily break down. Instead it can have a common start, i.e. common parts at play in the beginning of the process and also at the end of thereof, enabling the occurrence of the outcome, but some stages in between can be different because of the sensitiveness to context that fractionates the process in different paths.

Generalization is directly linked to the level of abstraction of a causal mechanism. Mechanisms in their more minimalist version are theorized in much more abstract language, i.e. the theoretical or empirical process is not really unpacked in any detail (see more in Beach and Pedersen 2019, 2016; Bennett and Checkel 2014; Falletti and Lynch 2009; George and Bennett 2005). This is a good choice when the researcher wants to explore the existence of evidence for certain mechanisms in early research process, because it facilitates to focus the attention on a handful number of mechanisms (Beach and Pedersen 2019, 3)—but it leads to make modest analytical inferences. By contrast, as mentioned above, mechanisms as a system are understood in a holistic way (Beach and Pedersen 2020, 38; Sawyer 2004), with entities engaged in activities in a productive way within a process where parts have no independent existence to produce the outcome (Beach and Pedersen 2019; Machamer 2004; Machamer, Darden, and Craver 2000). They are found in real world cases that are context sensitive.Footnote 4 The analytical value of in-depth PT here is in (1) identifying logical shortcomings in our theories; (2) learning about how a process works sheds light on the contexts that must play a key role for the mechanism to work; and (3) learning about the causal power of a condition to be able to trigger a mechanism.

The more abstract or superficial a mechanistic theorization, the more likely it is to cover more cases to explain and understand and to lower the degree of heterogeneity at the mechanistic level. However, the lower the level of abstraction, (i.e. a more detailed case description of the key steps in the process), the more difficult to cover cases and generalize, because the degree of heterogeneity is also potentially higher, other things equal. The approach to take here—abstract versus more context-bound—‘unpacked’ mechanistic theories depends on the aim of the multimethod research and also, of course, on the empirical material at hand to study and test the causal mechanism.

3 The quest for the appropriate solution type in practice

Having theoretically discussed the three key challenges, the question is now which solution type is most appropriate to address them in practice and why. The remainder of the article focuses on the merits of the conservative solution type, despite this type being a bit out of fashion in stand-alone QCA studies. As mentioned, we substantiate our arguments by relying on our experience in a recently completed evaluation study (Pattyn et al. 2020; Álamos-Concha et al. 2021) that was commissioned by the Flemish ESF (European Social Fund) Agency in Belgium. Before demonstrating the potential of the conservative solution in Sect. 3.2., we concisely introduce the key features of the evaluation study (Sect. 3.1).

3.1 Empirical illustration: training transfer effectiveness in Flemish firms

The evaluation study was launched in 2017 and was concluded in August 2020. The trainings that were to be evaluated took place at the end of 2018 and at the start of 2019. We contacted 203 employees, from 10 different organizations, that had followed a training on leadership skills or stress management and asked these employees to fill in a survey questionnaire before and after the training. Furthermore, we performed interviews with several trainees who had successfully transferred their training on leadership skills. We also collected additional documents (e-mails, attendance sheets, …) which were mostly used for the PT part of the study, as well as diverse data about the respective organizations. In the QCA, we solely worked with those employees (i.e. our cases) for which we had complete information on all conditions. This led to a total sample of 51 cases. In applying the possibility principle for selecting negative cases (Goertz 2005), one case was identified as irrelevant because it did not have a positive score on any of the potential conditions that could trigger learning transfer. Out of the 50 remaining cases, 15 had successfully transferred their training.Footnote 5 It would exceed the scope of the article to explain both the QCA and the PT analyses in depth.Footnote 6 We restrict ourselves to a brief explanation of (1) the QCA model and expected causal mechanisms; (2) the results of the QCA analysis; (3) and the theorized mechanism.

3.1.1 QCA model and causal mechanisms

Based on an in-depth review of the learning organization literature, we developed a QCA model presented in Fig. 2 representing how we expected to explain ‘effective employee social skills training transfer’ (i.e. the outcome). The model contains (1) conditions acting as contexts or ‘enablers’ of training transfer and (2) the causal conditions that trigger possible mechanisms to produce training transfer effectiveness. The causal conditions form a conjuncture of four conditions that are hypothesized to jointly act as sufficient to lead to a successful outcome, within certain contexts.Footnote 7

Fig. 2
figure 2

Source Own elaboration

QCA model

Full size image

Below, Fig. 3 displays the process within a mechanism-centered design (see Beach and Rohlfing 2018, 20) that is triggered by the conjuncture of causal conditions solely, i.e. the causal mechanisms that play a key role in the production of the outcome without including the contextual conditions. The challenge here is if we aim to determine the mechanism or mechanisms that tie the conjunction to the training transfer effectiveness or theorize it in a mono-mechanistic fashion—i.e. only one mechanism linking the condition with the outcome. We have opted by the conjuncture of mechanisms rather than mono-mechanistic expectation (see more in Beach and Rohlfing 2018). In empirical terms, the mechanisms are independent of each other because the presence of each condition suffices to launch a single mechanism. However, according to the theory the outcome training transfer effectiveness requires the presence of all four mechanisms which implies that they form a conjunction.

Fig. 3
figure 3

Source Own elaboration

Four-way conjunction of mechanisms

Full size image

3.1.2 QCA analysis

Based on the crisp set QCA analysis,Footnote 8 no single condition turned out to be individually necessary for training transfer effectiveness (see Appendix 1). The only condition closest to being necessary was the presence of the ‘training programme as active learning method’ context. However, with a very low coverage (31%) and relevance of necessity score (16%), this condition doesn’t have much explanatory potential. Neither did we identify disjunctions (or logical unions) of two or three conditions that were necessary for transfer effectiveness.

Turning then to the analysis of sufficiency, we found that no single condition was sufficient by itself for successful training transfer (for more details see Appendix 2). Instead, our analysis confirmed our expectation that conditions act in combination with other conditions to make training transfer successful, and that many different pathways can produce ‘success’. We identified no less than eight pathways, but with each configuration only covering a few cases. Table 1 presents the conservative solution for the individual employees who experienced effective training transfer (i.e. success cases, displaying the positive outcomeFootnote 9).

Table 1 Conservative solution (eight terms) for the positive outcome
Full size table

By way of illustration, one can read the second configuration as: when trainees received support from their peers and from their supervisors, without experiencing ‘sense of urgency’ and ‘relapse prevention and goal setting’, they were capable to transfer the content of the training programs at work. As facilitating contexts for this scenario, it turned out that it was important that the training programme made use of active learning methods and was set up in such a way that it featured identical recognizable elements of the working environment. Interestingly, the participants to whom this applied were not given any compensations in terms of workload to make sure that they had time to follow the training.

3.1.3 Process-tracing

Our findings show that with each of the configurations only applying to one or two cases, generalization is particularly challenging. Still, the added value is to know “what worked” in similar cases of success and to be able to replicate these practices to other cases sharing similar features. This being said, the extent of generalization also depends on the evidence collected. An in-depth PT application implies that core elements of the causal mechanism are to be unpacked theoretically and studied empirically in the form of so-called traces left by the activities associated with parts of the process (Beach and Pedersen 2019, 38).Footnote 10 It can be argued that the more supportive evidence one finds in more typical cases, the stronger the confidence of the finding for other unstudied typical cases (Beach and Rohlfing 2018, 15).

Let us observe the fourth configuration displayed in Table 1 and the causal mechanisms at play. Our QCA model in Fig. 2 proposes eight conditions, where four are triggers of causal mechanisms and four are contextual conditions that enable the well-functioning of the process. As mentioned earlier, this distinction is core in in-depth PT but not in QCA, where both conditions and contexts are considered causally relevant. In our empirical example, the cases N2 and B3 display a conjunction of two mechanisms that jointly produce training transfer effectiveness in this pair of cases. With every configuration being different, each one triggers a particular processes.

Empirically speaking, to know how many conjunctural causal processes are operating in our conservative solution, one can just observe how many configurations are displayed by the solution: eight configurations (terms), each one of which constitutes a conjunctural causal process. Each conjunctural causal process contains a number of conditions triggering a number of mechanisms. Figure 4 visualizes how configuration 4 constitutes a two-way conjunction of mechanisms triggered by two causal conditions within four contexts.

Fig. 4
figure 4

Source Own elaboration

Two-way conjunctions of mechanisms

Full size image

To understand how a conjuncture actually unfolds process wise, one best turns to the literature. For instance, and again considering Fig. 4, organisation learning scholarship hinted us at the importance of ‘self-management intervention’ and ‘signaling and retention’ in training transfer effectiveness. ‘Self-management intervention’ is defined as a series of methods or behavioral techniques oriented to facilitate positive transfer (Rahyuda, et al. 2014, 421). It has been studied mainly as occurring after training, however when linked to the self-efficacy its influence can be observed in the early stages of training transfer. Its definition integrates a new dimension: self-management in the acquisition of knowledge, going beyond retention of knowledge and also focused on the way in which trainees acquired correctly the knowledge and learn through self-efficacy (cfr. Gist et al. 1991, 837). ‘Signaling and retention’, is triggered by the supervisor support where they can use to influence the behavior of employees (Govaerts et al. 2017; Baldwin and Ford 1988). For instance, supervisors who are involved in the training, who know about the content and learning objectives, and who expect their employees to put the training to use, can send signals about the importance of training. These signals in turn can influence the efforts trainees put in retaining the training content, and in applying it in their work. It was particularly the ambition of our research project to disentangle the entire process of these complex mechanisms, and to test it. As it turned out from our PT analysis, the self-management mechanism consists of eight building blocks. Signaling and retention on its part consists of six building blocks which, in a productive manner, have connected the configuration with the outcome (training transfer).

Below Fig. 5 illustrates the building blocks of the conjuncture of the two causal mechanisms leading to training transfer effectiveness. For the purpose of this article, additional data are given in Appendices 3 and 4 where the single mechanism “signaling and retention” is fully unpacked. Appendix 5 in turn illustrates how we evaluated the evidence of this mechanism to conclude whether the within-case relationship is present or not. By way of illustration, we zoom into part 5 of the motivation to generalize building block.

Fig. 5
figure 5

Source Own elaboration

Building blocks of two-way conjunctions of mechanisms

Full size image

3.2 The merits of the conservative solution in a QCA/in-depth PT design

3.2.1 From omitted conditions to theoretical integrity

As discussed above, in our evaluation study, we distinguished between causal and contextual conditions. Again, the latter are core to the study of causal mechanisms, as a given outcome could be prevented from occurring if these contextual conditions are not present. This basic fact also requires that one should particularly be wary for known contextual conditions that are omitted in the QCA minimization. With only the conservative solution guaranteeing that all known contextual conditions that play an enabling role for particular groups of cases are included in the solution as a whole, one has no choice but to proceed from this more complex solution. As such, one does not need to sacrifice theoretical richness either.

As an illustration, let us consider again the fourth solution term indicated in Table 1i.e. [peer support* SUPERVISOR SUPPORT*RELAPSE PREVENTION AND GOAL SETTING*IDENTICAL ELEMENTS*TRAINING AS ACTIVE LEARNING METHOD*AUTONOMY *BALANCED WORKLOADFootnote 11], of the conservative solution. Let us suppose that the ‘balanced workload’ contextual condition would have been omitted from this QCA solution term. This would have probably been the case if we had selected the parsimonious solution. Such an omission would not be inconsequential for the functioning of the mechanism. As clarified in Appendix 3 outlining the mechanistic process, part 3b of the Facilitating training climate building block refers to ‘balanced workload’. As stated, 3b involves that “In parallel supervisor enables employees to follow the training by taking over the workload during the training period, with the aim that employees can be focused on learning the training content”. Should ‘balanced workload’ be absent from this QCA solution term, this would also imply that part 3b would not be present, resulting in a break-down of the theorized causal mechanism.

Approaching the same challenge from another lens: when studying causal mechanisms as systems or in-depth PT, the system as such needs to maintain its integrity. As Cartwright (2007, 239) argues: “There are any number of systems whose principals cannot be changed one at a time without either destroying the system or changing it into a system of a different kind”. Only the conservative solution offers the possibility to study causal mechanisms as a system, keeping the integrity of the principals of such a system. In contrast to the parsimonious solution, the QCA minimization process will not ignore known contextual conditions, at least in the solution as a whole (i.e. considering all the solution terms), but the conservative solution could still ignore particular contextual conditions in specific solution terms.

If a theorized mechanism did not operate as theorized in a particular case, this should result in a revision of the contexts that were omitted in the minimization and that could exert an impact on the process (Beach and Pedersen 2019, 114). As sketched above, to analyze the ‘signaling and retention’ causal mechanism, we selected cases that helped us to test the process linking the configuration (C) and training transfer effectiveness (O). The theorized mechanism presented in Appendix 3 can be conceived as the ideal-type process, given the presence of all contextual conditions derived from theory. All these contextual conditions turn out to be present for cases N2 and B3 (see Table 1), which suggests that this mechanism can be generalizable to both cases or other cases not included in the study that share similar features.

3.2.2 From mechanistic heterogeneity to mechanistic homogeneity

The same line of reasoning applies when considering the risk of mechanistic heterogeneity. As the conservative solution is more descriptive, it will better enable us to keep the integrity of the theoretical richness and therefore unpack a less abstract causal mechanism (more system oriented) within the group of positive cases. In Appendix 3, the N2 and B3 cases display causal homogeneity both at the level of causal conditions and at the level of the causal mechanism. The system approach enables us to thoroughly understand the whole story via in-depth case studies, thereby gaining deeper explanatory knowledge of the causal relationships and making stronger causal claims about what worked in both specific cases. As seen in Table 1, there are no omitted contexts in the fourth term of the conservative QCA solution: the four of them are present in N2 and B3, and it seems that they played a key role in the productivity of the process. It is particularly the presence of the same causes, the same causal mechanism and the contexts that enables us to confirm the existence of mechanistic homogeneity.

As also documented in Appendix 3, the signaling and retention causal mechanism is built up on the idea that when supervisor support takes place, this triggers a causal process that produces effective training transfer, during and after a training program and within a particular organizational context. Signaling and retention mechanism acts in the learning and performance stages of training. We thereby disentangled the process as a complex mechanism consisting of a single pathway, with a conjuncture of conditions, where supervisor support is the trigger of such a mechanism (consisting of six building block and seven parts). The process of signaling and retention starts in the learning process, with the supervisor ascribing importance to the training program and taking initiative to let the employees follow the training. The employees react by putting the training in their agenda. (ascribing importance of training building block). Everybody follows the training in group, in part because it was mandatory to do so by the supervisor (following the training building block). The supervisor enables employees to follow the training by taking over the workload during the training period. Employees can therefore focus on learning the training content (facilitating learning climate building block). This leads to an intermediate outcome, where a sort of organizational climate where employees perceive the importance of the training for their job is created, and where employees acknowledge the engagement of the supervisor encouraging this goal. Thanks to the perceived relevance of the training, the employees following the training feel motivated to use the learned content and to discuss with peers. Employees also try out/use the training in tasks-related matters, thereby keeping the level of motivation that ‘they just have to try to learn’ within an environment of trust and cohesion (motivation to generalize building block). Supervisors keep on reminding to use the training and provide feedback on the tasks related to the training application (keeping it alive signaling building block). [There is a feedback loop between the motivation to generalize and keeping the training alive], i.e.. Finally, thanks to the peer-supervisor engagement and trust, post-training evaluations feedback is implemented by supervisors until task-oriented new knowledge is retained and improved in its application by employees (increasing generalization building block). This process leads to training transfer effectiveness where employees are capable to use the learned knowledge (content, skills or attitudes) acquired in the training context to the workplace in the long-term.

To summarize, the above described building blocks of the ‘signaling and retention’ causal mechanism reveal part of the story about how employees transfer the training content acquired in the training program to the job context. This mechanism is integral and case specific but generalizable to other cases within the same configuration (fourth term of the QCA solution).

3.2.3 From flawed generalization to valid causal claims

As demonstrated above, working with the conservative solution to then investigate causal mechanisms as system can be a good way to deal with the issues of generalization, enabling more homogeneity at the causal level and taking care of avoiding omitted known conditions. This is because we theorized our causal mechanism by specifying the context within which it was expected to operate. Particularly, in the fourth term of the conservative solution, we have the same causes, same contexts, same causal mechanisms and same outcome taking place in the same cases.

Let us illustrate this situation as follows. The causal mechanism as system understanding contains seven parts acting in a productive relationship and together leading to the outcome (see Table 2). This mechanism explains and helps to understand how the process of training transfer took place in two particular cases.

Table 2 Causal mechanisms as a system (low level of abstraction)—4th term,
Full size table

Table 3 shows what the mechanism would be like for the second term [PEER SUPPORT*SUPERVISOR SUPPORT*sense of urgency*relapse prevention and goal setting*IDENTICAL ELEMENTS*TRAINING AS ACTIVE LEARNING METHOD*balanced workload]. The cases that are members of this configuration are B2 and K2.

Table 3 Causal mechanisms as a system (low level of abstraction)—2nd term,
Full size table

As discussed above, part 3b of the mechanism cannot occur when a balanced workload is not playing a role in enabling employees to be focused on learning the training content. The absence of part 3b can neither produce the appropriate organizational climate where employees would perceive the importance of the training (Intermediate outcome). This, in turn, prevents the employees from feeling motivated to use the learning content and to discuss it with peers (part 4). The lack of motivation will also prevent employees from using the training in task related matters and, further, employees will not be able to keep motivation alive or to create a climate of trust and cohesion (part 5). This situation will also prevent that the supervisor can act as a facilitator, because keeping motivation alive implies time management with the training-application process (part 6). Finally, post-training evaluations are not possible to occur under a workload scenario (part 7). And yet: the outcome was still produced for those case members of this configuration. The causal mechanism playing a role here can thus look different from the mechanism illustrated in Appendix 3. This suggests the existence of other causal dynamics producing effective training transfer that can be triggered by the role of peers in interaction with the supervisor.

To legitimize our choices, we followed the PT protocol developed by Beach and Pedersen (2016, 2019). We theorized the causal mechanisms (identified in previous QCA phase—as illustrated in Fig. 3) after getting the truth table, and we tested them by aggregating different type of data to each observation, and evaluated the empirical value of the different observations, and the reliability of the sources. This enabled us to minimize potential measurement errors (more details in Álamos-Concha et al. 2021).

As can be seen from this illustration, working with the conservative solution to study causal mechanisms is not such a straightforward strategy. It requires a clear purpose to combine QCA and in-depth PT as research strategies, an evaluation of the QCA model, a close scrutiny of the QCA solution, of the contexts and conditions possibly omitted or not considered in the solution terms, and of the number of cases sharing causal homogeneity for generalization purposes. A trade-off then needs to be made when aiming to generalize to the whole population of positive cases. As illustrated, the safer strategy is mapping the cases according to their membership in causes, contexts, mechanism and outcome. Thus, generalization is possible in those cases in which the same cause(s), the same context(s), the same mechanism and the same outcome are present (i.e. typical cases, positive cases). Even if the number of typical or positive cases grouped is low, the added value is to gather evidence of the presence of the mechanism in the within-case relationship and to know ‘what worked’ and what ‘did not work’ in such cases.

4 Conclusion: conservative QCA solutions for progress

The purpose of our article is to contribute to the discussion on which QCA solution type is most appropriate, in terms of causal validity, when engaging in a multimethod design that includes QCA and in-depth PT. We specifically focused on the design in which QCA is instrumentally applied first—following however a mechanistic approach to causality when performing QCA—and is followed by PT. With QCA we identified potential causally relevant conditions at the cross-case level—making PT more robust—whereas with PT, we unpacked the causal process triggered by those conditions at the within-case level.

Based on our empirical real-world illustration (evaluation study on the effectiveness of soft skills training in diverse organizations), we have identified three methodological challenges arising when combining QCA and in-depth PT: the question of omitted conditions, mechanistic heterogeneity, and generalization. We have distinguished between causal conditions proper, i.e. the triggers of the causal process leading to the outcome, and contextual conditions, i.e. the enablers of the correct functioning of such a causal process. This distinction can be tapped via QCA if one opts for the most appropriate type of QCA solution in order to study causal mechanisms. The bottom line is: the more complex the solution, the more theoretical richness is embraced at the level of the mechanism, and the more homogeneous the population of cases, as the latter then share the same causes, contexts, mechanism and outcome. Going for a longer QCA solution, i.e. the conservative one, enables one to know how the actual causal pathway actually worked in the positive cases, to avoid or at least mitigate the problem of mechanistic heterogeneity, and to facilitate generalization across similar cases. We have also illustrated that the quest for parsimony via the minimization process, i.e. going for intermediate or parsimonious QCA solutions, is not in line with in-depth process-tracing as system-understanding. Indeed, suppressing some conditions may affect the functioning of the causal mechanism, leading to flawed mechanistic causal claims and impeding generalizations about mechanisms.

Let us highlight three avenues for further discussion and future research. First, as we have re-introduced the conservative QCA solution and argued about its merits specifically when combining QCA and in-depth PT, it would be worth reexamining other potential advantages of such a ‘long’ and ‘complex’ QCA solution when using QCA as the single or main data analysis method. One potentially useful strategy, to be carefully weighted in terms of pro’s and con’s, could be to produce an ‘almost conservative’ QCA solution, i.e. only considering the ‘easiest’ logical remainders, thereby gaining a tiny bit of parsimony. Another, more case- and researcher-driven strategy could be to stick to the conservative solution and to exploit more systematically the different (manual) factoring out options, so as to focus one’s attention on some ‘key drivers’ somewhere along the path leading to the outcome.

Second, one should further examine the implications of going mixed/multimethod and including QCA in such a design. When combining QCA with in-depth PT, we anticipatedly adopt the mechanistic approach to causality when implementing QCA. By doing so, QCA is then used strategically as an instrument to identify potential relevant causes, but not necessarily as a method to establish a causal relationship (Beach and Kass 2020), making PT more robust via the production of the truth table. In-depth PT, thanks to this design maintains the integrity of a causal mechanism in a homogeneous population of cases and sheds light on how things worked in a holistic process between causes and outcomes.

Third and finally, we are facing an analytical trade-off when trying to exploit the strengths of both methods: from the QCA perspective, there is indeed added value going for a more abstract or minimalist understanding of causal mechanisms, and hence to go for more parsimony via ‘shorter’ QCA solutions. This nevertheless runs against the foundations of in-depth PT, but not necessarily against the minimalist understanding of PT, which does not include a ‘deep’, ‘long’ and empirically demanding conception of causality. In fact, researchers could apply a minimalist variant of PT in order to explore the field and have an idea that there might be something to look at when engaging in in-depth within-case analysis, because in-depth PT requires considerable resources to start with. Following this design, QCA would then not be applied as an instrument but as a counterfactual, set-theoretic method followed by a minimalistic PT. Thus, if, after the QCA, “within-case evidence is found for one or more mechanisms, this narrows the candidate field down to a manageable size, enabling the researcher to turn to more in-depth tracing of one or more causal mechanisms in more detail” (Beach and Pedersen 2019, 34).

All things considered, the broader issue at stake is: what do we lose and what do we gain when combining these two methods, in a mixed—or multimethod design? We may lose quite some from the perspective of the single method, but in return we may also, via the sequencing of complementary methods, gain a richer understanding of the social phenomenon under study.