Skip to main content
SpringerLink
Log in

Applying qualitative comparative analysis in large-N studies: a scoping review of good practices before, during, and after the analytic moment

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

Qualitative comparative analysis (QCA) was originally developed to address configural research questions with a small to medium number of cases (i.e., N < 50), allowing researchers to preserve the iterative nature of the data collection and interpretation that stems from a deep knowledge of cases. Recently, researchers have increasingly applied QCA within large-N applications involving anywhere from 50 to several thousand cases. Although the increasing popularity of using QCA in large-N research is promising, critical questions persist regarding QCA application. We conducted a scoping review of large-N QCA studies published in peer-reviewed journals over a 15-year period. Although the review showed some adherence to good practices for conducting QCA, it also revealed substantial gaps in large-N studies reporting analytic decision making, analysis, and results. We offer several recommendations for improving the reporting of large-N QCA studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. We used the following decision rules, based on the number of QCA models included within a study, to determine the number of cases in the study and whether the study met our large-N definition of 50 or more cases. If there were two or more separate QCA models with 50 or more cases reported, we summed the total number of cases included in the QCA models. If there was one model with 50 or more cases and one model with less than 50 cases, we reported only the number of cases from the model with 50 or more cases. If there were two or more models that each had fewer than 50 cases, the study did not meet our large-N definition and was excluded.

  2. Our definition of social science and health journals includes the fields of economics, education, health services, human services, public health, research methods, sociology, political science, and psychology. We manually reviewed journal names and editorial information (e.g., aims and scope, keywords, and abstracting and indexing information) to determine whether large-N QCA studies were published in social science or health journals. Studies published in other domains, such as business or engineering, were not selected for full-text reviews.

  3. We included studies that indicated what solution type (intermediate, conservative, parsimonious) was presented as specifying the approach to logical remainders, as the approach is implied by the solution type.

  4. The reviewed studies used myriad approaches to interpreting solution terms. We highlight one study (Dardanelli 2014), which used all of the approaches to interpret solution terms, as an example of how the various approaches to interpretation can be incorporated.

References

  • Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., and Consort Group: The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Annal. Int. Med. 134(8), 663–694 (2001)

    Article  Google Scholar 

  • Blackman, T.: Can smoking cessation services be better targeted to tackle health inequalities? Evidence from a cross-sectional study. Health Educ. J. 67(2), 91–101 (2008)

    Article  Google Scholar 

  • Chuang, E., Dill, J., Morgan, J., Konrad, T.: A configurational approach to the relationship between high-performance work practices and frontline health care worker outcomes. Health Serv. Res. 47(4), 1460–1481 (2012). https://doi.org/10.1111/j.1475-6773.2011.01366.x

    Article  Google Scholar 

  • Collier, D.: Comment: QCA should set aside the algorithms. Sociol. Methodol. 44(1), 122–126 (2014a). https://doi.org/10.1177/0081175014542568

    Article  Google Scholar 

  • Collier, D.: Problematic tools: introduction to symposium on set theory in social science. Qual. Multi-Method Res. 12(1), 2–9 (2014b)

    Google Scholar 

  • Cooper, B., Glaesser, J.: Exploring the robustness of set theoretic findings from a large n fsQCA: an illustration from the sociology of education. Int. J. Social Res. Methodol.: Theory Pract 19(4), 445–459 (2016). https://doi.org/10.1080/13645579.2015.1033799

    Article  Google Scholar 

  • Creswell, J.W., Creswell, J.: Research design, 2nd edn. Sage Publications (2003)

    Google Scholar 

  • Dardanelli, P.: European integration, party strategies, and state restructuring: a comparative analysis. Eur. Polit. Sci. Rev. 6(2), 213–236 (2014)

    Article  Google Scholar 

  • Emmenegger, P., Schraff, D., and Walter, A.: QCA, the truth table analysis and large-N survey data: The benefits of calibration and the importance of robustness tests (Compasss Working Paper 2014–79). www.compasss.org, (2014).

  • Finn, V.: A qualitative assessment of QCA: method stretching in large-N studies and temporality. Qual. Quant. 56(5), 3815–3830 (2022)

    Article  Google Scholar 

  • Goertz, G.: Social science concepts: A user’s guide. Princeton University Press (2006)

    Book  Google Scholar 

  • Greckhamer, T., Misangyi, V., and Fiss, P. C.: The two QCAs: From a small-N to a large-N set theoretic approach. In Configurational Theory and Methods in Organizational Research (pp. 49–75). (Research in the Sociology of Organizations; Vol. 38). Emerald Group Publishing Ltd. (2013). https://doi.org/10.1108/S0733-558X(2013)0000038007

  • Herrmann, A.M., Cronqvist, L.: When dichotomisation becomes a problem for the analysis of middle-sized datasets. Int. J. Soc. Res. Methodol. 12(1), 33–50 (2009)

    Article  Google Scholar 

  • Kahwati, L.C., Kane, H.L.: Qualitative comparative analysis in mixed methods research and evaluation. Sage Publications (2020)

    Book  Google Scholar 

  • Krogslund, C., Michel, K.: A larger-N, fewer variables problem? The counterintuitive sensitivity of QCA. Qual. Multi-Method Res. 12, 25–33 (2014)

    Google Scholar 

  • Longest, K., Thoits, P.: Gender, the stress process, and health: a configurational approach. Soc. Mental Health 2(3), 187–206 (2012). https://doi.org/10.1177/2156869312451151

    Article  Google Scholar 

  • Comparative Methods for Systematic Cross-Case Analysis. (n.d.). Bibliography of journal articles involving configurational comparative methods. https://www.zotero.org/groups/510780/compasss/library

  • Moller, J., Skaaning, S.E.: Set-theoretic methods in democratization research: an evaluation of their uses and contributions. Democratization 26(1), 78–96 (2019)

    Article  Google Scholar 

  • Munn, Z., Peters, M.D., Stern, C., Tufanaru, C., McArthur, A., Aromataris, E.: Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med. Res. Methodol. 18(1), 1–7 (2018)

    Article  Google Scholar 

  • Oana, I. E., and Schneider, C. Q. (2021). A robustness test protocol for applied QCA: theory and R software application. Sociological Methods and Research.

  • Ragin, C.C.: The comparative method: moving beyond qualitative and quantitative strategies. University of California Press (1987)

    Google Scholar 

  • Ragin, C.C.: Using qualitative comparative analysis to study causal complexity. Health Serv. Res. 34(5 Pt 2), 1225–1239 (1999)

    Google Scholar 

  • Ragin, C.C.: Fuzzy-set social science. The University of Chicago Press (2000)

    Google Scholar 

  • Ragin, C.C.: Redesigning social inquiry: Fuzzy sets and beyond. The University of Chicago Press (2008)

    Book  Google Scholar 

  • Rihoux, B.: Qualitative comparative analysis (QCA) and related systematic comparative methods: recent advances and remaining challenges for social science research. Int. Sociol. 21(5), 679–706 (2006)

    Article  Google Scholar 

  • Rihoux, B., Ragin, C.C.: Configurational comparative methods: Qualitative comparative analysis (QCA) and related techniques. Sage Publications (2008)

    Google Scholar 

  • Rihoux, B., Álamos-Concha, P., Bol, D., Marx, A., Rezsöhazy, I.: From niche to mainstream method? A comprehensive mapping of QCA applications in journal articles from 1984 to 2011. Political Res. Quart. 66(1), 175–184 (2013)

    Google Scholar 

  • Rubinson, C.: Presenting qualitative comparative analysis: Notation, tabular layout, and visualization. Methodol. Innovat. 12(2), 2059799119862110 (2019)

    Google Scholar 

  • Rutten, R.: Applying and assessing large-N QCA: Causality and robustness from a critical realist perspective. Sociological Methods and Research 51(3), 1211–1243 (2022)

    Article  Google Scholar 

  • Schneider, C. Q., and Wagemann, C: Set-theoretic methods for the social sciences: A guide to qualitative comparative analysis, Cambridge University Press. (2012)

  • Schneider, C.Q., Wagemann, C.: Standards of good practice in qualitative comparative analysis (QCA) and fuzzy-sets. Comp. Sociol. 9(3), 397–418 (2010)

    Article  Google Scholar 

  • Thiem, A., Dusa, A.: Qualitative comparative analysis with R: A user’s guide. Springer Science and Business Media, London (2012)

    Google Scholar 

  • Thomann, E., Ege, J.: Qualitative comparative analysis (QCA) in public administration. Oxford Research Encyclopedia of Politics (2020)

    Book  Google Scholar 

  • Thomann, E., Maggetti, M.: Designing research with qualitative comparative analysis (QCA): approaches, challenges, and tools. Sociol. Methods Res. 49(2), 356–386 (2020)

    Article  Google Scholar 

  • Thomann, E., Ege, J., Paustyan, E.: Approaches to qualitative comparative analysis and good practices: a systematic review. Swiss Political Sci. Rev. 28, 557–580 (2022). https://doi.org/10.1111/spsr.12503

    Article  Google Scholar 

  • Tricco, A.C., Lillie, E., Zarin, W., O’Brien, K.K., Colquhoun, H., Levac, D., Moher, D., Peters, M.D.J., Horsley, T., Weeks, L., Hempel, S., Akl, E.A., Chang, C., McGowan, J., Stewart, L., Hartling, L., Aldcroft, A., Wilson, M.G., Garritty, C., Straus, S.E.: PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Annal. Int. Med. 169(7), 467–473 (2018)

    Article  Google Scholar 

  • Verweij, S., Trell, E.M.: Qualitative comparative analysis (QCA) in spatial planning research and related disciplines: a systematic literature review of applications. J. Plan. Lit. 34(3), 300–317 (2019)

    Article  Google Scholar 

  • Wagemann, C., Schneider, C.Q.: Transparency standards in qualitative comparative analysis. Qual. Multi-Method Res. Newslett. 13, 38–42 (2015)

    Google Scholar 

  • Wagemann, C., Buche, J., Siewert, M.B.: QCA and business research: work in progress or a consolidated agenda? J. Bus. Res. 69(7), 2531–2540 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. RTI International, Research Triangle, USA

    Dallas J. Elgin, Erin Erickson, Meredith Crews, Leila C. Kahwati & Heather L. Kane

Authors
  1. Dallas J. Elgin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Erin Erickson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Meredith Crews
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Leila C. Kahwati
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Heather L. Kane
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dallas J. Elgin.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 31 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Elgin, D.J., Erickson, E., Crews, M. et al. Applying qualitative comparative analysis in large-N studies: a scoping review of good practices before, during, and after the analytic moment. Qual Quant (2024). https://doi.org/10.1007/s11135-024-01849-2

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11135-024-01849-2

Keywords

Search

Navigation