Skip to main content

What did I leave out? Omitted variables in regression and qualitative comparative analysis

European Political Science volume 18, pages 275–290 (2019)Cite this article

Abstract

Social scientists often face a fundamental problem: Did I leave something causally important out of my explanation? How do I diagnose this? Where do I look for solutions to this problem? We build bridges between regression models and qualitative comparative analysis by comparing diagnostics and solutions to the problem of omitted variables and conditions. We then discuss various approaches and tackle the theoretical issues around causality which must be addressed before attending to technical fixes. In the conclusions, we reflect on the bridges built between the two traditions and draw more general lessons about the logic of social science research.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Notes

  1. In QCA, the term of ‘omitted variables’ is not used, also because it is inappropriate to speak about ‘variables’ (see fn. 2). However, for reasons of readability, we use this expression not only for our discussion on regression, but also when we deal with QCA.

  2. This is more than just a linguistic matter of taste. Set-theoretic methods, such as QCA, do not look at variables but model various degrees of case membership in sets. The difference between the two understandings does not so much result in different values of the variable or the set membership, respectively, but lies in the perspective: variables look at a characteristics, while sets group cases according to a given case property. Sets, therefore, are very closely linked to concepts.

  3. An instrumental variable has first and foremost a technical definition, contrary to the conceptual approach we followed when dealing with the proxy.

  4. The tilde indicates the negation of a condition.

  5. The superficiality consists in the fact that our description simplifies actual XY plots a bit. There can also be inconsistent cases in the upper right and lower left corner, namely those below the diagonal. However, for the argument made here, this is not important.

  6. Since the extreme situation is already that every case has its ‘own’ explanation, there cannot be more explanatory paths than cases, unless we allow for values of ‘unique coverage’ being zero.

  7. Another source for this phenomenon might be the generally low number of cases.

  8. One may ask at this point whether it is then justified to lower expectations for large N QCAs and whether a low-expectation large N QCA is really desirable.

References

  • Angrist, J.D., and A.B. Krueger. 2001. Instrumental variables and the search for identification: From supply and demand to natural experiments. Journal of Economic Perspectives 15(4): 69–85.

    Article  Google Scholar 

  • Bennett, A., and J.T. Checkel. 2015. Process tracing: From philosophical roots to best practices. In Process tracing: From metaphor to analytical tool, ed. A. Bennett and J.T. Checkel, 3–38. Cambridge: Cambridge University Press.

    Google Scholar 

  • Blanchard, O., and F. Giavazzi. 2003. Macroeconomic effects of regulation and deregulation in goods and labor markets. Quarterly Journal of Economics 118(3): 879–907.

    Article  Google Scholar 

  • Calabresi, G., and D. Melamed. 1972. Property rules, liability rules and inalienability: One view of the cathedral. Harvard Law Review 85(6): 1089–1128.

    Article  Google Scholar 

  • Clarke, K.A. 2005. The phantom menace: Omitted variable bias in econometric research. Conflict Management and Peace Science 22(4): 341–352.

    Article  Google Scholar 

  • Clarke, K.A. 2009. Return of the phantom menace: Omitted variable bias in political research. Conflict Management and Peace Science 26(1): 46–66.

    Article  Google Scholar 

  • Collier, D. 2014. Comment: QCA should set aside the algorithms. Sociological Methodology 44(1): 122–126.

    Article  Google Scholar 

  • Djankov, S., C. McLiesh, and R.M. Ramalho. 2006. Regulation and growth. Economic Letters 92: 395–401.

    Article  Google Scholar 

  • Dougherty, C. 2011. Introduction to econometrics, 4th ed. Oxford: Oxford University Press.

    Google Scholar 

  • Fiss, P., D. Sharapov, and L. Cronqvist. 2013. Opposites attract? Opportunities and challenges for integrating large-N QCA and econometric analysis. Political Research Quarterly 66(1): 191–197.

    Google Scholar 

  • Gerring, J. 2014. Causal mechanisms: Yes, but…. Comparative Political Studies 43(11): 1499–1526.

    Article  Google Scholar 

  • Goertz, G. 2006. Assessing the trivialness, relevance, and relative importance of necessary and sufficient conditions in social science. Studies in Comparative International Development 41(2): 88–109.

    Article  Google Scholar 

  • Greene, W.H. 2003. Econometric analysis, 5th ed. Upper Saddle River, NJ: Pearson Education.

    Google Scholar 

  • Grofman, B., and C.Q. Schneider. 2009. An introduction to crisp set QCA, with a comparison to binary logistic regression. Political Research Quarterly 62(4): 662–672.

    Article  Google Scholar 

  • Hug, S. 2013. Qualitative comparative analysis: How inductive use and measurement error lead to problematic inference. Political Analysis 21(2): 252–265.

    Article  Google Scholar 

  • Jalilian, H., C. Kirkpatrick, and D. Parker. 2007. The impact of regulation on economic growth in developing countries: A cross-country analysis. World Development 35: 87–103.

    Article  Google Scholar 

  • King, G., R.O. Keohane, and S. Verba. 1994. Designing social inquiry: Scientific inference in qualitative research. Princeton: Princeton University Press.

    Book  Google Scholar 

  • Maggetti, M., F. Gilardi, and C.M. Radaelli. 2013. Designing research in the social sciences. Los Angeles: Sage.

    Book  Google Scholar 

  • Merton, R.K. 1957. On sociological theories of the middle range. In On theoretical sociology. Five essays, old and new, ed. R.K. Merton, 39–72. New York: The Free Press.

    Google Scholar 

  • Moses, J., and T. Knutsen. 2012. Ways of knowing: Competing methodologies in social and political research. Basingstoke: Palgrave Macmillan.

    Book  Google Scholar 

  • Most, B.A., and H. Starr. 1989. Inquiry logic and international politics. Columbia: University of South Carolina Press.

    Google Scholar 

  • Munck, G.L. 2016. Assessing set-theoretic comparative methods: A tool for qualitative comparativists? Comparative Political Studies 49(6): 775–780.

    Article  Google Scholar 

  • Paine, J. 2016a. Set-theoretic comparative methods: Less distinctive than claimed. Comparative Political Studies 49(6): 703–741.

    Article  Google Scholar 

  • Paine, J. 2016b. Still searching for the value added: Persistent concerns about set-theoretic comparative methods. Comparative Political Studies 49(6): 793–800.

    Article  Google Scholar 

  • Radaelli, C.M., and F. De Francesco. 2007. Regulatory quality in Europe: Concepts, measures and policy processes. Manchester: MUP.

    Google Scholar 

  • Ragin, C.C. 1987. The comparative method. Berkeley: The University of California Press.

    Google Scholar 

  • Ragin, C.C. 1994. Constructing social research: The unity and diversity of method. Thousand Oaks: Pine Forge Press.

    Google Scholar 

  • Ragin, C.C. 2000. Fuzzy-set social science. Chicago: Chicago University Press.

    Google Scholar 

  • Ragin, C.C. 2008. Redesigning social inquiry: Fuzzy sets and beyond. Chicago: University of Chicago Press.

    Book  Google Scholar 

  • Rihoux, B., and C.C. Ragin (eds.). 2009. Configurational comparative methods: Qualitative comparative analysis and related techniques. Thousand Oaks: SAGE.

    Google Scholar 

  • Rihoux, B., P. Álamos-Concha, D. Bol, A. Marx, and I. Rezsöhazy. 2013. From niche to mainstream? A comprehensive mapping of QCA applications in journal articles from 1984 to 2011. Political Research Quarterly 66(1): 175–184.

    Article  Google Scholar 

  • Schneider, C.Q. 2016. Real differences and overlooked similarities. Set-methods in comparative perspective. Comparative Political Studies 49(6): 781–792.

    Article  Google Scholar 

  • Schneider, C.Q., and I. Rohlfing. 2013. Combining QCA and process tracing in set-theoretic multi-method research. Sociological Methods and Research 42(4): 559–597.

    Article  Google Scholar 

  • Schneider, C.Q., and C. Wagemann. 2012. Set-theoretic methods for the social sciences. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Schneider, C.Q., and C. Wagemann. 2016. Assessing ESA on what it is designed for: A reply to Cooper and Glaesser. Field Methods 28(3): 316–321.

    Article  Google Scholar 

  • Radaelli, C.M., and Schrefler, L. 2011. Deregulation. In International encyclopedia of political science, ed. B. Badie, D. Berg-Schlosser, and L. Morlino. London: Sage.

    Google Scholar 

  • Seawright, J. 2005. Qualitative comparative analysis vis-à-vis regression. Studies in Comparative and International Development 40(1): 3–26.

    Article  Google Scholar 

  • Skaaning, S.E. 2011. Assessing the robustness of crisp-set and fuzzy-set QCA results. Sociological Methods and Research 40(2): 391–408.

    Article  Google Scholar 

  • Starr, H. 2005. Cumulation from proper specification: Theory, logic, research design, and ‘nice’ laws. Conflict Management and Peace Science 22(4): 353–363.

    Article  Google Scholar 

  • Thiem, A., M. Baumgartner, and D. Bol. 2016. Still lost in translation! A correction of three misunderstandings between configurational comparativists and regressional analysis. Comparative Political Studies 49(6): 742–774.

    Article  Google Scholar 

  • Vis, B. 2012. The comparative advantages of fsQCA and regression analysis for moderately large-N analyses. Sociological Methods and Research 41(1): 168–198.

    Article  Google Scholar 

  • Wagemann, C., J. Buche, and M.B. Siewert. 2016. QCA and business research: Work in progress or a consolidated agenda. Journal of Business Research 69(7): 2531–2540.

    Article  Google Scholar 

  • Woolridge, J.M. 2008. Introductory econometrics. Andover: Cengage Learning.

    Google Scholar 

Download references

Acknowledgements

We would like to thank the reviewers and the EPS co-editor Daniel Stockemer for their comments and suggestions. We also thank Fabrizio Gilardi, Jonathan C. Kamkhaji, Gabriel Katz and the participants to the ‘cake for comments’ seminar series 2016–2017 at the University of Exeter. We are grateful to Francesca Farmer for checking style and grammar. Research for this project was funded by the ERC project 694632 Protego, Procedural Tools for Effective Governance, and by the Goethe University of Frankfurt.

Author information

Authors and Affiliations

  1. Department of Politics, Centre for European Governance, University of Exeter, Exeter, UK

    Claudio M. Radaelli

  2. Department of Political Science, Faculty of Social Sciences, Goethe University, Frankfurt am Main, Germany

    Claudius Wagemann

Authors
  1. Claudio M. Radaelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Claudius Wagemann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Claudio M. Radaelli or Claudius Wagemann.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Radaelli, C.M., Wagemann, C. What did I leave out? Omitted variables in regression and qualitative comparative analysis. Eur Polit Sci 18, 275–290 (2019). https://doi.org/10.1057/s41304-017-0142-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/s41304-017-0142-7

Keywords

  • Causality
  • Econometrics
  • Qualitative comparative analysis
  • Research design
  • Regression analysis