European Political Science

, Volume 18, Issue 2, pp 275–290 | Cite as

What did I leave out? Omitted variables in regression and qualitative comparative analysis

  • Claudio M. RadaelliEmail author
  • Claudius WagemannEmail author


Social scientists often face a fundamental problem: Did I leave something causally important out of my explanation? How do I diagnose this? Where do I look for solutions to this problem? We build bridges between regression models and qualitative comparative analysis by comparing diagnostics and solutions to the problem of omitted variables and conditions. We then discuss various approaches and tackle the theoretical issues around causality which must be addressed before attending to technical fixes. In the conclusions, we reflect on the bridges built between the two traditions and draw more general lessons about the logic of social science research.


Causality Econometrics Qualitative comparative analysis Research design Regression analysis 



We would like to thank the reviewers and the EPS co-editor Daniel Stockemer for their comments and suggestions. We also thank Fabrizio Gilardi, Jonathan C. Kamkhaji, Gabriel Katz and the participants to the ‘cake for comments’ seminar series 2016–2017 at the University of Exeter. We are grateful to Francesca Farmer for checking style and grammar. Research for this project was funded by the ERC project 694632 Protego, Procedural Tools for Effective Governance, and by the Goethe University of Frankfurt.


  1. Angrist, J.D., and A.B. Krueger. 2001. Instrumental variables and the search for identification: From supply and demand to natural experiments. Journal of Economic Perspectives 15(4): 69–85.CrossRefGoogle Scholar
  2. Bennett, A., and J.T. Checkel. 2015. Process tracing: From philosophical roots to best practices. In Process tracing: From metaphor to analytical tool, ed. A. Bennett and J.T. Checkel, 3–38. Cambridge: Cambridge University Press.Google Scholar
  3. Blanchard, O., and F. Giavazzi. 2003. Macroeconomic effects of regulation and deregulation in goods and labor markets. Quarterly Journal of Economics 118(3): 879–907.CrossRefGoogle Scholar
  4. Calabresi, G., and D. Melamed. 1972. Property rules, liability rules and inalienability: One view of the cathedral. Harvard Law Review 85(6): 1089–1128.CrossRefGoogle Scholar
  5. Clarke, K.A. 2005. The phantom menace: Omitted variable bias in econometric research. Conflict Management and Peace Science 22(4): 341–352.CrossRefGoogle Scholar
  6. Clarke, K.A. 2009. Return of the phantom menace: Omitted variable bias in political research. Conflict Management and Peace Science 26(1): 46–66.CrossRefGoogle Scholar
  7. Collier, D. 2014. Comment: QCA should set aside the algorithms. Sociological Methodology 44(1): 122–126.CrossRefGoogle Scholar
  8. Djankov, S., C. McLiesh, and R.M. Ramalho. 2006. Regulation and growth. Economic Letters 92: 395–401.CrossRefGoogle Scholar
  9. Dougherty, C. 2011. Introduction to econometrics, 4th ed. Oxford: Oxford University Press.Google Scholar
  10. Fiss, P., D. Sharapov, and L. Cronqvist. 2013. Opposites attract? Opportunities and challenges for integrating large-N QCA and econometric analysis. Political Research Quarterly 66(1): 191–197.Google Scholar
  11. Gerring, J. 2014. Causal mechanisms: Yes, but…. Comparative Political Studies 43(11): 1499–1526.CrossRefGoogle Scholar
  12. Goertz, G. 2006. Assessing the trivialness, relevance, and relative importance of necessary and sufficient conditions in social science. Studies in Comparative International Development 41(2): 88–109.CrossRefGoogle Scholar
  13. Greene, W.H. 2003. Econometric analysis, 5th ed. Upper Saddle River, NJ: Pearson Education.Google Scholar
  14. Grofman, B., and C.Q. Schneider. 2009. An introduction to crisp set QCA, with a comparison to binary logistic regression. Political Research Quarterly 62(4): 662–672.CrossRefGoogle Scholar
  15. Hug, S. 2013. Qualitative comparative analysis: How inductive use and measurement error lead to problematic inference. Political Analysis 21(2): 252–265.CrossRefGoogle Scholar
  16. Jalilian, H., C. Kirkpatrick, and D. Parker. 2007. The impact of regulation on economic growth in developing countries: A cross-country analysis. World Development 35: 87–103.CrossRefGoogle Scholar
  17. King, G., R.O. Keohane, and S. Verba. 1994. Designing social inquiry: Scientific inference in qualitative research. Princeton: Princeton University Press.CrossRefGoogle Scholar
  18. Maggetti, M., F. Gilardi, and C.M. Radaelli. 2013. Designing research in the social sciences. Los Angeles: Sage.CrossRefGoogle Scholar
  19. Merton, R.K. 1957. On sociological theories of the middle range. In On theoretical sociology. Five essays, old and new, ed. R.K. Merton, 39–72. New York: The Free Press.Google Scholar
  20. Moses, J., and T. Knutsen. 2012. Ways of knowing: Competing methodologies in social and political research. Basingstoke: Palgrave Macmillan.CrossRefGoogle Scholar
  21. Most, B.A., and H. Starr. 1989. Inquiry logic and international politics. Columbia: University of South Carolina Press.Google Scholar
  22. Munck, G.L. 2016. Assessing set-theoretic comparative methods: A tool for qualitative comparativists? Comparative Political Studies 49(6): 775–780.CrossRefGoogle Scholar
  23. Paine, J. 2016a. Set-theoretic comparative methods: Less distinctive than claimed. Comparative Political Studies 49(6): 703–741.CrossRefGoogle Scholar
  24. Paine, J. 2016b. Still searching for the value added: Persistent concerns about set-theoretic comparative methods. Comparative Political Studies 49(6): 793–800.CrossRefGoogle Scholar
  25. Radaelli, C.M., and F. De Francesco. 2007. Regulatory quality in Europe: Concepts, measures and policy processes. Manchester: MUP.Google Scholar
  26. Ragin, C.C. 1987. The comparative method. Berkeley: The University of California Press.Google Scholar
  27. Ragin, C.C. 1994. Constructing social research: The unity and diversity of method. Thousand Oaks: Pine Forge Press.Google Scholar
  28. Ragin, C.C. 2000. Fuzzy-set social science. Chicago: Chicago University Press.Google Scholar
  29. Ragin, C.C. 2008. Redesigning social inquiry: Fuzzy sets and beyond. Chicago: University of Chicago Press.CrossRefGoogle Scholar
  30. Rihoux, B., and C.C. Ragin (eds.). 2009. Configurational comparative methods: Qualitative comparative analysis and related techniques. Thousand Oaks: SAGE.Google Scholar
  31. Rihoux, B., P. Álamos-Concha, D. Bol, A. Marx, and I. Rezsöhazy. 2013. From niche to mainstream? A comprehensive mapping of QCA applications in journal articles from 1984 to 2011. Political Research Quarterly 66(1): 175–184.CrossRefGoogle Scholar
  32. Schneider, C.Q. 2016. Real differences and overlooked similarities. Set-methods in comparative perspective. Comparative Political Studies 49(6): 781–792.CrossRefGoogle Scholar
  33. Schneider, C.Q., and I. Rohlfing. 2013. Combining QCA and process tracing in set-theoretic multi-method research. Sociological Methods and Research 42(4): 559–597.CrossRefGoogle Scholar
  34. Schneider, C.Q., and C. Wagemann. 2012. Set-theoretic methods for the social sciences. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  35. Schneider, C.Q., and C. Wagemann. 2016. Assessing ESA on what it is designed for: A reply to Cooper and Glaesser. Field Methods 28(3): 316–321.CrossRefGoogle Scholar
  36. Radaelli, C.M., and Schrefler, L. 2011. Deregulation. In International encyclopedia of political science, ed. B. Badie, D. Berg-Schlosser, and L. Morlino. London: Sage.Google Scholar
  37. Seawright, J. 2005. Qualitative comparative analysis vis-à-vis regression. Studies in Comparative and International Development 40(1): 3–26.CrossRefGoogle Scholar
  38. Skaaning, S.E. 2011. Assessing the robustness of crisp-set and fuzzy-set QCA results. Sociological Methods and Research 40(2): 391–408.CrossRefGoogle Scholar
  39. Starr, H. 2005. Cumulation from proper specification: Theory, logic, research design, and ‘nice’ laws. Conflict Management and Peace Science 22(4): 353–363.CrossRefGoogle Scholar
  40. Thiem, A., M. Baumgartner, and D. Bol. 2016. Still lost in translation! A correction of three misunderstandings between configurational comparativists and regressional analysis. Comparative Political Studies 49(6): 742–774.CrossRefGoogle Scholar
  41. Vis, B. 2012. The comparative advantages of fsQCA and regression analysis for moderately large-N analyses. Sociological Methods and Research 41(1): 168–198.CrossRefGoogle Scholar
  42. Wagemann, C., J. Buche, and M.B. Siewert. 2016. QCA and business research: Work in progress or a consolidated agenda. Journal of Business Research 69(7): 2531–2540.CrossRefGoogle Scholar
  43. Woolridge, J.M. 2008. Introductory econometrics. Andover: Cengage Learning.Google Scholar

Copyright information

© European Consortium for Political Research 2018

Authors and Affiliations

  1. 1.Department of Politics, Centre for European GovernanceUniversity of ExeterExeterUK
  2. 2.Department of Political Science, Faculty of Social SciencesGoethe UniversityFrankfurt am MainGermany

Personalised recommendations