Abstract
Interest in and use of mixed methods research in the social sciences has grown tremendously in recent years and has the potential to assist in addressing core challenges in causal inference. We discuss ways in which the addition of a qualitative component can serve multiple roles in causal analyses, including understanding treatment definition, concept measurement, selection into treatment, causal effect mechanisms, and effect heterogeneity. We also describe how quantitative and qualitative methodologies can be and have been combined in studies seeking to make causal claims, highlighting some of the key research design decisions in integrating qualitative and quantitative methodologies. Given the ability of qualitative research to greatly enhance quantitative studies concerned with causality, we argue that more causal studies should incorporate mixed methods approaches into their research designs.
We thank Jennifer Barber, Gary Goertz, and Stephen Morgan for helpful comments on a previous version of this paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Within the scholarly literature, a lively debate has emerged as to what constitutes mixed methods research and whether or not it constitutes a stand-alone method and perspective, rather than simply the combining of quantitative and qualitative methods. Engaging in this discussion is beyond the scope of the chapter, but see Small (2011) for an overview of the issue.
- 2.
In so doing, we should not be misunderstood as arguing that qualitative research cannot also stand on its own as a method for causal analysis under appropriate assumptions (just as quantitative analysis only produces valid causal inferences under appropriate assumptions).
- 3.
A mixed methods study could also use comparative historical methods that rely on data from archival research. This chapter does not discuss in detail this approach or some of the challenges particularly associated with it, such as case selection in small-n research. For a discussion of this and other issues related to comparative historical methods, see Mahoney and Goertz (2006) and Goertz and Mahoney (2012).
- 4.
According to Mahoney, a sufficient but not necessary cause is one experienced only by cases that had the outcome, but not all the cases with the outcome experience the cause. A necessary but not sufficient cause is one that all cases with the outcome experience, but not all those that experience the cause have the outcome. A necessary and sufficient cause is one experienced only by those with the outcome. If one has only cases that experienced the outcome, then at most one can make claims about necessary but not sufficient causes (see Harding et al. 2002 for an example). Mahoney (2000) argues that nominal comparison is particularly effective at eliminating alternative explanations. These ideas have been formalized and extended to more complex causal configurations by Ragin in his Qualitative Comparative Analysis (QCA) and “fuzzy set” Boolean algebra methods (Ragin 2000, 2008; see also Mahoney 2008).
- 5.
Particularly when one has many cases, one will likely find cases that do not fit with the modal patterns. However, such anomalous cases can help to elaborate one’s causal model or define the circumstances or types of cases or subjects to which it applies (just as outliers do in quantitative research).
- 6.
Although the literature on causal analysis in ethnography is considerably less developed, a similar logic has been proposed, albeit using different terminology. For example, Tavory and Timmermans (2012) discuss “a continuous stream of events in sequences,” and Lichterman and Reed (2012) discuss “chains of action.”
- 7.
Although it is possible to use instrumental variables analysis to addresses this concern, the researcher generally needs to have one instrument per mechanism.
- 8.
Surveys that collect information repeatedly and frequently could potentially obtain similarly detailed information and lead to an understanding of causal mechanisms. An example of such a study is the Relationship Dynamics and Social Life study, which, after a baseline interview, collected weekly information over the internet or telephone about relationships and pregnancy intentions from a sample of young women (Barber et al. 2011). However, quantitative studies with such frequent data collection are expensive and challenging to implement and are therefore very rare.
- 9.
Other authors have offered different typologies of the possible roles of qualitative research in mixed methods studies more generally. For example, Fearon and Laitin (2008) discuss three roles: (a) narrative accounts of causal processes, including interpretive data from key actors, (b) identifying omitted variables, and (c) evaluating measurement reliability and validity. Greene et al. (1989) describe five purposes of mixed methods research: triangulation, complementarity, development, initiation, and expansion.
- 10.
Although the Gautreaux evaluation did include a qualitative component, it started too late to study these selection processes.
- 11.
Open-ended questioning that attempts to discern how respondents arrive at the answer to a survey question is a tool used in cognitive interviewing, a method for diagnosing problems with survey questions (see Presser et al. 2004). For a review of methods for testing and evaluating survey questions, see Schaeffer and Presser (2003).
References
Achen, C. H., & Snidal, D. (1989). Rational deterrence theory and comparative case studies. World Politics, 41, 144–169.
Angrist, J. D., Imbens, G., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91, 444–472.
Axinn, W. G., & Pearce, L. D. (2006). Mixed method data collection strategies. Cambridge: Cambridge University Press.
Barber, J., Kusunoki, Y., & Gatny, H. (2011). Design and implementation of an online weekly journal to study unintended pregnancies. Vienna Yearbook of Population Research, 9, 327–334.
Brady, H. E., & Collier, D. (Eds.). (2010). Rethinking social inquiry: Diverse tools, shared standards (2nd ed.). New York: Rowman & Littlefield.
Brand, J., & Xie, Y. (2010). Who benefits most from college? Evidence for negative selection in heterogeneous economic returns to higher education. American Sociological Review, 75, 273–302.
Briggs, X. d. S., Popkin, S. J., & Goering, J. (2010). Moving to opportunity: The story of an American experiment to fight ghetto poverty. Oxford: Oxford University Press.
Bryman, A. (2006). Integrating quantitative and qualitative research: How is it done? Qualitative Research, 6, 97–113.
Burton, L. M., Purvin, D., & Garrett-Peters, R. (2009). Longitudinal ethnography: Uncovering domestic abuse in low-income women’s lives. In G. Elder & J. Z. Giele (Eds.), The craft of life course research (pp. 70–92). New York: Guilford Press.
Cherlin, A. J., Burton, L. M., Hurt, T. R., & Purvin, D. (2004). The influence of physical and sexual abuse on marriage and cohabitation. American Sociological Review, 69, 768–789.
Clampet-Lundquist, S., Edin, K., Kling, J. R., & Duncan, G. J. (2011). Moving teenagers out of high risk neighborhoods: How girls fare better than boys. The American Journal of Sociology, 116, 1154–1189.
Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks: Sage.
Dion, R. M., Hershey, A. M., Zaveri, H. H., Avellar, S. A., Strong, D. A., Silman, T., & Moore, R. (2008). Implementation of the building strong families program. Washington, DC: Mathematica Policy Research.
Durkheim, E. (1982 [1901]). In: S. Lukes (Ed.) The rules of sociological method (W.D. Halls, Trans.). New York: Free Press.
Fearon, J. D., & Laitin, D. D. (2008). Integrating qualitative and quantitative methods. In J. M. Box-Steffensmeier, H. E. Brady, & C. David (Eds.), Oxford handbook of political methodology (pp. 756–776). Oxford: Oxford University Press.
Gerring, J. (2007). Case study research: Principles and practices. Cambridge: Cambridge University Press.
Gibson-Davis, C. (2009). Money, marriage, and children: Testing the financial expectations and family formation theory. Journal of Marriage and Family, 71, 146–160.
Gibson-Davis, C., & Duncan, G. J. (2005). Qualitative/quantitative synergies in a random assignment program. In T. Weisner (Ed.), Discovering successful pathways in children’s development: New methods in the study of childhood and family life (pp. 283–303). Chicago: University of Chicago Press.
Gibson-Davis, C., Edin, K., & McLanahan, S. (2005). High hopes but even higher expectations: The retreat from marriage among low-income couples. Journal of Marriage and Family, 67, 1301–1312.
Glaser, B. G., & Strauss, A. L. (1977). The discovery of grounded theory: Strategies for qualitative research. New York: Aldine Publishing.
Goering, J., & Feins, J. D. (2003). Choosing a better life: Evaluating the moving to opportunity experiment. Washington, DC: Urban Institute Press.
Goertz, G., & Mahoney, J. (2012). A tale of two cultures: Contrasting qualitative and quantitative paradigms. Princeton: Princeton University Press.
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11, 255–274.
Harding, D. J. (2007). Cultural context, sexual behavior, and romantic relationships in disadvantaged neighborhoods. American Sociological Review, 72, 341–364.
Harding, D. J. (2010). Living the drama: Community, conflict, and culture among inner-city boys. Chicago: University of Chicago Press.
Harding, D. J., Fox, C., & Mehta, J. D. (2002). Studying rare events through qualitative case studies: Lessons from a study of rampage school shootings. Sociological Methods and Research, 31, 174–217.
Harding, D. J., Morenoff, J. D., & Herbert, C. (Forthcoming). Home is hard to find: Neighborhoods, institutions, and the residential trajectories of returning prisoners. Annals of the American Academy of Political and Social Science.
Harknett, K., & McLanahan, S. (2004). Explaining racial and ethnic differences in marriage among new, unwed parents. American Sociological Review, 69, 790–811.
Johnson, R. C., & Corcoran, M. (2003, Fall). The road to economic self-sufficiency: Job quality and job transition patterns after welfare reform. Journal of Policy Analysis & Management, 22(4), 615–639.
Keels, M., Duncan, G. J., DeLuca, S., Mendenhall, R., & Rosenbaum, J. E. (2005). Fifteen years later: Can residential mobility programs provide a permanent escape from neighborhood crime and poverty? Demography, 42, 51–73.
King, G., Keohane, R. O., & Verba, S. (1994). Designing social inquiry: Scientific inference in qualitative research. Princeton: Princeton University Press.
Kling, J. R., Liebman, J. B., & Katz, L. F. (2005). Bullets don’t got no name: Consequences of fear in the ghetto. In T. S. Weisner (Ed.), Discovering successful pathways in children’s development: Mixed methods in the study of childhood and family life (pp. 243–281). Chicago: University of Chicago Press.
Kling, J. R., Liebman, J. B., & Katz, L. F. (2007). Experimental analysis of neighborhood effects. Econometrica, 75, 83–119.
Lichterman, P., & Reed, I. A. (2012, March 8–9). Interpretation and explanation in ethnography: A pragmatist approach. Paper presented at the AJS Conference on Causal Thinking and Ethnographic Research, University of Chicago.
Lieberman, E. S. (2005, August). Nested analysis as a mixed-method strategy for comparative research. American Political Science Review, 99(3), 435–452.
Lieberson, S. (1991). Small N’s and big conclusions: An examination of the reasoning in comparative studies based on a small number of cases. Social Forces, 70, 307–320.
Lieberson, S. (1994). More on the uneasy case for using mills-type methods in small-N comparative studies. Social Forces, 72, 1225–1237.
Lin, A. C. (1998). Bridging positivist and interpretivist approaches to qualitative methods. Policy Studies Journal, 26, 162–180.
Lofland, J., & Lofland, L. H. (1995). Analyzing social settings: A guide to qualitative observation and analysis (3rd ed.). New York: Wadsworth.
Mahoney, J. (1999). Nominal, ordinal, and narrative appraisal in macrocausal analysis. The American Journal of Sociology, 104, 1154–1196.
Mahoney, J. (2000). Strategies of inference in small-N analysis. Sociological Methods and Research, 28, 387–424.
Mahoney, J. (2008). Toward a unified theory of causality. Comparative Political Studies, 41, 412–436.
Mahoney, J., & Goertz, G. (2006). A tale of two cultures: Contrasting qualitative and qualitative research. Political Analysis, 14, 227–249.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Thousand Oaks: Sage.
Morgan, S. L., & Winship, C. (2007). Counterfactuals and causal inference: Methods and principles for social research. Cambridge: Cambridge University Press.
Orr, L., Feins, J. D., Jacob, R., Beecroft, E., Sanbonmatsu, L., Katz, L. F., Liebman, J. B., & Kling, J. R. (2003). Moving to opportunity: Interim impacts evaluation. Washington, DC: Department of Housing and Urban Development.
Plano Clark, V. L., & Creswell, J. W. (Eds.). (2008). The mixed methods reader. Thousand Oaks: Sage.
Presser, S., Couper, M., Lessler, J., Martin, E., Martin, J., Rothgeb, J., & Singer, E. (2004). Methods for evaluating and testing survey questions. Public Opinion Quarterly, 68, 109–130.
Ragin, C. C. (2000). Fuzzy-set social science. Chicago: University of Chicago Press.
Ragin, C. C. (2008). Redesigning social inquiry: Fuzzy sets and beyond. Chicago: University of Chicago Press.
Ragin, C. C., & Becker, H. S. (Eds.). (1992). What is a case? Exploring the foundations of social inquiry. Cambridge: Cambridge University Press.
Roth, W. D., & Mehta, J. D. (2002). The Rashomon effect: Combining positivist and interpretivist approaches in the analysis of contested events. Sociological Methods and Research, 31, 131–173.
Rubinowitz, L. S., & Rosenbaum, J. E. (2000). Crossing the class and color lines: From public housing to white suburbia. Chicago: University of Chicago Press.
Sampson, R. J., & Sharkey, P. (2008). Neighborhood selection and the social reproduction of concentrated racial inequality. Demography, 45, 1–29.
Sanbonmatsu, L., Kling, J. R., Duncan, G. J., & Brooks-Gunn, J. (2006). Neighborhoods and academic achievement: Results from the moving to opportunity experiment. Journal of Human Resources, 41, 649–691.
Sale, J. E., Lohfeld, L. H., & Brazil, K. (2002). Revisiting the quantitative-qualitative debate: Implications for mixed-methods research. Quality and Quantity, 36, 43–53.
Schaeffer, N. C., & Presser, S. (2003). The science of asking questions. Annual Review of Sociology, 29, 65–88.
Seefeldt, K. S. (2008). Working after welfare: How women balance jobs and family in the wake of welfare reform. Kalamazoo: W.E. Upjohn Institute Press.
Sherwood, K. E., & Doolittle, F. (2003). What lies behind the impacts? In M. C. Lennon & T. Corbett (Eds.), Policy into action: Implementation research and welfare reform (pp. 193–234). Washington, DC: Urban Institute Press.
Small, M. L. (2009). Unanticipated gains: Origins of network inequality in everyday life. Oxford: Oxford University Press.
Small, M. L. (2011). How to conduct a mixed methods study: Recent trends in a rapidly growing literature. Annual Review of Sociology, 37, 57–86.
Staggenborg, S. (2008). Seeing mechanisms in action. Qualitative Sociology, 31, 341–344.
Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks: Sage.
Tashakkori, A., & Teddlie, C. (2003). Handbook of mixed methods in social and behavioral science. Thousand Oaks: Sage.
Tavory, I., & Timmermans, S. (2012, March 8–9). Consequences in action: A pragmatist approach to causality in ethnography. Paper presented at the AJS Conference on Causal Thinking and Ethnographic Research, University of Chicago.
Votruba, M. E., & Kling, J. R. (2008). Effects of neighborhood characteristics on the mortality of black male youth: Evidence from Gautreaux (Working Paper No. 08-03). Ann Arbor: University of Michigan, National Poverty Center.
Winship, C., & Morgan, S. L. (1999). The estimation of causal effects from observational data. Annual Review of Sociology, 25, 659–706.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Harding, D.J., Seefeldt, K.S. (2013). Mixed Methods and Causal Analysis. In: Morgan, S. (eds) Handbook of Causal Analysis for Social Research. Handbooks of Sociology and Social Research. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-6094-3_6
Download citation
DOI: https://doi.org/10.1007/978-94-007-6094-3_6
Published:
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-007-6093-6
Online ISBN: 978-94-007-6094-3
eBook Packages: Humanities, Social Sciences and LawSocial Sciences (R0)