Abstract
Quality is one of the most debated topics in the recent history of mixed methods research. A growing number of authors are currently discussing how the quality of mixed methods research should be conceptualised and operationalised, with the ultimate aim of promoting well designed and properly implemented mixed methods studies. These authors argue that mixed methods research has a number of unique features with respect to monomethod research and, as such, should be appraised according to its own set of quality criteria. Based on a systematic search of the literature, this review (1) examines the features and trends of the literature on the quality of mixed methods research published until February 2016; (2) provides a metasummary of the most prevalent quality criteria suggested in this literature; and (3) proposes a number of recommendations for future discussion and research on this topic. The review concludes that (1) publications on the quality of mixed methods research are increasingly more prevalent and elaborated; (2) a shared set of core quality criteria for appraising mixed methods research can be identified across publications; and (3) future work on this topic should focus on increasing the number of empirical publications on quality, achieving greater consistency in quality terminology, and reaching an agreement on core quality criteria.
Similar content being viewed by others
References
Brown, K.M., Elliott, S.J., Leatherdale, S.T., Robertson-Wilson, J.: Searching for rigour in the reporting of mixed methods population health research: a methodological review. Health Educ. Res. 30(6), 811–839 (2015)
Bryman, A.: Integrating quantitative and qualitative research: how is it done? Qual. Res. 6(1), 97–113 (2006a)
Bryman, A.: Paradigm peace and implications for quality. Int. J. Soc. Res. Method 9(2), 111–126 (2006b)
Bryman, A.: The research question in social research: what is its role? Int. J. Soc. Res. Method 10(1), 5–20 (2007)
Bryman, A.: June 1989 and beyond: Julia Brannen’s contribution to mixed methods research. Int. J. Soc. Res. Method 17(2), 121–131 (2014)
Bryman, A., Becker, S., Sempik, J.: Quality criteria for quantitative, qualitative and mixed methods research: a view from social policy. Int. J. Soc. Res. Method 11(4), 261–276 (2008)
Burrows, T.J.: A preliminary rubric design to evaluate mixed methods research. Doctoral Dissertation, Virginia Tech (2013)
Canales, G.: Transformative, mixed methods checklist for psychological research with mexican americans. J. Mixed Method Res. 7(1), 6–21 (2013)
Caracelli, V.J., Riggin, L.J.C.: Mixed-method evaluation: developing quality criteria through concept mapping. Eval. Pract. 15(2), 139–152 (1994)
Cheek, J.: It depends: possible impacts of moving the field of mixed methods research toward best practice guidelines. In: Hesse-Biber, S., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 624–636. Oxford University Press, New York (2015)
Collins, K.M.T.: Validity in multimethod and mixed research. In: Hesse-Biber, S., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 240–256. Oxford University Press, New York (2015)
Collins, K.M.T., Onwuegbuzie, A.J., Jiao, Q.G.: A mixed methods investigation of mixed methods sampling designs in social and health science research. J. Mixed Method Res. 1(3), 267–294 (2007)
Collins, K.M.T., Onwuegbuzie, A.J., Johnson, B.: Securing a place at the table: a review and extension of legitimation criteria for the conduct of mixed research. Am. Behav. Sci. 56(6), 849–865 (2012)
Creswell, J.W.: A Concise Introduction to Mixed Methods Research. Sage, Thousand Oaks (2015a)
Creswell, J.W.: Revisiting mixed methods and advancing scientific practices. In: Hesse-Biber, S., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 57–71. Oxford University Press, New York (2015b)
Creswell, J.W., Plano-Clark, V.L.: Designing and Conducting Mixed Methods Research, 2nd edn. Sage, Thousand Oaks (2011)
Creswell, J.W., Tashakkori, A.: Editorial: developing publishable mixed methods manuscripts. J. Mixed Method Res. 1(2), 107–111 (2007)
Creswell, J.W., Klassen, A.C., Plano Clark, V.L., Smith, K.C.: Best Practices for Mixed Methods Research in the Health Sciences. National Institutes of Health, Washington, DC (2011)
Curry, L.A., Nunez-Smith, M.: Mixed Methods in Health Sciences Research: A Practical Primer. Sage, Thousand Oaks (2015)
Dahlberg, B., Wittink, M.N., Gallo, J.J.: Funding and publishing integrated studies: Writing effective mixed methods manuscripts and grant proposals. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 775–802. Sage, Thousand Oaks (2010)
Dellinger, A.B., Leech, N.L.: Toward a unified validation framework in mixed methods research. J. Mixed Method Res. 1(4), 309–332 (2007)
Giddings, L.S., Grant, B.M.: From rigour to trustworthiness: validating mixed methods. In: Andrew, S., Halcomb, E.J. (eds.) Mixed Methods Research for Nursing and the Health Sciences, pp. 119–134. Wiley-Blackwell, Oxford (2009)
Greene, J.C.: Mixed Methods in Social Inquiry. Jossey-Bass, San Francisco (2007)
Hadi, M.A., Alldred, D.P., Closs, S.J., Briggs, M.: Mixed-methods research in pharmacy practice: recommendations for quality reporting (part 2). Int. J. Pharm. Pract. 22(1), 96–100 (2014)
Halcomb, E.J., Hickman, L.: Mixed methods research. Nurs. Stand. 29(32), 41–47 (2015)
Hammersley, M.: The issue of quality in qualitative research. Int. J. Res. Method Educ. 30(3), 287–305 (2007)
Hesse-Biber, S., Johnson, R.B.: Editorial: coming at things differently: future directions of possible engagement with mixed methods research. J. Mixed Method Res. 7(2), 103–109 (2013)
Heyvaert, M., Hannes, K., Maes, B., Onghena, P.: Critical appraisal of mixed methods studies. J. Mixed Method Res. 7(4), 302–327 (2013)
Ihantola, E.-M., Kihn, L.-A.: Threats to validity and reliability in mixed methods accounting research. Qual. Res. Acc. Manag. 8(1), 39–58 (2011)
Ivankova, N.V.: Implementing quality criteria in designing and conducting a sequential QUAN → QUAL mixed methods study of student engagement with learning applied research methods online. J. Mixed Method Res. 8(1), 25–51 (2014)
Ivankova, N.V.: Mixed Methods Applications in Action Research. Sage, Thousand Oaks (2015)
Ivankova, N.V., Kawamura, Y.: Emerging trends in the utilization of integrated designs in the social, behavioral, and health sciences. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 581–612. Sage, Thousand Oaks (2010)
Johnson, R.B., Onwuegbuzie, A.J., Turner, L.A.: Towards a definition of mixed methods research. J. Mixed Method Res. 1(2), 112–133 (2007)
Lavelle, E., Vuk, J., Barber, C.: Twelve tips for getting started using mixed methods in medical education research. Med. Teach. 35(4), 272–276 (2013)
Leech, N.L.: Interviews with the early developers of mixed methods research. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 253–272. Sage, Thousand Oaks (2010)
Leech, N.L.: Writing mixed research reports. Am. Behav. Sci. 56(6), 866–881 (2012)
Leech, N.L., Dellinger, A.B.: Validity: Mixed methods. In: Chapelle, C.A. (ed.) The Encyclopedia of Applied Linguistics, pp. 1–7. Wiley-Blackwell, Oxford (2013)
Leech, N.L., Onwuegbuzie, A.J.: A typology of mixed methods research designs. Qual. Quant. 43(2), 265–275 (2009)
Mertens, D.M.: Editorial: publishing mixed methods research. J. Mixed Method Res. 5(1), 3–6 (2011)
Mertens, D.M.: Mixed methods. In: Trainor, A.A., Graue, E. (eds.) Reviewing Qualitative Research in the Social Sciences, pp. 139–150. Routledge, New York (2013)
Mertens, D.M.: Research and Evaluation in Education and Psychology, 4th edn. Sage, Thousand Oaks (2015)
Mertens, D.M., Bazeley, P., Bowleg, L., Fielding, N., Maxwell, J., Molina-Azorin, J.F., Niglas, K.: The future of mixed methods: a five year projection to 2020. MMIRA (2016). https://mmira.wildapricot.org/resources/Documents/MMIRAtaskforcereportJan2016final.pdf
Molina-Azorin, J.F.: The use and added value of mixed methods in management research. J. Mixed Method Res. 5(1), 7–24 (2011)
Morse, J.M., Niehaus, L.: Mixed Method Design: Principles and Procedures. Left Coast Press, Walnut Creek (2009)
Nastasi, B.K., Hitchcock, J.H., Brown, L.M.: An inclusive framework for conceptualizing mixed methods design typologies: Moving toward fully integrated synergistic research models. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 305–338. Sage, Thousand Oaks (2010)
O’Cathain, A.: Assessing the quality of mixed methods research: Toward a comprehensive framework. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 531–555. Sage, Thousand Oaks (2010)
O’Cathain, A., Murphy, E., Nicholl, J.: The quality of mixed methods studies in health services research. J. Health Serv. Res. Policy 13(2), 92–98 (2008)
Onwuegbuzie, A.J., Corrigan, J.A.: Improving the quality of mixed research reports in the field of human resource development and beyond: a call for rigor as an ethical practice. Hum. Resour. Dev. Q. 25(3), 273–299 (2014)
Onwuegbuzie, A.J., Johnson, R.K.: The validity issue in mixed research. Res. Sch. 13(1), 48–63 (2006)
Onwuegbuzie, A.J., Poth, C.: Editors’ afterword: toward evidence-based guidelines for reviewing mixed methods research manuscripts submitted to journals. Int. J. Qual. Method 15(1), 1–13 (2016)
Onwuegbuzie, A.J., Johnson, R.B., Collins, K.M.T.: Assessing legitimation in mixed research: a new framework. Qual. Quant. 45(6), 1253–1271 (2011)
Papadimitriou, A., Ivankova, N.V., Hurtado, S.: Addressing challenges of conducting quality mixed methods studies in higher education. In: Tight, M. (ed.) Theory and Method in Higher Education Research, pp. 133–153. Bingley, Emerald (2013)
Plano-Clark, V.L., Ivankova, N.V.: Mixed Methods Research: A Guide to the Field. Sage, Thousand Oaks (2016)
Plano-Clark, V.L., Huddleston-Casas, C.A., Churchill, S.L., Green, D.O., Garrett, A.L.: Mixed methods approaches in family science research. J. Fam. Issues 29(11), 1543–1566 (2008)
Pluye, P., Gagnon, M.-P., Griffiths, F., Johnson-Lafleur, J.: A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in Mixed Studies Reviews. Int. J. Nurs. Stud. 46(4), 529–546 (2009)
Ridenour, C.S., Newman, I.: Mixed methods research: exploring the interactive continuum. Southern Illinois University Press, Illinois (2008)
Ritchie, J., Spencer, L., O’Connor, W.: Carrying out qualitative analysis. In: Ritchie, J., Lewis, J. (eds.) Qualitative Research Practice: A Guide for Social Science Students and Researchers, pp. 219–262. Sage, London (2003)
Sale, J.E.M., Brazil, K.: A strategy to identify critical appraisal criteria for primary mixed-method studies. Qual. Quant. 38(4), 351–365 (2004)
Sandelowski, M., Barroso, J.: Handbook for Synthesizing Qualitative Research. Springer, New York (2007)
Schifferdecker, K.E., Reed, V.A.: Using mixed methods research in medical education: basic guidelines for researchers. Med. Educ. 43(7), 637–644 (2009)
Seale, C.: The Quality of Qualitative Research. Sage, London (1999)
Tashakkori, A., Teddlie, C.: Quality of inferences in mixed methods research: calling for an integrative framework. In: Bergman, M.M. (ed.) Advances in Mixed Methods Research, pp. 101–119. Sage, London (2008)
Tashakkori, A., Teddlie, C.: Epilogue: Current developments and emerging trends in integrated research methodology. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 803–826. Sage, Thousand Oaks (2010)
Teddlie, C., Tashakkori, A.: Major issues and controversies in the use of mixed methods in the social and behavioral sciences. In: Tashakkori, A., Teddlie, C. (eds.) Handbook of Mixed Methods in Social and Behavioral Research, pp. 3–50. Sage, Thousand Oaks (2003)
Teddlie, C., Tashakkori, A.: Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences. Sage, Thousand Oaks (2009)
Teddlie, C., Tashakkori, A.: Overview of contemporary issues in mixed methods research. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 1–41. Sage, Thousand Oaks (2010)
Venkatesh, V., Brown, S.A., Bala, H.: Bridging the qualitative-quantitative divide: guidelines for conducting mixed methods research in Information Systems. MIS Q. 37(1), 21–54 (2013)
Wisdom, J.P., Cavaleri, M.A., Onwuegbuzie, A.J., Green, C.A.: Methodological reporting in qualitative, quantitative, and mixed methods health services research articles. Health Serv. Res. 47(2), 721–745 (2012)
Acknowledgments
The authors would like to thank Marie-Hélène Paré for her valuable comments on the manuscript.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fàbregues, S., Molina-Azorín, J.F. Addressing quality in mixed methods research: a review and recommendations for a future agenda. Qual Quant 51, 2847–2863 (2017). https://doi.org/10.1007/s11135-016-0449-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11135-016-0449-4