Advertisement

Quality & Quantity

, Volume 51, Issue 6, pp 2847–2863 | Cite as

Addressing quality in mixed methods research: a review and recommendations for a future agenda

  • Sergi FàbreguesEmail author
  • José F. Molina-Azorín
Article

Abstract

Quality is one of the most debated topics in the recent history of mixed methods research. A growing number of authors are currently discussing how the quality of mixed methods research should be conceptualised and operationalised, with the ultimate aim of promoting well designed and properly implemented mixed methods studies. These authors argue that mixed methods research has a number of unique features with respect to monomethod research and, as such, should be appraised according to its own set of quality criteria. Based on a systematic search of the literature, this review (1) examines the features and trends of the literature on the quality of mixed methods research published until February 2016; (2) provides a metasummary of the most prevalent quality criteria suggested in this literature; and (3) proposes a number of recommendations for future discussion and research on this topic. The review concludes that (1) publications on the quality of mixed methods research are increasingly more prevalent and elaborated; (2) a shared set of core quality criteria for appraising mixed methods research can be identified across publications; and (3) future work on this topic should focus on increasing the number of empirical publications on quality, achieving greater consistency in quality terminology, and reaching an agreement on core quality criteria.

Keywords

Mixed methods research Quality issues Quality criteria Review 

Notes

Acknowledgments

The authors would like to thank Marie-Hélène Paré for her valuable comments on the manuscript.

References

  1. Brown, K.M., Elliott, S.J., Leatherdale, S.T., Robertson-Wilson, J.: Searching for rigour in the reporting of mixed methods population health research: a methodological review. Health Educ. Res. 30(6), 811–839 (2015)CrossRefGoogle Scholar
  2. Bryman, A.: Integrating quantitative and qualitative research: how is it done? Qual. Res. 6(1), 97–113 (2006a)CrossRefGoogle Scholar
  3. Bryman, A.: Paradigm peace and implications for quality. Int. J. Soc. Res. Method 9(2), 111–126 (2006b)CrossRefGoogle Scholar
  4. Bryman, A.: The research question in social research: what is its role? Int. J. Soc. Res. Method 10(1), 5–20 (2007)CrossRefGoogle Scholar
  5. Bryman, A.: June 1989 and beyond: Julia Brannen’s contribution to mixed methods research. Int. J. Soc. Res. Method 17(2), 121–131 (2014)CrossRefGoogle Scholar
  6. Bryman, A., Becker, S., Sempik, J.: Quality criteria for quantitative, qualitative and mixed methods research: a view from social policy. Int. J. Soc. Res. Method 11(4), 261–276 (2008)CrossRefGoogle Scholar
  7. Burrows, T.J.: A preliminary rubric design to evaluate mixed methods research. Doctoral Dissertation, Virginia Tech (2013)Google Scholar
  8. Canales, G.: Transformative, mixed methods checklist for psychological research with mexican americans. J. Mixed Method Res. 7(1), 6–21 (2013)CrossRefGoogle Scholar
  9. Caracelli, V.J., Riggin, L.J.C.: Mixed-method evaluation: developing quality criteria through concept mapping. Eval. Pract. 15(2), 139–152 (1994)CrossRefGoogle Scholar
  10. Cheek, J.: It depends: possible impacts of moving the field of mixed methods research toward best practice guidelines. In: Hesse-Biber, S., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 624–636. Oxford University Press, New York (2015)Google Scholar
  11. Collins, K.M.T.: Validity in multimethod and mixed research. In: Hesse-Biber, S., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 240–256. Oxford University Press, New York (2015)Google Scholar
  12. Collins, K.M.T., Onwuegbuzie, A.J., Jiao, Q.G.: A mixed methods investigation of mixed methods sampling designs in social and health science research. J. Mixed Method Res. 1(3), 267–294 (2007)CrossRefGoogle Scholar
  13. Collins, K.M.T., Onwuegbuzie, A.J., Johnson, B.: Securing a place at the table: a review and extension of legitimation criteria for the conduct of mixed research. Am. Behav. Sci. 56(6), 849–865 (2012)CrossRefGoogle Scholar
  14. Creswell, J.W.: A Concise Introduction to Mixed Methods Research. Sage, Thousand Oaks (2015a)Google Scholar
  15. Creswell, J.W.: Revisiting mixed methods and advancing scientific practices. In: Hesse-Biber, S., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 57–71. Oxford University Press, New York (2015b)Google Scholar
  16. Creswell, J.W., Plano-Clark, V.L.: Designing and Conducting Mixed Methods Research, 2nd edn. Sage, Thousand Oaks (2011)Google Scholar
  17. Creswell, J.W., Tashakkori, A.: Editorial: developing publishable mixed methods manuscripts. J. Mixed Method Res. 1(2), 107–111 (2007)CrossRefGoogle Scholar
  18. Creswell, J.W., Klassen, A.C., Plano Clark, V.L., Smith, K.C.: Best Practices for Mixed Methods Research in the Health Sciences. National Institutes of Health, Washington, DC (2011)CrossRefGoogle Scholar
  19. Curry, L.A., Nunez-Smith, M.: Mixed Methods in Health Sciences Research: A Practical Primer. Sage, Thousand Oaks (2015)Google Scholar
  20. Dahlberg, B., Wittink, M.N., Gallo, J.J.: Funding and publishing integrated studies: Writing effective mixed methods manuscripts and grant proposals. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 775–802. Sage, Thousand Oaks (2010)CrossRefGoogle Scholar
  21. Dellinger, A.B., Leech, N.L.: Toward a unified validation framework in mixed methods research. J. Mixed Method Res. 1(4), 309–332 (2007)CrossRefGoogle Scholar
  22. Giddings, L.S., Grant, B.M.: From rigour to trustworthiness: validating mixed methods. In: Andrew, S., Halcomb, E.J. (eds.) Mixed Methods Research for Nursing and the Health Sciences, pp. 119–134. Wiley-Blackwell, Oxford (2009)Google Scholar
  23. Greene, J.C.: Mixed Methods in Social Inquiry. Jossey-Bass, San Francisco (2007)Google Scholar
  24. Hadi, M.A., Alldred, D.P., Closs, S.J., Briggs, M.: Mixed-methods research in pharmacy practice: recommendations for quality reporting (part 2). Int. J. Pharm. Pract. 22(1), 96–100 (2014)CrossRefGoogle Scholar
  25. Halcomb, E.J., Hickman, L.: Mixed methods research. Nurs. Stand. 29(32), 41–47 (2015)CrossRefGoogle Scholar
  26. Hammersley, M.: The issue of quality in qualitative research. Int. J. Res. Method Educ. 30(3), 287–305 (2007)CrossRefGoogle Scholar
  27. Hesse-Biber, S., Johnson, R.B.: Editorial: coming at things differently: future directions of possible engagement with mixed methods research. J. Mixed Method Res. 7(2), 103–109 (2013)CrossRefGoogle Scholar
  28. Heyvaert, M., Hannes, K., Maes, B., Onghena, P.: Critical appraisal of mixed methods studies. J. Mixed Method Res. 7(4), 302–327 (2013)CrossRefGoogle Scholar
  29. Ihantola, E.-M., Kihn, L.-A.: Threats to validity and reliability in mixed methods accounting research. Qual. Res. Acc. Manag. 8(1), 39–58 (2011)CrossRefGoogle Scholar
  30. Ivankova, N.V.: Implementing quality criteria in designing and conducting a sequential QUAN → QUAL mixed methods study of student engagement with learning applied research methods online. J. Mixed Method Res. 8(1), 25–51 (2014)CrossRefGoogle Scholar
  31. Ivankova, N.V.: Mixed Methods Applications in Action Research. Sage, Thousand Oaks (2015)Google Scholar
  32. Ivankova, N.V., Kawamura, Y.: Emerging trends in the utilization of integrated designs in the social, behavioral, and health sciences. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 581–612. Sage, Thousand Oaks (2010)CrossRefGoogle Scholar
  33. Johnson, R.B., Onwuegbuzie, A.J., Turner, L.A.: Towards a definition of mixed methods research. J. Mixed Method Res. 1(2), 112–133 (2007)CrossRefGoogle Scholar
  34. Lavelle, E., Vuk, J., Barber, C.: Twelve tips for getting started using mixed methods in medical education research. Med. Teach. 35(4), 272–276 (2013)CrossRefGoogle Scholar
  35. Leech, N.L.: Interviews with the early developers of mixed methods research. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 253–272. Sage, Thousand Oaks (2010)CrossRefGoogle Scholar
  36. Leech, N.L.: Writing mixed research reports. Am. Behav. Sci. 56(6), 866–881 (2012)CrossRefGoogle Scholar
  37. Leech, N.L., Dellinger, A.B.: Validity: Mixed methods. In: Chapelle, C.A. (ed.) The Encyclopedia of Applied Linguistics, pp. 1–7. Wiley-Blackwell, Oxford (2013)Google Scholar
  38. Leech, N.L., Onwuegbuzie, A.J.: A typology of mixed methods research designs. Qual. Quant. 43(2), 265–275 (2009)CrossRefGoogle Scholar
  39. Mertens, D.M.: Editorial: publishing mixed methods research. J. Mixed Method Res. 5(1), 3–6 (2011)CrossRefGoogle Scholar
  40. Mertens, D.M.: Mixed methods. In: Trainor, A.A., Graue, E. (eds.) Reviewing Qualitative Research in the Social Sciences, pp. 139–150. Routledge, New York (2013)Google Scholar
  41. Mertens, D.M.: Research and Evaluation in Education and Psychology, 4th edn. Sage, Thousand Oaks (2015)Google Scholar
  42. Mertens, D.M., Bazeley, P., Bowleg, L., Fielding, N., Maxwell, J., Molina-Azorin, J.F., Niglas, K.: The future of mixed methods: a five year projection to 2020. MMIRA (2016). https://mmira.wildapricot.org/resources/Documents/MMIRAtaskforcereportJan2016final.pdf
  43. Molina-Azorin, J.F.: The use and added value of mixed methods in management research. J. Mixed Method Res. 5(1), 7–24 (2011)CrossRefGoogle Scholar
  44. Morse, J.M., Niehaus, L.: Mixed Method Design: Principles and Procedures. Left Coast Press, Walnut Creek (2009)Google Scholar
  45. Nastasi, B.K., Hitchcock, J.H., Brown, L.M.: An inclusive framework for conceptualizing mixed methods design typologies: Moving toward fully integrated synergistic research models. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 305–338. Sage, Thousand Oaks (2010)CrossRefGoogle Scholar
  46. O’Cathain, A.: Assessing the quality of mixed methods research: Toward a comprehensive framework. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 531–555. Sage, Thousand Oaks (2010)CrossRefGoogle Scholar
  47. O’Cathain, A., Murphy, E., Nicholl, J.: The quality of mixed methods studies in health services research. J. Health Serv. Res. Policy 13(2), 92–98 (2008)CrossRefGoogle Scholar
  48. Onwuegbuzie, A.J., Corrigan, J.A.: Improving the quality of mixed research reports in the field of human resource development and beyond: a call for rigor as an ethical practice. Hum. Resour. Dev. Q. 25(3), 273–299 (2014)CrossRefGoogle Scholar
  49. Onwuegbuzie, A.J., Johnson, R.K.: The validity issue in mixed research. Res. Sch. 13(1), 48–63 (2006)Google Scholar
  50. Onwuegbuzie, A.J., Poth, C.: Editors’ afterword: toward evidence-based guidelines for reviewing mixed methods research manuscripts submitted to journals. Int. J. Qual. Method 15(1), 1–13 (2016)CrossRefGoogle Scholar
  51. Onwuegbuzie, A.J., Johnson, R.B., Collins, K.M.T.: Assessing legitimation in mixed research: a new framework. Qual. Quant. 45(6), 1253–1271 (2011)CrossRefGoogle Scholar
  52. Papadimitriou, A., Ivankova, N.V., Hurtado, S.: Addressing challenges of conducting quality mixed methods studies in higher education. In: Tight, M. (ed.) Theory and Method in Higher Education Research, pp. 133–153. Bingley, Emerald (2013)CrossRefGoogle Scholar
  53. Plano-Clark, V.L., Ivankova, N.V.: Mixed Methods Research: A Guide to the Field. Sage, Thousand Oaks (2016)Google Scholar
  54. Plano-Clark, V.L., Huddleston-Casas, C.A., Churchill, S.L., Green, D.O., Garrett, A.L.: Mixed methods approaches in family science research. J. Fam. Issues 29(11), 1543–1566 (2008)CrossRefGoogle Scholar
  55. Pluye, P., Gagnon, M.-P., Griffiths, F., Johnson-Lafleur, J.: A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in Mixed Studies Reviews. Int. J. Nurs. Stud. 46(4), 529–546 (2009)CrossRefGoogle Scholar
  56. Ridenour, C.S., Newman, I.: Mixed methods research: exploring the interactive continuum. Southern Illinois University Press, Illinois (2008)Google Scholar
  57. Ritchie, J., Spencer, L., O’Connor, W.: Carrying out qualitative analysis. In: Ritchie, J., Lewis, J. (eds.) Qualitative Research Practice: A Guide for Social Science Students and Researchers, pp. 219–262. Sage, London (2003)Google Scholar
  58. Sale, J.E.M., Brazil, K.: A strategy to identify critical appraisal criteria for primary mixed-method studies. Qual. Quant. 38(4), 351–365 (2004)CrossRefGoogle Scholar
  59. Sandelowski, M., Barroso, J.: Handbook for Synthesizing Qualitative Research. Springer, New York (2007)Google Scholar
  60. Schifferdecker, K.E., Reed, V.A.: Using mixed methods research in medical education: basic guidelines for researchers. Med. Educ. 43(7), 637–644 (2009)CrossRefGoogle Scholar
  61. Seale, C.: The Quality of Qualitative Research. Sage, London (1999)CrossRefGoogle Scholar
  62. Tashakkori, A., Teddlie, C.: Quality of inferences in mixed methods research: calling for an integrative framework. In: Bergman, M.M. (ed.) Advances in Mixed Methods Research, pp. 101–119. Sage, London (2008)Google Scholar
  63. Tashakkori, A., Teddlie, C.: Epilogue: Current developments and emerging trends in integrated research methodology. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 803–826. Sage, Thousand Oaks (2010)CrossRefGoogle Scholar
  64. Teddlie, C., Tashakkori, A.: Major issues and controversies in the use of mixed methods in the social and behavioral sciences. In: Tashakkori, A., Teddlie, C. (eds.) Handbook of Mixed Methods in Social and Behavioral Research, pp. 3–50. Sage, Thousand Oaks (2003)Google Scholar
  65. Teddlie, C., Tashakkori, A.: Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences. Sage, Thousand Oaks (2009)Google Scholar
  66. Teddlie, C., Tashakkori, A.: Overview of contemporary issues in mixed methods research. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 1–41. Sage, Thousand Oaks (2010)Google Scholar
  67. Venkatesh, V., Brown, S.A., Bala, H.: Bridging the qualitative-quantitative divide: guidelines for conducting mixed methods research in Information Systems. MIS Q. 37(1), 21–54 (2013)CrossRefGoogle Scholar
  68. Wisdom, J.P., Cavaleri, M.A., Onwuegbuzie, A.J., Green, C.A.: Methodological reporting in qualitative, quantitative, and mixed methods health services research articles. Health Serv. Res. 47(2), 721–745 (2012)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  1. 1.Department of Psychology and EducationUniversitat Oberta de CatalunyaBarcelonaSpain
  2. 2.Department of ManagementUniversity of AlicanteAlicanteSpain

Personalised recommendations