Advertisement

Quality & Quantity

, Volume 47, Issue 2, pp 659–676 | Cite as

Mixed methods research synthesis: definition, framework, and potential

  • M. HeyvaertEmail author
  • B. Maes
  • P. Onghena
Article

Abstract

Literature on the combination of qualitative and quantitative research components at the primary empirical study level has recently accumulated exponentially. However, this combination is only rarely discussed and applied at the research synthesis level. The purpose of this paper is to explore the possible contribution of mixed methods research to the integration of qualitative and quantitative research at the synthesis level. In order to contribute to the methodology and utilization of mixed methods at the synthesis level, we present a framework to perform mixed methods research syntheses (MMRS). The presented classification framework can help to inform researchers intending to carry out MMRS, and to provide ideas for conceptualizing and developing those syntheses. We illustrate the use of this framework by applying it to the planning of MMRS on effectiveness studies concerning interventions for challenging behavior in persons with intellectual disabilities, presenting two hypothetical examples. Finally, we discuss possible strengths of MMRS and note some remaining challenges concerning the implementation of these syntheses.

Keywords

Mixed methods research Mixed methodology Systematic review Research synthesis Intellectual disability Challenging behavior 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Andrew, S., Halcomb, E.J.: Mixed methods research is an effective method of enquiry for community health research. Adv. Contemp. Commun. Family Health (2006). doi: 10.5172/conu.2006.23.2.145
  2. Antonacci, D.J., Manuel, C., Davis, E.: Diagnosis and treatment of aggression in individuals with developmental disabilities. Psychiatry Q. (2008). doi: 10.1007/s11126-008-9080-4
  3. Attride-Stirling, J.: Thematic networks: an analytic tool for qualitative research. Qual. Res. (2001). doi: 10.1177/146879410100100307
  4. Balogh, R., Ouellette-Kuntz, H., Bourne, L., Lunsky, Y., Colantonio, A.: Organising health care services for persons with an intellectual disability. Cochrane Database Syst. Rev. (2008). doi: 10.1002/14651858.CD007492
  5. Bazeley P.: Issues in mixing qualitative and quantitative approaches to research. In: Buber, R., Gadner, J., Richards, L. (eds) Applying Qualitative Methods to Marketing Management Research, pp. 141–156. Palgrave Macmillan, UK (2004)Google Scholar
  6. Borenstein M., Hedges L.V., Higgins J., Rothstein H.: Introduction to Meta-Analysis. Wiley, Chichester (2009)CrossRefGoogle Scholar
  7. Bouras N.: Psychiatric and Behavioural Disorders in Developmental Disabilities and Mental Retardation. Cambridge University Press, Cambridge (1999)Google Scholar
  8. Braun, V., Clarke, V.: Using thematic analysis in psychology. Qual. Res. Psychol. (2006). doi: 10.1191/1478088706qp063oa
  9. Brylewski, J., Duggan, L.: Antipsychotic medication for challenging behaviour in people with intellectual disability: a systematic review of randomized controlled trials. J. Intellect. Disabil. Res. (1999). doi: 10.1046/j.1365-2788.1999.043005360.x
  10. Bryman, A.: Integrating quantitative and qualitative research: how is it done? Qual. Res. (2006). doi: 10.1177/1468794106058877
  11. Bryman, A.: Barriers to integrating quantitative and qualitative research. J. Mix. Methods Res. (2007). doi: 10.1177/2345678906290531
  12. Chalmers, I., Hedges, L.V., Cooper, H.: A brief history of research synthesis. Eval. Health Prof. (2002). doi: 10.1177/0163278702025001003
  13. Chan, S.W.C., Thompson, D.R., Chau, J.P.C., Tam, W.W.S., Chiu, I.W.S., Lo, S.H.S.: The effects of multisensory therapy on behaviour of adult clients with developmental disabilities. A systematic review. Int. J. Nurs. Stud. (2010). doi: 10.1016/j.ijnurstu.2009.08.004
  14. Collins K., O’Cathain A.: Ten points about mixed methods research to be considered by the novice researcher. Int. J. Multiple Res. Approaches 3, 2–7 (2009)CrossRefGoogle Scholar
  15. Cooper H.: Synthesizing Research: A Guide for Literature Reviews. 3rd edn. Sage, London (1998)Google Scholar
  16. Cooper H., Hedges L.V.: The Handbook of Research Synthesis. Russell Sage Foundation, New York (1994)Google Scholar
  17. Creswell J.W.: Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. 2nd edn. Sage, Thousand Oaks (2003)Google Scholar
  18. Creswell J.W., Plano Clark V.L.: Designing and Conducting Mixed Methods Research. Sage, Thousand Oaks (2007)Google Scholar
  19. Creswell, J.W., Tashakkori, A.: Developing publishable mixed methods manuscripts. J. Mix. Methods Res. (2007a). doi: 10.1177/1558689806298644
  20. Creswell, J.W., Tashakkori, A.: Differing perspectives on mixed methods research. J. Mix. Methods Res. (2007b). doi: 10.1177/1558689807306132
  21. Deb, S., Sohanpal, S.K., Soni, R., Lenotre, L., Unwin, G.: The effectiveness of antipsychotic medication in the management of behaviour problems in adults with intellectual disabilities. J. Intellect. Disabil. Res. (2007). doi: 10.1111/j.1365-2788.2007.00950.x
  22. Deb, S., Chaplin, R., Sohanpal, S.K., Unwin, G., Soni, R., Lenotre, L.: The effectiveness of mood stabilizers and antiepileptic medication for the management of behaviour problems in adults with intellectual disability: a systematic review. J. Intellect. Disabil. Res. (2008). doi: 10.1111/j.1365-2788.2007.00965.x
  23. Dellinger, A.B., Leech, N.L.: Toward a unified validation framework in mixed methods research. J. Mix. Methods Res. (2007). doi: 10.1177/1558689807306147
  24. Denscombe, M.: Communities of practice: a research paradigm for the mixed methods approach. J. Mix. Methods Res. (2008). doi: 10.1177/1558689808316807
  25. Didden R., Duker P.C., Korzilius H.: Meta-analytic study on treatment effectiveness for problem behaviors with individuals who have mental retardation. Am. J. Ment. Retard. 101, 387–399 (1997)Google Scholar
  26. Didden, R., Korzilius, H., van Oorsouw, W., Sturmey, P.: Behavioral treatment of challenging behaviors in individuals with mild mental retardation: meta-analysis of single-subject research. Am. J. Ment. Retard. (2006). doi: 10.1352/0895-8017(2006)111[290:BTOCBI]2.0.CO;2
  27. Dixon-Woods, M., Fitzpatrick, R., Roberts, K.: Including qualitative research in systematic reviews: opportunities and problems. J. Eval. Clin. Pract. (2001). doi: 10.1046/j.1365-2753.2001.00257.x
  28. Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A.: Synthesising qualitative and quantitative evidence: a review of possible methods. J. Health Serv. Res. Policy 10, 45–53 (2005)CrossRefGoogle Scholar
  29. Dösen A., Day K.: Treating mental illness and behaviour disorders in children and adults with mental retardation. American Psychiatric Publications, Washington, DC (2001)Google Scholar
  30. Doyle, L., Brady, A.-M., Byrne, G.: An overview of mixed methods research. J. Res. Nurs. (2009). doi: 10.1177/1744987108093962
  31. Emerson E.: Challenging Behaviour. Analysis and Intervention in People with Learning Difficulties. Cambridge University Press, Cambridge (1995)Google Scholar
  32. Eysenck, H.J.: An exercise in mega-silliness. Am. Psychol. (1978). doi: 10.1037/0003-066X.33.5.517.a
  33. Feilzer, M.Y.: Doing mixed methods research pragmatically: implications for the rediscovery of pragmatism as a research paradigm. J. Mix. Methods Res. (2010). doi: 10.1177/1558689809349691
  34. Forbes, A., Griffiths, P.: Methodological strategies for the identification and synthesis of ‘evidence’ to support decision-making in relation to complex health-care systems and practices. Nurs. Inq. (2002). doi: 10.1046/j.1440-1800.2002.00146.x
  35. Furlan, A.D., Pennick, V., Bombardier, C., Van Tulder, M.: 2009 Updated method guidelines for systematic reviews in the Cochrane Back Review Group. Spine (2009). doi: 10.1097/BRS.0b013e3181b1c99f
  36. Gelo, O., Braakmann, D., Benetka, G.: Quantitative and qualitative research: beyond the debate. Integr. Psychol. Behav. Sci. (2008). doi: 10.1007/s12124-008-9078-3
  37. Greene J.C.: Toward a methodology of mixed methods social inquiry. Res. Sch. 13, 93–98 (2006)Google Scholar
  38. Greene J.C.: Mixed Methods in Social Inquiry. Jossey-Bass, San Francisco (2007)Google Scholar
  39. Greene, J.C.: Is mixed methods social inquiry a distinctive methodology? J. Mix. Methods Res. (2008). doi: 10.1177/1558689807309969
  40. Greene J.C., Caracelli V.J.: Advances in Mixed-Method Evaluation: The Challenges and Benefits of Integrating Diverse Paradigms. Jossey-Bass, San Francisco (1997)Google Scholar
  41. Greene, J.C., Caracelli, V.J., Graham, W.F.: Toward a conceptual framework for mixed-method evaluation designs. Educ. Eval. Policy Anal. (1989). doi: 10.3102/01623737011003255
  42. Grey I.M., Hastings R.P.: Evidence-based practices in intellectual disability and behaviour disorders. Curr. Opin. Psychiatry 18, 469–475 (2005)CrossRefGoogle Scholar
  43. Gustafsson, C., Ojehagen, A., Hansson, L., Sandlund, M., Nyström, M., Glad, J., et al.: Effects of psychosocial interventions for people with intellectual disabilities and mental health problems. Res. Soc. Work Pract. (2009). doi: 10.1177/1049731508329403
  44. Hampton, J.R.: Evidence-based medicine, opinion-based medicine, and real-world medicine. Perspect. Biol. Med. (2002). doi: 10.1353/pbm.2002.0070
  45. Hanson, B.: Wither qualitative/quantitative? Grounds for methodological convergence. Qual. Quant. (2008). doi: 10.1007/s11135-006-9041-7
  46. Harden, A., Thomas, J.: Methodological issues in combining diverse study types in systematic reviews. Int. J. Soc. Res. Methodol. (2005). doi: 10.1080/13645570500155078
  47. Harden A., Thomas J.: Mixed methods and systematic reviews. In: Tashakkori, A., Teddlie, C. (eds) Sage Handbook of Mixed Methods in Social & Behavioral Research, pp. 749–774. Sage, Thousand Oaks (2010)Google Scholar
  48. Harden, A., Garcia, J., Oliver, S., Rees, R., Shepherd, J., Brunton, G., et al.: Applying systematic review methods to studies of people’s views: an example from public health. J. Epidemiol. Commun. Health (2004). doi: 10.1136/jech.2003.014829
  49. Hart, L.C., Smith, S.Z., Swars, S.L., Smith, M.E.: An examination of research methods in mathematics education (1995–2005). J. Mix. Methods Res. (2009). doi: 10.1177/1558689808325771
  50. Harvey, S.T., Boer, D., Meyer, L.H., Evans, I.M.: Updating a meta-analysis of intervention research with challenging behaviour: treatment validity and standards of practice. J. Intellect. Dev. Disabil. (2009). doi: 10.1080/13668250802690922
  51. Heyvaert, M., Maes, B., Onghena, P.: A meta-analysis of intervention effects on challenging behaviour among persons with intellectual disabilities. J. Intellect. Disabil. Res. (2010). doi: 10.1111/j.1365-2788.2010.01291.x
  52. Jang, E.E., McDougall, D.E., Pollon, D., Herbert, M., Russell, P.: Integrative mixed methods data analytic strategies in research on school success in challenging circumstances. J. Mix. Methods Res. (2008). doi: 10.1177/1558689808315323
  53. Jensen, L.A., Allen, M.N.: Meta-synthesis of qualitative findings. Qual. Health Res. (1996). doi: 10.1177/104973239600600407
  54. Johnson, R.B., Onwuegbuzie, A.J.: Mixed methods research: a research paradigm whose time has come. Educ. Res. (2004). doi: 10.3102/0013189X033007014
  55. Johnson, R.B., Onwuegbuzie, A.J., Turner, L.A.: Toward a definition of mixed methods research. J. Mix. Methods Res. (2007). doi: 10.1177/1558689806298224
  56. Khan, K.S., Riet, G., Popay, J., Nixon, J., Kleijnen, J.: Undertaking systematic reviews of research on effectiveness. Centre for Reviews and Dissemination report no. 4, pp. 1–20 (2001)Google Scholar
  57. Kuppens, S., Onghena, P.: Are there enough pieces to unravel the puzzle? A method to determine sufficiency in single-case research synthesis. Paper presented at the Annual Meeting of the American Educational Research Association, Denver, CO (2010)Google Scholar
  58. Lang, R., Koegel, L.K., Ashbaugh, K., Regester, A., Ence, W., Smith, W.: Physical exercise and individuals with autism spectrum disorders: a systematic review. Res. Autism Spectr. Disord. (2010). doi: 10.1016/j.rasd.2010.01.006
  59. Leech, N.L., Onwuegbuzie, A.J.: A typology of mixed methods research designs. Qual. Quant. (2009). doi: 10.1007/s11135-007-9105-3
  60. Major C.H., Savin-Baden M.: An Introduction to Qualitative Research Synthesis: Managing the Information Explosion in Social Science Research. Routledge, London (2010)Google Scholar
  61. Matson, J.L., Neal, D.: Psychotropic medication use for challenging behaviors in persons with intellectual disabilities: an overview. Res. Dev. Disabil. (2009). doi: 10.1016/j.ridd.2008.08.007
  62. Maxwell J.: Qualitative Research Design: An Interactive Approach. 2nd edn. Sage, Thousand Oaks (2005)Google Scholar
  63. Mays N., Roberts E., Popay J.: Synthesising research evidence. In: Fulop, N., Allen, P., Aileen, C., Black, B. (eds) Studying the Organisation and Delivery of Health Services: Research Methods, pp. 188–219. Routledge, London (2001)Google Scholar
  64. Mays, N., Pope, C., Popay, J.: Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J. Health Serv. Res. Policy (2005). doi: 10.1258/1355819054308576
  65. McGillivray, J.A., McCabe, M.P.: Emerging trends in the use of drugs to manage the challenging behaviour of people with intellectual disability. J. Appl. Res. Intellect. Disabil. (2006). doi: 10.1111/j.1468-3148.2005.00251.x
  66. McKenna H., Cutliffe J., McKenna P.: Evidence-based practice: demolishing some myths. Nurs. Stand. 14, 39–42 (1999)Google Scholar
  67. Mertens D.M.: Research and Evaluation in Education and Psychology: Integrating Diversity with Quantitative, Qualitative, and Mixed Methods. Sage, Thousand Oaks (2005)Google Scholar
  68. Mertens, D.M.: Transformative paradigm: Mixed methods and social justice. J. Mix. Methods Res. (2007). doi: 10.1177/1558689807302811
  69. Mertens, D.M.: Divergence and mixed methods. J. Mix. Methods Res. (2010). doi: 10.1177/1558689809358406
  70. Mitchell G.J.: Evidence-based practice: critique and alternative view. Nurs. Sci. Q. 12, 30–35 (1999)CrossRefGoogle Scholar
  71. Morgan, D.L.: Practical strategies for combining qualitative and quantitative methods: applications for health research. Qual. Health Res. (1998). doi: 10.1177/104973239800800307
  72. Morgan, D.L.: Paradigms lost and pragmatism regained: methodological implications of combining qualitative and quantitative methods. J. Mix. Methods Res. (2007). doi: 10.1177/2345678906292462
  73. Morse J.M.: Approaches to qualitative-quantitative methodological triangulation. Nurs. Res. 40, 120–123 (1991)CrossRefGoogle Scholar
  74. Morse J.M., Niehaus L.: Mixed Method Design: Principles and Procedures. Left Coast Press Inc., Walnut Creek (2009)Google Scholar
  75. Newman I., Benz C.R.: Qualitative-Quantitative Research Methodology: Exploring the Interactive Continuum. Southern University of Illinois Press, Carbondale (1998)Google Scholar
  76. Niaz, M.: A rationale for mixed methods (integrative) research programmes in education. J. Philos. Educ. (2008). doi: 10.1111/j.1467-9752.2008.00625.x
  77. Niglas, K.: Introducing the qualitative-quantitative continuum: an alternative view of teaching research methods courses. Paper and diagrams presented at the 2006 European Conference on Educational Research, Geneva, Switzerland (2006)Google Scholar
  78. O’Cathain, A., Murphy, E., Nicholl, J.: Integration and publications as indicators of “yield” from mixed methods studies. J. Mix. Methods Res. (2007). doi: 10.1177/1558689806299094
  79. Oliver S., Harden A., Rees R., Shepherd J., Brunton G., Garcia J. et al.: An emerging framework for integrating different types of evidence in systematic reviews for public health. Evaluation 11, 428–466 (2005)CrossRefGoogle Scholar
  80. Onwuegbuzie A.J., Johnson R.B.: The validity issue in mixed research. Res. Sch. 13, 48–63 (2006)Google Scholar
  81. Onwuegbuzie, A.J., Leech, N.L.: Taking the “Q” out of research: teaching research methodology courses without the divide between quantitative and qualitative paradigms. Qual. Quant. (2005). doi: 10.1007/s11135-004-1670-0
  82. Oxman A.D., Guyatt G.H.: Guidelines for reading literature reviews. Can. Med. Assoc. J. 138, 697–703 (1988)Google Scholar
  83. Paterson B.L., Thorne B.L., Canam C., Jillings C.: Meta-Study of Qualitative Health Research. Sage, Thousand Oaks (2001)Google Scholar
  84. Pawson R.: Method mix, technical hex, and theory fix. In: Bergman, M.M. (eds) Advances in Mixed Methods Research: Theories and Applications, pp. 120–137. Sage, Thousand Oaks (2008)Google Scholar
  85. Pawson, R., Greenhalgh, T., Harvey, G., Walshe, K.: Realist review—a new method of systematic review designed for complex policy interventions. J. Health Serv. Res. Policy (2005). doi: 10.1258/1355819054308530
  86. Plano Clark V.L., Creswell J.W., O’Neil Green D., Shope R.J.: Mixing quantitative and qualitative approaches: an introduction to emergent mixed methods research. In: Hesse-Biber, S.N., Leavy, P. (eds) Handbook of Emergent Methods, pp. 363–387. Guilford Press, New York (2008)Google Scholar
  87. Pluye, P., Gagnon, M.P., Griffiths, F., Johnson-Lafleur, J.: A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative, and mixed methods primary-level studies in mixed studies reviews. Int. J. Nurs. Stud. (2009). doi: 10.1016/j.ijnurstu.2009.01.009
  88. Rice, M.J.: Evidence-based practice in psychiatric and mental health nursing: qualitative meta-synthesis. J. Am. Psychiatr. Nurses Assoc. (2008). doi: 10.1177/1078390308326661
  89. Risjord, M.W., Dunbar, S.B., Moloney, M.F.: A new foundation for methodological triangulation. J. Nurs. Scholarsh. (2002). doi: 10.1111/j.1547-5069.2002.00269.x
  90. Robins, C.S., Ware, N.C., dosReis, S., Willging, C.E., Chung, J.Y., Lewis-Fernandez, R.: Dialogues on mixed-methods and mental health services research: anticipating challenges, building solutions. Psychiatr. Serv. (2008). doi: 10.1176/appi.ps.59.7.727
  91. Sandelowski, M., Docherty, S., Emden, C.: Qualitative metasynthesis: issues and techniques. Res. Nurs. Health (1997). doi: 10.1002/(SICI)1098-240X(199708)20:4<365::AID-NUR9>3.0.CO;2-E
  92. Sandelowski M., Voils C.I., Barroso J.: Defining and designing mixed research synthesis studies. Res. Sch. 13, 29–40 (2006)Google Scholar
  93. Sandelowski, M., Voils, C.I., Knaff, G.: On quantitizing. J. Mix. Methods Res. (2009). doi: 10.1177/1558689809334210
  94. Shogren, K.A., Faggella-Luby, M.N., Bae, S.J., Wehmeyer, M.L.: The effect of choice-making as an intervention for problem behavior: a meta-analysis. J. Posit. Behav. Interv. (2004). doi: 10.1177/10983007040060040401
  95. Sohanpal, S.K., Deb, S., Thomas, C., Soni, R., Lenotre, L., Unwin, G.: The effectiveness of antidepressant medication in the management of behaviour problems in adults with intellectual disabilities: a systematic review. J. Intellect. Disabil. Res. (2007). doi: 10.1111/j.1365-2788.2006.00935.x
  96. Suri, H., Clarke, D.: Advancements in research synthesis methods: from a methodologically inclusive perspective. Rev. Educ. Res. (2009). doi: 10.3102/0034654308326349
  97. Tashakkori, A., Creswell, J.W.: The new era of mixed methods. J. Mix. Methods Res. (2007). doi: 10.1177/2345678906293042
  98. Tashakkori A., Teddlie C.: Mixed Methodology: Combining Qualitative and Quantitative Approaches. Sage, Thousand Oaks (1998)Google Scholar
  99. Tashakkori A., Teddlie C.: The past and future of mixed methods research: from data triangulation to mixed model designs. In: Tashakkori, A., Teddlie, C. (eds) Handbook of Mixed Methods in Social and Behavioral Research, pp. 671–701. Sage, Thousand Oaks (2003a)Google Scholar
  100. Tashakkori A., Teddlie C. (eds.): Handbook of Mixed Methods in Social and Behavioural Research. Sage, Thousand Oaks (2003b)Google Scholar
  101. Teddlie C., Tashakkori A.: A general typology of research designs featuring mixed methods. Res. Sch. 13, 12–28 (2006)Google Scholar
  102. Thomas, J., Harden, A., Oakley, A., Oliver, S., Sutcliffe, K., Rees, R., et al.: Integrating qualitative research with trials in systematic reviews. Br. Med. J. (2004). doi: 10.1136/bmj.328.7446.1010
  103. Voils, C.I., Sandelowski, M., Barroso, J., Hasselblad, V.: Making sense of qualitative and quantitative findings in mixed research synthesis studies. Field Methods (2008). doi: 10.1177/1525822X07307463
  104. Walsh, D., Downe, S.: Meta-synthesis method for qualitative research: a literature review. J. Adv. Nurs. (2005). doi: 10.1111/j.1365-2648.2005.03380.x
  105. Whittemore, R., Knafl, K.: The integrative review: updated methodology. J. Adv. Nurs. (2005). doi: 10.1111/j.1365-2648.2005.03621.x
  106. Zimmer, L.: Qualitative meta-synthesis: a question of dialoguing with texts. J. Adv. Nurs. (2006). doi: 10.1111/j.1365-2648.2006.03721.x

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  1. 1.Research Foundation Flanders (FWO)BrusselsBelgium
  2. 2.Methodology of Educational Sciences Research Group, Faculty of Psychology and Educational SciencesKatholieke Universiteit LeuvenLeuvenBelgium
  3. 3.Parenting and Special Education Research Group, Faculty of Psychology and Educational SciencesKatholieke Universiteit LeuvenLeuvenBelgium

Personalised recommendations