Skip to main content
SpringerLink
Log in

Guidance for deciding upon use of primary mixed methods studies in research synthesis: lessons learned in childhood trauma

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

When reviewing literature, mixed methods studies (MMS) are increasingly retrieved, yet it is unclear how they should be dealt with in a research synthesis. In this article we examine the inclusion of primary MMS in research synthesis, based on experiences with a meta-analysis (MA) and a qualitative evidence synthesis (QES) in childhood trauma. The aim is to offer guidance for reviewers in deciding upon the use of MMS. This review article examines (1) the qualitative component, (2) the quantitative component, as well as (3) the third component of combined yield for use in a MA, a QES or a mixed studies review. A systematic search for MMS in the field of childhood trauma from January 1980 to October 2011 resulted in twelve MMS. Eight qualitative components, six quantitative components and one combined yield could have been included in a MA or QES. Exclusion of qualitative components was due to insufficient quality. Quantitative components were excluded because sample sizes were too small, different outcome measures than ours were used or no adequate statistics were provided. Yield could not be included because it was absent, outside the scope or otherwise unspecified. Finally, we offer flow charts with clear steps to assist researchers in deciding upon the use of components of MMS. Our study demonstrates that MMS can cover new areas and therefore cannot be neglected in a research synthesis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Alise, M.A., Teddlie, C.: A continuation of the paradigm wars? Prevalence rates of methodological approaches across the social/behavioral sciences. J. Mix. Methods Res. 4(2), 103–126 (2010)

    Article  Google Scholar 

  • Alisic, E., Jongmans, M.J., van Wesel, F., Kleber, R.J.: Building child trauma theory from longitudinal studies: a meta-analysis. Clin. Psychol. Rev. 31, 736–747 (2011)

    Google Scholar 

  • Boeije, H.R., Slagt, M., van Wesel, F.: The contribution of mixed methods research to the field of childhood trauma: a narrative review focused on data integration. J. Mix. Methods Res. (in preparation)

  • Bryman, A.: Barriers to integrating quantitative and qualitative research. J. Mix. Methods Res. 1(1), 8–22 (2007)

    Article  Google Scholar 

  • Bryman, A., Becker, S., Sempik, J.: Quality criteria for quantitative, qualitative and mixed methods research: a view from social policy. Int. J. Soc. Res. Method. 11(4), 261–276 (2008)

    Article  Google Scholar 

  • Candy, B., King, M., Jones, L., Oliver, S.: Using qualitative synthesis to explore heterogeneity of complex interventions. BMC Med. Res. Methodol. 11, 124–132 (2011)

    Article  Google Scholar 

  • Caracelli, V.J., Greene, J.C.: Crafting mixed-method evaluation designs. In: Greene, J.C., Caracelli, V.J. (eds.) Advances in Mixed-Method Evaluation: The Challenges and Benefits of Integrating Diverse Paradigms, pp. 19–32. Jossey-Bass, San Francisco (1997)

    Google Scholar 

  • Charles, N., Butera-Prinzi, F., Perlesz, A.: Families living with acquired brain injury: a multiple family group experience. NeuroRehabilitation 22, 61–76 (2007)

    Google Scholar 

  • Creswell, J.W., Plano Clark, V.L.: Designing and Conducting Mixed Methods Research. Sage, Thousand Oaks (2007)

    Google Scholar 

  • Dixon-Woods, M., Fitzpatrick, R., Roberts, K.: Including qualitative research in systematic reviews: opportunities and problems. J. Eval. Clin. Pract. 7(2), 125–133 (2001)

    Google Scholar 

  • Feeney, T.J., Ylvisaker, M.: Context-sensitive behavioral supports for young children with TBI. Short-term effects and long-term outcome. J. Head. Trauma. Rehabil. 18(1), 33–51 (2003)

    Article  Google Scholar 

  • Finfgeld, D.L.: Metasynthesis: the state of the art—so far. Qual. Health. Res. 13(7), 893–904 (2003)

    Article  Google Scholar 

  • Gaskell, S.L.: The challenge of evaluating rehabilitative activity holidays for burn-injured children: qualitative and quantitative outcome data from a Burns Camp over a five-year period. Dev. Neurohabil. 10(2), 149–160 (2007)

    Article  Google Scholar 

  • Green, J.C.: Is mixed methods social inquiry a distinctive methodology? J. Mix. Methods Res. 2(7), 7–22 (2008)

    Article  Google Scholar 

  • Haight, W., Black, J., Sheridan, K.: A mental health intervention for rural, foster children from methamphetamine-involved families: experimental assessment with qualitative elaboration. Child Youth Serv. Rev. 32, 1446–1457 (2010)

    Article  Google Scholar 

  • Harden, A., Brunton, G., Fletcher, A., Oakley, A.: Teenage pregnancy and social disadvantage: systematic review integrating controlled trials and qualitative studies. Br. Med. J. 339, b4254 (2009)

    Article  Google Scholar 

  • Harden, A., Thomas, J.: Methodological issues in combining diverse study types in systematic reviews. Int. J. Soc. Res. Method. 8(3), 257–271 (2005)

    Google Scholar 

  • Heyvaert, M., Maes, B., Onghena, P.: Mixed methods research synthesis: definition, framework, and potential. Qual. Quant. (2011). doi:10.1007/s11135-011-9538-6

  • Jones, L., Kafetsios, K.: Assessing adolescent mental health in war-effected societies: the significance of symptoms. Child Abuse Negl. 26, 1059–1080 (2002)

    Article  Google Scholar 

  • Jones, L., Kafetsios, K.: Exposure to political well-being in Bosnian adolescents: a mixed method approach. Clin. Child Psychol. Psychiatr. 10(2), 157–176 (2005)

    Article  Google Scholar 

  • Leech, N.L., Onwuegbuzie, A.J.: A typology of mixed methods research designs. Qual. Quant. 43, 265–275 (2009)

    Article  Google Scholar 

  • Lewin, S., Glenton, C., Oxman, A.D.: Use of qualitative methods alongside randomized controlled trials of complex healthcare interventions: methodological study. Br. Med. J. 339, b3496 (2009)

    Article  Google Scholar 

  • London, Bocknek E., Sanderson, J.: Ambiguous loss and posttraumatic stress in school-age children of prisoners. J. Child Fam. Stud. 18, 323–333 (2008)

    Google Scholar 

  • Madsen, K., Poulsen, H.S.: Needs for everyday life support for brain tumor patients’ relatives: systematic literature review. Eur. J. Cancer Care 20, 33–43 (2011)

    Article  Google Scholar 

  • Mays, N., Pope, C., Popay, J.: Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J. Health Serv. Res. Policy 10(1), 6–20 (2005)

    Article  Google Scholar 

  • Miller, K.E.: The effects of state terrorism and exile on indigenous Guatemalan refugee children: a mental health assessment and an analysis of children’s narratives. Child. Dev. 67, 89–106 (1996)

    Article  Google Scholar 

  • Miller, K.E., Fernando, G.A., Berger, D.E.: Daily stressors in the lives of Sri Lankan youth: a mixed methods approach to assessment in a context of war and natural disaster. Intervention 9(3), 187–203 (2009)

    Article  Google Scholar 

  • Molina-Azorín, J.F.: The use and added value of mixed methods in management research. J. Mix. Methods Res. 5(1), 7–24 (2011)

    Article  Google Scholar 

  • Nicolson, D.J.: Studies that mix methods and don’t identify as “Mixed methods research”: a nuisance for systematic reviews and a challenge for mixed methods research. Paper presented at: The 6th International Mixed Methods Conference, Baltimore (2010)

  • Nyamukapa, C.A., Gregson, S., Wambe, M., Mushore, P., Lopman, B., Mupambireyi, Z., Nhongo, K., Jukes, M.C.H.: Causes and consequences of psychological distress among orphans in eastern Zimbabwe. AIDS Care 22(8), 988–996 (2011)

    Article  Google Scholar 

  • O’Cathain, A., Murphy, E., Nicholl, J.: Integration and publications as indicators of “yield” from mixed methods studies. J. Mix. Methods Res. 1, 147–163 (2007)

    Article  Google Scholar 

  • O’Cathain, A., Murphy, E., Nicholl, J.: The quality of mixed methods studies in health services research. J. Health Serv. Res. Policy 13(2), 92–98 (2008)

    Google Scholar 

  • Ostler, T., Haight, W., Black, J., Choi, G., Kingery, L., Sheridan, K.: Case series: mental health needs and perspectives of rural children reared by parents who abuse methamphetamine. J. Am. Acad. Child Psychiatr. 46(4), 500–507 (2007)

    Article  Google Scholar 

  • Pace, R., Pluye, P., Bartlett, G., Macaulay, A.C., Salsberg, J., Jagosh, J., Seller, R.: Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review. Int. J. Nurs. Stud. 49, 47–53 (2012)

    Google Scholar 

  • Paterson, B.L.: It looks great but how do I know if it fits? An introduction to meta-synthesis research, choosing the right approach. In: Hannes, K., Lockwood, C. (eds.) Synthesizing Qualitative Research, pp. 1–20. Wiley-Blackwell, Oxford (2012)

    Chapter  Google Scholar 

  • Petticrew, M., Roberts, H.: Systematic Reviews in the Social Sciences. A Practical Guide. Blackwell, Oxford (2006)

    Book  Google Scholar 

  • Pluye, P., Gagnon, M., Griffiths, F., Johnson-Lafleur, J.: A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews. Int. J. Nurs. Stud. 46, 529–546 (2009)

    Article  Google Scholar 

  • Pope, C., Mays, N., Popay, J.: Synthesizing Qualitative and Quantitative Health Evidence. A Guide to Methods. Open University, Berkshire (2007)

    Google Scholar 

  • Roberts, Y.H., Mitchell, M.J., Witman, M.M.N., Taffaro, C.: Mental health symptoms in youth affected by hurricane Katrina. Prof. Psychol. Res. Pract. 41(1), 10–18 (2010)

    Article  Google Scholar 

  • Sale, J.E.M., Brazil, K.: A strategy to identify critical appraisal criteria for primary mixed-method studies. Qual. Quant. 38, 351–365 (2004)

    Article  Google Scholar 

  • Salter, K., Hellings, C., Floey, N., Teasell, R.: The experience of living with stroke: a qualitative meta-synthesis. J. Rehabil. Med. 40, 595–602 (2008)

    Article  Google Scholar 

  • Sandelowski, M., Voils, C.I., Barroso, J.: Defining and designing mixed research synthesis studies. Res. Schools 13(1), 29–40 (2006)

    Google Scholar 

  • Small, M.L.: How to conduct a mixed methods study: recent trends in a rapidly growing literature. Annu. Rev. Sociol. 37, 57–86 (2011)

    Article  Google Scholar 

  • Sommer, Harrits G.: More than method? A discussion of paradigm differences within mixed methods research. J. Mix. Methods Res. 5(2), 150–166 (2011)

    Article  Google Scholar 

  • Spillane, J.P., Stitziel, Pareja A.: Mixing methods in randomized controlled trials (RCTs): validation, contextualization, triangulation, and control. Educ. Assess. Eval. Account. 22, 5–28 (2010)

    Article  Google Scholar 

  • Talseth, A., Gilje, F.L.: Nurses’ responses to suicide and suicidal patients: a critical interpretive synthesis. J. Clin. Nurs. 20, 1651–1667 (2011)

    Article  Google Scholar 

  • van Wesel, F., Boeije, H.R., Alisic, E., Drost S.: I’ll be working my way back: a qualitative synthesis on the trauma experiences of children. Psychological trauma: theory, research, practice, and policy. 4(5), 516–526 (2012)

  • Vervoort, S., Borleffs, J.C.C., Hoepelman, A.I.M., Grypdonck, M.H.F.: Adherence in antiretroviral therapy: a review of qualitative studies. AIDS 21(3), 271–281 (2007)

    Article  Google Scholar 

  • Voils, C.I., Sandelowski, M., Barroso, J., Hasselblad, V.: Making sense of qualitative and quantitative findings in mixed research synthesis studies. Field Method. 20(1), 3–25 (2008)

    Article  Google Scholar 

  • Wiggins, B.J.: Confronting the dilemma of mixed methods. J. Theor. Philos. Psychol. 31(1), 44–60 (2011)

    Article  Google Scholar 

Download references

Acknowledgments

The authors wish to thank Eva Alisic, Ph.D., Monash Injury Research Institute (MIRI, Australia) for her co-operation in the meta-analysis and qualitative evidence synthesis, and Karin Hannes, Ph.D., and Mieke Heyvaert, Ph.D., Faculty of Psychology and Educational Sciences, Katholieke Universiteit Leuven, Belgium, for their generous comments on our paper.

Author information

Authors and Affiliations

  1. Department of Methodology and Statistics, Faculty of Social and Behavioral Sciences, Utrecht University, P.O. Box 80.140, 3508 TC, Utrecht, The Netherlands

    H. Boeije

  2. Department of Educational Neuroscience, Faculty of Psychology and Education, VU University Amsterdam, Amsterdam, The Netherlands

    F. van Wesel

  3. Department of Developmental Psychology, Faculty of Social and Behavioral Sciences, Utrecht University, Utrecht, The Netherlands

    M. Slagt

Authors
  1. H. Boeije
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. F. van Wesel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Slagt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to H. Boeije.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (doc 50 KB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Boeije, H., van Wesel, F. & Slagt, M. Guidance for deciding upon use of primary mixed methods studies in research synthesis: lessons learned in childhood trauma. Qual Quant 48, 1075–1088 (2014). https://doi.org/10.1007/s11135-012-9825-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-012-9825-x

Keywords

Search

Navigation