Evaluating Complex Interventions

  • Apostolos TsiachristasEmail author
  • Maureen P. M. H. Rutten-van Mölken


There is an increasing interest in evaluating complex interventions. This is because epidemiological changes increasingly call for composite interventions to address patients’ needs and preferences. It is also because such interventions increasingly require explicit reimbursement decisions. That was not the case in the past, when these interventions often entered the benefit package automatically, once they were considered standard medical practice. Nowadays, payers as well as care providers are intrigued to know not just if a health care intervention works but also when, for whom, how, and under which circumstances. In addition, there is broad recognition in the research community that evaluating complex interventions is a challenging task that requires adequate methods and scientific approaches. This chapter describes what an complex intervention is, discusses the rational for and challenges in the evaluation of complex interventions, and provides an overview of evaluation steps and methods.


  1. Altpeter, M., Gwyther, L. P., Kennedy, S. R., Patterson, T. R., & Derence, K. (2015). From evidence to practice: Using the RE-AIM framework to adapt the REACHII caregiver intervention to the community. Dementia (London), 14, 104–113.CrossRefGoogle Scholar
  2. Anderson, L. M., Petticrew, M., Chandler, J., Grimshaw, J., Tugwell, P., O’Neill, J., Welch, V., Squires, J., Churchill, R., & Shemilt, I. (2013). Introducing a series of methodological articles on considering complexity in systematic reviews of interventions. Journal of Clinical Epidemiology, 66, 1205–1208.CrossRefGoogle Scholar
  3. Boland, M. R., Kruis, A. L., Huygens, S. A., Tsiachristas, A., Assendelft, W. J., Gussekloo, J., Blom, C. M., Chavannes, N. H., & Rutten-Van Molken, M. P. (2015). Exploring the variation in implementation of a COPD disease management programme and its impact on health outcomes: A post hoc analysis of the RECODE cluster randomised trial. NPJ Primary Care Respiratory Medicine, 25, 15071.CrossRefGoogle Scholar
  4. Bonell, C., Fletcher, A., Morton, M., Lorenc, T., & Moore, L. (2012). Realist randomised controlled trials: A new approach to evaluating complex public health interventions. Social Science & Medicine, 75, 2299–2306.CrossRefGoogle Scholar
  5. Bradley, F., Wiles, R., Kinmonth, A. L., Mant, D., & Gantley, M. (1999). Development and evaluation of complex interventions in health services research: Case study of the Southampton heart integrated care project (SHIP). The SHIP Collaborative Group. BMJ, 318, 711–715.CrossRefGoogle Scholar
  6. Brewin, C. R., & Bradley, C. (1989). Patient preferences and randomised clinical trials. BMJ, 299, 313–315.CrossRefGoogle Scholar
  7. Brown, C. A., & Lilford, R. J. (2006). The stepped wedge trial design: A systematic review. BMC Medical Research Methodology, 6, 54.CrossRefGoogle Scholar
  8. Campbell, M., Fitzpatrick, R., Haines, A., Kinmonth, A. L., Sandercock, P., Spiegelhalter, D., & Tyrer, P. (2000). Framework for design and evaluation of complex interventions to improve health. BMJ, 321, 694–696.CrossRefGoogle Scholar
  9. Campbell, M. K., Piaggio, G., Elbourne, D. R., Altman, D. G., & Group, C. (2012). Consort 2010 statement: Extension to cluster randomised trials. BMJ, 345, e5661.CrossRefGoogle Scholar
  10. Collins, L. M., Murphy, S. A., Nair, V. N., & Strecher, V. J. (2005). A strategy for optimizing and evaluating behavioral interventions. Annals of Behavioral Medicine, 30, 65–73.CrossRefGoogle Scholar
  11. Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., Petticrew, M., & Medical Research Council Guidance. (2008). Developing and evaluating complex interventions: The new Medical Research Council guidance. BMJ, 337, a1655.CrossRefGoogle Scholar
  12. Craig, P., Cooper, C., Gunnell, D., Haw, S., Lawson, K., Macintyre, S., Ogilvie, D., Petticrew, M., Reeves, B., Sutton, M., & Thompson, S. (2012). Using natural experiments to evaluate population health interventions: New medical research council guidance. Journal of Epidemiology and Community Health, 66, 1182–1186.CrossRefGoogle Scholar
  13. Datta, J., & Petticrew, M. (2013). Challenges to evaluating complex interventions: A content analysis of published papers. BMC Public Health, 13, 568.CrossRefGoogle Scholar
  14. Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89, 1322–1327.CrossRefGoogle Scholar
  15. Glasgow, R. E., Klesges, L. M., Dzewaltowski, D. A., Estabrooks, P. A., & Vogt, T. M. (2006a). Evaluating the impact of health promotion programs: Using the RE-AIM framework to form summary measures for decision making involving complex issues. Health Education Research, 21, 688–694.CrossRefGoogle Scholar
  16. Glasgow, R. E., Nelson, C. C., Strycker, L. A., & King, D. K. (2006b). Using RE-AIM metrics to evaluate diabetes self-management support interventions. American Journal of Preventive Medicine, 30, 67–73.CrossRefGoogle Scholar
  17. Guyatt, G. H., Keller, J. L., Jaeschke, R., Rosenbloom, D., Adachi, J. D., & Newhouse, M. T. (1990). The n-of-1 randomized controlled trial: Clinical usefulness. Our three-year experience. Annals of Internal Medicine, 112, 293–299.CrossRefGoogle Scholar
  18. Hawe, P., Shiell, A., & Riley, T. (2004). Complex interventions: How “out of control” can a randomised controlled trial be? BMJ, 328, 1561–1563.CrossRefGoogle Scholar
  19. Hofman, C. S., Makai, P., Boter, H., Buurman, B. M., De Craen, A. J., Olde Rikkert, M. G., Donders, R. A., & Melis, R. J. (2014). Establishing a composite endpoint for measuring the effectiveness of geriatric interventions based on older persons’ and informal caregivers’ preference weights: A vignette study. BMC Geriatrics, 14, 51.CrossRefGoogle Scholar
  20. Knai, C., Nolte, E., Brunn, M., Elissen, A., Conklin, A., Pedersen, J. P., Brereton, L., Erler, A., Frolich, A., Flamm, M., Fullerton, B., Jacobsen, R., Krohn, R., Saz-Parkinson, Z., Vrijhoef, B., Chevreul, K., Durand-Zaleski, I., Farsi, F., Sarria-Santamera, A., & Soennichsen, A. (2013). Reported barriers to evaluation in chronic care: Experiences in six European countries. Health Policy (Amsterdam), 110, 220–228.CrossRefGoogle Scholar
  21. Kodner, D. L., & Spreeuwenberg, C. (2002). Integrated care: Meaning, logic, applications, and implications – a discussion paper. International Journal of Integrated Care, 2, e12.CrossRefGoogle Scholar
  22. Lamb, S. E., Becker, C., Gillespie, L. D., Smith, J. L., Finnegan, S., Potter, R., Pfeiffer, K., & Taxonomy, I. (2011). Reporting of complex interventions in clinical trials: Development of a taxonomy to classify and describe fall-prevention interventions. Trials, 12, 125.CrossRefGoogle Scholar
  23. Lamont, T., Barber, N., Pury, J., Fulop, N., Garfield-Birkbeck, S., Lilford, R., Mear, L., Raine, R., & Fitzpatrick, R. (2016). New approaches to evaluating complex health and care systems. BMJ, 352, i154.CrossRefGoogle Scholar
  24. Lancaster, G. A., Campbell, M. J., Eldridge, S., Farrin, A., Marchant, M., Muller, S., Perera, R., Peters, T. J., Prevost, A. T., & Rait, G. (2010). Trials in primary care: Statistical issues in the design, conduct and evaluation of complex interventions. Statistical Methods in Medical Research, 19, 349–377.CrossRefGoogle Scholar
  25. Marchal, B., Westhorp, G., Wong, G., Van Belle, S., Greenhalgh, T., Kegels, G., & Pawson, R. (2013). Realist RCTs of complex interventions – an oxymoron. Social Science & Medicine, 94, 124–128.CrossRefGoogle Scholar
  26. May, C. (2006). A rational model for assessing and evaluating complex interventions in health care. BMC Health Services Research, 6, 86.CrossRefGoogle Scholar
  27. Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010). CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. BMJ, 340, c869.CrossRefGoogle Scholar
  28. Mohler, R., Kopke, S., & Meyer, G. (2015). Criteria for reporting the development and evaluation of complex interventions in healthcare: Revised guideline (CReDECI 2). Trials, 16, 204.CrossRefGoogle Scholar
  29. Montgomery, P., Grant, S., Hopewell, S., Macdonald, G., Moher, D., Michie, S., & Mayo-Wilson, E. (2013a). Protocol for CONSORT-SPI: An extension for social and psychological interventions. Implementation Science, 8, 99.CrossRefGoogle Scholar
  30. Montgomery, P., Underhill, K., Gardner, F., Operario, D., & Mayo-Wilson, E. (2013b). The oxford implementation index: A new tool for incorporating implementation data into systematic reviews and meta-analyses. Journal of Clinical Epidemiology, 66, 874–882.CrossRefGoogle Scholar
  31. Moore, G., Audrey, S., Barker, M., Bond, L., Bonell, C., Cooper, C., Hardeman, W., Moore, L., O’Cathain, A., Tinati, T., Wight, D., & Baird, J. (2014). Process evaluation in complex public health intervention studies: The need for guidance. Journal of Epidemiology and Community Health, 68, 101–102.CrossRefGoogle Scholar
  32. Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O’Cathain, A., Tinati, T., Wight, D., & Baird, J. (2015). Process evaluation of complex interventions: Medical research council guidance. BMJ, 350, h1258.CrossRefGoogle Scholar
  33. Nolte, E., & Mckee, M. (2008). In E. Nolte & M. McKee (Eds.), Caring for people with chronic conditions: A health system perspective (p. 259). Maidenhead: Open University Press/McGraw-Hill, (European observatory on health systems and policies series), isbn: 978 0 335 23370 0.Google Scholar
  34. Oakley, A., Strange, V., Bonell, C., Allen, E., Stephenson, J., & Team, R. S. (2006). Process evaluation in randomised controlled trials of complex interventions. BMJ, 332, 413–416.CrossRefGoogle Scholar
  35. Ogilvie, D., Cummins, S., Petticrew, M., White, M., Jones, A., & Wheeler, K. (2011). Assessing the evaluability of complex public health interventions: Five questions for researchers, funders, and policymakers. The Milbank Quarterly, 89, 206–225.CrossRefGoogle Scholar
  36. Petticrew, M. (2011). When are complex interventions ‘complex’? When are simple interventions ‘simple’? European Journal of Public Health, 21, 397–398.CrossRefGoogle Scholar
  37. Rickles, D. (2009). Causality in complex interventions. Medicine, Health Care, and Philosophy, 12, 77–790.CrossRefGoogle Scholar
  38. Rychetnik, L., Frommer, M., Hawe, P., & Shiell, A. (2002). Criteria for evaluating evidence on public health interventions. Journal of Epidemiology and Community Health, 56, 119–127.CrossRefGoogle Scholar
  39. Shepperd, S., Lewin, S., Straus, S., Clarke, M., Eccles, M. P., Fitzpatrick, R., Wong, G., & Sheikh, A. (2009). Can we systematically review studies that evaluate complex interventions? PLoS Medicine, 6, e1000086.CrossRefGoogle Scholar
  40. Shiell, A., Hawe, P., & Gold, L. (2008). Complex interventions or complex systems? Implications for health economic evaluation. BMJ, 336, 1281–1283.CrossRefGoogle Scholar
  41. Steckler, A. B., Linnan, L., & Israel, B. A. (2002). Process evaluation for public health interventions and research. San Francisco, CA: Jossey-Bass.Google Scholar
  42. Stuart, E. A., Huskamp, H. A., Duckworth, K., Simmons, J., Song, Z., Chernew, M., & Barry, C. L. (2014). Using propensity scores in difference-in-differences models to estimate the effects of a policy change. Health Services & Outcomes Research Methodology, 14, 166–182.CrossRefGoogle Scholar
  43. Taylor, B. J., Francis, K., & Hegney, D. (2013). Qualitative research in the health sciences, Methodologies methods and processes. London: Routledge.Google Scholar
  44. Torgerson, D. J., & Sibbald, B. (1998). Understanding controlled trials. What is a patient preference trial? BMJ, 316, 360.CrossRefGoogle Scholar
  45. von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gotzsche, P. C., Vandenbroucke, J. P., & Initiative, S. (2007). The strengthening the reporting of observational studies in epidemiology (STROBE) statement: Guidelines for reporting observational studies. Lancet, 370, 1453–1457.CrossRefGoogle Scholar
  46. WHO. (2015). WHO global strategy on people-centred and integrated health services: Interim report. Geneva: World Health Organization.Google Scholar
  47. Zelen, M. (1979). A new design for randomized clinical trials. The New England Journal of Medicine, 300, 1242–1245.CrossRefGoogle Scholar
  48. Zwarenstein, M., Treweek, S., Gagnier, J. J., Altman, D. G., Tunis, S., Haynes, B., Oxman, A. D., Moher, D., Group, C., & Pragmatic Trials In Healthcare, Group. (2008). Improving the reporting of pragmatic trials: An extension of the CONSORT statement. BMJ, 337, a2390.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Apostolos Tsiachristas
    • 1
    Email author
  • Maureen P. M. H. Rutten-van Mölken
    • 2
  1. 1.Health Economics Research Centre, Nuffield Department of Population HealthUniversity of OxfordOxfordUK
  2. 2.Institute of Medical Technology Assessment, Institute of Health Care Policy and ManagementErasmus University RotterdamRotterdamThe Netherlands

Personalised recommendations