Skip to main content
SpringerLink
Log in

Systematic Review and Critique of Methods for Economic Evaluation of Digital Mental Health Interventions

  • Systematic Review
  • Published:
Applied Health Economics and Health Policy Aims and scope Submit manuscript

Abstract

Objectives

Investment in digital interventions for mental health conditions is growing rapidly, offering the potential to elevate systems that are currently overstretched. Despite a growing literature on economic evaluation of digital mental health interventions (DMHIs), including several systematic reviews, there is no conclusive evidence regarding their cost-effectiveness. This paper reviews the methodology used to determine their cost-effectiveness and assesses whether this meets the requirements for decision-making. In doing so we consider the challenges specific to the economic evaluation of DMHIs, and identify where consensus and possible further research is warranted.

Methods

A systematic review was conducted to identify all economic evaluations of DMHIs published between 1997 and December 2018. The searches included databases of published and unpublished research, reference lists and citations of all included studies, forward citations on all identified protocols and conference abstracts, and contacting authors researchers in the field. The identified studies were critiqued against a published set of requirements for decision-making in healthcare, identifying methodological challenges and areas where consensus is required.

Results

The review identified 67 papers evaluating DMHIs. The majority of the evaluations were conducted alongside trials, failing to capture all relevant available evidence and comparators, and long-term impact of mental health disorders. The identified interventions are complex and heterogeneous. As a result, there are a number of challenges specific to their evaluation, including estimation of all costs and outcomes, conditional on analysis viewpoint, and identification of relevant comparators. A taxonomy for DMHIs may be required to inform what interventions can reasonably be pooled and compared.

Conclusions

This study represents the first attempt to understand the appropriateness of the methodologies used to evaluate the value for money of DMHIs, helping work towards consensus and methods’ harmonisation on these complex interventions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Fig. 1

Similar content being viewed by others

References

  1. NHS England. Five year forward view. 2014. https://www.england.nhs.uk/wp-content/uploads/2014/10/5yfv-web.pdf. Accessed 7 Aug 2020.

  2. NHS. Personalised health and care 2020. Using data and technology to transform outcomes for patients and citizens. 2014. https://www.gov.uk/government/publications/personalised-health-and-care-2020. Accessed 7 Aug 2020.

  3. Craig P, Dieppe P, Macintyre S, Michie S, Nazzareth I, Petticrew M. Developing and evaluating complex interventions. 2019. https://www.mrc.ac.uk/complexinterventionsguidance. Accessed 7 Aug 2020.

  4. Sculpher MJ, Claxton K, Drummond M, McCabe C. Whither trial-based economic evaluation for health care decision making? Health Econ. 2006;15(7):677–87.

    Article  PubMed  Google Scholar 

  5. Hodgins DC, Peden N. Cognitive-behavioral treatment for impulse control disorders. Braz J Psychiatry. 2008;30:S31–S40.

    Article  PubMed  Google Scholar 

  6. Duarte A, Walker S, Littlewood E, Brabyn S, Hewitt C, Gilbody S, et al. Cost-effectiveness of computerized cognitive-behavioural therapy for the treatment of depression in primary care: findings from the Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy (REEACT) trial. Psychol Med. 2017;47(10):1825–35.

    Article  CAS  PubMed  Google Scholar 

  7. Littlewood E, Duarte A, Hewitt C, Knowles S, Palmer S, Walker S, et al. A randomised controlled trial of computerised cognitive behaviour therapy for the treatment of depression in primary care: the Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy (REEACT) trial. Health Technol Assess. 2015;19(101):viii, xxi-171. https://www.cochranelibrary.com/central/doi/10.1002/central/CN-01168627/full. Accessed 7 Aug 2020.

  8. Jones RB, Atkinson JM, Coia DA, Paterson L, Morton AR, McKenna K, et al. Randomised trial of personalised computer based information for patients with schizophrenia. Br Med J. 2001;322(7290):835–40.

    Article  CAS  Google Scholar 

  9. Naughton F, Cooper S, Foster K, Emery J, Leonardi-Bee J, Sutton S, et al. Large multi-centre pilot randomized controlled trial testing a low-cost, tailored, self-help smoking cessation text message intervention for pregnant smokers (MiQuit). Addiction. 2017;112(7):1238–49.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Wright B, Tindall L, Littlewood E, Allgar V, Abeles P, Trepel D, et al. Computerised cognitive-behavioural therapy for depression in adolescents: feasibility results and 4-month outcomes of a UK randomised controlled trial. BMJ Open. 2017;7(1):e012834.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Kenter RMF, van de Ven PM, Cuijpers P, Koole G, Niamat S, Gerrits RS, et al. Costs and effects of Internet cognitive behavioral treatment blended with face-to-face treatment: results from a naturalistic study. Internet Intery. 2015;2(1):77–83.

    Article  Google Scholar 

  12. Koeser L, Dobbin A, Ross S, McCrone P (2013) Economic evaluation of audio based resilience training for depression in primary care. J Affect Disord. 149(1–3):307–312

    Article  PubMed  Google Scholar 

  13. Lee YY, Barendregt JJ, Stockings EA, Ferrari AJ, Whiteford HA, Patton GA, et al. The population cost-effectiveness of delivering universal and indicated school-based interventions to prevent the onset of major depression among youth in Australia. Epidemiol Psychiatr Sci. 2017;26(5):545–64.

    Article  CAS  PubMed  Google Scholar 

  14. Naversnik K, Mrhar A. Cost-effectiveness of a novel e-health depression service. Telemed J E Health. 2013;19(2):110–6.

    Article  PubMed  Google Scholar 

  15. Lee Y-C, Gao L, Dear BF, Titov N, Mihalopoulos C. The cost-effectiveness of the online MindSpot Clinic for the treatment of depression and anxiety in Australia. J Ment Health Policy Econ. 2017;20(4):155–66.

    PubMed  Google Scholar 

  16. Solomon D, Proudfoot J, Clarke J, Christensen H. e-CBT (myCompass), antidepressant medication, and face-to-face psychological treatment for depression in australia: a cost-effectiveness comparison. J Med Internet Res. 2015;17(11):e255.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Mihalopoulos C, Shih ST, Kiropoulos L, Blashki G, Meadows G, Gunn J. Exploratory economic analyses of two primary care mental health projects: implications for sustainability. Med J Aust. 2005;183:S73–S7676.

    Article  PubMed  Google Scholar 

  18. Kumar S, Jones Bell M, Juusola JL. Mobile and traditional cognitive behavioral therapy programs for generalized anxiety disorder: a cost-effectiveness analysis. PLoS One. 2018;13(1):e0190554.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  19. Kass AE, Balantekin KN, Fitzsimmons-Craft EE, Jacobi C, Wilfley DE, Taylor CB. The economic case for digital interventions for eating disorders among United States college students. Int J Eat Disord. 2017;50(3):250–8.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Guerriero C, Cairns J, Roberts I, Rodgers A, Whittaker R, Free C. The cost-effectiveness of smoking cessation support delivered by mobile phone text messaging: Txt2stop. Eur J Heal Econ. 2013;14(5):789–97.

    Article  Google Scholar 

  21. Wu Q, Parrott S, Godfrey C, Gilbert H, Nazareth I, Leurent B, et al. Cost-effectiveness of computer-tailored smoking cessation advice in primary care: a randomized trial (ESCAPE). Nicotine Tob Res. 2014;16(3):270–8.

    Article  PubMed  Google Scholar 

  22. Burford O, Jiwa M, Carter O, Parsons R, Hendrie D. Internet-based photoaging within Australian pharmacies to promote smoking cessation: randomized controlled trial. J Med Internet Res. 2013;15(3):e64.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Graham AL, Chang Y, Fang Y, Cobb NK, Tinkelman DS, Niaura RS, et al. Cost-effectiveness of internet and telephone treatment for smoking cessation: an economic evaluation of The iQUITT Study. Tob Control. 2013;22(6):e11.

    Article  PubMed  Google Scholar 

  24. McCrone P, Marks IM, Mataix-Cols D, Kenwright M, McDonough M. Computer-aided self-exposure therapy for phobia/panic disorder: a pilot economic evaluation. Cogn Behav Ther. 2009;38(2):91–9.

    Article  PubMed  Google Scholar 

  25. Axelsson E, Andersson E, Ljotsson B, Hedman-Lagerlof E. Cost-effectiveness and long-term follow-up of three forms of minimal contact cognitive behaviour therapy for severe health anxiety: results from a randomised controlled trial. Behav Res Ther. 2018;107:95–105.

    Article  PubMed  Google Scholar 

  26. Bergström J, Andersson G, Ljótsson B, Rück C, Andréewitch S, Karlsson A, et al. Internet-versus group-administered cognitive behaviour therapy for panic disorder in a psychiatric setting: a randomised trial. BMC Psychiatry. 2010;10:54.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Joesch JM, Sherbourne CD, Sullivan G, Stein MB, Craske MG, Roy-Byrne P. Incremental benefits and cost of coordinated anxiety learning and management for anxiety treatment in primary care. Psychol Med. 2012;42(9):1937–48.

    Article  CAS  PubMed  Google Scholar 

  28. Ahern E, Kinsella S, Semkovska M. Clinical efficacy and economic evaluation of online cognitive behavioral therapy for major depressive disorder: a systematic review and meta-analysis. Vol. 18, expert review of pharmacoeconomics and outcomes research. London: Taylor and Francis Ltd; 2018. p. 25–41.

    Google Scholar 

  29. Hedman E, Ljótsson B, Lindefors N. Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert Rev Pharmacoecon Outcomes Res. 2012;12(6):745–64.

    Article  PubMed  Google Scholar 

  30. Kaltenthaler E, Brazier J, De Nigris E, Tumur I, Ferriter M, Beverley C, Parry G, Rooney G, Sutcliffe P. Computerised cognitive behaviour therapy for depression and anxiety update: a systematic review and economic evaluation. Health Technol Assess. 2006;10(33):iii, xi-xiv, 1-168. https://pubmed.ncbi.nlm.nih.gov/16959169/. Accessed 7 Aug 2020.

  31. Naidu VV, Giblin E, Burke KM, Madan I. Delivery of cognitive behavioural therapy to workers: a systematic review. Vol. 66, occupational medicine. Oxford: Oxford University Press; 2016. p. 112–117.

    Google Scholar 

  32. Kolovos S, van Dongen JM, Riper H, Buntrock C, Cuijpers P, Ebert DD, et al. Cost effectiveness of guided Internet-based interventions for depression in comparison with control conditions: an individual–participant data meta-analysis. Vol. 35, depression and anxiety. London: Blackwell Publishing Inc.; 2018. p. 209–219.

    Google Scholar 

  33. Paganini S, Teigelkötter W, Buntrock C, Baumeister H. Economic evaluations of internet- and mobile-based interventions for the treatment and prevention of depression: a systematic review. Vol. 225, J Affect Disord. Elsevier B.V.; 2018. p. 733–55.

  34. Arnberg FK, Linton SJ, Hultcrantz M, Heintz E, Jonsson U. Internet-delivered psychological treatments for mood and anxiety disorders: A systematic review of their efficacy, safety, and cost-effectiveness. Vol. 9, PLoS One. Public Library of Science; 2014.

  35. Richards D, Enrique A, Eilert N, Franklin M, Palacios J, Duffy D, Earley C, Chapman J, Jell G, Sollesse S, Timulak L. A pragmatic randomized waitlist-controlled effectiveness and cost-effectiveness trial of digital interventions for depression and anxiety. npj Digital Medicine. 2020;3 (85). https://www.nature.com/articles/s41746-020-0293-8. Accessed 7 Aug 2020.

Download references

Author information

Authors and Affiliations

  1. Centre for Health Economics, The University of York, Alcuin College, A Block, York, YO10 5DD, UK

    Dina Jankovic, Laura Bojke & Pedro Saramago Goncalves

  2. Centre for Reviews and Dissemination, University of York, York, UK

    David Marshall, Rachel Churchill & Hollie Melton

  3. Department of Health Sciences, University of York, York, UK

    Sally Brabyn

  4. Department of Health Sciences and Hull York Medical School, University of York, York, UK

    Lina Gega

Authors
  1. Dina Jankovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laura Bojke
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Marshall
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pedro Saramago Goncalves
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rachel Churchill
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hollie Melton
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sally Brabyn
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Lina Gega
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

DJ is a Research Fellow in Health Economics. She contributed to the screening of literature, coordinated the data extraction and methodological discussion, and led the paper submission. LB is a Reader in Health Economics. She supervised and contributed to the searches, data extraction and the methodological discussion. DM is a Research Fellow in Evidence Synthesis. He coordinated the literature searches, screened the retrieved records, and advised on and validated the data extraction. PSG is a Research Fellow in Health Economics. He contributed to the screening of literature, data extraction and methodological discussion. HM is a Research Fellow in Evidence Synthesis. She contributed to the screening of records. SB is a Research Fellow in Mental Health. She contributed to the scoping of the review and the literature searches. RC is a Chair in Evidence Synthesis. She supervised the work of colleagues from the Centre for Reviews and Dissemination and from the Cochrane Common Mental Disorders Group (DM and HM). LiG is a Reader in Mental Health and an Honorary Nurse Consultant. She conceived the programme of work that included this review, obtained the funding, and coordinated and supervised the project, contributed to and checked the literature searches, screening of records, data extractions, analyses and write-up.

Corresponding author

Correspondence to Dina Jankovic.

Ethics declarations

Funding

This work was supported by the UK’s National Institute for Health Research (NIHR), Health Technology Assessment (HTA) Programme, Grant number 17/93/06, 2019.

Disclaimer

The views expressed in this paper are those of the authors and not necessarily those of the UK’s National Health Service, the National Institute for Health Research (NIHR), or the Department of Health.

Conflict of interest

Lina Gega is the fund holder of the research grant that supported this work. Dina Jankovic, Laura Bojke, David Marshall, Pedro Saramago Goncalves, Rachel Churchill, Hollie Melton and Sally Brabyn declare that they have no conflicts of interest.

Availability of data and material

Details of extracted studies are provided in the Online Supplementary Material, no additional data or materials were used.

Code availability

N/A.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jankovic, D., Bojke, L., Marshall, D. et al. Systematic Review and Critique of Methods for Economic Evaluation of Digital Mental Health Interventions. Appl Health Econ Health Policy 19, 17–27 (2021). https://doi.org/10.1007/s40258-020-00607-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40258-020-00607-3

Search

Navigation