Skip to main content

Advertisement

Log in

Quality of health economic evaluations in emergency medicine journals: a systematic review

  • Original Research
  • Published:
Canadian Journal of Emergency Medicine Aims and scope Submit manuscript

Abstract

Objective

Health economic evaluations are used in decision-making regarding resource allocation and it is imperative that they are completed with rigor. The primary objectives were to describe the characteristics and assess the quality of economic evaluations published in emergency medicine journals.

Methods

Two reviewers independently searched 19 emergency medicine-specific journals via Medline and Embase from inception until March 3, 2022. Quality assessment was completed using the Quality of Health Economic Studies (QHES) tool, and the primary outcome was the QHES score out of 100. Additionally, we identified factors that may contribute to higher-quality publications.

Results

7260 unique articles yielded 48 economic evaluations that met inclusion criteria. Most studies were cost–utility analyses and of high quality, with a median QHES score of 84 (interquartile range, IQR: 72, 90). Studies based on mathematical models and those primarily designed as an economic evaluation were associated with higher quality scores. The most commonly missed QHES items were: (i) providing and justifying the perspective of the analysis, (ii) providing justification for the primary outcome, and (iii) selecting an outcome that was long enough to allow for relevant events to occur.

Conclusions

The majority of health economic evaluations in the emergency medicine literature are cost–utility analyses and are of high quality. Decision analytic models and studies primarily designed as economic analyses were positively correlated with higher quality. To improve study quality, future EM economic evaluations should justify the choice of the perspective of the analysis and the selection of the primary outcome.

Abstrait

Objectifs

Les évaluations économiques de la santé sont utilisées dans la prise de décisions concernant l’affectation des ressources et il est impératif qu’elles soient réalisées avec rigueur. Les principaux objectifs étaient de décrire les caractéristiques et d’évaluer la qualité des évaluations économiques publiées dans les revues de médecine d’urgence.

Méthodes

Deux examinateurs ont effectué une recherche indépendante dans 19 revues spécialisées en médecine d’urgence au moyen de Medline et d’Embase, du début jusqu’au 3 mars 2022. L’évaluation de la qualité a été effectuée à l’aide de l’outil Quality of Health Economic Studies (QHES), et le résultat principal a été la note de l’QHES sur 100. De plus, nous avons relevé des facteurs qui pourraient contribuer à des publications de meilleure qualité.

Résultats

7260 articles uniques ont produit 48 évaluations économiques qui répondaient aux critères d’inclusion. La plupart des études étaient des analyses coût-utilité et de haute qualité, avec un score médian de l’EPQE de 84 (plage interquartile, IQR : 72,90). Les études fondées sur des modèles mathématiques et celles conçues principalement comme une évaluation économique ont été associées à des notes de qualité supérieure. Les éléments les plus souvent omis de l’EPQH étaient : i) fournir et justifier la perspective de l’analyse, ii) fournir une justification pour le résultat principal, et iii) choisir un résultat qui était suffisamment long pour permettre aux événements pertinents de se produire.

Conclusions

La majorité des évaluations économiques de la santé dans la littérature sur la médecine d’urgence sont des analyses coût-utilité et sont de grande qualité. Les modèles d’analyse décisionnelle et les études principalement conçus comme des analyses économiques ont été positivement corrélés avec une qualité supérieure. Pour améliorer la qualité des études, les futures évaluations économiques des ME devraient justifier le choix de la perspective de l’analyse et le choix du résultat principal

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Drescher FS, Sirovich BE. Use of computed tomography in emergency departments in the united states: a decade of coughs and colds. JAMA Intern Med. 2016;176(2):273–5.

    PubMed  Google Scholar 

  2. Wong ML, Chiu D, Shapiro NI, Grossman SA. Utility of chest radiography in emergency department patients presenting with syncope. West J Emerg Med. 2016;17(6):698–701.

    PubMed  PubMed Central  Google Scholar 

  3. Baugh CW, Sun BC, Su E, Nicks BA, Shah MN, Adler DH, et al. Variation in diagnostic testing for older patients with syncope in the emergency department. Am J Emerg Med. 2019;37(5):810–6.

    PubMed  Google Scholar 

  4. Rubin R. The costs of US emergency department visits. JAMA. 2021;325(4):333.

    CAS  PubMed  Google Scholar 

  5. Global spending on health: Weathering the storm [Internet]. Available from: https://www.who.int/publications/i/item/9789240017788. Accessed 22 Sep 2021

  6. Chen W, Linthicum B, Argon NT, Bohrmann T, Lopiano K, Mehrotra A, et al. The effects of emergency department crowding on triage and hospital admission decisions. Am J Emerg Med. 2020;38(4):774–9.

    PubMed  Google Scholar 

  7. Fonterra J, Bhatt P, Lalchan R, Yaghi S, Ahuja T, Papadopolous J, et al. Cost comparison of andexanet versus prothrombin complex concentrates for direct factor Xa inhibitor reversal after hemorrhage. J Throm Haemost J. 2020;49(1):121–31.

    Google Scholar 

  8. Miciele A, Demchuck A, Wijeysundera H. Economic evaluation of andexanet versus prothrombin complex concentrate for reversal of factor Xa-associated intracranial hemorrhage. Stroke. 2021;52:1390–7. https://doi.org/10.1161/STROKEAHA.120.031108.

    Article  Google Scholar 

  9. Drummond M, Sculpher M, Claxton K, Stoddart G, Torrance G. Methods for the economic evaluation of health care programmes fourth. Oxford: Oxford University Press; 2015. p. 2–12.

    Google Scholar 

  10. Sassi F. Calculating QALYs, comparing QALY and DALY calculations. Health Policy Plan. 2006;21(5):402–8.

    PubMed  Google Scholar 

  11. Javidan AP, Naji F, Li A, Wu A, Srivatsav V, Rapanos T, et al. A Systematic review of economic evaluations in vascular surgery. Ann Vasc Surg. 2020;67:511–20.

    PubMed  Google Scholar 

  12. Liu CC, Lui J, Oddone Paolucci E, Rudmik L. Systematic review of the quality of economic evaluations in the otolaryngology literature. Otolaryngol Head Neck Surg (United States). 2015;152(1):106–15. https://doi.org/10.1177/0194599814556725.

    Article  Google Scholar 

  13. Ziolkowski NI, Voineskos SH, Ignacy TA, Thoma A. Systematic review of economic evaluations in plastic surgery. In: Plastic and Reconstructive Surgery, vol. 132. Philadelphia: Lippincott Williams and Wilkins; 2013. p. 191–203.

    Google Scholar 

  14. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;29:372.

    Google Scholar 

  15. Strings Attached: CADTH’s Database Search Filters|CADTH.ca [Internet]. Available from: https://www.cadth.ca/resources/finding-evidence/strings-attached-cadths-database-search-filters#syst. Accessed 2 Mar 2021

  16. Chiou CF, Hay JW, Wallace JF, Bloom BS, Neumann PJ, Sullivan SD, et al. Development and validation of a grading system for the quality of cost-effectiveness studies. Med Care. 2003;41(1):32–44.

    PubMed  Google Scholar 

  17. Spiegel BMR, Targownik LE, Kanwal F, Derosa V, Dulai GS, Gralnek IM, et al. The quality of published health economic analyses in digestive diseases: a systematic review and quantitative appraisal. Gastroenterology. 2004;127(2):403–11.

    PubMed  Google Scholar 

  18. Stevens ER, Farrell D, Jumkhawala SA, Ladapo JA. Quality of health economic evaluations for the ACC/AHA stable ischemic heart disease practice guideline: a systematic review. Am Heart J. 2018;1(204):17–33.

    Google Scholar 

  19. Walker DG, Walker RFW; RS; JB; LN; EBB; KF; DG, Sharma R, Bridges J, Niessen L, Bass EB, et al. Best Practices for Conducting Economic Evaluations in Health Care: A Systematic Review of Quality Assessment Tools [Internet]. Rockville, MD: Agency for Healthcare Research and Quality (US); 2012. 1–50 p. Available from: http://www.ncbi.nlm.nih.gov/pubmed/23230577. Accessed 2 Mar 2021

  20. Min C, Xue M, Haotian F, Jialian L, Lingli Z. An overview of the characteristics and quality assessment criteria in systematic review of pharmacoeconomics. PLoS One. 2021;16(2):e0246080.

    CAS  PubMed  PubMed Central  Google Scholar 

  21. Gray AM, Clarke PM, Wolstenholme JL, Wordsworth S. Applied methods of cost-effective analysis in health care. Oxford: Oxford University Press; 2011.

    Google Scholar 

  22. Jefferson T, Demicheli V, Vale L. Quality of systematic reviews of economic evaluations in health care. JAMA. 2002;287(21):2809–12.

    PubMed  Google Scholar 

  23. Husereau D, Drummond M, Augustovski F, de Bekker-Grob E, Briggs AH, Carswell C, et al. Consolidated Health economic evaluation reporting standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations. Value Heal. 2022;25(1):3–9.

    Google Scholar 

  24. Anis AH, Stiell IG, Stewart DG, Laupacis A. Cost-effectiveness analysis of the Ottawa Ankle Rules. Ann Emerg Med. 1995;26(4):422–8.

    CAS  PubMed  Google Scholar 

  25. Asha SE, Chan ACF, Walter E, Kelly PJ, Morton RL, Ajami A, et al. Impact from point-of-care devices on emergency department patient processing times compared with central laboratory testing of blood samples: a randomised controlled trial and cost-effectiveness analysis. Emerg Med J. 2014;31(9):714–9.

    PubMed  Google Scholar 

  26. Bosmans JE, Boeke AJ, Van Randwijck-Jacobze ME, Grol SM, Kramer MH, Van Der Horst HE, et al. Addition of a general practitioner to the accident and emergency department: a cost-effective innovation in emergency care. Emerg Med J. 2012;29(3):192–6.

    PubMed  Google Scholar 

  27. Boudreau DM, Guzauskas G, Villa KF, Fagan SC, Veenstra DL. A model of cost-effectiveness of tissue plasminogen activator in patient subgroups 3 to 4.5 hours after onset of acute ischemic stroke. Ann Emerg Med. 2013;61(1):46–55.

    PubMed  Google Scholar 

  28. Cheng I, Castren M, Kiss A, Zwarenstein M, Brommels M, Mittmann N. Cost-effectiveness of a physician-nurse supplementary triage assessment team at an academic tertiary care emergency department. CJEM. 2016;18(3):191–204.

    PubMed  Google Scholar 

  29. Collins SP, Schauer DP, Gupta A, Brunner H, Storrow AB, Eckman MH. Cost-effectiveness analysis of ED decision making in patients with non-high-risk heart failure. Am J Emerg Med. 2009;27(3):293–302.

    PubMed  Google Scholar 

  30. Dalziel K, Cheek JA, Fanning L, Borland ML, Phillips N, Kochar A, et al. A cost-effectiveness analysis comparing clinical decision rules PECARN, CATCH, and CHALICE with usual care for the management of pediatric head injury. Ann Emerg Med. 2019;73(5):429–39.

    PubMed  Google Scholar 

  31. Dowdy DW, Rodriguez RM, Bradley Hare C, Kaplan B. Cost-effectiveness of targeted human immunodeficiency virus screening in an urban emergency department. Acad Emerg Med. 2011;18(7):745–53.

    PubMed  Google Scholar 

  32. Dugas AF, Coleman S, Gaydos CA, Rothman RE, Frick KD. Cost-utility of rapid polymerase chain reaction-based influenza testing for high-risk emergency department patients. Ann Emerg Med. 2013;62(1):80–8.

    PubMed  PubMed Central  Google Scholar 

  33. Duriseti RS, Brandeau ML. Cost-effectiveness of strategies for diagnosing pulmonary embolism among emergency department patients presenting with undifferentiated symptoms. Ann Emerg Med [Internet]. 2010;56(4). Available from: https://pubmed-ncbi-nlm-nih-gov.proxy.bib.uottawa.ca/20605261/. Accessed 2 Jun 2022

  34. Duriseti RS, Shachter RD, Brandeau ML. Value of quantitative D-dimer assays in identifying pulmonary embolism: implications from a sequential decision model. Acad Emerg Med. 2006;13(7):755–66.

    PubMed  Google Scholar 

  35. Fitzgerald P, Goodacre SW, Cross E, Dixon S. Cost-effectiveness of point-of-care biomarker assessment for suspected myocardial infarction: the randomized assessment of treatment using panel Assay of cardiac markers (RATPAC) trial. Acad Emerg Med. 2011;18(5):488–95.

    PubMed  Google Scholar 

  36. Goldstein LN, Wells M, Vincent-Lambert C. The cost-effectiveness of upfront point-of-care testing in the emergency department: a secondary analysis of a randomised, controlled trial. Scand J Trauma Resusc Emerg Med [Internet]. 2019;27(1). Available from: https://pubmed-ncbi-nlm-nih-gov.proxy.bib.uottawa.ca/31829227/. Accessed 2 Jun 2022

  37. Goldstein LN, Wells M, Vincent-Lambert C. The cost of time: a randomised, controlled trial to assess the economic impact of upfront, point-of-care blood tests in the emergency centre. Afr J Emerg Med Rev africaine la Med d’urgence. 2019;9(2):57–63.

    Google Scholar 

  38. Goodacre S, Dixon S. Is a chest pain observation unit likely to be cost effective at my hospital? Extrapolation of data from a randomised controlled trial. Emerg Med J. 2005;22(6):418–22.

    CAS  PubMed  PubMed Central  Google Scholar 

  39. Goodacre S, Calvert N. Cost effectiveness of diagnostic strategies for patients with acute, undifferentiated chest pain. Emerg Med J. 2003;20(5):429–33.

    CAS  PubMed  PubMed Central  Google Scholar 

  40. Gordon JA. Cost-benefit analysis of social work services in the emergency department: a conceptual model. Acad Emerg Med. 2001;8(1):54–60.

    CAS  PubMed  Google Scholar 

  41. Herd DW, Newbury C, Brown PM. Cost benefit analysis of amethocaine (Ametop) compared with EMLA for intravenous cannulation in a children’s emergency department. Emerg Med J. 2010;27(6):456–60.

    PubMed  Google Scholar 

  42. Hohl CM, Nosyk B, Sadatsafavi M, Anis AH. A cost-effectiveness analysis of propofol versus midazolam for procedural sedation in the emergency department. Acad Emerg Med. 2008;15(1):32–9.

    PubMed  Google Scholar 

  43. Kabrhel C, Ali A, Choi JG, Hur C. Systemic thrombolysis, catheter-directed thrombolysis, and anticoagulation for intermediate-risk pulmonary embolism: a simulation modeling analysis. Acad Emerg Med. 2017;24(10):1235–43.

    PubMed  Google Scholar 

  44. Khare RK, Courtney DM, Powell ES, Venkatesh AK, Lee TA. Sixty-four-slice computed tomography of the coronary arteries: cost-effectiveness analysis of patients presenting to the emergency department with low-risk chest pain. Acad Emerg Med. 2008;15(7):623–32.

    PubMed  Google Scholar 

  45. Law S, Ghag D, Grafstein E, Stenstrom R, Harris D. A pharmacoeconomic study of traditional anticoagulation versus direct oral anticoagulation for the treatment of venous thromboembolism in the emergency department. CJEM. 2016;18(5):340–8.

    PubMed  Google Scholar 

  46. Leblanc J, Hejblum G, Costagliola D, Durand-Zaleski I, Lert F, de Truchis P, et al. Targeted HIV screening in eight emergency departments: the DICI-VIH cluster-randomized two-period crossover trial. Ann Emerg Med. 2018;72(1):41-53.e9.

    PubMed  Google Scholar 

  47. Lim A, Lim A. Cost-benefit analysis of retrospectively identifying missed compensable billings in the emergency department. Emerg Med Australas. 2020;32(6):1021–6.

    PubMed  Google Scholar 

  48. Mace SE, Harb G, Friend K, Turpin R, Armstrong EP, Lebel F. Cost-effectiveness of recombinant human hyaluronidase-facilitated subcutaneous versus intravenous rehydration in children with mild to moderate dehydration. Am J Emerg Med. 2013;31(6):928–34.

    PubMed  Google Scholar 

  49. Malhotra A, Wu X, Kalra VB, Schindler J, Forman HP. Cost-effectiveness analysis of follow-up strategies for thunderclap headache patients with negative noncontrast CT. Acad Emerg Med. 2016;23(3):243–50.

    PubMed  Google Scholar 

  50. Meltzer AC, Ward MJ, Gralnek IM, Pines JM. The cost-effectiveness analysis of video capsule endoscopy compared to other strategies to manage acute upper gastrointestinal hemorrhage in the ED. Am J Emerg Med. 2014;32(8):823–32.

    PubMed  Google Scholar 

  51. Mohr NM, Stoltze A, Ahmed A, Kiscaden E, Shane D. Using continuous quantitative capnography for emergency department procedural sedation: a systematic review and cost-effectiveness analysis. Intern Emerg Med. 2018;13(1):75–85. https://doi.org/10.1007/s11739-016-1587-3.

    Article  PubMed  Google Scholar 

  52. Moore S, Young T, Irving A, Goodacre S, Brennan A, Amos Y. Controlled observational study and economic evaluation of the effect of city-centre night-time alcohol intoxication management services on the emergency care system compared with usual care. Emerg Med J. 2021;38(7):504–10.

    PubMed  Google Scholar 

  53. Moretti ME, Ungar WJ, Freedman SB, Schuh S. Cost-effectiveness of preferred fluids versus electrolytes in pediatric gastroenteritis. CJEM. 2021;23(5):646–54.

    PubMed  Google Scholar 

  54. Nishijima DK, Yang Z, Clark JA, Kuppermann N, Holmes JF, Melnikow J. A cost-effectiveness analysis comparing a clinical decision rule versus usual care to risk stratify children for intraabdominal injury after blunt torso trauma. Acad Emerg Med. 2013;20(11):1131–8.

    PubMed  PubMed Central  Google Scholar 

  55. Nishijima DK, Yang Z, Urbich M, Holmes JF, Zwienenberg-Lee M, Melnikow J, et al. Cost-effectiveness of the PECARN rules in children with minor head trauma. Ann Emerg Med. 2015;65(1):72-80.e6.

    PubMed  Google Scholar 

  56. Oakley E, Carter R, Murphy B, Borland M, Neutze J, Acworth J, et al. Economic evaluation of nasogastric versus intravenous hydration in infants with bronchiolitis. Emerg Med Australas. 2017;29(3):324–9. https://doi.org/10.1111/1742-6723.12713.

    Article  PubMed  Google Scholar 

  57. Ong MEH, Coyle D, Lim SH, Stiell I. Cost-effectiveness of hair apposition technique compared with standard suturing in scalp lacerations. Ann Emerg Med. 2005;46(3):237–42.

    PubMed  Google Scholar 

  58. Overmann KM, Robinson BRH, Eckman MH. Cervical spine evaluation in pediatric trauma: a cost-effectiveness analysis. Am J Emerg Med. 2020;38(11):2347–55.

    PubMed  Google Scholar 

  59. Patterson BW, Khare RK, Courtney DM, Lee TA, Kyriacou DN. Cost-effectiveness of influenza vaccination of older adults in the ED setting. Am J Emerg Med. 2012;30(7):1072–9.

    PubMed  Google Scholar 

  60. Pham B, Teague L, Mahoney J, Goodman L, Paulden M, Poss J, et al. Early prevention of pressure ulcers among elderly patients admitted through emergency departments: a cost-effectiveness analysis. Ann Emerg Med [Internet]. 2011;58(5). Available from: https://pubmed-ncbi-nlm-nih-gov.proxy.bib.uottawa.ca/21820208/. Accessed 2 Jun 2022

  61. Quinn JV, McDermott D, Rossi J, Stein J, Kramer N. Randomized controlled trial of prophylactic antibiotics for dog bites with refined cost model. West J Emerg Med. 2010;11(5):435.

    PubMed  PubMed Central  Google Scholar 

  62. Rudis MI, Touchette DR, Swadron SP, Chiu AP, Orlinsky M. Cost-effectiveness of oral phenytoin, intravenous phenytoin, and intravenous fosphenytoin in the emergency department. Ann Emerg Med. 2004;43(3):386–97.

    PubMed  Google Scholar 

  63. Troyer JL, Jones AE, Shapiro NI, Mitchell AM, Hewer I, Kline JA. Cost-effectiveness of quantitative pretest probability intended to reduce unnecessary medical radiation exposure in emergency department patients with chest pain and dyspnea. Acad Emerg Med. 2015;22(5):525–35.

    PubMed  Google Scholar 

  64. Twiner MJ, Marinica AL, Kuper K, Goodman A, Mahn JJ, Burla MJ, et al. Screening and treatment for subclinical hypertensive heart disease in emergency department patients with uncontrolled blood pressure: a cost-effectiveness analysis. Acad Emerg Med. 2017;24(2):168–76.

    PubMed  PubMed Central  Google Scholar 

  65. Wang RC, Niedzwiecki MJ, Nishijima D, Rodriguez RM. An impact analysis of the NEXUS chest CT clinical decision rule. Am J Emerg Med. 2020;38(5):906–10.

    PubMed  Google Scholar 

  66. Ward MJ, Sodickson A, Diercks DB, Raja AS. Cost-effectiveness of lower extremity compression ultrasound in emergency department patients with a high risk of hemodynamically stable pulmonary embolism. Acad Emerg Med. 2011;18(1):22–31.

    PubMed  Google Scholar 

  67. Ward MJ, Eckman MH, Schauer DP, Raja AS, Collins S. Cost-effectiveness of telemetry for hospitalized patients with low-risk chest pain. Acad Emerg Med. 2011;18(3):279–86.

    PubMed  PubMed Central  Google Scholar 

  68. Ward MJ, Bonomo JB, Adeoye O, Raja AS, Pines JM. Cost-effectiveness of diagnostic strategies for evaluation of suspected subarachnoid hemorrhage in the emergency department. Acad Emerg Med. 2012;19(10):1134–44.

    PubMed  Google Scholar 

  69. Wu X, Malhotra A, Geng B, Liu R, Abbed K, Forman HP, et al. Cost-effectiveness of magnetic resonance imaging in cervical spine clearance of neurologically intact patients with blunt trauma. Ann Emerg Med. 2018;71(1):64–73.

    PubMed  Google Scholar 

  70. Yen Z-S, Davis MA, Chen S-C, Chen W-J. A cost-effectiveness analysis of treatment strategies for acute uncomplicated pyelonephritis in women. Acad Emerg Med. 2003;10(4):309–14.

    PubMed  Google Scholar 

  71. Yun BJ, Myriam Hunink MG, Prabhakar AM, Heng M, Liu SW, Qudsi R, et al. Diagnostic imaging strategies for occult hip fractures: a decision and cost-effectiveness analysis. Acad Emerg Med. 2016;23(10):1161–9.

    PubMed  Google Scholar 

Download references

Funding

This study was funded by a Grant from the Department of Emergency Medicine, The Ottawa Hospital. Ottawa, ON, Canada.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shawn Chhabra.

Ethics declarations

Conflict of interest

SC, AC, KT, LS and KY report no conflict of interest.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 882 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chhabra, S., Cameron, A., Thavorn, K. et al. Quality of health economic evaluations in emergency medicine journals: a systematic review. Can J Emerg Med 25, 676–688 (2023). https://doi.org/10.1007/s43678-023-00535-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s43678-023-00535-w

Keywords

Mots-clés

Navigation