Skip to main content
Log in

Propensity scores in intensive care and anaesthesiology literature: a systematic review

  • Review
  • Published:
Intensive Care Medicine Aims and scope Submit manuscript

Abstract

Introduction

Propensity score methods have been increasingly used in the last 10 years. However, the practical use of the propensity score (PS) has been reported as heterogeneous in several papers reviewing the use of propensity scores and giving some advice. No precedent work has focused on the specific application of PS in intensive care and anaesthesiology literature.

Objectives

After a brief development of the theory of propensity score, to assess the use and the quality of reporting of PS studies in intensive care and anaesthesiology, and to evaluate how past reviews have influenced the quality of the reporting.

Study design and setting

Forty-seven articles published between 2006 and 2009 in the intensive care and anaesthesiology literature were evaluated. We extracted the characteristics of the report, the type of analysis, the details of matching procedures, the number of patients in treated and control groups, and the number of covariates included in the PS models.

Results

Of the 47 articles reviewed, 26 used matching on PS, 12 used stratification on PS and 9 used adjustment on PS. The method used was reported in 81% of the articles, and the choice to conduct a paired analysis or not was reported in only 15%. The comparison with the previously published reviews showed little improvement in reporting in the last few years.

Conclusion

The quality of reporting propensity scores in intensive care and anaesthesiology literature should be improved. We provide some recommendations to the investigators in order to improve the reporting of PS analyses.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Ellenberg JH (1994) Selection bias in observational and experimental studies. Stat Med 13:557–567

    Article  CAS  PubMed  Google Scholar 

  2. Rossi P, Freeman H (1993) A systematic approach. Sage Publications, Inc, Newbury Park

    Google Scholar 

  3. Corrie P, Shaw J, Harris R (2003) Rate limiting factors in recruitment of patients to clinical trials in cancer research: descriptive study. BMJ 327:320–321

    Article  PubMed  Google Scholar 

  4. Fossa SD, Skovlund E (2002) Selection of patients may limit the generalizability of results from cancer trials. Acta Oncol 41:131–137

    Article  PubMed  Google Scholar 

  5. Guyatt GH, Sackett DL, Cook DJ (1994) Users’ guides to the medical literature. II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? Evidence-based medicine working group. Jama 271:59–63

    Article  CAS  PubMed  Google Scholar 

  6. Pocock SJ, Elbourne DR (2000) Randomized trials or observational tribulations? N Engl J Med 342:1907–1909

    Article  CAS  PubMed  Google Scholar 

  7. Rosenbaum P, Rubin D (1983) The central role of the propensity score in observational studies for causal effect. Biometrika 70:41–55

    Article  Google Scholar 

  8. Lunceford JK, Davidian M (2004) Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Stat Med 23:2937–2960

    Article  PubMed  Google Scholar 

  9. Rubin D (1976) Matching methods that are equal percent bias reducing: some examples. Biometrics 35:417–446

    Google Scholar 

  10. Rubin D (1979) Using multivariate matched sampling and regression adjustment to control bias in observational studies. J Am Stat Assoc 74:318–324

    Article  Google Scholar 

  11. Rubin D (1980) Bias reduction using Mahalanobis metric matching. Biometrics 36:293–298

    Article  Google Scholar 

  12. Carpenter R (1977) Matching when covariables are normally distributed. Biometrika 64:299–307

    Article  Google Scholar 

  13. Austin PC (2009) Some methods of propensity-score matching had superior performance to others: results of an empirical investigation and Monte Carlo simulations. Biom J 51:171–184

    Article  PubMed  Google Scholar 

  14. Rubin D (2001) Using propensity scores to help design observational studies. Health Serv Out Res Method 2:169–188

    Article  Google Scholar 

  15. Austin PC (2008) Assessing balance in measured baseline covariates when using many-to-one matching on the propensity-score. Pharmacoepidemiol Drug Saf 17:1218–1225

    Article  PubMed  Google Scholar 

  16. Aronow HD, Novaro GM, Lauer MS, Brennan DM, Lincoff AM, Topol EJ, Kereiakes DJ, Nissen SE (2003) In-hospital initiation of lipid-lowering therapy after coronary intervention as a predictor of long-term utilization: a propensity analysis. Arch Intern Med 163:2576–2582

    Article  PubMed  Google Scholar 

  17. Srinivasan AK, Shackcloth MJ, Grayson AD, Fabri BM (2003) Preoperative beta-blocker therapy in coronary artery bypass surgery: a propensity score analysis of outcomes. Interact Cardiovasc Thorac Surg 2:495–500

    Article  PubMed  Google Scholar 

  18. Vikram HR, Buenconsejo J, Hasbun R, Quagliarello VJ (2003) Impact of valve surgery on 6-month mortality in adults with complicated, left-sided native valve endocarditis: a propensity analysis. JAMA 290:3207–3214

    Article  CAS  PubMed  Google Scholar 

  19. Elad Y, French WJ, Shavelle DM, Parsons LS, Sada MJ, Every NR (2002) Primary angioplasty and selection bias inpatients presenting late (>12 h) after onset of chest pain and ST elevation myocardial infarction. J Am Coll Cardiol 39:826–833

    Article  PubMed  Google Scholar 

  20. Sabik JF, Gillinov AM, Blackstone EH, Vacha C, Houghtaling PL, Navia J, Smedira NG, McCarthy PM, Cosgrove DM, Lytle BW (2002) Does off-pump coronary surgery reduce morbidity and mortality? J Thorac Cardiovasc Surg 124:698–707

    Article  PubMed  Google Scholar 

  21. Austin PC, Grootendorst P, Anderson GM (2007) A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study. Stat Med 26:734–753

    Article  PubMed  Google Scholar 

  22. Weitzen S, Lapane KL, Toledano AY, Hume AL, Mor V (2005) Weaknesses of goodness-of-fit tests for evaluating propensity score models: the case of the omitted confounder. Pharmacoepidemiol Drug Saf 14:227–238

    Article  PubMed  Google Scholar 

  23. Austin PC (2009) Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat Med 28:3083–3107

    Article  PubMed  Google Scholar 

  24. Wijeysundera DN, Beattie WS, Austin PC, Hux JE, Laupacis A (2008) Epidural anaesthesia and survival after intermediate-to-high risk non-cardiac surgery: a population-based cohort study. Lancet 372:562–569

    Article  PubMed  Google Scholar 

  25. Austin PC (2007) The performance of different propensity score methods for estimating marginal odds ratios. Stat Med 26:3078–3094

    Article  PubMed  Google Scholar 

  26. Austin PC (2008) The performance of different propensity-score methods for estimating relative risks. J Clin Epidemiol 61:537–545

    Article  PubMed  Google Scholar 

  27. Almog Y, Novack V, Eisinger M, Porath A, Novack L, Gilutz H (2007) The effect of statin therapy on infection-related mortality in patients with atherosclerotic diseases. Crit Care Med 35:372–378

    Article  CAS  PubMed  Google Scholar 

  28. Annane D, Sebille V, Duboc D, Le Heuzey JY, Sadoul N, Bouvier E, Bellissant E (2008) Incidence and prognosis of sustained arrhythmias in critically ill patients. Am J Respir Crit Care Med 178:20–25

    Article  PubMed  Google Scholar 

  29. Biki B, Mascha E, Moriarty DC, Fitzpatrick JM, Sessler DI, Buggy DJ (2008) Anesthetic technique for radical prostatectomy surgery affects cancer recurrence: a retrospective analysis. Anesthesiology 109:180–187

    Article  PubMed  Google Scholar 

  30. Clec’h C, Alberti C, Vincent F, Garrouste-Orgeas M, de Lassence A, Toledano D, Azoulay E, Adrie C, Jamali S, Zaccaria I, Cohen Y, Timsit JF (2007) Tracheostomy does not improve the outcome of patients requiring prolonged mechanical ventilation: a propensity analysis. Crit Care Med 35:132–138

    Article  PubMed  Google Scholar 

  31. Combes A, Luyt CE, Nieszkowska A, Trouillet JL, Gibert C, Chastre J (2007) Is tracheostomy associated with better outcomes for patients requiring long-term mechanical ventilation? Crit Care Med 35:802–807

    Article  PubMed  Google Scholar 

  32. de Boer MT, Christensen MC, Asmussen M, van der Hilst CS, Hendriks HG, Slooff MJ, Porte RJ (2008) The impact of intraoperative transfusion of platelets and red blood cells on survival after liver transplantation. Anesth Analg 106:32–44 (table of contents)

    Article  PubMed  Google Scholar 

  33. Dhainaut JF, Payet S, Vallet B, Franca LR, Annane D, Bollaert PE, Le Tulzo Y, Runge I, Malledant Y, Guidet B, Le Lay K, Launois R (2007) Cost-effectiveness of activated protein C in real-life clinical practice. Crit Care 11:R99

    Article  PubMed  Google Scholar 

  34. Duncan AI, Koch CG, Xu M, Manlapaz M, Batdorf B, Pitas G, Starr N (2007) Recent metformin ingestion does not increase in-hospital morbidity or mortality after cardiac surgery. Anesth Analg 104:42–50

    Article  CAS  PubMed  Google Scholar 

  35. Duncan AI, Lin J, Koch CG, Gillinov AM, Xu M, Starr NJ (2006) The impact of gender on in-hospital mortality and morbidity after isolated aortic valve replacement. Anesth Analg 103:800–808

    Article  PubMed  Google Scholar 

  36. Ender J, Borger MA, Scholz M, Funkat AK, Anwar N, Sommer M, Mohr FW, Fassl J (2008) Cardiac surgery fast-track treatment in a postanesthetic care unit: six-month results of the Leipzig fast-track concept. Anesthesiology 109:61–66

    Article  PubMed  Google Scholar 

  37. Eurich DT, Marrie TJ, Johnstone J, Majumdar SR (2008) Mortality reduction with influenza vaccine in patients with pneumonia outside “flu” season: pleiotropic benefits or residual confounding? Am J Respir Crit Care Med 178:527–533

    Article  PubMed  Google Scholar 

  38. Fellahi JL, Parienti JJ, Hanouz JL, Plaud B, Riou B, Ouattara A (2008) Perioperative use of dobutamine in cardiac surgery and adverse cardiac outcome: propensity-adjusted analyses. Anesthesiology 108:979–987

    Article  CAS  PubMed  Google Scholar 

  39. Grathwohl KW, Black IH, Spinella PC, Sweeney J, Robalino J, Helminiak J, Grimes J, Gullick R, Wade CE (2008) Total intravenous anesthesia including ketamine versus volatile gas anesthesia for combat-related operative traumatic brain injury. Anesthesiology 109:44–53

    Article  CAS  PubMed  Google Scholar 

  40. Griesdale DE, Bosma TL, Kurth T, Isac G, Chittock DR (2008) Complications of endotracheal intubation in the critically ill. Intensive Care Med 34:1835–1842

    Article  PubMed  Google Scholar 

  41. Hix JK, Thakar CV, Katz EM, Yared JP, Sabik J, Paganini EP (2006) Effect of off-pump coronary artery bypass graft surgery on postoperative acute kidney injury and mortality. Crit Care Med 34:2979–2983

    PubMed  Google Scholar 

  42. Honiden S, Schultz A, Im SA, Nierman DM, Gong MN (2008) Early versus late intravenous insulin administration in critically ill patients. Intensive Care Med 34:881–887

    Article  CAS  PubMed  Google Scholar 

  43. Kertai MD, Westerhout CM, Varga KS, Acsady G, Gal J (2008) Dihydropiridine calcium-channel blockers and perioperative mortality in aortic aneurysm surgery. Br J Anaesth 101:458–465

    Article  CAS  PubMed  Google Scholar 

  44. Le Manach Y, Godet G, Coriat P, Martinon C, Bertrand M, Fleron MH, Riou B (2007) The impact of postoperative discontinuation or continuation of chronic statin therapy on cardiac outcome after major vascular surgery. Anesth Analg 104:1326–1333 (table of contents)

    Article  CAS  PubMed  Google Scholar 

  45. Ranucci M, Isgro G (2007) Minimally invasive cardiopulmonary bypass: does it really change the outcome? Crit Care 11:R45

    Article  PubMed  Google Scholar 

  46. Rowan KM, Welch CA, North E, Harrison DA (2008) Drotrecogin alfa (activated): real-life use and outcomes for the UK. Crit Care 12:R58

    Article  PubMed  Google Scholar 

  47. Scales DC, Thiruchelvam D, Kiss A, Redelmeier DA (2008) The effect of tracheostomy timing during critical illness on long-term survival. Crit Care Med 36:2547–2557

    Article  PubMed  Google Scholar 

  48. Schortgen F, Girou E, Deye N, Brochard L (2008) The risk associated with hyperoncotic colloids in patients with shock. Intensive Care Med 34:2157–2168

    Article  PubMed  Google Scholar 

  49. Thombs BD, Bresnick MG (2008) Mortality risk and length of stay associated with self-inflicted burn injury: evidence from a national sample of 30, 382 adult patients. Crit Care Med 36:118–125

    Article  PubMed  Google Scholar 

  50. Tritapepe L, De Santis V, Vitale D, Nencini C, Pellegrini F, Landoni G, Toscano F, Miraldi F, Pietropaoli P (2007) Recombinant activated factor VII for refractory bleeding after acute aortic dissection surgery: a propensity score analysis. Crit Care Med 35:1685–1690

    Article  CAS  PubMed  Google Scholar 

  51. Vandijck DM, Benoit DD (2008) Impact of recent intravenous chemotherapy on outcome in severe sepsis and septic shock patients with haematological malignancies: reply to letter by Meyer et al. Intensive Care Med 34:1930–1931

    Article  Google Scholar 

  52. Vincent JL, Sakr Y, Sprung C, Harboe S, Damas P (2008) Are blood transfusions associated with greater mortality rates? Results of the sepsis occurrence in acutely Ill patients study. Anesthesiology 108:31–39

    Article  PubMed  Google Scholar 

  53. Berger MM, Soguel L, Shenkin A, Revelly JP, Pinget C, Baines M, Chiolero RL (2008) Influence of early antioxidant supplements on clinical evolution and organ function in critically ill cardiac surgery, major trauma, and subarachnoid hemorrhage patients. Crit Care 12:R101

    Article  PubMed  Google Scholar 

  54. Constantinides VA, Tekkis PP, Fazil A, Kaur K, Leonard R, Platt M, Casula R, Stanbridge R, Darzi A, Athanasiou T (2006) Fast-track failure after cardiac surgery: development of a prediction model. Crit Care Med 34:2875–2882

    PubMed  Google Scholar 

  55. Meier R, Bechir M, Ludwig S, Sommerfeld J, Keel M, Steiger P, Stocker R, Stover JF (2008) Differential temporal profile of lowered blood glucose levels (3.5 to 6.5 mmol/l versus 5 to 8 mmol/l) in patients with severe traumatic brain injury. Crit Care 12:R98

    Article  PubMed  Google Scholar 

  56. Bagshaw SM, Lapinsky S, Dial S, Arabi Y, Dodek P, Wood G, Ellis P, Guzman J, Marshall J, Parrillo JE, Skrobik Y, Kumar A (2009) Acute kidney injury in septic shock: clinical outcomes and impact of duration of hypotension prior to initiation of antimicrobial therapy. Intensive Care Med 35:871–881

    Article  PubMed  Google Scholar 

  57. Kor DJ, Iscimen R, Yilmaz M, Brown MJ, Brown DR, Gajic O (2009) Statin administration did not influence the progression of lung injury or associated organ failures in a cohort of patients with acute lung injury. Intensive Care Med 35:1039–1046

    Article  CAS  PubMed  Google Scholar 

  58. Zarychanski R, Doucette S, Fergusson D, Roberts D, Houston DS, Sharma S, Gulati H, Kumar A (2008) Early intravenous unfractionated heparin and mortality in septic shock. Crit Care Med 36:2973–2979

    Article  CAS  PubMed  Google Scholar 

  59. Beattie WS, Karkouti K, Wijeysundera DN, Tait G (2009) Risk associated with preoperative anemia in noncardiac surgery: a single-center cohort study. Anesthesiology 110:574–581

    Article  PubMed  Google Scholar 

  60. Beattie WS, Wijeysundera DN, Karkouti K, McCluskey S, Tait G, Mitsakakis N, Hare GM (2009) Acute surgical anemia influences the cardioprotective effects of beta-blockade: a single-center, propensity-matched cohort study. Anesthesiology 112:25–33

    Article  Google Scholar 

  61. Christensen S, Thomsen RW, Johansen MB, Pedersen L, Jensen R, Larsen KM, Larsson A, Tonnesen E, Sorensen HT (2009) Preadmission statin use and one-year mortality among patients in intensive care—a cohort study. Crit Care 14:R29

    Article  Google Scholar 

  62. Devasia RA, Blackman A, Gebretsadik T, Griffin M, Shintani A, May C, Smith T, Hooper N, Maruri F, Warkentin J, Mitchel E, Sterling TR (2009) Fluoroquinolone resistance in Mycobacterium tuberculosis: the effect of duration and timing of fluoroquinolone exposure. Am J Respir Crit Care Med 180:365–370

    Article  CAS  PubMed  Google Scholar 

  63. Karkouti K, Wijeysundera DN, Yau TM, McCluskey SA, Tait G, Beattie WS (2009) The risk-benefit profile of aprotinin versus tranexamic acid in cardiac surgery. Anesth Analg 110:21–29

    Article  PubMed  Google Scholar 

  64. Kerger KH, Mascha E, Steinbrecher B, Frietsch T, Radke OC, Stoecklein K, Frenkel C, Fritz G, Danner K, Turan A, Apfel CC (2009) Routine use of nasogastric tubes does not reduce postoperative nausea and vomiting. Anesth Analg 109:768–773

    Article  PubMed  Google Scholar 

  65. Leslie K, Myles PS, Forbes A, Chan MT (2009) The effect of bispectral index monitoring on long-term survival in the B-aware trial. Anesth Analg 110:816–822

    Article  PubMed  Google Scholar 

  66. Lindenauer PK, Rothberg MB, Nathanson BH, Pekow PS, Steingrub JS (2010) Activated protein C and hospital mortality in septic shock: a propensity-matched analysis. Crit Care Med 38:1101–1107

    Article  PubMed  Google Scholar 

  67. Manrique A, Jooste EH, Kuch BA, Lichtenstein SE, Morell V, Munoz R, Ellis D, Davis PJ (2009) The association of renal dysfunction and the use of aprotinin in patients undergoing congenital cardiac surgery requiring cardiopulmonary bypass. Anesth Analg 109:45–52

    Article  CAS  PubMed  Google Scholar 

  68. Martin G, Brunkhorst FM, Janes JM, Reinhart K, Sundin DP, Garnett K, Beale R (2009) The international PROGRESS registry of patients with severe sepsis: drotrecogin alfa (activated) use and patient outcomes. Crit Care 13:R103

    Article  PubMed  Google Scholar 

  69. Payen JF, Bosson JL, Chanques G, Mantz J, Labarere J (2009) Pain assessment is associated with decreased duration of mechanical ventilation in the intensive care unit: a post Hoc analysis of the DOLOREA study. Anesthesiology 111:1308–1316

    Article  PubMed  Google Scholar 

  70. Renaud B, Santin A, Coma E, Camus N, Van Pelt D, Hayon J, Gurgui M, Roupie E, Herve J, Fine MJ, Brun-Buisson C, Labarere J (2009) Association between timing of intensive care unit admission and outcomes for emergency department patients with community-acquired pneumonia. Crit Care Med 37:2867–2874

    Article  PubMed  Google Scholar 

  71. Rioux JP, Lessard M, De Bortoli B, Roy P, Albert M, Verdant C, Madore F, Troyanov S (2009) Pentastarch 10% (250 kDa/0.45) is an independent risk factor of acute kidney injury following cardiac surgery. Crit Care Med 37:1293–1298

    Article  CAS  PubMed  Google Scholar 

  72. Surgenor SD, Kramer RS, Olmstead EM, Ross CS, Sellke FW, Likosky DS, Marrin CA, Helm RE Jr, Leavitt BJ, Morton JR, Charlesworth DC, Clough RA, Hernandez F, Frumiento C, Benak A, DioData C, O’Connor GT (2009) The association of perioperative red blood cell transfusions and decreased long-term survival after cardiac surgery. Anesth Analg 108:1741–1746

    Article  PubMed  Google Scholar 

  73. van Klei WA, Bryson GL, Yang H, Forster AJ (2009) Effect of beta-blocker prescription on the incidence of postoperative myocardial infarction after hip and knee arthroplasty. Anesthesiology 111:717–724

    Article  PubMed  Google Scholar 

  74. Wisnivesky JP, Halm E, Bonomi M, Powell C, Bagiella E (2009) Effectiveness of radiation therapy for elderly patients with unresected stage I and II non-small cell lung cancer. Am J Respir Crit Care Med 181:264–269

    Article  PubMed  Google Scholar 

  75. Harrel FE (2001) Overfitting and limits on number of predictors. In: Regression modeling strategies. Series in Statistics. Springer (ed), pp 60–61

  76. Weitzen S, Lapane KL, Toledano AY, Hume AL, Mor V (2004) Principles for modeling propensity scores in medical research: a systematic literature review. Pharmacoepidemiol Drug Saf 13:841–853

    Article  PubMed  Google Scholar 

  77. Shah BR, Laupacis A, Hux JE, Austin PC (2005) Propensity score methods gave similar results to traditional regression modeling in observational studies: a systematic review. J Clin Epidemiol 58:550–559

    Article  PubMed  Google Scholar 

  78. Sturmer T, Joshi M, Glynn RJ, Avorn J, Rothman KJ, Schneeweiss S (2006) A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. J Clin Epidemiol 59:437–447

    Article  PubMed  Google Scholar 

  79. Austin PC (2008) A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003. Stat Med 27:2037–2049

    Article  PubMed  Google Scholar 

  80. Austin PC (2007) Propensity-score matching in the cardiovascular surgery literature from 2004 to 2006: a systematic review and suggestions for improvement. J Thorac Cardiovasc Surg 134:1128–1135

    Article  PubMed  Google Scholar 

  81. Austin PC (2008) The performance of different propensity score methods for estimating marginal odds ratios, Statistics in Medicine 2007; 26:3078–3094. Stat Med 27:3918–3920

    Article  Google Scholar 

  82. Austin PC, Grootendorst P, Normand SL, Anderson GM (2007) Conditioning on the propensity score can result in biased estimation of common measures of treatment effect: a Monte Carlo study. Stat Med 26:754–768

    Article  PubMed  Google Scholar 

  83. Agresti A, Min Y (2005) Simple improved confidence intervals for comparing matched proportions. Stat Med 24:729–740

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Etienne Gayat.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gayat, E., Pirracchio, R., Resche-Rigon, M. et al. Propensity scores in intensive care and anaesthesiology literature: a systematic review. Intensive Care Med 36, 1993–2003 (2010). https://doi.org/10.1007/s00134-010-1991-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00134-010-1991-5

Keywords

Navigation