Extending iterative matching methods: an approach to improving covariate balance that allows prioritisation

  • Roland R. Ramsahai
  • Richard Grieve
  • Jasjeet S. Sekhon


Comparative effectiveness studies can identify the causal effect of treatment if treatment is unconfounded with outcome conditional on a set of measured covariates. Matching aims to ensure that the covariate distributions are similar between treatment and control groups in the matched samples, and this should be done iteratively by checking and improving balance. However, an outstanding concern facing matching methods is how to prioritise competing improvements in balance across different covariates. We address this concern by developing a ‘loss function’ that an iterative matching method can minimise. Our ‘loss function’ is a transparent summary of covariate imbalance in a matched sample and follows general recommendations in prioritising balance amongst covariates. We illustrate this approach by extending Genetic Matching (GM), an automated approach to balance checking. We use the method to reanalyse a high profile comparative effectiveness study of right heart catheterisation. We find that our loss function improves covariate balance compared to a standard GM approach, and to matching on the published propensity score.


Matching Propensity score Mahalanobis distance Balance measure Right heart catheterisation 


  1. Abadie, A.: Bootstrap tests for distributional treatment effects in instrumental variables models. J. Am. Stat. Assoc. 97, 284–292 (2002)CrossRefGoogle Scholar
  2. Abadie, A., Imbens, G.W.: Large sample properties of matching estimators for average treatment effects. Econometrica 74, 235–267 (2006)CrossRefGoogle Scholar
  3. Austin, P.C.: A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003. Stat. Med. 27, 2037–2049 (2008)PubMedCrossRefGoogle Scholar
  4. Austin, P.C.: Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat. Med. 28, 3083–3107 (2009)PubMedCrossRefGoogle Scholar
  5. Austin, P.C., Grootendorst, P., Anderson, G.M.: A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study. Stat. Med. 26, 734–753 (2007)PubMedCrossRefGoogle Scholar
  6. Brookhart, M.A., Schneeweiss, S., Rothman, K.J., Glynn, R.J., Avorn, J., Stürmer, T.: Variable selection for propensity score models. Am. J. Epidemiol. 163, 1149–1156 (2006)PubMedCrossRefGoogle Scholar
  7. Chittock, D., Dhingra, V., Ronco, J., Russell, J., Forrest, D., Tweeddale, M., Fenwick, J.: Severity of illness and risk of death associated with pulmonary artery catheter use. Crit. Care Med. 32, 911–915 (2004)PubMedCrossRefGoogle Scholar
  8. Cochran, W.G., Rubin, D.B.: Controlling bias in observational studies: a review. Sankhyā Indian J. Stat. A 35, 417–446 (1973)Google Scholar
  9. Connors, A.F., Speroff, T., Dawson, N.V., Thomas, C., Harrell, F.E., Wagner, D., Desbiens, N., Goldman, L., Wu, A.W., Califf, R.M., Fulkerson, W.J., Vidaillet, H., Broste, S., Bellamy, P., Lynn, J., Knaus, W.A.: The effectiveness of right heart catheterization in the intial care of critically ill patients. J. Am. Med. Assoc. 276, 889–897 (1996)CrossRefGoogle Scholar
  10. Dawid, A.P.: Conditional independence in statistical theory. J. R. Stat. Soc B 41, 1–31 (1979)Google Scholar
  11. Diamond, A., Sekhon, J.: Genetic matching for estimating causal effects: a general multivariate matching method for achieving balance in observational studies. In progress (2010); working paper available from
  12. Drake, C.: Effects of misspecification of the propensity score on estimators of treatment effect. Biometrics 49, 1231–1236 (1993)CrossRefGoogle Scholar
  13. Glance, L.G., Osler, T.M., Mukamel, D.B., Dick, A.W.: Use of a matching algorithm to evaluate hospital coronary artery bypass grafting performance as an alternative to conventional risk adjustment. Med. Care 45, 292–299 (2007)PubMedCrossRefGoogle Scholar
  14. Grootendorst, P.: A review of instrumental variables estimation of treatment effects in the applied health sciences. Health Serv. Outcome Res. Methodol. 7, 159–179 (2007)CrossRefGoogle Scholar
  15. Hansen, B.B.: The prognostic analogue of the propensity score. Biometrika 95, 481–488 (2008)CrossRefGoogle Scholar
  16. Hansen, B.B., Bowers, J.: Covariate balance in simple, stratified and clustered comparative studies. Stat. Sci. 23, 219–236 (2008)CrossRefGoogle Scholar
  17. Harvey, S., Harrison, D.A., Singer, M., Ashcroft, J., Jones, C.M., Elbourne, D., Brampton, W., Williams, D., Young, D., Rowan, K.: Assessment of the effectiveness of pulmonary artery catheters in management of patients in intensive care (PAC-Man): a randomised controlled trial. Lancet 366, 472–477 (2005)PubMedCrossRefGoogle Scholar
  18. Helfand, M.: Comparative effectiveness research. Med. Decis. Mak. 29, 641 (2009)CrossRefGoogle Scholar
  19. Hill, J., Reiter, J.P.: Interval estimation for treatment effects using propensity score matching. Stat. Med. 25, 2230–2256 (2006)PubMedCrossRefGoogle Scholar
  20. Hirano, K., Imbens, G.W.: Estimation of causal effects using propensity score weighting: an application to data on right heart catheterization. Health Serv. Outcome Res. Methodol. 2, 259–278 (2001)CrossRefGoogle Scholar
  21. Ho, D., Imai, K., King, G., Stuart, E.A.: Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Anal. 15, 199–236 (2007)CrossRefGoogle Scholar
  22. Imai, K., King, G., Stuart, E.A.: Misunderstandings between experimentalists and observationalists about causal inference. J. R. Stat. Soc. A 171, 481–502 (2008)CrossRefGoogle Scholar
  23. Knaus, W., Lynn, J.: Study to understand prognoses and preferences for outcomes and risks of treatment (SUPPORT) 1989-1997 [Computer file]. ICPSR version. George Washington University [producer], Washington, DC, 2000. Inter-university Consortium for Political and Social Research [distributor], Ann Arbor, MI, 2001 (1997). doi: 10.3886/ICPSR02957
  24. Lalonde, R.: Evaluating the econometric evaluations of training programs with experimental data. Am. Econ. Rev. 76, 604–620 (1986)Google Scholar
  25. Levy, A., Harrigan, B., Johnston, K., Briggs, A.: Comparative effectiveness research through the looking glass. Med. Decis. Mak. 29, N6–N8 (2009)CrossRefGoogle Scholar
  26. Pearl, J.: Causal diagrams for empirical research. Biometrika 82, 669–710 (1995)CrossRefGoogle Scholar
  27. Rosenbaum, P.R.: Observational Studies. Springer, New York (2002)Google Scholar
  28. Rosenbaum, P., Rubin, D.: The central role of the propensity score in observational studies for causal effects. Biometrika 70, 410–455 (1983)CrossRefGoogle Scholar
  29. Rosenbaum, P., Rubin, D.B.: Reducing bias in observational studies using subclassification on the propensity score. J. Am. Stat. Assoc. 79, 516–524 (1984)CrossRefGoogle Scholar
  30. Rosenbaum, P.R., Rubin, D.B.: Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. Am. Stat. 39, 33–38 (1985)CrossRefGoogle Scholar
  31. Rubin, D.B.: Multivariate matching methods that are equal percent bias reducing, I: some examples. Biometrics 32, 109–120 (1976)CrossRefGoogle Scholar
  32. Rubin, D.B.: The design versus the analysis of observational studies for causal effects: parallels with the design of randomized trials. Stat. Med. 26, 20–36 (2007)PubMedCrossRefGoogle Scholar
  33. Sekhon, J.S.: Multivariate and propensity score matching software with automated balance optimization: the matching package for R. J. Stat. Softw. 42, 1–52 (2011)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Roland R. Ramsahai
    • 1
  • Richard Grieve
    • 1
  • Jasjeet S. Sekhon
    • 2
  1. 1.Department of Health Services Research and PolicyLondon School of Hygiene and Tropical MedicineLondonUK
  2. 2.Travers Department of Political ScienceUniversity of California BerkeleyBerkeleyUSA

Personalised recommendations