Extending iterative matching methods: an approach to improving covariate balance that allows prioritisation

Abstract

Comparative effectiveness studies can identify the causal effect of treatment if treatment is unconfounded with outcome conditional on a set of measured covariates. Matching aims to ensure that the covariate distributions are similar between treatment and control groups in the matched samples, and this should be done iteratively by checking and improving balance. However, an outstanding concern facing matching methods is how to prioritise competing improvements in balance across different covariates. We address this concern by developing a ‘loss function’ that an iterative matching method can minimise. Our ‘loss function’ is a transparent summary of covariate imbalance in a matched sample and follows general recommendations in prioritising balance amongst covariates. We illustrate this approach by extending Genetic Matching (GM), an automated approach to balance checking. We use the method to reanalyse a high profile comparative effectiveness study of right heart catheterisation. We find that our loss function improves covariate balance compared to a standard GM approach, and to matching on the published propensity score.

Appendix: Example of a user defined loss function together with annotated R code

The function takes as input the variables matches , BM and nboots . The variable matches is a matrix with three columns, the indices of the treated and control units for a current matched sample and weights corresponding to each matched pair. The variable BM is the balance matrix containing the columns of the data for the variables used to define the loss function. The variable nboots is the number of bootstrap samples to use when computing the KS-bootstrap test p value.

• my.fitfunc <- function(matches, BM, nboots=0){

•      index.treated <- matches[,1]

•      index.control <- matches[,2]

•      weights <- matches[,3]

The variable nvars is the number of variables in the balance matrix. The variables bm1.nvars and bm2.nvars are the number of high and low priority variables respectively. The high priority variables are assumed to be at the beginning. The vector pvals is created as an empty vector to store the p values from the t-tests and KS-tests on the covariates.

• nvars <- ncol(BM)

• bm2.nvars <- nvars-bm1.nvars

• pvals <- c(rep(NA,2 * bm1.nvars))

The t-test and KS-test p values are computed for all of the high priority variables and stored in the vector pvals . The ks.boot function performs the KS-boot test with the number of bootstraps set by the parameter nboots but if the parameter nboots is 0 then the KS-test is performed. Here if the variable nboots is less than 500 then the parameter nboots is set to 0 and the warnings about a low number of bootstraps are suppressed else the parameter is set to the value of the variable nboots . The KS-test p values are stored at the beginning of the vector pvals then the t-test p values.

• for (i in 1:bm1.nvars){

•   if(nboots >= 500){

•     pvals[i] <- ks.boot(BM[index.treated,i], BM[index.control,i],

•      nboots=nboots)$ks.boot.pvalue

•   } else {

•     suppressWarnings(pvals[i] <- ks.boot(BM[index.treated,i],

•      BM[index.control,i],nboots=0)$ks$p.value)

•   }

•   pvals[i+bm1.nvars] <- balanceUV(BM[index.treated,i],

•     BM[index.control,i], paired=TRUE, weights=weights,

•     match=TRUE)$p.value

•   }

The p values are computed similarly for all of the low priority variables and stored in the vector pvals3 instead of pvals1 . To reference the appropriate point in BM , the loop is

• for (i in (bm1.nvars+1):nvars){ }

The variable indx is a vector of TRUE or FALSE corresponding to whether each p value is smaller in the current matched sample than the values stored in conpvals . The offset is needed to take into account the machine precision. Since the GM algorithm aims to minimise the output of the function and it is necessary to maximise the balance test p values, pvals is multiplied by −1. The variable pvals is copied to a variable named pvals1 .

• indx <- conpvals > pvals + sqrt(.Machine$double.eps)

• pvals <- -1*pvals

• pvals1<-pvals

one is added to the p values from the current matched sample which are lower than conpvals , i.e. indx=TRUE , making them positive. Since GM minimises the loss function, it moves towards matched samples which maximise the number of p values that are better than conpvals .

• for(i in 1:length(pvals)){

•   if(indx[i]){

•       pvals1[i] <- 1 + pvals[i]}

• }

This line is very important since it structures the output according to Fig. 4. The vector pvals1 contains the first two blocks and is sorted in decreasing order. Since the values in pvals1 corresponding to variables with worse balance than conpvals are positive then they will be in the first block and given highest priority. The vector pvals3 is given lowest priority and is sorted in decreasing order, as in Fig. 4.

•   loss <- c(sort(pvals1,decreasing=TRUE),sort(pvals3,decreasing=TRUE))

•   return(loss)

• }