Identifying reprioritization response shift in a stroke caregiver population: a comparison of missing data methods

Abstract

Purpose

Response shift (RS) is an important phenomenon that influences the assessment of longitudinal changes in health-related quality of life (HRQOL) studies. Given that RS effects are often small, missing data due to attrition or item non-response can contribute to failure to detect RS effects. Since missing data are often encountered in longitudinal HRQOL data, effective strategies to deal with missing data are important to consider. This study aims to compare different imputation methods on the detection of reprioritization RS in the HRQOL of caregivers of stroke survivors.

Methods

Data were from a Canadian multi-center longitudinal study of caregivers of stroke survivors over a one-year period. The Stroke Impact Scale physical function score at baseline, with a cutoff of 75, was used to measure patient stroke severity for the reprioritization RS analysis. Mean imputation, likelihood-based expectation–maximization imputation, and multiple imputation methods were compared in test procedures based on changes in relative importance weights to detect RS in SF-36 domains over a 6-month period. Monte Carlo simulation methods were used to compare the statistical powers of relative importance test procedures for detecting RS in incomplete longitudinal data under different missing data mechanisms and imputation methods.

Results

Of the 409 caregivers, 15.9 and 31.3 % of them had missing data at baseline and 6 months, respectively. There were no statistically significant changes in relative importance weights on any of the domains when complete-case analysis was adopted. But statistical significant changes were detected on physical functioning and/or vitality domains when mean imputation or EM imputation was adopted. There were also statistically significant changes in relative importance weights for physical functioning, mental health, and vitality domains when multiple imputation method was adopted. Our simulations revealed that relative importance test procedures were least powerful under complete-case analysis method and most powerful when a mean imputation or multiple imputation method was adopted for missing data, regardless of the missing data mechanism and proportion of missing data.

Conclusions

Test procedures based on relative importance measures are sensitive to the type and amount of missing data and imputation method. Relative importance test procedures based on mean imputation and multiple imputation are recommended for detecting RS in incomplete data.

Appendices

Appendix 1

Relative importance tests for response shift detection in incomplete longitudinal data.

To implement relative importance tests under complete-case analysis method, observations with at least one missing observation on any of the domains are deleted. The differences in relative weights are conducted on the complete data. Similarly, for the mean imputation and EM imputation methods, tests of differences in relative importance weights are conducted on the mean imputed and EM imputed datasets, respectively.

In contrast, for multiple imputation methods, the following steps are taking to implement the relative importance tests.

• 1. Estimate the observed differences in relative importance weights for domains on the original data. This is accomplished as follows:

  • a. Multiply impute the original incomplete longitudinal data by creating M copies of the imputed dataset.

  • b. For each imputed dataset, estimate the relative importance weights at each occasions (w _1mk and w _2mk ; m = 1, …, M; k = 1, …, p) and the differences in relative importance weights over the two occasions (i.e., W _mk  = w _1mk –w _2mk ).

  • c. Use Rubin’s rule as described in Sect. “Methods” to combine the estimates of the differences in relative importance weights for the M datasets. The average difference in relative importance weights is called the observed relative importance weights, $\hat{W}_{k}$.

• 2. Generate the distribution of the differences in relative importance weights by repeatedly sampling from the original incomplete data with replacement (say B times).

  • a. For each bootstrapped incomplete dataset, implement steps 1a–c to estimate the bootstrap-specific differences in relative weights $\hat{W}_{bk}$.

  • b. Sort the $\hat{W}_{bk}$ s (b = 1, 2, …, B) from the smallest to the highest. This is the bootstrap distribution of the difference in relative importance weights from the B bootstrap samples.

  • c. Estimate the W _bk s that corresponds to the 100 (α/2)th percentile (W _bkL ) and the 100 (1−α/2)th percentile (W _bkU ) of the bootstrap distribution.

• 3. Statistical significance of the differences in relative importance weights on a domain is considered present if W _k is outside the interval (W _bkL , W _bkU ).

Appendix 2

Description of missing data patterns in the longitudinal study of stroke caregivers.

See Table 5.

Table 5 Description of missing data patterns in the SF-36 domains of the longitudinal study of stroke caregivers

Appendix 3

See Tables 6 and 7.

Table 6 Percentile confidence intervals and p values for relative importance tests based on standardized discriminant function coefficients and standardized logistic regression coefficients for complete-case analysis and mean imputed data

Table 7 Percentile confidence intervals and p values for relative importance tests based on standardized discriminant function coefficients and standardized logistic regression coefficients for EM imputation and multiple imputation methods