Abstract
Patient-reported outcome and experience measures are increasingly important in health care and health research. The use of these measures is growing in the US and overseas, and performance measures that incorporate patient-reported outcomes are being considered, particularly in cancer. A major challenge for the use of these measures is patient non-response, especially for diseases such as cancer and dementia. A commonly used approach is to ask a proxy such as the patient’s spouse or child to complete the measure on their behalf. Proxy reporting is used in major surveys, including those used in pay-for-performance approaches. No standards exist regarding how to adjust for the use of proxy-reported measures in analyses. As patients requiring proxies likely differ in important ways from those who can self-report, adjusting for these differences is important. In this paper, we evaluate the use of propensity score models when adjusting for proxy-reported data, including weighting, matching with replacement, and non-parametric multiple imputation. Additionally, because previous analyses using propensity scores for proxy reports have employed stepwise or p value based algorithms, we evaluated the sensitivity of our results to the inclusion of respondent-sensitive variables such as proxy reports of patient health status, as well as auxiliary covariates. Under all propensity score methods, estimates obtained from propensity scores using respondent-insensitive variables were different from those obtained when respondent-sensitive variables were incorporated in the propensity score. Propensity score methods have limitations in these contexts and their assumptions should be carefully examined.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abadie, A., Imbens, G.W.: Large sample properties of matching estimators for average treatment effects. Econometrica 74, 235–267 (2006)
Austin, P.C.: Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies. Pharm. Stat. 10, 150–161 (2011)
Austin, P.C., Stuart, E.A.: Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Stat. Med. 34, 3661–3679 (2015)
Ayanian, J.Z., Zaslavsky, A.M., Arora, N.K., Kahn, K.L., Malin, J.L., Ganz, P.A., van Ryn, M., Hornbrook, M.C., Kiefe, C.I., He, Y., Urmie, J.M., Weeks, J.C., Harrington, D.P.: Patients’ experiences with care for lung cancer and colorectal cancer: findings from the Cancer Care Outcomes Research and Surveillance Consortium. J. Clin. Oncol. 28, 4154–4161 (2010)
Bertsekas, D.: Linear Network Optimization: Algorithms and Codes. MIT Press, Cambridge (1991)
Carpenter, J.R., Kenward, M.G.: Multiple Imputation and its Application. Wiley, West Sussex (2013)
Catalano, P.J., Ayanian, J.Z., Weeks, J.C., Kahn, K.L., Landrum, M.B., Zaslavsky, A.M., Lee, J., Pendergast, J., Harrington, D.P., Cancer Care Outcomes Research Surveillance, C.: Representativeness of participants in the cancer care outcomes research and surveillance consortium relative to the surveillance epidemiology and end results program. Med. Care 51, e9–e15 (2013)
Centers for Medicare and Medicaid Services: Oncology Care Model https://innovation.cms.gov/initiatives/oncology-care/ (2018). Accessed 6 Feb 2018
Cole, S.R., Hernan, M.A.: Constructing inverse probability weights for marginal structural models. Am. J. Epidemiol. 168, 656–664 (2008)
Elliott, M.N., Beckett, M.K., Chong, K., Hambarsoomians, K., Hays, R.D.: How do proxy responses and proxy-assisted responses differ from what Medicare beneficiaries might have reported about their health care? Health Serv. Res. 43, 833–848 (2008)
Ellis, B.H., Bannister, W.M., Cox, J.K., Fowler, B.M., Shannon, E.D., Drachman, D., Adams, R.W., Giordano, L.A.: Utilization of the propensity score method: an exploratory comparison of proxy-completed to self-completed responses in the Medicare Health Outcomes Survey. Health Qual. Life Outcomes 1, 47 (2003)
Evensen, C., Yost, K.J., Keller, S., Cowans, T., Frentzel, E., Jenkins, S., Egginton, J., Arora, N.K., Garfinkel, S.: CAHPS Cancer Care Survey: development, testing, and final content of a survey of patient experience with cancer care. J. Clin. Oncol. 35, 227 (2017)
Faries, D., Leon, A.C., Haro, J.M., Obenchain, R.L.: Analysis of Observational Health Care Data Using SAS. SAS Institute Inc, Cary (2010)
Fitzmaurice, G., Kenward, M.G., Molenberghs, G., Verbeke, G., Tsiatis, A.A.: Introduction and preliminaries. In: Molenberghs, G., Fitzmaurice, G., Kenward, M.G., Tsiatis, A., Verbeke, G. (eds.) Handbook of Missing Data Methodology, pp. 3–19. CRC Press, Boca Raton (2015)
Gu, C., Gutman, R.: Development of a common patient assessment scale across the continuum of care: a nested multiple imputation approach. Ann. Appl. Stat. 13, 466–491 (2019)
Gu, X.S., Rosenbaum, P.R.: Comparison of multivariate matching methods: structures, distances, and algorithms. J. Comput. Graph. Stat. 2, 405–420 (1993)
Hall, J.A., Milburn, M.A., Epstein, A.M.: A causal model of health status and satisfaction with medical care. Med. Care 31, 84–94 (1993)
Harel, O.: Strategies for Data Analysis with Two Types of Missing Values. Department of Statistics. Pennsylvania State University, Pennsylvania (2003)
He, Y., Zaslavsky, A.M., Landrum, M.B., Harrington, D.P., Catalano, P.: Multiple imputation in a large-scale complex survey: a practical guide. Stat. Methods Med. Res. 19, 653–670 (2010)
Higginson, I.J., Gao, W.: Caregiver assessment of patients with advanced cancer: concordance with patients, effect of burden and positivity. Health Qual. Life Outcomes 6, 42 (2008)
Hill, J.: Reducing Bias in Treatment Effect Estimation in Observational Studies Suffering from Missing Data. Institute for Social and Economic Research and Policy, New York (2004)
Hill, B.M., Shaw, A.: The wikipedia gender gap revisited: characterizing survey response bias with propensity score estimation. PLoS ONE 8, e65782 (2013)
Jacobs, B.L., Lopa, S.H., Yabes, J.G., Nelson, J.B., Barnato, A.E., Degenholtz, H.B.: Association of functional status and treatment choice among older men with prostate cancer in the medicare advantage population. Cancer 122, 3199–3206 (2016)
Kolen, M.J., Brennan, R.L.: Test Equating, Scaling, and Linking: Methods and Practices. Springer, New York (2014)
Lavori, P.W., Dawson, R., Shera, D.: A multiple imputation strategy for clinical trials with truncation of patient data. Stat. Med. 14, 1913–1925 (1995)
Lee, B.K., Lessler, J., Stuart, E.A.: Improving propensity score weighting using machine learning. Stat. Med. 29, 337–346 (2010)
Lee, B.K., Lessler, J., Stuart, E.A.: Weight trimming and propensity score weighting. PLoS ONE 6, e18174 (2011)
Lenis, D., Nguyen, T.Q., Dong, N., Stuart, E.A.: It’s All About Balance: Propensity Score Matching in the Context of Complex Survey Data. Johns Hopkins University, Department of Biostatistics, Baltimore (2017)
Leyrat, C., Seaman, S.R., White, I.R., Douglas, I., Smeeth, L., Kim, J., Resche-Rigon, M., Carpenter, J.R.,Williamson, E.J.: Propensity score analysis with partially observed confounders: how should multiple imputation be used? https://arxiv.org/abs/1608.05606 (2016). Accessed 3 Feb 2016
Li, M., Harris, I., Lu, Z.K.: Differences in proxy-reported and patient-reported outcomes: assessing health and functional status among medicare beneficiaries. BMC Med. Res. Methodol. 15, 62 (2015)
Little, R.J.A.: Missing-data adjustments in large surveys. J. Bus. Econ. Stat. 6, 287–296 (2012)
Little, R.J.A., Rubin, D.B.: Statistical Analysis with Missing Data. Wiley, Hoboken (2002)
Malin, J.L., Ko, C., Ayanian, J.Z., Harrington, D., Nerenz, D.R., Kahn, K.L., Ganther-Urmie, J., Catalano, P.J., Zaslavsky, A.M., Wallace, R.B., Guadagnoli, E., Arora, N.K., Roudier, M.D., Ganz, P.A.: Understanding cancer patients’ experience and outcomes: development and pilot study of the cancer care outcomes research and surveillance patient survey. Support. Care Cancer 14, 837–848 (2006)
Mattei, A.: Estimating and using propensity score in presence of missing background data: an application to assess the impact of childbearing on wellbeing. Stat. Methods Appl. 18, 257–273 (2009)
Milne, D.J., Mulder, L.L., Beelen, H.C., Schofield, P., Kempen, G.I., Aranda, S.: Patients’ self-report and family caregivers’ perception of quality of life in patients with advanced cancer: how do they compare? Eur. J. Cancer Care (Engl.) 15, 125–132 (2006)
Mitra, R., Reiter, J.P.: A comparison of two methods of estimating propensity scores after multiple imputation. Stat. Methods Med. Res. 25, 188–204 (2016)
Nekolaichuk, C.L., Bruera, E., Spachynski, K., MacEachern, T., Hanson, J., Maguire, T.O.: A comparison of patient and proxy symptom assessments in advanced cancer patients. Palliat. Med. 13, 311–323 (1999)
Patorno, E., Glynn, R.J., Hernandez-Diaz, S., Liu, J., Schneeweiss, S.: Studies with many covariates and few outcomes: selecting covariates and implementing propensity-score-based confounding adjustments. Epidemiology 25, 268–278 (2014)
Rubin, D.B.: Inference and missing data. Biometrika 63, 581–592 (1976)
Rubin, D.B.: The Bayesian bootstrap. Ann. Stat. 9, 130–134 (1981)
Rubin, D., Schenker, N.: Multiple imputation for interval estimation from simple random samples with ignorable nonresponse. J. Am. Stat. Assoc. 81, 366–374 (1986)
Sekhon, J.S.: Multivariate and propensity score matching software with automated balance optimization: the matching package for R. J. Stat. Soft. 42, 1–52 (2011)
Sneeuw, K.C., Aaronson, N.K., Sprangers, M.A., Detmar, S.B., Wever, L.D., Schornagel, J.H.: Comparison of patient and proxy EORTC QLQ-C30 ratings in assessing the quality of life of cancer patients. J. Clin. Epidemiol. 51, 617–631 (1998)
Sneeuw, K.C., Sprangers, M.A., Aaronson, N.K.: The role of health care providers and significant others in evaluating the quality of life of patients with chronic disease. J. Clin. Epidemiol. 55, 1130–1143 (2002)
Stuart, E.A.: Matching methods for causal inference: a review and a look forward. Stat. Sci. 25, 1–21 (2010)
Van der Wal, W.M., Geskus, R.B.: ipw: an R package for inverse probability weighting. J. Stat. Soft. 43, 1–23 (2011)
von Essen, L.: Proxy ratings of patient quality of life–factors related to patient-proxy agreement. Acta Oncol. 43, 229–234 (2004)
White, H.: A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica 48, 817–838 (1980)
Wolinsky, F.D., Jones, M.P., Wehby, G.L.: Gathering data from older adults via proxy respondents: research challenges. J. Comp. Eff. Res. 1, 467–470 (2012)
Acknowledgements
Jessica Roydhouse was supported by a Dissertation Support Grant from the Jayne Koskinas Ted Giovanis Foundation for Health and Policy. Nancy Keating is supported by K24CA181510 from the National Cancer Institute (NCI). Vince Mor is PI for HS-0011 from the Agency for Healthcare Research and Quality. The CanCORS consortium was funded by the following NCI Grants: U01CA093344, U01CA093332, U01CA093324, U01CA093348, U01CA093339, U01CA093326, and a Department of Veterans Affairs Grant, CRS02-164. The funding bodies had no role in the design of the study reported in this paper, nor in the collection, analysis, and interpretation of the data used in these analyses, nor in the writing of this manuscript. Parts of this work were presented at the ISOQOL 24th Annual Conference, Philadelphia, USA. Jessica Roydhouse’s attendance at the conference was supported by an ISOQOL New Investigator Scholarship and the Health Assessment Lab Tarlov and Ware Award.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
All procedures performed in CanCORS were in accordance with the ethical standards of the institutional research committees and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. When CanCORS data were collected, informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Roydhouse, J.K., Gutman, R., Keating, N.L. et al. Propensity scores for proxy reports of care experience and quality: are they useful?. Health Serv Outcomes Res Method 20, 40–59 (2020). https://doi.org/10.1007/s10742-019-00205-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10742-019-00205-4