Skip to main content

Advertisement

SpringerLink
Log in

Evaluating measurement invariance across assessment modes of phone interview and computer self-administered survey for the PROMIS measures in a population-based cohort of localized prostate cancer survivors

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

To evaluate measurement invariance (phone interview vs computer self-administered survey) of 15 PROMIS measures responded by a population-based cohort of localized prostate cancer survivors.

Methods

Participants were part of the North Carolina Prostate Cancer Comparative Effectiveness and Survivorship Study. Out of the 952 men who took the phone interview at 24 months post-treatment, 401 of them also completed the same survey online using a home computer. Unidimensionality of the PROMIS measures was examined using single-factor confirmatory factor analysis (CFA) models. Measurement invariance testing was conducted using longitudinal CFA via a model comparison approach. For strongly or partially strongly invariant measures, changes in the latent factors and factor autocorrelations were also estimated and tested.

Results

Six measures (sleep disturbance, sleep-related impairment, diarrhea, illness impact—negative, illness impact—positive, and global satisfaction with sex life) had locally dependent items, and therefore model modifications had to be made on these domains prior to measurement invariance testing. Overall, seven measures achieved strong invariance (all items had equal loadings and thresholds), and four measures achieved partial strong invariance (each measure had one item with unequal loadings and thresholds). Three measures (pain interference, interest in sexual activity, and global satisfaction with sex life) failed to establish configural invariance due to between-mode differences in factor patterns.

Conclusions

This study supports the use of phone-based live interviewers in lieu of PC-based assessment (when needed) for many of the PROMIS measures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., et al. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. Journal of Clinical Epidemiology, 63(11), 1179–1194. doi:10.1016/j.jclinepi.2010.04.011.

    Article  PubMed  PubMed Central  Google Scholar 

  2. DeWalt, D. A., Rothrock, N., Yount, S., & Stone, A. A. (2007). Evaluation of item candidates: The PROMIS qualitative item review. Medical Care, 45(5), S12–S21. doi:10.1097/01.mlr.0000254567.79743.e2.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., Thissen, D., Revicki, D. A., Weiss, D. J., Hambleton, R. K., Liu, H., Gershon, R., Reise, S. P., Lai, J., Cella, D., & PROMIS Cooperative Group. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the patient-reported outcomes measurement information system (PROMIS). Medical Care, 45(5), S22–S31. Retrieved from http://www.jstor.org/stable/40221455

  4. Alonso, J., Bartlett, S. J., Rose, M., Aaronson, N. K., Chaplin, J. E., Efficace, F., et al. (2013). The case for an international patient-reported outcomes measurement information system (PROMIS) initiative. Health and Quality of Life Outcomes, 11, 210. doi:10.1186/1477-7525-11-210.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Hahn, E. A., Cella, D., Dobrez, D., Shiomoto, G., Marcus, E., Taylor, S. G., et al. (2004). The talking touchscreen: A new approach to outcomes assessment in low literacy. Psycho-Oncology, 13(2), 86–95. doi:10.1002/pon.719.

    Article  PubMed  Google Scholar 

  6. Bjorner, J. B., Rose, M., Gandek, B., Stone, A. A., Junghaenel, D. U., & Ware, J. E., Jr. (2014). Difference in method of administration did not significantly impact item response: An IRT-based analysis from the patient-reported outcomes measurement information system (PROMIS) initiative. Quality of Life Research, 23(1), 217–227. doi:10.1007/s11136-013-0451-4.

    Article  PubMed  Google Scholar 

  7. Bjorner, J. B., Rose, M., Gandek, B., Stone, A. A., Junghaenel, D. U., & Ware, J. E., Jr. (2014). Method of administration of PROMIS scales did not significantly impact score level, reliability, or validity. Journal of Clinical Epidemiology, 67(1), 108–113. doi:10.1016/j.jclinepi.2013.07.016.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Chen, R. C., Carpenter, W. R., Kim, M., Hendrix, L. H., Agans, R. P., Meyer, A. M., et al. (2015). Design of the North Carolina prostate cancer comparative effectiveness and survivorship study (NC ProCESS). Journal of Comparative Effectiveness Research, 4(1), 3–9. doi:10.2217/cer.14.67.

    Article  PubMed  Google Scholar 

  9. Quach, C. W., Langer, M. M., Chen, R. C., Thissen, D., Usinger, D. S., Emerson, M. A., et al. (2016). Reliability and validity of PROMIS measures administered by telephone interview in a longitudinal localized prostate cancer study. Quality of Life Research, 25(11), 2811–2823. doi:10.1007/s11136-016-1325-3.

    Article  PubMed  Google Scholar 

  10. Jamshidian, M., & Jalal, S. (2010). Tests of homoscedasticity, normality, and missing completely at random for incomplete multivariate data. Psychometrika, 75(4), 649–674. doi:10.1007/s11336-010-9175-3.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Muthén, B. O., du Toit, S. H. C., & Spisic, D. (1997). Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes. Unpublished technical report. Retrieved from, https://www.statmodel.com/download/Article_075.pdf

  12. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. doi:10.1080/10705519909540118.

    Article  Google Scholar 

  13. Sass, D. A. (2011). Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. Journal of Psychoeducational Assessment, 29(4), 347–363. doi:10.1177/0734282911406661.

    Article  Google Scholar 

  14. Yu, C.-Y. (2002). Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes (Doctoral dissertation, University of California Los Angeles). Retrieved from, https://www.statmodel.com/download/Yudissertation.pdf

  15. Little, T. D. (2013). Longitudinal structural equation modeling. New York: Guilford Press.

    Google Scholar 

  16. Muthén, L. K., & Muthén, B. O. (1998–2017). Mplus user’s guide (8th edn). Los Angeles: Muthén & Muthén.

  17. Satorra, A., & Bentler, P. M. (2010). Ensuring positiveness of the scaled difference Chi square test statistic. Psychometrika, 75(2), 243–248. doi:10.1007/s11336-009-9135-y.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93(3), 568–592. doi:10.1037/0021-9010.93.3.568.

    Article  PubMed  Google Scholar 

  19. R Core Team. (2016). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. Retrieved from, http://www.R-project.org/

  20. Jamshidian, M., Jalal, S., & Jansen, C. (2014). MissMech: An R package for testing homoscedasticity, multivariate normality, and missing completely at random (MCAR). Journal of Statistical Software, 56(6), 1–31. doi:10.18637/jss.v056.i06.

    Article  Google Scholar 

  21. van Buuren, S., & Groothuis-Oudshoorn, K. (2011). Mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 1–67. doi:10.18637/jss.v045.i03.

    Article  Google Scholar 

  22. Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. doi:10.18637/jss.v048.i02.

    Article  Google Scholar 

  23. semTools Contributors. (2016). semTools: Useful tools for structural equation modeling. R package version 0.4-14. Retrieved from, https://CRAN.R-project.org/package=semTools

  24. Allison, P. D. (2003). Missing data techniques for structural equation modeling. Journal of Abnormal Psychology, 112(4), 545–557. doi:10.1037/0021-843X.112.4.545.

    Article  PubMed  Google Scholar 

  25. Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York: John Wiley & Sons.

    Book  Google Scholar 

  26. Asparouhov, T., & Muthén, B. (2016). IRT in Mplus. Version 2. Technical report. Retrieved from, https://www.statmodel.com/download/MplusIRT.pdf

  27. File, T. (2013). Computer and internet use in the United States. Current population survey reports, P20-568. U.S. Census Bureau, Washington, DC. Retrieved from, http://www.census.gov/prod/2013pubs/p20-569.pdf

  28. Kutner, M., Greenberg, E., & Baer, J. (2006). A first look at the literacy of America’s adults in the 21st century. NCES 2006-470. National Center for Education Statistics. Retrieved from, https://nces.ed.gov/naal/pdf/2006470_1.pdf

  29. Weinfurt, K. P., Lin, L., Bruner, D. W., Cyranowski, J. M., Dombeck, C. B., Hahn, E. A., et al. (2015). Development and initial validation of the PROMIS sexual function and satisfaction measures version 2.0. The Journal of Sexual Medicine, 12(9), 1961–1974. doi:10.1111/jsm.12966.

    Article  PubMed  Google Scholar 

Download references

Funding

This research was supported by grants from the Agency for Healthcare Research and Quality (HHSA29020050040ITO6) and the National Cancer Institute (R01CA174453).

Author information

Authors and Affiliations

  1. Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Mian Wang, Ronald C. Chen, Deborah S. Usinger & Bryce B. Reeve

  2. Cecil G. Sheps Center for Health Services Research, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Ronald C. Chen & Deborah S. Usinger

  3. Department of Radiation Oncology, School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Ronald C. Chen

  4. Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Bryce B. Reeve

Authors
  1. Mian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ronald C. Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Deborah S. Usinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bryce B. Reeve
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mian Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Appendices

Appendix A

See Table 1.

Appendix B: Tables

See Tables 2, 3 and 4.

Appendix C: Figure

See Fig. 1.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, M., Chen, R.C., Usinger, D.S. et al. Evaluating measurement invariance across assessment modes of phone interview and computer self-administered survey for the PROMIS measures in a population-based cohort of localized prostate cancer survivors. Qual Life Res 26, 2973–2985 (2017). https://doi.org/10.1007/s11136-017-1640-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-017-1640-3

Keywords

Search

Navigation