Advertisement

Quality of Life Research

, Volume 22, Issue 7, pp 1859–1869 | Cite as

Psychometric characteristics of daily diaries for the Patient-Reported Outcomes Measurement Information System (PROMIS®): a preliminary investigation

  • Stefan Schneider
  • Seung W. Choi
  • Doerte U. Junghaenel
  • Joseph E. Schwartz
  • Arthur A. Stone
Article

Abstract

Purpose

The Patient-Reported Outcomes (PRO) Measurement Information System (PROMIS®) has developed assessment tools for numerous PROs, most using a 7-day recall format. We examined whether modifying the recall period for use in daily diary research would affect the psychometric characteristics of several PROMIS measures.

Methods

Daily versions of short-forms for three PROMIS domains (pain interference, fatigue, depression) were administered to a general population sample (n = 100) for 28 days. Analyses used multilevel item response theory (IRT) models. We examined differential item functioning (DIF) across recall periods by comparing the IRT parameters from the daily data with the PROMIS 7-day recall IRT parameters. Additionally, we examined whether the IRT parameters for day-to-day within-person changes are invariant to those for between-person (cross-sectional) differences in PROs.

Results

Dimensionality analyses of the daily data suggested a single dimension for each PRO domain, consistent with PROMIS instruments. One-third of the daily items showed uniform DIF when compared with PROMIS 7-day recall, but the impact of DIF on the scale level was minor. IRT parameters for within-person changes differed from between-person parameters for 3 depression items, which were more sensitive for measuring change than between-person differences, but not for pain interference and fatigue items. Notably, mean scores from daily diaries were significantly lower than the PROMIS 7-day recall norms.

Conclusions

The results provide initial evidence supporting the adaptation of PROMIS measures for daily diary research. However, scores from daily diaries cannot be directly interpreted on PROMIS norms established for 7-day recall.

Keywords

PROMIS® Patient-reported outcomes Electronic diaries Item response theory 

Notes

Acknowledgments

This research was supported by a grant from the National Institutes of Health (1 U01AR057948-01). We would like to thank Gim Yen Toh and Laura Wolff for their assistance with data collection.

References

  1. 1.
    Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., et al. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology, 63(11), 1179–1194.PubMedCrossRefGoogle Scholar
  2. 2.
    DeWalt, D. A., Rothrock, N., Yount, S., & Stone, A. A. (2007). Evaluation of item candidates—The PROMIS qualitative item review. Medical Care, 45(5), S12–S21.PubMedCrossRefGoogle Scholar
  3. 3.
    Liu, H., Cella, D., Gershon, R., Shen, J., Morales, L. S., Riley, W., et al. (2010). Representativeness of the Patient-Reported Outcomes Measurement Information System Internet panel. Journal of Clinical Epidemiology, 63(11), 1169–1178.PubMedCrossRefGoogle Scholar
  4. 4.
    Institute of Medicine. (2011). Leading health indicators for healthy people 2020: Letter Report. Washington, DC: National Academies Press.Google Scholar
  5. 5.
    Leidy, N. K., Wilcox, T. K., Jones, P. W., Murray, L., Winnette, R., Howard, K., et al. (2010). Development of the EXAcerbations of Chronic Obstructive Pulmonary Disease Tool (EXACT): A Patient-Reported Outcome (PRO) Measure. Value in Health, 13(8), 965–975.PubMedCrossRefGoogle Scholar
  6. 6.
    Jim, H. S., Small, B., Faul, L. A., Franzen, J., Apte, S., & Jacobsen, P. B. (2011). Fatigue, depression, sleep, and activity during chemotherapy: daily and intraday variation and relationships among symptom changes. Annals of Behavioral Medicine, 42(3), 321–333.PubMedCrossRefGoogle Scholar
  7. 7.
    Chapman, C. R., Donaldson, G. W., Davis, J. J., & Bradshaw, D. H. (2011). Improving individual measurement of postoperative pain: The Pain Trajectory. Journal of Pain, 12(2), 257–262.PubMedCrossRefGoogle Scholar
  8. 8.
    Begg, A., Drummond, G., & Tiplady, B. (2003). Assessment of postsurgical recovery after discharge using a pen computer diary. Anaesthesia, 58(11), 1101–1105.PubMedCrossRefGoogle Scholar
  9. 9.
    Broderick, J. E., Schneider, S., Schwartz, J. E., & Stone, A. A. (2010). Interference with activities due to pain and fatigue: Accuracy of ratings across different reporting periods. Quality of Life Research, 19(8), 1163–1170.PubMedCrossRefGoogle Scholar
  10. 10.
    Broderick, J. E., Schwartz, J. E., Schneider, S., & Stone, A. A. (2009). Can end-of-day reports replace momentary assessment of pain and fatigue? Journal of Pain, 10(3), 274–281.PubMedCrossRefGoogle Scholar
  11. 11.
    Broderick, J. E., Schwartz, J. E., Vikingstad, G., Pribbernow, M., Grossman, S., & Stone, A. A. (2008). The accuracy of pain and fatigue items across different reporting periods. Pain, 139(1), 146–157.PubMedCrossRefGoogle Scholar
  12. 12.
    Schwarz, N. (2007). Retrospective and concurrent self-reports: The rationale for real-time data capture. In A. A. Stone, S. S. Shiffman, A. Atienza, & L. Nebeling (Eds.), The science of real-time data capture: Self-reports in health research (pp. 11–26). New York: Oxford University Press.Google Scholar
  13. 13.
    Winkielman, P., Knauper, B., & Schwarz, N. (1998). Looking back at anger: Reference periods change the interpretation of emotion frequency questions. Journal of Personality and Social Psychology, 75(3), 719–728.PubMedCrossRefGoogle Scholar
  14. 14.
    Roesch, S. C., Aldridge, A. A., Stocking, S. N., Villodas, F., Leung, Q., Bartley, C. E., et al. (2010). Multilevel factor analysis and structural equation modeling of daily diary coping data: Modeling trait and state variation. Multivariate Behav Res, 45(5), 767–789.PubMedCrossRefGoogle Scholar
  15. 15.
    Mehta, P. D., & Neale, M. C. (2005). People are variables too: Multilevel structural equations modeling. Psychological Methods, 10(3), 259–284.PubMedCrossRefGoogle Scholar
  16. 16.
    Zyphur, M. J., Kaplan, S. A., & Christian, M. S. (2008). Assumptions of cross-level measurement and structural invariance in the analysis of multilevel data: Problems and solutions. Group Dynamics-Theory Research and Practice, 12(2), 127–140.CrossRefGoogle Scholar
  17. 17.
    Skrondal, A., & Rabe Hesketh, S. (2004). Generalized latent variable modeling: multilevel, longitudinal, and structural equation models. Boca Raton, FL: Chapman & Hall.CrossRefGoogle Scholar
  18. 18.
    Amtmann, D., Cook, K. F., Jensen, M. P., Chen, W. H., Choi, S., Revicki, D., et al. (2010). Development of a PROMIS item bank to measure pain interference. Pain, 150(1), 173–182.PubMedCrossRefGoogle Scholar
  19. 19.
    Lai, J. S., Cella, D., Choi, S., Junghaenel, D. U., Christodoulou, C., Gershon, R., et al. (2011). How item banks and their application can influence measurement practice in rehabilitation medicine: A PROMIS fatigue item bank example. Archives of Physical Medicine and Rehabilitation, 92(10 Suppl), S20–S27.PubMedCrossRefGoogle Scholar
  20. 20.
    Pilkonis, P. A., Choi, S. W., Reise, S. P., Stover, A. M., Riley, W. T., Cella, D., et al. (2011). Item banks for measuring emotional distress from the patient-reported outcomes measurement information system (PROMIS (R)): Depression, Anxiety, and Anger. Assessment, 18(3), 263–283.PubMedCrossRefGoogle Scholar
  21. 21.
    Samejima, F. (1969). Estimation of Latent Ability Using a Response Pattern of Graded Scores. Psychometrika, 34, 100–114.Google Scholar
  22. 22.
    Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks - Plans for the patient-reported outcomes measurement information system (PROMIS). Medical Care, 45(5), S22–S31.PubMedCrossRefGoogle Scholar
  23. 23.
    Muthén, L. K., & Muthén, B. O. (1998-2010). Mplus user’s guide (6th ed.). Los Angeles, CA: Muthén & Muthén.Google Scholar
  24. 24.
    McDonald, R. P. (1982). Linear Versus Non-Linear Models in Item Response Theory. Applied Psychological Measurement, 6(4), 379–396.CrossRefGoogle Scholar
  25. 25.
    Grilli, L., & Rampichini, C. (2007). Multilevel factor models for ordinal variables. Structural Equation Modeling-a Multidisciplinary Journal, 14(1), 1–25.Google Scholar
  26. 26.
    Gregorich, S. E. (2006). Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Medical Care, 44(11), S78–S94.PubMedCrossRefGoogle Scholar
  27. 27.
    Mellenberg, G. J. (1982). Contingency table models for assessing item bias. Journal of Educational Statistics, 7, 105–108.CrossRefGoogle Scholar
  28. 28.
    Muthén, B. O., & Asparouhov, T. (2009). Beyond multilevel regression modeling: multilevel analysis in a general latent variable framework. In J. Hox & J. K. Roberts (Eds.), The Handbook of Advanced Multilevel Analysis (pp. 15–40). New York: Taylor and Francis.Google Scholar
  29. 29.
    Woods, C. M. (2009). Empirical Selection of Anchors for Tests of Differential Item Functioning. Applied Psychological Measurement, 33(1), 42–57.CrossRefGoogle Scholar
  30. 30.
    Stark, S., Chernyshenko, E. S., & Drasgow, F. (2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology, 91(6), 1292–1306.PubMedCrossRefGoogle Scholar
  31. 31.
    Benjamini, Y., & Hochberg, Y. (1995). Controlling the False Discovery Rate - a Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society Series B-Methodological, 57(1), 289–300.Google Scholar
  32. 32.
    Stone, A. A., Schwartz, J. E., Broderick, J. E., & Shiffman, S. S. (2005). Variability of momentary pain predicts recall of weekly pain: A consequence of the peak (or salience) memory heuristic. Personality and Social Psychology Bulletin, 31(10), 1340–1346.PubMedCrossRefGoogle Scholar
  33. 33.
    Schneider, S., Stone, A. A., Schwartz, J. E., & Broderick, J. E. (2011). Peak and End Effects in Patients’ Daily Recall of Pain and Fatigue: A Within-Subjects Analysis. Journal of Pain, 12(2), 228–235.PubMedCrossRefGoogle Scholar
  34. 34.
    Noonan, V. K., Cook, K. F., Bamer, A. M., Choi, S. W., Kim, J., & Amtmann, D. (2011). Measuring fatigue in persons with multiple sclerosis: creating a crosswalk between the Modified Fatigue Impact Scale and the PROMIS Fatigue Short Form. Quality of Life Research.Google Scholar
  35. 35.
    Thissen, D., Varni, J. W., Stucky, B. D., Liu, Y., Irwin, D. E., & Dewalt, D. A. (2011). Using the PedsQL 3.0 asthma module to obtain scores comparable with those of the PROMIS pediatric asthma impact scale (PAIS). Quality of Life Research, 20(9), 1497–1505.PubMedCrossRefGoogle Scholar
  36. 36.
    Gibbons, L. E., Feldman, B. J., Crane, H. M., Mugavero, M., Willig, J. H., Patrick, D., et al. (2011). Migrating from a legacy fixed-format measure to CAT administration: calibrating the PHQ-9 to the PROMIS depression measures. Quality of Life Research, 20(9), 1349–1357.PubMedCrossRefGoogle Scholar
  37. 37.
    Ackerman, T. A. (1992). A Didactic Explanation of Item Bias, Item Impact, and Item Validity from a Multidimensional Perspective. Journal of Educational Measurement, 29(1), 67–91.CrossRefGoogle Scholar
  38. 38.
    Muthén, B. O., & Curran, P. J. (1997). General longitudinal modeling of individual differences in experimental designs: A latent variable framework for analysis and power estimation. Psychological Methods, 2(4), 371–402.CrossRefGoogle Scholar
  39. 39.
    Cranford, J. A., Shrout, P. E., Iida, M., Rafaeli, E., Yip, T., & Bolger, N. (2006). A procedure for evaluating sensitivity to within-person change: Can mood measures in diary studies detect change reliably? Personality and Social Psychology Bulletin, 32(7), 917–929.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2012

Authors and Affiliations

  • Stefan Schneider
    • 1
  • Seung W. Choi
    • 2
  • Doerte U. Junghaenel
    • 1
  • Joseph E. Schwartz
    • 1
  • Arthur A. Stone
    • 1
  1. 1.Department of Psychiatry and Behavioral ScienceStony Brook UniversityStony BrookUSA
  2. 2.Department of Medical Social SciencesNorthwestern University Feinberg School of MedicineChicagoUSA

Personalised recommendations