Does recall period matter? Comparing PROMIS® physical function with no recall, 24-hr recall, and 7-day recall

  • David M. Condon
  • Robert Chapman
  • Sara Shaunfield
  • Michael A. Kallen
  • Jennifer L. Beaumont
  • Daniel Eek
  • Debanjali Mitra
  • Katy L. Benjamin
  • Kelly McQuarrie
  • Jamae Liu
  • James W. Shaw
  • Allison Martin Nguyen
  • Karen Keating
  • David CellaEmail author



To evaluate the influence of recall periods on the assessment of physical function, we compared, in cancer and general population samples, the standard administration of PROMIS Physical Function items without a recall period to administrations with 24-hour and 7-day recall periods.


We administered 31 items from the PROMIS Physical Function v2.0 item bank to 2400 respondents (n = 1001 with cancer; n = 1399 from the general population). Respondents were randomly assigned to one of three recall conditions (no recall, 24-hours, or 7-days) and one of two “reminder” conditions (with recall periods presented only at the start of the survey or with every item). We assessed items for potential differential item functioning (DIF) by recall time period. We then tested recall and reminder effects with analysis of variance controlling for demographics, English fluency, and co-morbidities.


Based on conservative pre-set criteria, no items were flagged for recall time period-related DIF. Using analysis of variance, each condition was compared to the standard PROMIS administration for Physical Function (no recall period). There was no evidence of significant differences among groups in the cancer sample. In the general population sample, only the 24-hour recall condition with reminders was significantly different from the “no recall” PROMIS standard. At the item level, for both samples, the number of items with non-trivial effect size differences across conditions was minimal.


Compared to no recall, the use of a recall period has little to no effect upon PROMIS physical function responses or scores. We recommend that PROMIS Physical Function be administered with the standard PROMIS “no recall” period.


Patient reported outcomes PROMIS Recall period Physical function 



The funding was provided by National Institutes of Health (Grant No. U2CCA186878), AbbVie, Amgen, AstraZeneca, Bayer, Bristol-Myers Squibb, Genentech, Janssen, Merck, Novartis, and Pfizer.

Supplementary material

11136_2019_2344_MOESM1_ESM.docx (32 kb)
Supplementary material 1 (DOCX 31 kb)
11136_2019_2344_MOESM2_ESM.docx (14 kb)
Supplementary material 2 (DOCX 16kb)


  1. 1.
    Weeks, W. B., & Weinstein, J. N. (2016). Patient-reported data can help people make better health care choices. New England Journal of Medicine. Retrieved from Accessed 18 Dec 2017.
  2. 2.
    Butt, Z., & Reeve, B. (2012). Enhancing the patient’s voice: Standards in the design and selection of patient-reported outcomes measures (PROMs) for use in patient-centered outcomes research. Patient-Centered Outcomes Research Institute.Google Scholar
  3. 3.
    Food and Drug Administration. (2009). Guidance for industry: Patient-reported outcome measures: Use in medical product development to support labeling claims. Federal Register,74(235), 65132–65133.Google Scholar
  4. 4.
    Brundage, M., Blazeby, J., Revicki, D., Bass, B., de Vet, H., Duffy, H., et al. (2013). Patient-reported outcomes in randomized clinical trials: Development of ISOQOL reporting standards. Quality of Life Research,22(6), 1161–1175. Scholar
  5. 5.
    Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook, K., Reeve, B., et al. (2007). The patient-reported outcomes measurement information system (PROMIS): Progress of an NIH roadmap cooperative group during its first two years. Medical Care,45(Suppl 1), S3–S11. Scholar
  6. 6.
    Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., et al. (2010). Initial adult health item banks and first wave testing of the patient-reported outcomes measurement information system (PROMIS) network: 2005–2008. Journal of Clinical Epidemiology,63(11), 1179–1194. Scholar
  7. 7.
    Calvert, M., Kyte, D., Mercieca-Bebber, R., Slade, A., Chan, A.-W., King, M. T., et al. (2018). Guidelines for inclusion of patient-reported outcomes in clinical trial protocols. JAMA,319(5), 483–494. Scholar
  8. 8.
    Cook, K., Jensen, S. E., Schalet, B. D., Beaumont, J. L., Amtmann, D., Czajkowski, S., et al. (2016). PROMIS measures of pain, fatigue, negative affect, physical function, and social function demonstrated clinical validity across a range of chronic conditions. Journal of Clinical Epidemiology,73, 89–102. Scholar
  9. 9.
    Jensen, R. E., Potosky, A. L., Reeve, B. B., Hahn, E., Cella, D., Fries, J., et al. (2015). Validation of the PROMIS physical function measures in a diverse US population-based cohort of cancer patients. Quality of Life Research,24(10), 2333–2344. Scholar
  10. 10.
    Lai, J.-S., Cella, D., Choi, S., Junghaenel, D. U., Christodoulou, C., Gershon, R., et al. (2011). How item banks and their application can influence measurement practice in rehabilitation medicine: A PROMIS fatigue item bank example. Archives of Physical Medicine and Rehabilitation,92(10), S20–S27. Scholar
  11. 11.
    Broderick, J. E., Schneider, S., Junghaenel, D. U., Schwartz, J. E., & Stone, A. A. (2013). Validity and reliability of patient-reported outcomes measurement information system (PROMIS) instruments in osteoarthritis. Arthritis Care & Research,65(10), 1625–1633. Scholar
  12. 12.
    Rothrock, N., Hays, R., Spritzer, K., Yount, S., Riley, W., & Cella, D. (2010). Relative to the general US population, chronic diseases are associated with poorer health-related quality of life as measured by the patient-reported outcomes measurement information system (PROMIS). Journal of Clinical Epidemiology,63(11), 1195–1204. Scholar
  13. 13.
    Reeve, B., Hays, R., Bjorner, J., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the patient-reported outcomes measurement information system (PROMIS). Medical Care,45(5), S22–S31. Scholar
  14. 14.
    Schalet, B. D., Revicki, D. A., Cook, K. F., Krishnan, E., Fries, J. F., & Cella, D. (2015). Establishing a common metric for physical function: Linking the HAQ-DI and SF-36 PF subscale to PROMIS physical function. Journal of General Internal Medicine,30(10), 1517–1523. Scholar
  15. 15.
    Riley, W., Rothrock, N., Bruce, B., Christodolou, C., Cook, K., Hahn, E., et al. (2010). Patient-reported outcomes measurement information system (PROMIS) domain names and definitions revisions: Further evaluation of content validity in IRT-derived item banks. Quality of Life Research,19(9), 1311–1321. Scholar
  16. 16.
    Hays, R. D., Spritzer, K. L., Schalet, B. D., & Cella, D. (2018). PROMIS-29 v2.0 profile physical and mental health summary scores. Quality of Life Research. Scholar
  17. 17.
    Cella, D., Lai, J.-S., Jensen, S. E., Christodoulou, C., Junghaenel, D. U., Reeve, B. B., et al. (2016). PROMIS® fatigue item bank has clinical validity across diverse chronic conditions. Journal of Clinical Epidemiology. Scholar
  18. 18.
    Liu, H., Cella, D., Gershon, R., Shen, J., Morales, L. S., Riley, W., et al. (2010). Representativeness of the patient-reported outcomes measurement information system internet panel. Journal of Clinical Epidemiology,63(11), 1169–1178. Scholar
  19. 19.
    Schalet, B. D., Cook, K. F., Choi, S. W., & Cella, D. (2014). Establishing a common metric for self-reported anxiety: Linking the MASQ, PANAS, and GAD-7 to PROMIS anxiety. Journal of Anxiety Disorders,28, 88–96. Scholar
  20. 20.
    DeWalt, D., Rothrock, N., Yount, S., & Stone, A. A. (2007). Evaluation of item candidates: The PROMIS qualitative item review. Medical Care,45(5 Suppl 1), S12–S21. Scholar
  21. 21.
    Garcia, S. F., Cella, D., Clauser, S. B., Flynn, K. E., Lad, T., Lai, J. S., et al. (2007). Standardizing patient-reported outcomes assessment in cancer clinical trials: A patient-reported outcomes measurement information system initiative,25(32), 5106–5112. Scholar
  22. 22.
    Schwarz, N., & Sudman, S. (2012). Autobiographical memory and the validity of retrospective reports. New York: Springer.Google Scholar
  23. 23.
    Bradburn, N. M., Rips, L. J., & Shevell, S. K. (1987). Answering autobiographical questions: The impact of memory and inference on surveys. Science,236(4798), 157–161. Scholar
  24. 24.
    Erskine, A., Morley, S., & Pain, S. P. (1990). Memory for pain: A review. Pain,41, 255–265. Scholar
  25. 25.
    Robinson, M. D., & Clore, G. L. (2002). Belief and feeling: Evidence for an accessibility model of emotional self-report. Psychological Bulletin,128(6), 934–960. Scholar
  26. 26.
    Gorin, A. A., & Stone, A. A. (2001). Recall biases and cognitive errors in retrospective self-reports: A call for momentary assessments. Handbook of Health Psychology,23, 405–413.Google Scholar
  27. 27.
    Redelmeier, D. A., & Pain, D. K. (1996). Patients’ memories of painful medical treatments: Real-time and retrospective evaluations of two minimally invasive procedures. Pain,66, 3–8. Scholar
  28. 28.
    Menon, G., & Yorkston, E. A. (1999). The use of memory and contextual cues in the formation of behavioral frequency judgments. In A. A. Stone, J. S. Turkkan, C. A. Bachrach, J. B. Jobe, H. S. Kurtzman, & V. S. Cain (Eds.), The science of self-report implications for research and practice (pp. 63–79). Mahwah: Lawrence Erlbaum Associates Publishers.Google Scholar
  29. 29.
    Rose, M., Bjorner, J. B., Becker, J., Fries, J. F., & Ware, J. E. (2008). Evaluation of a preliminary physical function item bank supported the expected advantages of the patient-reported outcomes measurement information system (PROMIS). Journal of Clinical Epidemiology,61(1), 17–33. Scholar
  30. 30.
    Rose, M., Bjorner, J. B., Gandek, B., Bruce, B., Fries, J. F., & Ware, J. E., Jr. (2014). The PROMIS physical function item bank was calibrated to a standardized metric and shown to improve measurement efficiency. Journal of Clinical Epidemiology,67(5), 516–526. Scholar
  31. 31.
    de Vries, S. T., Haaijer-Ruskamp, F. M., de Zeeuw, D., & Denig, P. (2014). The validity of a patient-reported adverse drug event questionnaire using different recall periods. Quality of Life Research,23(9), 2439–2445. Scholar
  32. 32.
    Broderick, J. E., Schwartz, J. E., Vikingstad, G., Pribbernow, M., Grossman, S., & Stone, A. A. (2008). The accuracy of pain and fatigue items across different reporting periods. Pain,139(1), 146–157. Scholar
  33. 33.
    Broderick, J. E., Schneider, S., Schwartz, J. E., & Stone, A. A. (2010). Interference with activities due to pain and fatigue: Accuracy of ratings across different reporting periods. Quality of Life Research,19(8), 1163–1170. Scholar
  34. 34.
    Stull, D. E., Leidy, N. K., Parasuraman, B., & Chassany, O. (2009). Optimal recall periods for patient-reported outcomes: Challenges and potential solutions. Current Medical Research and Opinion,25(4), 929–942. Scholar
  35. 35.
    Norquist, J. M., Girman, C., Fehnel, S., DeMuro-Mercon, C., & Santanello, N. (2012). Choice of recall period for patient-reported outcome (PRO) measures: Criteria for consideration. Quality of Life Research,21(6), 1013–1020. Scholar
  36. 36.
    Lai, J.-S., Cook, K., Stone, A., Beaumont, J., & Cella, D. (2009). Classical test theory and item response theory/Rasch model to assess differences between patient-reported fatigue using 7-day and 4-week recall periods. Journal of Clinical Epidemiology,62(9), 991–997. Scholar
  37. 37.
    Batterham, P. J., Sunderland, M., Carragher, N., & Calear, A. L. (2017). Psychometric properties of 7- and 30-day versions of the PROMIS emotional distress item banks in an Australian adult sample. Assessment. Scholar
  38. 38.
    Barta, W. D., Tennen, H., & Litt, M. D. (2012). Measurement reactivity in diary research. In M. R. Mehl & M. Connor (Eds.), Handbook of research methods for studying daily life (pp. 108–123). New York: Guilford Press.Google Scholar
  39. 39.
    Yost, K. J., Eton, D. T., Garcia, S. F., & Cella, D. (2011). Minimally important differences were estimated for six patient-reported outcomes measurement information system-cancer scales in advanced-stage cancer patients. Journal of Clinical Epidemiology,64(5), 507–516. Scholar
  40. 40.
    Northwestern University. (2018). HealthMeasures: Transforming how health is measured. Retrieved from Accessed 11 Jan 2018.
  41. 41.
    Hays, R., Bjorner, J., Revicki, D., Spritzer, K., & Cella, D. (2009). Development of physical and mental health summary scores from the patient-reported outcomes measurement information system (PROMIS) global items. European Journal of Cancer,18(7), 873–880. Scholar
  42. 42.
    Yanez, B., Pearman, T., Lis, C. G., Beaumont, J. L., & Cella, D. (2012). The FACT-G7: A rapid version of the functional assessment of cancer therapy-general (FACT-G) for monitoring symptoms and concerns in oncology practice and research. Annals of Oncology,24(4), 1073–1078. Scholar
  43. 43.
    Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin,107(2), 238–246. Scholar
  44. 44.
    Cook, K. F., Kallen, M. A., & Amtmann, D. (2009). Having a fit: Impact of number of items and distribution of data on traditional criteria for assessing IRT’s unidimensionality assumption. Quality of Life Research,18(4), 447–460. Scholar
  45. 45.
    Hatcher, L. (1994). A step-by-step approach to using SAS for factor analysis and structural equation modeling. Cary: SAS Institute Inc.Google Scholar
  46. 46.
    Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling a Multidisciplinary Journal,6(1), 1–55. Scholar
  47. 47.
    Kline, R. B. (2005). Principles and practice of structural equation modeling (2nd ed.). New York: Guilford Press.Google Scholar
  48. 48.
    McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah: Lawrence Erlbaum Associates Inc.Google Scholar
  49. 49.
    Reise, S. P., Morizot, J., & Hays, R. D. (2007). The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Quality of Life Research,16(Suppl 1), 19–31. Scholar
  50. 50.
    Reise, S. P., Moore, T. M., & Haviland, M. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment,92(6), 544–559. Scholar
  51. 51.
    Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment,95(2), 129–140. Scholar
  52. 52.
    Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods,21(2), 137–150. Scholar
  53. 53.
    Choi, S. W., Gibbons, L. E., & Crane, P. K. (2011). Lordif: An R package for detecting differential item functioning using iterative hybrid ordinal logistic regression/item response theory and monte carlo simulations. Journal of Statistical Software,39(8), 1–30.CrossRefGoogle Scholar
  54. 54.
    IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp. Released 2017.Google Scholar
  55. 55.
    Rosenthal, R., Rosnow, R. L., & Rubin, D. B. (2000). Contrasts and effect sizes in behavioral research: A correlational approach. Cambridge: Cambridge University Press.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • David M. Condon
    • 1
  • Robert Chapman
    • 1
  • Sara Shaunfield
    • 1
  • Michael A. Kallen
    • 1
  • Jennifer L. Beaumont
    • 1
    • 2
  • Daniel Eek
    • 3
  • Debanjali Mitra
    • 4
  • Katy L. Benjamin
    • 5
  • Kelly McQuarrie
    • 6
  • Jamae Liu
    • 7
  • James W. Shaw
    • 8
  • Allison Martin Nguyen
    • 9
  • Karen Keating
    • 10
  • David Cella
    • 1
    Email author
  1. 1.Department of Medical Social Sciences, Feinberg School of MedicineNorthwestern UniversityChicagoUSA
  2. 2.Terasaki Research InstituteLos AngelesUSA
  3. 3.AstraZenecaGothenburgSweden
  4. 4.Pfizer, Inc.New YorkUSA
  5. 5.Health Economics and Outcomes ResearchAbbvVie Inc.North ChicagoUSA
  6. 6.Janssen Global ServicesMalvernUSA
  7. 7.Health Economics and Outcomes Research, Novartis Pharmaceuticals CorporationEast HanoverUSA
  8. 8.Worldwide Health Economics and Outcomes ResearchBristol-Myers SquibbLawrencevilleUSA
  9. 9.Merck & Co., Inc.KenilworthUSA
  10. 10.Bayer HealthCare Pharmaceuticals, Inc.West HavenUSA

Personalised recommendations