Quality of Life Research

, Volume 28, Issue 1, pp 253–265 | Cite as

Development and content validity of a hemodialysis symptom patient-reported outcome measure

  • Jennifer E. FlytheEmail author
  • Adeline Dorough
  • Julia H. Narendra
  • Rebecca L. Wingard
  • Lorien S. Dalrymple
  • Darren A. DeWalt



To describe the process and preliminary qualitative development of a new symptom-based patient-reported outcome measure (PROM) intended to assess hemodialysis treatment-related physical symptoms.


Experienced interviewers conducted concept elicitation and cognitive debriefing interviews with individuals receiving in-center hemodialysis in the United States. Concept elicitation interviews involved eliciting spontaneous reports of symptom experiences and probing to further explore and confirm concepts. We used patient-reported concepts to generate a preliminary symptom PROM. We conducted 3 rounds of cognitive debriefing interviews to evaluate symptom relevance, item interpretability, and draft item structure. We iteratively refined the measure based on cognitive interview findings.


Forty-two adults receiving in-center hemodialysis participated in the concept elicitation interviews. A total of 12 symptoms were reported by > 10% of interviewees. We developed a 13-item initial draft instrument for testing in 3 rounds of cognitive interviews with an additional 52 hemodialysis patients. Participant responses and feedback during cognitive interviews led to changes in symptom descriptions, division of the single item “nausea/vomiting” into 2 distinct items, removal of daily activity interference items, addition of instructions, and clarification about the recall period, among other changes.


Symptom Monitoring on Renal Replacement Therapy-Hemodialysis (SMaRRT-HD™) is a 14-item PROM intended for use in hemodialysis patents. SMaRRT-HD™ uses a single treatment recall period and a 5-point Likert scale to assess symptom severity. Qualitative interview data provide evidence of its content validity. SMaRRT-HD™ is undergoing additional testing to assess measurement properties and inform measure scoring.


Symptoms Hemodialysis Content validity Concept elicitation Cognitive debriefing interview End-stage kidney disease Patient-reported outcome measure 



The authors would like to thank all study participants for sharing their experiences and perspectives on hemodialysis-related symptoms and the symptom measure. The authors also thank Dr. Emaad Abdel-Rahman and research liaisons, Lisa Johnson and Nino McHedlishvili, for their help in facilitating concept elicitation interviews at the University of Virginia. Finally, the authors thank the advisors who contributed to measure development: Derek Forfang, Lori Hartwell, Robert Kossmann, Mahesh Krishnan, Francesca Tentori, and David Thissen.


This work was supported by an unrestricted, investigator-initiated research Grant (A17-1082) from Renal Research Institute (RRI), a subsidiary of Fresenius Medical Care (FMC), North America. RRI played no role in study design; collection, analysis, and interpretation of data; writing the report; or the decision to submit the report for publication. Dr. Flythe is supported by National Institute of Diabetes and Digestive and Kidney Diseases of the National Institute of Health Grant K23 DK109401.

Compliance with ethical standards

Conflict of interest

Dr. Flythe has received investigator-initiated research funding from the Renal Research Institute, a subsidiary of Fresenius Medical Care, North America. In the past 2 years, Dr. Flythe has received consulting fees from Fresenius Medical Care, North America and speaking honoraria from American Renal Associates, American Society of Nephrology, Baxter, National Kidney Foundation, and multiple universities. Dr. Dalrymple and Ms Wingard are employees of Fresenius Medical Care, North America and have performance share-options/stock-options. Miss Dorough, Mrs. Narendra, and Dr. DeWalt declare that they have no conflicts of interest.

Ethics approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. This study was approved by the Institutional Review Boards of the University of North Carolina at Chapel Hill (17-0038 and 17-1252) and the University of Virginia (19928).

Informed consent

Informed consent was obtained from all individual participants included in the study.


  1. 1.
    Saran, R., Robinson, B., Abbott, K. C., Agodoa, L. Y., Albertus, P., Ayanian, J., et al. (2017) U.S. renal data system 2016 annual data report: Epidemiology of kidney disease in the United States. American Journal of Kidney Diseases, 69(Suppl 1), A7–A8.CrossRefGoogle Scholar
  2. 2.
    Abdel-Kader, K., Unruh, M. L., & Weisbord, S. D. (2009). Symptom burden, depression, and quality of life in chronic and end-stage kidney disease. Clinical Journal of the American Society of Nephrology, 4(6), 1057–1064.CrossRefGoogle Scholar
  3. 3.
    Merkus, M. P., Jager, K. J., Dekker, F. W., de Haan, R. J., Boeschoten, E. W., & Krediet, R. T. (1999). Physical symptoms and quality of life in patients on chronic dialysis: Results of The Netherlands Cooperative Study on Adequacy of Dialysis (NECOSAD). Nephrology Dialysis Transplantation, 14(5), 1163–1170.CrossRefGoogle Scholar
  4. 4.
    Weisbord, S. D., Fried, L. F., Arnold, R. M., Fine, M. J., Levenson, D. J., Peterson, R. A., et al. (2005). Prevalence, severity, and importance of physical and emotional symptoms in chronic hemodialysis patients. Journal of the American Society of Nephrology, 16(8), 2487–2494.CrossRefGoogle Scholar
  5. 5.
    Cox, K. J., Parshall, M. B., Hernandez, S. H., Parvez, S. Z., & Unruh, M. L. (2017). Symptoms among patients receiving in-center hemodialysis: A qualitative study. Hemodialysis International, 21(4), 524–533.CrossRefGoogle Scholar
  6. 6.
    Flythe, J. E., Dorough, A., Narendra, J. H., Forfang, D., Hartwell, L., & Abdel-Rahman, E. (2018) Perspectives on symptom experiences and symptom reporting among individuals on hemodialysis. Nephrology Dialysis Transplantation. (Epub ahead of print).Google Scholar
  7. 7.
    Evangelidis, N., Tong, A., Manns, B., Hemmelgarn, B., Wheeler, D. C., Tugwell, P., et al. (2017). Developing a set of core outcomes for trials in hemodialysis: An International Delphi survey. American Journal of Kidney Diseases, 70(4), 464–475.CrossRefGoogle Scholar
  8. 8.
    Manns, B., Hemmelgarn, B., Lillie, E., Dip, S. C., Cyr, A., Gladish, M., et al. (2014). Setting research priorities for patients on or nearing dialysis. Clinical Journal of the American Society of Nephrology, 9(10), 1813–1821.CrossRefGoogle Scholar
  9. 9.
    Weisbord, S. D., Fried, L. F., Mor, M. K., Resnick, A. L., Unruh, M. L., Palevsky, P. M., et al. (2007). Renal provider recognition of symptoms in patients on maintenance hemodialysis. Clinical Journal of the American Society of Nephrology, 2(5), 960–967.CrossRefGoogle Scholar
  10. 10.
    Basch, E., Deal, A. M., Kris, M. G., Scher, H. I., Hudis, C. A., Sabbatini, P., et al. (2016). Symptom monitoring with patient-reported outcomes during routine cancer treatment: A randomized controlled trial. Journal of Clinical Oncology, 34(6), 557–565.CrossRefGoogle Scholar
  11. 11.
    Peipert, J. D., Bentler, P. M., Klicko, K., & Hays, R. D. (2018). Psychometric properties of the kidney disease quality of life 36-item short-form survey (KDQOL-36) in the United States. American Journal of Kidney Diseases, 71(4), 461–468.CrossRefGoogle Scholar
  12. 12.
    Patrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E., et al. (2011). Content validity-establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 1—eliciting concepts for a new PRO instrument. Value Health, 14(8), 967–977.CrossRefGoogle Scholar
  13. 13.
    Patrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E., et al. (2011). Content validity-establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: Part 2—assessing respondent understanding. Value Health, 14(8), 978–988.CrossRefGoogle Scholar
  14. 14.
    U.S. Department of Health and Human Services FaDA. (2009). Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER), Center for Devices and Radiological Health (CDRH). Guidance for Industry Patient Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Silver Spring: U.S. Food and Drug Administration.Google Scholar
  15. 15.
    Flythe, J. E., Powell, J. D., Poulton, C. J., Westreich, K. D., Handler, L., Reeve, B. B., et al. (2015). Patient-reported outcome instruments for physical symptoms among patients receiving maintenance dialysis: A systematic review. American Journal of Kidney Diseases, 66(6), 1033–1046.CrossRefGoogle Scholar
  16. 16.
    Sandelowski, M. (2000). Combining qualitative and quantitative sampling, data collection, and analysis techniques in mixed-method studies. Research in Nursing & Health, 23(3), 246–255.CrossRefGoogle Scholar
  17. 17.
    Patton, M. (2014). Qualitative research and evaluation methods (4th ed.). Thousand Oaks: Sage.Google Scholar
  18. 18.
    Paige, S. R., Krieger, J. L., Stellefson, M., & Alber, J. M. (2016). eHealth literacy in chronic disease patients: An item response theory analysis of the eHealth literacy scale (eHEALS). Patient Education and Counseling, 100(2), 320–326.CrossRefGoogle Scholar
  19. 19.
    Norman, C. D., & Skinner, H. A. (2006). eHEALS: The eHealth literacy scale. Journal of Medical Internet Research, 8(4), e27.CrossRefGoogle Scholar
  20. 20.
    Norman, C. D., & Skinner, H. A. (2006). eHealth literacy: Essential skills for consumer health in a Networked world. Journal of Medical Internet Research, 8(2), e9.CrossRefGoogle Scholar
  21. 21.
    Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87, 215–251.CrossRefGoogle Scholar
  22. 22.
    Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks: Sage.CrossRefGoogle Scholar
  23. 23.
    Morris, N. S., MacLean, C. D., Chew, L. D., & Littenberg, B. (2006). The single item literacy screener: Evaluation of a brief instrument to identify limited reading ability. BMC Family Practice, 7, 21.CrossRefGoogle Scholar
  24. 24.
    Borson, S., Scanlan, J. M., Chen, P., & Ganguli, M. (2003). The Mini-Cog as a screen for dementia: Validation in a population-based sample. Journal of the American Geriatrics Society, 51(10), 1451–1454.CrossRefGoogle Scholar
  25. 25.
    Lindsay, R. M., Heidenheim, P. A., Nesrallah, G., Garg, A. X., Suri, R., & Daily Hemodialysis Study Group London Health Sciences Centre (2006). Minutes to recovery after a hemodialysis session: A simple health-related quality of life question that is reliable, valid, and sensitive to change. Clinical Journal of the American Society of Nephrology, 1(5), 952–959.CrossRefGoogle Scholar
  26. 26.
    Davison, S. N., Jhangri, G. S., & Johnson, J. A. (2006). Longitudinal validation of a modified Edmonton symptom assessment system (ESAS) in haemodialysis patients. Nephrology Dialysis Transplantation, 21(11), 3189–3195.CrossRefGoogle Scholar
  27. 27.
    Virga, G., Mastrosimone, S., Amici, G., Munaretto, G., Gastaldon, F., et al. (1998). Symptoms in hemodialysis patients and their relationship with biochemical and demographic parameters. The International Journal of Artificial Organs, 21(12), 788–793.CrossRefGoogle Scholar
  28. 28.
    Scherer, J. S., Combs, S. A., & Brennan, F. (2017). Sleep disorders, restless legs syndrome, and uremic pruritus: Diagnosis and treatment of common symptoms in dialysis patients. American Journal of Kidney Diseases, 69(1), 117–128.CrossRefGoogle Scholar
  29. 29.
    Hays, R. D., Kallich, J. D., Mapes, D. L., Coons, S. J., & Carter, W. B. (1994). Development of the kidney disease quality of life (KDQOL) instrument. Quality of Life Research, 3(5), 329–338.CrossRefGoogle Scholar
  30. 30.
    Rao, S., Carter, W. B., Mapes, D. L., Kallich, J. D., Kamberg, C. J., Spritzer, K. L., et al. (2000). Development of subscales from the symptoms/problems and effects of kidney disease scales of the kidney disease quality of life instrument. Clinical Therapeutics, 22(9), 1099–1111.CrossRefGoogle Scholar
  31. 31.
    Weisbord, S. D., Fried, L. F., Arnold, R. M., Rotondi, A. J., Fine, M. J., Levenson, D. J., et al. (2004). Development of a symptom assessment instrument for chronic hemodialysis patients: The Dialysis Symptom Index. Journal of Pain and Symptom Management, 27(3), 226–240.CrossRefGoogle Scholar
  32. 32.
    Davison, S. N., Jhangri, G. S., & Johnson, J. A. (2006). Cross-sectional validity of a modified Edmonton symptom assessment system in dialysis patients: A simple assessment of symptom burden. Kidney International, 69(9), 1621–1625.CrossRefGoogle Scholar
  33. 33.
    Flythe, J. E., Hilliard, T., Castillo, G., Ikeler, K., Orazi, J., Abdel-Rahman, E., et al. (2018). Symptom prioritization among adults receiving in-center hemodialysis: A mixed methods study. Clinical Journal of the American Society of Nephrology, 13(5), 735–745.CrossRefGoogle Scholar
  34. 34.
    Tong, A., Manns, B., Hemmelgarn, B., Wheeler, D. C., Evangelidis, N., Tugwell, P., et al. (2017). Establishing core outcome domains in hemodialysis: Report of the standardized outcomes in nephrology-hemodialysis (SONG-HD) consensus workshop. American Journal of Kidney Diseases, 69(1), 97–107.CrossRefGoogle Scholar
  35. 35.
    Jhamb, M., Liang, K., Yabes, J., Steel, J. L., Dew, M. A., Shah, N., et al. (2013). Prevalence and correlates of fatigue in chronic kidney disease and end-stage renal disease: Are sleep disorders a key to understanding fatigue? The American Journal of Nephrology, 38(6), 489–495.CrossRefGoogle Scholar
  36. 36.
    Ju, A., Unruh, M. L., Davison, S. N., Dapueto, J., Dew, M. A., Fluck, R., et al. (2018). Patient-reported outcome measures for fatigue in patients on hemodialysis: A systematic review. American Journal of Kidney Diseases, 71(3), 327–343.CrossRefGoogle Scholar
  37. 37.
    Gedney, J. J., Logan, H., & Baron, R. S. (2003). Predictors of short-term and long-term memory of sensory and affective dimensions of pain. The Journal of Pain, 4(2), 47–55.CrossRefGoogle Scholar
  38. 38.
    Brodie, E. E., & Niven, C. A. (2000). Remembering an everyday pain: The role of knowledge and experience in the recall of the quality of dysmenorrhoea. Pain, 84(1), 89–94.CrossRefGoogle Scholar
  39. 39.
    Lavin, R., & Park, J. (2014). A characterization of pain in racially and ethnically diverse older adults: A review of the literature. Journal of Applied Gerontology, 33(3), 258–290.CrossRefGoogle Scholar
  40. 40.
    Bagayogo, I. P., Interian, A., & Escobar, J. I. (2013). Transcultural aspects of somatic symptoms in the context of depressive disorders. Advances in Psychosomatic Medicine, 33, 64–74.CrossRefGoogle Scholar
  41. 41.
    Koffman, J., Morgan, M., Edmonds, P., Speck, P., & Higginson, I. (2008). Cultural meanings of pain: A qualitative study of Black Caribbean and White British patients with advanced cancer. Palliative Medicine, 22(4), 350–359.CrossRefGoogle Scholar
  42. 42.
    Hoeppner, B. B., Kelly, J. F., Urbanoski, K. A., & Slaymaker, V. (2011). Comparative utility of a single-item versus multiple-item measure of self-efficacy in predicting relapse among young adults. The Journal of Substance Abuse Treatment, 41(3), 305–312.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Division of Nephrology and Hypertension, Department of MedicineUniversity of North Carolina (UNC) Kidney Center, UNC School of MedicineChapel HillUSA
  2. 2.Cecil G. Sheps Center for Health Services ResearchUniversity of North CarolinaChapel HillUSA
  3. 3.Medical OfficeFresenius Medical Care North AmericaWalthamUSA
  4. 4.Division of General Medicine and Clinical Epidemiology, Department of MedicineUniversity of North Carolina, UNC School of MedicineChapel HillUSA

Personalised recommendations