The quality of patient-reported outcome (PRO) data can be compromised by non-response (NR) to scheduled questionnaires, particularly if reasons for NR are related to health problems, which may lead to unintended bias. The aim was to investigate whether electronic reminders and real-time monitoring improve PRO completion rate.
The population-based study “Quality of life in Danish multiple myeloma patients” is a longitudinal, multicentre study with consecutive inclusion of treatment-demanding newly diagnosed or relapsed patients with multiple myeloma. Education of study nurses in the avoidance of NR, electronic reminders, 7-day response windows and real-time monitoring of NR were integrated in the study. Patients complete PRO assessments at study entry and at 12 follow-up time points using electronic or paper questionnaires. The effect of the electronic reminders and real-time monitoring were investigated by comparison of proportions of completed questionnaires before and after each intervention.
The first 271 included patients were analysed; of those, 249 (85%) chose electronic questionnaires. Eighty-four percent of the 1441 scheduled PRO assessments were completed within the 7-day response window and 11% after real-time monitoring, achieving a final PRO completion rate of 95%. A significant higher proportion of uncompleted questionnaires were completed after the patients had received the electronic reminder and after real-time monitoring.
Electronic reminders and real-time monitoring contributed to a very high completion rate in the study. To increase the quality of PRO data, we propose integrating these strategies in PRO studies, however highlighting that an increase in staff resources is required for implementation.
Multiple myeloma (MM) is an incurable malignancy of plasma cells in the bone marrow. MM is associated with severe morbidity, specifically caused by bone destruction and pathological bone fractures, renal dysfunction, high infection rate and potential physical disability [1, 2]. The prognosis of MM has improved markedly over the past 20 years, and the median survival of MM patients under the age of 70 has increased from 3 years  to 6–7 years [4, 5]. The improved prognosis is mediated by the introduction of high dose chemotherapy with autologous stem cell support (HDT) in the 1990s, new treatment options with immunomodulatory drugs (IMiDs), such as thalidomide, lenalidomide and pomalidomide [6,7,8], and the proteasome inhibitors, bortezomib, carfilzomib and ixazomib [9,10,11]. Most recently, the monoclonal antibodies elotuzumab and daratumumab [12, 13] have been introduced, and therefore the prognosis is expected to improve even further in coming years .
Treatment choice in MM depends on the patients’ age, disease complications, existing comorbidity and whether the patient is judged fit for specific regimens, such as HDT. Treatment usually involves repeated cycles of a 2–3 drug combination therapy with a proteasome inhibitor, IMiD, cytostatic agent and/or steroid. Treatment implies a risk of both acute adverse events, such as infections, as well as late effects, such as peripheral neuropathy and fatigue [15,16,17,18].
Patients with MM report a high symptom burden. Common symptoms include fatigue, pain, constipation, insomnia and tingling in the hands/feet, with a consequent decrease in physical and cognitive functioning [19,20,21]. Compared to patients with other haematological malignancies, patients with MM report low health-related quality of life (HRQoL) [21, 22]. Longitudinal HRQoL studies of patients with MM suggest that clinically beneficial improvements in HRQoL are more likely during primary treatments than during treatment for relapse .
The patients’ experience of symptoms and HRQoL can be validly and reliably captured with patient-reported outcome (PRO) questionnaires . Integration of PROs into routine care management of patients with advanced cancer is associated with increased overall survival and reduction of the symptom burden [25, 26]. Typically, PRO assessments are scheduled at key time points and when a questionnaire is not completed at a scheduled time, it is referred to as a non-response (NR). Consequently, missing data can lead to a variety of problems, more so as NR rates increase, including loss of study power and precision [27, 28]. Further, there is risk of bias if the reason for NR is related to the patient’s health status and appropriate analysis methods are not applied. For example, if only complete case analysis methods are used, there is a risk of overestimated HRQoL and underestimated toxicity [27,28,29]. This is due to the analysis being based on available data from patients with completed PRO assessments and who presumably have better HRQoL, while patients who drop out might have more toxicity and a worse HRQoL [27, 30,31,32,33,34]. Thus, NR represents a threat to internal and external validity and is one of the inherent barriers in establishing high quality PRO data for use in patient-centred care [35,36,37]. Several strategies designed to minimize NR have been proposed, and these can be integrated into the study design, protocol and implementation procedures for the PRO study . These include ensuring that staff are aware of the importance of reducing NR and have access to written study procedure guidelines and support [29, 31]. Also, given the time-sensitive nature of PRO data, real-time monitoring of PRO completion rates during the study is recommended . However, we are not aware of any studies that have assessed and documented the effectiveness of such strategies in reducing NR.
The study “Quality of life in Danish multiple myeloma patients” (QoL-MM) is a national multicentre, prospective, observational and primarily web-based survey with real-time monitoring of NRs. Several strategies to reduce NR have been implemented including education of study nurses, electronic reminders and real-time monitoring of PRO completion. The aim of this analysis was to investigate whether electronic reminders and real-time monitoring of NRs improve PRO completion rate.
The QoL-MM study includes newly diagnosed or relapsed, treatment-demanding patients with MM who, according to International Myeloma Working Group (IMWG) criteria, are eligible for inclusion . Broad inclusion criteria ensure inclusion of a population-based, representative cohort of MM patients. Only, patients who are not able to understand the Danish language or who are diagnosed with a psychiatric condition are ineligible. Study sites include all 10 Danish departments of haematology. The goal is to recruit 800 patients, and each patient is followed for 24 months or until early drop-out due to withdrawal of consent, death or permanent lack of ability to fill out the questionnaires. The patients are introduced to the study by their treating physician or nurse, and written informed consent is obtained prior to inclusion. Demographic data are collected as part of the inclusion interview by a local study nurse. Moreover, the patients provide information related to activity of daily living (ADL), instrumental ADL and self-reported diseases, summarized into the Charlson Comorbidity Index [39,40,41]. This information is used to calculate the Freiburger Comorbidity Index of 0-3 and the IMWG myeloma frailty score which divides the patients into three categories of “Frail”, “Intermediate Fitness” or “Fit” [42, 43]. The patient’s Karnofsky Performance status was assessed by the local study nurse . Clinical data, e.g. date of diagnosis, MM subtype and the prognostic score, International Staging System, are collected from The Danish Multiple Myeloma Registry [45, 46]. Data on admissions, discharges and other hospital procedures are captured from The National Registry of Patients .
PRO study design
Patients complete questionnaires at study entry and at 12 follow-up time points during a 24 month period. The patient left the study before 24 months in case of death, withdrawn consent or entering a state with permanently inability to complete a questionnaire. The follow-up target dates for completion of questionnaires are every 4 weeks for the first 6 months and thereafter every 3 months until 24 months. Depending on the PRO assessment time point, the patient completes between two and four PRO instruments, equivalent to 50–85 items. Each set of questionnaires is sequenced to begin with the cancer specific instrument European Organisation for Research and Treatment of Cancer Quality of life QLQ-C30 (EORTC QLQ-C30)  followed by the Multiple Myeloma module QLQ-MY20 (EORTC QLQ-MY20) , the Chemotherapy-Induced Peripheral Neuropathy module (EORTC QLQ-CIPN20)  and the Short-form health survey version 2 (SF12v2) .
PRO data collection procedures
Participants are asked to complete the entire set of questionnaires at one time, preferably on the target date and no later than 7 days hereafter, which is the 7-day response window for all follow-up questionnaires. The patients are furthermore encouraged to use the web-based questionnaire method, where a link is sent to the patients’ email on the target date. However, patients can choose the paper-and-pencil method, if preferred. A REDCap database automatically sends the emails on the target date and email reminders .
At baseline, the patient may complete the questionnaires alone or with assistance from the study nurse using a tablet or paper. The baseline questionnaires can also be completed at home by computer or tablet; however the study nurse must ensure that the questionnaires have been completed no later than on the day of anti-myeloma treatment. The patient is excluded as a screening failure if there is an uncompleted baseline questionnaire or missed completion of one or more of the four baseline PRO instruments by 3 days after start of anti-myeloma treatment.
Patients who choose to use paper questionnaires receive three sets of questionnaires at the inclusion interview to complete at four, eight and 12 weeks follow-up. The target date of each set of questionnaires is written on the cover page with the contact information of the local study nurse. The patients are asked to bring the completed questionnaires to the outpatient clinic at a scheduled appointment. To remind the patient to complete the first follow-up paper questionnaire, the local study nurse contacts the patient at 4 weeks. After week 12, it is the responsibility of the local study nurses to provide the next three sets of questionnaires to the patient for completion at 16 and 20 weeks and 6 months. In this way, the local study nurse provides paper questionnaires to the patients who chose to complete the follow-up questionnaires in paper, three times during the 24 months. Completed paper questionnaires are uploaded to the REDCap database by the local study nurse and centrally entered continuously during the study period by the staff at the study office.
Strategies to minimize intermittent non-responses
All local study nurses are informed about the importance of minimizing missing items and NR to a questionnaire or set of questionnaires and are trained in appropriate procedures to minimize missing data. Local study nurses are permitted to provide support to frail patients or patients who temporarily lack the ability to independently complete questionnaires. In such cases, the local study nurse reads the items and response categories aloud and marks the patient’s answers on his or her behalf. The aim is for this to be done within the 7-day response window after the target date. If the patient does not have an appointment in the outpatient clinic within this time frame, but has a scheduled appointment a few days before the target date, the study nurse provides the patient with the questionnaire at that appointment. Otherwise, the questionnaire is completed after the 7-day response window.
For patients using the web-based completion method who have not completed the electronic questionnaire at day 4, a reminder is automatically sent to the patients email. If a patient has still not answered the questionnaire on day 7 after the target date, the local study nurse is notified by the central study office during week days, as part of real-time monitoring of NR. In this situation, the local study nurse has 2 week days to contact the patient, ascertain and document the reason for NR and to invite the patient to complete the questionnaire. The study nurses have access to a written guideline of all project related tasks, and in case there is need for further clarification of a project procedure, the study office can be contacted by telephone or email during week days.
Real-time monitoring of PRO completion or NR for both web-based and paper-based PRO questionnaire completion is carried out by the study office. If a patient has completed some or all items of the EORTC QLQ-C30, which is the first questionnaire at every scheduled PRO assessment time point, the follow-up PRO assessment is defined as completed. Missing items and partly completed questionnaires or sets of questionnaires are not part of the real-time monitoring.
The participating departments are financially compensated for managing a NR, providing guidance to a patient to complete a questionnaire and when three completed paper questionnaires are either provided or collected.
Information to the participants
As part of the inclusion interview, all patients are informed about the importance of completing the follow-up questionnaires within the 7-day response window, and that the study nurse will contact them if they have not completed a questionnaire by the seventh day. A part of the information provided to the patients choosing the electronic platform is that they will receive an email reminder, if they have not completed a scheduled questionnaire within 4 days. The patients are informed about the reason why they will be contacted in case of an NR within the 7-day response window, including that this could be due to a decline in the patients’ HRQoL. Patients are informed that participation and completing questionnaires in the study are voluntary and that choosing to skip a scheduled questionnaire will not have any consequences for the patient. All patients receive the study nurses’ contact information and are encouraged to seek support in case of questions, technical challenges or deciding to change the method of completion.
Patient cohort and data analysis
The patient cohort for this paper include all patients who consented to QoL-MM by 16th August 2018 and who reached at least the first follow-up PRO assessment time point at week 4. Questionnaires which are completed before or within the 7-day response window are defined as “on-time responses”. In case the patient complete the questionnaire at day 7 after the target date or later, the response is defined as “salvage response”, and the remainder were categorized as a “never response”. The PRO completion rate was calculated as the number of completed on-time and salvage responses as a proportion of the number of scheduled PRO assessments expected to be completed . Also, we calculated the rate of on-time and salvage responses. The effect of electronic reminders was estimated by comparing the number of completed questionnaires completed by patients who had chosen to complete the follow-up questionnaires electronically at day four compared to day three in relation to the number of questionnaires still not completed. The same was carried out to estimate the effect of real-time monitoring, where the number of electronic completed questionnaires at day 6 and day 7 was compared. Chi square test was used as statistical analysis method for the effect analyses. We investigated seven baseline patient characteristics as predictors for on-time response and questionnaire completion (age, gender, Karnofsky Performance Status Scale, IMWG frailty score, CCI, relapsed disease and completion of the questionnaires by paper). The analyses were performed by mixed effects logistic regression with a random effect to take account non-independence of observations from the same patient. P-values < 0.05 were considered statistically significant. Data are presented by descriptive analyses using Stata version 15.
As of August 16th 2018, 481 patients were found eligible for the QoL-MM study, and hereof 292 provided written consent for participation and inclusion in the study. Of the 292 patients included, 271 had reached at least the first follow-up PRO assessment time point at 4 weeks and were included in the analyses. Patient and disease characteristics are presented in Table 1. Of all patients, 55% had newly diagnosed symptomatic MM of which 52% started an induction regimen with planned HDT at study entry. A daratumumab containing regimen was the most frequently used regimen in treatment of relapse MM and 24% of the patients with relapse started fifth or later line of therapy. Twenty-four percent of both newly diagnosed and relapse MM were 76 years or older, and 17% of the NDMM patients and 12% of the RMM were characterized as “Frail” according to the IMWG myeloma frailty score.
Electronic completion of follow-up questionnaires was chosen by 85% of the 271 patients, and 15% chose the paper-and-pencil method. Three patients changed mode of answering during follow-up, two from electronic method to paper method, as the electronic method was found to be too complicated.
PRO completion rate
Per protocol, for the study cohort (n = 271), 1441 scheduled follow-up questionnaires were expected to be completed at the time of analysis. The number of patients and completed questionnaires (on-time and salvage) and never-responses at each follow-up time point are presented in Fig. 1. The reasons for reduction in number of patients in the flow diagram are due to early drop-out or end of follow-up. The largest proportion of never-responses with the first year of follow-up was at 4 weeks 7% (19 questionnaires out of 271 expected).
Of the 1441 scheduled questionnaires, 1214 (84%) were completed on-time. Of the 227 questionnaires that were not completed on-time, 153 (67%) were salvaged responses and 74 (33%) were never completed. When adding the salvage responses to the on-time responses, a total 1367 of the scheduled questionnaires were completed, equivalent to a PRO completion rate of 95%.
Pattern of response and effect of electronic reminder and real-time monitoring
Questionnaire completion patterns are presented in Fig. 2; Table 2. Of the 1367 scheduled questionnaires, 553 (40%) were answered on the target date, 417 (31%) were completed on days one to three. For estimation of the effect of the reminder, at day 4, 100 out of 392 not completed electronic questionnaires were completed, which is 25.5% (95% CI 21.3%; 30.1%). At day 3, 75 out of 467 not completed questionnaires were completed, which is 16.1% (95% CI 12.8; 19.7). A significantly higher proportion of uncompleted questionnaires were completed after the patients had received the electronic reminder (p < 0.001). For estimation of the effect of real-time monitoring, at day 7, 58 out of 189 not completed questionnaires were completed (30.7%, 95% CI 24.2%; 37.8%). This is a significantly higher proportion (p < 0.001) than at day 6, where 35 out of 223 not completed questionnaires were completed (15.7%, 95% CI 11.2%; 21.1%).
Predictors for non-completion
The strongest predictors for non-completion of scheduled questionnaires were IMWG frailty score (Intermediate fitness; p-value 0.009, Frail; p-value 0.001) and choosing paper questionnaires (p-value 0.005). A weak association for non-completion was found for CCI 1 (p-value 0.042) and a weak association for completion was found for male gender (p-value 0.044). When we made the analysis for on-time completion only, the association between paper completion method and non-completion was persistent (p-value 0.047) and Karnofsky Performance Status Scale ≤ 70% (p-value 0.025) appeared as a predictor for non-completion, too. The results of the mixed effects logistic regression analysis are presented in Table 3. We repeated the analysis but adjusted for time point and found similar results (data not shown).
The aim of the analysis was to investigate whether the strategies used to minimize NRs to scheduled questionnaires could increase PRO completion rate in a longitudinal study of MM patients receiving anti-myeloma treatment. Our initiatives included education of study nurses, electronic reminders and real-time monitoring of NR. A significant higher proportion of uncompleted questionnaires were completed after the patients had received the electronic reminder and after real-time monitoring. Those strategies resulted in a very high PRO completion rate of 95%, with just 5% non-response.
PRO completion rates in longitudinal studies of patients with MM are often not reported, and of the studies, where this information was reported, the PRO completion rate was 78–98% . Only one study has reported a higher PRO completion rate compared to our findings. This was the NMSG 4/90 study by Gulbrandsen et al. , where the PRO completion rate of the historical control group of newly diagnosed patients with MM was 98%. This historical control group originates from the EORTC QLQ-C30 validation study of patients with MM . One of the aims in the validation study was to evaluate the applicability of the questionnaire in a cohort of patients with MM and included sampling of data concerning the patient’s need for assistance in completing the questionnaires. The authors found that up to 30% of the MM patients reported need of assistance in completing the questionnaire. Other strategies of how this high PRO completion rate was achieved are not described in the paper .
When the QoL-MM study was designed, there had been particular focus on how we could minimize NR. We introduced real-time monitoring of NR and provided the patients with reminders. Staff resources were dedicated for this purpose, software as well as a high proportion of the patients choosing the web-based questionnaire method made it possible, and we succeeded in reaching a high PRO completion rate. Still, some NR could not be avoided. We found that low performance status at the time of inclusion and choosing paper-and-pencil method were predictors for not completing the questionnaires on-time. Risk of never response was higher for females, un-fit patients at baseline and choosing paper method for questionnaire completion. Special attention and guidance should be provided from the study staff to avoid NR from those patients. Information about the clinical status of patients when they fail to complete a scheduled questionnaire and the reason for NR might assist the PRO researchers in making the correct assumption for the underlying missing data mechanisms [27, 28, 56]. The link between the documented reasons, the predictors for NR and the missing data mechanisms as well as estimation of the impact of NR and patient drop-out on HRQoL results will be investigated in future analyses of the QoL-MM study.
The overall aim of the QoL-MM study is to describe the quality of life of the general population of patients with MM from diagnosis to late, advanced disease throughout different kinds of anti-myeloma therapies . Methodological considerations concerning PRO assessment time points were included as part of the study planning. Clinical visits are frequently chosen time points for PRO assessment in clinical trials of MM patients . This decision has the advantage of reducing the risk of NR, since patients have the opportunity to complete the questionnaires at the hospital with assistance from the study nurse. Disadvantages in using day 1 of treatment cycles include a potential risk of underestimation of toxicities that occur after day 1 and not capturing periods with temporary decline in HRQoL, resulting in rescheduling of chemotherapy . We chose to collect the PRO data in QoL-MM at predefined calendar time points to meet the overall study aim and thereby capture HRQoL at regular non-clinic time points throughout the MM patients´ diverse disease trajectories. This decision could have made the study vulnerable for low PRO completion rates, since the general population of patient with MM can be frail and are at risk of adverse events, hospital admissions, as well as risk of physical and mental disabilities caused by the disease and therapy. Therefore, we implemented the educational and procedural strategies to reduce NRs. The same strategies might be useful to ensure high PRO completion rate in routine assessment and clinical utilization of PRO measures in the broad group of cancer patients .
Another strategie we chose was to use a pre-planned time frame of 7 days for each scheduled questionnaire. This allowed the study nurses to clearly communicate the expectations to the patients participating in the study and systematically capture reasons for NR from every patient who failed to complete the questionnaire within the 7-day response window [60, 61]. Whether patients who were not able to complete the questionnaires within the expected time frame have a poorer HRQoL will be investigated as part of the QoL-MM study.
We evaluated strategies to maximize PRO completion in a longitudinal cohort study of patients with MM receiving anti-myeloma treatment. Real-time monitoring of NR, electronic reminders and education of study nurses are effective strategies that resulted in a questionnaire completion rate of 95%. To our knowledge, the QoL-MM study is the first study to provide insight into how to ensure high PRO completion rates in a cohort of cancer patients receiving chemotherapy. We propose our applied strategies as a model for improving PRO completion rates in clinical trials, registries and routine care management to increase the quality and value of PRO data in patient-centred care.
Kyle, R. A., & Rajkumar, S. V. (2008). Multiple myeloma. Blood,111(6), 2962–2972. https://doi.org/10.1182/blood-2007-10-078022.
Rajkumar, S. V., Dimopoulos, M. A., Palumbo, A., Blade, J., Merlini, G., Mateos, M. V., et al. (2014). International Myeloma Working Group updated criteria for the diagnosis of multiple myeloma. The lancet Oncology,15(12), e538–e548. https://doi.org/10.1016/s1470-2045(14)70442-5.
Kumar, S. K., Rajkumar, S. V., Dispenzieri, A., Lacy, M. Q., Hayman, S. R., Buadi, F. K., et al. (2008). Improved survival in multiple myeloma and the impact of novel therapies. Blood,111(5), 2516–2520.
Kumar, S. K., Dispenzieri, A., Lacy, M. Q., Gertz, M. A., Buadi, F. K., Pandey, S., et al. (2014). Continued improvement in survival in multiple myeloma: Changes in early mortality and outcomes in older patients. Leukemia,28(5), 1122–1128. https://doi.org/10.1038/leu.2013.313.
Blade, J., & Rosinol, L. (2009). Changing paradigms in the treatment of multiple myeloma. Haematologica,94(2), 163–166. https://doi.org/10.3324/haematol.2008.002766.
Waage, A., Gimsing, P., Juliusson, G., Turesson, I., Gulbrandsen, N., Eriksson, T., et al. (2004). Early response predicts thalidomide efficiency in patients with advanced multiple myeloma. British Journal of Haematology,125(2), 149–155. https://doi.org/10.1111/j.1365-2141.2004.04879.x.
Miguel, J. S., Weisel, K., Moreau, P., Lacy, M., Song, K., Delforge, M., et al. (2013). Pomalidomide plus low-dose dexamethasone versus high-dose dexamethasone alone for patients with relapsed and refractory multiple myeloma (MM-003): A randomised, open-label, phase 3 trial. The lancet Oncology,14(11), 1055–1066. https://doi.org/10.1016/s1470-2045(13)70380-2.
Weber, D. M., Chen, C., Niesvizky, R., Wang, M., Belch, A., Stadtmauer, E. A., et al. (2007). Lenalidomide plus dexamethasone for relapsed multiple myeloma in North America. New England Journal of Medicine,357(21), 2133–2142. https://doi.org/10.1056/NEJMoa070596.
Moreau, P., Masszi, T., Grzasko, N., Bahlis, N. J., Hansson, M., Pour, L., et al. (2016). Oral Ixazomib, lenalidomide, and dexamethasone for multiple myeloma. The New England Journal of Medicine,374(17), 1621–1634. https://doi.org/10.1056/NEJMoa1516282.
Stewart, A. K., Rajkumar, S. V., Dimopoulos, M. A., Masszi, T., Spicka, I., Oriol, A., et al. (2015). Carfilzomib, lenalidomide, and dexamethasone for relapsed multiple myeloma. The New England Journal of Medicine,372(2), 142–152. https://doi.org/10.1056/nejmoa1411321.
Richardson, P. G., Barlogie, B., Berenson, J., Singhal, S., Jagannath, S., Irwin, D., et al. (2003). A phase 2 study of Bortezomib in relapsed, refractory myeloma. New England Journal of Medicine,348(26), 2609–2617. https://doi.org/10.1056/NEJMoa030288.
Palumbo, A., Chanan-Khan, A., Weisel, K., Nooka, A. K., Masszi, T., Beksac, M., et al. (2016). Daratumumab, bortezomib, and dexamethasone for multiple myeloma. New England Journal of Medicine,375(8), 754–766. https://doi.org/10.1056/NEJMoa1606038.
Lonial, S., Dimopoulos, M., Palumbo, A., White, D., Grosicki, S., Spicka, I., et al. (2015). Elotuzumab therapy for relapsed or refractory multiple myeloma. New England Journal of Medicine,373(7), 621–631. https://doi.org/10.1056/NEJMoa1505654.
Ocio, E. M., Richardson, P. G., Rajkumar, S. V., Palumbo, A., Mateos, M. V., Orlowski, R., et al. (2014). New drugs and novel mechanisms of action in multiple myeloma in 2013: A report from the International Myeloma Working Group (IMWG). Leukemia,28(3), 525–542. https://doi.org/10.1038/leu.2013.350.
Boland, E., Eiser, C., Ezaydi, Y., Greenfield, D. M., Ahmedzai, S. H., & Snowden, J. A. (2013). Living with advanced but stable multiple myeloma: A study of the symptom burden and cumulative effects of disease and intensive (hematopoietic stem cell transplant-based) treatment on health-related quality of life. Journal of Pain and Symptom Management,46(5), 671–680. https://doi.org/10.1016/j.jpainsymman.2012.11.003.
Holmstrom, M. O., Gimsing, P., Abildgaard, N., Andersen, N. F., Helleberg, C., Clausen, N. A., et al. (2015). Causes of early death in multiple myeloma patients who are ineligible for high-dose therapy with hematopoietic stem cell support: A study based on the nationwide Danish Myeloma Database. American Journal of Hematology,90(4), E73–E74. https://doi.org/10.1002/ajh.23932.
Mateos, M. V. (2010). Management of treatment-related adverse events in patients with multiple myeloma. Cancer Treatment Reviews,36(SUPPL. 2), S24–S32. https://doi.org/10.1016/S0305-7372%2810%2970009-8.
Richardson, P. G., Delforge, M., Beksac, M., Wen, P., Jongen, J. L., Sezer, O., et al. (2012). Management of treatment-emergent peripheral neuropathy in multiple myeloma. Leukemia,26(4), 595–608. https://doi.org/10.1038/leu.2011.346.
Ramsenthaler, C., Kane, P., Gao, W., Siegert, R. J., Edmonds, P. M., Schey, S. A., et al. (2016). Prevalence of symptoms in patients with multiple myeloma: A systematic review and meta-analysis. European Journal of Haematology,97(5), 416–429. https://doi.org/10.1111/ejh.12790.
Jordan, K., Proskorovsky, I., Lewis, P., Ishak, J., Payne, K., Lordan, N., et al. (2014). Effect of general symptom level, specific adverse events, treatment patterns, and patient characteristics on health-related quality of life in patients with multiple myeloma: Results of a European, multicenter cohort study. Supportive Care in Cancer: Official Journal of the Multinational Association of Supportive Care in Cancer,22(2), 417–426. https://doi.org/10.1007/s00520-013-1991-4.
Johnsen, A. T., Tholstrup, D., Petersen, M. A., Pedersen, L., & Groenvold, M. (2009). Health related quality of life in a nationally representative sample of haematological patients. European Journal of Haematology,83(2), 139–148. https://doi.org/10.1111/j.1600-0609.2009.01250.x.
Priscilla, D., Hamidin, A., Azhar, M. Z., Noorjan, K. O. N., Salmiah, M. S., & Bahariah, K. (2011). Quality of life among patients with hematological cancer in a Malaysian hospital. Medical Journal of Malaysia,66(2), 117.
Nielsen, L. K., Jarden, M., Andersen, C. L., Frederiksen, H., & Abildgaard, N. (2017). A systematic review of health-related quality of life in longitudinal studies of myeloma patients. European Journal of Haematology,99(1), 3–17. https://doi.org/10.1111/ejh.12882.
Aaronson, N. K., Ahmedzai, S., Bergman, B., Bullinger, M., Cull, A., Duez, N. J., et al. (1993). The European Organization for Research and Treatment of Cancer QLQ-C30: A quality-of-life instrument for use in international clinical trials in oncology. Journal of the National Cancer Institute,85(5), 365–376.
Basch, E., Deal, A. M., Dueck, A. C., Scher, H. I., Kris, M. G., Hudis, C., et al. (2017). Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA,318(2), 197–198. https://doi.org/10.1001/jama.2017.7156.
Nipp, R. D., El-Jawahri, A., Ruddy, M., Fuh, C., Temel, B., D’Arpino, S. M., et al. (2019). Pilot randomized trial of an electronic symptom monitoring intervention for hospitalized patients with cancer. Annals of Oncology,30(2), 274–280. https://doi.org/10.1093/annonc/mdy488.
Bell, M. L., & Fairclough, D. L. (2014). Practical and statistical issues in missing data for longitudinal patient-reported outcomes. Statistical Methods in Medical Research,23(5), 440–459. https://doi.org/10.1177/0962280213476378.
Fairclough, D. L. (2010). Design and analysis of quality of life studies in clinical trials (2nd ed.). Boca Raton: Chapman & Hall/CRC.
Mercieca-Bebber, R., Palmer, M. J., Brundage, M., Calvert, M., Stockler, M. R., & King, M. T. (2016). Design, implementation and reporting strategies to reduce the instance and impact of missing patient-reported outcome (PRO) data: A systematic review. The BMJ Open,6(6), e010938. https://doi.org/10.1136/bmjopen-2015-010938.
Mercieca-Bebber, R. L., Price, M. A., Bell, M. L., King, M. T., Webb, P. M., & Butow, P. N. (2017). Ovarian cancer study dropouts had worse health-related quality of life and psychosocial symptoms at baseline and over time. Asia-Pacific Journal of Clinical Oncology,13(5), e381–e388. https://doi.org/10.1111/ajco.12580.
Mercieca-Bebber, R., Calvert, M., Kyte, D., Stockler, M., & King, M. T. (2018). The administration of patient-reported outcome questionnaires in cancer trials: Interviews with trial coordinators regarding their roles, experiences, challenges and training. Contemporary Clinical Trials Communications,9, 23–32.
Mercieca-Bebber, R., Kyte, D. G., Calvert, M. J., Stockler, M., & King, M. T. (2017). Challenges and training needs of study coordinators responsible for administration of patient-reported outcome questionnaires in cancer trials [Conference Abstract]. Quality of Life Research,26(1 Supplement 1), 102.
Fairclough, D. L., Peterson, H. F., Cella, D., & Bonomi, P. (1998). Comparison of several model-based methods for analysing incomplete quality of life data in cancer clinical trials. Statistics in Medicine,17(5–7), 781–796.
Bernhard, J., Cella, D. F., Coates, A. S., Fallowfield, L., Ganz, P. A., Moinpour, C. M., et al. (1998). Missing quality of life data in cancer clinical trials: Serious problems and challenges. Statistics in Medicine,17(5–7), 517–532.
Bottomley, A., Pe, M., Sloan, J., Basch, E., Bonnetain, F., Calvert, M., et al. (2016). Analysing data from patient-reported outcome and quality of life endpoints for cancer clinical trials: A start in setting international standards. The lancet Oncology,17(11), e510–e514. https://doi.org/10.1016/s1470-2045(16)30510-1.
Basch, E. (2018). Patient-reported outcomes: An essential component of oncology drug development and regulatory review. The lancet Oncology,19(5), 595–597. https://doi.org/10.1016/s1470-2045(18)30141-4.
Palmer, M. J., Mercieca-Bebber, R., King, M., Calvert, M., Richardson, H., & Brundage, M. (2018). A systematic review and development of a classification framework for factors associated with missing patient-reported outcome data. Clinical Trials,15(1), 95–106. https://doi.org/10.1177/1740774517741113.
Basch, E., Abernethy, A. P., Mullins, C. D., Reeve, B. B., Smith, M. L., Coons, S. J., et al. (2012). Recommendations for incorporating patient-reported outcomes into clinical comparative effectiveness research in adult oncology. Journal of Clinical Oncology,30(34), 4249–4255. https://doi.org/10.1200/jco.2012.42.5967.
Charlson, M. E., Pompei, P., Ales, K. L., & MacKenzie, C. R. (1987). A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation. Journal of Chronic Diseases,40(5), 373–383.
Katz, S., Ford, A. B., Moskowitz, R. W., Jackson, B. A., & Jaffe, M. W. (1963). Studies of illness in the aged. The index of Adl: A standardized measure of biological and psychosocial function. JAMA,185, 914–919.
Lawton, M. P., & Brody, E. M. (1969). Assessment of older people: Self-maintaining and instrumental activities of daily living. The Gerontologist,9(3), 179–186.
Kleber, M., Ihorst, G., Terhorst, M., Koch, B., Deschler, B., Wasch, R., et al. (2011). Comorbidity as a prognostic variable in multiple myeloma: Comparative evaluation of common comorbidity scores and use of a novel MM-comorbidity score. Blood Cancer Journal,1(9), e35. https://doi.org/10.1038/bcj.2011.34.
Palumbo, A., Bringhen, S., Mateos, M. V., Larocca, A., Facon, T., Kumar, S. K., et al. (2015). Geriatric assessment predicts survival and toxicities in elderly myeloma patients: An International Myeloma Working Group report. Blood,125(13), 2068–2074. https://doi.org/10.1182/blood-2014-12-615187.
Mor, V., Laliberte, L., Morris, J. N., & Wiemann, M. (1984). The Karnofsky Performance Status Scale. An examination of its reliability and validity in a research setting. Cancer,53(9), 2002–2007.
Gimsing, P., Holmstrom, M. O., Klausen, T. W., Andersen, N. F., Gregersen, H., Pedersen, R. S., et al. (2016). The Danish national multiple myeloma registry. Clinical Epidemiology,8, 583–587. https://doi.org/10.2147/clep.s99463.
Greipp, P. R., San Miguel, J., Durie, B. G., Crowley, J. J., Barlogie, B., Blade, J., et al. (2005). International staging system for multiple myeloma. Journal of Clinical Oncology,23(15), 3412–3420. https://doi.org/10.1200/jco.2005.04.242.
Folmer Andersen, T., Madsen, M., Jørgensen, J., Mellemkjær, L., & Olsen, J. H. (1999). The Danish National Hospital Register: A valuable source of data for modern health sciences. Danish Medical Bulletin,46, 263–268.
Cocks, K., Cohen, D., Wisloff, F., Sezer, O., Lee, S., Hippe, E., et al. (2007). An international field study of the reliability and validity of a disease-specific questionnaire module (the QLQ-MY20) in assessing the quality of life of patients with multiple myeloma. European Journal of Cancer (Oxford, England: 1990),43(11), 1670–1678.
Postma, T. J., Aaronson, N. K., Heimans, J. J., Muller, M. J., Hildebrand, J. G., Delattre, J. Y., et al. (2005). The development of an EORTC quality of life questionnaire to assess chemotherapy-induced peripheral neuropathy: The QLQ-CIPN20. European Journal of Cancer (Oxford, England: 1990),41(8), 1135–1139.
Ware, J., Jr., Kosinski, M., & Keller, S. D. (1996). A 12-Item Short-Form Health Survey: Construction of scales and preliminary tests of reliability and validity. Medical Care,34(3), 220–233.
Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009). Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics,42(2), 377–381. https://doi.org/10.1016/j.jbi.2008.08.010.
Osoba, D., Bezjak, A., Brundage, M., Zee, B., Tu, D., & Pater, J. (2005). Analysis and interpretation of health-related quality-of-life data from clinical trials: Basic approach of The National Cancer Institute of Canada Clinical Trials Group. European Journal of Cancer,41(2), 280–287. https://doi.org/10.1016/j.ejca.2004.10.017.
Nielsen, L. K., Abildgaard, N., Jarden, M., & Klausen, T. W. (2019). Methodological aspects of health-related quality of life measurement and analysis in patients with multiple myeloma. British Journal of Haematology. https://doi.org/10.1111/bjh.15759.
Gulbrandsen, N., Wisloff, F., Brinch, L., Carlson, K., Dahl, I. M., Gimsing, P., et al. (2001). Health-related quality of life in multiple myeloma patients receiving high-dose chemotherapy with autologous blood stem-cell support. Medical oncology (Northwood, London, England),18(1), 65–77.
Wisloff, F., Eika, S., Hippe, E., Hjorth, M., Holmberg, E., Kaasa, S., et al. (1996). Measurement of health-related quality of life in multiple myeloma. Nordic Myeloma Study Group. British Journal of Haematology,92(3), 604–613.
Little, R. J., D’Agostino, R., Cohen, M. L., Dickersin, K., Emerson, S. S., Farrar, J. T., et al. (2012). The prevention and treatment of missing data in clinical trials. The New England Journal of Medicine,367(14), 1355–1360. https://doi.org/10.1056/NEJMsr1203730.
Nielsen, L., Jarden, M., Andersen, C., Frederiksen, H., Gregersen, H., Klostergaard, A., et al. (2018). Quality of life in newly diagnosed and relapsed Danish multiple myeloma patients—A national longitudinal study of Danish Myeloma Study Group [Conference abstract]. HemaSphere,2(S1), 294–295.
Giesinger, J. M., Wintner, L. M., Zabernigg, A., Gamper, E. M., Oberguggenberger, A. S., Sztankay, M. J., et al. (2014). Assessing quality of life on the day of chemotherapy administration underestimates patients’ true symptom burden. BMC Cancer,14, 758. https://doi.org/10.1186/1471-2407-14-758.
Girgis, A., Durcinoska, I., Gerges, M., Kaadan, N., Arnold, A., Descallar, J., et al. (2018). Study protocol for a controlled trial of an eHealth system utilising patient reported outcome measures for personalised treatment and care: PROMPT-Care 20. BMC Cancer,18(1), 845. https://doi.org/10.1186/s12885-018-4729-3.
Klee, M. C., King, M. T., Machin, D., & Hansen, H. H. (2000). A clinical model for quality of life assessment in cancer patients receiving chemotherapy. Annals of Oncology,11(1), 23–30.
Calvert, M., Kyte, D., Mercieca-Bebber, R., Slade, A., Chan, A. W., King, M. T., et al. (2018). Guidelines for inclusion of patient-reported outcomes in clinical trial Protocols: The SPIRIT-PRO extension. The Journal of the American Medical Association,319(5), 483–494. https://doi.org/10.1001/jama.2017.21903.
This study was funded by The Danish Cancer Society (Grant Number R150-A10023), The Faculty of Health Sciences at The University of Southern Denmark, Amgen A/S, Celgene A/S, Takeda A/S and Janssen A/S. This work is funded by The Danish Cancer Society, Celgene, Amgen, Takeda and Janssen.
Conflict of interest
The authors declare that they have no conflicts of interest.
The study was approved by the Danish Data Protection Agency, registered at ClinicalTrials.gov by number NCT02892383 and carried out in accordance with the Helsinki Declaration and Good Clinical Practice guidelines. Informed consent was obtained from all participants included in the study.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Nielsen, L.K., King, M., Möller, S. et al. Strategies to improve patient-reported outcome completion rates in longitudinal studies. Qual Life Res 29, 335–346 (2020). https://doi.org/10.1007/s11136-019-02304-8
- Missing data
- Health-related quality of life
- Patient-reported outcomes
- Patient-reported outcomes completion rate
- Multiple myeloma