Abstract
Purpose
The purpose of this study was to investigate the impact of the type of data capture on the time and help needed for collecting patient-reported outcomes as well as on the proportion of missing scores.
Methods
In a multinational prospective study, thyroid cancer patients from 17 countries completed a validated questionnaire measuring quality of life. Electronic data capture was compared to the paper-based approach using multivariate logistic regression.
Results
A total of 437 patients were included, of whom 13% used electronic data capture. The relation between data capture and time needed was modified by the emotional functioning of the patients. Those with clinical impairments in that respect needed more time to complete the questionnaire when they used electronic data capture compared to paper and pencil (ORadj 24.0; p = 0.006). This was not the case when patients had sub-threshold emotional problems (ORadj 1.9; p = 0.48). The odds of having the researcher reading the questions out (instead of the patient doing this themselves) (ORadj 0.1; p = 0.01) and of needing any help (ORadj 0.1; p = 0.01) were lower when electronic data capture was used. The proportion of missing scores was equivalent in both groups (ORadj 0.4, p = 0.42).
Conclusions
The advantages of electronic data capture, such as real-time assessment and fewer data entry errors, may come at the price of more time required for data collection when the patients have mental health problems. As this is not uncommon in thyroid cancer, researchers need to choose the type of data capture wisely for their particular research question.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Background
Assessing patient-reported outcomes (PRO) has become a standard now both in clinical trials and routine care worldwide [1, 2]; it is also relevant many years after the diagnosis [3]. In a busy clinic or in multinational trials, using electronic data capture instead of paper is often preferred for several reasons: real-time assessment and consequently immediate feedback on the patients’ well-being [4], easier logistics once the system is established [5], the possibility for the adaptive presentation of questions [6], avoidance of secondary data entry errors [7], and ease of using different language versions.
However, there are also disadvantages of this mode of data capture: costs and time involved to develop and set up the information technology (IT) infrastructure [8], increased workload for the health care professionals [9, 10], challenges with data security [11] or digital exclusion [12]. There are also reports that electronic assessments may be more challenging for some patients [13], although other studies found that most patients prefer electronic symptom monitoring [14, 15]. Generally, it seems that patients agree to report on their symptoms electronically but they want to talk to a person about their problems as soon as there is something of concern [16, 17]. Preferences and participation are also related to the patients’ age, education, and digital literacy [9, 18, 19].
Another much-debated question is whether electronic data capture versus paper and pen or a life telephone interview is equivalent in terms of the outcomes obtained. While most studies find that equivalence is sufficient [20,21,22,23,24,25,26,27], others have reported that this is not always the case [28,29,30,31]. For example, a randomized trial found that patients reported more severe problems with an automated voice response system compared to during a nurse-led live telephone interview [30].
There is conflicting evidence whether the mode of assessment affects the proportion of missing data. While some studies report better completion with electronic data capture [32, 33], others found the opposite [34]. Forced answer options usually result in no missing items at all; participants might, however, just stop with questionnaire completion.
Regarding the time needed to complete a questionnaire, evidence is also inconclusive. In a systematic review, only two out of nine studies found that the completion time was shorter with electronic devices [35]. This question, however, depends on what time is taken into account, just the time needed for filling out the questionnaire or also the time a tablet takes to start or an online version to load.
Due to these open questions, it is generally recommended to avoid mixing the modes of data capture in a given study, unless there is evidence indicating that, for the particular instrument and patient population, the discrepancies are minor [7, 36]. The aim of the present analysis was to investigate whether there were differences in time required, or the need for assistance in completing a newly developed questionnaire measuring quality of life in thyroid cancer patients [37,38,39]. We also examined the proportion of missing data by mode of data capture. In particular, our questions were:
-
1.
Is the type of data capture (electronic vs. paper-based) associated with the time required to complete the questionnaire?
-
2.
Do patients more frequently complete the questionnaire themselves (as opposed to the researcher asking the questions and entering the data/completing the forms on behalf of the patient) when they use electronic rather than paper-and-pen data capture?
-
3.
Is the type of data capture (electronic vs. paper-based) associated with whether or not help is required to complete the questionnaire?
-
4.
Does the proportion of items missing a response vary by type of data capture?
Methods
Study design
We used data from the phase IV field validation study of the European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life Questionnaire Thyroid Module (EORTC QLQ-THY34; study number EORTC 002/2017), to address our research questions. This was a prospective multinational study including patients with thyroid cancer from 21 different institutions in 17 countries.
Two groups of patients were enrolled. The first group comprised patients about to undergo treatment, named Group 1 (treatment). They completed the questionnaire on three occasions: before onset of treatment or best supportive care (t1) as well as 6 weeks (t2) and 6 months thereafter (t3). If a collaborating institution was not able to include patients at t1, patients could be enrolled at t2 (then defined as 6 weeks after first day of initial treatment). The second group comprised people who had been diagnosed with thyroid cancer ≥24 months prior to enrollment, without structural evidence of disease based on imagingFootnote 1, and without anti-neoplastic treatment during the past 12 months, named Group 2 (survivors). They completed the questionnaire only once.Footnote 2 As there is no generally accepted definition of when a patient could be considered to be a survivor, we chose the 24-month time interval, together with the other criteria, assuming that the participants would then not be in an acute situation anymore.
Eligibility criteria for all participants were: diagnosed thyroid cancer, sufficient language proficiency (the questionnaire was available in the local language of the participating institutions) and cognitive functioning to understand and complete the forms (as judged by the local investigator), age ≥16 years, and written informed consent.
Institutional Review Board Approval was obtained from the Ethics Committee of Rhineland-Palatinate Medical Association with reference number 837.406.17 (11240) as well as from all participating sites, according to the respective national requirements. More details about the study design and conduct are published elsewhere [37].
Mode of questionnaire administration and data capture
Electronic data capture was undertaken using the Computer Based Health Evaluation System (CHES) [40, 41]. Each site was free to decide what type of data capture (paper vs. electronic) they used, depending on the local infrastructure.
At the first time-point (t1), all patients were seen in person as per the study protocol. Electronic data capture could be used by handing them a tablet or using an online system.
Instruments
The participants received two questionnaires: the core instrument of the EORTC, the EORTC QLQ-C30 [42], and the thyroid cancer-specific module, the EORTC QLQ-THY34 [37].
The thresholds published by Giesinger et al. [43] were used to define what patients had clinically relevant impairments of their subjective emotional and cognitive functioning, which are 71 and 75, respectively.
The local investigator asked a few debriefing questions at t1 regarding the time needed to complete the questionnaire (core questionnaire and module together), type of completion (self-completed vs. researcher read questions to participants and recorded their response), and whether any help was required to complete it (no help, practical help [for example when the patient did not have their glasses so the questionnaire had to be read to them, or they had shaky hands and had difficulty writing], supportive help [such as interviewer just sitting with the patient while they completed the questionnaire], or help with understanding the questionnaire).
Statistical analysis
Data analysis comprising descriptive statistics and multivariable binary logistic regression analyses was performed with STATA (StataCorp. 2017. Stata Statistical Software: Release 16. College Station, TX: StataCorp LP).
The exposure variable was the type of data capture (paper vs. CHES) and the outcomes were: time needed (<10 vs. ≥10 min), type of completion (self-completed vs. orally), help required (any help vs. no help), and proportion of missing scale scores. According to the EORTC scoring manual [44], a score of a certain scale can be calculated when at least half of the items of that scale are completed. Hence, if more than half of the items are not completed, no score is computed and is therefore missing. We first calculated the number of missing scores per participant and then created a binary variable “any missing score vs. no missing score”. The latter was used as an outcome variable.
We explored the potential for effect modification by the following variables: age (<75 vs. ≥75 years), education (<10 years, 10 years, >10 years), clinical impairment of cognitive or emotional functioning at baseline; effect modification was explored using the Mantel-Haenszel method and consequently tested with likelihood ratio tests in the regression models. If there was no evidence for effect modification, we did not specifically mention that in the following results section.
The following variables were adjusted for language, UICC stage, performance status, comorbidity (ascertained using the Charlson Comorbidity Score), exhaustion at t1 (measured with the EORTC QLQ-THY34). We did not adjust for further individual patient characteristics because the type of data capture did not vary within one center, for logistic reasons. Once the center had decided about the type of data capture, this was not changed later on. For that reason, the results are not confounded by this.
As patients with anaplastic thyroid cancer and those receiving best supportive care differ in certain clinical aspects from those with other histologies and treatments, we explored in a sensitivity analysis whether the effect of data capture on the various outcomes is different in these patients.
Results
Sample characteristics
A total of 437 patients participated (see Fig. 1 for details). The majority (84%) had differentiated thyroid cancer, 11% had medullary, 4% anaplastic, and 1% other types of thyroid cancer. About 9% were 75 years or older. The median age was 51 years (mean: 51 years, standard deviation 16). Most (71%) had received more than 10 years of education (Table 1). At the time of entry into the study, 278 (64%) participants had received total thyroidectomy, 35 (8%) partial thyroidectomy, 72 (16%) no surgery, and for 52 (12%) no information about surgery was available. By the end of the study, 364 (83%) had received total thyroidectomy, 44 (10%) partial thyroidectomy, and 29 (7%) no surgery.
Before t1, 10 (2%) patients had had radiotherapy for local control and 8 for distant metastases; 10 (2%) had received tyrosine kinase inhibitors (TKI); radioactive iodine (RAI) was received by 36 (8%) for ablation and 69 (16%) for therapy. By the end of the study, 275 (63%) participants had ever received RAI and 36 (8%) TKI. There were 4 patients who received the best supportive care.
Clinically relevant impairment of cognitive and emotional functioning as defined by Giesinger et al. (2020) was present at baseline in 36% and 50%, respectively (Table 2).
Type of data capture
A total of 54 (12%) participants from three sites (Kobe, Japan; Innsbruck, Austria; Pamplona, Spain) used CHES for electronic data capture. The remaining data were captured on paper.
Description of outcomes
Time required
About 42% of the participants needed <10 min and 55% needed ≥10 min. For 12 participants (3%), the time required to complete the questionnaire was not documented. The numbers and percentages broken down by type of data capture are displayed in Table 3.
Type of completion
The questionnaires were self-completed by 68% of participants, by the researcher in 30%, and in 2% this was not documented (Table 3).
The type of completion was associated with the language used. The proportion of oral completion (researcher read out the questions) was as follows: Swedish 75%, Greek 61%, Portuguese 53%, French 30%, Spanish 30%, Italian 28%, German 28%, Japanese 27%, English 7%, Dutch 7%, Norwegian 4%, Arabic 2% (p < 0.01).
Help needed
The majority (74%) of the participants required no help to complete the questionnaires. A quarter (24%) needed some type of help, and for 2% whether help was needed was not documented (Table 3).
Proportion of missing scale scores
At t1, there were 16 participants (4% of those participating at t1) with missing scale scores; in 13 of these cases, only 1 score was missing.
At t2, there were 10 participants (3% of those participating at t2) with missing scale scores; in 8 of them, only 1 score was missing.
At t3, there were 14 participants (6% of those participating at t3) with missing scale scores; in 11 of them, only 1 score was missing.
For the 6 participants who entered the study at t2, their completion data were used from t2 for the following analyses. They all had no missing scores at t2. Hence, the final number for the regression models was 16 (4%) with at least one missing scale score.
Association of type of data capture with time or help required and with missing scores
Is the type of data capture associated with the time data collection needs?
There was a social gradient with less time needed the higher the education level was (≥10 min needed in participants with less than 10 years versus 10 years versus more than 10 years of schooling: 76% vs. 60% vs. 53%), but effect modification was not observed. A similar pattern was seen regarding cognitive functioning. Emotional functioning, however, did modify the effect of data capture on time needed. Consequently, stratum-specific effect estimates for patients with and without clinical impairment of emotional functioning are reported in the following, and the other variables were treated as potential confounders. Hence, the final list of variables adjusted for the regression model were age, education, cognitive functioning, language, UICC stage, status of disease, performance status, comorbidity, and exhaustion.
With this model, there was no evidence that patients without clinically relevant emotional problems differed in the time needed to complete the questionnaires (adjusted odds ratio [ORadj] 1.9, 95% confidence interval [CI] 0.3–11.4; p = 0.48). In contrast to that, patients with clinically relevant emotional problems more often needed ≥10 min to complete the questionnaires when they used electronic data capture compared to paper and pencil (ORadj 24.0, CI 2.4–235.8; p = 0.006; Fig. 2).
When we ignore effect modification by emotional functioning, the odds of needing more than 10 min for questionnaire completion was 8 times higher in patients who used electronic data capture compared to paper and pencil (ORadj 5.5, CI 1.1–27.3; p = 0.04).
Do patients more frequently self-complete the questionnaire when they use electronic data capture?
The odds of having the researcher reading the questions out (instead of the patient doing this themselves) were considerably lower when electronic data capture was used (ORadj 0.1, CI 0.03–0.6; p = 0.01) compared to paper-based data collection.
Is the type of data capture associated with the help required for data collection?
The odds of needing any help were lower when electronic data capture was used (ORadj 0.1, CI 0.02–0.6; p = 0.01) compared to paper-based data collection.
Does the proportion of missing scale scores differ by type of data capture?
The proportion of missing scale scores was similar in both types of data capture: 4% vs. 2% had at least one missing scale score in paper vs. electronic data capture (see Table 3 for details). In line with that, when adjusting for potential confounders, we found no evidence for an effect of data capture on the likelihood of missing scores (ORadj 0.4, CI 0.04–4.0; p = 0.42).
Sensitivity analysis
Anaplastic cancer versus other types of histology
Of the 19 patients with anaplastic cancer, 3 had used CHES and the remaining used paper for their data capture. More than half (58%, n = 11) reported clinically relevant levels of emotional problems. The majority (84%, n = 16) needed more than 10 min for completing the questionnaires. Fifty-eight percent (n = 11) completed the questionnaires on their own. Ten percent (n = 2) had at least one missing score at t1. The anaplastic cancer patients were considerably older than the patients with other histologies (median 72 years, mean 70, range 43–83 years).
Due to the low number of patients using CHES in this group of patients, regression models using the same specifications as with the entire sample (variables taken into account for adjustment and effect modification) could not be computed. We had to restrict the adjustment variables to age and stratify by emotional functioning for the model where we had found effect modification in the entire sample. In those with increased levels of emotional problems, the age-adjusted OR for electronic data capture on time needed to complete the questionnaire was 0.13 (p = 0.24). For the eight patients with sub-threshold emotional problems and for all other outcomes, no regression models could be computed due to the low number of cases.
Best supportive care versus other treatment
All of the four patients with the best supportive care had used paper for data capture. Hence, no comparisons between different types of data capture were possible here.
Discussion
This analysis set out to investigate the effect of electronic versus paper-based data capture on the time and help needed to complete the questionnaires for a newly developed questionnaire to measure the quality of life in thyroid cancer patients [37]. We were also interested in whether the proportion of missing scores differs between these two options.
We found that the effect of data capture on the time needed is probably modified by the emotional well-being of the patients. In those without clinically relevant mental health problems, the questionnaire completion time is similar for both types of data capture. This is in contrast to other studies where less time was needed for electronic data capture [45], though the majority of studies did not find a difference in time for completion [35]. Those exceeding the threshold for clinical importance, however, needed in our study more time when electronic data capture was used. This finding is of relevance because emotional problems are common among cancer patients; about a third suffer from co-morbid mental health conditions [46,47,48,49]. As thyroid cancer involves the endocrine system, which is related to mental health, this topic is of particular concern in this group of patients [50,51,52,53,54,55,56]. They often suffer from depression, anxiety and poor emotional functioning, even for long periods of time after the diagnosis [3, 57,58,59]. In our study, half of the participants indicated emotional problems of clinical relevance. The instrument used to capture emotional functioning, the EORTC QLQ-C30, subsumes the following constructs under it: tension, worry, irritability, and depression. Especially irritability and tension are known to be related to hyperthyroidism, while depression is more common during hypothyroid states. Thyroid cancer patients can suffer from both.
It is well-known that individuals with emotional problems may slow down in their capabilities of processing thoughts and deciding [60,61,62,63]. Recent data on digital interventions also suggest that depressive patients need professional support by a human being and should not be left alone with a tablet or computer [64]. This underlines that using the latest technology is not always the best option for data capture [24] and researchers need to think carefully about their target population when deciding about the methods of data collection, be it “paper or plastic” [23].
Apart from that particular problem, our data show that electronic data capture may be associated with needing less help in filling out the questionnaires and with a higher probability of self-completion, which is usually desired in large-scale observational studies and clinical trials. This might seem somewhat contradictory to the findings discussed above about the time needed. It might imply that the electronic data capture procedures were clear to the patients and the material was easy to use, but that some of the patients still needed more time to complete the forms electronically, for other reasons than needing help.
The proportion of missing scores was equal in both types of data capture, which is in contrast to Blondin’s study where more missing values were found with electronic data capture [34]. This difference could be explained by the different modes of data acquisition: they used an interactive voice response system, whereas we presented a questionnaire on a tablet. Moreover, they elicited daily reports from the patients, whereas our study only had three timepoints for data collection, with a longer time in between.
The results of our study should be interpreted in the light of its limitations. First of all, the type of data capture was not randomly allocated to either institutions or individual patients. The differences found could therefore also be due to residual confounding. There could also be more effect modifiers that we did not take into account. We selected only four variables for test of effect modification in order to keep the power of the tests at an acceptable level. This brings us to the next limitation which is the relatively low proportion of sites (and therefore participants) using electronic data capture. Consequently, the statistical tests used do not have the ability to detect smaller effects and effect estimates may be imprecise. Moreover, the three sites using CHES are most likely not representative for other institutions because they had a preference for this type of data capture. We also learned that one of these sites included patients at the nuclear medicine department, where internet connections were slow in some rooms due to necessary radiation protection measures there. Finally, the time required for completing the questionnaires was not measured exactly but recorded by the researchers. It is possible that it was remembered differently when using paper versus electronic data capture, thereby introducing an information bias that cannot be controlled for.
Strengths of our study include that all data collection that we considered in this analysis was done in the hospitals, so the home environment of the patients could not affect the results.
In summary, the obvious advantages of electronic data capture such as real-time assessment [4] and fewer data entry errors [7] may come at the price of additional time required for data collection on the part of the patients when they have mental health problems. As such problems are very common among thyroid cancer patients [57, 65, 66], researchers and clinicians should take these aspects into consideration when choosing a type of data capture for a particular research question or clinical application, respectively, in this patient population.
Data availability
The data of this study are stored in the EORTC data repository and can be accessed by other researchers.
Notes
Survivors could have biochemically incomplete response or indeterminate response.
For the purpose of testing retest reliability, the questionnaire was presented a second time shortly after the first measurement, but these data were not used in the current analysis.
References
H. Ishiki, Y. Kikawa, M. Terada, J. Mizusawa, M. Honda, T. Iwatani, T. Mizutani, K. Mori, N. Nakamura, T. Miyaji et al. Patient-reported outcome and quality of life research policy: Japan Clinical Oncology Group (JCOG) policy. Jpn. J. Clin. Oncol. 53, 195–202 (2023)
G. Velikova, C. Coens, F. Efficace, E. Greimel, M. Groenvold, C. Johnson, S. Singer, L. van de Poll-Franse, T. Young, A. Bottomley, Health-related quality of life in EORTC clinical trials – 30 years of progress from methodological developments to making a real impact on oncology practice. Eur. J. Cancer Suppl. 10, 141–149 (2012)
M. Dionisi-Vici, M. Fantoni, R. Botto, A. Nervo, F. Felicetti, R. Rossetto, M. Gallo, E. Arvat, R. Torta, P. Leombruni, Distress, anxiety, depression and unmet needs in thyroid cancer survivors: a longitudinal study. Endocrine 74, 603–610 (2021)
A.P. Abernethy, A. Ahmad, S.Y. Zafar, J.L. Wheeler, J.B. Reese, H.K. Lyerly, Electronic patient-reported data capture as a foundation of rapid learning cancer care. Med. Care 48, S32–S38 (2010)
P. Cramon, A.K. Rasmussen, S.J. Bonnema, J.B. Bjorner, U. Feldt-Rasmussen, M. Groenvold, L. Hegedus, T. Watt, Development and implementation of PROgmatic: A clinical trial management system for pragmatic multi-centre trials, optimised for electronic data capture and patient-reported outcomes. Clin. Trials 11, 344–354 (2014)
M.A. Petersen, N.K. Aaronson, J. Arraras, W.C. Chie, T. Conroy, A. Costantini, L. Dirven, P.M. Fayers, E.M. Gamper, J.M. Giesinger et al. The EORTC CAT Core-The computer adaptive version of the EORTC QLQ-C30 questionnaire. Eur. J. Cancer 100, 8–16 (2018)
P. O’Donohoe, D.S. Reasner, S.M. Kovacs, B. Byrom, S. Eremenco, A.I. Barsdorf, V. Arnera, S.J. Coons, Updated recommendations on evidence needed to support measurement comparability among modes of data collection for patient-reported outcome measures: a good practices report of an ISPOR Task Force. Value Health 26, 623–633 (2023)
V. Zebralla, N. Pohle, S. Singer, T. Neumuth, A. Dietz, M. Stier-Jarmer, A. Boehm, Vorstellung des Screeningsystems (OncoFunction) für Funktionsstörungen im Kopf-Hals-Tumor-follow-up. Laryngorhinootologie 95, 118–124 (2016).
C. Steiert, J. Lambeck, T.D. Grauvogel, J. Beck, J. Grauvogel, Digital patient-reported outcome measures assessing health-related quality of life in skull base diseases-analysis of feasibility and pitfalls two years after implementation. Healthcare 11, 13 (2023)
T.R. Kiderlen, A. Schnack, M. de Wit Essential barriers and considerations for the implementation of electronic patient-reported outcome (ePRO) measures in oncological practice: contextualizing the results of a feasibility study with existing literature. Z. Gesundh. Wiss. 1–18 (2022)
C. Tran, A. Dicker, B. Leiby, E. Gressen, N. Williams, H. Jim, Utilizing digital health to collect electronic patient-reported outcomes in prostate cancer: single-arm pilot trial. J. Med. Internet Res. 22, 11 (2020)
J. Fischer, K. Vltavska: digital literacy and digital exclusion of poor, unemployed, uneducated and pensioners: who is the most threatened?, 27th Interdisciplinary Information Management Talks Conference (IDIMT). Schriftenreihe Informatik. Kutna Hora, Trauner Verlag, 2019, pp 75–82
B.Q. Zhu, T.N. Feng, B. Izci-Balserak, Using research electronic data capture for longitudinal assessment among older adults with diabetes enhances real-time data collection. Cin. Comput. Inform. Nurs. 39, 32–41 (2021)
L.M. Wintner, J.M. Giesinger, A. Zabernigg, G. Rumpold, M. Sztankay, A.S. Oberguggenberger, E.M. Gamper, B. Holzner, Evaluation of electronic patient-reported outcome assessment with cancer patients in the hospital and at home. BMC Med. Inform. Decis. Mak. 15, 10 (2015)
G. Velikova, E.P. Wright, A.B. Smith, A. Cull, A. Gould, D. Forman, T. Perren, M. Stead, J. Brown, P.J. Selby, Automated collection of quality-of-life data: a comparison of paper and computer touch-screen questionnaires. J. Clin. Oncol. 17, 998–1007 (1999)
K.E. Alexander, T. Ogle, H. Hoberg, L. Linley, N. Bradford, Patient preferences for using technology in communication about symptoms post hospital discharge. BMC Health Serv. Res. 21, 11 (2021)
S.S. Olmsted, J.D. Grabenstein, A.K. Jain, N. Lurie, Patient experience with, and use of, an electronic monitoring system to assess vaccination responses. Health Expect. 9, 110–117 (2006)
J. Graf, N. Sickenberger, K. Brusniak, L.M. Matthies, T.M. Deutsch, E. Simoes, C. Plappert, L. Keilmann, A. Hartkopf, C.B. Walter et al. Implementation of an electronic patient-reported outcome app for health-related quality of life in breast cancer patients: evaluation and acceptability analysis in a two-center prospective trial. J. Med. Internet Res. 24, 10 (2022)
L. Kosowicz, K. Tran, T.T. Khanh, T.H.A. Dang, V. Pham, H.T.T. Kim, H.T.B. Duong, T.D. Nguyen, A.T. Phuong, T.H. Le et al. Lessons for Vietnam on the use of digital technologies to support patient-centered care in low- and middle-income countries in the Asia-Pacific region: scoping review. J. Med. Internet Res. 25, 14 (2023)
C.J. Gwaltney, A.L. Shields, S. Shiffman, Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health 11, 322–333 (2008)
W. Muehlhausen, H. Doll, N. Quadri, B. Fordham, P. O’Donohoe, N. Dogar, D.J. Wild, Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013. Health Qual. Life Outcomes 13, 20 (2015)
H. Romero, D. DeBonis, P. O’Donohoe, K.W. Wyrwich, V. Arnera, J.V. Platko, T. Willgoss, K. Harris, M. Crescioni, S. Steele et al. Recommendations for the electronic migration and implementation of clinician-reported outcome assessments in clinical trials. Value Health 25, 1090–1098 (2022)
A.S. Green, E. Rafaeli, N. Bolger, P.E. Shrout, H.T. Reis, Paper or plastic? Data equivalence in paper and electronic diaries. Psychol. Methods 11, 87–105 (2006)
K. Weiler, A.M. Christ, G.G. Woodworth, R.L. Weiler, J.M. Weiler, Quality of patient-reported outcome data captured using paper and interactive voice response diaries in an allergic rhinitis study: is electronic data capture really better?. Ann. Allergy Asthma Immunol. 92, 335–339 (2004)
C. Rutherford, D. Costa, R. Mercieca-Bebber, H. Rice, L. Gabb, M. King, Mode of administration does not cause bias in patient-reported outcome results: a meta-analysis. Qual. Life Res. 25, 559–574 (2016)
N. Kamo, S.V. Dandapani, R.A. Miksad, M.J. Houlihan, I. Kaplan, M. Regan, T.K. Greenfield, M.G. Sanda, Evaluation of the SCA instrument for measuring patient satisfaction with cancer care administered via paper or via the Internet. Ann. Oncol. 22, 723–729 (2011)
S.L. Rasmussen, L. Rejnmark, E. Ebbehoj, U. Feldt-Rasmussen, A.K. Rasmussen, J.B. Bjorner, T.Watt, High level of agreement between electronic and paper mode of administration of a thyroid-specific patient-reported outcome, ThyPRO. Eur. Thyroid J. 5, 65–72 (2016).
M.K. Lee, T.J. Beebe, K.J. Yost, D.T. Eton, P.J. Novotny, A.C. Dueck, M. Frost, J.A. Sloan, Score equivalence of paper-, tablet-, and interactive voice response system-based versions of PROMIS, PRO-CTCAE, and numerical rating scales among cancer patients. J. Patient Rep. Outcomes 5, 19 (2021)
J.K. Schmier, D.W. Kane, M.T. Halpern, Practical applications of usability theory to electronic data collection for clinical trials. Contemp. Clin. Trials 26, 376–385 (2005)
A. Sikorskii, C.W. Given, B. Given, S. Jeon, M. You, Differential symptom reporting by mode of administration of the assessment automated voice response system versus a live telephone interview. Med. Care 47, 866–874 (2009)
E.F. Juniper, J.M. Langlands, B.A. Juniper, Patients may respond differently to paper and electronic versions of the same questionnaires. Respir. Med. 103, 932–934 (2009)
B. Gurland, P.C. Alves-Ferreira, T. Sobol, R.P. Kiran, Using technology to improve data capture and integration of patient-reported outcomes into clinical care: pilot results in a busy colorectal unit. Dis. Colon Rectum 53, 1168–1175 (2010)
J. Roick, H. Danker, A. Kersting, A. Briest, A. Dietrich, A. Dietz, J. Einenkel, K. Papsdorf, F. Lordick, J. Meixensberger et al. Factors associated with non-participation and dropout among cancer patients in a cluster-randomised controlled trial. Eur. J. Cancer Care 27, e12645 (2018)
J.M. Blondin, K.S. Abu-Hasaballah, H. Tennen, R.V. Lalla, Electronic versus paper diaries: a pilot study of concordance and adherence in head and neck cancer patients receiving radiation therapy. Head. Neck Oncol. 2, 5 (2010)
O. Dale, K.B. Hagen, Despite technical problems personal digital assistants outperform pen and paper when collecting patient diary data. J. Clin. Epidemiol. 60, 8–17 (2007)
S. Eremenco, S.J. Coons, J. Paty, K. Coyne, A.V. Bennett, D. McEntegart, I.P.M.M.T. Force, PRO data collection in clinical trials using mixed modes: report of the ISPOR PRO mixed modes good research practices task force. Value Health 17, 501–516 (2014)
S. Singer, A. Al-Ibraheem, M. Pinto, I. Iakovou, A. Osthus, E. Hammerlid, L.D. Locati, E.M. Gamper, J. Arraras, S. Jordan, et al. International phase IV field study for the reliability and validity of the European Organisation for Research and Treatment of Cancer Thyroid Cancer Module EORTC QLQ-THY34. Thyroid 33, 1078–1089 (2023). https://doi.org/10.1089/thy.2023.0221
S. Singer, S. Jordan, L. Locati, M. Pinto, I.M. Tomaszewska, C. Araújo, E. Hammerlid, E. Vidhubala, O. Husson, N. Kiyota et al. The EORTC module for quality of life in patients with thyroid cancer: phase III. Endocr. Relat. Cancer 24, 197–207 (2017)
S. Singer, O. Husson, I.M. Tomaszewska, L. Locati, N. Kiyota, U. Scheidemann-Wesp, D. Hofmeister, M. Winterbotham, C. Brannan, C. Araújo et al. Quality-of-life priorities in patients with thyroid cancer: a multinational European organisation for research and treatment of cancer phase I study. Thyroid 26, 1605–1613 (2016)
B. Holzner, J.M. Giesinger, J. Pinggera, S. Zugal, F. Schopf, A.S. Oberguggenberger, E.M. Gamper, A. Zabernigg, B. Weber, G. Rumpold, The Computer-based Health Evaluation Software (CHES): a software for electronic patient-reported outcome monitoring. BMC Med. Inform. Decis. Mak. 12, 11 (2012)
A. Erharter, J. Giesinger, G. Kemmler, G. Schauer-Maurer, G. Stockhammer, A. Muigg, M. Hutterer, G. Rumpold, B. Sperner-Unterweger, B. Holzner, Implementation of computer-based quality-of-life monitoring in brain tumor outpatients in routine clinical practice. J. Pain. Symptom Manage. 39, 219–229 (2010)
N. Aaronson, S. Ahmedzai, B. Bergmann, M. Bullinger, A. Cull, N.J. Duez, A. Filiberti, H. Flechtner, J.C.J.M. de Haes, S. Kaasa et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J. Nat. Cancer Inst. 85, 365–376 (1993)
J.M. Giesinger, F.L.C. Loth, N.K. Aaronson, J.I. Arraras, G. Caocci, F. Efficace, M. Groenvold, M. van Leeuwen, M.A. Petersen, J. Ramage et al. Thresholds for clinical importance were established to improve interpretation of the EORTC QLQ-C30 in clinical practice and research. J. Clin. Epidemiol. 118, 1–8 (2020)
P. Fayers, N. Aaronson, K. Bjordal, M. Groenvold, D. Curran, A. Bottomley: EORTC QLQ-C30 Scoring Manual, 3rd edn. (EORTC, Brüssel, 2001)
J.Y. Yu, T. Goldberg, N.I. Lao, B.M. Feldman, Y.I. Goh, Electronic forms for patient reported outcome measures (PROMs) are an effective, time-efficient, and cost-minimizing alternative to paper forms. Pediatr. Rheumatol. 19, 9 (2021)
A.J. Mitchell, M. Chan, H. Bhatti, M. Halton, L. Grassi, C. Johansen, N. Meader, Prevalence of depression, anxiety, and adjustment disorder in oncological, haematological, and palliative-care settings: a meta-analysis of 94 interview-based studies. Lancet Oncol. 12, 160–174 (2011)
S. Singer, J. Das-Munshi, E. Brähler, Prevalence of mental health conditions in cancer patients in acute care – a meta-analysis. Ann. Oncol. 21, 925–930 (2010)
T. Akechi, T. Okuyama, Y. Sugawara, T. Nakano, Y. Shima, Y.Uchitomi, Major depression, adjustment disorders, and post-traumatic stress disorder in terminally ill cancer patients: associated and predictive factors. J. Clin. Oncol. 22, 1957–1965 (2004).
S. Singer, C. Szalai, S. Briest, A. Brown, A. Dietz, J. Einenkel, S. Jonas, A. Konnopka, K. Papsdorf, D. Langanke et al. Co-morbid mental health conditions in cancer patients at working age – prevalence, risk profiles, and care uptake. Psycho. Oncol. 22, 2291–2297 (2013)
H. Himmerich, S. Fulda, J. Linseisen, H. Seiler, G. Wolfram, S. Himmerich, K. Gedrich, S. Kloiber, S. Lucae, M. Ising et al. Depression, comorbidities and the TNF-alpha system. Eur. Psychiatry 23, 421–429 (2008)
S. Tagay, S. Herpertz, M. Langkafel, Y. Erim, L. Freudenberg, N. Schopper, A. Bockisch, W. Senf, R.Gorges, Health-related quality of life, anxiety and depression in thyroid cancer patients under short-term hypothyroidism and TSH-suppressive levothyroxine treatment. Eur. J. Endocrinol. 153, 755–763 (2005).
R. Larisch, K. Kley, S. Nikolaus, W. Sitte, M. Franz, H. Hautzel, W. Tress, H.W. Muller, Depression and anxiety in different thyroid function states. Horm. Metab. Res. 36, 650–653 (2004)
M. Büttner, D. Krogh, H. Siggelkow, S. Singer, What are predictors of impaired quality of life in patients with hypoparathyroidism? – Results of a patient survey. Clin. Endocrinol. 97, 268–275 (2022)
M. Büttner, D. Krogh, H. Siggelkow, S. Singer, Impairments in quality of life and predictors of symptom burden in patients with hypoparathyroidism: results from a population-based survey. Endocrine 82, 419–426 (2023)
M.L. Monzani, F. Piccinini, G. Boselli, R. Corleto, G. Margiotta, R.P. Peeters, M. Simoni, G. Brigante, Changes in quality of life after thyroidectomy in subjects with thyroid cancer in relation to the dose of levothyroxine. J. Endocrinol. Investig. 46, 319–326 (2023)
W.L. Chan, H.C.W. Choi, B. Lang, K.P. Wong, K.K. Yuen, K.O. Lam, V.H.F. Lee, D. Kwong. Health-related quality of life in asian differentiated thyroid cancer survivors. Cancer Control. 28, 10732748211029726 (2021)
S. Singer, T. Lincke, E. Gamper, S. Schreiber, A. Hinz, K. Bhaskaran, T. Schulte, Quality of life in patients with thyroid cancer compared with the general population. Thyroid 22, 117–124 (2012)
E.M. Gamper, L.M. Wintner, M. Rodrigues, S. Buxbaum, B. Nilica, S. Singer, J.M. Giesinger, B. Holzner, I.Virgolini, Persistent quality of life impairments in differentiated thyroid cancer patients: results from a monitoring program. Eur. J. Nucl. Med. Mol. Imaging 42, 1179–1188 (2015)
S.J. Yang, X.Q. Xu, Anxiety and quality of life among papillary thyroid cancer patients awaiting final pathology results after surgery. Endocrine 76, 377–384 (2022)
P. Yudilevich, E. BenEliahu, Bion’s alpha-function as a key for understanding emotional problems in ADHD. Psychoanal. Psychol. 39, 302–310 (2022)
B.C. Schneider, S. Diedrich, M. Hauschildt, S.V. Biedermann, S. Arlt, S. Moritz, L. Jelinek, Changes in processing speed, cognitive flexibility, and selective attention over a four-week treatment period in inpatients with moderate to severe depression. Z. Fur Neuropsychologie. 32, 129–140 (2021)
A. Mougias, F. Christidi, M. Synetou, I. Kotrotsou, P. Valkimadi, A. Politis, Differential effect of demographics, processing speed, and depression on cognitive function in 755 non-demented community-dwelling elderly individuals. Cogn. Behav. Neurol. 32, 236–246 (2019)
M. Blair, S. Gill, I. Gutmanis, K. Smolewska, E. Warriner, S.A. Morrow, The mediating role of processing speed in the relationship between depressive symptoms and cognitive function in multiple sclerosis. J. Clin. Exp. Neuropsychol. 38, 782–794 (2016)
J. Posselt, R. Klawunn, M.-L. Dierks, Digital health intervention prescriptions for people with depressive disorders: results of a qualitative study. Z. für. Allgemeinmedizin. 99, 145–150 (2023)
S. Tagay, S. Herpertz, M. Langkafel, Y. Erim, A. Bockisch, W. Senf, R. Gorges, Health-related quality of life, depression and anxiety in thyroid cancer patients. Qual. Life Res. 15, 695–703 (2006)
C. O’Neill, M. Carlson, C. Rowe, E. Fradgley, C. Paul, Hearing the voices of Australian thyroid cancer survivors: qualitative thematic analysis of semistructured interviews identifies unmet support needs. Thyroid 33, 1455–1464 (2023). https://doi.org/10.1089/thy.2023.0080
Acknowledgements
We would like to acknowledge the work of colleagues who helped with data collection: Gisele da Rocha Santa, Micha Pilz, Hisayo Doi, Emily Dickson; for setting up this study in local institutions: Cláudia Áraújo.
Funding
This study was funded by the European Organisation for Research and Treatment of Cancer, Quality of Life Group (project number 002/2017). The EORTC Quality of Life Group business model involves charges for commercial companies using EORTC instruments. Academic use of EORTC instruments is free of charge. In addition, this work was partially supported by the Japan Agency for Medical Research and Development (AMED), grant number 21ck0106498h0003, for the participation of the Japanese site. Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Contributions
S.S., N.K., G.S., A.A.-I., M.P., I.I., A.Ø., E.H., L.D.L., E.M.G., J.I.A., S.J., G.I., O.H., L.M., J.I., G.A., and H.R. contributed to the study conception and design. Data collection was performed by G.S., A.A.-I., M.P., I.I., A.Ø., E.H., L.D.L., E.M.G., J.I.A., S.J., N.K, M.B., R.C., G.I., O.H., R.R.G., G.F., L.M., J.I., G.A. Data analysis was performed by S.S. following a statistical analysis plan that had been drafted by S.S. and discussed with N.K., G.S., R.R.G., A.A.-I., M.P., I.I., A.Ø., E.H., L.D.L., E.M.G., M.B., G.I., O.H., G.F., and J.I. The first draft of the manuscript was written by S.S. and all authors could comment on it and on subsequent versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
S.S. has received honoraria from Lilly for reviewing papers for their Quality of Life Award and from Eisei for advice in writing a paper, outside of the submitted work. N.K. reports honoraria from ONO PHARMACEUTICAL, Bristol Meyers Squibb, Merck Biopharma, Astra-Zeneca, Merck Sharp & Dohme, Eisai, Bayer and Chugai Pharmaceutical, all outside the submitted work. M.P. reports speaker fees from Meeting&Words and from Hinovia S.r.l., outside the submitted work. G.S. has received research support in the form of donations from Merck Sharp & Dohme, IBSA and Alpha-Sigma, which partially defrayed costs associated with recruiting patients in this study. M.B. has received speaker fees from Lilly and Takeda outside of the submitted work. All authors have a special interest in quality of life in thyroid cancer patients and survivors. The authors declare that they have no conflict of interest that could be perceived as prejudicing the impartiality of the research reported.
Consent to participate
Written informed consent was obtained from all individual participants included in the study.
Ethics approval
All procedures performed were in accordance with the ethical standards of the institutional research committees and with the 1964 Helsinki Declaration and its later amendments of comparable ethical standards. All sites obtained ethical approval in accordance with regional and national requirements. Approval number from the principal investigator’s institution: 837.406.17 (11240).
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Singer, S., Sykiotis, G., Al-Ibraheem, A. et al. The impact of electronic versus paper-based data capture on data collection logistics and on missing scores in thyroid cancer patients. Endocrine 84, 635–645 (2024). https://doi.org/10.1007/s12020-023-03628-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12020-023-03628-9