Usability of an online application for reporting the burden of side effects in cancer patients

  • Mies C. van EenbergenEmail author
  • Corina van den Hurk
  • Floortje Mols
  • Lonneke V. van de Poll-Franse
Original Article



In the Netherlands, online patient reporting of side effects is a new phenomenon. The aim of this study was to gain insight into patients’ user experiences and the benefits of the web application BijKanker (‘AlongsideCancer’).


Patients in seven hospitals were asked to make entries in BijKanker at least once a week. On logging in to the application for the first time (T1), patients were asked to complete a questionnaire on information needs and Internet use. Four months after starting their treatment (T2), they were asked to complete a second questionnaire on their experiences with BijKanker and its four functions: information; reporting side effects; communication with oncology nurses and clinicians; and data feedback.


Ninety-nine patients logged in to BijKanker, 60 patients (61%) had completed the first questionnaire (T1) and 40 (40%) had also completed the second questionnaire (T2). In total, 1661 side effects were reported. Generally, patients experienced BijKanker as user-friendly and patients appreciated the attention given to their side effects.


The user-centred design gives ample insight into user experiences and usability. The results provide useful starting points for improvements to the online application. We recommend to put much effort into supporting oncology nurses in the implementation of the application.


Cancer Side effects Online application Patient-reported outcomes Chemotherapy 



Common Terminology Criteria for Adverse Events


Health-related quality of life


The Netherlands Comprehensive Cancer Organisation


Nurse Problem Analysis


Patient-Reported Outcomes Common Terminology Criteria for Adverse Events


Randomised clinical trial



We wish to thank the patients and their nurses and doctors at the following hospitals in the Netherlands for their participation in this study: Bernhoven Uden, Elisabeth-TweeSteden Hospital Tilburg, Haven Hospital Rotterdam, Maastricht UMC, Medical Centre Alkmaar, Spaarne Hospital Haarlem, and ZorgSaam Terneuzen. Special thanks are due to Josca Heier-van Leerdam for her role as an advisor.

Compliance with ethical standards

A declaration of no objection was granted by the Medical Ethics Review Committee Midden-Brabant (NW2015-17). All participants provided informed consent online.

Conflict of interest

The authors declare that they have no conflict of interest.


All the authors have approved the contents of this manuscript. This manuscript is not being considered for publication elsewhere and its findings have not been previously published.

Supplementary material

520_2019_4639_MOESM1_ESM.xlsx (44 kb)
ESM 1 (XLSX 44 kb)


  1. 1.
    Mols F, Beijers T, Lemmens V, van den Hurk CJ, Vreugdenhil G, van de Poll-Franse LV (2013) Chemotherapy-induced neuropathy and its association with quality of life among 2- to 11-year colorectal cancer survivors: results from the population-based PROFILES registry. J Clin Oncol 31(21):2699–2707. CrossRefPubMedGoogle Scholar
  2. 2.
    Trajkovic-Vidakovic M, de Graeff A, Voest EE, Teunissen SC (2012) Symptoms tell it all: a systematic review of the value of symptom assessment to predict survival in advanced cancer patients. Crit Rev Oncol Hematol 84(1):130–148. CrossRefPubMedGoogle Scholar
  3. 3.
    Foote M (1998) The importance of planned dose of chemotherapy on time: do we need to change our clinical practice? Oncologist 3(5):365–368PubMedGoogle Scholar
  4. 4.
    Goldhirsch A, Coates AS, Colleoni M, Castiglione-Gertsch M, Gelber RD (1998) Adjuvant chemoendocrine therapy in postmenopausal breast cancer: cyclophosphamide, methotrexate, and fluorouracil dose and schedule may make a difference. International Breast Cancer Study Group. J Clin Oncol 16(4):1358–1362. CrossRefPubMedGoogle Scholar
  5. 5.
    Beijers AJ, Mols F, van den Hurk CJ, Vreugdenhil A (2016) Are chemotherapy-associated symptoms underestimated? A view beyond common toxicity criteria. Acta Oncol 55(4):516–518. CrossRefPubMedGoogle Scholar
  6. 6.
    Kuderer NM, Wolff AC (2014) Enhancing therapeutic decision making when options abound: toxicities matter. J Clin Oncol 32(19):1990–1993. CrossRefPubMedGoogle Scholar
  7. 7.
    Judson TJ, Bennett AV, Rogak LJ, Sit L, Barz A, Kris MG, Hudis CA, Scher HI, Sabattini P, Schrag D, Basch E (2013) Feasibility of long-term patient self-reporting of toxicities from home via the Internet during routine chemotherapy. J Clin Oncol 31(20):2580–2585. CrossRefPubMedPubMedCentralGoogle Scholar
  8. 8.
    Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, Rogak L, Bennett AV, Dueck AC, Atkinson TM, Chou JF, Dulko D, Sit L, Barz A, Novotny P, Fruscione M, Sloan JA, Schrag D (2016) Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol 34(6):557–565. CrossRefPubMedGoogle Scholar
  9. 9.
    Basch E, Iasonos A, Barz A, Culkin A, Kris MG, Artz D, Fearn P, Speakman J, Farquhar R, Scher HI, McCabe M, Schrag D (2007) Long-term toxicity monitoring via electronic patient-reported outcomes in patients receiving chemotherapy. J Clin Oncol 25(34):5374–5380. CrossRefPubMedGoogle Scholar
  10. 10.
    Pietanza MC, Basch EM, Lash A, Schwartz LH, Ginsberg MS, Zhao B, Shouery M, Shaw M, Rogak LJ, Wilson M, Gabow A, Latif M, Lin KH, Wu Q, Kass SL, Miller CP, Tyson L, Sumner DK, Berkowitz-Hergianto A, Sima CS, Kris MG (2013) Harnessing technology to improve clinical trials: study of real-time informatics to collect data, toxicities, image response assessments, and patient-reported outcomes in a phase II clinical trial. J Clin Oncol 31(16):2004–2009. CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Denis F, Lethrosne C, Pourel N, Molinier O, Pointreau Y, Domont J et al. Overall survival in patients with lung cancer using a web-application-guided follow-up compared to standard modalities: results of phase III randomized trial. . 2016 ASCO Annual Meeting Chicago: J Clin Oncol 34, 2016 (suppl; abstr LBA9006); 2016Google Scholar
  12. 12.
    Denis F, Lethrosne C, Pourel N, Molinier O, Pointreau Y, Domont J et al. Randomized trial comparing a web-mediated follow-up with routine surveillance in lung cancer patients. Journal of the National Cancer Institute. 2017;109(9).
  13. 13.
    Berry DL, Hong F, Halpenny B, Partridge A, Fox E, Fann JR, Wolpin S, Lober WB, Bush N, Parvathaneni U, Amtmann D, Ford R (2014) The electronic self report assessment and intervention for cancer: promoting patient verbal reporting of symptom and quality of life issues in a randomized controlled trial. BMC Cancer 14:513. CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Ruland CM, Andersen T, Jeneson A, Moore S, Grimsbo GH, Borosund E et al (2013) Effects of an internet support system to assist cancer patients in reducing symptom distress: a randomized controlled trial. Cancer Nurs 36(1):6–17. CrossRefPubMedGoogle Scholar
  15. 15.
    Bennett AV, Dueck AC, Mitchell SA, Mendoza TR, Reeve BB, Atkinson TM et al. Mode equivalence and acceptability of tablet computer-, interactive voice response system-, and paper-based administration of the U.S. National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Health and quality of life outcomes. 2016;14:24.
  16. 16.
    Haynes RB, Ackloo E, Sahota N, McDonald HP, Yao X (2008) Interventions for enhancing medication adherence. The Cochrane database of systematic reviews 2:CD000011. CrossRefGoogle Scholar
  17. 17.
    Nieuwlaat R, Wilczynski N, Navarro T, Hobson N, Jeffery R, Keepanasseril A, Agoritsas T, Mistry N, Iorio A, Jack S, Sivaramalingam B, Iserman E, Mustafa RA, Jedraszewski D, Cotoi C, Haynes RB, Cochrane Consumers and Communication Group (2014) Interventions for enhancing medication adherence. The Cochrane database of systematic reviews 11:CD000011. CrossRefGoogle Scholar
  18. 18.
    Bennett AV, Jensen RE, Basch E (2012) Electronic patient-reported outcome systems in oncology clinical practice. CA Cancer J Clin 62(5):337–347. CrossRefPubMedGoogle Scholar
  19. 19.
    Basch E, Artz D, Iasonos A, et al (2007) Evaluation of an online platform for cancer patient self-reporting of chemotherapy toxicities. J Am Med Inform Assoc.14(3):264–268.
  20. 20.
    Kirkovits T, Schinkoethe T, Drewes C, Gehring C, Bauerfeind I, Harbeck N et al (2016) eHealth in modern patient-caregiver communication: high rate of acceptance among physicians for additional support of breast cancer patients during long-term therapy. J Med Internet Res Cancer(2):e14. CrossRefGoogle Scholar
  21. 21.
    Patmon FL, Gee PM, Rylee TL, Readdy NL (2016) Using interactive patient engagement technology in clinical practice: a qualitative assessment of nurses’ perceptions. J Med Internet Res 18(11):e298. CrossRefPubMedPubMedCentralGoogle Scholar
  22. 22.
    Hershman DL, Lacchetti C, Dworkin RH, Lavoie Smith EM, Bleeker J, Cavaletti G, Chauhan C, Gavin P, Lavino A, Lustberg MB, Paice J, Schneider B, Smith ML, Smith T, Terstriep S, Wagner-Johnston N, Bak K, Loprinzi CL, American Society of Clinical Oncology (2014) Prevention and management of chemotherapy-induced peripheral neuropathy in survivors of adult cancers: American Society of Clinical Oncology clinical practice guideline. J Clin Oncol 32(18):1941–1967. CrossRefPubMedGoogle Scholar
  23. 23.
    Cancer Therapy Evaluation Program. Common Toxicity Criteria (CTC). National Cancer Institute, Website National Cancer Institute. 2016.
  24. 24.
    Johnson CM, Johnson TR, Zhang J (2005) A user-centered framework for redesigning health care interfaces. J Biomed Inform 38(1):75–87. CrossRefPubMedGoogle Scholar
  25. 25.
    van de Poll-Franse LV, van Eenbergen MC (2008) Internet use by cancer survivors: current use and future wishes. Support Care CancerGoogle Scholar
  26. 26.
    Nuver J, Boer H, Bunskoek S, Siesling S, Berendsen AJ, Gietema JA (2013) Late effects of cancer treatment: shared care. Huisarts Wet 56(7):342–345CrossRefGoogle Scholar
  27. 27.
    Menning S, de Ruiter MB, Kieffer JM, Agelink van Rentergem J, Veltman DJ, Fruijtier A et al (2016) Cognitive impairment in a subset of breast cancer patients after systemic therapy-results from a longitudinal study. J Pain Symptom Manag 52(4):560–569 e1. CrossRefGoogle Scholar
  28. 28.
    Li J, Talaei-Khoei A, Seale H, Ray P, Macintyre CR (2013) Health care provider adoption of eHealth: systematic literature review. Int J Med Res 2(1):e7. CrossRefGoogle Scholar
  29. 29.
    Djamasbi S, Fruhling A, Loiacono E. The influence of affect, attitude and usefulness in the acceptance of telemedicine systems. J Inf Technol Theory Appl. 2009;10(1)Google Scholar
  30. 30.
    Nicolaije KA, Husson O, Ezendam NP, Vos MC, Kruitwagen RF, Lybeert ML et al (2012) Endometrial cancer survivors are unsatisfied with received information about diagnosis, treatment and follow-up: a study from the population-based PROFILES registry. Patient Educ Couns 88(3):427–435. CrossRefPubMedGoogle Scholar
  31. 31.
    Lamers RE, Cuypers M, Husson O, de Vries M, Kil PJ, Ruud Bosch JL et al (2016) Patients are dissatisfied with information provision: perceived information provision and quality of life in prostate cancer patients. Psycho-Oncology 25(6):633–640. CrossRefPubMedGoogle Scholar
  32. 32.
    Husson O, Mols F, Oranje WA, Haak HR, Nieuwlaat WA, Netea-Maier RT, Smit JWA, van de Poll-Franse LV (2014) Unmet information needs and impact of cancer in (long-term) thyroid cancer survivors: results of the PROFILES registry. Psycho-Oncology 23(8):946–952. CrossRefPubMedGoogle Scholar
  33. 33.
    Fletcher C, Flight I, Chapman J, Fennell K, Wilson C (2017) The information needs of adult cancer survivors across the cancer continuum: a scoping review. Patient Educ Couns 100(3):383–410. CrossRefPubMedGoogle Scholar
  34. 34.
    Basch E, Wood WA, Schrag D, Sima CS, Shaw M, Rogak LJ, Kris MG, Shouery M, Bennett A, Atkinson T, Pietanza MC (2016) Feasibility and clinical impact of sharing patient-reported symptom toxicities and performance status with clinical investigators during a phase 2 cancer treatment trial. Clin Trials 13(3):331–337. CrossRefPubMedGoogle Scholar
  35. 35.
    Basch E, Reeve BB, Mitchell SA, Clauser SB, Minasian LM, Dueck AC et al (2014) Development of the National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). J Natl Cancer Inst 106(9).
  36. 36.
    Bruner DW, Hanisch LJ, Reeve BB, Trotti AM, Schrag D, Sit L, Mendoza TR, Minasian L, O’Mara A, Denicoff AM, Rowland JH, Montello M, Geoghegan C, Abernethy AP, Clauser SB, Castro K, Mitchell SA, Burke L, Trentacosti AM, Basch EM (2011) Stakeholder perspectives on implementing the National Cancer Institute’s patient-reported outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Transl Behav Med 1(1):110–122. CrossRefPubMedPubMedCentralGoogle Scholar
  37. 37.
    Dueck AC, Mendoza TR, Mitchell SA, Reeve BB, Castro KM, Rogak LJ, Atkinson TM, Bennett AV, Denicoff AM, O’Mara AM, Li Y, Clauser SB, Bryant DM, Bearden JD 3rd, Gillis TA, Harness JK, Siegel RD, Paul DB, Cleeland CS, Schrag D, Sloan JA, Abernethy AP, Bruner DW, Minasian LM, Basch E, National Cancer Institute PRO-CTCAE Study Group (2015) Validity and reliability of the US National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). JAMA Oncol 1(8):1051–1059.
  38. 38.
    Hay JL, Atkinson TM, Reeve BB, Mitchell SA, Mendoza TR, Willis G et al (2014) Cognitive interviewing of the US National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Qual Life Res 23(1):257–269. CrossRefPubMedGoogle Scholar
  39. 39.
    Kluetz PG, Chingos DT, Basch EM, Mitchell SA (2016) Patient-reported outcomes in cancer clinical trials: measuring symptomatic adverse events with the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). American Society of Clinical Oncology educational book / ASCO American Society of Clinical Oncology Meeting 35:67–73. CrossRefGoogle Scholar
  40. 40.
    Atkinson TM, Ryan SJ, Bennett AV, Stover AM, Saracino RM, Rogak LJ, Jewell ST, Matsoukas K, Li Y, Basch E (2016) The association between clinician-based common terminology criteria for adverse events (CTCAE) and patient-reported outcomes (PRO): a systematic review. Support Care Cancer 24(8):3669–3676. CrossRefPubMedPubMedCentralGoogle Scholar
  41. 41.
    Basch E, Iasonos A, McDonough T, Barz A, Culkin A, Kris MG, Scher HI, Schrag D (2006) Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: results of a questionnaire-based study. Lancet Oncol 7(11):903–909. CrossRefPubMedGoogle Scholar
  42. 42.
    Cirillo M, Venturini M, Ciccarelli L, Coati F, Bortolami O, Verlato G (2009) Clinician versus nurse symptom reporting using the National Cancer Institute-Common Terminology Criteria for Adverse Events during chemotherapy: results of a comparison based on patient’s self-reported questionnaire. Ann Oncol 20(12):1929–1935. CrossRefPubMedGoogle Scholar
  43. 43.
    Coolbrandt A, Van den Heede K, Vanhove E, De Bom A, Milisen K, Wildiers H (2011) Immediate versus delayed self-reporting of symptoms and side effects during chemotherapy: does timing matter? Eur J Oncol Nurs 15(2):130–136. CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of ResearchNetherlands Comprehensive Cancer Organisation (IKNL)UtrechtNetherlands
  2. 2.Department of Medical and Clinical PsychologyTilburg UniversityTilburgNetherlands
  3. 3.Division of Psychosocial Research & EpidemiologyThe Netherlands Cancer InstituteAmsterdamNetherlands

Personalised recommendations