Journal of Cancer Education

, Volume 33, Issue 4, pp 926–944 | Cite as

Accuracy and Readability of Websites on Kidney and Bladder Cancers

  • Samy A AzerEmail author
  • Maha M Alghofaili
  • Rana M Alsultan
  • Najla S Alrumaih


The aim of this study was to assess the scientific accuracy and the readability level of websites on kidney and bladder cancers. The search engines Google™, Yahoo™ and Bing™ were searched independently by assessors in November 2014 using the following keywords: “bladder cancer”, “kidney cancer”, “patient bladder cancer”, “patient kidney cancer” and “bladder and kidney cancer”. Only English-language websites were selected on the bases of predetermined inclusion and exclusion criteria. Assessors independently reviewed the findings and evaluated the accuracy and quality of each website by using the DISCERN and the LIDA instruments. The readability of the websites was calculated using the Flesch-Kincaid Grade Level Index and the Coleman-Liau Readability Index. Sixty-two websites were finally included in the study. The overall accuracy scores varied; for the DISCERN, the range was 28 to 76; out of 80 (mean ± SD, 47.1 ± 12.1; median = 46.0, interquartile range (IQR) = 19.2), and for the LIDA, the range was 52 to 125; out of 144 (mean ± SD, 101.9 ± 15.2; median, 103; IQR, 16.5). The creators of these websites were universities and research centres (n = 25, 40%), foundations and associations (n = 10, 16%), commercial and pharmaceutical companies (n = 13, 21%), charities and volunteer work (n = 4, 6%) and non-university educational bodies (n = 10, 16%). The readability scores (mean ± SD) were 11.2 ± 2.2 for the Flesch-Kincaid Grade Level Index and 11.2 ± 1.6 for the Coleman-Liau Readability Index. The accuracy and the quality of the websites on kidney and bladder cancers varied. In most websites, there were deficiencies in clarity of aims, presenting symptoms, investigations and treatment options. The readability matched grades 10–11 literacy levels—a level above the public readability level. The study highlights the needs for further improvement of the online information created for public and patients with kidney and bladder cancers.


Kidney cancer Bladder cancer The Internet Online resources Patients’ education 


Compliance with Ethical Standards


This work was funded by the College of Medicine Research Center, Deanship of Scientific Research, King Saud University, Riyadh, Saudi Arabia.

Institutional Review Board Statement

The Institutional Review Board (IRB), College of Medicine King Saud University, has approved the project, and the approval number is F05/2014.

Author Contribution

Azer SA participated in the study design and development and analysis and interpretation of data, construction of figures and tables and writing the manuscript. Alghofaili MM, AlSultan RM and Alrumaih NS participated in the data collection, data analysis, interpretation of data, construction of tables and review of the manuscript.

Conflict of Interest

The authors declare that they have no competing interests.

Data Sharing Statement

No additional data are available.


  1. 1.
    Miller DC, Ruterbusch J, Colt JS, Davis FG, Linehan WM, Chow WH, Schwartz K (2010) Contemporary clinical epidemiology of renal cell carcinoma: insight from a population based case-control study. J Urol 184(6):2254–2258CrossRefPubMedPubMedCentralGoogle Scholar
  2. 2.
    Capitanio U, Montorsi F (2016) Renal cancer. Lancet 387(10021):894–906CrossRefPubMedGoogle Scholar
  3. 3.
    Leibovich BC, Lohse CM, Crispen PL, Boorjian SA, Thompson RH, Blute ML, Cheville JC (2010) Histological subtype is an independent predictor of outcome for patients with renal cell carcinoma. J Urol 183(4):1309–1315CrossRefPubMedGoogle Scholar
  4. 4.
    Curti BD (2004) Renal cell carcinoma. JAMA 292(1):97–100CrossRefPubMedGoogle Scholar
  5. 5.
    Gofrit ON, Orevi M (2016) Diagnostic challenges of kidney cancer: a systematic review of the role of positron emission tomography-computerized tomography. J Urol 196(3):648–657CrossRefPubMedGoogle Scholar
  6. 6.
    Chang SS, Boorjian SA, Chou R, Clark PE, Daneshmand S, Konety BR, Pruthi R, Quale DZ, Ritch CR, Seigne JD, Skinner EC, Smith ND, McKiernan JM (2016) Diagnosis and treatment of non-muscle invasive bladder cancer: AUA/SUO guideline. J Urol 196(4):1021–1029CrossRefPubMedGoogle Scholar
  7. 7.
    Bellmunt J, Orsola A, Leow JJ, Wiegel T, De Santis M, Horwich A, ESMO Guidelines Working Group (2014) Bladder cancer: ESMO practice guidelines for diagnosis, treatment and follow-up. Ann Oncol 25(Suppl 3):iii40–iii48CrossRefPubMedGoogle Scholar
  8. 8.
    Freedman ND, Silverman DT, Hollenbeck AR, Schatzkin A, Abnet CC (2011) Association between smoking and risk of bladder cancer among men and women. JAMA 306(7):737–745CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Kiriluk KJ, Prasad SM, Patel AR, Steinberg GD, Smith ND (2012) Bladder cancer risk from occupational and environmental exposures. Urol Oncol 30(2):199–211CrossRefPubMedGoogle Scholar
  10. 10.
    Burger M, Catto JW, Dalbagni G, Grossman HB, Herr H, Karakiewicz P, Kassouf W, Kiemeney LA, La Vecchia C, Shariat S, Lotan Y (2013) Epidemiology and risk factors of urothelial bladder cancer. Eur Urol 63(2):234–241CrossRefPubMedGoogle Scholar
  11. 11.
    Fernandes ET, Manivel JC, Reddy PK, Ercole CJ (1996) Cyclophosphamide associated bladder cancer--a highly aggressive disease: analysis of 12 cases. J Urol 156(6):1931–1933CrossRefPubMedGoogle Scholar
  12. 12.
    Pandya E, Bajorek BV (2016) Assessment of web-based education resources informing patients about stroke prevention in atrial fibrillation. J Clin Pharm Ther 41(6):667–676CrossRefPubMedGoogle Scholar
  13. 13.
    Anderson JG, Rainey MR, Eysenbach G (2003) The impact of CyberHealthcare on the physician-patient relationship. J Med Syst 27(1):67–84CrossRefPubMedGoogle Scholar
  14. 14.
    Eysenbach G (2003) The impact of the Internet on cancer outcomes. CA Cancer J Clin 53(6):356–371CrossRefPubMedGoogle Scholar
  15. 15.
    Top 15 Most Popular Search Engines. Available at: . [Accessed 15 Nov. 2016].
  16. 16.
    Azer SA, AlOlayan TI, AlGhamdi MA, M AlSanea MA. Inflammatory bowel disease: an evaluation of health information on the Internet. World J Gastroenterol 2017 In Press.Google Scholar
  17. 17.
    Black PC, Penson DF (2006) Prostate cancer on the Internet—information or misinformation? J Urol 175(5):1836–1842CrossRefPubMedGoogle Scholar
  18. 18.
    Azer SA (2015) Is Wikipedia a reliable learning resource for medical students? Evaluating respiratory topics. Adv Physiol Educ 39(1):5–14CrossRefPubMedGoogle Scholar
  19. 19.
    Walsh TM, Volsko TA (2008) Literacy assessment of Internet based consumer healthcare information. Respir Care 53:1310–1315PubMedGoogle Scholar
  20. 20.
    National Institutes of Health. How to write easy to read health materials. National Library of Medicine Web site. Available at: [Accessed 15 Nov. 2016].
  21. 21.
    Azer SA (2014) Evaluation of gastroenterology and hepatology articles on Wikipedia: are they suitable as learning resources for medical students? Eur J GastroenterolHepatol 26(2):155–163CrossRefGoogle Scholar
  22. 22.
    Charnock D, Shepperd S, Needham G, Gann R (1999) DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 53:105–111CrossRefPubMedPubMedCentralGoogle Scholar
  23. 23.
    Danino J, Muzaffar J, Mitchell-Innes A, Howard J, Coulson C (2016) Quality of information available via the Internet for patients with otological conditions. Otol Neurotol 37(8):1063–1065CrossRefPubMedGoogle Scholar
  24. 24.
    Schreuders EH, Grobbee EJ, Kuipers EJ, Spaander MC, Veldhuyzen van Zanten SJ.Variable Quality and readability of patient-oriented websites on colorectal cancer screening. Clin Gastroenterol Hepatol. 2016 Jul 9.
  25. 25.
    Grewal P, Alagaratnam S (2013) The quality and readability of colorectal cancer information on the Internet. Int J Surg 11:410–413CrossRefPubMedGoogle Scholar
  26. 26.
    Yeung TM, D'Souza ND (2013) Quality analysis of patient information on surgical treatment of haemorrhoids on the Internet. Ann R Coll Surg Engl 95:341–344CrossRefPubMedPubMedCentralGoogle Scholar
  27. 27.
    Charnock D The DISCERN handbook. Quality criteria for consumer health information on treatment choices. Radcliffe Medical Press, Ltd, Oxon, United Kingdom Available at: [Accessed 15 November 2016].
  28. 28.
    Jayaweera JM, De Zoysa MI (2016) Quality of information available over Internet on laparoscopic cholecystectomy. J Minim Access Surg 12(4):321–324CrossRefPubMedPubMedCentralGoogle Scholar
  29. 29.
    Azer SA, Azer S (2016) Bibliometric analysis of the top-cited gastroenterology and hepatology articles. BMJ Open 6:e009889CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Kutner M, Greenberg E, Jin Y, Paulsen C. The health literacy of America’s adults: results from the 2003 National Assessment of Adult Literacy. Washington: Department of Education (US); 2006. Available at: URL: .asp?pubid=2006483 [Accessed 15 Nov. 2016].
  31. 31.
    Friedman DB, Hoffman-Goetz L (2006) A systematic review of read- ability and comprehension instruments used for print and web-based cancer information. Health Educ Behav 33:352–373CrossRefPubMedGoogle Scholar
  32. 32.
    Eltorai AE, Naqvi SS, Ghanian S, Eberson CP, Weiss AP, Born CT et al (2015) Readability of invasive procedure consent forms. Clin Transl Sci 8:830–833CrossRefPubMedPubMedCentralGoogle Scholar
  33. 33.
    Tian C, Champlin S, Mackert M, Lazard A, Agrawal D (2014) Readability, suitability, and health content assessment of web-based patient education materials on colorectal cancer screening. Gastrointest Endosc 80:284–290CrossRefPubMedGoogle Scholar
  34. 34.
    Azer SA, AlSwaidan NM, Alshwairikh LA, AlShammari JM (2015) Accuracy and readability of cardiovascular entries on Wikipedia: are they reliable learning resources for medical students? BMJ Open 5:e008187CrossRefPubMedPubMedCentralGoogle Scholar
  35. 35.
    Polit DF, Beck CT (2010) Essentials of nursing research: appraising evidence for nursing practice, 7th edn. Lippincott Williams & Wilkins, Wolters KluwerGoogle Scholar
  36. 36.
    Powell JA, Darvell M, Gray JAM (2003) The doctor, the patient and the world-wide web: how the Internet is changing healthcare. J R Soc Med 96:74–76CrossRefPubMedPubMedCentralGoogle Scholar
  37. 37.
    Cline RJ, Haynes KM (2001) Consumer health information seeking on the Internet: the state of the art. Health Educ Res 16(6):671–692Google Scholar
  38. 38.
    Best J, Muzaffar J, Mitchell-Innes A (2015) Quality of information available via the Internet for patients with head and neck cancer: are we improving? Eur Arch Otorhinolaryngol 272(11):3499–3505Google Scholar
  39. 39.
    Patient information forum. Making the case for information. The evidence for investing in high quality health information for patients and the public. The full report. The patient information forum. May 2013. Available at: [Accessed 15 Nov. 2016].
  40. 40.
    Morgan T, Schmidt J, Haakonsen C, Lewis J, Della Rocca M, Morrison S et al (2014) Using the Internet to seek information about genetic and rare diseases: a case study comparing data from 2006 and 2011. JMIR Res Protoc 3:e10CrossRefPubMedPubMedCentralGoogle Scholar
  41. 41.
    Carlsson T, Bergman G, Karlsson A-M, Mattsson E (2015) Content and quality of information websites about congenital heart defects following a prenatal diagnosis. Interactive Interact J Med Res 4(1):e4Google Scholar
  42. 42.
    Goslin RA, Elhassan HA (2013) Evaluating Internet health resources in ear, nose, and throat surgery. Laryngoscope 123(7):1626–1631CrossRefPubMedGoogle Scholar
  43. 43.
    Fast AM, Deibert CM, Hruby GW, Glassberg KI (2013) Evaluating the quality of Internet health resources in pediatric urology. J Pediatr Urol 9(2):151–156CrossRefPubMedGoogle Scholar
  44. 44.
    Borgmann H, Wölm JH, Vallo S, Mager R, Huber J, Breyer J, Salem J, Loeb S, Haferkamp A, Tsaur I 2017. Prostate cancer on the web-expedient tool for patients’ decision-making? J Cancer Educ 32(1):135–140.Google Scholar
  45. 45.
    Paasche-Orlow MK, Taylor HA, Brancati FL (2003) Readability standards for informed-consent forms as compared with actual readability. N Engl J Med 348:721–772CrossRefPubMedGoogle Scholar

Copyright information

© American Association for Cancer Education 2017

Authors and Affiliations

  • Samy A Azer
    • 1
    • 2
    • 3
    Email author
  • Maha M Alghofaili
    • 1
  • Rana M Alsultan
    • 1
  • Najla S Alrumaih
    • 1
  1. 1.Curriculum Development and Research Unit, College of MedicineKing Saud UniversityRiyadhSaudi Arabia
  2. 2.Curriculum Development and Research Unit, Medical Education Department, College of MedicineKing Saud UniversityRiyadhSaudi Arabia
  3. 3.Australian Professional TeachingMelbourneAustralia

Personalised recommendations