Understandable health information is essential for treatment adherence and improved health outcomes. For readability testing, several instruments analyze the complexity of sentence structures, e.g., Flesch-Reading Ease (FRE) or Vienna-Formula (WSTF). Moreover, the vocabulary is of high relevance for readers. The aim of this study is to investigate the agreement of sentence structure and vocabulary-based (SVM) instruments. A total of 52 freely available German patient information booklets on cancer were collected from the Internet. The mean understandability level L was computed for 51 booklets. The resulting values of FRE, WSTF, and SVM were assessed pairwise for agreement with Bland–Altman plots and two-sided, paired t tests. For the pairwise comparison, the mean L values are LFRE = 6.81, LWSTF = 7.39, LSVM = 5.09. The sentence structure-based metrics gave significantly different scores (P < 0.001) for all assessed booklets, confirmed by the Bland–Altman analysis. The study findings suggest that vocabulary-based instruments cannot be interchanged with FRE/WSTF. However, both analytical aspects should be considered and checked by authors to linguistically refine texts with respect to the individual target group. Authors of health information can be supported by automated readability analysis. Health professionals can benefit by direct booklet comparisons allowing for time-effective selection of suitable booklets for patients.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K (2011) Low health literacy and health outcomes: an updated systematic review. Ann Intern Med 155:97–107. https://doi.org/10.7326/0003-4819-155-2-201107190-00005
Coulter A, Ellins J (2007) Effectiveness of strategies for informing, educating, and involving patients. BMJ 335:24–27. https://doi.org/10.1136/bmj.39246.581169.80
Dirmaier J, Härter M (2012) Partizipative Entscheidungsfindung: Patientenbeteiligung bei Behandlungsentscheidungen in der medizinischen Versorgung [Shared decision making: patient involvement in treatment decisions in medical care]. BARMER GEK Gesundheitswesen Aktuell 212–235
Gorgojo L, Harris M, Garcia-Lopez E, Group CW (2012) National cancer control programmes: analysis of primary data from questionnaires Final Prelim Rep Eur Partnersh Action Cancer EPAACURL Httpwww Epaac Eunational-Cancer-Plans Accessed Novemb 2015
Bundesministerium für Gesundheit (2012) , Nationaler Krebsplan - Handlungsfelder, Ziele und Umsetzungsempfehlungen [National cancer plan—fields of action, objectives and implementation of recommendations]. Druckerei im Bundesministerium für Arbeit und Soziales, Berlin
Huebner J, Muenstedt K, Prott FJ, Stoll C, Micke O, Buentzel J, Muecke R, Senf B (2014) Online survey of patients with breast cancer on complementary and alternative medicine. Breast Care Basel Switz 9:60–63. https://doi.org/10.1159/000360381
Paul M, Davey B, Senf B, Stoll C, Münstedt K, Mücke R, Micke O, Prott FJ, Buentzel J, Hübner J (2013) Patients with advanced cancer and their usage of complementary and alternative medicine. J Cancer Res Clin Oncol 139:1515–1522. https://doi.org/10.1007/s00432-013-1460-y
Keinki C, Seilacher E, Ebel M, Ruetters D, Kessler I, Stellamanns J, Rudolph I, Huebner J (2015) Information needs of cancer patients and perception of impact of the disease, of self-efficacy, and locus of control. J Cancer Educ Off J Am Assoc Cancer Educ 31:610–616. https://doi.org/10.1007/s13187-015-0860-x
Huebner J, Micke O, Muecke R, Buentzel J, Prott FJ, Kleeberg U, Senf B, Muenstedt K, PRIO (Working Group Prevention and Integrative Oncology of the German Cancer Society) (2014) User rate of complementary and alternative medicine (CAM) of patients visiting a counseling facility for CAM of a German comprehensive cancer center. Anticancer Res 34:943–948
Flesch R (1948) A new readability yardstick. J Appl Psychol 32:221–233
Friedman DB, Hoffman-Goetz L (2006) A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ Behav off Publ Soc public Health Educ 33:352–373. https://doi.org/10.1177/1090198105277329
Amstad T (1978) Wie verständlich sind unsere Zeitungen? [How readable are our newspapers?]. Doctoral thesis, Universität Zürich, Switzerland
Bamberger R, Vanacek (1984) Lesen-Verstehen-Lernen-Schreiben [Reading-Comprehension-Learning-Writing]. Diesterweg
Liebl P, Seilacher E, Koester M-J, Stellamanns J, Zell J, Hübner J (2015) What cancer patients find in the internet: the visibility of evidence-based patient information—analysis of information on German websites. Oncol Res Treat 38:212–218. https://doi.org/10.1159/000381739
Keinki C, Zowalla R, Wiesner M, Koester MJ, Huebner J (2016) Understandability of patient information booklets for patients with cancer. J Cancer Educ Off J Am Assoc Cancer Educ. https://doi.org/10.1007/s13187-016-1121-3
Robert Koch-Institut, die Gesellschaft der epidemiologischen Krebsregister in Deutschland e.V. (2013) Krebs in Deutschland 2009/2010 [Cancer in Germany 2009/2010]. Robert Koch-Institut, Berlin
Leroy G, Miller T, Rosemblat G, Browne A (2008) A balanced approach to health information evaluation: a vocabulary-based naïve Bayes classifier and readability formulas. J Am Soc Inf Sci Technol 59:1409–1419. https://doi.org/10.1002/asi.20837
Zowalla R, Wiesner M, Pfeifer D (2014) Automatically assessing the expert degree of online health content using SVMs. Stud Health Technol Inform 202:48–51
Joachims T (1998) Text categorization with support vector machines: learning with many relevant features. In: Nédellec C, Rouveirol C (eds) Machine learning: ECML-98. Springer Berlin Heidelberg, pp 137–142
Salton G, Buckley C (1988) Term-weighting approaches in automatic text retrieval. Inf Process Manag 24:513–523. https://doi.org/10.1016/0306-4573(88)90021-0
Platt JC (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In: Advances in large margin classifiers, MIT Press pp 61–74
Bland JM, Altman DG (1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet Lond Engl 1:307–310
Bland JM, Altman DG (1999) Measuring agreement in method comparison studies. Stat Methods Med Res 8:135–160. https://doi.org/10.1177/096228029900800204
Hoefert H-W (2011) Wandel der Patientenrolle: neue Interaktionsformen im Gesundheitswesen [Change of the patient role: new forms of interaction in healthcare]. Hogrefe, Göttingen
Fagerlin A, Zikmund-Fisher BJ, Ubel PA (2011) Helping patients decide: ten steps to better risk communication. J Natl Cancer Inst 103:1436–1443. https://doi.org/10.1093/jnci/djr318
Butow P, Brindle E, McConnell D, Boakes R, Tattersall M (1998) Information booklets about cancer: factors influencing patient satisfaction and utilization. Patient Educ Couns 33:129–141
Cooley ME, Moriarty H, Berger MS, Selm-Orr D, Coyle B, Short T (1995) Patient literacy and the readability of written cancer educational materials. Oncol Nurs Forum 22:1345–1351
Garcia SF, Hahn EA, Jacobs EA (2010) Addressing low literacy and health literacy in clinical oncology practice. J Support Oncol 8:64–69
Nicholls S, Hankins M, Hooley C, Smith H (2009) A survey of the quality and accuracy of information leaflets about skin cancer and sun-protective behaviour available from UK general practices and community pharmacies. J Eur Acad Dermatol Venereol JEADV 23:566–569. https://doi.org/10.1111/j.1468-3083.2008.03017.x
Singh J (2003) Reading grade level and readability of printed cancer education materials. Oncol Nurs Forum 30:867–870. https://doi.org/10.1188/03.ONF.867-870
Weintraub D, Maliski SL, Fink A, Choe S, Litwin MS (2004) Suitability of prostate cancer education materials: applying a standardized assessment tool to currently available materials. Patient Educ Couns 55:275–280. https://doi.org/10.1016/j.pec.2003.10.003
Hasan M, Kotov A, Carcone A et al (2016) A study of the effectiveness of machine learning methods for classification of clinical interview fragments into a large number of categories. J Biomed Inform 62:21–31. https://doi.org/10.1016/j.jbi.2016.05.004
Müller K (2001) Automatic detection of syllable boundaries combining the advantages of treebank and bracketed corpora training. In: Proceedings of the 39th annual meeting on Association for Computational Linguistics Association for Computational Linguistics, pp 410–417
Conflict of Interest
The authors declare that they have no conflict of interest.
This article does not contain any studies with human participants performed by any of the authors.
About this article
Cite this article
Keinki, C., Zowalla, R., Pobiruchin, M. et al. Computer-Based Readability Testing of Information Booklets for German Cancer Patients. J Canc Educ 34, 696–704 (2019). https://doi.org/10.1007/s13187-018-1358-0
- Evidence-based health information
- Patient information booklets
- Support vector machine