Introduction

Skin cancer is the commonest malignancy worldwide [1], with 1 in 5 Americans developing skin cancer in their lifetime [2]. The incidence of basal cell carcinoma has risen yearly [3], with a predicted doubling in the next 30 years, along with a fall in the average age of first diagnosis [4]. Skin cancer affects those from all socioeconomic, education, literacy and geographical backgrounds, and in many developed nations is predicted to cause a significant public health dilemma.

Treatment for skin cancer is wide ranging, and while surgical excision with a margin of non-involved tissue is typically favoured in Europe and the USA [5], many other options for non-melanoma skin cancer exist, such a topical chemotherapy [6], radiotherapy [7] and in some cases laser treatment [8]. In today’s era of shared decision-making between patient and clinician, all feasible options should be discussed with the patient [9]. To help facilitate this decision-making process, information should be given by health care staff. However, many patients also seek information elsewhere, often from online sources.

The scale of online health consumerism is huge, with approximately 37% of all Internet traffic related to health information [10]. There are estimated to be 3.6 billion Internet users, accounting for 49% of the Earth’s population [11]. Around 80% of those in the USA who have access to the Internet have used it to access health information [12].

While the global advancement of health care information available online has brought many benefits at local, regional and national levels [13], it is only of use to the individual consumer if they are able to comprehend the information available. Many, however, do not; in a 2006 study of American citizens, 25% of people questioned felt ‘overwhelmed’ when using the Internet for health related information [14].

One reason for this feeling of being ‘overwhelmed’ relates to poor health literacy within the population. Health literacy has been defined as ‘The degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions’ [15]. A major barrier to adequate health literacy is poor reading ability. In the United Kingdom, 1 in 6 people have general literacy levels below that expected of a 6th-grade student (UK 11-year-old) [16], while in the USA 45 million adults (or just over 1 in 5) are functionally illiterate as they read below the fifth-grade (10-year-old) level.

This degree of inadequate health literacy is a significant public health concern. Poor health literacy has been associated with poorer overall health status, higher mortality and decreased ability to engage in shared-decision making with regards to health management [17,18,19]. In fact, almost any health outcome that one cares to investigate has a positive correlation between the worsening of that metric and lower health literacy [20]. As a result of this correlation between health literacy and overall health outcomes (and the obvious link between reading ability and health literacy), the US Department of Health and Human Services (USDHHS) has recommended that all patient-orientated health literature be written at or below the 6th-grade level, equating to a UK reading age of 11–12 years [21]. There is no such guidance in the UK and therefore, as with other papers [22], the USDHHS guidelines are taken as the gold standard.

The skin cancer population is diverse and many will turn to the Internet for further information. The aim of this study was therefore to identify the most frequently accessed online resources relating to the treatment of skin cancer of any type and assess their readability. While others have explored general patient information for melanoma [23], this is, to our knowledge, the first time information specifically pertaining to the treatment options for all skin cancers has been assessed.

Methods

Searches were performed on the three largest Internet search engines (Google, Bing and Yahoo) [24], which are used for approximately 90% of all Internet searches. Previous studies have shown that the search terminology can have a significant impact on search outcomes and therefore the likely readability and accuracy of the information [25]. To avoid bias towards more scientific websites we used the search term ‘skin cancer treatment’ as would be used by a member of the lay public. This term was trialed on lay acquaintances of the study team to make sure it was an appropriate search term. The search was performed on all three search engines on the same day by two independent investigators. All location and account information was disabled on each computer and the Internet search history and cookies deleted between each search to avoid inadvertent search bias. Each investigator then independently documented the top 20 non-sponsored results as they appeared on each search engine.

Once the top 20 websites had been identified from each search engine they were tabulated in Excel (Microsoft Excel for Mac, Microsoft Corp, Redmond, WA) and given a reverse score from 20 to 1 (therefore the top website was given a score of 20 and the last website a score of 1). Scores were combined across search engines to give a ranking of websites based on their cumulative score. From this the top 10 websites were selected for further analysis. The reason for selecting 10 websites was based on this being the average number of non-sponsored hits for the first page of a search engine and it having previously been shown that 92% of people look only at the first page of results [26].

All 10 websites were then accessed one-by-one and searched for information relating to the treatment of skin cancer. Articles were defined as anything with information pertaining to the treatment of skin cancer; all articles that were linked to the original website were accessed. Each article was downloaded exactly as it appeared and pasted into Microsoft Word 2011 for Mac (Microsoft Corp, Redmond, WA, USA), with a separate document used for each website. Only English language websites and articles were included.

Each document was then edited, using a standardised set of procedures. All non-text features—images, figures, hyperlinks, references, non-standard symbols such as copyright and trademark symbols, and advertisements—were removed. Passages of text that were repeated on different web pages of a single website were also removed to prevent re-analysing the same passage when random samples of text were analysed. Further editing, as recommended by Flesch and others [27], was undertaken to avoid over-estimation. Thus, headings, bullet points, abbreviations, colons, semi-colons and dashes were all removed. Any typographical errors were also corrected.

Each text was then analysed for reading difficulty using a variety of techniques; these are summarized in Table 1. These techniques vary and some use the whole text to form an estimate of readability. Others use sampling where multiple random sections of text, usually 100 words in length, were then analysed using a set of readability scores, calculated with standardised formulas (Table 1), using the following online calculators (http://www.readabilityformulas.com/flesch-reading-ease-readability-formula.php and http://www.readabilityformulas.com/free-fry-graph-test.php). It has been demonstrated that for health-related literature where 100% comprehension is the goal, a combination of two or more formulae should be used, including the SMOG [28]. Most give a predicted ‘grade score’ of readability—the level of education, based on the year group system used in public schools in the United States, expected for the text to be understood.

Table 1 Readability formulae used in the assessment of online patient-orientated skin cancer treatment literature
Full size table

It is also possible to calculate the lexical sophistication of these texts. Lexical sophistication is a measure of the percentage of infrequent words in a text, with an infrequent word usually considered to be a word not included in the most frequent 2000 word families as identified in reliable frequency lists. A high percentage of infrequent words can make a text difficult to understand and The Guardian newspaper, which is considered by many university level students to be a difficult read, scores approximately 20% on this measure [29]. Lexical sophistication has been calculated using Cobb’s online profiling program, https://www.lextutor.ca/, which is now a standard tool in making such calculations and is based on British Nation Corpus (BNC) frequency information [30] and the British National Corpus and the Corpus of Contemporary American English (BNC/COCA) lists.

All data were analysed using SPSS for Mac V22 licensed to Swansea University. Mann–Whitney U test for nonparametric data was used for statistical comparison. Results are presented as median values with range. This article does not contain any new studies with human or animal subjects and was reviewed by our institutional review board.

Results

The top 10 websites as averaged across Google, Bing and Yahoo were: cancerresearchuk.org; cancer.gov; cancer.org; www.webmd.com; www.nhs.uk; www.skincancer.org; www.macmillan.org.uk; www.verywell.com; www.medicinenet.com; www.bupa.com. Of these, four are the websites of commercial companies, while the remaining six relate to not-for-profit organizations (Table 2). This did not make any significant difference to the reading difficulty of the website, with the median grade score for all commercial websites being 10.5 and 10.0 for not-for-profit sites (U = 10.00, Z = −0.430, p = 0.667).

Table 2 Top 10 websites in order of most popular across Google, Yahoo and Bing searches. Source of funding and total number of words per website also shown
Full size table

A total of 91 pages were accessed across all 10 websites. After editing of the text had been performed, as per the strategy outlined in the methods, the median word count was 4046, with a range of 522–13,622.

Table 3 demonstrates the approximate US school grade and UK reading age for each website, calculated from pooled scores across all readability formulae used. The corresponding USDHHS classification of difficulty is also shown. It can be seen from these data that none of the top 10 websites achieves the recommended 6th grade or below, with many considered ‘difficult’ by the USDHHS classification.

Table 3 Readability scores for each website using the Gunning Fog, Flesch-Kincaid Grade Level, Coleman-Liao Index, SMOG Index, Automated Readability Index and Linsear Write Formula
Full size table

When the individual formulae are looked at in detail there is, as expected, a range of grade scores for the same website depending on the formulae used. The Flesch-Kincaid Grade Level Formula, designed to evaluate technical documents, demonstrated a median grade of 9.65 (7.2–14.5) across all websites. The Gunning Fog median was 12.25 (9.5–17.4). The Coleman-Liao Index showed a grade median of 10.5 (8–14). The Automated Readability Index and Linsear Write Formula grade medians were 9.5 (6.5–12.7) and 11.05 (7.3–15.9), respectively. The SMOG index is a validated formula that accurately measures the reading grade required for full text comprehension. SMOG analysis revealed a grade median across all websites of 9.0 (6.9–12.7), Table 3. The median grade score and range across these six readability formulae is shown graphically in Fig. 1.

Fig. 1
figure 1

Median grade level for individual websites across six readability formulae (Gunning Fog, Flesch-Kincaid Grade Level, Coleman-Liao Index, SMOG Index, Automated Readability Index and Linsear Write Formula). It can be seen that all websites are above the USDHHS recommended 6th grade level

Full size image

The Flesch Reading Ease formula produces an index score, rather than a US reading grade, ranging from 0 to 100. A score of 100 is the easiest to read, with lower scores progressively more difficult. There is an approximately inverse relationship with the grade score on the similarly designed Flesch–Kincaid Grade Level. The median Flesch Reading Ease score for all the articles in this series was 55.7 (30.7–68.3), which correspond to around the 10th–12th grade level.

The Fry Graph Readability formula was used to calculate a score for all the articles on or linked to each website. These are depicted in Fig. 2, where the average number of sentences per 100 words is plotted against the average number of syllables per 100 words. It can be seen that once again this score puts all of the sites above the 6th grade level, with two sites calculated as requiring a reading grade above the collegiate level.

Fig. 2
figure 2

Fry Readability Graph. The dots represent each individual website with the average number of sentences per 100 words plotted against the average number of syllables per 100 words. All can be seen to be over the 6th grade level

Full size image

Finally, the percentage of words that fall outside the top 2000 has been calculated for each website (Table 4). Looking at the values for websites discussing the treatment of skin cancer, only three out of ten fall below the 20% figure of The Guardian, with the majority being higher. This again lends further evidence that all of the websites sampled in this study are written to a level higher than the recommended 6th grade and significantly harder than many people can manage to read.

Table 4 Percentage of words that fall outside of the commonest 2000 word families based on the British National Corpus (BNC) and the Corpus of Contemporary American English (COCA)
Full size table

Discussion

Use of the Internet to access health-related literature has become commonplace [12]. However, the information available is of use only if patients and their relatives are able to comprehend it. This degree of comprehension is dependent on the relationship between the patient’s literacy level and the readability of the provided information. There is a wide variation in literacy levels in the general population [16, 31], strongly connected with the number of years spent in education. A large proportion of populations will struggle to handle text of a technical or specialist nature unless it is carefully written to be accessible to a non-specialist audience. Therefore, it is recommended by the USDHHS that all patient information be written at or below the 6th grade level (UK age 11–12) [21]. In this study of the most popular online resources describing the treatment of skin cancer, none were found to be written at this recommended level. Furthermore, many were written significantly above this, with two having a readability level in keeping with university-level reading ability. These results are in keeping with many other studies, which have shown that online patient information in breast cancer [32], cleft lip and palate [33], neurosurgery [34] and stroke [22] are all written above the recommended 6th grade level. It is interesting to note that the problem is therefore not condition specific and has not been improving with time, despite increased awareness and reporting of the issue.

There is a need to improve the readability of patient-orientated literature across all medical domains, but how can this be done? If one looks at the formulae used for obtaining a readability score, most involve a combination of sentence length, word length and syllables per word. In reducing these parameters, the reading age of a piece of text can be reduced. However, this may not always improve understanding in the context of health literacy. Health literature is made up of a number of complex, often polysyllabic words, often for which no simpler word exists. In those formulae where syllable count is used, the use of medical terminology could significantly increase the reading level score despite the remainder of the text being simplified. This is likely to be one reason why the readability scores were so poor in this study. A study of printed patient information leaflets from a family medicine clinic in the USA looked at the difference in readability scores before and after removing medical terminology [35]. It was demonstrated using the Fry and SMOG formulae that there was a significant reduction in reading age after removal of medical terminology, as well as there being on average between 3.2 and 4.3 syllables per medical term [35]. Another finding was that the medical terms that were used were generally poorly explained, with one brochure only explaining 9% of the included medical terms. It is for this reason that the Centre for Health Care Strategies recommends that medical terms be replaced by simpler explanatory words [36], while it is also important that if medical terms cannot be removed that they are defined within the accompanying material.

It is, however, important to remember that removal of all medical terminology or replacement with simple words is impractical and unrealistic. It is probable that to do so would result in the whole meaning of the text being lost and it no longer imparting useful information to the patient. Furthermore, it is important in today’s world of shared decision making that patients and clinicians be able to understand one another and use the same terminology. If none of the words used to describe a condition or its treatment are used in online information, patients will not be able to understand and communicate with their treating clinicians. It is also interesting to note that not all multisyllabic medical terms are complex and unfamiliar to patients. For example the word ‘operation’ is a four-syllable word, which is likely to be recognized and understood by most lay people. However, there are many shorter or seemingly less grammatically complex words that it is likely that a layperson will never have come across. For example, BCC is an example of an acronym, which is a short word and would therefore ‘score well’, however it is likely to be incomprehensible to those who are not medically trained. It is with this in mind that one should be cautious about removing all multi-syllable medical words from a piece of text just to improve the score it achieves on a readability assessment. Instead, not only should the whole document be written in simple, short sentences, but any medical terminology that needs to be used to covey the appropriate message should be backed up with a well designed and coherent definition. We would therefore advocate the use of a glossary of terms in conjunction with any written medical text, especially in web-based applications where this can be easily presented. This and other recommendations are included in the plain language initiative which aims to make it easier for people to find what they need in a piece of text, understand it and then act appropriately on it [37].

Another avenue to improving understandability of a piece of written text could be through the novel use of audiovisual material. No specific measure was made of the number of audiovisual clips that were removed during the processing phase of this study, however relative to the volume of text on each site, the number was low. Audiovisual material could, however, hold the key to improving health literacy for those accessing this information via the Internet.

Our study is limited by a number of factors. While we used a more comprehensive range of readability formulae than most other studies, the resulting outputs of these formulae cannot completely represent the understandability of the information. What is important, however, is that the scores of different websites can be compared with one another and that the results from different studies are comparable. This study did not assess the quality, accuracy or relevance of the information published. While other studies have looked into this [38, 39] it was not the aim of this work. Furthermore, previous studies have demonstrated that the quality of information was variable and did not correlate with its readability [40]. While the accuracy of the information is undeniably important, it is of little value if it is incomprehensible to those patients reading it due to being written at too high a reading level. To overcome some of the issues with readability formulae further work could be done in which a cohort of lay people were asked to review the above websites for ease of reading and information retrieval, providing ‘real-life’ data.

Finally, the aim of our search term was to replicate a typical layperson’s search for skin cancer treatment related information, thus trying to avoid bias towards more scientific articles as noted in a previous study [25]. We, however, acknowledge that altering the search terms used will alter the results and may therefore prioritize websites with a higher or lower readability scores.

Conclusion

We have shown in this study that the information that patients will be likely to access is generally written above the desired readability level. There is also a considerable lottery as to the website which is picked, with a range of reading grades from 7 to 15 (UK ages 11 to University) noted. We hope that by further raising the issue of poor adherence to guidelines on the reading age to which patient-orientated information should be written, it will encourage those writing information on skin cancer treatment to critically appraise their work with regard to this criterion before publishing it. As discussed above, writing medical information at the 6th grade level while maintaining its meaning and usefulness is not an easy task, however the importance of aiming to achieve this goal in order to reduce health inequality is paramount.