Introduction

Health information helps people to understand aspects of their own health and engage in self-management. The recent developments in information technology have provided the general public with free access to a wide range of health information. However, the efficacious use of this information requires that individuals are health literate, i.e., that they possess adequate cognitive and social skills underlying the motivation and ability of individuals to gain access to, understand, and use information in ways which promote and maintain good health [1]. Low literacy has negative impacts on various patient behaviors and health outcomes [2–4], and health literacy (HL) is now recognized as a key factor in terms of both clinical “risk” and personal “assets” [5].

A number of tools for measuring HL have been developed and used in research studies. Most of these were designed to assess reading comprehension and numeracy in English speakers [6, 7]. The concept of HL has recently begun to attract notice in Japan, where nearly 100 % of the population over the age of 15 can read and write the Japanese language. There have been some attempts to develop self-rated scales for measuring HL in Japanese speakers [8–12]. One of these, developed by Ishikawa and colleagues [8], has the advantage of dealing with all three levels of HL: (1) functional literacy—sufficient basic skills in reading and writing to be able to function effectively in everyday situations, a definition which is broadly compatible with the narrow definition of HL; (2) communicative literacy—more advanced skills to participate actively in everyday activities, to extract information and derive meaning from different forms of communication, and to apply new information to changing circumstances; (3) critical literacy—more advanced skills to analyze information critically and to use this information to exert greater control over life events and situations [13]. Unfortunately, the HL scale is not generic, rather it is specific to diabetic patients. In order to develop a generic HL measure for Japanese adults, we adapted the HL scale specific to diabetic patients developed by Ishikawa and colleagues [8] for use with various people. We assessed the reliability and validity of our HL scale among middle-aged Japanese men and women.

Methods

Subjects

A questionnaire survey was conducted between September and December 2010 at the Japanese Red Cross Kumamoto Health Care Center. About 2,000 participants in multiphasic health examinations were asked to fill out a questionnaire anonymously. Details of the questionnaire survey have been described elsewhere [14]. The study protocol was approved by the ethics committees of the Japanese Red Cross Kumamoto Healthcare Center and Daito Bunka University, and the study was conducted in accordance with the Guidelines for Epidemiological Studies by the Ministry of Health, Labour, and Welfare and the Ministry of Education, Culture, Sports, Science, and Technology.

A total of 1,978 participants who agreed to participate in the questionnaire survey were given the questionnaire to complete then and there. After the exclusion of those who did not answer age or gender questions (N = 105) and those who (had) worked in the medical and pharmaceutical fields (N = 166), 1,707 subjects (1,056 men and 651 women) remained. Of these, those aged less than 30 years (N = 4) and those aged ≥70 years (N = 81) were excluded because there was an insufficient number of subjects available in these age categories. Among the remaining 1,622 subjects (998 men and 624 women) aged 30–69 years, the 1,507 subjects (946 men and 561 women) who completely filled out the HL scale were included in this study.

Measures

The questionnaire consisted of three parts. In the first part, the participants were asked about their experiences and perspectives on seeking medicine information and making medication decisions. In the second part, their HL was measured using the 14-item health literacy scale (HLS-14) that was adapted from the HL scale specific to diabetes patients developed by Ishikawa and colleagues [8]. In the third part, their quality of life was measured using the SF-8 Japanese version. The answers to the questions in the second part, together with those in the first part, were analyzed in this study.

The HLS-14 is shown in the “Appendix”. It contains five items pertaining to functional HL, five items pertaining to communicative HL, and four items pertaining to critical HL. The 14 items are the same as those on the original HL scale [8], while the interrogative sentences have been modified so as not to be specific to diabetic patients. Response options have been changed from a 4-point scale that indicates how often the item happens to them (‘never’ to ‘often’) [8] to a 5-point scale that indicates how much the respondent agrees or disagrees with the item (‘strongly disagree’ to ‘strongly agree’) [9]. The scores on the items were summed up for each respondent to give the total HL score, as well as functional, communicative, and critical HL scores. Higher scores indicate a better HL.

Statistical analysis

Reliability

Explanatory factor analysis with promax rotation was performed to determine the factor structure of the HLS-14. Factor loadings of ≥0.4 were considered to be appropriate. Internal consistency was assessed by Cronbach’s alpha, where a value of ≥0.7 was considered satisfactory [15, 16]. The percentages of subjects with the lowest and highest scores were calculated to identify floor and ceiling effects. If the percentage was >15 %, the effect was considered to be present [16].

Validity

Confirmatory factor analysis was performed to assess the construct validity of the HLS-14. Model fitness was assessed by the comparative fit index (CFI), normed fit index (NFI), and root mean square error of approximation (RMSEA). For CFI and NFI, a value closer to 1 indicates better fit, and for RMSEA, a value of <0.10 is considered to be acceptable [17].

To examine whether the HL scores were associated with experiences and perspectives on seeking medicine information and making medication decisions, the study subjects were divided into two groups according to a total HL score of above or below the median (50). The percentages of affirmative answers were compared between the higher and lower scoring groups using the chi-square test. Age and gender differences in the HL scores were examined using two-way analysis of variance (ANOVA).

All statistical analyses except for the confirmatory factor analysis were performed using SAS ver. 9.2 software (SAS Institute, Cary, NC). Confirmatory factor analysis was performed using the IBM SPSS Amos ver. 20.0 (IBM Corp, Armonk, NY). Significant levels were set at p < 0.05.

Results

Subjects

Of the 1,622 nonmedical respondents aged 30–69 years, 1,507 (92.9 %) completely filled out the HLS-14 questionnaire. The response rate for each item ranged from 96.4 % (Q12: I consider whether the information is credible) to 99.5 % (Q2: The print is too small for me). Table 1 shows the characteristics of the study subjects.

Table 1 Characteristics of the study subjects
Full size table

Reliability

The explanatory factor analysis revealed a factor structure that was very similar to that of the original HL scale [8]. Table 2 shows the factor structure of the HLS-14. The initial factor solution indicated three factors with eigenvalues of 3.86, 2.53, and 0.71, respectively, which jointly accounted for 109 % of the total variance. The promax rotation indicated that all five items for functional HL loaded on the second factor and that all five items for communicative HL loaded on the first factor. Among the four items for critical HL, three loaded on the third factor; the remaining item ‘Q11: I consider whether the information is applicable to me’ loaded mainly on the first factor, with a factor loading of <0.4 on the third factor.

Table 2 Factor structure of the 14-item health literacy scale (HLS-14)
Full size table

Table 3 shows the internal consistency and floor and ceiling effects of the HLS-14. The internal consistency of each HL score, as indicated by Cronbach’s alpha, was satisfactory. The total correlations for each set of items ranged from 0.44 to 0.76 for the functional HL score, from 0.58 to 0.72 for the communicative HL score, and from 0.45 to 0.71 for the critical HL score. When the item ‘Q11: I consider whether the information is applicable to me’ was removed from the critical HL items, Cronbach’s alpha of the critical HL score, as well as that of the total HL score, decreased by 0.01–0.02. Similar to the original HL scale [8], we decided to count this item among the critical HL items. There were no floor or ceiling effects in each HL score.

Table 3 Internal consistency and floor and ceiling effects of the 14-item health literacy scale (HLS-14)
Full size table

Validity

The confirmatory factor analysis revealed an acceptable fit of the three-factor model, with an CFI = 0.912, NFI = 0.905, and RMSEA = 0.082 (90 % confidence interval 0.078–0.088). Figure 1 shows the path diagrams of the confirmatory factor model. In the preliminary analysis, no significant correlation was found between the functional HL score and the critical HL score (γ = 0.00, p = 0.90). Therefore, the correlation between functional HL and critical HL was not incorporated into the confirmatory factor model. Standardized factor loadings ranged from 0.46 (Q5: I need someone to help me read them) to 0.89 (Q3: The content is too difficult for me). There was a positive correlation between the communicative HL score and the critical HL score, while the functional HL score was not correlated with the communicative HL score.

Fig. 1
figure 1

Path diagrams of the confirmatory factor model. Rectangles Observed variables (items), ellipses latent variables (factors), values on the single-headed arrows standardized factor loadings, values on the double-headed arrows correlation coefficients. Model fitness: comparative fit index = 0.912, normed fit index = 0.905, root mean square error of approximation = 0.082 (90 % confidence interval 0.078–0.088)

Full size image

Table 4 shows the comparison between the higher and lower scoring groups in seeking medicine information and making medication decisions. Compared with the lower scoring group, the higher scoring group tended to use more than one information source for seeking medicine information. Those who could obtain all the information they want and those who had never seen unknown medical words at hospitals or pharmacies were more frequently observed in the higher scoring group. Those who wanted their views taken into account in medication decisions and those who preferred to choose between various alternative medicines accounted for 66 and 75 % of subjects, respectively, in the higher scoring group and 50 and 58 %, respectively, of subjects in the lower scoring group.

Table 4 Comparison between the higher and lower scoring groups in seeking medicine information and making medication decisions
Full size table

Table 5 shows the age and gender differences in the HL scores. Two-way ANOVA revealed significant age and gender differences in the total, functional, communicative, and critical HL scores. When the comparison test was performed by age group, women had significantly higher scores than men in all age groups except for the age groups 50–59 and 60–69 years for the functional HL score.

Table 5 Age and gender differences in the health literacy scores
Full size table

Discussion

It is essential to measure HL in target populations to offer health information according to their HL level. We propose a generic HL measure for Japanese adults that is based on the HL scale specific to diabetic patients developed by Ishikawa and colleagues [8]. Most existing tools for measuring HL focus on reading comprehension and numeracy (i.e., functional HL) in English speakers [6, 7]. To our knowledge, our HL scale (the HLS-14) is a unique tool that aims to measure functional, communicative, and critical HL in both the clinical and public health contexts which has been proven to have adequate reliability and validity among middle-aged Japanese men and women.

Explanatory factor analysis produced a three-factor solution and confirmatory factor analysis revealed an acceptable fit of the three-factor model. The internal consistency of each HL score was satisfactory. These results confirm that the 14 items on the HLS-14 represent functional, communicative, and critical HL as originally designed. In the explanatory factor analysis, the item ‘Q11: I consider whether the information is applicable to me’ loaded both on the first factor, indicating communicative HL, and on the third factor, indicating critical HL. A similar result was obtained for the original HL scale [8]. This item is counted among the critical HL items, but it may measure advanced skills that combine communicative HL and critical HL. Overall, the HLS-14 has a successful structure for measuring HL at three different levels (i.e., functional, communicative, and critical HL) [13].

The mean score on each item tended to be higher for the functional HL items than for the communicative HL items and the critical HL items (Table 2). Functional HL is defined as basic skills, while communicative HL and critical HL are defined as advanced skills [13]. The higher scores on functional HL items are consistent with the definitions of three levels of HL. As shown in the confirmatory factor model, the functional HL score was not correlated with the communicative HL score or the critical HL score (Fig. 1). This result suggests that the measurement of functional HL cannot substitute for the measurement of communicative HL and critical HL, although communicative HL and critical HL must be based on functional HL. Due to a lack of tools for measuring communicative HL and critical HL, epidemiological data on these advanced skills are scarce. The use of HLS-14 may contribute to promoting a better understanding of advanced skills beyond reading comprehension and numeracy.

The item ‘Q5: I need someone to help me read them’ had the minimum factor loading among the 14 items in the confirmatory factor model. The score on this item showed a skewed distribution, with a mean of 4.5 on a 5-point scale, whereas no ceiling effect was found in the functional HL score. The study subjects were recruited from participants in multiphasic health examinations, who may be healthier and less impaired in reading and writing skills than clinical patients. The skewed distribution of scores may also be due partly to the study setting. This item is commonly used as a single-item screener to identify patients with inadequate HL [6, 7]. However, according to our results, caution should be taken when applying the single-item screener to relatively healthy people in a public health context or using it in a well-educated population like the Japanese.

The HLS-14 is relatively a short questionnaire and easy to use. Because of the self-administered character of the questionnaire (interviews are not required), it easy to use in a large-scale survey in both the clinical and public health contexts. The response rate for each item ranged from 96.4 to 99.5 %, indicating that the HLS-14 was well accepted. The <100 % response rate may be due to the voluntary response character of the questionnaire survey. Another possible explanation is that people without experience of suffering from a disease may have found some items difficult to answer. A qualitative interview survey among Dutch patients suggested the need for revisions in the Dutch version of the HL scale [18]. Similarly, a qualitative assessment of the questionnaire may help improve the questionnaire to achieve a 100 % response rate.

When the subjects were categorized into two groups based on the total HL score of above and below the median (50), those who could obtain medication information satisfactorily and those who wanted to participate in making medication decisions were more frequently observed in the higher scoring group. This result suggests the possibility that the use of HLS-14 may help identify individuals with a potential ability to share in decision-making [19]. The HL scores showed significant differences between age groups, but there was no age-related trend between age 30 and 69 years. Women had significantly higher scores than men independently of age. It is known that women are more sensitive to discomfort and more inclined to report discomforts to family, friends, and professionals, whereas men are more reluctant to seek help [20–22]. A better awareness of symptoms and more willingness to seek help in women may increase their access to health information, and subsequently, may contribute to the development of HL. Further studies using the HLS-14 should be conducted to confirm the impact of HL on health behavior and to identify influential factors in the development of HL.

This study provides evidence for the reliability and validity of the HLS-14; however, it has a number of potential limitations. First, popular HL measures, such as the Rapid Estimate of Adult Literacy in Medicine (REALM) and the Test of Functional Health Literacy in Adults (TOFHLA), are unavailable in Japanese. We therefore could not examine the correlations between our HL scale and these measures, although this study revealed that the HL scores were significantly associated with experiences and perspectives on seeking medicine information and making medication decisions. Second, the study subjects were recruited from participants in multiphasic health examinations, who may be relatively healthy people with an interest in health issues. Moreover, because of the voluntary response on the questionnaire survey, people who found it difficult to fill in the questionnaire may have been not returned a completed form and therefore have been excluded from the analysis. Further study is needed to examine whether the HLS-14 is acceptable to people with a wide range of HL level. Third, sociodemographic characteristics were not collected in the questionnaire survey. Therefore, we could not adjust for socioeconomic status in the comparison of HL scores. The age and gender differences in the HL scores must be confirmed in a population-based study with adjustment for socioeconomic status.

In conclusion, the HLS-14 demonstrated adequate reliability and validity as a generic HL measure for Japanese adults. This scale can be utilized for measuring functional, communicative, and critical HL in both the clinical and public health contexts. The use of HLS-14 may contribute to promoting a better understanding of advanced skills beyond reading comprehension and numeracy.