Background

Insomnia is highly prevalent in modern society. The Insomnia Severity Index (ISI) [Bastien et al. 2001] and Athens Insomnia Scale (AIS) [Soldatos et al. 2000; Okajima et al. 2013] were developed based on the standard diagnostic criteria for insomnia. These tools are widely used for assessing an individual’s risk of insomnia [Lomeli et al. 2008].

To date, few studies have directly compared the ISI and AIS. One meta-analysis estimated and compared the diagnostic accuracy of the ISI and AIS and found that these tools yield comparable diagnostic properties for insomnia screening [Chiu et al. 2016]. However, this meta-analysis did not directly compare the ISI and AIS, but rather performed a comparison of sensitivity and specificity using references such as the International Classification of Sleep Disorders, Second Edition [American Academy of Sleep Medicine 2005] and the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition [American Psychiatric Association 2010]. A few studies have directly compared the ISI and AIS [Jeong et al. 2015; Sierra et al. 2008; Chung et al. 2011]. A strong positive correlation was found between the scores of the AIS and ISI in Korean firefighters (r = 0.85) [Jeong et al. 2015] and elderly Spanish individuals (r = 0.93) [Sierra et al. 2008]. Another study used the AIS and ISI simultaneously and assessed their internal consistency, reliability, and validity compared to individual clinical diagnoses [Chung et al. 2011].

Prior studies have confirmed that insomnia is associated with depression [Knekt et al. 2011], work productivity loss [Bolge et al. 2009], and reduced quality of life (QOL) [Ishak et al. 2012]. Patients with insomnia report experiencing a variety of symptoms, including daytime sleepiness, fatigue, cognitive impairment, symptoms of depression and anxiety, health decrement, and impairment in social and occupational function [Krystal 2007]. Insomnia is a frequent complaint of individuals with depression. Eighty five percent of patients with depression reportedly have insomnia [Sunderajan et al. 2010]. Insomnia remains the most common unresolved symptom of depression even after mood improvement by pharmacological treatment [Ishak et al. 2012]. Research has shown that patients with insomnia have 24.2% greater work impairment (work productivity loss) and 18.0% greater activity impairment than those without insomnia [Bolge et al. 2009]. It has also been reported that the individual-level decrements in work performance caused by insomnia have a human capital value of $2280, which is equivalent to an annualized population-level estimate of $63.2 billion in the USA [Kessler et al. 2011]. Clearly, impairments in health, function, and QOL are central features of insomnia, which can lead to significant economic burden. Therefore, diagnosing and treating insomnia appropriately by improving the perceived health, function, and QOL of patients with this condition is essential [Krystal 2007].

Significant economic and social burden is caused by depression [Global Burden of Disease Study 2013 Collaborators. 2015; Kadotani et al. 2014] and insomnia [Daley et al. 2009]. Questionnaire surveys are commonly used to assess depression and insomnia in large-scale community settings or in working environments. Increasing the response rate in surveys helps minimize bias and maximize the generalizability of the findings [Blair and Zinkhan. 2006]. To this end, minimizing the length of the questionnaire was reported to significantly increase the response rate [Sahlqvist et al. 2011]. Therefore, ff a questionnaire designed to measure insomnia can also detect depression with acceptable accuracy, it may help reduce the length of the questionnaires distributed in large-scale surveys and correspondingly increase the response rate.

Here, we performed a direct comparison between the ISI and AIS in a Japanese population. Both tools were expected to show a similar diagnostic performance with respect to the health outcomes associated with insomnia.

Methods

Participants

A cross-sectional questionnaire-based study was conducted as part of the Night in Japan Home Sleep Monitoring (NinJaSleep) sleep and mental health epidemiological study. Participants were government employees of Koka city, which is a rural city in the Shiga Prefecture of Japan. Employees for whom the consent of a legal representative was required for participation or who had taken extended leaves from employment were excluded. Among the 2119 employees, 62 were excluded because of extended leaves, including sick, maternity, and childcare leaves. Thus, a total of 2057 participants were included. Questionnaires were distributed on September 6, 2017.

Questionnaires

The AIS and ISI were used to assess insomnia. Depression, sleepiness, QOL, and work performance were assessed using the Patient Health Questionnaire (PHQ)-9 [Kroenke et al. 2001; Gilbody et al. 2007], a Japanese version of the Epworth Sleepiness Scale (ESS) [Johns 1991; Takegami et al. 2009], the Short Form-8 Health Survey Questionnaire (SF-8) [Ware et al. 2001; Fukuhara and Suzukamo 2004], and the World Health Organization Health and Work Performance Questionnaire (WHO-HPQ) [Kessler et al. 2003; Suzuki et al. 2014], respectively. Participants’ bedtime, sleep latency, and waking time on weekdays were also recorded in the questionnaires.

The AIS evaluates the following eight items: AIS_1) sleep initiation; AIS_2) awakening during the night; AIS_3) early morning awakening; AIS_4) total sleep duration; AIS_5) overall QOL; AIS_6) problems with sense of well-being; AIS_7) overall functioning; and AIS_8) daytime sleepiness [Soldatos et al. 2000; Okajima et al. 2013]. The ISI examines the following seven items: ISI_1) sleep-onset; ISI_2) sleep maintenance; ISI_3) early morning awakening; ISI_4) satisfaction with current sleep pattern; ISI_5) interference with daily functioning; ISI_6) noticeability of impairment attributed to sleep problems; and ISI_7) level of distress caused by sleep problems [Bastien et al. 2001]. For the ISI, total scores of 8–14 and ≥ 15 are classified as subthreshold insomnia and clinical insomnia, respectively [Bastien et al. 2001]. Subjects with AIS scores of < 6 can be reliably considered as not having insomnia [Soldatos et al. 2003]. An AIS score of 6 is the optimum cutoff based on the balance between sensitivity and specificity [Soldatos et al. 2003]. Subjects with AIS scores of ≥10 are expected to be diagnosed with insomnia [Soldatos et al. 2003]. Thus, we classified AIS total scores of 6–7 and ≥ 10 as suspected and definite insomnia, respectively.

The PHQ-9 is a reliable and valid instrument for screening individuals for major depressive disorder [Kroenke et al. 2001; Gilbody et al. 2007]. In previous studies, participants with a PHQ-9 ≥ 10 [Kroenke et al. 2001; Gilbody et al. 2007] were classified as having depression, and this same cutoff was used here. As for the ESS, in line with prior studies, a score of > 10 was considered to indicate sleepiness [Johns 1991; Takegami et al. 2009]. General health-related QOL was assessed using the SF-8, which consists of eight items and is divided into physical component summary (PCS) and mental component summary (MCS) scores [Ware et al. 2001; Fukuhara et al. 2004]. Higher PCS and MCS scores indicate better health. In the general Japanese population, scores > 50 and those < 50 are considered above and below the average, respectively [Fukuhara et al. 2004]. Here, poor physical and mental QOL were defined as the lowest tertiles of the PCS and MCS QOL scores from the SF-8, respectively. Productivity loss at work because of health problems is called presenteeism and this parameter can be measured using the WHO-HPQ [Kessler et al. 2003; Suzuki et al. 2014]. Absolute presenteeism in the WHO-HPQ represents actual performance. In the present study, we used absolute presenteeism to assess work productivity loss.

Participants in the lowest tertile of the absolute presenteeism score in the WHO-HPQ were classified as having poor work productivity [Suzuki et al. 2014]. Regarding participants’ waking times and sleep latency, participants who woke up in the earliest tertile (before 06:00) and those in the shortest tertile for total sleep time (TST < 6 h) were classified as having early awakening and a short TST, respectively. Additionally, in adults, an onset latency > 30 min typically has clinical significance [American Academy of Sleep Medicine 2014]; therefore, here, participants with a sleep latency > 30 min were classified as having a long sleep latency.

Statistical analyses

Pearson correlation coefficient analyses were performed to determine the strength of the association between two variables. One-way analyses of variance were used to identify differences among three or more groups, and post hoc testing was performed using Scheffé’s method. Additionally, receiver operating characteristic (ROC) curve analysis was performed to compare the screening performance of the questionnaires. Pairwise comparisons of the ROC curves were performed by calculating the standard error of the area under the curve (AUC) and the difference between the two AUCs. Cohen’s kappa coefficient was used for agreement evaluation. Statistical analyses were performed using MedCalc Version 17.9.7 (MedCalc Software, Mariakerke, Belgium). P < 0.05 was considered statistically significant.

Results

A total of 1685 (81.9%; 1685/2057) city government employees had returned the questionnaires by September 30, 2017. Fifty-two of the individuals who returned the survey worked the night shift. Data were missing for six participants. Data from the remaining 1627 (79.1%: 1627/2057) participants were analyzed in this study. Table 1 and Additional file 1: Figure S1 summarize the participant characteristics.

Table 1 Participant characteristics
Full size table

Both the AIS and ISI had moderate reproducibility (weighted kappa: 0.58). A strong positive correlation was found between the total scores of the AIS and ISI (r = 0.80, p < 0.01). The distribution of the AIS and ISI scores is presented in Fig. 1 and Additional file 2: Figure S2. When we compared the categories of the AIS (0–5: no pathological insomnia, 6–9: suspected insomnia, and 10–24: definite insomnia) and the ISI (0–7: no clinically significant insomnia, 8–14: subthreshold insomnia, and 15–28: clinical insomnia), the weighted kappa was 0.578 (Table 2). The prevalence of clinical insomnia (ISI ≥ 15) and definite insomnia (AIS ≥ 10) in this population was 6.5 and 10.8%, respectively.

Fig. 1
figure 1

Athens Insomnia Scale (AIS) and Insomnia Severity Index (ISI) three-dimensional histogram

The height (z-axis) represents the number of subjects, with the ISI and AIS scores shown in the x- and y-axis, respectively.

Full size image
Table 2 AIS and ISI total scores
Full size table

Both the AIS and ISI were associated with low physical and mental QOL, depression, and productivity loss at work (Fig. 2 and Additional file 3: Table S1). Furthermore, both the AIS and ISI showed a weak negative correlation with the PCS (r = − 0.37, p < 0.01 and r = − 0.32, p < 0.01, respectively) and absolute presenteeism (r = − 0.32, p < 0.01 and r = − 0.28, p < 0.01, respectively) and a moderate negative correlation with the MCS (r = − 0.53, p < 0.01 and r = − 0.43, p < 0.01, respectively) (Additional file 3: Table S1).

Fig. 2
figure 2

Receiver operating characteristic (ROC) analysis of the Athens Insomnia Scale (AIS) and Insomnia Severity Index (ISI) scores and poor QOL, depression, and poor work performance. Health-related quality of life (QOL) as assessed by the Short Form-8 Health Survey Questionnaire (SF-8) was summarized into the physical continuous summary (PCS) and mental continuous summary (MCS) scores. Work performance was assessed by the absolute presenteeism score using the World Health Organization Health and Work Performance Questionnaire (WHO-WPQ). The lowest tertile of the PCS, MCS, and absolute presenteeism scores were classified as poor physical QOL, poor mental QOL, and poor work productivity, respectively. A Patient Health Questionnaire-9 (PHQ-9) score ≥ 10 was classified as depression. The p values represent the pairwise comparisons for the ROC curves of the ISI and AIS

Full size image

We performed ROC analyses to compare the ability of the AIS and ISI to detect poor QOL (lowest tertiles of the PCS and MCS from the SF-8), depression (PHQ-9 ≥ 10), and poor work performance (lowest tertile of the absolute presenteeism score in the WHO-HPQ) (Fig. 2). All the ROC curves presented in Fig. 2 had significantly high AUCs (p < 0.01, compared with AUC = 0.5), which suggested that the AIS and ISI can identify low physical QOL, low mental QOL, depression, and productivity loss at work. The AIS had a significantly higher AUC than did the ISI for physical QOL, mental QOL, and depression. The AUCs to detect depression exceeded 0.85. The Pearson correlation coefficients were significantly different between the AIS and ISI compared with the PHQ-9 and MCS (Additional file 4: Table S2).

Each item of the AIS was compared with the corresponding item of the ISI. AIS_1 and ISI_1 assess problems with sleep initiation. AIS_3 and ISI_3 assess problems with early awakening (Additional file 5: figure S3). These items yielded similar AUCs (p = 0.81 comparing AIS_1 with ISI _1; p = 0.50 comparing AIS_3 with ISI_3) (Fig. 3). AIS_4, AIS_6, and AIS_8 assess sleep duration, problems with sense of well-being, and sleepiness, respectively (Additional file 5: Figure S3). The scores for these items were compared with the short TST (shortest tertile < 6 h), depression (PHQ-9 ≥ 10), and sleepiness (ESS > 10) data, respectively. The AUCs for AIS_4, AIS_6, and AIS_8 were 0.70, 0.84, and 0.67, respectively (Fig. 4).

Fig. 3
figure 3

Receiver operating characteristic (ROC) analysis of the Athens Insomnia Scale (AIS) and Insomnia Severity Index (ISI) scores, long sleep latency, and early waking. ISI_1, first ISI item: sleep-onset; AIS_1, first AIS item: sleep initiation; ISI_3, third ISI item: early morning awakening; AIS_3: third AIS item: early morning awakening. The p values represent the pairwise comparisons of the ROC curves of the ISI and AIS

Full size image
Fig. 4
figure 4

Receiver operating characteristic (ROC) analysis of the Athens Insomnia Scale (AIS) item scores for the comparison of short sleep duration and sleepiness. AIS_4, fourth AIS item: total sleep duration; AIS_6, sixth AIS item: problems with sense of well-being; AIS_8, eighth AIS item: sleepiness during the day

Full size image

A ROC analysis was performed using each item of the AIS and ISI as test variables with depression, poor physical QOL, poor mental QOL, and poor work performance as outcomes. Of these ROC analyses, only AIS_6 and AIS_7 had an AUC > 0.8 when compared with depression (Additional file 6: Table S3).

Discussion

In this study, the presence of insomnia and its outcomes were assessed simultaneously with the AIS and ISI in city government employees in Japan. A strong positive correlation was found between the AIS and ISI. Both tools were associated with QOL, depression, and productivity loss at work (Fig. 1), suggesting comparable properties.

Ten percent of the general population is expected to have chronic insomnia [American Academy of Sleep Medicine 2014]. Our prevalence of clinical insomnia (ISI ≥ 15: 6.5%) and definite insomnia (AIS ≥ 10.8%) may represent the prevalence of chronic insomnia in city government employees in Japan.

Major depressive disorder is commonly comorbid with insomnia [Riemann and Voderholzer, 2003; Tsuno et al. 2005; Kadotani et al. 2017]. Among the three outcomes (QOL, depression, and productivity) compared in this study, depression had the highest AUC (> 0.85; Fig. 2). This finding suggests that both the AIS and ISI had moderate accuracy for detecting depression in this population. AIS_6 and AIS_7 also had similar AUCs (0.84 and 0.82, respectively; p = 0.05; Additional file 6: Table S3 and Additional file 7: Figure S4). These items alone may be useful for screening for depression in the general population.

The present findings are in contrast to those of our previous study, which did not reveal an association between insomnia and depression when using the Pittsburgh Sleep Quality Index and clinical interview in a different population of working Japanese men (the Pittsburgh Sleep Quality Index values in the depression and control groups were 5.58 ± 2.28 and 4.76. ± 1.97, respectively; p = 0.05) [Kadotani et al. 2017]. However, the sample size of our previous study was 314. Therefore, the use of the AIS/ISI and/or the larger sample size may have been useful in detecting depression with the AIS/ISI in this study.

The total AIS score had a significantly higher AUC for QOL and depression and had a slightly higher AUC for productivity loss than did the ISI (Fig. 2). AIS_1/ISI_1 and AIS_3/ISI_3 assess problems in sleep latency and early morning awakening, respectively. The AUCs for these items did not show a significant difference between the AIS and ISI (Fig. 3). The AIS and ISI have 8 and 7 items, respectively. Each item yielded comparable results and the total score had a higher AUC in the AIS than in the ISI. The AIS (AIS_6) assesses well-being, but the ISI does not. AIS_6 had the highest AUC among all the items of the AIS and ISI (Additional file 6: Table S3). Thus, the different AUCs for the AIS and ISI total scores might have been due to the differences in the number of their items.

In the current study, five items of the AIS (AIS_1, AIS_3, AIS_4, AIS_6, and AIS_8) were compared with the outcomes related to sleep latency, early morning awakening, short TST, sleepiness, and depression (Figs. 3 and 4). Two items in the ISI (ISI_1 and ISI_3) were also compared with the outcomes related to sleep latency and early morning awakening (Fig. 3). The other items in the AIS and ISI were not analyzed in this study because we did not have data sources for sleep maintenance problems (or awakening during the night), satisfaction with current sleep patterns (or sleep quality), or distress caused by sleep problems. Daily functioning (functional capacity during the day) may be associated with QOL; however, the SF-8 can only yield physical and mental summary scores, not a total summary score.

Our results showed that AIS_3 and ISI_3 were not associated with early awakening (earliest tertile of awakening time; Fig. 3). The earliest tertile of awakening time may not have been a suitable outcome for the comparison. The cutoff threshold using the tertile was earlier than 6:00 am, whereas the mean awakening time was 6:00 am (Table 1). We used this cutoff value because we used tertiles for most of the outcomes in this study. However, the awakening time or awakening time tertile may not have been suitable for detecting problems with early awakening.

This study had some limitations. First, our participants were city government employees in Japan. Therefore, while they may share similarities with the general population of their particular city, they may not have been representative of the general Japanese population. Second, this study used only questionnaire data; no objective data or clinical diagnoses were used. Although we plan to perform portable electroencephalography [Matsuo et al. 2016] and sleep respiratory monitoring to analyze sleep apnea in this population (NinJaSleep Study), such data were not collected in the present study. Finally, this was a cross-sectional study; thus, we could not show a causal relationship. We plan to conduct a longitudinal study in this population to further investigate this relationship.

Conclusions

The AIS and ISI were simultaneously administered to assess insomnia and its outcomes in Japanese city government employees. Both the AIS and ISI were associated with a low physical and mental QOL, depression, and productivity loss at work. A strong positive correlation was found between the total scores of the AIS and ISI. Moreover, the AIS, ISI, AIS_6, and AIS_7 showed a moderate accuracy for detecting depression in this population. Both the AIS and ISI may be useful for screening Japanese workers for not only insomnia, but also depression.