Background

Approximately 90,000 to 120,000 Norwegians have known diabetes [1], among whom reported prevalence of diabetic retinopathy (DR) ranges from 11% to 28% [2–5]. In the working age group, DR is a leading cause of visual impairment [6]. Among people with diabetes, 1% to 13% develop sight-threatening diabetic retinopathy (STDR) and 0.4% to 1.3% are visually impaired because of DR [7–13]. Regular eye examinations and early treatment of retinopathy can prevent visual loss [9, 14–16], so screening for DR is cost-effective [17]. Dilated retinal digital photography with the additional use of ophthalmoscopy is the most effective and robust method of DR screening [18, 19]. In Norway, the national guidelines for diabetes [20] and The Norwegian College of General Practitioners [21] recommend either regular eye examinations by an ophthalmologist or the use of retinal photography. The Norwegian Association of Optometry has issued clinical guidelines for optometric practice [22] which include guidelines for the examination and management of patients with diabetes.

People with diabetes are commonly examined in optometric practice due to having refractive errors. Norwegian optometric practice may represent a low threshold setting for case-finding of DR [23]. Studies in other countries have shown that optometrists are able to detect and grade DR [24] and specially trained optometrists perform well when screening for STDR (sensitivity 73%-97% and specificity 83%-99%) [25–29]. Since 1988, the profession in Norway has developed from being populated by opticians to being an approved healthcare profession, populated by optometrists. Consequently, Norwegian optometrists are a heterogeneous group with regard to formal education [30]. In 2004, optometrists were granted the right to prescribe diagnostic ocular drugs and since 2009 they have been able to refer patients directly to an ophthalmologist, without the patient first seeing a gate-keeping general practitioner (GP). These two responsibilities warrant a high standard of performance on the part of the optometrist.

Sensitivity and specificity define the ability of a clinical test to correctly identify people with and without a specific disease. For low prevalence diseases, a high specificity is required to avoid large numbers of false positive results. The British Diabetic Association (now Diabetes UK) has set a required screening standard for DR of at least 80% sensitivity and 95% specificity [31]. The aims of the current study were to assess the sensitivity and specificity of the optometrists’ diagnosis of DR and to assess sensitivity and specificity with respect to the optometrists’ formal education. Furthermore, we wanted to investigate how the optometrists intended to follow up their cases.

Methods

A cross-sectional experimental design was employed. The study population, from which study participants were drawn, comprised authorized optometrists in Norway (n≈1850). Members of the Norwegian Association of Optometry (NOF) (n=1028) were invited to participate by e-mail. Only those optometrists who were currently working in private practice, who had worked in private practice for the previous 6 months and who intended to continue working in private practice for the following 6 months were eligible for inclusion in the study.

Those optometrists who responded positively to our e-mail and were subsequently accepted for inclusion in the study were sent an interactive web-based visual identification and management of ophthalmological conditions (VIMOC) examination that used Question Writer 4 software. A VIMOC examination tests clinical competency using cases and/or images with accompanying multiple choice questions [32]. The examination consisted of 14 retinal images which the optometrists were to assess with respect to the presence or absence of DR, without grading severity. Additionally, they were to decide on patient management, based solely on retinal findings and making the assumption that the patient had never been examined by an ophthalmologist. No grading scales or patient management guidelines were provided and the optometrists were not given any patient information, such as visual acuity data. It was possible to move back and forth in the VIMOC exam to review the images and revise prior assessments before submitting a final response. In addition, a questionnaire was included to gather information regarding the participants’ work experience, education, preferred method of retinal examination, methods used for retinal examination in patients with diabetes, and methods available to them for retinal examination and imaging. Optometrists used their own computers with screen resolution and colour set to maximum. Screen resolution ranged from 1024×600 to 2560×1440 pixels.

The VIMOC retinal images were obtained from a previous Norwegian population survey [2]. The study followed the tenets of the Declaration of Helsinki for research involving humans and was approved by the Regional Committee for Medical Research Ethics, REC Central (January 19. 2009). Blinded to patient information, all images had been independently assessed by two ophthalmologists who graded the presence of retinopathy according to the Diabetic Retinopathy Disease Severity Scale [33]. The ophthalmologists viewed the images on a 21” monitor with screen resolution of 1600×1200 pixels. From a total of 239 images, only those that had been graded with full agreement between the two ophthalmologists (n=217) were considered for inclusion in our study. Seven images of retinas affected by non-proliferative diabetic retinopathy (NPDR) and seven images of retinas unaffected by DR were randomly selected. The DR images included five examples of mild NPDR (Figure 1) and two examples of moderate, potentially sight threatening NPDR (Figure 2). To reduce the possibility of falsely high specificity occurring by chance, half of the presented images were of retinas that were not affected by DR. The diagnoses of the two ophthalmologists for each image were used as a “gold standard” against which the performance of the study participants was assessed.

Figure 1
figure 1

Mild non-proliferative diabetic retinopathy.

Full size image
Figure 2
figure 2

Moderate non-proliferative diabetic retinopathy.

Full size image

The required sample size of participants was calculated based on the following: 50% prevalence of DR in the image sample, a standard deviation of true sensitivity and specificity for individual optometrists’ image evaluation of 0.2, and 50% sensitivity and specificity for detecting DR by individual optometrists to allow maximum variance. It was calculated that a CI < 0.05 for sensitivity and specificity for any diabetic retinopathy (ADR) could be achieved with 100 study participants, meaning that sensitivity and specificity was calculated with 95% confidence interval for any retinopathy. Study participants were not asked to grade DR; the sensitivity of mild and moderate NPDR was assessed in terms of detection of retinopathy in images with mild and moderate NPDR, respectively. The screening standard established by the British Diabetic Association (Diabetes UK) of at least 80% sensitivity and 95% specificity for ADR [31] was used as the screening standard in our study. Potential associations between test performance and formal education were investigated and analysed using Pearson Chi-square and student-t tests; a p-value ≤ 0.05 was regarded as significant.

Data were collected in the period between 28th February and 14th March 2011; reminders requesting participants to complete the test were sent once.

Results

In all, 112 (11%) members of NOF responded positively to the e-mail and volunteered to participate in this study. Of these 112, 101 (90%) met the inclusion criteria, and 74 (73%) completed the study. Participants were generally educated to a higher level than the average for Norwegian optometrists (Table 1).

Table 1 Characteristics of Norwegian optometrists
Full size table

The optometrists’ preferred methods of retinal examination were reported to include undilated indirect ophthalmoscopy (47% of participants) and undilated retinal fundus photography (34% of participants). Multiple examination methods were reported for patients with diabetes (Table 2). Twenty-three percent of participants reported that they undertook dilated retinal examinations in patients with diabetes.

Table 2 Characteristics of participating optometrists by formal education
Full size table

The optometrists’ assessments of each of the 14 VIMOC images are presented in Table 3. Optometrists with higher optometric education (a Master of Science in clinical optometry [MSc]) demonstrated significantly higher sensitivity than those who had a basic optometric education (Table 4). The specificity was not influenced by the optometrist education level.

Table 3 Optometrists’ VIMOC evaluations of retinal images and corresponding ophthalmologist grading and patient glucose status
Full size table
Table 4 Optometrists’ sensitivity and specificity for identifying diabetic retinopathy, presented by formal education level
Full size table

No association was found either between sensitivity or specificity and the number of years of experience in optometric practice, or between sensitivity or specificity and the participants’ preferred method of retinal examination. The screening standard for sensitivity of at least 80% and specificity of at least 95%, for ADR, was met by 24 (32%) and 31 (42%) optometrists, respectively, overall. The standards for sensitivity and specificity were met by 50% and 45%, respectively, of optometrists who held an MSc and by 25% and 39%, respectively, who had a basic optometric education. Only four optometrists (5%) met the required standard for both sensitivity and specificity.

Patient management decisions were dependent on retinal findings (Table 5). Report/referral to a GP and/or an ophthalmologist was regarded as appropriate for 99% and 96% of true- and false-positive findings, respectively. The rate of referral to an ophthalmologist was higher for moderate than for mild NPDR (92% vs. 62%). No further management was considered appropriate in 68% and 66% of cases of true- and false-negative findings, respectively.

Table 5 Individual image evaluation and suggested follow-up
Full size table

Discussion

Only 5% of the responding optometrists satisfied the screening standard established by the British Diabetic Association of at least 80% sensitivity and 95% specificity [31]. Overall, sensitivity for detecting ADR was low and specificity was moderate. Sensitivity for detecting potential STDR was, however, high. This suggests that the optometrists’ assessment of retinal images is an unreliable method of screening for DR. The sensitivity and specificity of detection of DR in the current study and previous studies is presented in Table 6. It is not possible to make a direct comparison between the current study and previous studies that involved community optometrists [28, 34], as those studies did not report sensitivity and specificity levels for individual optometrists. However, based on the reported mean levels of sensitivity and specificity, it is unlikely that individual optometrists in those studies would have met the British Diabetic Association screening criteria. The sensitivity for detecting ADR in our study (67%) was lower than that reported by Gibbins et al. (86-88%) [28]. However, the sensitivity for detecting STDR was higher in our study than in either the Gibbins et al. or Buxton et al studies (100% vs. 47-97%) [28, 34] and the specificity was similar (84% vs. 83-95%). The greater sensitivity for detecting STDR in the current study could have been a result of the higher prevalence of STDR in our VIMOC sample compared with these earlier studies, which may have inflated sensitivity by chance. The prevalence of ADR in our study was comparable with the prevalence of ADR in the study by Gibbins et al. [28]. In that study, optometrists had received special training in the identification and grading of DR, which could explain the relatively high sensitivity levels observed.

Table 6 Optometrists’ sensitivity and specificity for identifying diabetic retinopathy as reported in the current study and previous studies
Full size table

We have found that sensitivity, but not specificity, was influenced by the level of formal education the participants had received. Optometrists with an MSc had a significantly higher sensitivity than optometrist with a basic optometric education. This suggests that our results give a better estimate of sensitivity and specificity in general optometric practice, as our study included optometrists who had not had any special training in screening for DR.

The sensitivity observed in this study is in line with that observed in a previous study we undertook to investigate Norwegian general optometric practice [23], where sensitivity ranged from 61% to 65%, based on an assumption of 14% prevalence of DR among patients with diabetes [3]. Specificity in the current study was, however, lower than in our previous study (84% vs. 98-100%), which could be explained by the difference in prevalence between the two studies. In the current study, 98% of findings of DR (both true- and false-positives) were considered to warrant a report/referral to a physician. This is higher than the rate of 57% reported in a national practice registration in Norway [23]. The experimental design in our study, where optometrists were blinded to patient information but assumed that the patient had never been examined by an ophthalmologist, may have led to an increased tendency to recommend referral to a physician.

Assuming the prevalence of DR is 14% [3], the negative and positive predictive values of the optometrists’ evaluation of DR in our study would be 94% and 41%, respectively. Based on this and on the fact that Norwegian optometrists undertake approximately 1 million eye examinations per year (of which approximately 4% are in patients with diabetes [30]), our findings suggest that each year approximately 5500 patients without DR are referred based on a false positive result, while in approximately 1300 patients with DR, no further action is taken. However, if the British Diabetic Association screening criteria were met in Norwegian optometric practice, these figures would be 1700 and 800, respectively. Our results suggest that an excessive workload is being placed on healthcare services by inaccurate referral practices. However, the national guidelines recommend eye examination by ophthalmologists [20, 21], thus the report/referral of a patient who has not previously been seen by an ophthalmologist should not be discouraged. Of greater concern is the false security given to those patients with DR who are not referred to an ophthalmologist.

The strengths of this study are the use of standardised images in the VIMOC exam and the use of a diagnostic “gold standard” based on 100% agreement between two independent ophthalmologists. The experimental design allowed the calculation of sensitivity and specificity with acceptable precision in a relatively large nationwide sample of optometrists, something that was not achieved in previous studies [25–28, 34]. In terms of gender, number of years in practice and geographical location, our sample of optometrists is representative of members of the NOF and of optometrists who participated in a previous study of Norwegian optometric practice [23, 30]. The potential for knowledge bias and overestimation of sensitivity and specificity for general optometric practice was reduced in the current study because the optometrists were not provided with grading scales, nor were they given specific training prior to the study.

One potential limitation of the study was the possibility of selection bias, as optometrists with a specific interest in diabetes may have been more likely to accept the invitation to participate and hence may have been overrepresented in the study. This could have inflated the sensitivity levels observed, compared with general optometric practice. On the other hand, participating optometrists did not have specific training in screening for DR, nor were they provided with a DR grading scale or a computer screen that would facilitate classification of DR. Variable viewing conditions may have influenced the detection rate of DR. Small screen size, low screen resolution and inadequate colour setting may have led to lower sensitivity for detecting mild DR. On the other hand, the optometrists’ use of their own facilities simulated real practice, something that the use of perfect viewing conditions could not have done.

Conclusions

Our study is likely to have given a better representation of general optometric practice than previous studies [25–28, 34]. However, our findings indicate that at present case-finding of DR in Norwegian optometric practice is unreliable. Formal optometric training in screening for DR and continuing education may improve diagnostic sensitivity. Further research will be needed to evaluate the effectiveness of measures undertaken to improve optometrists’ diagnostic accuracy for case-finding of DR.

Authors’ information

VS is the program coordinator for postgraduate courses in Optometry and Visual Science at the Department of Optometry and Visual Science, Faculty of Health Sciences, Buskerud University College.