Background

Globally, 285 million people of all ages suffer from visual impairment [1]. Long-term ocular conditions, including both ocular diseases (e.g. glaucoma, diabetic retinopathy, age-related macular degeneration and cataract) and uncorrected refractive errors are the major causes of visual impairment worldwide [2]. The prevalence of vision problems is strongly associated with ageing and this compromised visual function affects individuals’ ability to perform activities of daily living [3]. Common eye diseases can often be detected early and their visual impact minimised or they can be prevented by appropriate eye care services, including routine eye examinations [4,5,6]. Due to the growing demand for eye care in the context of resource scarcity, interest in measuring and improving the appropriateness of eye care delivery is growing [7, 8]. Appropriate care is defined as provision of evidence-based care that is relevant to the patient’s needs and based on established standards [9].

Translation of best available evidence into clinical practice is important, ensuring that both efficacy and cost-effectiveness of patient management is maintained [10]. Evidence-based guidelines aim to translate well conducted scientific trials into easy to apply recommendations. Such guidelines intend to guide practitioners and help them to improve their professional practice and optimize patient care [11]. Evidence-based guidelines are not always adhered to and/or fully implemented in the clinical setting. Adherence to guidelines can be quantitatively measured using quality indicators of appropriateness of care delivery. Quality Indicators can be defined as “measurable components of a standard or guideline, with explicit criteria for inclusion, exclusion, time frame, setting and compliance action” [12].

Evidence of suboptimal care being delivered exist, arising from several large studies assessing appropriateness of care across different health conditions. The RAND study conducted in 2000 in the United States evaluated performance on 439 quality indicators of appropriateness of care for 30 acute and chronic conditions as well as preventive care. The RAND study showed that American adults received recommended care only 55% (range 11–79%) of the time [13]. More recently, the CareTrack study in Australia showed similar results with 57% (range 13–90%) of Australian adults receiving appropriate care across 22 conditions [12]. Ocular conditions were not included in the CareTrack study [12]. Defining existing eye care practice patterns and current variation from best practices is an important component of a systemic approach to improving appropriateness of eye care [14, 15].

Purpose

This paper aimed to review findings from published studies about the appropriateness of eye care delivery, using existing published evidence and/or experts’ practice. A secondary aim was to describe and compare the variety of methods used to measure appropriateness of eye care.

Methods

Data sources and searches

A systematic search was conducted using Medline, Embase and the Cumulative Index to Nursing and Allied Health Literature (CINAHL) electronic databases to identify studies related to the appropriateness of eye care. The search strategy was reviewed and tested by an academic librarian and reviewed by content experts (IJ and FS). The literature review process followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) procedures [16] and the review protocol was published on PROSPERO (http://www.crd.york.ac.uk/prospero/, reference CRD42016049974). As eye conditions with higher prevalence and heavier burden on the health system, the emphasis was put on glaucoma, diabetic retinopathy, refractive error, cataract and macular degeneration [17]. The search incorporated the three elements:

  1. 1)

    Profession-specific terms: “Optometr*”, “Ophthalmolog*”, “General practitioner*”, “Orthopt*”, “Ophthalmic nurse*”, “Ophthalmic practitioner*”.

  2. 2)

    Subject headings: Exp”Quality of Health Care” in Medline, Exp”Health care quality” in Embase, MH”Health Services Research+” in CINAHL.

  3. 3)

    Condition-specific terms: Exp Glaucoma, Exp diabetic retinopathy, Exp refractive errors, Exp macular degeneration, Exp cataract.

An example of the full electronic search strategy for Medline database is illustrated in Additional file 1.

Study selection

Reference lists and citations were used to cross-check the results of our search. The reference details and abstracts of the 5596 articles retrieved from the literature search after duplicates removal were reviewed by one reviewer (KCH). Studies assessing the processes of eye care delivery against existing published evidence and experts’ practice (e.g. consultant ophthalmologists’ practice) were included. Studies assessing outcomes of care delivery such as patient satisfaction or those assessing structural aspects of care delivery such as workforce characteristics, infrastructure, regulations and policies were excluded from analysis in this review. The search was not restricted by type of study design, and no other limitations (e.g. population, intervention, comparison, length of follow-up) were set. The search was limited to English and 10 years to the search date (2006 to 16th September 2016). Studies conducted more than 10 years ago were excluded, on the basis that appropriateness of care was likely to change over time, and that older studies might not reflect recent changes in care delivery standards [18]. The references were narrowed to 65 articles after title and abstract screening following the application of exclusion criteria (Fig. 1). A further six articles were excluded after full text review with three that did not access process of care and three that did not measure against existing published evidence or experts’ practice.

Fig. 1
figure 1

PRISMA flow diagram for appropriateness of eye care delivery

Full size image

Data extraction and quality assessment

Each paper was reviewed and information was extracted based on the following characteristics:

  • Country

  • Condition(s) – the eye condition(s) for which the appropriateness of care was assessed

  • Professions – the health professions delivering the care of the assessed eye condition

  • Methods – the method used to assess the appropriateness of eye care delivered

  • Sample size

  • Response rate

  • Evidence sources – the reference standard used to assess the appropriateness of eye care delivered

  • Settings – classification based on whether study was conducted in hospital or independent practice

  • Number of sites – the number of sites that the study was conducted at

  • Timing – the timing and visit types assessed in the article (e.g. at diagnosis, follow-up, etc)

  • Percentage of encounters with appropriate eye care – the number of quality indicators met over the total number of relevant quality indicators

Taking into consideration the diversity of study types (e.g. descriptive, interventional and observational studies, record reviews, and surveys), two reviewers (KCH and SA) independently assessed the quality of each article using a validated critical appraisal tool [19]. The applied tool was modified by adding questions from other validated critical appraisal tools including Critical Appraisal Skills Programme (CASP) diagnostic checklist [20], National Institutes of Health (NIH) Quality Assessment Tool For Observational Cohort And Cross-Sectional Studies [21], Joanna Briggs Institute (JBI) Critical Appraisal Checklist For Studies Reporting Prevalence Data [22], Effective Public Health Practice Project (EPHPP) Quality assessment tool for quantitative studies [23].

The modified quality assessment tool included 17 individual criterions with questions from validated critical appraisal tools [20,21,22,23] (Additional file 2) and grouped in the seven categories listed below:

  • Quality of reporting (adequate description of the context [19], clearly stated aims [19,20,21], eligibility [21], methods and findings [20])

  • Selection bias (representative of the selected individuals [22, 23], response rate at least 50% [21], and sample size justification [21])

  • Study design (presence of randomisation [23], presence of control group [19, 23])

  • Blinding (blinding of outcome assessors to the intervention or exposure status of participants [20, 21, 23], blinding of participants to research question [23], and blinding of decision making between participants and experts [20])

  • Data collection tools (reliability of the data collection tool [22, 23] and valid reference used to assess the appropriateness of care [20])

  • Analysis (sufficient rigorous data analysis [19, 22, 23])

  • Limitations (key potential confounders are identified and accounted for [21,22,23])

The number of criteria used varied depending on the study design of the publication being reviewed. An overall rating was allocated for each paper as a percentage based on the number of criteria met over the number of relevant criteria for the corresponding study design. If less than 60% criteria relevant to the study design was met, this item was scored as Weak in the quality assessment tool. It was scored moderate if 60–79% of criteria were met and strong if 80–100% of criteria were met. A third reviewer (IJ) resolved any disagreements and consensus was reached through discussion. All articles were included, and the results of critical appraisal are provided in Additional file 3.

Data synthesis and analysis

Due to the anticipated heterogeneity of included studies, no plans were made to pool the results statistically, therefore a meta-analysis was not undertaken. For each study, the range of percentage of appropriate care (summary data from published reports, but not individual patient-level data) and the number of quality indicators were separated according to the nature of the quality indicators into the following six domains of care: ‘history taking’, ‘physical examination’, ‘management’, ‘recall period’, ‘referral’ and ‘patient education’. On occasion, data provided in the papers had to be reclassified to fit these proposed domains of care. Data were also reanalysed as required so that the results could be presented in terms of appropriateness to prescribed care and not the reverse (i.e. percentage with inappropriate care).

Results

Of 6472 citations, 57 articles met the inclusion (see Fig. 1). The characteristics of these studies are presented in Table 1. The majority of the studies were from the United Kingdom (UK) (n = 25) and the United States of America (USA) (n = 15), with Australia (n = 5), Australia and New Zealand (NZ) (n = 2) and other countries accounting for the remainder. Among the 57 papers, two-thirds examined eye care delivery for glaucoma (n = 28) and diabetic retinopathy (n = 11). The majority of papers assessed the care delivered by optometrists (n = 22) and ophthalmologists (n = 19), with another seven studies including both professions. Half of the studies were rated moderate (60–79% of quality criteria met) for the methodological quality (n = 29), another one-third were rated strong (80–100% of quality criteria met) (n = 19) and the remainder were rated weak (< 60% of quality criteria met) (n = 9). For all conditions but diabetic retinopathy, a similar pattern of distribution of methodological quality (i.e. mostly moderate) was observed. However, for diabetic retinopathy most of the studies (73%) were rated strong in methodological quality.

Table 1 Study Characteristics (n = 57). USA = United States of America, UK=United Kingdom, NZ = New Zealand, A&E = accident and emergency
Full size table

Record review (26 of 57 studies) and practitioner survey with or without case vignettes (15 of 57 studies) were the most commonly used methods, with one study using a combination of both methods and one study using both methods with claims data and patient survey. When eye care appropriateness was measured using record review, assessments were most frequently conducted at a single site (n = 19) and in these cases, studies were conducted in a hospital setting (Fig. 2). Use of a single site reduces logistical challenges, but the results may not be generalisable to other environments with a different location, business models and case-mix. For example, the record review conducted in the Department of Veterans Affairs, which caters to a population that is predominantly male, may not be generalised to clinic settings and patient populations outside the Veterans Affairs system [50].

Fig. 2
figure 2

Review Site Characteristics (n = 29). The number on each bar indicates the number of included studies (a) assessed within the corresponding settings, (b) conducted at single or multiple sites. When eye care appropriateness was measured using record review, assessments were most frequently conducted at a single site (n = 19) and in these cases, studies were conducted in a hospital setting. Use of a single site reduces logistical challenges, but the results may not be generalisable to other environments with a different location, practice types and case-mix

Full size image

Appropriateness of eye care was generally measured as compliance against scientific evidence or consensus with clinical experts in the field with around two-thirds of the articles having measured eye care appropriateness against recommendations from clinical practice guidelines (n = 38) and 16% having used experts’ opinions (n = 9).

A small number of studies measured eye care appropriateness against expert care rather than against clinical practice guidelines, where the same patients are examined twice, once by the practitioners and once by experts [36, 135, 143].

Eye care appropriateness results are summarized in Table 2. It is important to note at the outset that the timing (e.g. once during a period, at the diagnosis visit, etc.), type of visits (e.g. first visit, follow-up visit, etc.), the health professions and settings assessed, and the method used to collect the data (e.g. record review) vary between studies (see Table 2) and may confound the appropriateness of eye care results.

Table 2 Appropriateness of eye care by domain of care. Numbers are percentage of encounters with appropriate care (number of quality indicators). If more than one quality indicator was assessed, the percentage of encounters with appropriate care is presented as a range of percentage. NZ = New Zealand, A&E = accident and emergency, N/A = not applicable as no specific timing was measured
Full size table

Twenty-eight studies reporting on eye care appropriateness in glaucoma screening, glaucoma suspects and/or glaucoma patients were included. In more than half of the studies (15 of 28), the appropriateness of glaucoma care was measured via a review of hospital records. Appropriate ‘management’ and ‘recall period’ for glaucoma were reported most of the time, whereas ‘physical examination’ and ‘referral’ for glaucoma were not delivered as appropriately at times (Fig. 3a and b). Overall, the appropriateness of glaucoma care ranged widely from 2 to 100%. The appropriateness of glaucoma care assessed using clinical agreement with experts was the only method where appropriate care was delivered consistently at least 50% of the time. Although studies investigated the appropriateness of glaucoma delivered by optometrists and ophthalmologists, no obvious differences between professions were noted.

Fig. 3
figure 3

Appropriateness of eye care for glaucoma (a, b) and diabetic retinopathy (c, d) for various domains of care by profession (a, c) and methods (b, d). All quality indicators from the included studies were pooled together. Each data point represented the percentage of compliance against a quality indicator. a Overall, the appropriateness of glaucoma care ranged widely from 2 to 100%. The appropriateness of glaucoma delivered by optometrists and ophthalmologists appeared similar. b When appropriateness of glaucoma care was assessed using clinical agreement with experts, care was delivered appropriately at least 50% of the time. The appropriateness of glaucoma care assessed using other methods ranged more widely. c, d The appropriateness of diabetic eye care ranged widely from 0 to 100%. The wide range and the relatively small number of studies measuring appropriateness of diabetic eye care limited our ability to detect obvious patterns in individual domains for diabetes care

Full size image

Eleven studies have reported on appropriateness of eye care delivery in diabetic patients. Overall, diabetes eye care compliance also ranged widely from 0 to 100%. That wide range and the relatively small number of studies available makes it challenging to detect obvious patterns in individual domains for diabetes care (Fig. 3c and d). For example, only a single study with three quality indicators sampled the appropriateness of ‘patient education’ in diabetes eye care at a single site and reported a below 50% appropriateness of ‘patient education’ overall.

Appropriateness of eye care delivery has been measured for cataract, age-related macular degeneration, preventative eye care and five other ocular conditions in 18 separate articles (Table 2). Eye care appropriateness also ranged widely in those studies, for example from 0 to 100% for dry eye care [134] and for the referral of cataract surgery [107].

Very few studies examined or reported on factors that can modulate appropriateness of eye care delivery. Modifiable factors that have been shown to impact appropriateness of eye care delivery include data entry system (i.e. electronic or paper records) [134], health insurance coverage [76], higher eye care provider density [76], awareness of clinical practice guidelines availability [142], procedural confidence and therapeutic endorsement of optometrists [56] and specialty training conducted in a supportive environment [43]. Non-modifiable factors that may impact appropriateness of eye care include the severity of patients’ eye condition [71], patient’s age and ethnicity [54], and practitioner’s age [72, 129], gender [129] and years of experience [88]. These factors must therefore be measured and controlled for in any future studies assessing the appropriateness of eye care delivery.

Discussion

This systematic literature review summarises studies reporting the process of eye care delivery in many different countries using existing published evidence and/or experts’ practice to measure appropriateness of eye care. The appropriateness of eye care delivered was found to vary widely for the most commonly reported conditions (glaucoma and diabetic eye care) from 0 to 100%. Appropriate ‘management’ and ‘recall period’ for glaucoma were observed. Record review was most commonly used to assess the appropriateness of eye care delivery; this may be explained by the ease of administration and low cost associated with this method, especially when conducted at a single site.

The methodological quality was rated as moderate on average across all methods. Different quality assessment tools were used for to appraise studies with different study design, where some criteria were the same between tools. With consideration of the variety of the study designs and the total numbers of included studies, it was considered beneficial to use a modified quality assessment tool with all questions sourced from existing validated critical appraisal tools (Additional file 2). The quality of the included studies should not be different when different tools are used, when the studies are assessed against the same questions from the existing validated critical appraisal tools.

Comparison of the overall appropriateness of eye care versus the appropriateness for individual domains of eye care between studies presented some challenges for the following reasons:

  1. 1)

    Differences in the number of quality indicators used. Seven quality indicators were used in the Zebardast et al. [48] study, but 19 quality indicators were used by Ong et al. [50] Although both studies assessed appropriateness of eye care against the same glaucoma guidelines, the overall result cannot be easily compared, unless this is done by comparing appropriateness of care of individual quality indicators used by both studies.

  2. 2)

    Differences in eligibility criteria and time frame of quality indicators. Quigley et al. [52] assessed whether practitioners have performed gonioscopy at least once within the previous 6 years for all patients with open-angle glaucoma and found that appropriate care was delivery only 50% of the time. Conversely, Ong et al. [50] reported 90% appropriate care for performing gonioscopy on indication. A possible conclusion may be that practitioners in the latter study performed much better than in the former. However, careful observation of the study population characteristics reveals that this appropriateness of care results simply reflects how often practitioners perform gonioscopy in open angle glaucoma in the first instance and use of gonioscopy in cases with a suspicious angle in the latter study.

  3. 3)

    Differences in time interval. Chawla et al. [27] assessed both planned and actual review interval for glaucoma against the guidelines whereas Ong et al. [50] only assessed if the planned follow-up complied with guidelines.

  4. 4)

    Different aspects of the quality indicator are assessed. Appropriateness of ‘referral’ can be considered in terms of the appropriateness of the referral criteria, the timing of the referral or in terms of the appropriateness and contents of referral letters. Appropriateness of referral often describes whether patients were referred to the correct people or facilities. Appropriateness and contents of referral letters typically considers if the referral letters contained the required information, according to guidelines or specialist’s opinions. However, the percentage of appropriate care of these two aspects may not directly be comparable. Appropriateness of referral pathway or criteria is not necessarily equivalent to an appropriate referral letter and vice versa. For example, Ang et al. [62] reported that the appropriateness of referral letters from optometrists referring for glaucoma progression was 32% whereas the appropriateness and contents of their referral letters exhibited 24–93% compliance against the seven quality indicators used.

  5. 5)

    Differences in quality indicator weighing. Most studies weighed all quality indicators evenly, but some assigned different weightings for different quality indicators. Quigley et al. [52] assigned weighting (0, 1, 2 or 3) according to the imputed importance of individual items. Gupta et al. [140] defined appropriate care as the practitioners documenting 50% or more of the sub-indicators listed for each element. For example, once 2 or more of the 4 sub-indicators (frequency of deviation, date of onset, and presence of diplopia or squint) of ocular signs and symptoms were documented, this quality indicator was counted as compliant.

The findings of this systematic review are limited by the lack of a standardised method to measure and report the appropriateness of eye care delivery. The extent to which eye care appropriateness may have been under or overestimated may be significantly influenced by the choice of method used to assess care delivery in these studies. Two-thirds of the included articles measured compliance against recommendations from clinical practice guidelines, which are likely to have been developed using similar evidence sources. In this review, this is likely to have manifested as reporting the appropriateness of eye care according to a somewhat narrow evidence base. However, clinical practice guidelines are primarily developed for and made available to clinicians for the purposes of guiding evidence-based care, which lends credibility to their use as a compliance tool. In addition, studies conducted in one country might not reflect the appropriateness of eye care received in a different country where the health care and education systems, values and expectations could be significantly different [144]. Given that and the diversity of countries where eye care appropriateness has been measured, the generalisability of the various reported findings to other countries is uncertain.

Conclusion

Studies reporting the appropriateness of eye care delivery in Australia and other developed mainly English-speaking countries, indicated a wide range of appropriateness of care delivery, for glaucoma and diabetic eye care. Existing eye-related studies have assessed a single condition, a sample from a single centre or a single domain of care even as specific as only one examination technique such as gonioscopy. Consequently, none of the studies identified in the literature review attempted to measure the overall appropriateness of care provided in eye care. One important purpose of measuring appropriateness of care is to help policy makers to allocate limited health resources. Future research would benefit from a more comprehensive approach where appropriateness of eye care delivery is measured across multiple conditions with a single methodology to guide priorities within eye care delivery and monitor quality improvement initiatives.