FormalPara Key Summary Points

The development of a disease activity index could be a major advance in clinical trials as well as in evaluation of treatment response of patients with uveitis in routine clinical practice.

Our group developed a composite disease activity index for patients with uveitis which has been validated, showing considerable discriminative power in classifying inflammatory activity.

The aim of this study was to determine the interobserver reliability of the index and assess its sensitivity to change in patients receiving pharmacologic treatment. The study population was patients diagnosed with noninfectious uveitis and inflammatory activity at baseline.

The interobserver reliability of UVEDAI was high in the overall sample, and the composite index was sufficiently sensitive for determining the change in inflammatory activity after treatment.

We believe that UVEDAI is a disease activity index that enables objective comparison of results in both routine clinical practice and clinical trials.

Introduction

Uveitis is defined as inflammation of the uvea, the middle vascular layer of the eye. Nowadays, however, uveitis is a generic term used to describe a heterogeneous group of diseases characterized by intraocular inflammation [1] and a variable clinical course. It is classified according to anatomic location (anterior, intermediate, and posterior uveitis and panuveitis), although its course and other morphologic parameters also play a relevant role [2]. Clinical activity is evaluated based on the combination of clinical signs and the opinion of the ophthalmologist. Uveitis is one of the main causes of visual disability and blindness in developed countries [3,4,5]; therefore, assessment of inflammation for early and appropriate treatment is very important. Of note, the types of uveitis that most frequently threaten visual acuity are those that affect the posterior segment (intermediate and posterior uveitis and panuveitis). These three anatomic sites usually require similar therapeutic strategies [i.e., systemic corticosteroids, synthetic disease-modifying antirheumatic drugs (s DMARDs), and biologic disease-modifying antirheumatic drugs (b DMARDs) and are grouped together to enable clinical studies on treatment], even if their etiologies differ [6, 7]. Reliable and sensitive assessment tools are essential if we are to monitor disease activity effectively and evaluate response to treatment.

Assessment of the clinical course of uveitis can be hampered by the lack of standardized and validated outcome measures of disease activity, which makes it difficult to compare efficacy and response to treatment [8,9,10]. Along these lines, Denniston et al. [11] performed a systematic review of clinical trials and studies of treatments for uveitis. The authors highlighted the heterogeneity of the primary results owing to “a lack of consensus over which outcome measure(s) to use, and how to measure them, results in disparity of study design which limits evidence synthesis,” also stating that the “ability to compare new results to other studies is often a key requirement of regulatory authorities when evaluating and licensing novel therapeutics.” Therefore, a composite index of ocular disease activity for comparing outcomes would be very useful in efficacy studies and clinical trials for the development of new therapies. Similarly, a recent review of the literature on clinical trials [12] insisted on the poor definition of primary outcome measures in up to 12% of trials [11] and reported that > 20% of trials use multiple primary outcome measures, in contrast to the recommendations of the CONSORT (Consolidated Standards of Reporting Trials) guidelines [13]. The literature contains at least two composite indices [14,15,16,17], which, although not validated, have been used in some studies.

In 2014, this working group developed a composite uveitis disease activity index, UVEDAI, to assess overall ocular inflammatory activity [18]. The index included seven variables used in daily clinical practice, each of which has a specific weight in the calculation of the index depending on its contribution to the inflammatory activity of the eye. The score obtained classifies the level of ocular activity as mild, moderate, or severe, and its discriminatory capacity is very high (> 85%). The first part of the study was performed in 2019, when the index was validated (construct validity and criterion validity) [19]. In this second part, the interobserver reliability and sensitivity to change of the UVEDAI index are assessed to provide valuable information on its performance as a standardized and objective tool for the assessment of uveitis.

The aims of this study were to determine the interobserver reliability of the UVEDAI index and to assess its sensitivity to change in patients with noninfectious uveitis receiving pharmacologic treatment.

Methods

Design

This was a multicenter, prospective, longitudinal, observational study in eight multidisciplinary uveitis units of Spanish tertiary hospitals.

Study Population and Recruitment

The study population comprised patients aged ≥ 18 years with a diagnosis of noninfectious uveitis and involvement of any anatomic location (anterior, intermediate, posterior, and panuveitis), with inflammatory activity at the time of the visit, and for whom pharmacologic treatment was prescribed. Patients who were in complete remission, those participating in a clinical trial or an associated research project, and those with postsurgical or traumatic uveitis were excluded. The patients in this study differed from those recruited in the first part (validation of the score) [19] to avoid circularity or contamination of the dependent variable. The recruitment period ran from September 2020 to March 2022. The study was performed at the specialized care level of the National Health Service. We invited eight multidisciplinary uveitis units from National Health Service hospitals to participate. All the centers had complete ophthalmologic examination equipment (as reported elsewhere) [18], which included spectral-domain optical coherence tomography (OCT) devices.

Sample Size Determination

The sensitivity to change of the UVEDAI was assessed based on the results of the first part of the study, the validation, and a pilot test carried out in daily clinical practice with 31 patients. The investigator group considered that the clinically relevant change (CRC) had to be at least 0.8 points between the assessment at baseline and at 4 weeks. This value was obtained from the average UVEDAI, which was simulated considering the criteria for improvement of inflammatory activity used in routine clinical practice. The sample size necessary for a CRC of ≥ 0.8 points based on standard alpha and beta levels of 0.05 and 0.2 (contrast power 0.8) and bilateral contrast was 110 patients. Assuming possible losses of 15%, the final sample size was set at 127 patients. The statistical analysis was performed using the GPower 3.0 software application.

Ethics Statement

This study involved human participants and was approved by the Hospital Clínico San Carlos Ethics Committee (Title: Validation and Sensitivity to Change of an Ocular Inflammatory Activity Index: UVEDAI. Internal Code: 18/196-O_SP. Sponsor and Funder: Spanish Society of Rheumatology). Participants signed an informed consent (IC) before participating in the study.

The study has been carried out in accordance with the principles outlined by the Declaration of Helsinki in its latest revision. International standards regarding the execution of epidemiologic studies, included in the International Guidelines for Ethical Review of Epidemiological Studies (Council for the International Organizations of Medical Sciences-CIOMS, Geneva, 1991), have been followed, in addition to the recommendations of the Spanish Epidemiological Society (SEE) regarding the review of all ethical aspects of the epidemiologic study.

The IC of patients has been obtained and has been an indispensable requirement for inclusion in the study.

This study did not involve animal subjects.

Patient and Public Involvement

No patients were involved.

Study Development

At the baseline visit, two ophthalmologists (investigator/ophthalmologist 1 and collaborator/ophthalmologist 2) independently evaluated the level of ocular inflammation based on the UVEDAI variables. In addition, to minimize variability, both ophthalmologists evaluated the patient independently during the visit. In addition, sociodemographic variables, the anatomic location of the uveitis, etiologic diagnosis, patient's previous treatments, and pharmacologic treatment prescribed by ophthalmologist 1 were collected during this visit. Ophthalmologist 1 prescribed the pharmacologic treatment according to local clinical practice criteria and saw the patient again at 4 weeks to determine ocular inflammatory activity using UVEDAI and collected adverse effects by means of open-ended questions according to standard clinical practice. The time window between the estimated 4-week visit date and the actual date did not exceed ± 3 days.

Operational Variables and Definitions

An electronic case report form (eCRF) was prepared to collect sociodemographic variables (sex, age, educational level), decimal best-corrected visual acuity (BCVA), UVEDAI index variables [anterior chamber cell grade, vitreous haze, macular edema, number of chorioretinal lesions, vasculitis, papillitis, and patient assessment measured using a visual analog scale (VAS), scored from 0 to 10, where 0 is “very good” and 10 “very bad”], site affected by uveitis, etiologic diagnosis, previous treatments, treatment prescribed at the baseline visit, index variables at the 4-week visit, and adverse effects. The eCRF can be used to calculate the score at each visit based on the data for the seven variables. The index classifies ocular inflammatory activity into three levels: mild, ≤ 1.05; moderate, between 1.05 and 4.86; and severe, > 4.86. The operational definitions of the variables have been described elsewhere. [18]

Statistical Analysis

A descriptive analysis of sociodemographic variables and variables of interest was performed for the overall sample. Symmetric continuous variables were expressed as mean and standard deviation. Categorical variables were expressed as absolute frequency and percentages.

Interobserver Reliability

Interobserver reliability was assessed based on variables of the UVEDAI that were registered by the two ophthalmologists in each specialist uveitis unit at each participating center at the baseline visit. Interobserver reliability was measured by calculating the intraclass correlation coefficient (ICC) [20], which incorporates sources of variability from different observers and measurement error in the analysis. The result is interpreted as a percentage of variability that depends on variability between participants. An ICC value between 0.75 and 0.9 indicates good reliability [21]. Moreover, the mean UVEDAI scores obtained by the two ophthalmologists from the more active eye were compared using the repeated-measures t-test.

Sensitivity to Change

To evaluate sensitivity to change, i.e., the CRC, the mean of the initial UVEDAI score collected by ophthalmologist 1 from the more active eye at baseline was compared with the mean of the final UVEDAI score obtained after 4 weeks from the same eye. The mean change between both measures was compared using the repeated-measures t-test [22, 23]. The analyses were performed on the global sample and by differentiating according to the anatomic location of uveitis (anterior vs. intermediate/posterior/panuveitis).

The statistical analysis was performed using the STATA statistical package, version 13.1 (copyright 1985–2013 StataCorp LP Statistics/Data Analysis. StataCorp 4905 Lakeway Drive College Station, Texas 77845, USA) and SPSS version 21.0 (IBM Corp. Released 2012. IBM SPSS Statistics for Windows, version 21.0. Armonk, NY: IBM Corp.).

Results

General and Sociodemographic Data

A total of 111 patients were included in the interobserver reliability phase [60 males (54.1%) and 51 females (45.9%); mean age 49.9 ± 15.8 years]. Table 1 shows the percentage of uveitis by etiology and by anatomic pattern. Regarding visual acuity, those patients with anterior uveitis had better visual acuity than those with intermediate or posterior uveitis. In terms of treatment, 63 (56.8%) patients were on uveitis medication before the baseline visit, the most frequent drugs being topical corticosteroids (Supplementary Table 1). At baseline, the principal ophthalmologist prescribed the new treatment, which was topical, systemic, or both. Most patients (88.1%) received topical treatment, the most frequent being corticosteroids in monotherapy or with a cycloplegic (Supplementary Table 2). In 46% of patients, systemic treatment was indicated, the most common being corticosteroids, followed by conventional s DMARDs [6, 9], the most common being methotrexate [24, 25]. Of the b DMARDs, anti–tumor necrosis factor agents (anti-TNFα) [26, 27] were the most frequently prescribed.

Table 1 Sociodemographic characteristics, clinical assessment (uveitis activity at baseline visit in the more active eye by ophthalmologist 1), and UVEDAI score
Full size table

The mean UVEDAI value collected in the more active eye at the baseline visit by ophthalmologist 1 was 1.9 ± 1.8 (Table 2). If we differentiate by anatomic location, the mean UVEDAI value was 1.2 ± 1.6 in anterior uveitis and 2.8 ± 1.8 in intermediate/posterior/panuveitis (Table 2). The UVEDAI score recorded by ophthalmologist 2 was similar (Supplementary Table 3).

Table 2 Differences in the UVEDAI value for the active eye at the baseline visit and ICC values of the UVEDAI variables obtained by ophthalmologists 1 and 2
Full size table

The UVEDAI value classifies the level of ocular inflammatory activity, not the severity of uveitis, into three categories: mild, ≤ 1.05; moderate, between 1.05 and 4.86; high/severe, > 4.86. At the baseline visit, 40.5% of patients had mild activity, 51.4% had moderate activity, and only 8.1% had high activity (Table 3).

Table 3 Clinical assessment at 4 weeks. Inflammatory activity by anatomical location and difference in UVEDAI value measured by ophthalmologist 1 in the active eye at baseline and at 4 weeks
Full size table

Interobserver Reliability

The ICC for the UVEDAI value was 0.9, which means that 90% of the variance of the UVEDAI values depends only on the variability of the patients. Since the ICC is very reliable for values between 0.75 and 0.9 [21], agreement between the score obtained by the two ophthalmologists was good. Analysis of the variables that make up the index (minus the variables for macular edema and patient assessment, in which the two ophthalmologists reported the same value) revealed that the highest ICC value was 0.87 for the vasculitis and papillitis variables and the lowest was 0.75 for the vitreous haze variable (Table 2). Likewise, the index values of the two ophthalmologists by anatomic location and overall were, respectively, 1.2 ± 1.6 and 1.2 ± 1.6 for anterior uveitis, 2.8 ± 1.8 and 2.7 ± 1.7 for intermediate/posterior uveitis, and 1.9 ± 1.8 and 1.8 ± 1.8 overall (Table 2). In 64 patients (57.7%), the ophthalmologists obtained the same UVEDAI score. For the remaining patients, the differences in index scores between the ophthalmologists ranged from 0.09 to 2.35 points (Fig. 1). Comparison of the mean UVEDAI values revealed no statistically significant differences (p value > 0.05, Table 2) for the total sample or after differentiation by anatomic location. However, when the anatomic location was intermediate/posterior/panuveitis, the difference in the activity index score between the ophthalmologists was greater.

Fig. 1
figure 1

Differences in UVEDAI score between ophthalmologist 1 and 2 at baseline visit

Full size image

Sensitivity to Change

Sensitivity to change was assessed in 110 patients in the eye with higher activity at baseline after re-evaluation at 4 weeks (53.2% right eyes and 46.8% left eyes), as established by ophthalmologist 1. There were statistically significant differences in the mean UVEDAI values between the baseline visit and 4 weeks, in both the overall sample and after differentiation by anatomic location of uveitis (Table 3). In all cases, the index value decreased significantly by > 1 point at 4 weeks of treatment. The VAS also improved significantly from 5.4 ± 2.4 at baseline to 1.8 ± 1.8 at the follow-up visit (Tables 1, 3).

If we look at the change in uveitis activity in the overall sample after treatment, of the nine cases with initially high values, these became mild in five (55.6%), moderate in three (33.3%), and remained unchanged in one. Of the 57 cases of uveitis with moderate activity initially, this became mild in 36 (63.2%) and remained unchanged in the other 21 patients (36.8%). Of the 45 cases of initially mild uveitis, 1 became moderate after treatment, while the other 44 (97.8%) remained mild or went into remission (Table 3).

Discussion

The objective of this study was to determine the interobserver reliability and sensitivity to change of UVEDAI, a score that assesses ocular inflammatory activity as a whole in patients with uveitis who were receiving pharmacologic treatment. The results confirm that interobserver reliability for the index was good, with an ICC of 0.9 [21]. As for the sensitivity of the index to change at 4 weeks, statistically significant differences were found for both the sample as a whole and the different anatomic sites. In all cases, the value of the index decreased significantly by > 1 point after treatment. The results provide valuable insights into the performance and usefulness of UVEDAI for assessing uveitis activity and monitoring treatment efficacy and could be used in clinical trials.

The demographic data we recorded were similar to those reported in other studies [28], namely, the pattern of anterior uveitis was the most common [29, 30] and the most prescribed treatment was topical (corticosteroids and cycloplegics). Systemic therapy was administered in intermediate/posterior/panuveitis, sometimes combined with topical therapy. This was often initiated at the baseline visit. The most common drugs were corticosteroids, the most common s DMARDs was methotrexate [24, 25], and the most common b DMARDs were anti-TNFα agents [26, 27]. Notably, oral acetazolamide [31] was used in some cases, possibly to reduce macular edema.

Interobserver reliability is a crucial aspect when applying an assessment tool in clinical practice, after construct and criterion validation [19]. Our results showed that the ICC value for UVEDAI was good both overall and for each of the variables, with the lowest value recorded for vitreous turbidity being 0.75. While vitreous turbidity has been used as an evaluation criterion in multiple clinical trials, it is measured on an ordinal scale (0+ to 4+) and is an indirect marker of inflammation. Moreover, it has been called into question because it is not linear, it is subjective, and it is difficult to distinguish between the subtle differences in vitreous haze at lower grades [32, 33]. In UVEDAI, vitreous haze was categorized as mild (1+ to 2+) and moderate (3+ to 4+) [18] and was the variable for which the greatest interobserver difference was identified, even though this was not significant. No significant differences were found when the mean value of the index was compared at baseline in the sample overall and by anatomic location between the two ophthalmologists. The high ICC value of 0.9 indicates strong agreement between the two observers in evaluating uveitis activity using UVEDAI. This suggests that the index is consistent and reliable, enabling different clinicians to obtain similar results when assessing the activity of uveitis. Furthermore, it implies that the index provides a standardized and objective approach for evaluating ocular inflammatory activity in patients with uveitis. This consistency in assessment is essential for effective communication among healthcare providers and for monitoring disease progression.

As for sensitivity to change of the index after 4 weeks’ treatment, we found statistically significant differences in the sample overall and at the different anatomic sites. In all cases, the value of the index decreased significantly by > 1 point at 4 weeks, as did the patients’ overall evaluation of their disease. In terms of anatomic site, activity of anterior uveitis became mild or the disease went into remission in 95% of cases, whereas 48% of patients still had moderate activity in intermediate/posterior/panuveitis, although most had improved. This was probably because intermediate/posterior/panuveitis requires longer term systemic treatment with corticosteroids and b/s DMARDs and improvement is more gradual, although the index reflected a significant change in the score. Uveitis is a complex and heterogeneous condition, and its activity can vary over time. This finding suggests that UVEDAI is sensitive to changes in the activity of uveitis, making it a valuable tool for monitoring response to treatment, deciding on therapy, minimizing disease-related complications, and improving patient outcomes. Therefore, we believe that these data show that the UVEDAI scoring system will enable objective and unbiased evaluations of patients in daily practice. It will also minimize errors in the measurement, analysis, and interpretation of clinical trial outcomes, as has already been achieved with other scores in different pathologies [32].

Strengths and Limitations

The strengths of the study include the fact that it was performed over a long period in daily clinical practice (eye examination without invasive techniques) at the same centers in all the phases of the study and by professionals with experience in evaluating the variables. In addition, we used spectral domain OCT to prevent deviations in the value of macular edema. To our knowledge, compared with other indices [15, 33], UVEDAI is the only ocular inflammation index to be validated, and we believe that it could prove useful as a standardized tool based on exploratory parameters and OCT data. It could also prove an essential element for evaluation of patients with uveitis, obviating the need for complex equipment (e.g., fluorescein angiography, autofluorescence).

It is important to acknowledge the limitations of this study. First, we were unable to reach the initially calculated sample size owing to difficulties recruiting patients during the COVID pandemic. However, we did achieve a significant sample; this enabled us to perform robust analyses and draw significant conclusions. Moreover, only one loss to follow-up was recorded, thus minimizing the potential bias associated with a small sample. Second, only noninfectious uveitis was included to ensure that the sample was more homogeneous; however, we believe that the index can be applied in any type of uveitis, since it evaluates the degree of inflammation and not the etiology of the disease. Lastly, while UVEDAI makes it possible to evaluate inflammatory activity in the same way as other scores used in other diseases [32, 34], it does not evaluate the severity of uveitis, which is determined by the ophthalmologist based on data from the examination, activity index, and his/her clinical opinion.

Conclusion

The interobserver reliability of UVEDAI was high in both the overall sample and for the different variables. Similarly, the index was sufficiently sensitive for determining the change in inflammatory activity after treatment in anterior uveitis and in intermediate/posterior/panuveitis. We believe that UVEDAI is suitable for clinical practice and for clinical trials and studies, enabling objective comparison of outcomes.