FormalPara Key Summary Points

Why carry out this study?

Remote monitoring of visual function in patients with maculopathy may ease the burden on healthcare resources and enable detection of disease progression between clinic visits.

We aimed to determine whether the myVisionTrack app tests of visual acuity and contrast sensitivity are repeatable and whether they agree with standard clinical tests of visual function.

What was learned from this study?

In the Aphelion study, a single-arm study of 122 participants with maculopathy, myVisionTrack tests were repeatable and demonstrated agreement with standard clinical tests in the clinic setting.

Participants found the tests easy to learn and use.

This study demonstrates the feasibility of deploying the myVisionTrack tests for the self-assessment of visual function in patients with maculopathy.

Introduction

Diseases of the macula (maculopathies), such as age-related macular degeneration (AMD) and diabetic macular edema (DME), exhibit variable rates of progression [1,2,3,4]. Timely intervention when disease activity is detected is essential to preserve sight [5,6,7,8]. As such, the fundamental goal of patient self-testing of vision is to reveal disease activity occurring between clinic visits, so that patients can be brought in promptly for clinical evaluation and, if warranted, treatment. This goal becomes ever more important with longer-acting treatments extending the interval between clinic visits [9,10,11,12], and with the pressures of an aging population increasing the burden on eye clinics.

Remote monitoring solutions could decrease this burden, but they must be evidence based, provide clinically valid, accurate information, and be usable by patients and clinicians. Effective remote vision monitoring has the potential to be implemented across the continuum of care, including monitoring quiescent disease, the early detection of progression from intermediate to advanced disease or disease onset in the fellow-eye, and treatment optimization in patients receiving anti-vascular endothelial growth factor (VEGF) and other therapies.

Remote self-testing in patients with maculopathy typically relies on the decades-old paper Amsler grid with which patients subjectively report the perception of distortions or missing fields. This tool is limited by the lack of a quantifiable measure of visual function to detect changes over time, and by the poor validity for the detection of retinal defects [13]. Indeed, a meta-analysis published in 2023 concluded that the Amsler grid performed at a sensitivity of less than the typically recommended level of 70% [14]. With the increasing usage of smartphones [15], including among older adults [16], it is now feasible to make digital vision function tests available to a sizable proportion of patients with maculopathy. Furthermore, smartphone-based visual function tests benefit from a lower cost burden and simpler rollout than hardware-based solutions (such as home optical coherence tomography) [17, 18]. Accordingly, several software-based devices have been developed to track changes in vision [19, 20].

One such software-based solution is myVisionTrack (mVT; or Home Vision Monitor). The mVT solution, which utilizes the shape discrimination hyperacuity (SDH) test, is a prescription-only, software-based medical device intended for the detection and characterization of central three degrees metamorphopsia in patients with maculopathy [19]. Hyperacuity requires global integration of visual stimuli over a large portion of the retina and can detect the progression of maculopathy [19, 21, 22]. During self-monitoring from the mVT app at home, a change from a patient’s baseline visual function triggers an alert to their clinician. In a real-world study during the COVID-19 pandemic, the mVT solution was deployed to patients at the Moorfields Eye Hospital [23]. Of the patients for whom the app alerted their clinician to a deterioration in vision, 42% had their in-clinic appointment brought forward, and 85% were confirmed as having active disease requiring anti-VEGF treatment [23]. This study illustrated the potential role for remote vision monitoring to offer patients and clinicians a safety net for early vision loss versus standard of care scheduling.

An expanded version of the mVT solution, mVTx, has been developed to include digital near visual acuity (VA) and contrast sensitivity tests to provide a more comprehensive vision testing suite that could expand the applicability of the mVT solution to a wider range of diseases and could provide complementary information on visual function. VA and contrast sensitivity measurements are routinely used in clinical practice and are, therefore, more familiar to clinicians than the SDH test.

The purpose of the Aphelion study was to evaluate the repeatability, accuracy, and usability of these additional digital visual function tests: VA measured by both a tumbling E and a Landolt C stimulus, and a tumbling E contrast sensitivity test.

Methods

Setting and Patient Population

Aphelion was a single-arm, prospective study. Study procedures were approved by the Alpha Independent Review Board (IRB00006205). The study was conducted in conformance with the Guidelines for Good Pharmacoepidemiology, and informed consent was obtained from all study participants. Adults with any maculopathy and a habitual VA in the study eye of 0.7 log of minimum angle of resolution (logMAR) (20/100 Snellen equivalent) or better, measured by the Early Treatment Diabetic Retinopathy Study (ETDRS) chart at 4 m, were recruited consecutively at two sites in the USA, Andover Eye Associates and the Retina Foundation of the Southwest, between August 20, 2020 and December 18, 2020. Participants were excluded if they had received an intravitreal anti-VEGF injection in the study eye within 1 week prior to their study visit. Participants were also excluded if their vision was limited by a comorbidity in addition to a maculopathy that would require surgical intervention to prevent or to treat vision loss (e.g., cataract, glaucoma) or that would affect the interpretation of the study results (e.g., glaucoma, epiretinal membrane, amblyopia, strabismus). Participants were excluded if they had any limitation that would prevent them from performing the mVTx tests (e.g., insufficient knowledge of the English language, physical or mental inability to give informed consent, or inability to participate in examinations) or if they were participating in another Roche- or Genentech-sponsored clinical trial.

Description of mVTx Tests

All mVTx tests were performed on an iPad Air 3 (Apple, Cupertino, CA, USA). The mVTx tests all employ the same underlying strategy in which the user makes a four-alternative forced choice (Fig. 1).

Fig. 1
figure 1

The patient view during app testing. a Tumbling E. b Landolt C. c Contrast sensitivity. d SDH test. SDH shape discrimination hyperacuity

Full size image

For both the tumbling E (Fig. 1a) and Landolt C (Fig. 1b) tests, the user is presented with four optotypes simultaneously in one of four orientations and is asked to identify the single optotype that has a different orientation compared with the other three. The contrast sensitivity test (Fig. 1c) also uses a tumbling E stimulus with one optotype in a differing orientation. The SDH test (Fig. 1d) is a measure of hyperacuity and has been described previously [19, 22, 24]. During the test, the user is presented with three undistorted circles and one radially distorted circular shape and asked to identify the distorted shape. In each test the size, contrast, or extent of radial distortion is reduced or increased in a stepwise manner depending on the user’s previous response and the score calculated using a maximum likelihood fitting procedure. Contrast sensitivity was reported in log contrast threshold (logCT), the threshold value at which an object (the tumbling E optotype) can be distinguished from the background. The logCT expresses the relative difference in luminance between the object and the background on a log scale. VA and SDH test results were reported in logMAR. Although the results for the VA and the SDH tests are both reported in logMAR, each test measures a different aspect of vision, and the normal ranges for these measures will differ substantially between tests [25].

Test Procedure

All assessments were conducted in the clinic. Participants used their habitual spectacle or contact lens prescription they would normally use for near-vision activities during testing. Testing was performed monocularly, with the nonstudy eye covered by an eye patch or spectacle occluder. The device screen was set at 80% brightness. Room illumination was not specified and followed the standard clinical procedure in place at each study site. The requested distance between the patient and the screen was 40 cm; a chin rest was not used in order to simulate real-world conditions.

The study visit could be combined with screening (Fig. S1 in the Supplementary Materials). Participants first performed a set of mVTx practice tests to become familiar with the test format and to verify that they could perform them without issues. Then participants completed the first round of testing (the mVTx and the standard clinical tests), a user survey, and the mVTx retests. To reduce order effects of fatigue or practice, participants were randomized to perform the tests in one of 12 different order permutations, with approximately 10 participants allocated to each sequence (Table S3). Order permutations were such that participants completed either the mVTx or the standard clinical tests first; 62 participants completed the mVTx tests first and 60 participants completed the standard clinical tests first. While the order of the mVTx tests was randomized, the SDH test was always completed last. There was no maximum time limit to complete the tests. Participants could take a short break between each test, but not during a test.

Outcome Measures and Data Analysis

The repeatability of the mVTx tests was assessed by comparing the results at first test with the results of the second test (retest).

The agreement between the mVTx tests and the clinical tests of visual function was assessed by comparing the first mVTx test with the corresponding clinical standard. Tests of VA (the tumbling E and Landolt C tests) were compared with both the near (40 cm) and distance (4 m) ETDRS charts. The mVTx contrast sensitivity test was compared with the Pelli–Robson chart at 1 m. As a comparator, the agreement between the near and distance ETDRS chart results was also assessed.

Bland–Altman analyses [26, 27] and calculation of the 95% limit of agreement (LoA) from the standard deviation (SD) of the difference of paired comparisons were performed for all test–retest and clinical-test comparisons. Spearman’s rank correlation was also reported.

A user survey recorded participants’ opinions on their ability to learn and use the mVTx tests, and their access to a smartphone or tablet (Table S1 in the Supplementary Materials).

The time taken to perform each mVTx test was also recorded.

Results

Patient Population

Of the 122 participants included in the study, 44.3% were men and most (90.2%) were White (Table 1). The mean age of participants was 67 years (19–86 years). The mean distance VA at baseline was 0.3 logMAR (SD, 0.2 logMAR) and the most common maculopathy was AMD (42% of patients).

Table 1 Baseline participant demographics and disease characteristics
Full size table

Test–Retest Repeatability

There was no consistent bias between test and retest scores for all the mVTx tests (Fig. 2). Among the additional mVTx tests, the tumbling E test (Fig. 2a) had a narrower 95% LoA of ± 0.18 logMAR (mean difference, + 0.01 logMAR) than the Landolt C test (Fig. 2b) with an LoA of ± 0.23 logMAR (mean difference, + 0.01 logMAR). The contrast sensitivity test (Fig. 2c) had an LoA of ± 0.32 logCT (mean difference, 0.00 logCT). For the original SDH test (Fig. 2d), the 95% LoA was ± 0.24 logMAR (mean difference, + 0.01 logMAR).

Fig. 2
figure 2

Bland–Altman plots for the test–retest repeatability of a tumbling E, b Landolt C, c contrast sensitivity, and d SDH test. Blue (central) and red (upper and lower bound) shading indicate the 95% CIs of the mean difference and LoAs, respectively. logMAR log of minimum angle of resolution, LoA limit of agreement, CI confidence interval, logCT log contrast threshold, SDH shape discrimination hyperacuity

Full size image

In accordance with the narrow LoA, the test and retest scores were correlated for all mVTx tests (tumbling E, r = 0.89; Landolt C, r = 0.84; contrast sensitivity, r = 0.74; SDH, r = 0.90).

Some participants reached the limit of the mVTx contrast sensitivity scale (scale range from − 2.0 to − 0.1 logCT): 17 participants (14%) on first test and 19 participants (16%) on retest.

Agreement with Standard Clinical Tests

Bland–Altman plots comparing each mVTx test with the corresponding clinical test are shown in Fig. 3. The summary data for all comparisons are shown in Fig. 4.

Fig. 3
figure 3

The agreement between the mVTx app and standard clinical tests. Panels show Bland–Altman plots for the agreement between a the mVTx tumbling E and the 4 m ETDRS chart, b the mVTx Landolt C and the 4 m ETDRS chart, and c the mVTx contrast sensitivity and the Pelli–Robson chart. Blue (central) and red (upper and lower bound) shading indicate the 95% CIs of the mean difference and LoAs, respectively. ETDRS Early Treatment Diabetic Retinopathy Study, logMAR log of minimum angle of resolution, LoA limit of agreement, CI confidence interval, logCT log contrast threshold, mVTx myVisionTrack expanded version

Full size image
Fig. 4
figure 4

The agreement between the mVTx app and standard clinical tests. aThe agreement between the two clinical tests of VA, near and distance ETDRS, is included for comparison. LoA limit of agreement, ETDRS Early Treatment Diabetic Retinopathy Study, mVTx myVisionTrack expanded version, VA visual acuity

Full size image

The 95% LoA between the distance ETDRS and the tumbling E test was ± 0.35 logMAR (mean difference, − 0.07 logMAR; Figs. 3a and 4a). For the near ETDRS test, the 95% LoA with the tumbling E test was ± 0.32 logMAR (mean difference, − 0.13 logMAR; Fig. 4b).

When the Landolt C test was compared with the distance ETDRS chart, the 95% LoA was ± 0.39 logMAR (mean difference, + 0.03 logMAR; Figs. 3b and 4c), and the 95% LoA between the near ETDRS chart and the Landolt C test was ± 0.34 logMAR (mean difference, − 0.03 logMAR; Fig. 4d).

Both mVTx VA tests were correlated with the distance ETDRS test (tumbling E, r = 0.59; Landolt C, r = 0.55). Correlation coefficients were higher between both mVTx VA tests and the near ETDRS test (tumbling E, r = 0.73; Landolt C, r = 0.67).

The 95% LoA between the Pelli–Robson chart and the mVTx contrast sensitivity test was ± 0.30 logCT (mean difference, − 0.25 logCT; Figs. 3c and 4e) and scores were correlated (r = 0.75).

In comparison, the 95% LoA between the two standard clinical tests of VA, the near and distance ETDRS charts, was ± 0.36 logMAR (mean difference, − 0.06 logMAR; Fig. 4f), demonstrating a level of inter-test agreement similar to that of the mVTx tests.

Patient Usability Survey Data

Of the 122 study participants, 108 completed the usability survey. Nearly all the respondents had a smartphone (94%) or tablet (58%). Most (81%) reported that they would be comfortable taking the tests at home.

The survey data indicated that the tests were quick and easy to learn. Of 108 respondents, 92 (85%) agreed that they were able to learn the mVTx tests quickly. The majority of respondents found the mVTx tests easy to use (Fig. S2 in the Supplementary Materials). The tumbling E test scored highest on ease of use (72% agreed that it was easy to use), followed by the SDH test (62%), the Landolt C test (60%), and the contrast sensitivity test (58%). When the two mVTx VA tests were directly compared, 81/108 (75%) of respondents found the tumbling E test easier to use than the Landolt C test.

Most participants expressed confidence that they were performing the tests to the best of their ability (tumbling E test, 93%; Landolt C test, 92%; contrast sensitivity test, 92%; SDH test, 93%).

Of the respondents, 34% were willing to spend 11–15 min twice a week completing the test suite.

Time Taken to Perform the Tests

The median time taken to complete each mVTx test ranged from 2.28 to 3.75 min (Table S2 in the Supplementary Materials).

Discussion

Patients with maculopathy require regular in-clinic examination to detect new or recurrent exudative disease in the treatment or fellow-eye, and to determine the optimum treatment intervals if undergoing anti-VEGF therapy [5]. This places a significant burden on healthcare systems and patients, with patients with AMD and DME attending on average 8–10 and 13–14 clinic appointments per year, respectively [28,29,30]. The ongoing development of a port delivery system for anti-VEGF therapy [9, 31] and longer-acting treatments [10, 11, 32,33,34] could enable a reduction in the frequency of clinic visits. However, disease reactivation could go unnoticed during prolonged periods between examinations [23]. Remote monitoring has the potential to function as a safety net, and provide both clinicians and patients with the confidence to extend the interval between clinic visits and enable patients to participate in their care.

In the Moorfields Eye Hospital real-world study of the mVT solution, remote monitoring of hyperacuity via the SDH test effectively identified vision deterioration and enabled clinicians to bring forward in-clinic appointments and treat when necessary [23]. We now provide evidence that including digital VA and contrast sensitivity tests within the mVT solution is feasible for assessing visual function in patients with maculopathy. These digital tests produced repeatable results, in agreement with standard clinical tests, and were acceptable to patients. Our simple, but relatively large, study involving 122 participants was adequately powered to assess differences among tests and provides the first evidence of convergent validity with standard clinical tests. In addition, supplementary results from the usability survey indicate that participants found the mVTx tests quick to learn and easy to use.

The repeatability of the mVTx tests was similar to published test–retest values for the ETDRS and Snellen charts at distance (LoA, 0.28 logMAR and 0.36 logMAR, respectively) [35], despite the mVTx app testing near vision. Moreover, there was no meaningful learning effect on retest, as indicated by the mean test–retest differences being close to zero. Eye test results, especially those for near vision, often show high test–retest variability [35,36,37,38,39,40,41,42], which is reflected in the study results and consistent with the literature. Indeed, test–retest variability for measures of VA is often higher in eyes affected by maculopathy than in healthy eyes [42,43,44].

The mVTx tests of VA compare favorably with other remote monitoring apps. The Peek Acuity, Vision@Home, and OdySight apps all use a tumbling E stimulus to measure VA. When compared with the distance ETDRS test, the reported LoA for these apps ranges from ± 0.22 to ± 0.41 logMAR [45,46,47]. Different study conditions may account for this range. For instance, the OdySight study used a chin rest to control the distance between the patient and the screen [47], whereas the Peek Acuity and Vision@Home studies advised a testing distance but did not use a chin rest [45, 46]. Like Aphelion, these studies were performed under supervised conditions, with less potential for variability than self-assessment.

Indirect comparison with test–retest data for other digital contrast sensitivity tests suggests that the mVTx test has similar, or in some cases better [48], repeatability. For instance, studies of the PeekCS app and a fast-screening iPad app reported 95% LoA from 0.24 to 0.56 logCT [48,49,50]. For mVTx, the equivalent value was 0.32 logCT. However, direct comparisons between these digital tests should be treated with caution owing to the differing patient populations and study settings. We found the Pelli–Robson scores to be, on average, 0.25 logCT poorer than those measured by the mVTx test. Post-study photometric measurements revealed the mVTx contrast to be accurate, but the contrast of the Pelli–Robson chart to be lower than specified, which is likely to account for the large difference between the two tests. Differences between the Pelli–Robson chart and digital contrast sensitivity tests have also been reported in other studies [49, 51].

This study adds to the growing body of literature on remote vision monitoring. A review of the evidence supporting remote monitoring of visual function via mobile app tests in patients with an existing maculopathy identified seven app-based tests [52]. The extent of supporting evidence ranges considerably between apps. Most apps are supported by studies of convergence with in-clinic visual function tests and rates of patient satisfaction or compliance, but test–retest repeatability studies are available for only two apps. For some, such as the SDH test, there are multiple studies in large numbers of patients, including real-world evidence of the app’s utility. Other apps are supported by only one publication with relatively few patients. With the exception of the SDH test, these apps lack real-world studies with long-term follow-up, and therefore data assessing the impact of remote vision monitoring on disease progression and patient outcomes are limited. Aphelion provides initial evidence on the repeatability and validity of the mVTx tests of visual acuity and contrast sensitivity. Future work will build on these initial findings to further validate the mVTx tests in non-supervised settings, under real-world use and evaluate patient outcomes over a longer follow-up.

Although Aphelion was designed to produce robust evidence on the repeatability and validity in a large population with diverse maculopathies, several limitations must be considered. First, the study would have benefited from a test–retest comparison of the standard clinical tests to provide a benchmark of the variability in this study population. Second, some participants reached the limit of the contrast sensitivity test scale. Since the study was completed, the range of contrast values recorded by the mVTx test has been extended to − 2.3 logCT. Third, variations in the testing distance may have occurred owing to the absence of a chin rest. Use of a chin rest would reduce the variability of the mVTx tests and improve the agreement with the standard clinical tests; however, variations in testing distance in the absence of a chin rest are likely to reflect the real-world use of the app. Fourth, this study was conducted in an artificial setting in the clinic. It remains to be seen how the variability of these mVTx tests would be affected by at-home, unsupervised use. As such, further studies are warranted to determine the clinical utility of these tests for home testing and which combinations of the available mVTx tests provide the most useful data for clinicians while maintaining ease of use for patients. Ongoing studies are evaluating unsupervised use of the mVTx tests as well as the effect of variations in test distance. Fifth, participants used their habitual correction for which their prescription may need updating. Contrast sensitivity and the SDH test are more resistant to errors in refractive correction than VA [53,54,55]. Finally, most participants were White; future studies should include a more diverse patient population.

Conclusions

Emerging digital tools, such as the mVTx app, offer the potential for remote monitoring of vision. We anticipate that these digital measurements of VA and contrast sensitivity will complement the existing SDH test to provide a more comprehensive coverage of visual function with the aim of more readily detecting progression from intermediate to advanced disease and to remotely monitor treatment effectiveness. The inclusion of digital VA and contrast sensitivity tests would offer clinicians familiar measures of visual function in addition to testing for changes in hyperacuity. For the patient, we anticipate that the time invested in taking the additional tests will be offset by the reassurance that their clinician can monitor several aspects of their vision in the periods between clinic visits.