Introduction

The Amsler charts were first used seven decades ago by Swiss ophthalmologist Marc Amsler. These “metamorphosia” detection instruments were considered to be inspired from similar charts consisting of parallel lines with a central fixation spot used for the same purpose in 1874 [1]. Ever since, Amsler grid testing has been widely used for the detection and monitoring of scotomas and metamorphopsia in the central 20° of the visual field, thus helping in diagnosis of diseases such as maculopathy, including Stargardt's disease, age-related macular degeneration (AMD), retinopathy (including diabetic, central serous), retinal detachments, and optic neuropathies.

Despite its historic acceptability due to low cost and ease of training the patients, the Amsler grid test has several limitations. Previous studies have reported that the Amsler test indicated the presence of scotomas in about 2% of control subjects without any scotoma [2]. Schuchard reported that the sensitivity of the Amsler chart was as low as 56% in comparison with fundus microperimetry [3]. Additionally, the patients with smaller scotomas (of less than 6° diameter) had a false negative rate of 77%. In another study by Lowenstein et al., the authors reported that the rate of the Amsler chart for the detection of AMD varied between 9% in early AMD and 34% in late AMD with choroidal neovascularization [4] The limitations of the Amsler test such as its non-interactive nature, missing fixation control, need for reasonable reading vision to discern the lines, low sensitivity due to a suprathreshold stimulus, its poor performance due to the “crowding effect,” and limited awareness of visual field defects until the scotoma is significantly large in size due to “filling-in phenomenon” amplified the need for the development of sensitive novel tools with a more user-friendly interface [5, 6] In this study, we assess metamorphopsia using a new software based on AMD-A Metamorphopsia Detector® [7, 8]. The MacuFix® test (app4eyes, developed by Ronald Krüger, patent DE 10 2019 205 318 A1) is available for use on a screen (personal computer[PC], smartphone, tablet, iPad) both for Android and iOS platforms. We determined the “user-friendly” interface using the “System Usability Score (SUS)” questionnaire and compared the adherence of self-monitoring with the Amsler grid versus the software MacuFix® [9, 10].

Materials and methods

This observational pilot study included 47 patients who were recruited from the patient pool of an ophthalmologic group practice and subjected to the study examination as part of their medically necessary control examination. The approval of the Ethics Committee of the North Rhine Medical Association (No. 600213225, dated April 1, 2020) was obtained to recruit patients for a prospective, controlled study. MacuFix® app as a home test (supplementary to the standardized clinical diagnostic protocol) and to start. A detailed signed consent was obtained from all the patients included in the study. The study was performed in strict compliance with the tenets of the Declaration of Helsinki of 1975, as revised in 2013.

Inclusion criteria

The patients of either sex above 18 years of age diagnosed with metamorphosia in at least one eye were identified in the clinic patient database using ICD code (H 53.15). The metamorphosia in the patients included in this study was detected using an Amsler grid and had a best-corrected visual acuity (BCVA) of at least 20/200. The patients who did not fulfill the inclusion criteria or those who had advanced glaucoma or intraocular surgery other than cataract surgery or vitrectomy within the prior three months of enrollment were not included in the study.

Study methodology

The primary endpoint investigated in the observational study was the SUS questionnaire score. The secondary endpoint investigated in the prospective, controlled interventional study arm was the frequency of Amsler test or MacuFix® use measured as the time interval between two measurements to assess test adherence. As part of a routine examination, each participating patient performed the MacuFix® test with each eye with adequate near correction once in the period May 4, 2020, and June 30, 2020. In addition, BCVA was measured, and retinal examination and spectral-domain Optical Coherence Tomography (SD-OCT, CIRRUS ™ HD-OCT, Carl Zeiss Meditec) were performed. All study patients filled out the pseudonymized SUS questionnaire after completing the MacuFix® test. The patients were provided with the link to the MacuFix® for home test using a PC. Alternatively, the patients could download the test as an application for use with an iPad or smartphone. This offer was completely voluntary. Patients could decide whether they wanted to use the app. On the study day, each patient was asked about the frequency of use of the Amsler test from January 2020 to March 2020 (baseline period). The standardized question was: “I would like to know from you how often you have used the Amsler Grid in the past. The following question refers to the period from January 2020 to March 2020: How many days usually elapsed after a test with the Amsler Grid until the next test?” At 20–24 weeks after the study day, the patients were telephonically contacted and were asked a standardized question referring to the previous 3 months (comparison period)—“I would like to ask you two questions about the Amsler Grid and the Macufix® app. With my question, I would like to record how often you have carried out one of the two tests in the past three months. Did you use one of the two test methods?” The following question, if applicable, was, “How many days usually elapsed after a test with the method you chose until the next time you used this test method?”.

MacuFix® test procedure

The interactive test shows four square fields on a PC screen with a grid pattern of horizontal and vertical lines. All four fields have lines that are partially distorted (wavy). However, one of the four squares differs from the remaining three squares by more strongly distorted lines (Fig. 1). The difference between this grid pattern and the remaining three can be small or noticeable. The patients are required to select the more distorted field when viewing with one eye wearing appropriate near correction. The selection can be made by pointing, verbally with help, or by tapping on the touch screen. Each test includes a maximum of 20 attempts per eye to reach a result, and an algorithm determines the smallest distortion difference that was correctly chosen in at least 80% of attempts. This is given as a class, where each class stands for the smallest distortion difference. The procedure takes about two minutes per eye, and the encrypted results can be sent to the consultant ophthalmologist via email. The test is repeated for the other eye as well, and all the associated data remain on the device during the MacuFix® test. The test complies with the regulations of the German data protection regulations (DSGVO).

Fig. 1
figure 1

MacuFix used on a smartphone

Full size image
Fig. 2
figure 2

SD-OCT of the left eye of Subject No. 45 with medium drusen and no pigmentary abnormalities and an example of the distortion difference 1: The upper left quadrant contains lines that are more distorted than those in the remaining three quadrants

Full size image
Fig. 3
figure 3

SD-OCT Subject No. 14 left eye and Macufix distortion difference 2

Full size image
Fig. 4
figure 4

SD-OCT Subject No. 10 left eye and MacuFix distortion difference 4

Full size image
Fig. 5
figure 5

SD-OCT Subject No. 8 left eye and MacuFix distortion difference 5

Full size image

System usability scale (SUS) questionnaire and test frequency survey

The SUS questionnaire was developed to determine how users perceive the ease of use of the software [10]. It consists of ten statements based on a Likert scale, each with five possible answers ranging from complete rejection to complete agreement. The SUS questionnaire contains five positive and five negative statements about the usability of the system to be evaluated. Thus, data are collected, which can be quantitatively assessed and interpreted, and the result is a percentage usability value of the application. The SUS questionnaire contains the following ten statements:

  • I can well imagine using the test regularly.

  • I find the test unnecessarily complicated.

  • I find the test easy to use.

  • I think I would need technical support to use the test.

  • I find that the various functions of the test are easy to use.

  • I think that there are too many inconsistencies in the test.

  • I can imagine that most people learn to master the test quickly.

  • I find the operation very complicated.

  • I have felt very confident in using the test.

  • I had to learn a lot of things before I could work with the test.

For each statement, the participant gives his agreement or disagreement in the form of a scale ranging from 0 (strong disagreement) to 4 (strong agreement). The results of the SUS questionnaire are used to calculate a numerical value (the SUS score). The categories in the SUS questionnaire are coded with values from 0 to 4. The coding depends on the formulation: The numbers obtained in the ten questions are added together—the sum is between 0 and 40—and then multiplied by 2.5. As part of the study investigation, the study participants were about the time elapsed between two Amsler grid tests. The response spectrum could range from “1” for daily performance to “90” for a maximum of once per day for three months. Three months after the study, the study participants were called and asked how many days elapsed between the performances of two self-tests.

Statistical analysis

Since the study had a primarily exploratory character, no priory sample size was carried out. The study had two groups of patients, the group of patients who continued to use the Amsler test after the study day (group “Stay”) and the group of patients who used the application MacuFix® after the study day (group “Switch”). For the prospective, controlled study on adherence using a self-test, the null hypothesis was that there is no difference in the mean values of the two populations with respect to the frequency of use of the Amsler grid and the MacuFix® software (an interindividual comparison using t-test for unrelated samples). The alternative hypothesis was that there is a difference in the mean values of the two populations.

The null hypothesis for the intraindividual comparison was the frequency of use changed in the group of patients who stayed with the Amsler grid in the same way as those who switched to the MacuFix® test (an intraindividual comparison using the t-test for paired samples). The alternative hypothesis was that the change in test frequency was different in the group of patients who stayed with the Amsler grid compared to those who switched to the MacuFix® test.

In addition, the t-test for unrelated samples was used to compare whether the “Stay” group differed from the “Switch” group in the frequency of self-testing prior to the study day in order to rule out a selection bias. The statistical method used for this intraindividual comparison was the two-sided t-test for paired samples. All statistical analyses were performed with the statistical software “R” (version 3.6.1., R Foundation, R Core Team), a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria (http://www.R-project.org).

Results

Composition of the investigated collective

The initial study cohort included 46 patients. During the course of the study, one patient withdrew from the study, and the remaining 45 patients (18 females and 27 males) completed the SUS questionnaire after the MacuFix® test had been performed and responded, referring to the test frequency of the tests. Among the cohort, 35 study participants decided to use the MacuFix® test as a home test after the study; three of them used a PC, one patient used an iPad, and 31 used a smartphone. (Fig. 1) All subjects agreed to a telephone survey to be conducted three months later on the frequency of using a home test and were reached within the appropriate time period.

The mean age was 68 ± 9.7 years (SD). The mean visual acuity was 20/30 (Snellen fraction) or 0.6 ± 0.25 (decimal), respectively. Among the cohort, 18 eyes showed no abnormal macular findings; there was an early macular degeneration with small to medium-sized drusen but no changes in the retinal pigment epithelium in 22 eyes. Sixteen eyes showed intermediate  AMD with large drusen or at least medium-sized drusen associated with pigment epithelial changes. Fourteen eyes suffered from AMD (four geographic atrophy and ten neovascular AMD). In six cases, epiretinal gliosis or vitreomacular traction was present, one eye showed a macular hole, and one patient had diabetic macular edema (DME) in both eyes. In one eye, macular edema was due to retinal vein occlusion, in two eyes it was due to uveitis, and two eyes showed no edema after treatment for uveitic edema. Among the cohort, four eyes had developed Irvine–Gass Syndrome between 4 and 12 weeks (average 7.5 weeks) after cataract surgery, while two eyes had central serous chorioretinopathy. In the cohort, out of the 90 eyes of 45 study participants, two eyes could not be measured with Macufix due to central scotoma caused by geographic atrophy.

Sensitivity and specificity of macufix to identify metamorphopsia

When examined with the Amsler Grid, 42 eyes perceived metamorphopsia, 46 did not see metamorphopsia. Referring to the Amsler Grid as a gold standard, MacuFix® measurements were correct positive in 38, false positive in 4, correct negative in 43, and false negative in 3 cases. This led to a sensitivity of the Application MacuFix® of 92.7% and a specificity of 91.5%.

SUS score

Average values of the individual questions in the SUS questionnaire are listed in Table 1. The application received a higher rating in questions associated with learning the use of the application. The average score of the SUS questionnaire in this study was 76.7 (SD =  ± 15.5), which corresponds to the score “excellent,” as per categorization by Bangor et al. [9]

Table 1 Average values of individual questions in the SUS questionnaire
Full size table

Choice of test procedure

Of 38 persons who used the Amsler test before the study day, eight persons also carried out a self-test after the study day using Amsler grids (group “Stay”). The remaining 30 patients decided to use the MacuFix test instead of the Amsler test (group “Switch”). Seven of the 45 patients did not use any self-test to check for metamorphopsia in the baseline period, and two of them continued not to use a self-test in the comparison period, while the remaining five patients used the MacuFix® application. None of these patients used the Amsler grid after the study day. The average interval between two Amsler tests was 22.5 days before the study day (SD =  ± 14.1; confidence interval (CI) = 18.1; 27.0). The average interval between two Amsler tests after the study day was 19.4 days (SD =  ± 10.8; CI = 11.9; 26.9). The average interval between performing two MacuFix tests after the study day was 5.8 (SD =  ± 6.4; CI = 3.7; 7.9).

Comparison between stay and switch groups

In the “Stay” group, the interval between two Amsler tests averaged 26 days in the baseline period and 19 days in the comparison period. In the group “Stay,” the critical value is 2.365 with seven degrees of freedom and a probability of error (α) = 0.05. Since the test statistic of 1.697 is not higher than the critical value, the test frequency with the Amsler test in the baseline period did not differ significantly from the comparison period (t-test for paired samples: (|− 1.697|< 2.365; p > 0.05; n = 8)). In the “switch” group, the test frequency increased statistically significantly after switching from the Amsler grid to MacuFix. The time interval between two Amsler tests averaged 31 days in the baseline period. In the comparison period, the average interval between two MacuFix tests was six days. In the "Switch" group, the critical value is 2.032 at 35 degrees of freedom and α = 0.05. Since the test statistic of 6.135 is higher than the critical value, the difference is significant (t-test for paired samples: (|− 6.135|> 2.032; p < 0.001, n = 36)). In the comparison period, the average time interval between two tests in the Amsler group was 19 days, whereas in the MacuFix® group, the next test was performed after an average of six days. The average time to the next test was 13 days shorter in the MacuFix® group (95% CI: [3.82; 23.37]), t (7.98) = 3.21). This difference was statistically significant (p < 0.05). The Welch test was chosen because of the different sample sizes.

In order to determine whether the group of patients who used the Amsler test throughout the study (group “Stay”) differed in their general self-behavior from those who switched to the Application MacuFix® (group “Switch”), the test frequency data were compared before the study day. In the baseline period from January to March 2020, the average time interval between two Amsler tests in the “Stay” group was 26 days. In the “Switch” group, the next test was performed after an average of 31 days. The time to the next test showed no statistically significant difference when using the Welch test in both groups (95%-KI: [− 21.60; 10.66]), t (15.91) = − 0.72; p > 0.05. In the period before the study day, subjects who used the Amsler test continuously and those who switched to the Application MacuFix® later showed no difference in the test behavior. Six of the seven subjects who did not use a home test before the study were men. Two of these men continued not to use any home tests in the comparison period, four of them used the MacuFix® test. Thus, 60% of the men and one woman decided to use the MacuFix® test after the study while they had not applied any home test previously. The group of subjects who used the Amsler test before the study consisted of 18 females and 20 males. Eight female subjects from this group continued to perform the Amsler test. The remaining ten females switched to the MacuFix® test after the study, as did all 20 males in this group. Thus, in the group that had already used the Amsler test before the study, 55% of the women and 100% of the men decided to use the MacuFix® test instead of the Amsler tests for self-monitoring in the future. Prior to the study, seven of 45 patients (15.5%) did not use any home test to evaluate their macular function. After the study, five of these patients used the Application MacuFix®, whereas two of 45 patients (4.4%) continued not to use any home test. The odds ratio for persons previously not using a home test to imply a home test in their routine was 4.5 (OR = 7/38: 2/43 = 4) due to MacuFix®.

Discussion

This study evaluates the comparative data of 45 patients using the application MacuFix® and the standard functional test, Amsler Grid. As regards sensitivity and specificity, comparability of MacuFix® and the Amsler Grid is to some extent limited, because MacuFix® tests the central 4° of the visual field, whereas the Amsler Grid examines 10° when used at the intended distance.

According to the questionnaire, the study participants found the test easy to learn and uncomplicated, which may also be due to the interactive interface of the application. When using the SUS questionnaire, the application received the worst score on question no. 6, “I think there are too many inconsistencies in the test.” In order to reduce barriers to implementing the intervention, the help function for using the application has been revised in the meantime with the cooperation of patients, educators, and linguists, and the display of the test results has been improved. An improvement in health care through self-monitoring tools may possibly improve diagnostic and subsequently therapeutic adherence and persistence, thus leading to better maintenance of the patient's vision. Moreover, these tools using computer-based testing are especially important in low-resource settings, for less mobile patients such as those in senior care homes or when reduced contacts are desirable, as has been highlighted by the ongoing COVID-19 pandemic [11,12,13,14].

Amsler grid test is popular as it is inexpensive and can be easily explained to the patients for assessment of metamorphopsia. The unmet needs in-home monitoring of macular function is derived from the limitations of the Amsler grid, such as the non-interactive nature of the test, the missing fixation control during the test, the need for reasonable reading vision to discern the lines, a low sensitivity due to a suprathreshold stimulus, its poor performance due to the “crowding effect,” and limited awareness of visual field defects until the scotoma is significantly large in size due to “filling-in phenomenon” [15, 16]. Some of the ophthalmic home-based tests developed so far do not satisfy the needs of patients, among others due to complicated handling, high purchase price, or regulated access. These patients either cannot afford, do not have access to the devices for testing due to regulatory hurdles, or do not want or are unable to use new technologies. The Age-Related Eye Disease Study 2–HOme Monitoring of the Eye Study (AREDS2-HOME) revealed that 20% of patients who were offered the hyperacuity-based ForeseeHome monitoring device (Notal Vision Ltd, Tel Aviv, Israel) were unable to use it successfully due to visual field defects or problems with its application [17].

Diagnostic and therapeutic adherence is crucial for treatment persistence. Ehlken et al. observed that 44% of their patients did not reveal sufficient adherence in the first year of treatment [18]. The AURA study highlighted the role that regular monitoring plays in guiding neovascular AMD (nAMD) therapy [19]. The OCEAN study showed that a timely start of therapy leads to an improved outcome, but unfortunately, only 60% of patients continued the therapy after two years [20]. The POLARIS study found that the adherence of patients with DME is lower compared to AMD patients [21]. The ANDROMEDA study investigates the factors that influence adherence in patients with AMD [22].

Delayed diagnosis or detection of progression of metamorphopsia accounts for poor visual prognosis, and this opens up an area of application for home monitoring and telemedicine. Pinnacle clinical trials evaluating anti-vascular endothelial growth factor (VEGF) therapies in management protocols for nAMD have demonstrated significant visual acuity gains, yet these same benefits are not always reflected in real-world patient analyses. In a systematic review, Carrasco et al. described real-world outcomes in the treatment of nAMD and found that the amount of the visual acuity gain as seen in the studies was superior to the visual acuity gain seen in the real-world settings [23].

Angermann et al. retrospectively analyzed data of 1264 patients with diabetic retinopathy or nAMD receiving treatment with anti-VEGF between 2015 and 2018 [24]. Multivariate regression analysis showed that advanced age, lack of mobility, and need for assisted transport, poor final visual acuity despite treatment, and decrease in vision during the observational period were independent risk factors for terminating the treatment. The authors concluded that taking the risk of disease progression into account, strategies for better compliance and adherence to therapy should be considered to optimize patient care. The aspect of improving adherence is crucial as well for partner eyes bearing an annual risk of conversion to nAMD of 24% [25]. The authors of the post hoc analysis of the VIEW study requested close monitoring of these eyes at risk [26].

Macufix® has been shown to be a reliable tool for metamorphopsia detection in AMD patients [27]. In the present study, the usability of the “free” Macufix® Application was rated as excellent, and its availability increased the test frequency by a factor of 3. Therefore, we suggest that MacuFix® has the potential to serve as a cost-effective home monitoring solution that can connect patients or individuals at risk for macular disease with their eye care professionals and thus improve patient self-management. As early as 2008, Meyer et al. observed in a pilot study in 15 patients a good correlation between the standard examination including BCVA and OCT and a computer-based assessment of central vision, subjective categorization of their visual acuity development, and a digital marking of the metamorphopsia area [28]. In a larger group of 1444 patients, the paper-based ACTO self-test consisting of a reading chart gradated in decimal steps and six questions regarding the quality of the Amsler grid was accompanied by telephone contact with a coordinating center. The authors found a good correlation between the steps of the self-test and standard visual acuity at 4 m [29]. Against the background of a 48% drop-out rate in the ACTO study, attributed to decreased visual acuity, additional examinations by an ophthalmologist, or reluctance to continue telephone follow-up, a study with a longer follow-up would be desirable to determine the drop-out rate of MacuFix users and to investigate the influence of self-monitoring on functional outcomes.

The validity of the present study may be limited because of the range of diagnoses and the sometimes small number of cases for each diagnosis. The risk of statistical bias was reduced in our study by intraindividual comparison. A selection bias may be due to the fact that study participants were recruited from our own patient pool who may have been particularly motivated to perform and rate the test positively in the sense of social desirability. An observation bias may have arisen from the announcement and conduct of a telephone interview and thus have influenced adherence.

In the present study, randomization was deliberately avoided since it does not represent the everyday practice pattern; patients select a home monitoring test mainly based on their own decision and will accomplish it based on intrinsic motivation. The allocation in the context of randomization can falsify this picture and assign patients to a test, which they would not select of their own free will. In diseases that result in metamorphopsia, a variety of factors such as age, gender, cultural background, experience with electronic media, visual acuity, other diseases affecting visual acuity, or the visual field such as cataracts or glaucoma can influence the test adherence to an application. Their role as confounders or effect modifiers was not examined in this study. The risk of statistical bias was reduced in our study by intraindividual comparison.

Therapeutic agents and dosing strategies designed to overcome treatment burden by extending the time between dosing intervals continue to evolve and thus have the potential to improve quality of life and visual outcomes in patients with AMD, particularly nAMD, when integrated into clinical practice. To avoid discrepancies between clinical trials and real-world data due to the undertreatment of patients with nAMD or DME, even with these upcoming treatment algorithms, strategies that strengthen patient adherence will become more important. The high user satisfaction and increased testing frequency observed with the use of MacuFix® may lead to improved outcomes in the treatment of macular disease.