Introduction

Epiretinal membrane (ERM) is an age-related disease caused by the proliferation of fibrous cell membranes formed on the retina’s inner surface [1]–[3]. It can cause visual distortion, monocular double vision, and even vision loss [4, 5]. According to the different reasons for the formation of fibrotic cell membranes, epiretinal membrane can be divided into two types: idiopathic epiretinal membrane (iERM) and the secondary epiretinal membrane (sERM). IERMs occur when there is no associated ocular abnormality or only posterior vitreous detachment (PVD), PVD presents 78% [6] to 95% [7] of iERMs, suggesting its importance in the pathogenic process. Secondary ERM refers to ERMs thought to be due to co-existing or preceding ocular disease. The Blue Mountains Eye Study [8] and the Beaver Dam Eye Study [9] were two early extensive population studies on the prevalence of ERM. The Blue Mountains Eye Study reported that the prevalence of iERM was 7%, with the present rate of 31% in the opposite eye. The Beaver Dam Eye Study reported the prevalence of the iERM was 11.8%, and the present rate in the opposite eye was 19.5%. Therefore, accurate screening of iERM can allow the elderly to receive treatment in a timely manner, which will greatly avoid the development of the disease process and reduce the occurrence of vision loss.

Due to the difficulty in distinguishing ERM from fundus images, the main methods for diagnosing ERM are optical coherence tomography (OCT) and optical coherence tomography angiography (OCTA) [10]. ERM formation is a fibrotic process of myofibroblasts, and exposes the retina to anteroposterior and tangential forces [11]. Additionally, tangential forces drag the retina from its original position, and can be clearly identified when looking at the vascular displacement [12]. Anteroposterior forces cause vertical traction, resulting in leads to a thickening of the macular, which can easily be assessed by OCT [9].

Due to the vertical traction caused by the generation of ERM, the retinal morphology and blood supply of the retinal blood vessels will be affected. Several studies have been made to analyze structural changes in ERM. In the study of elderly French subjects, Delyfer et al. [13] found that the central foveal thickness (CFT) increased with ERM. Osada et al. studied the changes in retinal structures (layers) before and after microincision vitrectomy surgery [14]. They found that central retinal thickness (CRT) and inner retinal thickness (IRT) decreased significantly after surgery. At the same time, vessel density and foveal avascular zone (FAZ) did not show any significant changes. Chen et al. analyzed microvasculature features before and after vitrectomy with iERM [15].

The oxygen saturation (SO2) level is a functional parameter that shows the blood supply of the retinal blood vessels. In the study of ERM for retinal SO2, Nakano et al. found that postoperative patients’ arterial and venous blood SO2 were significantly higher than those before surgery [16]. Zhixin et al. [17] performed similar studies and found that the postoperative arterial blood SO2 was considerably higher than before the vitrectomy surgery. Lim et al. also performed analysis of retinal vessel SO2 before and after vitrectomy [18].

As shown in Fig. 1, in order to realize the accurate diagnosis of iERM by analyzing the SO2 and vascular morphological features of retinal vessels based on fundus images, we obtained dual-modal data from iERM patients and control subjects. We analyzed the SO2 level and morphological characteristics of blood vessels. Then, accurate classification was achieved using these functional and structural features. According to our knowledge, this study is the first analysis of iERM’s functional and structural features on fundus image, and it is the first proposed method using a dual-modal fundus camera to diagnose iERM. Compared with OCT and OCTA, a dual-modal fundus camera is cheaper and can be used widely, especially in rural areas, which is more conducive to extensive fundus screening of iERM.

Fig. 1
figure 1

a A dual-modal fundus camera was used to collect data. b Using CRU-Net to segment arterial, venous and blood vessels. c Functional and structural analysis based on vessel segmentation. d Feature fusion by machine learning and iERM classification

Full size image

Methods

Data acquisition and participants

Different from other fundus cameras, the dual-modal (structural and functional) fundus camera (OT-110 M, Hefei Orbis Biotech, LTD, China)Footnote 1 captures the 570-nm and 610-nm.

images while taking the color fundus image. The device applies two different wavelengths of light to simultaneously acquire two wavelength-dependent images of the fundus. The content ratio of oxyhemoglobin and deoxyhemoglobin in the blood reflects the degree of SO2 [19]. These two proteins are insensitive to 570-nm light (isoabsorption), and sensitive to 610-nm light (unequal absorption), so the 570- and 610-nm images can reflect the SO2 situation. Color fundus image was used to evaluate the vascular morphology, and the 570-nm and 610-nm images were used to evaluate the blood vessel SO2 level. Figure 2a shows a dual-modal fundus camera, and the acquired images. The resolution of the color fundus image is \(2304\times 1728,\) the resolution of the 570-nm image and 610-nm image is \(1936\times 1456\), the resolution of OT-110 M is \(4.54\mu m\), and the field of view is 45°.

Fig. 2
figure 2

Dual-modal fundus camera and the acquired images. a A dual-modal fundus camera. (b–d) The obtained color fundus image, the corresponding 570-nm image, and 610-nm image. (e) Pseudo-color image of blood oxygen saturation (SO2)

Full size image

In this study, the exclusion criteria were as follows: keratoconus, high myopia (more severe than -6 diopters or ocular axial length > 26.5 mm), prior intraocular surgery or any co-existing ocular disease. Under this criterion, 72 image sets (color fundus image, 570-nm image, and the 610-nm image) were obtained from 51 Chinese ethnic subjects. Among them, 23 people were patients diagnosed with iERM by two physicians with clinical experience from Wenzhou Medical University, and 28 were healthy control subjects. The average age of the iERM group was \(59.7\pm 6.0\), with 8 males and 15 females. The average age of the control group was \(57.6\pm 6.1\), with 9 males and 19 females. There was no significant difference in age between iERM and the normal control group, and neither had other systemic diseases. The demographic data of iERM and the normal control group are shown in Table1.

Table 1 Demographic data of idiopathic epiretinal membrane (iERM) and normal control group. Number of subjects, age, and male-to-female ratio are matched between these two groups
Full size table

Preprocessing

Since the images acquired by the dual-modal camera are not completely registered in position, in order to accurately evaluate blood vessel SO2, we used the method proposed by Dou et al. [20] and registered 570-nm and 610-nm images with the reference of color fundus image.

The extraction of blood vessels is the prerequisite for our blood vessel SO2 and blood vessel morphology analysis. Some recent works have effectively segmented blood vessels [21, 22]. However, considering the particularity of data, we used the CRU-Net proposed by Zhang et al. [23] for processing dual-modal data, and segmented the arteriovenous vessels based on the registered images.

Owing to iERM mainly affecting the macular area, our study focused on the macular-centered region of interest (MROI). MROI is a semicircle with the center of the macular as the center and the distance from the macular center to the outer edge of the optical disk (OD) as the radius. To obtain the position of OD, an optimized U-Net network was applied to segment OD from color fundus image [24]. Figure 3a shows the registered image with segmentation results of OD and arteriovenous blood vessels. We further located the macular center with an algorithm proposed in [25]. Many studies have proved machine learning methods work well for medical image problems, and the methods we used above [23]–[25] fit well on our data and greatly reduced the workload. Our analysis of retinal blood vessels of iERM patients and control subjects was performed on MROI, as shown in Fig. 3b. Moreover, the image quality was improved using the method proposed by Luo et al. [26] to help vessel segmentation. All the deep learning methods above were used out-of-the-box, for our data types are consistent with theirs.

Fig. 3
figure 3

a Registered image with optic disk (OD) and arteriovenous blood vessel segmentation results. b Selected macular-centered region of interest (MROI) used to calculate blood vessel SO2 and vascular morphology

Full size image

Retinal Oximetry

Due to the formation of iERM will affect the blood supply of the fundus, and we analyze the blood vessel SO2. In order to evaluate SO2, optical density is obtained and calculated as:

$$\mathrm{Optical} \mathrm{Density}= \mathrm{log}\left(\frac{{I}_{out}}{{I}_{in}}\right)$$
(1)

where \({I}_{out}\) and \({I}_{\mathrm{in}}\) represent the internal and external intensity of the blood vessel, respectively [19]. Then the Hammer method was used to calculate the corrected optical density ratio (ODR) [27]. Using the linear relationship between blood SO2 and ODR in the study of Harris et al. [28], we evaluated the blood SO2 level based on the method provided by Chen et al. [29] with images acquired by our dual-modal fundus camera. Details can be found in [20].

Vascular morphology analysis

In order to study the correlation between vascular morphological features and the appearance of iERM, we introduced the following indicators to assess changes in vascular morphology.

The fractal dimension (FD) summarizes the whole branching pattern of the retinal vascular tree [30]. Larger FD values indicate a more complex branching pattern. The box-counting method proposed by Mainster & Martin [31] and Stosic & Stosic [32] was applied to evaluate the FD of fundus vessels.

Tortuosity is used to measure the degree of curvature of blood vessels. Lager tortuosity denotes blood vessels are more curved. There are different definitions of tortuosity [33, 34]. We applied arc length-normalized total squared curvature as the measurement of tortuosity, as proposed by Hart et al. [33], and further improved by Turior et al.[35]. The blood vessels in MROI are used to calculate the tortuosity.

Blood vessel density was calculated as the ratio of vessel area to the MROI area without OD.

Equivalent vessel diameter is used to measure the thickness of blood vessels. A larger diameter indicates a larger retinal vascular caliber. To obtain vessel diameter, we used the calculation of the retinal vascular caliber of the improved Knudtson–Parr–Hubbard formula [36].

The branching angle (BA) is the angle between two child blood vessels at the bifurcation. To evaluate BA, we applied the definition in the study of Baker et al. [37] and calculated the branching angles at the branching points. Branching coefficient (BC) was calculated using the width of the three vessels at the branching points. The branching coefficient was calculated using the definition in the study of Patton et al. [38]

Asymmetry factor (AF) was used to describe the asymmetry at the branching points of the blood vessels. The calculation of the AF is given by Eq. 2 for the case of \({d}_{1}\gg {d}_{2}\):

$$\mathrm{AF}=\frac{{d}_{1}^{2}}{{d}_{2}^{2}}$$
(2)

where \({d}_{1}\) and \({d}_{2}\) are the width of the two daughter vessels. The optimality ratio (OR) is also to measure the relationship between parent and daughter vessels at vascular bifurcations, and is defined as:

$$\mathrm{OR}=\sqrt[1/3]{\frac{{d}_{1}^{3}+{d}_{2}^{3}}{2\times {d}_{0}^{3}}}$$
(3)

where \({d}_{1}\) and \({d}_{2}\) are the width of the two daughter vessels and \({d}_{0}\) is the width of the parent vessel.

Statistical analysis

To compare blood vessel SO2 level and vascular morphologies in the normal control group and iERM group, we first applied the Shapiro–Wilk test to test the normality, and then used Student’s t-test and Kolmogorov–Smirnov test to test whether there is a significant difference between two groups [39]. All data were analyzed using R version 4.1.2 (Foundation for Statistical Computing, Vienna, Austria). All tests were two-sided, p < 0.1 was considered marginally statically significant, and p < 0.05 was considered statically significant (*p < 0.1; **p < 0.05; ***p < 0.01).

Classification

Random forest (RF) [40] and support vector machine (SVM) [41] were applied as the classifiers to differentiate the iERM patients and healthy subjects.

The RF classifier is an ensemble classifier that uses randomly selected training samples and variables to create multiple decision trees. The RF classifier uses a set of classification and regression trees to make predictions. The trees are produced by drawing a subset of training samples through replacement [41].

The goal of SVM is to find a hyperplane to separate different classes with the greatest margin [42]. For each category, the data closest to the hyperplane is defined as the support vector [43].

To demonstrate the synergistic effect of combining the blood vessel SO2 level with the vascular morphology features on the performance of the classifier, we designed ablation experiments. First, separate blood vessel SO2 and vascular morphological features were taken as input and sent to the classifiers. Then we fed the combination of blood vessel SO2 and vascular morphological features into the machine learning classifiers. When using the combined features as input, we selected the vascular morphological features that had the highest classification accuracy when input alone.

In order to verify the robustness of the experiment, the threefold cross-validation was applied to assess the classifier’s performance on the image level. For RF classification, the number of trees was set to 500, and the number of features was set to “sqrt” for searching the best split. For building the SVM model, we normalized the input variable and set the kernel function to “rbf.”

Evaluation

Accuracy (Acc), Precision (Pre), Recall (Rec), and F1-score (F1) were used to evaluate the effect of classification. They are defined as follows:

$$\mathrm{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN}$$
(4)
$$\mathrm{Precision}=\frac{TP}{TP+FP}$$
(5)
$$\mathrm{Recall}=\frac{TP}{TP+FN}$$
(6)
$$F1-score=2\times \frac{\mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}$$
(7)

where TP, TN, FP, and FN, respectively, represent true positive, true negative, false positive, and false negative.

Results

SO2 in iERM and control groups

We calculated the arterial and venous SO2 in the MROI area of the control group and the iERM group. In the iERM group, the arterial SO2, veinous SO2, and arteriovenous difference (AVD) of SO2 are \(104.6\pm 8.2\%\), \(71.1\pm 8.7\%\), and \(33.5\pm 5.7\%\), respectively, while the numbers in the control groups are \(95.1\pm 4.8\%\), \(64.3\pm 5.7\%\), and \(30.9\pm 5.2\%\). Figure 4 shows the boxplot of arterial and venous SO2 and the difference between them. In the MROI area, the arterial SO2 and venous SO2 in the iERM group are significantly higher than those in the control group, while the AVD of SO2 has no significant difference between these two groups.

Fig. 4
figure 4

Mean values of arterial SO2 and venous SO2 in the idiopathic epiretinal membrane (iERM) group are significantly higher than in the control group. No significant difference is found in arteriovenous difference of SO2 (***: P < 0.01)

Full size image

Comparison of vascular morphology

Table 2 and Fig. 5 show the morphological features of blood vessels in the iERM group and the control group from a quantitative and qualitative perspective, respectively.

Table 2 Quantitative comparison of arteriovenous vascular morphological features measured in iERM eyes and control eyes. Significant differences of arteriovenous vascular morphological features of iERM and control groups are found. Bold fonts in the table indicate that the values have marginally significant differences (p < 0.1) or significant differences (p < 0.05)
Full size table
Fig. 5
figure 5

Comparison of vascular morphological features among iERM and control groups. (a-d) Boxplot of fractal dimension (FD), tortuosity, density, and equivalent diameter (*: P < 0.1, **: P < 0.05, ***: P < 0.01)

Full size image

Figure 5 shows the boxplot of FD, tortuosity, density, and equivalent diameter. Significant differences in these vascular morphological features exist in arteries and veins between the iERM group and the normal control group.

As shown in Table 2, for the arteries, the iERM subjects has an FD of 1.11, which is significantly lower than that of the control subjects’ FD of 1.13 (p < 0.05). Vessel density in iERM patients is 4.5, which is significantly lower than 4.9 in controls (p < 0.01). The AF of iERM patients is 2.2, which is marginally significantly lower than the controls’ AF of 2.6 (p < 0.1). For veins, the tortuosity of the iERM patients is \(2.1\times {10}^{-4}\), which is significantly greater than the controls’ of \(1.6\times {10}^{-4}\) (p < 0.01). The iERM patients has an equivalent diameter of 13.1, which is marginally significantly larger than the control group’s diameter of 12.8 (p < 0.1).

There is no significant difference between the BA, BC, and OR in the arteries and veins of iERM patients and control subjects.

Performance of classification

We used blood vessel SO2 and vascular morphological features extracted from fundus images as input and applied RF and SVM as classifiers to classify iERM and control groups. Quantitative and qualitative comparisons have been made to analyze the classification performance of blood vessel SO2 and vascular morphological features alone and the combination of the two as the input.

The quantitative results are shown in Table 3, and the receiver operating characteristic (ROC) curves as our qualitative measurements are shown in Fig. 6.

Table 3 Classification results of random forest (RF) and support vector machine (SVM). Combining blood vessel morphology features and blood oxygen saturation (SO2), SVM and RF outperform the two features used alone
Full size table
Fig. 6
figure 6

Receiver operating characteristic (ROC) curve of threefold cross-validation. From top to bottom are the ROC results of random forest (RF) and support vector machine (SVM) classification, respectively. The three columns from left to right represent the features used to classify: SO2, vascular morphological features, and all features. RF and SVM have the best performance by using SO2 and vascular morphological features. In the experiment of SVM using morphological features, the classification results of twofolds were the same, so the green line overlapped with the blue one

Full size image

For the RF classifier, when only blood vessel SO2 was used as an input feature, the classification accuracy was \(72.30\pm 3.08\mathrm{\%}\). The best classification accuracy of using structural features appeared when choosing the combination of equivalent vessel diameter, tortuosity, fractal dimension, density, and asymmetry factor as the vascular morphological features input (\(78.62\pm 5.91\mathrm{\%}\)). Based on this, we combined blood vessel SO2 and the selected vascular morphologies as the input features for classification, and the accuracy reached \(82.85\pm 7.11\%\), which was better than the two separate inputs.

For the SVM classifier, similar results showed up. When only blood vessel SO2 was used as a feature for classification, the accuracy was \(72.13\pm 2.78\mathrm{\%}\). We also selected several morphological features (density, tortuosity, fractal dimension, and asymmetry factor) that had the best classification accuracy when only vascular morphological features as inputs. The accuracy was \(80.13\pm 6.81\mathrm{\%}\). Afterward, we combined blood vessel SO2 and the selected vascular morphologies as the input features for SVM classifier, and the accuracy reached \(82.91\pm 2.78\mathrm{\%}\), which was also better than the two separate inputs.

Discussion

In our work, we have studied the functional and structural changes caused by the formation of iERM in fundus images. Significant differences in vessel blood SO2 and vascular morphologies were found in the iERM group and the normal control group. Finally, we applied RF and SVM as machine learning methods to classify iERM patients and normal subjects by combining vessel blood SO2 and vascular morphologies. We achieve the accurate diagnosis of iERM with an accuracy of over 82% on dual-modal fundus images.

Reports on using blood vessel SO2 to analyze iERM patients and normal subjects are limited. Some studies focused on the changes in blood vessel SO2 before and after vitrectomy surgery. Li et al. [17] studied 24 eyes of 24 consecutive patients who underwent vitrectomy. The preoperative retinal arterial and venous SO2 were \(89.9\pm 11.3\%\) and \(56.1\pm 11.0\%\). In the study of Lim et al. [18], it was also found that the SO2 of the arteries and veins increased after vitrectomy. Correspondingly, our arterial and venous SO2 were \(104.6\pm 8.2\%\) and \(71.7\pm 8.7\%\). In their research using Oxymap T1, the selected region of interest (ROI) was an optic disk-centered, 1.5-disk to 3-disk diameter area, and 1 disk diameter away from the disk margin. Since iERM is a disease that occurs in the macular, we selected a semicircle with the center of the macular as the center and the distance from the center of the macular to the outer edge of the OD as the radius (MROI). The proposed MROI is more in line with the pathological characteristics of iERM. Possibly due to the selection of different ROIs, the arterial and venous SO2 in our study is higher than in the above-mentioned studies.

By analyzing in the MROI, we found that the mean values of arterial and venous SO2 of iERM patients were significantly higher than that of the control subjects. Retinal arterial SO2 reflects the ability of hemoglobin to bind oxygen. Compared with control subjects, the arterial SO2 of iERM patients was higher, indicating that the retinal artery tissue of iERM patients has a stronger ability to acquire oxygen. Venous SO2 reflects the relationship between oxygen supply and oxygen demand. Higher venous SO2 in iERM patients indicates sufficient local oxygen supply of the retina. Compared with control subjects, there was no significant difference in AVD, indicating that there is no significant difference in the oxygen consumption of the affected tissues.

At the same time, we found significant differences in several morphological descriptors between iERM patients and the healthy controls. The arterial FD of iERM patients is significantly lower than that of control subjects. FD represents the complexity of the vascular structures. This difference could be due to changes in the development of the vascular system caused by iERM, such as the smallest friction between the blood flow and the vascular wall, the best heart rate for proper blood supply, the shortest transportation distance and other factors [44]. Unlike the arteries, the most significant differences in the veins between the patients and the healthy controls are the tortuosity and the vessel width. In the patients, the veins are wider and curvy. This difference in tortuosity is consistent with the findings of Chen et al. [15]. We found iERM patients’ venous vessels are wider than that of the control subjects. Nakano et al. [16] reported vessel diameter increased at 6 months after cataract surgery, which is consistent with our study. BA and BC represent branching angle and branching coefficient at the branch of the blood vessel, respectively. In our study, there was no significant difference in arterial and venous on BA and BC, indicating iERM does not affect the branches of blood vessels. AF represents the asymmetry factor of two daughter vessels at a branching point. In our study, arterial AF in iERM groups was smaller than that in the control group, while no difference was found in venous vessels. This indicates that iERM will make the difference in vessel width between the two daughter vessels of the artery smaller.

Thanks to the analysis of blood vessel SO2 and vascular morphological features, we have achieved accurate diagnosis of iERM and found the synergy between the two features. Regardless of whether RF or SVM was used as a classifier, the classification accuracy was over 82%, while the best classification accuracy using blood vessel SO2 or vascular morphological features was \(80.13\pm 6.81\%\). To the best of our knowledge, this is the first study to classify iERM using only the blood vessel features in dual-modal fundus images. Moreover, we found the complementarity between the morphological features of blood vessels. For example, the difference in AF between the iERM group and the normal control group is not statically significant (p < 0.1), but the addition of AF will improve the accuracy of classification. Others morphological features like BA and BC showed no significant differences between the two groups, so the classification accuracy was not good enough when input containing these features. For the iERM classification using machine learning methods, there are several other studies [45, 46]. However, their classification methods are based on OCT data or 3D-OCT data, which is not comparable to ours.

Despite OCT/OCTA have been studied extensively, they are not widespread in the assessment of retinal diseases [47], due to the cost. In addition, the recent COVID-19 pandemic has had a significant negative impact on ophthalmologists’ ability to conduct in-clinic exams [48], demonstrated the limitations of the current state of teleophthalmology [49], and highlighted the need for developing effective, low cost, and reliable alternatives for both in-home and in-clinic measurements. OCT/OCTA imaging requires clinic visits, preventing truly ubiquitous use as an in-home alternative. The ERM screening method based on the retinal image proposed in this paper is novel, high-efficiency, convenient, and low cost, which is helpful to prepare for and adapt to a post-COVID-19 world.

This study has some limitations. First of all, the current study had a relatively small sample size of iERM data, and we are not able to model the real-world data distribution well in our classification experiments. So far, there is no staging data of iERM patients; our analysis of iERM is not thorough enough. Next, we will include more detailed staging information, and conduct more in-depth research on iERM. Secondly, the retinal oximetry method depends on the analysis of the reflected light, fundus pigmentation, and diameter of retinal vessels would influence the measurement of retinal SO2. Although we have screened based on image quality, it might nevertheless affect the measurement of SO2. Finally, there is no measurement of internal factors related to technicians. Biases by the technician might still be possible despite thorough training and standardized routine in his workdays.

Conclusions

Benefitting from the dual-modal fundus camera capable of capturing functional and structural features, we can fully analyze these blood vessel SO2 and vascular morphological features of iERM patients. As iERM mainly affects the macular area, we performed the analysis in a macular-centered region of interest (MROI) and found significant differences between iERM and the control group, which has clinical significance. Machine learning methods address small data problems and can conduct efficient information mining in fundus images. By analyzing the SO2 and the morphological features of blood vessels, we made full use of the complementarity and synergy between them, and achieved excellent diagnosis performance of iERM based only on fundus images using machine learning methods (accuracy over 82%).

The acquisition of dual-modal fundus images is low cost and convenient. We hope that it will be widely adopted in iERM screening in future. Moreover, there are many other diseases that affect the blood supply and structure of the fundus. The proposed dual-modal fundus camera-based method may also be extended to the diagnosis of these diseases (such as Alzheimer’s disease and stroke) in future [37, 50].