FormalPara Key Summary Points

Why carry out this study?

Ultrasound biomicroscopy (UBM) and anterior segment optical coherent tomography (ASOCT) are the most widely used instruments to objectively visualize and evaluate anterior segment parameters. Both demonstrate excellent repeatability and reproducibility, while each has its own specific limitations.

A new technology that combines the advantages of both devices is necessary; such a tool might have the potential to become the gold standard method for measuring anterior segment parameters.

We have previously shown that a generative adversarial network (GAN) framework of synthetic OCT images provides good quality images for clinical evaluation and can also be used for developing deep learning algorithms. Recently, the cycle-consistent generative adversarial network framework (CycleGAN) was introduced to generate images from different imaging modalities.

This aim of this study was to investigate the feasibility of generating synthesized UBM from swept-source anterior segment optical coherent tomography (SS-ASOCT) using CycleGAN for iridociliary assessment.

What was learned from the study?

Our results showed that there was good to excellent correlation of anterior segment parameters measured from the synthetic images and those from real UBM images.

The CycleGAN-based deep learning technique provides a promising strategy to assess iridociliary using easy-to-use and non-contact methods.

Introduction

Precise ocular biometry is crucial for the diagnosis and treatment of ocular disorders. For anterior ocular biometric measurements, newer imaging instruments have been developed to objectively visualize and evaluate anterior segment parameters instead of traditional subjective techniques, such as slip-lamp examination and gonioscopy. Among these devices, ultrasound biomicroscopy (UBM) and anterior segment optical coherence tomography (ASOCT) are the most widely used imaging modalities. UBM allows high-resolution visualization of the anterior segment and angle structures at an ultrasonic frequency of 35–100 MHz, providing additional information on the posterior chamber not otherwise available through clinical examination [1]. ASOCT is a computerized imaging technology providing optical cross-sectional images of ocular structures, and when updated with the newer technology of swept-source OCT (SS-OCT), it provides a larger number of non-contact higher resolution images, more detailed information, and precise parameters of the angle, corneal, iris, and anterior chamber volumes; in addition, the fast scan rate effectively decreases motion artifact [2].

Research has demonstrated excellent repeatability and reproducibility of UBM and SS-ASOCT [3,4,5]. However, despite their high value as diagnostic tools, both techniques have limitations. UBM is a contact, non-invasive method that takes a relatively longer time, and capturing diagnostic images and evaluation requires great skill and experience of the operator. SS-ASOCT is based on the optical principle. Since the examining beam cannot penetrate through the iris, certain structures in the posterior chamber, such as ciliary processes, zonule fibers, and cysts or intraocular foreign body behind the iris, can not be captured and imaged. Should a specific new technological method be developed that could combine the advantages of both devices, namely, a device that could automatically capture the non-contact high-resolution images of the anterior segment and angle structures with posterior chamber information, as well as unique anterior chamber parameters under rapid scan, it might have the potential to become the gold standard method for measuring the anterior segment parameters.

The recent development of deep learning methods, especially domain adaption using generative adversarial networks (GANs) [6], has attracted much interest in the field of medical imaging analysis. Several studies have demonstrated the capability of GANs to generate synthetic computed tomography (CT) or magnetic resonance (MRI) images of prostate [7], liver [8], brain [9], and head and neck cancer [10]. We have previously shown that GANs of synthetic OCT images have good quality for clinical evaluation and can also be used for developing deep learning algorithms [11,12,13]. More recently, a cycle-consistent generative adversarial network framework (CycleGAN) was introduced to generate images from different imaging modalities. For example, several researchers have demonstrated a CycleGAN-based domain transfer between CT and MRI images [14], traditional retinal fundus photographs, and ultra-widefield images [15], among others. However, cross-modality image transfer between SS-ASOCT and UBM has not yet been reported. Inspired by this domain transfer, the aim of this study was to build a CycleGAN-based deep learning model for the domain transfer from SS-ASOCT to UBM. We also conducted experiments to demonstrate the effectiveness of the CycleGAN method by qualitatively and quantitatively measuring iridociliary parameters from synthetic UBM using this technique with an independent dataset.

Methods

Study Design

We retrospectively collected the development datasets from the PACS ((picture archiving and communication system) of the department of ophthalmology, Xinhua Hospital, Medicine School of Shanghai Jiaotong University. The raw development datasets consisted of 2314 SS-ASOCT images and 2417 UBM images from 163 and 612 patients, respectively, between 16 September 2020, and 10 December 2021 (Fig. 1a). All subjects had visited the glaucoma clinic to screen for primary angle-closure glaucoma, which is a major cause of blindness in Chinese population. We excluded patients with a prior history of trauma, intraocular tumor, intraocular surgery, and laser iridoplasty. Eyes with gross iris atrophy and uveitis were also excluded. Details of SS-ASOCT and UBM imaging have been described previously [16, 17]. In brief, SS-ASOCT (model CASIA2; Tomey, Nagoya, Japan) is a novel imaging modality which, compared to earlier generations of ASOCT, such as time-domain ASOCT (Visante OCT; Carl Zeiss Meditec, Dublin, CA, USA), has a faster scan speed (50,000 vs. 2000 A-scans/s), provides deeper imaging (11 vs. 8 mm), and has a higher resolution (2.4 vs. 10 µm). In the present study, SS-ASOCT (model CASIA2; Tomey Corp., Nagoya, Japan) images were obtained for all participants. A total of 128 two-dimensional cross-sectional SS-ASOCT images were acquired per scan. All UBM images were taken by UBM (model SW-3200L; Suoer Electronic Ltd., Tianjin, China). During the UBM examination, subjects lay supine under standardized dark conditions (illumination 60–70 lx). The probe was then placed perpendicular to the ocular surface, and images of all four quadrants were obtained. A single operator imaged all subjects. Only those images were included in the analysis which clearly visualized the scleral spur, drainage angle, ciliary body (for UBM images only), and a half chord of the iris.

Fig. 1
figure 1

Schematic of image preprocessing, CycleGAN model development, and evaluation of synthetic UBM images. ACA Anterior chamber angle, AS-OCT anterior segment optical coherence tomography CycleGAN cycle-consistent generative adversarial network, UBM ultrasound biomicroscopy

Full size image

To evaluate the GAN model, we also collected two independent testing datasets as the in-distribution and out-of-distribution testing datasets, respectively. The in-distribution testing dataset included SS-ASOCT images from the same hospital between 1 March 2021 and 4 May 2021. The out-of-distribution testing datasets involved time-domain ASOCT images acquired at a different hospitals from our previous study. All subjects underwent an ophthalmic examination, including best-corrected visual acuity, refraction, slit-lamp examination, and gonioscopic anterior chamber angle (ACA) evaluation by a fellowship-trained glaucoma specialist (Goldmann 2-mirror lens; Haag-Streit AG, Bern, Switzerland). The inclusion and exclusion criteria were identical for the training and independent testing datasets. In the two testing datasets, ASOCT and UBM were performed in terms of the nasal and temporal angle.

This study was approved by the Institutional Review Board of Xinhua Hospital, Medicine School of Shanghai Jiaotong University (identifier: XHEC-D-2021-114). The study was carried out in accordance with the ethical standards set by the Declaration of Helsinki of 1964 as amended in 2008. All SS-ASOCT and UBM images were anonymized and de-identified according to the Health Insurance Portability and Accountability Act Safe Harbor before analysis [18]. Informed consent was exempted by the IRB in the retrospectively collected development and validation datasets. In the prospectively collected testing dataset, informed consent for publication was obtained from all enrolled patients or from their guardians.

CycleGAN Architecture and Generation of Synthesized UBM Images

To generate the synthesized UBM images, we adopted a CycleGANs model to translate images from the SS-ASOCT domain to UBM using unpaired data (Fig. 1b). The details of the CycleGAN architecture have been previously described by Zhu et al. In brief, CycleGAN is a type of unsupervised machine learning technique used for mapping different image domains [19]. The whole neural network of CycleGAN consists of two generator-discriminator networks. Figure 1b shows the schematic of the CycleGAN model in the current study. The forward cycle learns a mapping of the first generator to translate SS-ASOCT images to synthetic UBM images and attempts to make synthetic UBM images that are as real as real UBM images to fool the discriminator. On the other hand, the backward cycle learns a mapping of the second generator to transform synthetic UBM images back into synthetic SS-ASOCT images, and to make synthetic SS-ASOCT images as real as the real SS-ASOCT images. This cycle consistency allows CycleGANs to capture the characteristics of two image domains and automatically learn how these characteristics should be translated to transfer the domains without any paired datasets [20].

We adopted the codes of CycleGANs from the Tensorflow tutorial webpage (https://www.tensorflow.org/tutorials/generative/CycleGANs). Since each raw SS-ASOCT image contains two ACA regions, we split each image into two ACA images. Both SS-ASOCT and UBM ACA images were then rescaled to 256 × 256 pixels, normalized between − 1 and + 1, and augmented by random flip, cropping, and contrast perturbations. For training, we used an Adam optimizer learning rate of 0.0002 and a batch size of 1, as described by the same webpage. The model was trained on the Kaggle platform (www.kaggle.com), a free cloud service for deep learning research. Kaggle cloud platform provides free access to NVIDIA TESLA P100 GPUs with 16 GB RAM.

Evaluation of CycleGANs Synthetic UBM Images for Iridociliary Assessment

We sought to determine whether the CycleGANs synthetic UBM images could be utilized to assess the iridociliary. First, a visual Turing test [21] was performed to evaluate the image quality of synthetic UBM images. Then, we applied the CycleGAN to the independent testing dataset, and 30 synthetic UBM images were generated accordingly. All images were displayed on a laptop screen with 256 × 256 pixels. Two glaucoma specialists (HFY and YQH with more than 5 and 10 years of glaucoma experience, respectively) from the same center manually assessed the image quality of both the synthetic and real UBM images. To avoid confirmation bias, the glaucoma specialists were not told that there a

were any synthetic images among the UBM images. We have devised an image quality grading scheme for evaluating the GAN synthetic ophthalmic images [11, 12] that includes: (1) visibility of the scleral spurs, which was defined as the point where there was a change in the curvature of the inner surface of the angle wall [22]; (2) presence of continuity in the anterior segment structures, including iridociliary [11]; and (3) the absence of motion artifacts [23]. After image quality evaluation, two glaucoma specialists were asked to classify every image in that dataset as being either real or fake.

To quantitively assess the synthetic UBM images in the in-distribution (SS-ASOCT) and out-of-distribution (time-domain ASOCT) datasets, imaging analysis in the testing dataset was further processed using q-customized software (Anterior Segment Analysis Program [ASAP] [24]) by a single experienced observer (YW) who was masked to the clinical data. ASAP works as a plug-in of public domain software (ImageJ version 1.38x; public domain software: http://imagej.nih.gov/ij) and based on traditional image processing algorithms. The algorithm then automatically calculated the anterior angle and iridociliary parameters. The following parameters were measured as described previously: (1) ciliary body thickness at the point of the scleral spur (CT0) and at the distance of 1000 um (CT1000) from the scleral spur [25]; (2) angle opening distance (AOD500) [26], which was calculated as the perpendicular distance measured from the trabecular meshwork at 500 um anterior to the scleral spur as described previously; (3) iris thickness (IT500), measured at 500 μm from the scleral spur; (4) trabecular-ciliary process angle (TCA), measured as the angle between the corneal endothelium and superior surface of the ciliary process; and (4) trabecularciliary process distance (TCPD), defined as the length of a line 500 μm from the scleral spur extending from the corneal endothelium, perpendicular through the posterior surface of the iris, to the ciliary process (Fig. 1c).

To further measure the performance of the CycleGAN model, we chose the Fréchet Inception Distance (FID) to evaluate the synthetic images. FID is a widely used metric for evaluating the distance between the distributions of synthetic data and real data by calculating the Wasserstein-2 distance in the feature space of an Inception-v3 network.

Statistical Analysis

The quality of the synthetic and real UBM images’ was graded by two glaucoma specialists, using Pearson’s χ2 test. To evaluate the agreement between the measurements of the devices, Bland–Altman analysis (mean difference and limits of agreement [LoA]) was performed. The measurement correlations were calculated using intra-class correlation coefficients (ICCs) and coefficients of variation (CoVs). An ICC of < 0.4 indicated poor reproducibility, ICC between 0.4 and 0.75 indicated fair to good reproducibility, and ICC > 0.75 indicated excellent reproducibility. All statistical tests were performed using the SPSS version 26.0 software (SPSS IBM Corp., Armonk, NY, USA).

Results

In this study, the CycleGAN model generated UBM images (pixel resolution of 256 × 256) using SS-ASOCT images. After image grading and preprocessing, the trained dataset included 4037 SS-ASOCT ACA and 2206 UBM ACA images. The processing time for one synthetic UBM image was approximately 1.1 s using a commercial laptop (Apple MacBook Air M1; Apple Inc., Cupertino, CA, USA). The examples of synthetic UBM images and corresponding SS-ASOCT images are shown in Fig. 2. Overall, our approach is capable of generating OCT images that are realistic. The results of UBM image quality grading by two glaucoma specialists are shown in Table 1. Both glaucoma specialists graded synthetic UBM images as having approximately the same proportion of good quality images as the real UBM images (all p > 0.05, Pearson Chi-square). Table 2 also summarizes the second part of the Turing test. It was notable that the accuracy for distinguishing between true and fake images was 56.7% and 60%, respectively, which is only slightly better than chance.

Fig. 2
figure 2

Examples of SS-ASOCT images (a, d), real UBM images (b, e), and synthetic UBM images (c, f). SS-ASOCT Swept-source anterior segment optical coherent tomography

Full size image
Table 1 Synthetic and real UBM images' quality grading by 2 glaucoma specialists
Full size table
Table 2 Synthetic vs. real UBM images distinguished by 2 glaucoma specialists
Full size table

In the in-distribution (SS-ASOCT) dataset, there was fair to excellent agreement between the measurements of real and synthetic UBM images of the anterior chamber and of iridociliary parameters (Table 3). ICC values were 0.74 (CT1000), 0.86 (CT0), 0.97 (AOD500), 0.48 (IT500), 0.81 (TCA) and 0.80 (TCPD). Figure 3a shows the Bland–Altman plot for the anterior chamber parameter (AOD500). The mean difference was − 0.06 mm at 95% LoA (− 0.11 to − 0.01 mm). The Bland–Altman plots for iridociliary parameters CT0, CT1000, TCA, and TCPD are shown in Fig. 3b, c, e, and f, respectively; the mean difference was − 0.10 mm (95% LoA − 0.34 to 0.12), − 0.01 mm (95% LoA − 0.12 to 0.09), − 5.1° (95% LoA − 21.91° to 11.71°), and 0.02 mm (95% LoA; − 0.22 to 0.25), respectively. Figure 3d shows the Bland–Altman plot for the iris parameter (IT500); the mean difference was − 0.07 mm at 95% LoA of − 0.20 to 0.07 mm (Fig. 3; Table 3). The CoVs were 7.7% (CT1000), 13.3% (CT0), 25.8% (AOD500), 16.5% (IT500), 8.4% (TCA), and 9.6% (TCPD).

Table 3 Comparison of anterior chamber angle and ciliary body measurements between synthetic and real ultrasound biomicroscopy images
Full size table
Fig. 3
figure 3

Evaluation of agreement between real and synthetic UBM images measurements of anterior chamber and iridociliary parameters: Bland–Altman plot for AOD500 (a), CT0 (b), CT1000 (c), IT500 (d), TCA (e), and TCPD (f). AOD500 Angle opening distance, CTO ciliary body thickness at the point of the scleral spur, CT1000 ciliary body thickness at the distance of 1000 um from the scleral spur, IT500 iris thickness, measured at 500 μm from the scleral spur. TCA trabecular-ciliary process angle, PCPD trabecularciliary process distance; for more detail, see text

Full size image

In the out-of-distribution (time-domain ASOCT) dataset, the fair to excellent agreements were also observed between real and synthetic UBM image measurements of the anterior chamber and iridociliary parameters (Table 4). ICC values were 0.70 (CT1000), 0.82 (CT0), 0.86 (AOD500), 0.52 (IT500), 0.73 (TCA), and 0.81 (TCPD).

Table 4 Repeatability of anterior chamber angle and ciliary body measurements between synthetic and real ultrasound biomicroscopy images from he out-of-distribution dataset
Full size table

As no previous data are available on the question addressed in our study, we were unable to determine whether the FID measured in our experiment was good or not. Therefore, we added the synthetic ASOCT images from a precious study performed by our group as reference. Relative to calculations based on the real UBM images, the FID was 21.3, 24.1, and 102.8 for synthetic UBM images from the in-distribution dataset, synthetic UBM images from out-of-distribution dataset, and synthetic ASOCT images, respectively (Fig. 4). It was noted that the FIDs obtained from in-distribution and out-of-distribution UBM images were much smaller than those obtained by synthetic images from different anterior chamber imaging modalities.

Fig. 4
figure 4

FID for synthetic UBM images from in-distribution dataset, synthetic UBM images from out-of-distribution dataset, and the synthetic ASOCT images, respectively. FID Fréchet Inception Distance

Full size image

We also noted several cases of failure, as shown in Fig. 5. If an input SS-ASOCT image shows an open angle with shallow anterior chamber, the CycleGAN would not be able to generate angle structure correctly.

Fig. 5
figure 5

Sample with failed UBM image generation

Full size image

Discussion

In this study, we proposed a CycleGAN-based deep learning technique for generating UBM images using SS-ASOCT images as the inputs. Our experiments suggest that realistic synthetic UBM images can be generated from SS-ASOCT images using CycleGANs. Moreover, the CycleGAN synthetic UBM images can reveal iridociliary structure. The measurements of iridociliary parameters from the CycleGAN synthetic UBM images showed fair to excellent reproducibility compared with those from the real UBM images. Although the measurements of anterior segment parameters obtained with real and synthetic UBM images cannot be considered interchangeable, there is fair to excellent correlation between them. The potential application of this technique is promising, as most clinical researchers still use UBM as the angle image modality in diagnosing angle-closure.

Assessment of angle structure is an essential part of diagnosing and determining the management of individuals with angle-closure [17, 27]. UBM allows visualization of the anterior segment and angle structures at an ultrasonic frequency of 35–100 MHz, providing high-resolution (50 μm) images of the angle and iridociliary, which adds valuable information regarding causal mechanisms of angle-closure [28]. UBM is characterized to assess the pathologies behind the iris, such as, for example, plateau iris, lens-induced glaucoma, ciliary block, cysts, and solid tumors of the anterior segment. On the other hand, SS-ASOCT has a fast scan speed (50,000 A-scans/s), allows deep imaging (11 mm), and has a higher resolution (2.4 µm); in addition, it provides a simple, user-friendly, and non-contact method of assessing the angle structure and providing more detailed information and precise parameters of the angle and corneal, iris, and anterior chamber volumes. The fast scan rate of SS-ASOCT effectively decreases motion artifacts [2]. Unfortunately, SS-ASOCT cannot visualize structures posterior to the iris [29], which precludes assessment of the impact of the iridociliary in some angle-closure mechanisms, such as plateau iris which is more common in the Chinese population. Therefore, it would be clinically advantageous to develop a diagnostic modality in which these two anterior segment imaging modalities are integrated. For example, Kwon et al. assessed eyes with primary angle-closure (PAC) using both SS-ASOCT and UBM [17]. Based on their results, these authors suggested that by using UBM, clinicians may obtain more clues on the mechanisms of PAC. Moreover, prior SS-ASOCT data can be transferred to UBM and further combined into an anterior segment dataset for medical follow-up, clinical research, and deep learning analysis.

GANs offer a novel method to generate new medical images. Recent studies suggest possible applications of generative methods for retinal image registration and fundus or optical coherence tomography image denoising [30]. Traditional GANs require a large training dataset to synthesize realistic medical images. For example, in our previous studies [11], we proposed a GAN approach to generate realistic ASOCT images using more than 20,000 ASOCT images. Moreover, traditional GAN models, such as progressively grown GANs or deep convolutional GANs, can only work in images from the same domain. To synthesize medical images from different domains, several generative networks have been suggested, including unsupervised GAN models, such as CycleGAN, and supervised networks, such as Pix2Pix GAN. Pix2Pix techniques have shown good performance in image translation settings. However, the critical shortage of paired dataset restricts the real application of supervised GANs. UBM requires a contact immersion scanning technique that is run by skilled operators. Therefore, it is difficult to collect a sufficiently large paired dataset for supervised GAN training, which is also a common issue in medical imaging research. In addition, it is inevitable that the body or eyes will move during different scanning procedures, such as CT and MRI, ASOCT and UBM, which make it a challenge to match or register two different domain images. CycleGAN is a type of unsupervised machine learning technique with the significant advantage of being able to utilize the unpaired dataset with two different domains. Recently, several researchers have demonstrated a CycleGAN-based domain transfer between different image modalities. For example, Li et al. proposed a CycleGAN architecture to synthesize MRI images from brain CT images for MR-guided radiotherapy [31]. In another study, Muhaddisa et al. reported an effective domain adaptation method based on CycleGANs to map MR images from different datasets taken from different hospitals using different scanners and settings [32]. Their technique effectively enlarged the MRI dataset, and the proposed scheme was shown to achieve good diagnostic performance for predicting molecular subtypes in low-grade gliomas. Using a similar CycleGAN model, Yoo et al. showed this technique can synthesize traditional fundus photography images directly from ultra-widefield fundus photography without a manual pre-conditioning process [15]. However, cross-modality domain transfer between SS-ASOCT and UBM is challenging due to the high variability of angle tissue or structure appearance caused by different imaging mechanisms [33]. In the current study, we built a CycleGAN-based deep learning model for the domain transfer from SS-ASOCT to UBM. Our result was encouraging in that we found that the CycleGAN model can generate realistic UBM images of high quality, as assessed by human experts in our Turing test.

Despite the above, our study has a few limitations. First, our CycleGAN model can only generate UBM images with 256 × 256 pixel resolution, which is lower than that of the Casia2 SS-ASOCT system (2129 × 1464 pixel resolution) and the Suoer UBM (1024 × 655 pixel resolution) system. For the evaluation of some small lesions, such as ciliary body cysts or tumors, a higher resolution may be needed. As only part of the iris and lens were synthesized in the UBM image, we also could not assess the reproducibility of some lens or iris parameters (lens vault or iris convexity). On the other hand, some angle structure, such as iris thickness measured at 500 μm from the scleral spur, only has a length of a few pixels which could cause lower ICC in our study. The aim of the present study was to build a CycleGAN-based deep learning model for the domain transfer from SS-ASOCT to UBM. It is possible to generate higher resolutions (e.g., 1024 × 1024 or above), as reported in our previous study [12]. Future work will involve generating small lesions or angle structures, which might help improve the clinical application of GAN models. Second, most of the subjects in our testing datasets had an open-angle. Other angle-closure mechanisms, like pupil block, plateau iris configuration, thick peripheral iris roll, and exaggerated lens vault, were not evaluated in the current study [34]. Generating the iridociliary for different angle-closure mechanisms might help improve our understanding of angle-closure. Third, although we assessed the generalizability of the CycleGAN model by testing it within independent external datasets, the whole dataset is collected from the same center, using the same devices, following the same process. This will result in the method being vulnerable to the domain shift and will also cause over-optimism about the results. The latest module of SS-ASOCT, CASIA2, is only available in very few centers in mainland China, which prevented us from collecting more data from other centers. The application of the CycleGAN technique requires further validation in multi-center and multi-ethnic trials. Fourth, the CycleGAN model does not work in a pairwise fashion. For paired image-to-image translation (i.e., ASOCT images to UBM images with the same corresponding clock-hour position), Pix2Pix architecture (a supervised GAN) is the better model to learn a mapping from input images to corresponding output images. However, supervised GAN needs paired datasets that are not available in the current study. Overlapped examinations of both SS-ASOCT and UBM may cost the patients more and increase the use of human resources. The objective of this study was to evaluate the feasibility of CycleGANs to generate synthesized UBM images from SS-ASOCT images. The real-world applications of UBM include tumors, trauma, and surgical complications, all of which cannot be easily synthesized using the CycleGAN model. Future work is need to evaluate the iridociliary and angle pathologies using other GAN techniques, such as meta-learning-based GANs or semi-supervised GANs, by collecting pairs of ASOCT and UBM images of different pathologies.

Conclusion

In conclusion, we developed a CycleGAN model for generating UBM images using SS-ASOCT images as the inputs. Our preliminary results showed that there has fair to excellent correlation between anterior segment parameters measured from the synthetic and real UBM images. The CycleGAN-based deep learning technique presents a promising way to assess the iridociliary with non-contact methods.