Abstract
Purpose
Retinopathy of prematurity (ROP) is a retinal vascular proliferative disease common in low birth weight and premature infants and is one of the main causes of blindness in children.
In the context of predictive, preventive and personalized medicine (PPPM/3PM), early screening, identification and treatment of ROP will directly contribute to improve patients’ long-term visual prognosis and reduce the risk of blindness. Thus, our objective is to establish an artificial intelligence (AI) algorithm combined with clinical demographics to create a risk model for ROP including treatment-requiring retinopathy of prematurity (TR-ROP) infants.
Methods
A total of 22,569 infants who underwent routine ROP screening in Shenzhen Eye Hospital from March 2003 to September 2023 were collected, including 3335 infants with ROP and 1234 infants with TR-ROP among ROP infants. Two machine learning methods of logistic regression and decision tree and a deep learning method of multi-layer perceptron were trained by using the relevant combination of risk factors such as birth weight (BW), gestational age (GA), gender, whether multiple births (MB) and mode of delivery (MD) to achieve the risk prediction of ROP and TR-ROP. We used five evaluation metrics to evaluate the performance of the risk prediction model. The area under the receiver operating characteristic curve (AUC) and the area under the precision-recall curve (AUCPR) were the main measurement metrics.
Results
In the risk prediction for ROP, the BW + GA demonstrated the optimal performance (mean ± SD, AUCPR: 0.4849 ± 0.0175, AUC: 0.8124 ± 0.0033). In the risk prediction of TR-ROP, reasonable performance can be achieved by using GA + BW + Gender + MD + MB (AUCPR: 0.2713 ± 0.0214, AUC: 0.8328 ± 0.0088).
Conclusions
Combining risk factors with AI in screening programs for ROP could achieve risk prediction of ROP and TR-ROP, detect TR-ROP earlier and reduce the number of ROP examinations and unnecessary physiological stress in low-risk infants. Therefore, combining ROP-related biometric information with AI is a cost-effective strategy for predictive diagnostic, targeted prevention, and personalization of medical services in early screening and treatment of ROP.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Introduction
Retinopathy of Prematurity (ROP) is a pathological process that occurs in the immature retinal tissue of low birth weight preterm infants [1,2,3]. It is a developmental disorder of retinal vessel that can develop into tractional retinal detachment in the later stage, which may lead to vision loss and is one of the leading cause of blindness in infants [4]. According to a retrospective analysis [5], ROP caused as much as 8% of blindness in children worldwide. During the first epidemic of ROP in the mid-twentieth century, the main risk factor at the time were unmonitored supplemental oxygen, the use of high concentration of oxygen and unrestricted oxygen therapy to improve survival of preterm infants, mainly affecting preterm infants in industrialized countries such as the United States and Western Europe [3, 6]. In the following two decades, as oxygen use in newborns was controlled, the blindness of ROP declined and the survival rate of extremely preterm infants greatly improved, and a second epidemic of ROP emerged in the industrialized countries in the 1970s [7]. With the widespread promotion of monitored oxygen inhalation for preterm infants, ROP screening and treatment have been popularized. In particular, developed countries have formulated various systems to strictly control the blinding rate of ROP, which has stabilized the incidence of ROP. Nevertheless, due to large differences between countries in preterm birth rates, the level of neonatal intensive care, the number of trained neonatal nurses, and the effectiveness of screening and treatment programs for ROP [8], blindness due to ROP varied widely among countries, especially with higher occurrence of severe ROP in preterm infants in low- and middle-income countries [9]. Since then, the epidemiology of ROP in preterm infants has changed both geographically and temporally, primarily related to two major factors: a progressively lower neonatal mortality rate and the implementation of stringent oxygen monitoring protocols [10, 11].
To mitigate the risk of vision loss caused by ROP, there are the following two prevention strategies: (1) Primary prevention of ROP, through strict oxygen titration, effectively reduces the incidence of treatment-requiring retinopathy of prematurity (TR-ROP) [12]; (2) Secondary prevention of ROP, providing a higher proportion of oxygen, and implementing timely ROP screening in the neonatal intensive care units (NICUs) for all at-risk newborns [13]. However, there is a delicate balance: a lower ratio of oxygen is effective in reducing all subsequent morbidities in preterm infants but thus increases the probability of mortality, and vice versa [14]. ROP screening was recommended for preterm infants with gestational age (GA) < 31 weeks or birth weight (BW) < 1501 g based on demographic criteria in the United States [12], whereas the range of BW and GA of affected infants was much broader in low- and middle-income countries [8, 15, 16]. ROP Examinations should be scheduled based on the preterm infant’s gestational age at birth and the presence and severity of subsequent disease [1]. ROP screening helps identify preterm infants who progress to TR-ROP so that timely treatment can be provided. Over the past few decades, the incidence of blindness due to ROP had shown a rapid increase in many low- and middle-income countries. Many NICUs in these countries lacked the material resources required for primary prevention of ROP, such as oxygen blenders and pulse oxygenation monitors [9], which further contributed to higher incidence and severity of ROP in these countries [17], with even mildly preterm infants at risk for blinding ROP, and therefore, more infants needing to be screened [1, 18]. As emphasized in the European association for predictive, preventive and personalized Medicine (EPMA) white paper [19], identifying infants at high risk of ROP is an crucial step in preventing blindness due to ROP.
Working hypothesis and study aims in the PPPM framework
Artificial intelligence (AI) algorithms were already widely used to assist in the diagnosis of ophthalmic diseases, which will be able to ease the burden to some extent in countries or regions where the burden of these diseases far exceeds existing screening capabilities [20, 21]. The diagnosis of ROP involved three main clinical features: Zone, indicating the location of retinal vascularization that provides an indication of infant maturity and the risk of ROP development; Stage, describing the characteristics when acute ROP vascular features appear at the vascular-avascular juncture; and plus disease [4], where retinal vessels exhibit dilation and tortuosity [4, 12, 22]. Deep learning algorithms based on imaging and informatics for ROP primarily focus on the three main clinical characteristics mentioned above, including automatic diagnosis of plus disease [23], quantitative assessment of retinal vascular abnormalities [24, 25], and automatic diagnosis and staging of ROP [26,27,28]. However, these studies had predominantly utilized imaging data to achieve automatic diagnosis of ROP-related diseases, but few studies had combined the ROP risk factors to build ROP prediction models, which are crucial for the prediction and screening of ROP [29]. Even though some studies had combined two major risk factors such as BW and GA for risk prediction of ROP [13], they had ignored the exploration of crucial risk factors such as mode of delivery (MD), gender and multiple births (MB) [29]. Beyond this, an AI algorithm that showed high diagnostic accuracy in a particular research dataset or in a particular demographic population may not be generalizable to other populations [30]. To close this knowledge gap, we utilized five ROP-related risk factors, including BW, GA, MD, gender, and MB, to explore whether effective ROP risk and TR-ROP risk prediction can be constructed solely based on biometric information. In this study, we retrospectively evaluated the risk prediction performance of AI-based ROP and TR-ROP risk prediction models in a real-world dataset from the Shenzhen Eye Hospital informatics ROP cohort study. We further evaluated the hypothesis that risk prediction models based solely on biometric information can be used to predict the risk of ROP and TR-ROP, aiming to assess potential differences in neonatal care within this population. Therefore, this study aimed to explore the relationship between the biometric information of preterm infants and the risk of ROP and TR-ROP, and to explore the feasibility of the PPPM concept in mitigating blindness caused by ROP in Chinese preterm infants.
Methods
Dataset preparation and analysis
This study had received approval from the Medical Ethics Committee of Shenzhen Eye Hospital (SZEH) and was conducted in accordance with the principles of the Helsinki Declaration. Written informed consent had been obtained from the parents of infants when they come to the clinic for ROP screening. As part of the informatics Retinopathy of Prematurity (i-ROP) cohort study, ROP screening was conducted on 22,569 unique patients (BW < 1533 g or GA < 32 weeks) between March 2003 and September 2023. The final diagnostic results were consistently determined by three ROP experts using the comprehensive international classification of ROP (ICROP) standards [4, 12]. Statistical single-factor analysis was performed using IBM SPSS Statistics 25 software for both ROP and TR-ROP risk prediction, considering five candidate risk factors: BW, GA, gender, MB, and MD (eutocia or caesarean section). For continuous variables such as BW and GA, Mann-Whitney U test [31] was employed in the single-factor analysis, while for categorical variables such as gender, MB, and MD, Chi-Squared test was applied. Two-side p values < 0.05 were considered statistically significant. As can be seen from Table 1, in the risk prediction for both ROP and TR-ROP, four factors, namely BW (p < 0.001, p < 0.001), GA (p < 0.001, p < 0.001), gender (p < 0.001, p < 0.001), and MD (p < 0.001, p < 0.001) exhibited significant statistical differences (p < 0.05), while MB (p = 0.243, p = 0.878) did not exhibit significant statistical differences.
Subsequently, based on the four risk factors identified with significant statistical differences in the single-factor analysis, a binary logistic regression model was employed to conduct a multi-factor analysis for both ROP and TR-ROP risk factors. The results of the multi-factor analysis were presented in Table 2. After analysis, we finally identified four factors, including BW (p < 0.001, p < 0.001), GA (p < 0.001, p < 0.001), gender (p < 0.001, p < 0.001), and MD (p = 0.008, p < 0.001), as predictive risk factors for ROP and TR-ROP.
Finally, these 22,569 patients were evenly divided into five groups based on their final diagnostic results respectively, and these five groups were used in subsequent five-fold cross-validation experiments. The patients in the training and testing datasets were mutually exclusive.
Risk model development
From the single-factor analysis and multi-factor analysis, we can observe that BW, GA, Gender, and MD may be the major factors in risk prediction of ROP and TR-ROP. In this study, two machine learning methods of logistic regression and decision tree and a deep learning method of multilayer perceptron (MLP) were used to achieve ROP and TR-ROP risk prediction. All three models utilized a five-fold cross-validation strategy for model optimization. Logistic regression incorporated L2 regularization as a penalty term and lbfgs as the optimization algorithm. Decision tree utilized entropy based on Shannon information gain to measure the quality of a split, employing a random strategy for selecting the best random split at each node. In MLP, five linear layers were designed, with the output of the first four linear layers sequentially processed through a ReLU activation layer and a batch normalization layer. The last linear layer was used for the final classification.
Model evaluation
The dataset used in this study had class imbalance, with the proportion of preterm infants with ROP being 14.8% compared to no-ROP. Among ROP patients, TR-ROP accounts for 37%. Therefore, we mainly chose the area under the precision-recall curve (AUCPR) and the area under the receiver operating characteristic curve (AUC) as the main measures of model performance. DeLong’s test was used to compare the AUCs of different models. Additionally, evaluation metrics such as accuracy (Acc), sensitivity (Sen) and specificity (Spec) are included. Acc calculates the ratio of the number of correctly predicted samples (true positives and true negatives) to the total number of samples (true positives, false negatives, false positives, and true negatives). Sen measures the proportion of true positive samples predicted by the model among all actual positive samples. Spec measures the proportion of true negative samples predicted by the model among all actual negative samples. To calculate these metrics, we utilize the following formulas:
TP, TN, FP, FN and TN represent true positives, true negatives, false positives, false negatives, true negatives respectively.
Results
Data characteristics
This study used demographic data and clinical outcomes from 22,569 infants enrolled in the SZEH between 2003 and 2023. Tables 1 and 2 displays relevant demographic data and clinical outcomes from the dataset. Among those infants, 9915 were female (43.9%). The mean (SD) GA was 31.4 (2.4) weeks, and the mean (SD) BW was 1480.5 (334.3 g). Among those infants, 3335 were diagnosed with ROP, with 1234 infants identified as TR-ROP. The median (interquartile range, IQR) BW for no-ROP infants is 1570 g (1300.0–1800.0 g), while for infants with ROP, it is 1160 g (980.0–1400.0 g). The median (IQR) GA for no-ROP infants is 32.0 weeks (30.3–33.4 weeks), whereas for ROP infants, it is 29.0 weeks (27.6–30.7 weeks). The median (IQR) BW for no-TR-ROP infants was 1499.6 g (1495.2–1504.0 g), while for TR-ROP infants, it was 1150.5 g (1133.5–1167.6 g). The median (IQR) GA for no-TR-ROP infants was 31.6 weeks (31.569–31.631 weeks), whereas for TR-ROP infants, it was 28.6 weeks (28.5–28.7 weeks). Figure 1 illustrated the distribution of infants in the dataset based on BW and GA.
Performance in ROP risk prediction models
We employed two machine learning methods and one deep learning method to predict the risk of ROP. From the multi-factor analysis results of ROP risk prediction in Table 2, it was evident that BW (p < 0.001), GA (p < 0.001), gender (p < 0.001), and MD (p = 0.008) were the primary predictive risk factors for ROP risk. We designed four different combinations of risk factors, and all three models explored the optimal combination of risk factors within these four combinations.
The experimental results were shown in Table 3. In the five-fold cross-validation of logistic regression, the combination GA + BW performed the best among the three metrics: AUCPR, Acc, and Sen. Although it did not achieve the highest AUC compared to the combination GA + BW + Gender (81.36 (80.90–81.82)), the difference was only 0.12%. DeLong’s test showed no significant difference in AUC between them (p = 0.973). Except for the Spec, the comprehensive performance of the combination GA + BW + Gender + MD + MB was the worst (GA + BW vs GA + BW + Gender + MD + MB, p < 0.001). The combination GA + BW + Gender + MD had an AUC 0.83% lower than GA + BW, although it did not show a significant difference (p = 0.072). In decision tree, the combination GA + BW achieved the best performance in all four metrics, and AUC showed a significant difference compared to the other three combinations (GA + BW vs GA + BW+ Gender, p = 0.021; GA + BW vs GA + BW + Gender + MD, p < 0.001; GA + BW vs GA + BW + Gender + MD + MB, p < 0.001). In the results of the four combinations in MLP, the AUC were relatively close, with no significant differences observed (GA + BW vs GA + BW+ Gender, p = 0.958; GA + BW vs GA + BW + Gender + MD, p = 0.329; GA + BW vs GA + BW + Gender + MD + MB, p = 0.828). According to the comprehensive results of the three models, the combination GA + BW had certain advantages in the risk prediction of ROP, and the logistic regression had the best effect among the three methods. Although in the DeLong’s test results for AUC under the GA + BW combination, MLP did not show significant differences (logistic regression vs decision tree, p < 0.001; logistic regression vs MLP, p = 0.5413). The comparative ROC and PR curves and confusion matrix for the three models were shown in Figs. 2 and 3.
Performance in TR-ROP risk prediction models
The results of the three TR-ROP risk prediction models were presented in Table 4. In logistic regression, the combination GA + BW + Gender + MD + MB demonstrated the best overall performance, with the highest AUC of 83.28 (82.19–84.37). With this combination of risk factors, the diagnostic Sen reached 75.36% and the Spec reached 74.83%, which meant that the logistic regression was able to identify about 75% of infants who need treatment and nearly 75% of infants who did not need treatment. Although there were no significant differences compared to other combinations (GA + BW + Gender + MD + MB vs GA + BW, p = 0.601; GA + BW + Gender + MD + MB vs GA + BW+ Gender, p = 0.760; GA + BW + Gender + MD + MB vs GA + BW + Gender + MD, p = 0.948). In decision tree, the combination GA + BW performed best in overall performance, showing significant differences in AUC compared to other combinations (p < 0.001 for all). However, its highest AUC is only 73.68 (71.66–75.70). In MLP, the combination GA + BW achieved the best AUC and AUCPR, but there were no significant differences compared to other combinations (GA + BW vs GA + BW+ Gender, p = 0.485; GA + BW vs GA + BW + Gender + MD, p = 0.215; GA + BW vs GA + BW + Gender + MD + MB, p = 0.101). Among the three models, logistic regression had the highest AUC and AUCPR in the combination GA + BW + Gender + MD + MB. Compared to the best combinations in the other two models, DeLong’s test results showed significant differences in decision tree (p < 0.001) and no significant differences in MLP (p = 0.348). The comparative ROC and PR curves for the three models were shown in Fig. 4.
Discussion
Predictive approach
Early detection, prevention, and intervention of ROP are key to reducing incidence and mortality and alleviating the huge socio-economic burden caused by ROP. Screening out the factors with significant statistical differences from numerous risk factors and exploring the best combination of risk factors are an effective strategy for predictive diagnostic, targeted prevention and personalized treatment of ROP [13, 29]. Low-risk preterm infants should receive fewer examinations to alleviate scarce medical resources, while high-risk infants (premature infants with ROP who require treatment) should be treated with medication or laser surgical interventions to reduce the risk of blindness, which effectively promotes the paradigm shift from reactive medicine to the advanced approach by utilizing the 3 PM framework. The active promotion of the PPPM framework has been applied in many fields, making great contributions to the health quality improvement including fundus diseases [32], diabetes [33], cancers [34], etc. In this study, we retrospectively evaluated an AI system for ROP risk and TP-ROP risk prediction developed for preterm infants in Chinese population based on the i-ROP cohort study dataset from SZEH. First, statistical single-factor analysis was used to determine the risk factors with statistically significant differences among the five risk factors for ROP and TR-ROP. Then, based on the risk factors with significant statistical differences found in the single-factor analysis, a binary logistic regression model was used for multi-factor analysis to determine the final risk prediction factors. Finally, with statistically significant risk factors as input, two machine learning methods of logistic regression and decision tree and a deep learning method of MLP were used to explore the optimal prediction method for ROP and TR-ROP risk prediction.
We observed that among the three AI-based ROP risk prediction models, logistic regression has better comprehensive performance. Only using BW and GA can identify approximately 23% of infants who are at risk of ROP before being diagnosed with ROP, while simultaneously excluding 97.75% of those at low risk (Table 3, Fig. 3-A). When other risk factors were added to the combination of GA + BW, both logistic regression and MLP showed improvement in specificity, which indicated that more low-risk people could be excluded. In MLP, adding gender can achieve better sensitivity compared to the combination GA + BW (Fig. 3). In addition, we knew that in the results of single-factor analysis and multi-factor analysis for risk factors (Tables 1 and 2), MD is a risk factor with significant statistical difference, but there is no significant improvement after adding MD to the ROP risk prediction methods. Instead, a certain performance decline was observed in all three methods (AUC and AUCPR exhibited a decrease in Fig. 2). On the other hand, although there is no statistically significant difference in the risk factor MB, the model’s specificity was improved by adding MB to logistic regression (Fig. 3). Among the three AI-based TR-ROP risk prediction models, logistic regression demonstrated relatively superior performance. Using the risk factor combination GA + BW + Gender + MD + MB, 75.36% of children who need treatment could be identified, while excluding nearly 75% of infants who did not require treatment (Table 3, Fig. 5-D). Moreover, we found that there was no significant difference in AUC between the two methods of logistic regression and MLP in the four risk factor combinations (Table 4).
The comprehensive confusion matrix for each model’s five-fold cross-validation results in ROP risk prediction. A: GA + BW, B: GA + BW + Gender, C: GA + BW + Gender + MD, D: GA + BW + Gender + MD + MB, for logistic regression. E: GA + BW, F: GA + BW + Gender, G: GA + BW + Gender + MD, H: GA + BW + Gender + MD + MB, for decision tree. I: GA + BW, J: GA + BW + Gender, K: GA + BW + Gender + MD, L: GA + BW + Gender + MD + MB, for MLP
The comprehensive confusion matrix for each model’s five-fold cross-validation results in TR-ROP risk prediction. A: GA + BW, B: GA + BW + Gender, C: GA + BW + Gender + MD, D: GA + BW + Gender + MD + MB, for logistic regression. E: GA + BW, F: GA + BW + Gender, G: GA + BW + Gender + MD, H: GA + BW + Gender + MD + MB, for decision tree. I: GA + BW, J: GA + BW + Gender, K: GA + BW + Gender + MD, L: GA + BW + Gender + MD + MB, for MLP
It can be seen from the result analysis that the comprehensive performance of logistic regression is better, BW + GA risk factor combination can obtain the best performance in ROP risk prediction, and the risk factor combination GA + BW+ Gender + MD + MB has the best performance in TR-ROP risk prediction. We established a new ROP and TP-ROP prediction model based on the combination of these risk factors. The formula of ROP risk prediction model was as follows:
The formula of TR-ROP prediction model is as follows:
Where x1 represents GA, x2 represents BW, x3 represents gender, x4 represents MD, and x5 represents MB. In the context of PPPM [35], efforts to incorporate this risk prediction paradigm into current clinical workflows would be advantageous, as it has the potential to become a promising tool for early detection and treatment of ROP disorders in infants.
Targeted prevention
In terms of prevention [35], ROP was one of the main causes of blindness in children, and early detection can help prevent further deterioration of the condition in time and stop the progression of the disease. Our findings indicated that using the five easily available ROP risk factors for early ROP risk and TR-ROP risk prediction can help reduce the burden on ophthalmologists and improve screening efficiency, especially in low and middle-income regions. In this study, for a targeted ROP prevention model designed for a specific Chinese population, we further simplified the model into nomograms as shown in Fig. 6, so that it can be more convenient to enter the clinical application.
The above situation was obtained from the analysis of a specific Chinese population, and the model is unlikely to generalize well “out of the box” to populations different from the Chinese screening population. Depending on epidemiological and demographic risk factors, the prediction models need to be re-adjusted based on local disease epidemiology, and different risk factor combinations may have different performance because the incidence of ROP varies geographically and over time [29, 30]. We believed that these results prove that the AI-based risk prediction model can effectively reduce the number of ROP examinations and associated physiological stress in low-risk infants, thereby improving the efficiency of ROP screening, and can also be used as an epidemiological tool to monitor NICU-level ROP risk and TP-ROP risk prediction across regions and time. Finally, quantitative monitoring that combines GA and BW, the two strongest risk factors for developing ROP, with other risk factors could allow for earlier and more consistent diagnosis of TR-ROP, thereby minimizing the overall risk of adverse outcomes.
This model may also be easier to implement than the existing ROP risk prediction models. Coyner et al. [13] proposed a method for TR-ROP risk prediction using a combination of GA and Vascular Severity Score (VSS) factors and achieved good performance. However, in this method, an additional AI model needs to be trained to obtain the VSS based on the patient’s fundus images, which is relatively cumbersome in clinical practice because the images are not part of the standard of care and digital fundus cameras may be expensive, so in some low- or middle-income countries or regions, fundus images of infants are not easily available. Moreover, the best AUC obtained by this method was 0.82, while our prediction model could reach 0.83. The specificity of our proposed risk prediction model was much higher than that of the ROP prediction model of the Children’s Hospital of Philadelphia [36], which uses BW + GA + weight gain to predict the risk of type II ROP and TR-ROP, and the highest specificity was 53.4%, while in in our TR-ROP prediction model, the specificity reached 74.83%.
Personalization of medical services
There are many risk factors for ROP [29], and this study was based on the analysis of a specific Chinese population. In practice, when the model was extended to a population different from the Chinese screening population, it was recommended to appropriately adjust the input parameters of the model according to the local disease epidemiology. Different combinations of risk factors may have different impacts on the performance of the risk prediction model. This paradigm of ROP and TR-ROP risk prediction by using ROP risk factors as input to the model will enable high-risk infants receive earlier and more accurate diagnosis and treatment, while minimizing the over-examination of low-risk infants, and better reduce or eliminate the occurrence of ROP-related blindness. In this study, the overall sensitivity of the ROP risk prediction model was relatively low, and it may be necessary to continue to integrate oxygen exposure, intraventricular hemorrhage, neonatal sepsis, necrotizing enterocolitis, thrombocytopenia, and other previously associated risk factors [29] to further improve the model’s sensitivity.
For the automatic identification of ROP, many deep learning (DL) algorithms were currently developed based on color fundus photography images. For example, Brown et al. [23] used two CNNs to realize the automatic diagnosis of plus disease in ROP. First, a U-Net architecture was used to segment blood vessels, and then an Inception network [37] was used to realize the classification task of plus disease in ROP. Furthermore, there were studies using DL algorithms to achieve quantitative assessment of ROP risk severity. For example, Taylor et al. [24] used a DL algorithm for vascular severity scoring to identify TR-ROP by quantifying clinical disease progression, while Kellyn et al. [38] used a vascular severity score based on DL to describe aggressive posterior retinopathy of prematurity quantitatively. In addition, there are also some studies to implement automatic classification tasks of ROP. For example, Zhang et al. [39] used a VGG-16 network to realize the automatic classification task of ROP. Wang et al. [40] developed an automatic ROP detection system using two deep neural networks, in which the Id-Net network was used to identify the ROP, the Gr-Net network was further used to grade the severity of ROP. With the popularity of smartphones, smartphone-based cameras may become a substitute for professional color fundus photography, and the acquisition of infant fundus images may become more convenient. According to fundus images, more clinical characteristics can be obtained, such as the development of blood vessels in different fundus zones. Whether combining that information with the factors previously involved can further improve the prediction performance of the model is worth exploring. If possible, this would not only greatly reduce the burden of ROP screening, but also enable high-risk infants to receive timely treatment and reduce the burden on society. In any case, the risk prediction model proposed in this study can lay a basic framework for a new ROP prediction model, which can at least achieve ROP risk prediction and TR-ROP risk prediction only by inputting GA and BW, without requiring a complete ophthalmoscopy. This paradigm allows low-risk infants to receive fewer examinations, while high-risk infants receive earlier, more precise diagnosis and treatment. This could allow rural areas and low- and middle-income countries to make better use of scarce resources.
Conclusion, outlook in the framework of PPPM/3 PM, and limitations of the study
Conclusion
Our study found that birth weight, gestational age, multiple births; mode of delivery, gender were related to the occurrence of ROP and TR-ROP in specific Chinese populations. We employed multiple risk factors to explore predictive models for ROP risk and TR-ROP risk, identifying the best model through rich comparative experiments of three different models. In the SZEH’s i-ROP cohort study dataset, we demonstrated that using only the two main risk factors of BW and GA not only could identify approximately 23% of infants at risk for ROP before ROP is diagnosed, but excluded 97.75% of low-risk infants. It can also identify about 75% of infants who need treatment and nearly 75% of children who did not need treatment were detected. We also constructed a simple and easy to operate nomogram model for clinical staff to use. The implementation of this model can significantly reduce the number of ROP examinations in low-risk infants, and can identify TR-ROP infants at an early stage and give timely treatment, so as to better utilize ROP screening resources. Future work will verify the effectiveness of this paradigm in a larger population and explore whether using more relevant risk factors can further improve the predictive performance of the model and better reduce or eliminate the occurrence of blindness due to ROP.
Limitations and outlook in the framework of PPPM/3 PM
This study has several limitations. Our i-ROP cohort study dataset is relatively small, contained only Chinese individuals, and was limited to a single geographic region, limiting its generalizability. Therefore, in future work, databases from different regions and other parts of the world should be expanded to increase the data diversity and improve the applicability of risk prediction models. The ROP risk prediction model designed in this study is a non-invasive automated system for early prediction of ROP and TR-ROP in infants. It also aligns with a key aspect of the PPPM/3 PM approach highlighted at the 2019 EPMA World Congress, which focuses on customizing and continuously monitoring patients’ clinical parameters to improve treatment outcomes [41]. Additionally, this study considered only five factors associated with ROP risk, and the overall sensitivity of the ROP disease risk prediction model is relatively low. In the future, incorporating more risk factors for analysis should be considered to enhance the sensitivity of the prediction model. To facilitate future applications of AI in this field, it is strongly recommended to consider the following two points:
-
(1)
Ensure that biological information about ROP is accurately recorded at birth, including but not limited to risk factors such as BW, GA, gender, MD, and MB, so as to provide real and effective data support for the subsequent disease risk prediction models.
-
(2)
In order to be able to predict disease progression comprehensively, it is recommended to register follow-up records in a timely manner to facilitate the evaluation of treatment decisions relevant to the clinical setting, so that predictive models can be applied to clinical risk monitoring at an early stage. Guided by the PPPM/3 PM approach, better clinical outcomes for patients are provided by providing complete prediction from diagnosis to treatment decisions.
Data availability
No datasets were generated or analysed during the current study.
Code availability
The source code used in this study is available upon request from the first author only.
Abbreviations
- ROP:
-
Retinopathy of prematurity
- TR-ROP:
-
Treatment-requiring retinopathy of prematurity
- AI:
-
Artificial intelligence
- i-ROP:
-
Informatics Retinopathy of Prematurity
- BW:
-
Birth weight
- GA :
-
Gestational age
- MB:
-
Multiple births
- MD :
-
Mode of delivery
- SD :
-
Standard deviations
- CI:
-
Confidence interval
- CS:
-
Caesarean section
- MLP:
-
Multilayer perceptron
- AUCPR:
-
Precision-recall curve area under the curve
- AUC:
-
Receiver operating characteristic curve area under the curve
- Acc:
-
Accuracy
- Sen:
-
Sensitivity
- Spec:
-
Specificity
- IQR:
-
Interquartile range;
- ROC:
-
Receiver operating characteristic
- SZEH:
-
Shenzhen Eye Hospital
- NICUs:
-
Neonatal Intensive Care Units
References
Fierson WM, Chiang MF, Good W, Phelps D, Reynolds J, Robbins SL, et al. Screening examination of premature infants for retinopathy of prematurity. Pediatrics. 2018;142:e20183061.
Early Treatment for Retinopathy of Prematurity Cooperative Group. Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial. Arch Ophthalmol. 2003;121:1684–94.
Gilbert C. Retinopathy of prematurity: a global perspective of the epidemics, population of babies at risk and implications for control. Early Hum Dev. 2008;84:77–82.
Chiang MF, Quinn GE, Fielder AR, Ostmo SR, Chan RP, Berrocal A, et al. International classification of retinopathy of prematurity. Third Edition Ophthalmol. 2021;128:E51–68.
Ren H, Su G, Xu S, Zhou L, Cai S. The effects of Combuxil and Leizumab on retinal function and serum interleukin-17A in premature infants with retinopathy. Comput Math Methods Med. 2022;2022:6371994.
Jefferson E. Retrolental fibroplasia. Arch Dis Child. 1952;27:329–36.
Varughese S, Jain S, Gupta N, Singh S, Tyagi V, Puliyel JM. Magnitude of the problem of retinopathy of prematurity: experience in a large maternity unit with a medium size level-3 nursery. Indian J Ophthalmol. 2001;49:187–8.
Gilbert C, Fielder A, Gordillo L, Quinn G, Semiglia R, Visintin P, et al. Characteristics of infants with severe retinopathy of prematurity in countries with low, moderate, and high levels of development: implications for screening programs. Pediatrics. 2005;115:e518–e25.
Blencowe H, Moxon S, Gilbert C. Update on blindness due to retinopathy of prematurity globally and in India. Indian Pediatr. 2016;53:S89–92.
Quinn GE, Barr C, Bremer D, Fellows R, Gong A, Hoffman R, et al. Changes in course of retinopathy of prematurity from 1986 to 2013: comparison of three studies in the United States. Ophthalmology. 2016;123:1595–600.
Lawn JE, Davidge R, Paul VK, Sv X, de Graft JJ, Costello A, et al. Born too soon: care for the preterm baby. Reprod Health. 2013;10:1–19.
International Committee for the Classification of Retinopathy of Prematurity. The International Classification of Retinopathy of Prematurity revisited. Arch Ophthalmol. 2005;123:991–9.
Coyner AS, Chen JS, Singh P, Schelonka RL, Jordan BK, McEvoy CT, et al. Single-examination risk prediction of severe retinopathy of prematurity. Pediatrics. 2021;148:e2021051772.
Campbell JP, Singh P, Redd TK, Brown JM, Shah PK, Subramanian P, et al. Applications of artificial intelligence for retinopathy of prematurity screening. Pediatrics. 2021;147:e2020016618.
Chen Y, Li X. Characteristics of severe retinopathy of prematurity patients in China: a repeat of the first epidemic? Br J Ophthalmol. 2006;90:268–71.
Trinavarat A, Atchaneeyasakul L-o, Udompunturak S. Applicability of American and British criteria for screening of the retinopathy of prematurity in Thailand. Jpn J Ophthalmol. 2004;48:50–3.
Blencowe H, Lawn JE, Vazquez T, Fielder A, Gilbert C. Preterm-associated visual impairment and estimates of retinopathy of prematurity at regional and global levels for 2010. Pediatr Res. 2013;74:35–49.
Shukla R, Murthy G, Gilbert C, Vidyadhar B, Mukpalkar S. Operational guidelines for ROP in India: a summary. Indian J Ophthalmol. 2020;68:S108–S14.
Golubnitschaja O, Epma CV. General Report & Recommendations in Predictive, Preventive and Personalised Medicine 2012: White paper of the European Association for Predictive, Preventive and Personalised Medicine. EPMA J. 2012;3:14.
Ting DSW, Pasquale LR, Peng L, Campbell JP, Lee AY, Raman R, et al. Artificial intelligence and deep learning in ophthalmology. Br J Ophthalmol. 2019;103:167–75.
Wang M, Lin T, Wang L, Lin A, Zou K, Xu X, et al. Uncertainty-inspired open set learning for retinal anomaly identification. Nat Commun. 2023;14:6757.
Committee for the Classification of Retinopathy of Prematurity. An international classification of retinopathy of prematurity. Arch Ophthalmol. 1984;102:1130–4.
Brown JM, Campbell JP, Beers A, Chang K, Ostmo S, Chan RVP, et al. Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks. JAMA Ophthalmol. 2018;136:803–10.
Taylor S, Brown JM, Gupta K, Campbell JP, Ostmo S, Chan RVP, et al. Monitoring disease progression with a quantitative severity scale for retinopathy of prematurity using deep learning. JAMA Ophthalmol. 2019;137:1022–8.
Travis KR, John Peter C, James MB, Sang Jin K, Susan O, Robison Vernon Paul C, et al. Evaluation of a deep learning image assessment system for detecting severe retinopathy of prematurity. Br J Ophthalmol. 2019;103:580–4.
Yo-Ping H, Haobijam B, Eugene Yu-Chuan K, Kuan-Jen C, Yih-Shiou H, Chi-Chun L, et al. Automated detection of early-stage ROP using a deep convolutional neural network. Br J Ophthalmol. 2021;105:1099–103.
Xie H, Liu Y, Lei H, Song T, Yue G, Du Y, et al. Adversarial learning-based multi-level dense-transmission knowledge distillation for AP-ROP detection. Med Image Anal. 2023;84:102725.
Chen S, Zhang R, Chen G, Zhao J, Wang T, Zhang G, et al. Attention-guided Deep Multi-instance Learning for Staging Retinopathy of Prematurity. 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI). 2021; https://doi.org/10.1109/ISBI48211.2021.9434012.
Kim SJ, Port AD, Swan R, Campbell JP, Chan RVP, Chiang MF. Retinopathy of prematurity: a review of risk factors and their clinical significance. Surv Ophthalmol. 2018;63:618–37.
Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann EK. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study. PLoS Med. 2018;15:e1002683.
Fay MP, Proschan MA. Wilcoxon-Mann-Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. Stat Surv. 2010;4:1–39.
Liu Y, Xie H, Zhao X, Tang J, Yu Z, Wu Z, et al. Automated detection of nine infantile fundus diseases and conditions in retinal images using a deep learning system. EPMA J. 2024;15:39–51.
Cui C, Zhang T, Qi Y, Chu J, Xu H, Sun C, et al. Diabetes, glycemic control and arterial stiffness: a real-world cohort study in the context of predictive, preventive, and personalized medicine. EPMA J. 2023;14:663–72.
Fang J, Wu J, Hong G, Zheng L, Yu L, Liu X, et al. Cancer screening in hospitalized ischemic stroke patients: a multicenter study focused on multiparametric analysis to improve management of occult cancers. EPMA J. 2024;15:53–66.
Golubnitschaja O, Baban B, Boniolo G, Wang W, Bubnov R, Kapalla M, et al. Medicine in the early twenty-first century: paradigm and anticipation - EPMA position paper 2016. EPMA J. 2016;7:23.
Binenbaum G, Ying G-s, Quinn GE, Huang J, Dreiseitl S, Antigua J, et al. The CHOP postnatal weight gain, birth weight, and gestational age retinopathy of prematurity risk model. Arch Ophthalmol. 2012;130:1560–5.
Szegedy C, Ioffe S, Vanhoucke V, Alemi AA. Inception-v4, inception-ResNet and the impact of residual connections on learning. Proceedings of the AAAI Conference on Artificial Intelligence 2017; https://doi.org/10.1609/aaai.v31i1.11231.
Bellsmith KN, Brown J, Kim SJ, Goldstein IH, Coyner A, Ostmo S, et al. Aggressive posterior retinopathy of prematurity: clinical and quantitative imaging features in a large north American cohort. Ophthalmology. 2020;127:1105–12.
Zhang Y, Wang L, Wu Z, Zeng J, Chen Y, Tian R, et al. Development of an automated screening system for retinopathy of prematurity using a deep neural network for wide-angle retinal images. IEEE Access. 2019;7:10232–41.
Wang J, Ju R, Chen Y, Zhang L, Hu J, Wu Y, et al. Automated retinopathy of prematurity screening using deep neural networks. EBioMedicine. 2018;35:361–8.
Golubnitschaja O, Topolcan O, Kucera R, Costigliola V, Akopyan M, Akulov SN, et al. 10th anniversary of the European Association for Predictive, preventive and personalised (3P) medicine - EPMA world congress supplement 2020. EPMA J. 2020;11:1–133.
Funding
This study was supported by Macao Polytechnic University (Nos. RP/FCSD-01/2022 and RP/FCA-10/2023), National Natural Science Foundation of China (Nos. 82271103 and 82301269), Guangdong Basic and Applied Basic Research Foundation (2022A1515012326), Shenzhen Medical Research Fund (C2301005), Shenzhen Key Medical Discipline Construction Fund (SZXK038), Sanming Project of Medicine in Shenzhen (SZSM202311018) and supported by Shenzhen Fund for Guangdong Provincial High level Clinical Key Specialties (SZGSP014).
Author information
Authors and Affiliations
Contributions
Shaobin Chen, Yue Sun and Tao Tan were involved in all aspects of the study, including conceptualizing and designing the study, analyzing the data, drafting the initial manuscript, and reviewing and revising the manuscript. Xinyu Zhao, Zhenquan Wu, Yulin Zhang and Guoming Zhang were involved in data collection, coordinated and supervised data collection, carried out the initial data analyses, and critically reviewed and revised the manuscript for important intellectual content. Chan-Tong Lam, Yanwu Xu and Kangyang Cao critically reviewed and revised the manuscript.
Corresponding authors
Ethics declarations
Ethics approval
This diagnostic study was approved by the Ethics Committee of Shenzhen Eye Hospital. All institutions abided by the tenets of the Declaration of Helsinki.
Consent to participate
Not applicable.
Consent for publication
This article has been approved for publication by the authors.
Conflicts interests
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Chen, S., Zhao, X., Wu, Z. et al. Multi-risk factors joint prediction model for risk prediction of retinopathy of prematurity. EPMA Journal 15, 261–274 (2024). https://doi.org/10.1007/s13167-024-00363-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13167-024-00363-7