Skip to main content
SpringerLink
Log in

ResDeepSurv: A Survival Model for Deep Neural Networks Based on Residual Blocks and Self-attention Mechanism

  • Original research article
  • Published:
Interdisciplinary Sciences: Computational Life Sciences Aims and scope Submit manuscript

Abstract

Survival analysis, as a widely used method for analyzing and predicting the timing of event occurrence, plays a crucial role in the medicine field. Medical professionals utilize survival models to gain insight into the effects of patient covariates on the disease, and the correlation with the effectiveness of different treatment strategies. This knowledge is essential for the development of treatment plans and the enhancement of treatment approaches. Conventional survival models, such as the Cox proportional hazards model, require a significant amount of feature engineering or prior knowledge to facilitate personalized modeling. To address these limitations, we propose a novel residual-based self-attention deep neural network for survival modeling, called ResDeepSurv, which combines the benefits of neural networks and the Cox proportional hazards regression model. The model proposed in our study simulates the distribution of survival time and the correlation between covariates and outcomes, but does not impose strict assumptions on the basic distribution of survival data. This approach effectively accounts for both linear and nonlinear risk functions in survival data analysis. The performance of our model in analyzing survival data with various risk functions is on par with or even superior to that of other existing survival analysis methods. Furthermore, we validate the superior performance of our model in comparison to currently existing methods by evaluating multiple publicly available clinical datasets. Through this study, we prove the effectiveness of our proposed model in survival analysis, providing a promising alternative to traditional approaches. The application of deep learning techniques and the ability to capture complex relationships between covariates and survival outcomes without relying on extensive feature engineering make our model a valuable tool for personalized medicine and decision-making in clinical practice.

Graphical Abstract

The proposed architecture framework. (a) provides a comprehensive overview of the essential procedures involved in the ResDeepSurv model. These procedures encompass data preprocessing, the underlying network architecture of the model, and the resultant final output. During the data preprocessing stage, the input data are appropriately prepared to ensure its compatibility with the ResDeepSurv model. Next, the preprocessed data are fed into the network architecture of the ResDeepSurv model. After the data have been transmitted through the network architecture, the proposed model generates the final output, which may encompass prognostic or personalized treatment recommendations. (b) illustrates the network architecture of ResDeepSurv, which encompasses the residual block structure responsible for capturing nonlinear relationships among features. Additionally, it incorporates a self-attention mechanism that learns the relative importance of the output features, which may consist of predictions or personalized treatment recommendations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Availability of Data and Materials

The source code of ResdeepSurv is freely available at https://github.com/HaoWuLab-Bioinformatics/ResDeepSurv.

References

  1. George B, Seals S, Aban I (2014) Survival analysis and regression models. J Nucl Cardiol 21:686–694. https://doi.org/10.1007/s12350-014-9908-2

    Article  PubMed  PubMed Central  Google Scholar 

  2. Cox DR (1972) Regression models and life-tables. J R Stat Soc B 34(2):187–202. https://doi.org/10.1111/j.2517-6161.1972.tb00899.x

    Article  MathSciNet  Google Scholar 

  3. Van Belle V, Pelckmans K, Van Huffel S et al (2011) Improved performance on high-dimensional survival data by application of survival-svm. Bioinformatics 27(1):87–94. https://doi.org/10.1093/bioinformatics/btq617

    Article  CAS  PubMed  Google Scholar 

  4. Bair E, Tibshirani R (2004) Semi-supervised methods to predict patient survival from gene expression data. PLoS Biol 2(4):e108. https://doi.org/10.1371/journal.pbio.0020108

    Article  PubMed  PubMed Central  Google Scholar 

  5. Royston P, Altman DG (2013) External validation of a cox prognostic model: principles and methods. Bmc Med Res Methodol 13:1–15. https://doi.org/10.1186/1471-2288-13-33

    Article  Google Scholar 

  6. Zhang P, Wu Y, Zhou H et al (2022) Clnn-loop: a deep learning model to predict ctcf-mediated chromatin loops in the different cell lines and ctcf-binding sites (cbs) pair types. Bioinformatics 38(19):4497–4504. https://doi.org/10.1093/bioinformatics/btac575

    Article  CAS  PubMed  Google Scholar 

  7. Zhang P, Zhang H, Wu H (2022) Ipro-wael: a comprehensive and robust framework for identifying promoters in multiple species. Nucleic Acids Res 50(18):10278–10289. https://doi.org/10.1093/nar/gkac824

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Lv Y, Wang S, Meng F et al (2015) Identifying novel associations between small molecules and Mirnas based on integrated molecular networks. Bioinformatics 31(22):3638–3644. https://doi.org/10.1093/bioinformatics/btv417

    Article  CAS  PubMed  Google Scholar 

  9. Jarrett D, Yoon J, van der Schaar M (2019) Dynamic prediction in clinical survival analysis using temporal convolutional networks. IEEE J Biomed Health 24(2):424–436. https://doi.org/10.1109/JBHI.2019.2929264

    Article  Google Scholar 

  10. Ishwaran H, Kogalur UB (2007) Random survival forests for R. Ann Appl Stat 7(2):25–31. https://doi.org/10.1214/08-AOAS169

    Article  Google Scholar 

  11. Ishwaran H, Gerds TA, Kogalur UB et al (2014) Random survival forests for competing risks. Biostatistics 15(4):757–773. https://doi.org/10.1093/biostatistics/kxu010

    Article  PubMed  PubMed Central  Google Scholar 

  12. Fouodo CJ, König IR, Weihs C et al (2018) Support vector machines for survival analysis with R. R J. https://doi.org/10.32614/RJ-2018-005

    Article  Google Scholar 

  13. Klein JP, Moeschberger ML et al (2003) Survival analysis: techniques for censored and truncated data. Technometrics. https://doi.org/10.1198/jasa.2004.s348

    Article  Google Scholar 

  14. Yu D, Deng L (2010) Deep learning and its applications to signal and information processing [exploratory dsp]. IEEE Signal Process Mag 28(1):145–154. https://doi.org/10.1109/MSP.2010.939038

    Article  ADS  Google Scholar 

  15. Peng L, Wang F, Wang Z et al (2022) Cell–cell communication inference and analysis in the tumour microenvironments from single-cell transcriptomics: data resources and computational strategies. Briefings Bioinf 23(4):bbac234. https://doi.org/10.1093/bib/bbac234

    Article  CAS  Google Scholar 

  16. Wang Y, Lian B, Zhang H et al (2023) A multi-view latent variable model reveals cellular heterogeneity in complex tissues for paired multimodal single-cell data. Bioinformatics 39(1):btad005. https://doi.org/10.1093/bioinformatics/btad005

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Faraggi D, Simon R (1995) A neural network model for survival data. Stat Med 14(1):73–82. https://doi.org/10.1002/sim.4780140108

    Article  CAS  PubMed  Google Scholar 

  18. Sargent DJ (2001) Comparison of artificial neural networks with other statistical approaches: results from medical data sets. Support Care Cancer 91(S8):1636–1642. https://doi.org/10.1002/1097-014

    Article  CAS  Google Scholar 

  19. Xiang A, Lapuerta P, Ryutov A et al (2000) Comparison of the performance of neural network methods and cox regression for censored survival data. Comput Stat Data Anal 34(2):243–257. https://doi.org/10.1002/S0167947399000985

    Article  Google Scholar 

  20. Mariani L, Coradini D, Biganzoli E et al (1997) Prognostic factors for metachronous contralateral breast cancer: a comparison of the linear cox regression model and its artificial neural network extension. Breast Cancer Res Treat 44:167–178. https://doi.org/10.1023/A:1005765403093

    Article  CAS  PubMed  Google Scholar 

  21. Katzman JL, Shaham U, Cloninger A et al (2018) Deepsurv: personalized treatment recommender system using a cox proportional hazards deep neural network. MC Med Res Methodol 18(1):1–12. https://doi.org/10.1186/s12874-018-0482-1

    Article  Google Scholar 

  22. Li Y, Wang L, Wang J, et al (2016) Transfer learning for survival analysis via efficient l2, 1-norm regularized cox regression. In: ICDM IEEE, pp 231–240, https://doi.org/10.1109/ICDM.2016.0034

  23. Goldberg RJ, Gore JM, Alpert JS et al (1986) Recent changes in attack and survival rates of acute myocardial infarction (1975 through 1981): the Worcester heart attack study. JAMA 255(20):2774–2779. https://doi.org/10.1001/jama.1986.03370200076031

    Article  CAS  PubMed  Google Scholar 

  24. Knaus WA, Harrell FE, Lynn J et al (1995) The support prognostic model: objective estimates of survival for seriously ill hospitalized adults. Ann Intern Med 122(3):191–203. https://doi.org/10.7326/0003-4819-122-3-199502010-00007

    Article  CAS  PubMed  Google Scholar 

  25. Curtis C, Shah SP, Chin SF et al (2012) The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature 486(7403):346–352. https://doi.org/10.1038/nature10983

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Foekens JA, Peters HA, Look MP et al (2000) The urokinase system of plasminogen activation and prognosis in 2780 breast cancer patients. Cancer Res 60(3):636–643 (60/3/636/507065)

    CAS  PubMed  Google Scholar 

  27. Schumacher M, Bastert G, Bojar H et al (1994) Randomized 2 x 2 trial evaluating hormonal treatment and the duration of chemotherapy in node-positive breast cancer patients. german breast cancer study group. J Clin Oncol 12(10):2086–2093. https://doi.org/10.1200/jco.1994.12.10.2086

    Article  CAS  PubMed  Google Scholar 

  28. Wang P, Li Y, Reddy CK (2019) Machine learning for survival analysis: a survey. ACM Comput Surv 51(6):1–36. https://doi.org/10.1145/3214306

    Article  Google Scholar 

  29. Kvamme H, Borgan Ø, Scheel I (2019) Time-to-event prediction with neural networks and cox regression. Arxiv:1907.00825

  30. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, pmlr, pp 448–456, arxiv:1502.03167

  31. Harrell FE Jr, Lee KL, Califf RM et al (1984) Regression modelling strategies for improved prognostic prediction. Stat Med 3(2):143–152. https://doi.org/10.1002/sim.4780030207

    Article  PubMed  Google Scholar 

  32. Harrell FE, Califf RM, Pryor DB et al (1982) Evaluating the yield of medical tests. JAMA 247(18):2543–2546. https://doi.org/10.1001/jama.1982.03320430047030

    Article  PubMed  Google Scholar 

  33. Rufibach K (2010) Use of brier score to assess binary predictions. J Clin Epidemiol 63(8):938–939. https://doi.org/10.1016/j.jclinepi.2009.11.009

    Article  MathSciNet  PubMed  Google Scholar 

  34. Graf E, Schmoor C, Sauerbrei W et al (1999) Assessment and comparison of prognostic classification schemes for survival data. Stat Med 18(17–18):2529–2545. https://doi.org/10.1002/(SICI)1097-0258(19990915/30)18:17/18<2529::AID-SIM274>3.0.CO;2-5

    Article  CAS  PubMed  Google Scholar 

  35. Chirag N, Steve Y, Negar R, et al (2021) Deep cox mixtures for survival regression. ArXiv:2101.06536. https://api.semanticscholar.org/CorpusID:231632302

  36. BingZhong J, Tao Z, Zixian W et al (2019) A deep survival analysis method based on ranking. Artif Intell Med 98:1–9. https://doi.org/10.1093/bioinformatics/btad005

    Article  CAS  Google Scholar 

  37. Lee C, Zame W, Yoon J, et al (2018) Deephit: A deep learning approach to survival analysis with competing risks. In: AAAI Conf Artif Intell, https://doi.org/10.1609/aaai.v32i1.11842

  38. Nakagawa S, Cuthill IC (2007) Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol Rev 82(4):591–605. https://doi.org/10.1111/j.1469-185X.2007.00027.x

    Article  PubMed  Google Scholar 

  39. Kaplan EL, Meier P (1958) Nonparametric estimation from incomplete observations. J Am Stat Assoc 53(282):457–481. https://doi.org/10.1080/01621459.1958.10501452

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors would like to thank members of the group for their valuable discussions and comments. The scientific calculations in this paper have been done on the HPC Cloud Platform of Shandong University.

Funding

This work is supported by the National Key Research and Development Program (Grant No. 2021YFF0704103) and the National Natural Science Foundation of China (Grant Nos. 62272278 & 61972322). The funders did not play any role in the design of the study, the collection, analysis, and interpretation of data, or the writing of the manuscript.

Author information

Authors and Affiliations

  1. School of Software, Shandong University, Jinan, 250101, China

    Yuchen Wang, Lizhen Cui & Hao Wu

  2. Department of Pediatric Surgery, Heze Municipal Hospital, Heze, 274000, China

    Xianchun Kong

  3. School of Mathematics, Shandong University, Jinan, 250100, China

    Xiao Bi

  4. School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, 400065, China

    Hong Yu

Authors
  1. Yuchen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xianchun Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao Bi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lizhen Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hong Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Hao Wu, Lizhen Cui, Hong Yu, and Yuchen Wang conceived the entire experiment, Yuchen Wang implemented the entire experiment, and Xianchun Kong and Xiao Bi assisted in analyzing the data. The paper writing is mainly completed by Yuchen Wang and reviewed by Hao Wu.

Corresponding author

Correspondence to Hao Wu.

Ethics declarations

Conflict of Interest

None declared.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Kong, X., Bi, X. et al. ResDeepSurv: A Survival Model for Deep Neural Networks Based on Residual Blocks and Self-attention Mechanism. Interdiscip Sci Comput Life Sci (2024). https://doi.org/10.1007/s12539-024-00617-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12539-024-00617-y

Keywords

Search

Navigation