Abstract
Recent image-based survival models rely on discriminative patch labeling, which are both time consuming and infeasible to extend to large scale cancer datasets. Different from the existing works on learning using key patches or clusters from WSIs, we take advantages of a deep multiple instance learning to encode all possible patterns from WSIs and consider the joint effects from different patterns for clinical outcomes prediction. We evaluate our model in its ability to predict patients’ survival risks across the Lung and Brain tumors from two large whole slide pathological images datasets. The proposed framework can improve the prediction performances compared with existing state-of-the-arts survival analysis approaches. Results also demonstrate the effectiveness of the proposed method as a recommender system to provide personalized recommendations based on an individual’s calculated risk.
This work was partially supported by US National Science Foundation IIS-1718853 and the NSF CAREER grant IIS-1553687.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bychkov, D., et al.: Deep learning based tissue analysis predicts outcome in colorectal cancer. Sci. Rep. 8(1), 3395 (2018)
Carpenter, A.E., et al.: CellProfiler: image analysis software for identifying and quantifying cell phenotypes. Genome Biol. 7(10), R100 (2006)
Ishwaran, H., Gerds, T.A., Kogalur, U.B., Moore, R.D., Gange, S.J., Lau, B.M.: Random survival forests for competing risks. Biostatistics 15(4), 757–773 (2014)
Lee, E.T., Wang, J.: Statistical Methods for Survival Data Analysis, vol. 476. Wiley, Hoboken (2003)
Li, R., Yao, J., Zhu, X., Li, Y., Huang, J.: Graph CNN for survival analysis on whole slide pathological images. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 174–182. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_20
Li, Y., Wang, J., Ye, J., Reddy, C.K.: A multi-task learning formulation for survival analysis. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2016 (2016)
Reddy, C.K., Li, Y.: A review of clinical prediction models. In: Healthcare Data Analytics, pp. 343–378. Chapman and Hall/CRC (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Steck, H., Krishnapuram, B., Dehing-Oberije, C., Lambin, P., Raykar, V.C.: On ranking in survival analysis: bounds on the concordance index. In: Advances in Neural Information Processing Systems, pp. 1209–1216 (2008)
Tang, B., Li, A., Li, B., Wang, M.: CapSurv: capsule network for survival analysis with whole slide pathological images. IEEE Access 7, 26022–26030 (2019)
Tibshirani, R., et al.: The lasso method for variable selection in the cox model. Stat. Med. 16(4), 385–395 (1997)
Wang, H., Xing, F., Su, H., Stromberg, A., Yang, L.: Novel image markers for non-small cell lung cancer classification and survival prediction. BMC Bioinform. 15(1), 310 (2014). http://www.biomedcentral.com/1471-2105/15/310
Wang, S., Yao, J., Xu, Z., Huang, J.: Subtype cell detection with an accelerated deep convolution neural network. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 640–648. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_74
Yang, H., Zhou, J.T., Cai, J., Ong, Y.S.: MIML-FCN+: multi-instance multi-label learning via fully convolutional networks with privileged information. In: CVPR, pp. 1577–1585 (2017)
Yang, Y., Zou, H.: A cocktail algorithm for solving the elastic net penalized cox’s regression in high dimensions. Stat. Interface 6(2), 167–173 (2012)
Yao, J., Wang, S., Zhu, X., Huang, J.: Imaging biomarker discovery for lung cancer survival prediction. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 649–657. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_75
Yao, J., Zhu, X., Zhu, F., Huang, J.: Deep correlational learning for survival prediction from multi-modality data. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 406–414. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66185-8_46
Yu, K.H., et al.: Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features. Nat. Commun. 7, 12474 (2016)
Zhu, X., Yao, J., Huang, J.: Deep convolutional neural network for survival analysis with pathological images. In: 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 544–547. IEEE (2016)
Zhu, X., Yao, J., Zhu, F., Huang, J.: WSISA: making survival prediction from whole slide histopathological images. In: CVPR, pp. 7234–7242 (2017)
Acknowledgments
The authors would like to thank the National Cancer Institute for access to NCI’s data collected by the National Lung Screening Trial. The statements contained herein are solely of the authors and do not represent or imply concurrence or endorsement by NCI.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Yao, J., Zhu, X., Huang, J. (2019). Deep Multi-instance Learning for Survival Prediction from Whole Slide Images. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11764. Springer, Cham. https://doi.org/10.1007/978-3-030-32239-7_55
Download citation
DOI: https://doi.org/10.1007/978-3-030-32239-7_55
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32238-0
Online ISBN: 978-3-030-32239-7
eBook Packages: Computer ScienceComputer Science (R0)
