Abstract
Immunotherapy has great potential in the treatment of Non-Small-Cell Lung Cancer (NSCLC). The treatment decision for patients is based on the pathologist’s analysis of NSCLC biopsy images. Using deep learning (DL) methods to automatically segment and classify tissues enable quantitative analysis of biopsy images. However, distinguishing between positive tumor tissues and immune tissues remains challenging due to the similarity between these two types of tissues. In this paper, we present a two-branch convolutional neural network (TBNet) combining segmentation network and patch-based classification network. The segmentation branch feeds additional information to the classification branch to improve the performance of patch classification. Then the classification results are fed back to the segmentation branch to obtain segmented tissue regions classified as positive tumor or immune. The experimental results show that the proposed method improves the classification accuracy by an average of 4.3% over a single classification model and achieves 0.864 and 0.907 dice coefficient of positive tumor tissue region and immune tissue region in segmentation task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
American Cancer Society. https://www.cancer.org/cancer/lung-cancer.html
Alom, M.Z., Yakopcic, C., Shamima, M.: Histopathological images with inception recurrent residual convolutional neural network. J. Digit. Imaging 32(4), 605–617 (2019). https://doi.org/10.1007/s10278-019-00182-7
Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017). https://doi.org/10.1109/TPAMI.2016.2644615
Coudray, N., Ocampo, P.S., Sakellaropoulos, T., Narula, N., Snuderl, M.: Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24(10), 1559–1567 (2018). https://doi.org/10.1038/s41591-018-0177-5
Deng, J., Dong, W., Socher, R., Li, L., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 248–255 (2009). https://doi.org/10.1109/CVPR.2009.5206848
Gertych, A., et al.: Convolutional neural networks can accurately distinguish four histologic growth patterns of lung adenocarcinoma in digital slides. Sci. Rep. 9(1), 1483 (2019). https://doi.org/10.1038/s41598-018-37638-9
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
Kapil, A., et al.: Deep semi supervised generative learning for automated tumor proportion scoring on NSCLC tissue needle biopsies. Sci. Rep. 8(1), 17343 (2018). https://doi.org/10.1038/s41598-018-35501-5
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2017)
Lerousseau, M., et al.: Weakly supervised multiple instance learning histopathological tumor segmentation. In: Martel, A.L., et al. (eds.) MICCAI 2020, Part V. LNCS, vol. 12265, pp. 470–479. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_45
Li, W., Manivannan, S., Akbar, S., Zhang, J., Trucco, E., McKenna, S.J.: Gland segmentation in colon histology images using hand-crafted features and convolutional neural networks. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 1405–1408 (2016). https://doi.org/10.1109/ISBI.2016.7493530
Lin, M., Chen, Q., Yan, S.: Network in network (2014)
Lin, T., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. IEEE Trans. Pattern Anal. Mach. Intell. 42(2), 318–327 (2020). https://doi.org/10.1109/TPAMI.2018.2858826
Litjens, G., Sánchez, C., Timofeeva, N., Hermsen, M.: Nagtegaal: deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci. Rep. 6(1), 26286 (2016)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440 (2015). https://doi.org/10.1109/CVPR.2015.7298965
Minaee, S., Boykov, Y., Porikli, F., Plaza, A., Kehtarnavaz, N., Terzopoulos, D.: Image segmentation using deep learning: a survey (2020)
Qaiser, T., Tsang, Y.-W., Epstein, D., Rajpoot, N.: Tumor segmentation in whole slide images using persistent homology and deep convolutional features. In: Valdés Hernández, M., González-Castro, V. (eds.) MIUA 2017. CCIS, vol. 723, pp. 320–329. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60964-5_28
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015, Part III. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Shen, H., et al.: Deep active learning for breast cancer segmentation on immunohistochemistry images. In: Martel, A.L., et al. (eds.) MICCAI 2020, Part V. LNCS, vol. 12265, pp. 509–518. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_49
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2015)
Sirinukunwattana, K., Raza, S.E.A., Tsang, Y., Snead, D.R.J., Cree, I.A., Rajpoot, N.M.: Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans. Med. Imaging 35(5), 1196–1206 (2016). https://doi.org/10.1109/TMI.2016.2525803
Szegedy, C., et al.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015). https://doi.org/10.1109/CVPR.2015.7298594
Takahama, S., et al.: Multi-stage pathological image classification using semantic segmentation. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10701–10710 (2019). https://doi.org/10.1109/ICCV.2019.01080
Udall, M., et al.: PD-L1 diagnostic tests: a systematic literature review of scoring algorithms and test-validation metrics. Diagn. Pathol. 13(1), 12 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Gao, B., Ding, G., Fang, K., Chen, P. (2021). A Two-Branch Neural Network for Non-Small-Cell Lung Cancer Classification and Segmentation. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2021. ICANN 2021. Lecture Notes in Computer Science(), vol 12893. Springer, Cham. https://doi.org/10.1007/978-3-030-86365-4_53
Download citation
DOI: https://doi.org/10.1007/978-3-030-86365-4_53
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86364-7
Online ISBN: 978-3-030-86365-4
eBook Packages: Computer ScienceComputer Science (R0)