Skip to main content
View expanded cover

International Conference on Medical Image Computing and Computer-Assisted Intervention

MICCAI 2020: Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 pp 540–549Cite as

Deep Interactive Learning: An Efficient Labeling Approach for Deep Learning-Based Osteosarcoma Treatment Response Assessment

Part of the Lecture Notes in Computer Science book series (LNIP,volume 12265)

Abstract

Osteosarcoma is the most common malignant primary bone tumor. Standard treatment includes pre-operative chemotherapy followed by surgical resection. The response to treatment as measured by ratio of necrotic tumor area to overall tumor area is a known prognostic factor for overall survival. This assessment is currently done manually by pathologists by looking at glass slides under the microscope which may not be reproducible due to its subjective nature. Convolutional neural networks (CNNs) can be used for automated segmentation of viable and necrotic tumor on osteosarcoma whole slide images. One bottleneck for supervised learning is that large amounts of accurate annotations are required for training which is a time-consuming and expensive process. In this paper, we describe Deep Interactive Learning (DIaL) as an efficient labeling approach for training CNNs. After an initial labeling step is done, annotators only need to correct mislabeled regions from previous segmentation predictions to improve the CNN model until the satisfactory predictions are achieved. Our experiments show that our CNN model trained by only 7 h of annotation using DIaL can successfully estimate ratios of necrosis within expected inter-observer variation rate for non-standardized manual surgical pathology task.

Keywords

  • Computational pathology
  • Interactive learning
  • Osteosarcoma

D.J. Ho and N.P. Agaram—The first two authors contributed equally.

M.R. Hameed and T.J. Fuchs—The last two authors contributed equally.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-59722-1_52
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   119.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-59722-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   159.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.

References

  1. Arunachalam, H.B., et al.: Viable and necrotic tumor assessment from whole slide images of osteosarcoma using machine-learning and deep-learning models. PLoS ONE 14(4), e0210706 (2019)

    CrossRef  Google Scholar 

  2. Bandi, P., et al.: From detection of individual metastases to classification of lymph node status at the patient level: the CAMELYON17 challenge. IEEE Trans. Med. Imaging 38(2), 550–560 (2019)

    CrossRef  Google Scholar 

  3. Bankhead, P., et al.: QuPath: open source software for digital pathology image analysis. Sci. Rep. 7, 16878 (2017)

    CrossRef  Google Scholar 

  4. Berg, S., et al.: ilastik: interactive machine learning for (bio)image analysis. Nat. Methods 16, 1226–1232 (2019)

    CrossRef  Google Scholar 

  5. Bokhorst, J.M., et al.: Learning from sparsely annotated data for semantic segmentation in histopathology images. In: Proceedings of the International Conference on Medical Imaging with Deep Learning, pp. 84–91 (2019)

    Google Scholar 

  6. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    CrossRef  Google Scholar 

  7. Ehteshami Bejnordi, B., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318(22), 2199–2210 (2017)

    CrossRef  Google Scholar 

  8. Fails, J.A., Olsen, D.R.: Interactive machine learning. In: Proceedings of the International Conference on Intelligent User Interfaces, pp. 39–45 (2003)

    Google Scholar 

  9. Fu, C., et al.: Nuclei segmentation of fluorescence microscopy images using convolutional neural networks. In: Proceedings of the IEEE International Symposium on Biomedical Imaging, pp. 704–708 (2017)

    Google Scholar 

  10. Fuchs, T.J., Buhmann, J.M.: Computational pathology: challenges and promises for tissue analysis. Comput. Med. Imaging Graph. 35(7), 515–530 (2011)

    CrossRef  Google Scholar 

  11. Glorot. X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)

    Google Scholar 

  12. Ho, D.J., et al.: Deep multi-magnification networks for multi-class breast cancer image segmentation. arXiv preprint, arXiv:1910.13042 (2019)

  13. Huvos, A.G., Rosen, G., Marcove, R.C.: Primary osteogenic sarcoma: pathologic aspects in 20 patients after treatment with chemotherapy en bloc resection, and prosthetic bone replacement. Arch. Pathol. Lab. Med. 101(1), 14–18 (1977)

    Google Scholar 

  14. Lee, B., Paeng, K.: A robust and effective approach towards accurate metastasis detection and pN-stage classification in breast cancer. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 841–850 (2018)

    Google Scholar 

  15. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

    Google Scholar 

  16. Ottaviani, G., Jaffe, N.: The epidemiology of osteosarcoma. Pediatric and Adolescent Osteosarcoma 152, 3–13 (2009)

    CrossRef  Google Scholar 

  17. Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: Proceedings of the Neural Information Processing Systems, pp. 8024–8035 (2019)

    Google Scholar 

  18. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241 (2015)

    Google Scholar 

  19. Rosen, G., et al.: Preoperative chemotherapy for osteogenic sarcoma: selection of postoperative adjuvant chemotherapy based on the response of the primary tumor to preoperative chemotherapy. Cancer 49(6), 1221–1230 (1982)

    CrossRef  Google Scholar 

  20. Schüffler, P.J., Fuchs, T.J., Ong, C.S., Wild, P., Buhmann, J.M.: TMARKER: a free software toolkit for histopathological cell counting and staining estimation. J. Pathol. Inform. 4(2), S2 (2013)

    Google Scholar 

  21. Sommer, C., Straehle, C., Koethe, U., Hamprecht, F. A.: ilastik: interactive learning and segmentation toolkit. In: Proceedings of the IEEE International Symposium on Biomedical Imaging, pp. 230–233 (2011)

    Google Scholar 

  22. Srinidhi, C. L., Ciga, O., Martel, A. L.: Deep neural network models for computational histopathology: a survey. arXiv preprint, arXiv:1912.12378 (2019)

  23. Viray, H., et al.: A prospective, multi-institutional diagnostic trial to determine pathologist accuracy in estimation of percentage of malignant cells. Arch. Pathol. Lab. Med. 137(11), 1545–1549 (2013)

    CrossRef  Google Scholar 

  24. Wang, D., Khosla, A., Gargeya, R., Irshad, H., Beck, A. H.: Deep learning for identifying metastatic breast cancer. arXiv preprint, arXiv:1606.05718 (2016)

Download references

Author information

Authors and Affiliations

  1. Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, 10065, USA

    David Joon Ho, Narasimhan P. Agaram, Peter J. Schüffler, Chad M. Vanderbilt, Marc-Henri Jean, Meera R. Hameed & Thomas J. Fuchs

  2. Weill Cornell Graduate School for Medical Sciences, New York, NY, 10065, USA

    Thomas J. Fuchs

Authors
  1. David Joon Ho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Narasimhan P. Agaram
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter J. Schüffler
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chad M. Vanderbilt
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marc-Henri Jean
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Meera R. Hameed
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Thomas J. Fuchs
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Joon Ho .

Editor information

Editors and Affiliations

  1. University of Toronto, Toronto, ON, Canada

    Prof. Anne L. Martel

  2. The University of British Columbia, Vancouver, BC, Canada

    Purang Abolmaesumi

  3. University College London, London, UK

    Danail Stoyanov

  4. École Centrale de Nantes, Nantes, France

    Diana Mateus

  5. EURECOM, Biot, France

    Maria A. Zuluaga

  6. Chinese Academy of Sciences, Beijing, China

    S. Kevin Zhou

  7. Sorbonne University, Paris, France

    Daniel Racoceanu

  8. The Hebrew University of Jerusalem, Jerusalem, Israel

    Prof. Leo Joskowicz

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Ho, D.J. et al. (2020). Deep Interactive Learning: An Efficient Labeling Approach for Deep Learning-Based Osteosarcoma Treatment Response Assessment. In: , et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12265. Springer, Cham. https://doi.org/10.1007/978-3-030-59722-1_52

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-59722-1_52

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-59721-4

  • Online ISBN: 978-3-030-59722-1

  • eBook Packages: Computer ScienceComputer Science (R0)