Variational Networks for Joint Image Reconstruction and Classification of Tumor Immune Cell Interactions in Melanoma Tissue Sections
Immunotherapy is currently revolutionizing the treatment of cancer. Detailed analyses of tumor immune cell interaction in the tumor microenvironment will facilitate an accurate prediction of a patient’s clinical response. The automatic and reliable pre-screening of histological tissue sections for tumor infiltrating immune cells (TILs) will support the development of TIL-based predictive biomarkers for checkpoint immunotherapy. In this paper, a learning approach for image classification is presented, which allows various pattern inquires for different types of tissue section images. The underlying trainable reaction diffusion model combines classification and denoising. The model is trained using a stochastic generation of training data. The effectiveness of this approach is demonstrated for immunofluorescent and for Hematoxylin and Eosin (H&E) stained melanoma section images. A particular focus is on the classification of TILs in the proximity to melanoma cells in an experimental melanoma mouse model and in human melanoma. This new learning approach for images of melanoma tissue sections will refine the strategy for the practical clinical application of biomarker research.
Unable to display preview. Download preview PDF.
- 1.Tumeh PC, Harview CL, Yearley JH, et al. PD-1 blockade induces responses by inhibiting adaptive immune resistance. Nature. 2014;515:568–571.Google Scholar
- 2.Jacquelot N, Roberti MP, Enot DP, et al. Predictors of responses to immune checkpoint blockade in advanced melanoma. Nat Commun. 2017;8.Google Scholar
- 3.Sirinukunwattana K, Raza SEA, Tsang YW, et al. Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images. IEEE Trans Med Imaging. 2016;35(5):1196–1206.Google Scholar
- 4.Xu J, Luo X, Wang G, et al. A Deep Convolutional Neural Network for segmenting and classifying epithelial and stromal regions in histopathological images. Neurocomputing. 2016;191:214–223.Google Scholar
- 5.Janowczyk A, Madabhushi A. Deep Learning for Digital Pathology Image Analysis: A Comprehensive Tutorial with Selected Use Cases. J Pathol Inform. 2016;7.Google Scholar
- 6.Chen Y, Pock T. Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. IEEE Trans Pattern Anal Mach Intell. 2017;39(6):1256–1272.Google Scholar
- 7.Kobler E, Klatzer T, Hammernik K, et al. Variational Networks: Connecting Variational Methods and Deep Learning. In: Ger Pattern Recognit Conf; 2017. p. 281–293.Google Scholar
- 8.Kingma DP, Ba JL. Adam: AMethod for Stochastic Optimization. In: International Conference on Learning Representations; 2015.Google Scholar
- 9.Landsberg J, Kohlmeyer J, Renn M, et al. Melanomas resist T-cell therapy through inflammation-induced reversible dedifferentiation. Nature. 2012;490:412–416.Google Scholar