Predicting Cancer with a Recurrent Visual Attention Model for Histopathology Images

  • Aïcha BenTaiebEmail author
  • Ghassan Hamarneh
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11071)


Automatically recognizing cancers from multi-gigapixel whole slide histopathology images is one of the challenges facing machine and deep learning based solutions for digital pathology. Currently, most automatic systems for histopathology are not scalable to large images and hence require a patch-based representation; a sub-optimal solution as it results in important additional computational costs but more importantly in the loss of contextual information. We present a novel attention-based model for predicting cancer from histopathology whole slide images. The proposed model is capable of attending to the most discriminative regions of an image by adaptively selecting a limited sequence of locations and only processing the selected areas of tissues. We demonstrate the utility of the proposed model on the slide-based prediction of macro and micro metastases in sentinel lymph nodes of breast cancer patients. We achieve competitive results with state-of-the-art convolutional networks while automatically identifying discriminative areas of tissues.



We thank NVIDIA Corporation for GPU donation and The Natural Sciences and Engineering Research Council of Canada (NSERC) for funding.


  1. 1.
    Agarwalla, A., Shaban, M., Rajpoot, N.M.: Representation-aggregation networks for segmentation of multi-gigapixel histology images. arXiv preprint arXiv:1707.08814 (2017)
  2. 2.
    Bejnordi, B.E., et al.: Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images. MedIA 4(4), 044504 (2017)Google Scholar
  3. 3.
    BenTaieb, A., et al.: A structured latent model for ovarian carcinoma subtyping from histopathology slides. MedIA 39, 194–205 (2017)Google Scholar
  4. 4.
    Brunye, T.T., et al.: Eye movements as an index of pathologist visual expertise: a pilot study. PloS one 9(8), e103447 (2014)CrossRefGoogle Scholar
  5. 5.
    Golden, J.A.: Deep learning algorithms for detection of lymph node metastases from breast cancer: helping artificial intelligence be seen. JAMA 318(22), 2184–2186 (2017)CrossRefGoogle Scholar
  6. 6.
    Komura, D., Ishikawa, S.: Machine learning methods for histopathological image analysis. arXiv preprint arXiv:1709.00786 (2017)
  7. 7.
    Mercan, C., et al.: Multi-instance multi-label learning for multi-class classification of whole slide breast histopathology images. IEEE TMI 37(1), 316–325 (2018)Google Scholar
  8. 8.
    Mnih, V., et al.: Recurrent models of visual attention. In: NIPS, pp. 2204–2212 (2014)Google Scholar
  9. 9.
    Sermanet, P., Frome, A., Real, E.: Attention for fine-grained categorization. arXiv preprint arXiv:1412.7054 (2014)
  10. 10.
    Wang, D., et al.: Deep learning for identifying metastatic breast cancer. arXiv preprint arXiv:1606.05718 (2016)

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Computing ScienceSimon Fraser UniversityBurnabyCanada

Personalised recommendations