Deep Multiscale Convolutional Feature Learning for Weakly Supervised Localization of Chest Pathologies in X-ray Images

  • Suman SedaiEmail author
  • Dwarikanath Mahapatra
  • Zongyuan Ge
  • Rajib Chakravorty
  • Rahil Garnavi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11046)


Localization of chest pathologies in chest X-ray images is a challenging task because of their varying sizes and appearances. We propose a novel weakly supervised method to localize chest pathologies using class aware deep multiscale feature learning. Our method leverages intermediate feature maps from CNN layers at different stages of a deep network during the training of a classification model using image level annotations of pathologies. During the training phase, a set of layer relevance weights are learned for each pathology class and the CNN is optimized to perform pathology classification by convex combination of feature maps from both shallow and deep layers using the learned weights. During the test phase, to localize the predicted pathology, the multiscale attention map is obtained by convex combination of class activation maps from each stage using the layer relevance weights learned during the training phase. We have validated our method using 112000 X-ray images and compared with the state-of-the-art localization methods. We experimentally demonstrate that the proposed weakly supervised method can improve the localization performance of small pathologies such as nodule and mass while giving comparable performance for bigger pathologies e.g., Cardiomegaly.


Weakly supervised learning X-ray pathology classification 


  1. 1.
    Avni, U., Greenspan, H., Goldberger, J.: X-ray categorization and spatial localization of chest pathologies. In: Fichtinger, G., Martel, A., Peters, T. (eds.) Proceedings of the MICCAI, pp. 199–206 (2011)Google Scholar
  2. 2.
    Ding, J., Li, A., Hu, Z., Wang, L.: Accurate pulmonary nodule detection in computed tomography images using deep convolutional neural networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) Proceedings of the MICCAI, pp. 559–567 (2017)Google Scholar
  3. 3.
    Girshick, R.: Fast R-CNN. In: IEEE ICCV, pp. 1440–1448 (2015)Google Scholar
  4. 4.
    Huang, G., Liu, Z., v. d. Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: IEEE Conference on CVPR, pp. 2261–2269 (2017)Google Scholar
  5. 5.
    Hwang, S., Kim, H.E.: Self-transfer learning for weakly supervised lesion localization. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) Proceedings of the MICCAI, pp. 239–246 (2016)CrossRefGoogle Scholar
  6. 6.
    Liao, S., Gao, Y., Lian, J., Shen, D.: Sparse patch-based label propagation for accurate prostate localization in CT images. IEEE Trans. Med. Imaging 32(2), 419–434 (2013)CrossRefGoogle Scholar
  7. 7.
    Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Is object localization for free? - weakly-supervised learning with convolutional neural networks. In: IEEE Conference on CVPR, pp. 685–694 (2015)Google Scholar
  8. 8.
    Pesce, E., Ypsilantis, P., Withey, S., Bakewell, R., Goh, V., Montana, G.: Learning to detect chest radiographs containing lung nodules using visual attention networks. CoRR, arXiv:1712.00996 (2017)
  9. 9.
    Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) Proceedings of the MICCAI, pp. 234–241 (2015)Google Scholar
  10. 10.
    Sedai, S., Tennakoon, R., Roy, P., Cao, K., Garnavi, R.: Multi-stage segmentation of the fovea in retinal fundus images using fully convolutional neural networks. In: ISBI, pp. 1083–1086 (2017)Google Scholar
  11. 11.
    Sedai, S., Mahapatra, D., Hewavitharanage, S., Maetschke, S., Garnavi, R.: Semi-supervised segmentation of optic cup in retinal fundus images using variational autoencoder. In: MICCAI, pp. 75–82 (2017)Google Scholar
  12. 12.
    Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. CoRR, arXiv:1705.02315 (2017)
  13. 13.
    Xu, Y., Zhu, J.Y., Chang, E.I.C., Lai, M., Tu, Z.: Weakly supervised histopathology cancer image segmentation and classification. Med. Image Anal. 18(3), 591–604 (2014)CrossRefGoogle Scholar
  14. 14.
    Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: IEEE Conference on CVPR, pp. 2921–2929 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Suman Sedai
    • 1
    Email author
  • Dwarikanath Mahapatra
    • 1
  • Zongyuan Ge
    • 1
  • Rajib Chakravorty
    • 1
  • Rahil Garnavi
    • 1
  1. 1.IBM Research - AustraliaMelbourneAustralia

Personalised recommendations