Improving Deep Lesion Detection Using 3D Contextual and Spatial Attention

  • Qingyi TaoEmail author
  • Zongyuan Ge
  • Jianfei Cai
  • Jianxiong Yin
  • Simon See
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11769)


Lesion detection from computed tomography (CT) scans is challenging compared to natural object detection because of two major reasons: small lesion size and small inter-class variation. Firstly, the lesions usually only occupy a small region in the CT image. The feature of such small region may not be able to provide sufficient information due to its limited spatial feature resolution. Secondly, in CT scans, the lesions are often indistinguishable from the background since the lesion and non-lesion areas may have very similar appearances. To tackle both problems, we need to enrich the feature representation and improve the feature discriminativeness. Therefore, we introduce a dual-attention mechanism to the 3D contextual lesion detection framework, including the cross-slice contextual attention to selectively aggregate the information from different slices through a soft re-sampling process. Moreover, we propose intra-slice spatial attention to focus the feature learning in the most prominent regions. Our method can be easily trained end-to-end without adding heavy overhead on the base detection network. We use DeepLesion dataset and train a universal lesion detector to detect all kinds of lesions such as liver tumors, lung nodules, and so on. The results show that our model can significantly boost the results of the baseline lesion detector (with 3D contextual information) but using much fewer slices.

Supplementary material

490281_1_En_21_MOESM1_ESM.pdf (313 kb)
Supplementary material 1 (pdf 313 KB)


  1. 1.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: NIPS, pp. 91–99 (2015)Google Scholar
  2. 2.
    Dai, J., Li, Y., He, K., Sun, J.: R-FCN: object detection via region-based fully convolutional networks. In: NIPS, pp. 379–387 (2016)Google Scholar
  3. 3.
    Lam, C., Caroline, Y., Huang, L., Rubin, D.: Retinal lesion detection with deep learning using image patches. Investig. Ophthalmol. Vis. Sci. 59(1), 590–596 (2018)CrossRefGoogle Scholar
  4. 4.
    Li, Z., et al.: Thoracic disease identification and localization with limited supervision. In: IEEE CVPR, pp. 8290–8299 (2018)Google Scholar
  5. 5.
    Liao, F., Liang, M., Li, Z., Hu, X., Song, S.: Evaluate the malignancy of pulmonary nodules using the 3-D deep leaky noisy-or network. IEEE Trans. Neural Netw. Learn. Syst. (2019) Google Scholar
  6. 6.
    Dou, Q., Chen, H., Lequan, Y., Qin, J., Heng, P.-A.: Multilevel contextual 3-D CNNs for false positive reduction in pulmonary nodule detection. IEEE Trans. Biomed. Eng. 64(7), 1558–1567 (2017)CrossRefGoogle Scholar
  7. 7.
    Yan, K., Bagheri, M., Summers, R.M.: 3D context enhanced region-based convolutional neural network for end-to-end lesion detection. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 511–519. Springer, Cham (2018). Scholar
  8. 8.
    Wang, F., et al.: Residual attention network for image classification. In: IEEE CVPR, pp. 3156–3164 (2017)Google Scholar
  9. 9.
    Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: CBAM: convolutional block attention module. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 3–19. Springer, Cham (2018). Scholar
  10. 10.
    Yan, K., et al.: Deep lesion graphs in the wild: relationship learning and organization of significant radiology image findings in a diverse large-scale lesion database. In: IEEE CVPR, pp. 9261–9270 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Qingyi Tao
    • 1
    • 2
    Email author
  • Zongyuan Ge
    • 1
    • 3
  • Jianfei Cai
    • 2
  • Jianxiong Yin
    • 1
  • Simon See
    • 1
  1. 1.NVIDIA AI Technology CenterSanta ClaraUSA
  2. 2.Nanyang Technological UniversitySingaporeSingapore
  3. 3.Monash UniversityMelbourneAustralia

Personalised recommendations