Skip to main content
SpringerLink
Log in

Local Axial Scale-Attention for Universal Lesion Detection

  • Original Paper
  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Universal lesion detection (ULD) in computed tomography (CT) images is an important and challenging prerequisite for computer-aided diagnosis (CAD) to find abnormal tissue, such as tumors of lymph nodes, liver tumors, and lymphadenopathy. The key challenge is that lesions have a tiny size and high similarity with non-lesions, which can easily lead to high false positives. Specifically , non-lesions are nearby normal anatomy that include the bowel, vasculature, and mesentery, which decrease the conspicuity of small lesions since they are often hard to differentiate. In this study, we present a novel scale-attention module that enhances feature discrimination between lesion and non-lesion regions by utilizing the domain knowledge of radiologists to reduce false positives effectively. Inspired by the domain knowledge that radiologists tend to divide each CT image into multiple areas, then detect lesions in these smaller areas separately, a local axial scale-attention (LASA) module is proposed to re-weight each pixel in a feature map by aggregating local features from multiple scales adaptively. In addition, to keep the same weight, a combination of axial pixels in the height- and width-axes is designed, attached with position embedding. The model can be used in CNNs easily and flexibly. We test our method on the DeepLesion dataset. The sensitivities at 0.5, 1, 2, 4, 8, and 16 false positives (FPs) per image and average sensitivity at [0.5, 1, 2, 4] are calculated to evaluate the accuracy. The sensitivities are 78.30%, 84.96%, 89.86%, 93.14%, 95.36%, and 95.54% at 0.5, 1, 2, 4, 8, and 16 FPs per image; the average sensitivity is 86.56%, outperforming the former methods. The proposed method enhances feature discrimination between lesion and non-lesion regions by adding LASA modules. These encouraging results illustrate the potential advantage of exploiting the domain knowledge for lesion detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data Availability

The DeepLesion dataset is openly available at https://nihcc.box.com/v/DeepLesion.

References

  1. R. Bellotti et al., A completely automated CAD system for mass detection in a large mammographic database, Medical physics 33, 3066–3075 (2006).

    Article  CAS  PubMed  Google Scholar 

  2. R. Bellotti et al., A CAD system for nodule detection in low-dose lung CTs based on region growing and a new active contour model, Medical Physics 34, 4901–4910 (2007).

    Article  CAS  PubMed  Google Scholar 

  3. L. Li, Y. Zheng, L. Zhang, and R. A. Clark, False-positive reduction in CAD mass detection using a competitive classification strategy, Medical physics 28, 250–258 (2001).

    Article  CAS  PubMed  Google Scholar 

  4. H. Yoshida and J. Näppi, CAD in CT colonography without and with oral contrast agents: progress and challenges, Computerized Medical Imaging and Graphics 31, 267–284 (2007).

    Article  PubMed  Google Scholar 

  5. B. Sahiner, A. Pezeshk, L. M. Hadjiiski, X. Wang, K. Drukker, K. H. Cha, R. M. Summers, and M. L. Giger, Deep learning in medical imaging and radiation therapy, Medical physics 46, e1–e36 (2019).

    Article  PubMed  Google Scholar 

  6. D. Gu, G. Liu, and Z. Xue, On the performance of lung nodule detection, segmentation and classification, Computerized Medical Imaging and Graphics 89, 101886 (2021).

    Article  PubMed  Google Scholar 

  7. B. Wang, G. Qi, S. Tang, L. Zhang, L. Deng, and Y. Zhang, Automated pulmonary nodule detection: High sensitivity with few candidates, in International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 759–767, Springer, 2018.

  8. F. Liao, M. Liang, Z. Li, X. Hu, and S. Song, Evaluate the malignancy of pulmonary nodules using the 3-d deep leaky noisy-or network, IEEE transactions on neural networks and learning systems 30, 3484–3495 (2019).

    Article  PubMed  Google Scholar 

  9. S.-g. Lee, J. S. Bae, H. Kim, J. H. Kim, and S. Yoon, Liver lesion detection from weakly-labeled multi-phase CT volumes with a grouped single shot multibox detector, in International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 693–701, Springer, 2018.

  10. T. Liu, J. Liu, Y. Ma, J. He, J. Han, X. Ding, and C.-T. Chen, Spatial feature fusion convolutional network for liver and liver tumor segmentation from CT images, Medical Physics 48, 264–272 (2021).

    Article  PubMed  Google Scholar 

  11. X. Wang, S. Han, Y. Chen, D. Gao, and N. Vasconcelos, Volumetric attention for 3D medical image segmentation and detection, in International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 175–184, Springer, 2019.

  12. D. Bouget, A. Jørgensen, G. Kiss, H. O. Leira, and T. Langø, Semantic segmentation and detection of mediastinal lymph nodes and anatomical structures in CT data for lung cancer staging, International journal of computer assisted radiology and surgery 14, 977–986 (2019).

    PubMed  Google Scholar 

  13. J. Liu, J. Hoffman, J. Zhao, J. Yao, L. Lu, L. Kim, E. B. Turkbey, and R. M. Summers, Mediastinal lymph node detection and station mapping on chest CT using spatial priors and random forest, Medical physics 43, 4362–4374 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  14. K. Yan, M. Bagheri, and R. M. Summers, 3D context enhanced region-based convolutional neural network for end-to-end lesion detection, in International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 511–519, Springer, 2018.

  15. C. Jiang, S. Wang, X. Liang, H. Xu, and N. Xiao, Elixirnet: Relation-aware network architecture adaptation for medical lesion detection, in Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 11093–11100, 2020.

    Article  Google Scholar 

  16. M. Zlocha, Q. Dou, and B. Glocker, Improving RetinaNet for CT lesion detection with dense masks from weak RECIST labels, in International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 402–410, Springer, 2019.

  17. Y.-B. Tang, K. Yan, Y.-X. Tang, J. Liu, J. Xiao, and R. M. Summers, ULDor: a universal lesion detector for CT scans with pseudo masks and hard negative example mining, in 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pages 833–836, IEEE, 2019.

  18. K. Yan, Y. Tang, Y. Peng, V. Sandfort, M. Bagheri, Z. Lu, and R. M. Summers, MULAN: multitask universal lesion analysis network for joint lesion detection, tagging, and segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 194–202, Springer, 2019.

  19. Z. Li, S. Zhang, J. Zhang, K. Huang, Y. Wang, and Y. Yu, MVP-Net: Multi-view FPN with position-aware attention for deep universal lesion detection, in International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 13–21, Springer, 2019.

  20. F. Lyu, B. Yang, A. J. Ma, and P. C. Yuen, A Segmentation-Assisted Model for Universal Lesion Detection with Partial Labels, in International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 117–127, Springer, 2021.

  21. T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, Feature pyramid networks for object detection, in Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2117–2125, 2017.

  22. H. R. Boveiri, R. Khayami, R. Javidan, and A. Mehdizadeh, Medical image registration using deep neural networks: A comprehensive review, Computers & Electrical Engineering 87, 106767 (2020).

    Article  Google Scholar 

  23. H. Wang, Y. Zhu, B. Green, H. Adam, A. Yuille, and L.-C. Chen, Axial-deeplab: Stand-alone axial-attention for panoptic segmentation, in European Conference on Computer Vision, pages 108–126, Springer, 2020.

  24. K. Yan, X. Wang, L. Lu, L. Zhang, A. P. Harrison, M. Bagheri, and R. M. Summers, Deep lesion graphs in the wild: relationship learning and organization of significant radiology image findings in a diverse large-scale lesion database, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9261–9270, 2018.

  25. M. J. Willemink, W. A. Koszek, C. Hardell, J. Wu, D. Fleischmann, H. Harvey, L. R. Folio, R. M. Summers, D. L. Rubin, and M. P. Lungren, Preparing medical imaging data for machine learning, Radiology 295, 4–15 (2020).

    Article  PubMed  Google Scholar 

  26. K. He, G. Gkioxari, P. Dollár, and R. Girshick, Mask r-cnn, in Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017.

  27. G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, Densely connected convolutional networks, in Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017.

  28. X. Wang, R. Girshick, A. Gupta, and K. He, Non-local neural networks, in Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7794–7803, 2018.

  29. P. Ramachandran, N. Parmar, A. Vaswani, I. Bello, A. Levskaya, and J. Shlens, Stand-alone self-attention in vision models, arXiv preprint http://arxiv.org/abs/1906.05909arXiv:1906.05909 (2019).

  30. H. Zhao, J. Jia, and V. Koltun, Exploring self-attention for image recognition, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10076–10085, 2020.

Download references

Acknowledgements

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. School of Electrical and Information Engineering, Tianjin University, Tianjin, 300072, China

    Chuanyu Liu, Yonghong Hou, Pengyu Zhao, Zihui Guo & Haoyuan Zhang

  2. Department of Cardiology, Tianjin Chest Hospital, No. 261 Taierzhuang S Rd, Tianjin, 300350, China

    Yueyang Li & Jia Zhou

Authors
  1. Chuanyu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yonghong Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pengyu Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zihui Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yueyang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Haoyuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jia Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yonghong Hou.

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, C., Hou, Y., Zhao, P. et al. Local Axial Scale-Attention for Universal Lesion Detection. J Digit Imaging 36, 1208–1215 (2023). https://doi.org/10.1007/s10278-022-00748-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-022-00748-y

Keywords

Search

Navigation