Skip to main content
SpringerLink
Log in

Mitigating the Class Overlap Problem in Discriminative Localization: COVID-19 and Pneumonia Case Study

  • Chapter
  • First Online:
Explainable AI Within the Digital Transformation and Cyber Physical Systems

  • 771 Accesses

Abstract

Early detection and diagnosis of the SARS-CoV-2 virus which causes COVID-19 remains a difficult and time-sensitive process. Though progress has been made in the development and administration of tests, the time required to obtain a confident response can be prohibitive in areas of dense outbreaks. For these reasons, there has been great interest in adopting machine learning solutions to detect COVID-19 in prospective patients and aid clinicians in its localization. However, the adoption of deep learning approaches for these tasks is faced with multiple key challenges. In this work, we present a novel discriminative localization approach based on the computation of Class Activation Maps (CAMs) that addresses the problem of class overlap and imbalance within training data. In particular, we train a Convolutional Neural Network (CNN) on viral pneumonia and COVID-19 labels and compute the difference in scaled activations of the features in the last convolutional layer of the CNN for both classes. In doing so, we exploit the feature similarity between the two diseases for training purposes, thereby mitigating issues arising due to the scarcity of COVID-19 data. In models with overlapping classes, this can yield a neuron co-adaptation problem wherein the model is less confident in the outputs of similar classes, resulting in large CAMs with decreased localization certainty. Increasing the final mapping resolution of the last convolutional layer can reduce the overall area of such CAMs, but does not address class overlap, because CAMs computed for such classes exhibit significant spatial overlap, making it difficult to distinguish which spatial regions contribute the most to a specific target class. Our approach mitigates this by computing directed differences in scaled activations of pneumonia and COVID-19 CAMs, amplifying these differences, and projecting the results on to the original image in order to aid clinicians in the early detection of COVID-19. Specifically, our method clearly delineates spatial regions of an image that the classifier model considers as the most relevant for a given classification, even when standard CAMs for multiple classes within the model exhibit significant spatial overlap due to similarity in feature activations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ai, T., Yang, Z., Hou, H., Zhan, C., Chen, C., Lv, W., Tao, Q., Sun, Z., & Xia, L. (2020). Correlation of chest CT and RT-PCR testing in coronavirus disease 2019 (COVID-19) in China: A report of 1014 cases. Radiology, 296(2), 200642.

    Article  Google Scholar 

  2. Alejo, R., Valdovinos, R. M., García, V., & Pacheco-Sanchez, J. H. (2013). A hybrid method to face class overlap and class imbalance on neural networks and multi-class scenarios. Pattern Recognition Letters, 34(4), 380–388.

    Article  Google Scholar 

  3. Bernheim, A., Mei, X., Huang, M., Yang, Y., Fayad, Z. A., Zhang, N., Diao, K., Lin, B., Zhu, X., Li, K., et al. (2020). Chest CT findings in coronavirus disease-19 (COVID-19): Relationship to duration of infection. Radiology 295(3), 200463.

    Article  Google Scholar 

  4. Bullock, J., Pham, K. H., Lam, C. S. N., Luengo-Oroz, M., et al. (2020). Mapping the landscape of artificial intelligence applications against COVID-19. arXiv preprint arXiv:2003.11336.

    Google Scholar 

  5. Butt, C., Gill, J., Chun, D., & Babu, B. A. (2020). Deep learning system to screen coronavirus disease 2019 pneumonia. Applied Intelligence 1.

    Google Scholar 

  6. Cao, Y., Xu, Z., Feng, J., Jin, C., Han, X., Wu, H., & Shi, H. (2020). Longitudinal assessment of covid-19 using a deep learning–based quantitative CT pipeline: Illustration of two cases. Radiology: Cardiothoracic Imaging, 2(2), e200082.

    Google Scholar 

  7. Chen, J., Wu, L., Zhang, J., Zhang, L., Gong, D., Zhao, Y., Hu, S., Wang, Y., Hu, X., Zheng, B., et al. (2020). Deep learning-based model for detecting 2019 novel coronavirus pneumonia on high-resolution computed tomography: A prospective study. MedRxiv.

    Google Scholar 

  8. Chowdhury, M. E., Rahman, T., Khandakar, A., Mazhar, R., Kadir, M. A., Mahbub, Z. B., Islam, K. R., Khan, M. S., Iqbal, A., Al-Emadi, N., et al. (2020). Can AI help in screening viral and COVID-19 pneumonia? arXiv preprint arXiv:2003.13145.

    Google Scholar 

  9. Cohen, J. P., Morrison, P., & Dao, L. (2020). COVID-19 image data collection. arXiv preprint arXiv:2003.11597.

    Google Scholar 

  10. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009). ImageNet: A large-scale hierarchical image database. In CVPR09.

    Google Scholar 

  11. Erhan, D., Bengio, Y., Courville, A., & Vincent, P. (2009). Visualizing higher-layer features of a deep network.

    Google Scholar 

  12. Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., & Brendel, W. (2019). ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations.

    Google Scholar 

  13. Ghoshal, B., & Tucker, A. (2020). Estimating uncertainty and interpretability in deep learning for coronavirus (COVID-19) detection. arXiv preprint arXiv:2003.10769.

    Google Scholar 

  14. Gozes, O., Frid-Adar, M., Greenspan, H., Browning, P. D., Zhang, H., Ji, W., Bernheim, A., & Siegel, E. (2020). Rapid AI development cycle for the coronavirus (COVID-19) pandemic: Initial results for automated detection & patient monitoring using deep learning CT image analysis. arXiv preprint arXiv:2003.05037.

    Google Scholar 

  15. Huang, L., Han, R., Ai, T., Yu, P., Kang, H., Tao, Q., & Xia, L. (2020). Serial quantitative chest CT assessment of COVID-19: Deep-learning approach. Radiology: Cardiothoracic Imaging, 2(2), e200075.

    Google Scholar 

  16. Hurt, B., Kligerman, S., & Hsiao, A. (2020). Deep learning localization of pneumonia: 2019 coronavirus (COVID-19) outbreak. Journal of Thoracic Imaging, 35(3), W87–W89.

    Article  Google Scholar 

  17. Kanne, J. P. (2020). Chest CT findings in 2019 novel coronavirus (2019-NCOV) infections from Wuhan, China: Key points for the radiologist.

    Google Scholar 

  18. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In F. Pereira, C. J. C. Burges, L. Bottou & K. Q. Weinberger (Eds.), Advances in neural information processing systems (Vol. 25, pp. 1097–1105). Curran Associates.

    Google Scholar 

  19. Li, L., Qin, L., Xu, Z., Yin, Y., Wang, X., Kong, B., Bai, J., Lu, Y., Fang, Z., Song, Q., et al. (2020). Artificial intelligence distinguishes COVID-19 from community acquired pneumonia on chest CT. Radiology, 296(2), E65–E71.

    Article  Google Scholar 

  20. Liang, T., et al. (2020). Handbook of COVID-19 prevention and treatment. The First Affiliated Hospital, Zhejiang University School of Medicine. Compiled According to Clinical Experience (Vol. 68).

    Google Scholar 

  21. Long, Y., Gong, Y., Xiao, Z., & Liu, Q. (2017). Accurate object localization in remote sensing images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 55(5), 2486–2498.

    Article  Google Scholar 

  22. Loshchilov, I., & Hutter, F. (2019). Decoupled weight decay regularization. In International Conference on Learning Representations.

    Google Scholar 

  23. Narin, A., Kaya, C., & Pamuk, Z. (2020). Automatic detection of coronavirus disease (COVID-19) using x-ray images and deep convolutional neural networks. arXiv preprint arXiv:2003.10849.

    Google Scholar 

  24. Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2015). Is object localization for free? Weakly-supervised learning with convolutional neural networks. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 685–694).

    Google Scholar 

  25. Ozturk, T., Talo, M., Yildirim, E. A., Baloglu, U. B., Yildirim, O., & Acharya, U. R. (2020). Automated detection of covid-19 cases using deep neural networks with X-ray images. Computers in Biology and Medicine, 121, 103792.

    Article  Google Scholar 

  26. Qi, X., Jiang, Z., Yu, Q., Shao, C., Zhang, H., Yue, H., Ma, B., Wang, Y., Liu, C., Meng, X., et al. (2020). Machine learning-based CT radiomics model for predicting hospital stay in patients with pneumonia associated with sars-cov-2 infection: A multicenter study. medRxiv.

    Google Scholar 

  27. Rahman, T., Chowdhury, M., & Khandakar, A. (2020). COVID-19 radiology database. Retrieved September, 2020 from https://www.kaggle.com/tawsifurrahman/covid19-radiography-database

  28. Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D., Bagul, A., Langlotz, C., Shpanskaya, K., et al. (2017). CheXNet: Radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv preprint arXiv:1711.05225.

    Google Scholar 

  29. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 234–241). Springer.

    Google Scholar 

  30. Roy, S., Menapace, W., Oei, S., Luijten, B., Fini, E., Saltori, C., Huijben, I., Chennakeshava, N., Mento, F., Sentelli, A., et al. (2020). Deep learning for classification and localization of COVID-19 markers in point-of-care lung ultrasound. IEEE Transactions on Medical Imaging, 39(8), 2676–2687.

    Article  Google Scholar 

  31. Shi, F., Wang, J., Shi, J., Wu, Z., Wang, Q., Tang, Z., He, K., Shi, Y., & Shen, D. (2020). Review of artificial intelligence techniques in imaging data acquisition, segmentation and diagnosis for COVID-19. IEEE Reviews in Biomedical Engineering, 14, 4–15.

    Article  Google Scholar 

  32. Simonyan, K., Vedaldi, A., & Zisserman, A. (2014). Deep inside convolutional networks: Visualising image classification models and saliency maps. CoRR, abs/1312.6034.

    Google Scholar 

  33. van Sloun, R. J., & Demi, L. (2019). Localizing b-lines in lung ultrasonography by weakly supervised deep learning, in-vivo results. IEEE Journal of Biomedical and Health Informatics, 24(4), 957–964.

    Article  Google Scholar 

  34. Verenich, E., Velasquez, A., Khan, N., & Hussain, F. (2020). Improving explainability of image classification in scenarios with class overlap: application to COVID-19 and pneumonia. In Proceedings of the 19th IEEE International Conference on Machine Learning and Applications.

    Google Scholar 

  35. Wang, L., & Wong, A. (2020). COVID-net: A tailored deep convolutional neural network design for detection of COVID-19 cases from chest x-ray images. arXiv preprint arXiv:2003.09871.

    Google Scholar 

  36. Wang, S., Kang, B., Ma, J., Zeng, X., Xiao, M., Guo, J., Cai, M., Yang, J., Li, Y., Meng, X., et al. (2020). A deep learning algorithm using CT images to screen for corona virus disease (COVID-19). MedRxiv.

    Google Scholar 

  37. Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., &Summers, R. M. (2017). Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 3462–3471).

    Google Scholar 

  38. Won, J., Lee, S., Park, M., Kim, T. Y., Park, M. G., Choi, B. Y., Kim, D., Chang, H., Kim, V. N., & Lee, C. J. (2020). Development of a laboratory-safe and low-cost detection protocol for SARS-COV-2 of the coronavirus disease 2019 (COVID-19). Experimental Neurobiology, 29(2), 107.

    Article  Google Scholar 

  39. Xie, X., Zhong, Z., Zhao, W., Zheng, C., Wang, F., & Liu, J. (2020). Chest CT for typical 2019-NCOV pneumonia: Relationship to negative RT-PCR testing. Radiology, 296(2), 200343.

    Article  Google Scholar 

  40. Zhang, J., Xie, Y., Li, Y., Shen, C., & Xia, Y. (2020). COVID-19 screening on chest x-ray images using deep learning based anomaly detection. arXiv preprint arXiv:2003.12338.

    Google Scholar 

  41. Zheng, C., Deng, X., Fu, Q., Zhou, Q., Feng, J., Ma, H., Liu, W., & Wang, X. (2020). Deep learning-based detection for COVID-19 from chest CT using weak label. medRxiv.

    Google Scholar 

  42. Zheng, Z., Yao, Z., Wu, K., and Zheng, J. (2020). The diagnosis of pandemic coronavirus pneumonia: A review of radiology examination and laboratory test. Journal of Clinical Virology, 128, 104396.

    Article  Google Scholar 

  43. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016). Learning deep features for discriminative localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2921–2929).

    Google Scholar 

  44. Zhou, Z., Siddiquee, M. M. R., Tajbakhsh, N., & Liang, J. (2018). UNet+ +: A nested u-net architecture for medical image segmentation. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support (pp. 3–11). Springer.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Clarkson University, Potsdam, NY, USA

    Edward Verenich

  2. Air Force Research Laboratory, Rome, NY, USA

    Edward Verenich

  3. Clarkson University, Potsdam, NY, USA

    M. G. Sarwar Murshed & Faraz Hussain

  4. Department of Computer Science, University of the Punjab, Lahore, Pakistan

    Nazar Khan

  5. Air Force Research Laboratory, Rome, NY, USA

    Alvaro Velasquez

Authors
  1. Edward Verenich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. G. Sarwar Murshed
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nazar Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alvaro Velasquez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Faraz Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Edward Verenich .

Editor information

Editors and Affiliations

  1. Institute Mines-Telecom Lille Douai, Douai, France

    Moamar Sayed-Mouchaweh

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Verenich, E., Murshed, M.G.S., Khan, N., Velasquez, A., Hussain, F. (2021). Mitigating the Class Overlap Problem in Discriminative Localization: COVID-19 and Pneumonia Case Study. In: Sayed-Mouchaweh, M. (eds) Explainable AI Within the Digital Transformation and Cyber Physical Systems. Springer, Cham. https://doi.org/10.1007/978-3-030-76409-8_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-76409-8_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-76408-1

  • Online ISBN: 978-3-030-76409-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation