Skip to main content

Advertisement

Log in

Cross-domain learning for pulmonary nodule detection using Gestalt principle of similarity

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Transfer learning is a trending concept in computer vision that is based on the transfer of knowledge between the source and target domains. In the distant domain problem addressed in this paper, the source and target domains are totally unrelated but have similar visual structures, thereby infusing explainability in transfer learning. We specifically focus on the pulmonary nodule detection problem in which the task is to distinguish the image patches that contain lung nodules from those that do not. The Gestalt principle of similarity states that the human mind tends to club visually similar structures together based on some object attributes such as shape and color. Though some structural differences may exist, these are imperceptible to the human eye since it focusses on “what it wants to see.” This is the central idea behind our work that trains a deep convolutional neural network on commonly found natural scene images containing visual structures similar in appearance to cropped images of pulmonary nodules from computed tomography (CT) scans for the purpose of cancer diagnosis. Our transfer learning module comprises a deep convolutional autoencoder (CAE) that is pre-trained on a source domain comprising of a small and selective subset of only two objects: flowers and rivers that are selected by voting by human annotators to visually correlate with images of lung nodules and non-nodules, respectively. Our work thus presents a mechanism to make the learning process both human-interactive and explainable. Deep tuning our network on images from the benchmark Lung Image Database Consortium and Infectious Disease Research Institute (LIDC/IDRI) database yields higher classification scores as compared to the state-of-the-art. Cost-sensitive learning or data augmentation can be additionally used to further improve the performance of the proposed model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are available in the Lung Image Database Consortium and Infectious Disease Research Institute (LIDC/IDRI) database repository [https://wiki.cancerimagingarchive.net/display/Public/LIDC-IDRI].

Notes

  1. https://drive.google.com/drive/folders/1Uk3VtwWBwjspiIhLOot0P7fAiCeodV_Y?usp=sharing

  2. https://drive.google.com/file/d/1Ew0qGaBTwfQJxyOn-W7F0JbcTeGzHjWf/view?usp=sharing

  3. https://drive.google.com/drive/folders/1Uk3VtwWBwjspiIhLOot0P7fAiCeodV_Y?usp=sharing

  4. https://www.kaggle.com/datasets/alxmamaev/flowers-recognition

  5. https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/8FDC7P

  6. https://github.com/sebasusan/Cross-Domain-Learning-for-Pulmonary-Nodule-Detection

References

  • Antropova N, Huynh BQ, Giger ML (2017) A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets. Med Phys 44(10):5162–5171

    Article  Google Scholar 

  • Armato SG III, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP, Clarke LP (2011) The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys 38(2):915–931

    Article  Google Scholar 

  • Bar Y, Diamant I, Wolf L, Greenspan H (2015) Deep learning with non-medical training used for chest pathology identification. In: Medical imaging 2015: computer-aided diagnosis (vol 9414, p 94140V). International Society for Optics and Photonics

  • Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) Greedy layer-wise training of deep networks. In: Advances in neural information processing systems (pp 153–160)

  • Bishop CM (1995) Neural networks for pattern recognition. Oxford University Press, Oxford

    MATH  Google Scholar 

  • Blitzer J, Crammer K, Kulesza A, Pereira F, Wortman J (2008) Learning bounds for domain adaptation

  • Cao C, Cui Z, Wang L, Wang J, Cao Z, Yang J (2021) Cost-sensitive awareness-based SAR automatic target recognition for imbalanced data. IEEE Trans Geosci Remote Sens 60:1–16

    Google Scholar 

  • Caruana R (1997) Multitask Learn Mach Learn 28(1):41–75

    Article  MathSciNet  Google Scholar 

  • Fang Y, Zhang X, Yuan F, Imamoglu N, Liu H (2019) Video saliency detection by gestalt theory. Pattern Recogn 96:106987

    Article  Google Scholar 

  • Fernández A, García S, Galar M, Prati RC, Krawczyk B, Herrera F, Herrera F (2018) Cost-sensitive learning. Learning from imbalanced data sets, pp 63–78

  • Gogineni AK, Kishore R, Raj P, Naik S, Sahu KK (2020) Unsupervised Clustering algorithm as region of interest proposals for cancer detection using CNN. In: Computational vision and bio-inspired computing: ICCVBIC 2019 (pp 1386–1396). Springer International Publishing

  • Griffin G, Holub A, Perona P (2007) The caltech-256. Caltech Technical Report, 1

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 770–778)

  • Holzinger A, Goebel R, Fong R, Moon T, Müller KR, Samek W (2022) xxAI-beyond explainable artificial intelligence. In: xxAI-Beyond explainable ai: international workshop, held in conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and extended papers (pp. 3–10). Springer International Publishing, Cham

  • Huang Bo, Juaneda C, Sénécal S, Léger P-M (2021) “Now You See Me”: the attention-grabbing effect of product similarity and proximity in online shopping. J Interact Mark 54:1–10

    Article  Google Scholar 

  • Kaushik A, Susan S (2021) Two-way metric learning with majority and minority subsets for classification of large extremely imbalanced face dataset. Jordanian J Comput Inf Technol (JJCIT) 7(04)

  • Kim HE, Cosa-Linan A, Santhanam N, Jannesari M, Maros ME, Ganslandt T (2022) Transfer learning for medical image classification: a literature review. BMC Med Imaging 22(1):69

    Article  Google Scholar 

  • Krawczyk B, Galar M, Jeleń Ł, Herrera F (2016) Evolutionary undersampling boosting for imbalanced classification of breast cancer malignancy. Appl Soft Comput 38:714–726

    Article  Google Scholar 

  • Lai KD, Nguyen TT, Le TH (2021) Detection of lung nodules on ct images based on the convolutional neural network with attention mechanism. Ann Emerg Technol Comput (AETiC) 5(2):78–89

    Article  Google Scholar 

  • LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature (2015). May; 521 (7553): 436 https://doi.org/10.1038/nature14539

  • Li L, Wang B, Verma M, Nakashima Y, Kawasaki R, Nagahara H (2021) SCOUTER: Slot attention-based classifier for explainable image recognition. In: Proceedings of the IEEE/CVF international conference on computer vision (pp. 1046–1055)

  • Lu J, Behbood V, Hao P, Zuo H, Xue S, Zhang G (2015) Transfer learning using computational intelligence: a survey. Knowl-Based Syst 80:14–23

    Article  Google Scholar 

  • Martinez-Murcia FJ, Ortiz A, Gorriz JM, Ramirez J, Castillo-Barnes D, Salas-Gonzalez D, Segovia F (2018) Deep convolutional autoencoders vs PCA in a highly-unbalanced Parkinson’s disease dataset: a DaTSCAN study. In: The 13th international conference on soft computing models in industrial and environmental applications (pp 47–56). Springer, Cham

  • Masci J, Meier U, Cireşan D, Schmidhuber J (2011) Stacked convolutional auto-encoders for hierarchical feature extraction. In: International conference on artificial neural networks (pp 52–59). Springer, Berlin

  • Nilsback ME, Zisserman A (2006). A visual vocabulary for flower classification. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06) (vol 2, pp 1447–1454). IEEE

  • Peterson DJ, Berryhill ME (2013) The Gestalt principle of similarity benefits visual working memory. Psychon Bull Rev 20(6):1282–1289

    Article  Google Scholar 

  • Saini M, Susan S (2019). Data augmentation of minority class with transfer learning for classification of imbalanced breast cancer dataset using inception-V3. In: Iberian conference on pattern recognition and image analysis (pp 409–420). Springer, Cham

  • Setio AAA, Traverso A, De Bel T, Berens MSN, Van Den Bogaard C, Cerello P, Chen H et al. (2017) Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge. Med Image Anal 42:1–13.

  • Song Q, Zhao L, Luo X, Dou X (2017) Using deep learning for classification of lung nodules on computed tomography images. J Healthcare Eng 2017

  • Sui Y, Wei Y, Zhao D (2015) Computer-aided lung nodule recognition by SVM classifier based on combination of random undersampling and SMOTE. Comput Math Methods Med 2015

  • Susan S, Sethi D, Arora K (2021) CW-CAE: pulmonary nodule detection from imbalanced dataset using class-weighted convolutional autoencoder. In: International conference on innovative computing and communications (pp 825–833). Springer, Singapore

  • Susan S, Kumar A (2021) The balancing trick: optimized sampling of imbalanced datasets—a brief survey of the recent State of the Art. Eng Rep 3(4):e12298

    Google Scholar 

  • Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016). Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition (pp 2818–2826)

  • Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J (2016) Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Med Imag 35(5):1299–1312

    Article  Google Scholar 

  • Tan B, Zhang Y, Pan S, Yang Q (2017). Distant domain transfer learning. In: Proceedings of the AAAI conference on artificial intelligence (vol 31, no 1)

  • Thrun S, Pratt L (1998) Learning to learn: Introduction and overview. In: Learning to learn (pp 3–17). Springer, Boston, MA

  • Todorovic D (2008) Gestalt Principles Scholarpedia 3(12):5345

    Article  Google Scholar 

  • Wörgötter F, Krüger N, Pugeault N, Calow D, Lappe M, Pauwels K, Johnston A (2004) Early cognitive vision: Using gestalt-laws for task-dependent, active image-processing. Nat Comput 3(3):293–321

  • Yang Y, Newsam S (2010). Bag-of-visual-words and spatial extensions for land-use classification. In: Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems (pp 270–279)

  • Yildirim O, Baloglu UB, Tan RS, Ciaccio EJ, Acharya UR (2019) A new approach for arrhythmia classification using deep coded features and LSTM networks. Comput Methods Prog Biomed 176:121–133

    Article  Google Scholar 

Download references

Funding

The authors declare that no funds, grants or other support were received during the preparation of this manuscript.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to the study conception and design. Material preparation and data collection were performed by DS and KA. All authors contributed to the analysis of results. The manuscript was written by SS. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Seba Susan.

Ethics declarations

Conflict of interest

The three authors are affiliated with Delhi Technological University, Delhi, India.

Ethical approval

The authors have followed the code of ethics in the experiments involving human participation.

Informed consent

The consent of all human participants has been recorded for the dissemination of research.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Susan, S., Sethi, D. & Arora, K. Cross-domain learning for pulmonary nodule detection using Gestalt principle of similarity. Soft Comput (2023). https://doi.org/10.1007/s00500-023-08252-3

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00500-023-08252-3

Keywords

Navigation