Skip to main content

Advertisement

SpringerLink
Log in

An Efficient Content-Based Image Retrieval System for the Diagnosis of Lung Diseases

  • Original Paper
  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

The main problem in content-based image retrieval (CBIR) systems is the semantic gap which needs to be reduced for efficient retrieval. The common imaging signs (CISs) which appear in the patient’s lung CT scan play a significant role in the identification of cancerous lung nodules and many other lung diseases. In this paper, we propose a new combination of descriptors for the effective retrieval of these imaging signs. First, we construct a feature database by combining local ternary pattern (LTP), local phase quantization (LPQ), and discrete wavelet transform. Next, joint mutual information (JMI)–based feature selection is deployed to reduce the redundancy and to select an optimal feature set for CISs retrieval. To this end, similarity measurement is performed by combining visual and semantic information in equal proportion to construct a balanced graph and the shortest path is computed for learning contextual similarity to obtain final similarity between each query and database image. The proposed system is evaluated on a publicly available database of lung CT imaging signs (LISS), and results are retrieved based on visual feature similarity comparison and graph-based similarity comparison. The proposed system achieves a mean average precision (MAP) of 60% and 0.48 AUC of precision-recall (P-R) graph using only visual features similarity comparison. These results further improve on graph-based similarity measure with a MAP of 70% and 0.58 AUC which shows the superiority of our proposed scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Liu X., Ma L., Song L., Zhao Y., Zhao X., Zhou C.: Recognizing common ct imaging signs of lung diseases through a new feature selection method based on fisher criterion and genetic optimization. IEEE journal of biomedical and health informatics 19 (2): 635–647, 2014

    PubMed  Google Scholar 

  2. Han G., Liu X., Han F., Santika I. N. T., Zhao Y., Zhao X., Zhou C.: The liss —a public database of common imaging signs of lung diseases for computer-aided detection and diagnosis research and medical education. IEEE Transactions on Biomedical Engineering 62 (2): 648–656, 2014

    PubMed  Google Scholar 

  3. Rizzo S., Botta F., Raimondi S., Origgi D., Fanciullo C., Morganti A. G., Bellomi M.: Radiomics: the facts and the challenges of image analysis. European radiology experimental 2 (1): 1–8, 2018

    Google Scholar 

  4. Peeken J. C., Bernhofer M., Wiestler B., Goldberg T., Cremers D., Rost B., Wilkens J. J., Combs S. E., Nüsslin F.: Radiomics in radiooncology–challenging the medical physicist. Physica Medica 48: 27–36, 2018

    PubMed  Google Scholar 

  5. Gillies R. J., Kinahan P. E., Hricak H.: Radiomics: images are more than pictures, they are data. Radiology 278 (2): 563–577, 2016

    PubMed  Google Scholar 

  6. Liu Y., Zhang D., Lu G., Ma W. Y.: A survey of content-based image retrieval with high-level semantics. Pattern recognition 40 (1): 262–282, 2007

    Google Scholar 

  7. Dubey S. R., Singh S. K., Singh R. K.: A multi-channel based illumination compensation mechanism for brightness invariant image retrieval. Multimedia Tools and Applications 74 (24): 11,223–11,253, 2015

    Google Scholar 

  8. Rashedi E., Nezamabadi-Pour H., Saryazdi S.: A simultaneous feature adaptation and feature selection method for content-based image retrieval systems. Knowledge-Based Systems 39: 85–94, 2013

    Google Scholar 

  9. ElAlami M. E.: A new matching strategy for content based image retrieval system. Applied Soft Computing 14: 407–418, 2014

    Google Scholar 

  10. Alajlan N., Kamel M. S., Freeman G. H.: Geometry-based image retrieval in binary image databases. IEEE Transactions on Pattern Analysis and Machine Intelligence 30 (6): 1003–1013, 2008

    PubMed  Google Scholar 

  11. Behnam M., Pourghassem H.: Optimal query-based relevance feedback in medical image retrieval using score fusion-based classification. Journal of digital imaging 28 (2): 160–178, 2015

    PubMed  Google Scholar 

  12. Larsen A. B. L., Vestergaard J. S., Larsen R.: Hep-2 cell classification using shape index histograms with donut-shaped spatial pooling. IEEE transactions on medical imaging 33 (7): 1573–1580, 2014

    PubMed  Google Scholar 

  13. Dubey S. R., Singh S. K., Singh R. K.: Local wavelet pattern: a new feature descriptor for image retrieval in medical ct databases. IEEE Transactions on Image Processing 24 (12): 5892–5903, 2015

    PubMed  Google Scholar 

  14. Chun Y. D., Kim N. C., Jang I. H.: Content-based image retrieval using multiresolution color and texture features. IEEE Transactions on multimedia 10 (6): 1073–1084, 2008

    Google Scholar 

  15. Ojala T., Pietikäinen M., Mäenpää T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis & Machine Intelligence 7: 971–987, 2002

    Google Scholar 

  16. Dubey S. R., Singh S. K., Singh R. K.: Local diagonal extrema pattern: a new and efficient feature descriptor for ct image retrieval. IEEE Signal Processing Letters 22 (9): 1215–1219, 2015

    Google Scholar 

  17. Quellec G., Lamard M., Cazuguel G., Cochener B., Roux C.: Wavelet optimization for content-based image retrieval in medical databases. Medical image analysis 14 (2): 227–241, 2010

    CAS  PubMed  Google Scholar 

  18. Liu G. H., Yang J. Y.: Content-based image retrieval using color difference histogram. Pattern recognition 46 (1): 188–198, 2013

    Google Scholar 

  19. Chatzichristofis S. A., Zagoris K., Boutalis Y. S., Papamarkos N.: Accurate image retrieval based on compact composite descriptors and relevance feedback information. International Journal of Pattern Recognition and Artificial Intelligence 24 (02): 207–244, 2010

    Google Scholar 

  20. Guyon I., Elisseeff A.: An introduction to variable and feature selection. Journal of machine learning research 3: 1157–1182, 2003. Mar

    Google Scholar 

  21. Guldogan E., Gabbouj M.: Feature selection for content-based image retrieval. Signal, Image and Video Processing 2 (3): 241–250, 2008

    Google Scholar 

  22. Chun Y. D., Seo S. Y., Kim N. C.: Image retrieval using bdip and bvlc moments. IEEE transactions on circuits and systems for video technology 13 (9): 951–957, 2003 bioinformatics

    Google Scholar 

  23. Saeys Y., Inza I., Larrañaga P.: A review of feature selection techniques in bioinformatics. bioinformatics 23 (19): 2507–2517, 2007

    CAS  PubMed  Google Scholar 

  24. Cho H.C., Hadjiiski L., Sahiner B., Chan H.P., Helvie M., Paramagul C., Nees A.V.: Similarity evaluation in a content-based image retrieval (cbir) cadx system for characterization of breast masses on ultrasound images. Medical physics 38 (4): 1820–1831, 2011

    PubMed  PubMed Central  Google Scholar 

  25. Yue J., Li Z., Liu L., Fu Z.: Content-based image retrieval using color and texture fused features. Mathematical and Computer Modelling 54 (3-4): 1121–1127, 2011

    Google Scholar 

  26. Yu J., Amores J., Sebe N., Radeva P., Tian Q.: Distance learning for similarity estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence 30 (3): 451–462, 2008

    PubMed  Google Scholar 

  27. Sethi I. K., Coman I. L., Stan D.: Mining association rules between low-level image features and high-level concepts.. In: Data mining and knowledge discovery: theory, tools, and technology III, vol. 4384, pp. 279–290. International Society for Optics and Photonics, 2001

  28. Mezaris V., Kompatsiaris I., Strintzis M. G.: An ontology approach to object-based image retrieval.. In: Proceedings 2003 International Conference on Image Processing (Cat. No. 03CH37429), vol. 2, pp. II–511. IEEE, 2003

  29. Vailaya A., Figueiredo M. A., Jain A. K., Zhang H. J.: Image classification for content-based indexing. IEEE transactions on image processing 10 (1): 117–130, 2001

    CAS  PubMed  Google Scholar 

  30. Chen Y., Wang J. Z., Krovetz R.: An unsupervised learning approach to content-based image retrieval.. In: Seventh international symposium on signal processing and its applications, 2003. Proceedings., vol. 1, pp. 197–200. IEEE, 2003

  31. André B., Vercauteren T., Buchner A. M., Wallace M. B., Ayache N.: Learning semantic and visual similarity for endomicroscopy video retrieval. IEEE Transactions on Medical Imaging 31 (6): 1276–1288, 2012

    PubMed  Google Scholar 

  32. Zhu L., Shen J., Xie L., Cheng Z.: Unsupervised visual hashing with semantic assistant for content-based image retrieval. IEEE Transactions on Knowledge and Data Engineering 29 (2): 472–486, 2016

    Google Scholar 

  33. Pedronette D. C. G.: Torres, R.d.S.: Exploiting contextual information for image re-ranking and rank aggregation. International Journal of Multimedia Information Retrieval 1 (2): 115–128, 2012

    Google Scholar 

  34. Perronnin F., Liu Y., Renders J. M.: A family of contextual measures of similarity between distributions with application to image retrieval.. In: 2009 IEEE Conference on computer vision and pattern recognition, pp. 2358–2365. IEEE, 2009

  35. Schwander O., Nielsen F.: Reranking with contextual dissimilarity measures from representational bregman k-means.. In: VISAPP (1), Pp. 118–123, 2010

  36. El-Naqa I., Yang Y., Galatsanos N. P., Nishikawa R. M., Wernick M. N.: A similarity learning approach to content-based image retrieval: application to digital mammography. IEEE transactions on medical imaging 23 (10): 1233–1244, 2004

    PubMed  Google Scholar 

  37. Bai S., Bai X.: Sparse contextual activation for efficient visual re-ranking. IEEE Transactions on Image Processing 25 (3): 1056–1069, 2016

    PubMed  Google Scholar 

  38. Bai X., Yang X., Latecki L. J., Liu W., Tu Z.: Learning context-sensitive shape similarity by graph transduction. IEEE Transactions on Pattern Analysis and Machine Intelligence 32 (5): 861–874, 2009

    Google Scholar 

  39. Bai S., Sun S., Bai X., Zhang Z., Tian Q.: Improving context-sensitive similarity via smooth neighborhood for object retrieval. Pattern Recognition 83: 353–364, 2018

    Google Scholar 

  40. Bai S., Sun S., Bai X., Zhang Z., Tian Q.: Smooth neighborhood structure mining on multiple affinity graphs with applications to context-sensitive similarity.. In: European conference on computer vision, pp. 592–608. Springer, 2016

  41. Ma L., Liu X., Gao Y., Zhao Y., Zhao X., Zhou C.: A new method of content based medical image retrieval and its applications to ct imaging sign retrieval. Journal of biomedical informatics 66: 148–158, 2017

    PubMed  Google Scholar 

  42. Rahman M. M., Desai B. C., Bhattacharya P.: Image retrieval-based decision support system for dermatoscopic images.. In: 19Th IEEE symposium on computer-based medical systems (CBMS’06), pp. 285–290. IEEE, 2006

  43. Ballerini L., Li X., Fisher R. B., Rees J.: A query-by-example content-based image retrieval system of non-melanoma skin lesions.. In: MICCAI International workshop on medical content-based retrieval for clinical decision support, pp. 31–38. Springer, 2009

  44. Dhara A. K., Mukhopadhyay S., Dutta A., Garg M., Khandelwal N.: Content-based image retrieval system for pulmonary nodules: Assisting radiologists in self-learning and diagnosis of lung cancer. Journal of digital imaging 30 (1): 63–77, 2017

    PubMed  Google Scholar 

  45. Wei G., Cao H., Ma H., Qi S., Qian W., Ma Z.: Content-based image retrieval for lung nodule classification using texture features and learned distance metric. Journal of medical systems 42 (1): 13, 2018

    Google Scholar 

  46. Suganya R., Rajaram S.: Content based image retrieval of ultrasound liver diseases based on hybrid approach. American Journal of Applied Sciences 9 (6): 938, 2012

    Google Scholar 

  47. Akakin H. C., Gurcan M. N.: Content-based microscopic image retrieval system for multi-image queries. IEEE transactions on information technology in biomedicine 16 (4): 758–769, 2012

    PubMed  PubMed Central  Google Scholar 

  48. Qayyum A., Anwar S. M., Awais M., Majid M.: Medical image retrieval using deep convolutional neural network. Neurocomputing 266: 8–20, 2017

    Google Scholar 

  49. Deserno T. M., Antani S., Long R.: Ontology of gaps in content-based image retrieval. Journal of digital imaging 22 (2): 202–215, 2009

    PubMed  Google Scholar 

  50. Smeulders A. W., Worring M., Santini S., Gupta A., Jain R.: Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis & Machine Intelligence 12: 1349–1380, 2000

    Google Scholar 

  51. Ojansivu V., Heikkilä J.: Blur insensitive texture classification using local phase quantization.. In: International conference on image and signal processing, pp. 236–243. Springer, 2008

  52. Tan X., Triggs W.: Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE transactions on image processing 19 (6): 1635–1650, 2010

    PubMed  Google Scholar 

  53. Orozco H. M., Villegas O. O. V., Sánchez V. G. C., Domínguez H.d.J.O., Alfaro M.d.J.N.: Automated system for lung nodules classification based on wavelet feature descriptor and support vector machine. Biomedical engineering online 14 (1): 9 , 2015

    Google Scholar 

  54. Yang H., Moody J.: Feature selection based on joint mutual information.. In: Proceedings of international ICSC symposium on advances in intelligent data analysis, pp. 22–25. Citeseer, 1999

  55. Messay T., Hardie R. C., Rogers S. K.: A new computationally efficient cad system for pulmonary nodule detection in ct imagery. Medical image analysis 14 (3): 390–406, 2010

    PubMed  Google Scholar 

  56. Akram S., Javed M. Y., Akram M. U., Qamar U., Hassan A.: Pulmonary nodules detection and classification using hybrid features from computerized tomographic images. Journal of Medical Imaging and Health Informatics 6 (1): 252–259, 2016

    Google Scholar 

  57. Tartar A., Kilic N., Akan A. (2013) Classification of pulmonary nodules by using hybrid features. Computational and Mathematical Methods in Medicine, 2013

  58. Wang Z., Chi Z., Feng D.: Shape based leaf image retrieval. IEE Proceedings-Vision. Image and Signal Processing 150 (1): 34–43, 2003

    Google Scholar 

  59. Chandrashekar G., Sahin F.: A survey on feature selection methods. Computers & Electrical Engineering 40 (1): 16–28, 2014

    Google Scholar 

  60. Xue B., Zhang M., Browne W. N., Yao X.: A survey on evolutionary computation approaches to feature selection. IEEE Transactions on Evolutionary Computation 20 (4): 606–626, 2015

    Google Scholar 

  61. Brown G., Pocock A., Zhao M. J., Luján M.: Conditional likelihood maximisation: a unifying framework for information theoretic feature selection. Journal of machine learning research 13: 27–66, 2012. Jan

    Google Scholar 

  62. Platt J., et al.: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10 (3): 61–74, 1999

    Google Scholar 

  63. Shaffer C. A.: Data structures and algorithm analysis. Update 3: 0–10, 2013

    Google Scholar 

  64. Fleuret F.: Fast binary feature selection with conditional mutual information. Journal of Machine learning research 5: 1531–1555, 2004. Nov

    Google Scholar 

  65. Peng H., Long F., Ding C.: Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis & Machine Intelligence 8: 1226–1238, 2005

    Google Scholar 

  66. Jakulin A. (2005) Machine learning based on attribute interactions: phd dissertation. Ph.D. thesis, Univerza v Ljubljani, Fakulteta za računalništvo in informatiko

  67. Lin D., Tang X.: Conditional infomax learning: an integrated framework for feature extraction and fusion.. In: European conference on computer vision, pp. 68–82. Springer, 2006

  68. Meyer P. E., Bontempi G.: On the use of variable complementarity for feature selection in cancer classification.. In: Workshops on applications of evolutionary computation, pp. 91–102. Springer, 2006

  69. Banerjee I., Kurtz C., Devorah A. E., Do B., Rubin D. L., Beaulieu C. F.: Relevance feedback for enhancing content based image retrieval and automatic prediction of semantic image features: Application to bone tumor radiographs. Journal of biomedical informatics 84: 123–135, 2018

    PubMed  Google Scholar 

  70. Khatami A., Babaie M., Tizhoosh H. R., Khosravi A., Nguyen T., Nahavandi S.: A sequential search-space shrinking using cnn transfer learning and a radon projection pool for medical image retrieval. Expert Systems with Applications 100: 224–233, 2018

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Electronics and Electrical Engineering, Univeristy of Engineering & Technology, Taxila, Pakistan

    Muhammad Kashif, Gulistan Raja & Furqan Shaukat

Authors
  1. Muhammad Kashif
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gulistan Raja
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Furqan Shaukat
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gulistan Raja.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kashif, M., Raja, G. & Shaukat, F. An Efficient Content-Based Image Retrieval System for the Diagnosis of Lung Diseases. J Digit Imaging 33, 971–987 (2020). https://doi.org/10.1007/s10278-020-00338-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-020-00338-w

Keywords

Search

Navigation