Advertisement

Local bit-plane decoded convolutional neural network features for biomedical image retrieval

  • Shiv Ram DubeyEmail author
  • Swalpa Kumar Roy
  • Soumendu Chakraborty
  • Snehasis Mukherjee
  • Bidyut Baran Chaudhuri
Original Article
  • 93 Downloads

Abstract

Biomedical image retrieval is a challenging problem due to the varying contrast and size of structures in the images. The approaches for biomedical image retrieval generally rely on the feature descriptors to characterize the images. The feature descriptor of query image is compared with the descriptors of images from the database, to find the best matches. Several hand-crafted feature descriptors have been proposed so far for biomedical image retrieval by exploiting the local relationship of neighboring image pixels. It is observed in the literature that the local bit-plane decoded features are well suited for this retrieval task. Moreover, in recent past, it is also observed that the convolutional neural network-based features such as AlexNet, Vgg16, GoogleNet and ResNet perform well in many computer vision-related tasks. Motivated by the success of the deep learning-based approaches, this paper proposes a local bit-plane decoding-based AlexNet descriptor (LBpDAD) for biomedical image retrieval. The proposed LBpDAD is computed by max-fusing the ReLU operated feature maps of pre-trained AlexNet at a particular layer, obtained from the original and local bit-plane decoded images. The proposed approach is also compared with Vgg16, GoogleNet and ResNet models. The experiments on the proposed method over three benchmark biomedical databases of different modalities such as MRI, CT and microscopic show the efficacy of the proposed descriptor.

Keywords

Biomedical images Convolutional neural networks AlexNet Image retrieval Local bit-plane decoding 

Notes

Acknowledgements

This research is funded by Science and Engineering Research Board (SERB), Govt. of India through Project Sanction No. ECR/2017/000082. The authors would like to thank NVIDIA Corporation for the support of 2 GeForce Titan X Pascal GPU donated to Computer Vision Group, IIIT Sri City.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

References

  1. 1.
    Boland MV, Murphy RF (2001) A neural network classifier capable of recognizing the patterns of all major subcellular structures in fluorescence microscope images of hela cells. Bioinformatics 17(12):1213–1223CrossRefGoogle Scholar
  2. 2.
    Carneiro G, Nascimento J, Bradley AP (2015) Unregistered multiview mammogram analysis with pre-trained deep learning models. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 652–660Google Scholar
  3. 3.
    Chakraborty S, Singh S, Chakraborty P (2016) Local gradient hexa pattern: a descriptor for face recognition and retrieval. IEEE Trans Circuits Syst Video Technol 28:171–180CrossRefGoogle Scholar
  4. 4.
    Chakraborty S, Singh SK, Chakraborty P (2017) Local directional gradient pattern: a local descriptor for face recognition. Multimed Tools Appl 76(1):1201–1216CrossRefGoogle Scholar
  5. 5.
    Chang YY, Tai SC, Lin JS (2012) Segmentation of multispectral mr images through an annealed rough neural network. Neural Comput Appl 21(5):911–919CrossRefGoogle Scholar
  6. 6.
    Chowdhury M, Bulo SR, Moreno R, Kundu MK, Smedby Ö (2016) An efficient radiographic image retrieval system using convolutional neural network. In: 2016 23rd international conference on pattern recognition (ICPR). IEEE, New York, pp 3134–3139Google Scholar
  7. 7.
    Chung YA, Weng WH (2017) Learning deep representations of medical images using siamese CNNs with application to content-based image retrieval. Preprint. arXiv:1711.08490
  8. 8.
    Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M et al (2013) The cancer imaging archive (TCIA): maintaining and operating a public information repository. J Dig Imaging 26(6):1045–1057CrossRefGoogle Scholar
  9. 9.
    Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: IEEE conference on computer vision and pattern recognition, CVPR 2009. IEEE, New York, pp 248–255Google Scholar
  10. 10.
    Dubey SR, Chakraborty S (2018) Average biased ReLU based CNN descriptor for improved face retrieval. Preprint. arXiv:1804.02051
  11. 11.
    Dubey SR, Mukherjee S (2018) Ldop: local directional order pattern for robust face retrieval. Preprint. arXiv:1803.07441
  12. 12.
    Dubey SR, Singh SK, Singh RK (2014) Rotation and illumination invariant interleaved intensity order-based local descriptor. IEEE Trans Image Process 23(12):5323–5333MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Dubey SR, Singh SK, Singh RK (2015) Local diagonal extrema pattern: a new and efficient feature descriptor for CT image retrieval. IEEE Signal Process Lett 22(9):1215–1219CrossRefGoogle Scholar
  14. 14.
    Dubey SR, Singh SK, Singh RK (2015) Local wavelet pattern: a new feature descriptor for image retrieval in medical CT databases. IEEE Trans Image Process 24(12):5892–5903MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Dubey SR, Singh SK, Singh RK (2016) Local bit-plane decoded pattern: a novel feature descriptor for biomedical image retrieval. IEEE J Biomed Health Inf 20(4):1139–1147CrossRefGoogle Scholar
  16. 16.
    Dubey SR, Singh SK, Singh RK (2016) Multichannel decoded local binary patterns for content-based image retrieval. IEEE Trans Image Process 25(9):4018–4032MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    Dubey SR, Singh SK, Singh RK (2016) Novel local bit-plane dissimilarity pattern for computed tomography image retrieval. Electron Lett 52(15):1290–1292CrossRefGoogle Scholar
  18. 18.
    Ge Y, Jiang S, Xu Q, Jiang C, Ye F (2017) Exploiting representations from pre-trained convolutional neural networks for high-resolution remote sensing image retrieval. Multimed Appl 77:1–27Google Scholar
  19. 19.
    Guo K, Duan G (2014) 3D image retrieval based on differential geometry and co-occurrence matrix. Neural Comput Appl 24(3–4):715–721CrossRefGoogle Scholar
  20. 20.
    He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: 2017 IEEE international conference on computer vision (ICCV). IEEE, New York, pp 2980–2988Google Scholar
  21. 21.
    He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778Google Scholar
  22. 22.
    Hermessi H, Mourali O, Zagrouba E (2018) Convolutional neural network-based multimodal image fusion via similarity learning in the shearlet domain. Neural Comput Appl 30:1–17CrossRefGoogle Scholar
  23. 23.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems conference, pp 1097–1105Google Scholar
  24. 24.
    Lan R, Zhou Y (2017) Medical image retrieval via histogram of compressed scattering coefficients. IEEE J Biomed Health Inf 21(5):1338–1346MathSciNetCrossRefGoogle Scholar
  25. 25.
    Liu P, Guo JM, Wu CY, Cai D (2017) Fusion of deep learning and compressed domain features for content-based image retrieval. IEEE Trans Image Process 26(12):5706–5717MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Lu K, He N, Xue J, Dong J, Shao L (2015) Learning view-model joint relevance for 3D object retrieval. IEEE Trans Image Process 24(5):1449–1459MathSciNetzbMATHCrossRefGoogle Scholar
  27. 27.
    Marcus DS, Wang TH, Parker J, Csernansky JG, Morris JC, Buckner RL (2007) Open access series of imaging studies (OASIS): cross-sectional mri data in young, middle aged, nondemented, and demented older adults. J Cogn Neurosci 19(9):1498–1507CrossRefGoogle Scholar
  28. 28.
    Müller H, Michoux N, Bandon D, Geissbuhler A (2004) A review of content-based image retrieval systems in medical applications—clinical benefits and future directions. Int J Med Inf 73(1):1–23CrossRefGoogle Scholar
  29. 29.
    Murala S, Maheshwari R, Balasubramanian R (2012) Local tetra patterns: a new feature descriptor for content-based image retrieval. IEEE Trans Image Process 21(5):2874–2886MathSciNetzbMATHCrossRefGoogle Scholar
  30. 30.
    Murala S, Wu QJ (2013) Local ternary co-occurrence patterns: a new feature descriptor for MRI and CT image retrieval. Neurocomputing 119:399–412CrossRefGoogle Scholar
  31. 31.
    Murala S, Wu QJ (2014) Local mesh patterns versus local binary patterns: biomedical image indexing and retrieval. IEEE J Biomed Health Inf 18(3):929–938CrossRefGoogle Scholar
  32. 32.
    Nanni L, Brahnam S, Ghidoni S, Lumini A (2018) Bioimage classification with handcrafted and learned features. IEEE/ACM Trans Comput Biol Bioinform.  https://doi.org/10.1109/TCBB.2018.2821127 CrossRefGoogle Scholar
  33. 33.
    Nanni L, Lumini A (2008) A reliable method for cell phenotype image classification. Artif Intell Med 43(2):87–97CrossRefGoogle Scholar
  34. 34.
    Nanni L, Lumini A, Brahnam S (2010) Local binary patterns variants as texture descriptors for medical image analysis. Artif Intell Med 49(2):117–125CrossRefGoogle Scholar
  35. 35.
    Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987zbMATHCrossRefGoogle Scholar
  36. 36.
    Paci M, Nanni L, Lahti A, Aalto-Setala K, Hyttinen J, Severi S (2013) Non-binary coding for texture descriptors in sub-cellular and stem cell image classification. Curr Bioinform 8(2):208–219CrossRefGoogle Scholar
  37. 37.
    Pang S, Yu Z, Orgun MA (2017) A novel end-to-end classifier using domain transferred deep convolutional neural networks for biomedical images. Comput Methods Programs Biomed 140:283–293CrossRefGoogle Scholar
  38. 38.
    Pietikäinen M, Hadid A, Zhao G, Ahonen T (2011) Computer vision using local binary patterns, vol 40. Springer, BerlinCrossRefGoogle Scholar
  39. 39.
    Qayyum A, Anwar SM, Awais M, Majid M (2017) Medical image retrieval using deep convolutional neural network. Neurocomputing 266:8–20CrossRefGoogle Scholar
  40. 40.
    Qiu C, Cai Y, Gao X, Cui Y (2017) Medical image retrieval based on the deep convolution network and hash coding. In: 2017 10th international congress on image and signal processing, biomedical engineering and informatics (CISP-BMEI). IEEE, New York, pp 1–6Google Scholar
  41. 41.
    Rajpurkar P, Irvin J, Zhu K, Yang B, Mehta H, Duan T, Ding D, Bagul A, Langlotz C, Shpanskaya K et al (2017) Chexnet: radiologist-level pneumonia detection on chest X-rays with deep learning. Preprint. arXiv:1711.05225
  42. 42.
    Ramírez I, Cuesta-Infante A, Pantrigo JJ, Montemayor AS, Moreno JL, Alonso V, Anguita G, Palombarani L (2018) Convolutional neural networks for computer vision-based detection and recognition of dumpsters. Neural Comput Appl.  https://doi.org/10.1007/s00521-018-3390-8 CrossRefGoogle Scholar
  43. 43.
    Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems conference, pp 91–99Google Scholar
  44. 44.
    Roy SK, Chanda B, Chaudhuri B, Ghosh DK, Dubey SR (2017) A complete dual-cross pattern for unconstrained texture classification. In: 4th IAPR Asian conference on pattern recognition (ACPR 2017), Nanjing, pp 741–746Google Scholar
  45. 45.
    Roy SK, Chanda B, Chaudhuri BB, Banerjee S, Ghosh DK, Dubey SR (2017) Local jet pattern: a robust descriptor for texture classification. Preprint. arXiv:1711.10921
  46. 46.
    Roy SK, Chanda B, Chaudhuri BB, Banerjee S, Ghosh DK, Dubey SR (2018) Local directional ZigZag pattern: a rotation invariant descriptor for texture classification. Pattern Recognit Lett 108:23–30CrossRefGoogle Scholar
  47. 47.
    Sezer A, Sezer HB, Albayrak S (2017) Hermite-based texture feature extraction for classification of humeral head in proton density-weighted MR images. Neural Comput Appl 28(10):3021–3033CrossRefGoogle Scholar
  48. 48.
    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Preprint. arXiv:1409.1556
  49. 49.
    Singh GAP, Gupta P (2018) Performance analysis of various machine learning-based approaches for detection and classification of lung cancer in humans. Neural Comput Appl.  https://doi.org/10.1007/s00521-018-3518-x CrossRefGoogle Scholar
  50. 50.
    Sirinukunwattana K, Raza SEA, Tsang YW, Snead DR, Cree IA, Rajpoot NM (2016) Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans Med Imaging 35(5):1196–1206CrossRefGoogle Scholar
  51. 51.
    Sorensen L, Shaker SB, De Bruijne M (2010) Quantitative analysis of pulmonary emphysema using local binary patterns. IEEE Trans Med Imaging 29(2):559–569CrossRefGoogle Scholar
  52. 52.
    Srivastava D, Rajitha B, Agarwal S et al (2018) Pattern-based image retrieval using GLCM. Neural Comput Appl.  https://doi.org/10.1007/s00521-018-3611-1 CrossRefGoogle Scholar
  53. 53.
    Suri JS, Wilson D, Laxminarayan S (2005) Handbook of biomedical image analysis, vol 2. Springer, BerlinCrossRefGoogle Scholar
  54. 54.
    Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A et al (2015) Going deeper with convolutions. In: CVPRGoogle Scholar
  55. 55.
    Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19(6):1635–1650MathSciNetzbMATHCrossRefGoogle Scholar
  56. 56.
    Wang DH, Conilione P (2012) Machine learning approach for face image retrieval. Neural Comput Appl 21(4):683–694CrossRefGoogle Scholar
  57. 57.
    Wang G, Xu X, Jiang X, Ding S (2016) Medical image registration based on self-adapting pulse-coupled neural networks and mutual information. Neural Comput Appl 27(7):1917–1926CrossRefGoogle Scholar
  58. 58.
    Wang Z, Fan B, Wu F (2011) Local intensity order pattern for feature description. In: 2011 IEEE international conference on computer vision (ICCV). IEEE, New York, pp 603–610Google Scholar
  59. 59.
    Wu Y, Wang L, Cui F, Zhai H, Dong B, Wang JY (2016) Cross-model convolutional neural network for multiple modality data representation. Neural Comput Appl 30:1–11Google Scholar
  60. 60.
    Yang Z, Yu W, Liang P, Guo H, Xia L, Zhang F, Ma Y, Ma J (2018) Deep transfer learning for military object recognition under small training set condition. Neural Comput Appl.  https://doi.org/10.1007/s00521-018-3468-3 CrossRefGoogle Scholar
  61. 61.
    Yao J, Liu F, Geng Y (2017) Query-specific optimal convolutional neural ranker. Neural Comput Appl.  https://doi.org/10.1007/s00521-017-3257-4 CrossRefGoogle Scholar
  62. 62.
    Zhang B, Gao Y, Zhao S, Liu J (2010) Local derivative pattern versus local binary pattern: face recognition with high-order local pattern descriptor. IEEE Trans Image Process 19(2):533–544MathSciNetzbMATHCrossRefGoogle Scholar
  63. 63.
    Zhang L, Lu L, Nogues I, Summers RM, Liu S, Yao J (2017) Deeppap: deep convolutional networks for cervical cell classification. IEEE J Biomed Health Inf 21(6):1633–1643CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Computer Vision GroupIndian Institute of Information TechnologySri City, ChittoorIndia
  2. 2.Computer Vision and Pattern Recognition UnitIndian Statistical InstituteKolkataIndia
  3. 3.Computer Science and Engineering DepartmentIndian Institute of Information TechnologyLucknowIndia

Personalised recommendations