Abstract
In this paper, a new content-based medical image retrieval (CBMIR) framework using an effective classification method and a novel relevance feedback (RF) approach are proposed. For a large-scale database with diverse collection of different modalities, query image classification is inevitable due to firstly, reducing the computational complexity and secondly, increasing influence of data fusion by removing unimportant data and focus on the more valuable information. Hence, we find probability distribution of classes in the database using Gaussian mixture model (GMM) for each feature descriptor and then using the fusion of obtained scores from the dependency probabilities, the most relevant clusters are identified for a given query. Afterwards, visual similarity of query image and images in relevant clusters are calculated. This method is performed separately on all feature descriptors, and then the results are fused together using feature similarity ranking level fusion algorithm. In the RF level, we propose a new approach to find the optimal queries based on relevant images. The main idea is based on density function estimation of positive images and strategy of moving toward the aggregation of estimated density function. The proposed framework has been evaluated on ImageCLEF 2005 database consisting of 10,000 medical X-ray images of 57 semantic classes. The experimental results show that compared with the existing CBMIR systems, our framework obtains the acceptable performance both in the image classification and in the image retrieval by RF.
Similar content being viewed by others
References
Tamura H, Yokoya N: Image database systems: a survey. Pattern Recognit 17(1):29–43, 1984
Chang SK, Hsu A: Image information systems: where do we go from here? IEEE Trans Knowl Data Eng 5(5):431–442, 1992
Smeulders AWM, Worring M, Santini S, Gupta A, Jain R: Content-based image retrieval at the end of the early years, IEEE Trans. Pattern Anal Mach Intell 22(12):1349–1380, 2000
Rui Y, Huang TS, Chang S-F: Image retrieval: current techniques, promising directions, and open issues. J Vis Commun Image Represent 10(4):39–62, 1999
Jafari Fesharaki N, Pourghassem H: Medical x-ray image hierarchical classification using a merging and splitting scheme in feature space. J Med Signals Sensors 3(3):150–163, 2013
Liu Y, Zhanga D, Lua G, Ma WY: A survey of content-based image retrieval with high-level semantics. Pattern Recognit 40(1):262–282, 2007
Pourghassem H, Ghassemian H: Content-based medical image classification using a new hierarchical merging scheme. J Comput Med Imaging Graph 32(8):651–661, 2008
N. Jafari Fesharaki, H. Pourghassem: Medical X-ray Images Classification Based on Shape Features and Bayesian Rule, Fourth International Conference on Computational Intelligence and Communication Networks (CICN2012): Mathura, Uttar Pradesh, India 3–5 Nov. 2012, pp 369-373
Rahman MM, Desai BC, Bhattacharya P: Medical image retrieval with probabilistic multi-class support vector machine classifiers and adaptive similarity fusion. J Comput Med Imaging Graph 32(2):95–108, 2008
Greenspan H, Pinhas AT: Medical image categorization and retrieval for PACS using the GMM-KL framework. IEEE Trans Inf Tech Bio 11(2):190–202, 2007
Marakakis A, Siolas G, Galatsanos N, Likas A, Stafylopatis A: Relevance feedback approach for image retrieval combining support vector machines and adapted Gaussian mixture models. IET Image Process 5(6):531–540, 2011
Xie Z, Wang S, Hu D: New insight at level set & Gaussian mixture model for natural image segmentation. SIViP 7(3):521–536, 2013
Chatzichristofis SA, Zagoris K, Boutalis Y, Arampatzis A: A Fuzzy Rank-Based Late Fusion Method for Image Retrieval. 18th International Conference, MMM(2012). Springer Berlin Heidelberg, Klagenfurt, Austria, 2012, pp 463–472
M. Jović, Y. Hatakeyama, F. Dong, K. Hirota: Image Retrieval Based on Similarity Score Fusion from Feature Similarity Ranking Lists, Third International Conference, FSKD 2006, Xi’an: China, Sep. 2006, pp 461–470
Su JH, Huang WH, Yu PS, Tseng VS: Efficient relevance feedback for content-based image retrieval by mining user navigation patterns. IEEE Trans Knowl Data Eng 23(3):360–372, 2011
J.J. Rocchio: Relevance Feedback in Information Retrieval: The SMART Retrieval System—Experiments in Automatic Document Processing, 1971, pp 313–323
Y. Rui, T. S. Huang, and S. Mehrotra: Content-Based Image Retrieval with Relevance Feedback in MARS: Proc. IEEE Int’l Conf. Image Processing, Oct. 1997, pp 815–818
Rui Y, Huang TS, Ortega M, Mehrotra S: Relevance feedback: a power tool for interactive content-based image retrieval. IEEE Trans Circ Syst Video Techno 8(5):644–655, 1998
K. Porkaew, K. Chakrabarti, and S. Mehrotra: Query Refinement for Multimedia Similarity Retrieval in MARS: Proc. ACM Int’l Multimedia Conf. (ACMMM), 1999, pp 235–238
Rahman MM, Bhattacharya P, Desai BC: A framework for medical image retrieval using machine learning and statistical similarity matching techniques with relevance feedback. IEEE Trans Inf Tech Bio 11(1):58–69, 2007
Pourghassem H, Daneshvar S: A framework for medical image retrieval using merging-based classification with dependency probability-based relevance feedback. Turk J Electr Eng Comput Sci 21(3):882–896, 2013
Rahman MM, Antani SK, Thoma GR: A learning-based similarity fusion and filtering approach for biomedical image retrieval using SVM classification and relevance feedback. IEEE Trans Inf Technol Biomed 15(4):640–646, 2011
Yang L, Jin R, Mummert L, Sukthankar R, Goode A, Zheng B, Hoi SCH, Satyanarayanan M: A boosting framework for visuality-preserving distance metric learning and its application to medical image retrieval. IEEE Trans Pattern Anal Mach Intell 32(1):30–44, 2010
Duda RO, Hart PE, Stork DG: Pattern Classification, 2nd edition. John Wiley & Sons, New York, NY, 2001
Nurmohamadi M, Pourghassem H: Clavulanic acid production estimation based on color and structural features of streptomyces clavuligerus bacteria using self-organizing map and genetic algorithm. Comput Methods Programs Biomed 114(3):337–348, 2014
M. Behnam, H. Pourghassem: Feature Descriptor Optimization in Medical Image Retrieval Based on Genetic Algorithm: 20th Iranian Conference on Biomedical Engineering (ICBME2013), Tehran, Iran: 18–20 Dec, 2013, pp 280–285
da Silva SF, Ribeiro MX, Neto JB, TrainaJr C, Traina AJM: Improving the ranking quality of medical image retrieval using a genetic feature selection method. Decis Support Syst 51(4):810–820, 2011
Steji’c Z, Takama Y, Hirota K: Genetic algorithms for a family of image similarity models incorporated in the relevance feedback mechanism. Appl Soft Comput 2(4):306–327, 2003
Lai CC, Chen YC: A user-oriented image retrieval system based on interactive genetic algorithm. IEEE Trans Instrum Meas 60(10):3318–3325, 2011
Ojala T, Pietikäinen M, Mäenpää T: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987, 2002
Zhao Y, Huang DS, Jia W: Completed local binary count for rotation invariant texture classification. IEEE Trans Image Process 21(10):4492–4497, 2012
Guo ZH, Zhang L, Zhang D: A completed modeling of local binary pattern operator for texture classification. IEEE Trans Image Process 19(6):1657–1663, 2010
Persoon E, Fu K: Shape discrimination using fourier descriptors. IEEE Trans Pattern Anal Mach Intell 8(3):388–397, 1977
Zahn CT, Roskies RZ: Fourier descriptors for plane closed curves. IEEE Trans Comput 21(3):269–281, 1972
Tahmasebi A, Pourghassem H: A novel intra-class distance-based signature identification algorithm using weighted gabor features and dynamic characteristics. Arab J Sci Eng 38(11):3019–3029, 2013
Zhang DS, Lu G: Generic fourier descriptor for shape-based image retrieval. IEEE Int Conf Multimed Expo 1:425–428, 2002
D. S. Zhang and G. Lu: Enhanced generic Fourier descriptors for object-based image retrieval: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP2002), 2002, pp 3668_3671
Shahbeig S, Pourghassem H: A fast and automatic algorithm for optic disc extraction in retinal images using PCA-based pre-processing and curvelet transform. J Opt Soc Am A 30(1):13–21, 2013
Pourghassem H: A novel material detection algorithm based on 2D GMM-based power density function and image detail addition scheme in dual energy x-ray images. J X-Ray Sci Technol (IOS press) 20(2):213–228, 2012
Pourghassem H: A relevance feedback approach based on modification of similarity measure using particle swarm optimization in a medical x-ray image retrieval system. Majlesi J Electrical Eng 4(2):9–17, 2010
P. Clough, H. Muller, T. Deselaers, M. Grubinger, T. M. Lehmann, J. Jensen, W. Hersh: The CLEF 2005 Cross–Language Image Retrieval Track: 6th Workshop of the Cross-Language Evaluation Forum, CLEF 2005, Vienna, Austria: 21–23, 2006, September, 2005, pp. 535–557
Smith JR, Chang S: Tools and techniques for color image retrieval. Storage Retrieval Image Video Databases IV 2670:1–12, 1996
Deselaers T, Keysers D, Ney H: Classification error rate for quantitative evaluation of content-based image retrieval systems. Proc 17th International Conf Pattern Recognit 2:505–508, 2004
Acknowledgments
The authors would like to thank the IRMA Group, Aachen, Germany, for making the database available for the experiments.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Behnam, M., Pourghassem, H. Optimal Query-Based Relevance Feedback in Medical Image Retrieval Using Score Fusion-Based Classification. J Digit Imaging 28, 160–178 (2015). https://doi.org/10.1007/s10278-014-9730-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10278-014-9730-z