Abstract
Image retrieval is an important problem for researchers in computer vision and content-based image retrieval (CBIR) fields. Over the last decades, many image retrieval systems were based on image representation as a set of extracted low-level features such as color, texture and shape. Then, systems calculate similarity metrics between features in order to find similar images to a query image. The disadvantage of this approach is that images visually and semantically different may be similar in the low level feature space. So, it is necessary to develop tools to optimize retrieval of information. Integration of vector space models is one solution to improve the performance of image retrieval. In this paper, we present an efficient and effective retrieval framework which includes a vectorization technique combined with a pseudo relevance model. The idea is to transform any similarity matching model (between images) to a vector space model providing a score. A study on several methodologies to obtain the vectorization is presented. Some experiments have been undertaken on Wang, Oxford5k and Inria Holidays datasets to show the performance of our proposed framework.
Similar content being viewed by others
References
Aidos H, Duarte JMM, Fred ALN (2014) Identifying regions of interest for discriminating Alzheimer’s disease from mild cognitive impairment. IEEE International Conference on Image Processing (ICIP), pp 21–25
Angelova A, Zhu S (2013) Efficient object detection and segmentation for fine-grained recognition. IEEE Conference on Computer Vision and Pattern Recognition, pp 811–818
Anh ND, Bao PT, Nam BN, Hoang NH (2010) A New CBIR System Using SIFT Combined with Neural Network and Graph-Based Segmentation. Intelligent Information and Database Systems, Second International Conference, ACIIDS, pp 294–301
Alpkocak A, Kilinc D, Expansion TB (2010) Re-ranking approaches for multimodal image retrieval using text-based methods ImageCLEF, Experimental Evaluation in Visual Information Retrieval, pp 261–275
Arandjelovic R, Gronat P, Torii A, Pajdla T, Sivic J (2015) NetVLAD: CNN architecture for weakly supervised place recognition
Arandjelovic R, Zisserman A (2012) Three things everyone should know to improve object retrieval. IEEE Conference on Computer Vision and Pattern Recognition, pp 2911–2918
Atreya A, Elkan C (2010) Latent semantic indexing (LSI) fails for TREC collections. SIGKDD Explorations, pp 5–10
Babenko A, Lempitsky VS (2015) Aggregating Deep Convolutional Features for Image Retrieval
Babenko A, Slesarev A, Chigorin A, Lempitsky VS (2014) Neural codes for image retrieval. Computer Vision - ECCV, pp 584–599
Becker CJ, Rigamonti R, Lepetit V, Fua P (2013) Supervised feature learning for curvilinear structure segmentation. Medical Image Computing and Computer-Assisted Intervention - MICCAI, pp 526– 533
Canny J (1986) A computational approach to edge detection. IEEE Transaction on Pattern Analysis Machine Intelligence. pp 679–698
Chatzichristofis SA, Yiannis SB (2008) CEDD color and edge directivity descriptor: A compact descriptor for image indexing and retrieval. Computer Vision Systems, 6th International Conference, pp 312–322
Claveau V, Tavenard R, d’appariement LA (2010) Vectorisation des processus document-requte. Conference en Recherche d’Infomations et Applications. Conference en Recherche d’Infomations et Applications (CORIA), pp 313–324
Delaitre V, Laptev I, Sivic J (2010) Recognizing human actions in still images: a study of bag-of-features and part-based representations. British Machine Vision Conference - BMVC, pp 1–11
Deserno TM, Guld MO, Plodowski B, Spitzer K, Wein BB, Schubert H, Ney H, Seidl T (2008) Extended query refinement for medical image retrieval. Journal Digital Imaging pp 280–289
Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Trevor D (2014) DeCAF A deep convolutional activation feature for generic visual recognition. Proceedings of the 31th International Conference on Machine Learning - ICML, pp 647–655
Douze M, Jegou H, Sandhawalia H, Amsaleg L, Schmid C (2009) Evaluation of GIST descriptors for web-scale image search. Proceedings of the 8th ACM International Conference on Image and Video Retrieval CIVR
Flickner M, Sawhney HS, Ashley J, Huang Q, Dom B, Gorkani M, Hafner J, Lee D, Petkovic D, Steele D, Yanker P (1995) Query by image and video content: The QBIC System. IEEE Computer, pp 23–32
Gong Y, Lazebnik S (2011) Iterative quantization: A procrustean approach to learning binary codes. The 24th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 817–824
Gong Y, Wang L, Guo R, Lazebnik S (2014) Multi-scale orderless pooling of deep convolutional activation features. Computer Vision - ECCV, pp 392–407
Gony J, Cord M, Philipp-Foliguet S, Gosselin Philippe H, Precioso F, Jordan M (2007) RETIN: a smart interactive digital media retrieval system. Proceedings of the 6th ACM International Conference on Image and Video Retrieval, pp 93–96
Hsu W, Rodney Long L, Antani SK (2007) SPIRS: A framework for content-based image retrieval from large biomedical databases. (MEDINFO)- Proceedings of the 12th World Congress on Health (Medical) Informatics - Building Sustainable Health Systems, pp 188–192
Jain AK, Nandakumar K, Ross A (2005) Score norMalization in multimodal biometric systems. Pattern Recognition, pp 2270–2285
Jegou H, Chum O (2012) Negative Evidences and Co-occurences in Image Retrieval: The Benefit of PCA and Whitening. Computer Vision. ECCV 12th European Conference on Computer Vision, pp 774–787
Jegou H, Zisserman A (2014) Triangulation embedding and democratic aggregation for image search. IEEE Conference on Computer Vision and Pattern Recognition - CVPR, pp 3310–3317
Jordan C, Watters CR (2004) Extending the Rocchio relevance feedback algorithm to provide contextual retrieval dvances in Web intelligence. Second International Atlantic Web Intelligence Conference (AWIC), pp 135–144
Kalantidis Y, Mellina C, Osindero S (2016) Cross-dimensional weighting for aggregated deep convolutional features. Computer Vision - ECCV, pp 685–701
Karamti H (2013) Vectorisation du modle d’appariement pour la recherche d’images par le contenu. Conference en Recherche d’Infomations et Applications (CORIA), pp 335–340
Karamti H, Tmar M, Benammar A (2012) A new relevance feedback approach for multimedia retrieval. IKE, pp 129–135
Karamti H, Tmar M, Gargouri F (2014) Content-based image retrieval system using neural network. 11th IEEE/ACS International Conference on Computer Systems and Applications (AICCSA), pp 723–728
Karamti H, Tmar M, Gargouri F (2015) Content-based image retrieval system with relevance feedback. WEBIST 2015 - Proceedings of the 11th International Conference on Web Information Systems and Technologies, pp 287–292
Kasutani E, Yamada A (2001) The MPEG-7 color layout descriptor: a compact image feature description for high-speed image/video segment retrieval. ICIP (1), pp 674–677
Kulkarni S, Srinivasan B, Ramakrishna MV (1999) Vector-space image model (VSIM) for content-based retrieval. 10th International Workshop on Database and Expert Systems Applications, pp 899–903
Lee H, Grosse RB, Ranganath R, Ng AY (2011) Unsupervised learning of hierarchical representations with convolutional deep belief networks Commun. ACM, pp 95–103
Lindeberg T (1996) Edge Detection and Ridge Detection with Automatic Scale Selection. Conference on Computer Vision and Pattern Recognition CVPR, pp 465–470
Liu W, Wang J, Ji R, Jiang Y-G, Chang S-F (2012) Supervised hashing with kernels. Conference on Computer Vision and Pattern Recognition, Providence, RI, pp 2074–2081
Lowe DG (1999) Object recognition from local scale-invariant features. ICCV, pp 1150–1157
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, pp 91–110
Mandal S, Sudarshan VP, Nagaraj Y, Dean-Ben XL, Razansky D (2015) Multiscale edge detection and parametric shape modeling for boundary delineation in optoacoustic images. 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp 707–710
Manning CD, Raghavan P, Schutze H (2008) Introduction to information retrieval. Cambridge University Press
Miao J, Huang JX, Ye Z (2012) Proximity-based rocchio’s model for pseudo relevance. The 35th International ACM-SIGIR conference on research and development in Information Retrieval, pp 535–544
Ming A, Ma H (2007) A blob detector in color images. Proceedings of the 6th ACM International Conference on Image and Video Retrieval, pp 364–370
Ng JY-H, Yang F, Davis LS (2015) Exploiting local features from deep networks for image retrieval. IEEE Conference on Computer Vision and Pattern Recognition Workshops - CVPR, pp 53–61
Ortiz-Jaramillo B, Benitez-Restrepo H, Garcia-Alvarez JC, Castellanos-Dominguez CG (2010) Region of Interest Extraction based on Multiresolution Analysis for Infrared Nondestructive Testing. 10th Quantitative Infrared Thermography Conference QIRT
Park DK, Jeon YS, Won CS (2000) Efficient use of local edge histogram descriptor. Proceedings of the ACM Multimedia 2000 Workshops, pp 51–54
Paulin M, Douze M, Harchaoui Z, Mairal J, Perronnin F, Schmid C (2015) Local convolutional features with unsupervised training for image retrieval. IEEE International Conference on Computer Vision - ICCV, pp 91–99
Philbin J, Chum O, Isard M, Sivic J, Zisserman A (2007) Object retrieval with large vocabularies and fast spatial matching. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)
Philbin J, Isard M, Sivic J, Zisserman Andrew (2010) Descriptor learning for efficient retrieval. Computer Vision - (ECCV) 2010, 11th European Conference on Computer Vision, pp 677–691
Ramanathan V, Mishra S, Mitra P (2011) Quadtree decomposition based extended vector space model for image retrieval. IEEE Workshop on Applications of Computer Vision WACV, pp 139– 144
Ramos H, Ribeiro AL, Giro PM (1994) A Two-dimensional Vector Model of Ferromagnetic Hysteresis. Journal of Magnetism and Magnetic Materials, pp 574–577
Rosten E, Drummond T (2006) Machine learning for high-speed corner detection. Computer Vision - ECCV, 9th European Conference on Computer Vision, ppp 430–443
Ruthven I, Lalmas M (2003) A survey on the use of relevance feedback for information access systems Knowledge Engineering Review, pp 95–145
Salembier P (2002) Overview of the MPEG-7 Standard and of Future Challenges for Visual Information Analysis. EURASIP Journal of Advanced Signal Proceedings, pp 343–353
Salembier P, Sikora T (2002) Introduction to MPEG-7: Multimedia content description interface. Wiley,
Salton G, Wong A, Yang CS (1975) A Vector Space Model for Automatic Indexing Commun. ACM, pp 613–620
Schettini R, Ciocca G, Gagliardi I (2009) Feature extraction for content-based image retrieval. Encyclopedia of Database Systems, pp 1115–1119
Sheikholeslami G, Chang W, SemQuery AZ (2002) Semantic clustering and querying on heterogeneous features for visual data. IEEE Transactions on Knowledge and Data Engineering, pp 988– 1002
Smeulders AWM, Worring M, Santini S, Gupta A, Jain RC (2000) Content-based image retrieval at the end of the early years. IEEE Transactions Pattern Analysis Machine Intelligence, pp 1349– 1380
Stathopoulos V, Jose JM (2011) Bayesian probabilistic models for image retrieval. Proceedings of the Second Workshop on Applications of Pattern Analysis (WAPA), pp 41–47
Teran L, Mordohai P (2014) 3D interest point detection via discriminative learning. Computer Vision - ECCV - 13th European Conference, pp 159–173
Tolias G (2015) Ronan Sicre and Herve Jegou Particular object retrieval with integral max-pooling of CNN activations
Tsai C-F, Hu Y-H, Chen Z-Y (2015) Factors affecting rocchio-based pseudorelevance feedback in image retrieval. JASIST, pp 40–57
Wang D, Tan X (2014) C-SVDDNet: An Effective Single-Layer Network for Unsupervised Feature Learning
Wang Y, Gong M, Wang T, Cohen-Or D, Zhang H, Chen B (2013) Projective analysis for 3D shape segmentation. ACM Transactions, pp 192–192
Wengert C, Douze M, Jegou H (2011) Bag-of-colors for improved image search. Proceedings of the 19th International Conference on Multimedia, pp 1437–1440
Westerveld T, De Vries AP, Van Ballegooij A, De Jong F, Hiemstra D (2003) A Probabilistic Multimedia Retrieval Model and Its Evaluation. EURASIP Journal of Advanced Signal Proceedings, pp 186– 198
Winder SAJ, Hua G, Brown MA (2009) Picking the best DAISY. IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR, pp 178–185
Yan R, Hauptmann AG, Jin R (2003) Negative pseudo-relevance feedback in content-based video retrieval. Proceedings of the Eleventh ACM International Conference on Multimedia, pp 343– 346
Ye Z, Huang JX (2014) A simple term frequency transformation model for effective pseudo relevance feedback. The 37th International ACM-SIGIR Conference on Research and Development in Information Retrieval, pp 323–332
Yue J, Li Z, Lu L, Fu Z (2011) Content-based image retrieval using color and texture fused features. Mathematical and Computer Modelling, pp 1121–1127
Zhang D, Lu G (2001) Content-Based Shape Retrieval Using Different Shape Descriptors: A Comparative Study. Proceedings of the IEEE International Conference on Multimedia and Expo
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Karamti, H., Tmar, M., Visani, M. et al. Vector space model adaptation and pseudo relevance feedback for content-based image retrieval. Multimed Tools Appl 77, 5475–5501 (2018). https://doi.org/10.1007/s11042-017-4463-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-017-4463-x