Abstract
Content based image retrieval systems rely heavily on the set of features extracted from images. Effective image representation emerges as a crucial step in such systems. A key challenge in visual content representation is to reduce the so called ‘semantic gap’. It is the inability of existing methods to describe contents in a human-oriented way. Content representation methods inspired by the human vision system have shown promising results in image retrieval. Considerable work has been carried out during the past two decades for developing methods to extract descriptors inspired by the human vision system and attempt to retrieve visual contents efficiently according to the user needs, thereby reducing the semantic gap. Despite the extensive research being conducted in this area, limitations in current image retrieval systems still exist. This paper presents a descriptor for personalized social image collections which utilizes the local structure patterns in salient edge maps of images at multiple scales. The human visual system at the basic level is sensitive to edges, corners, intersections, and other such intensity variations in images generating local structure patterns. Analyzing these patterns at multiple scales allow the most salient fine-grained and coarse-grained features to be captured. The features are accumulated in a local structure patterns histogram to index images allowing flexible querying of visual contents. The retrieval results show that the proposed descriptor ranks well among similar state-of-the-art methods for large social image collections.
Similar content being viewed by others
References
Ahmad J, Jan Z, Khan SM (2014) A Fusion of Labeled-Grid Shape Descriptors with Weighted Ranking Algorithm for Shapes Recognition. World Applied Sciences Journal 31(6):1207–1213
Ahmad J, Sajjad M, Mehmood I, Baik SW (2015) SSH : Salient structures histogram for content based image retrieval. In: 18th IEEE International Conference on Network-Based Information Systems (NBiS), Taipei, Taiwan, 2015
Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: Application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(12):2037–2041
Canny J (1986) A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 6:679–698
Corel Dataset. http://wang.ist.psu.edu/docs/related.shtml. Accessed 10 July 2015
Emerson CW, Siu-Ngan Lam N, Ouattrochi D (1999) Multi-scale fractal analysis of image texture and patterns. Photogrammetric Engineering and Remote Sensing 65:51–62
Facebook Has a Quarter of a Trillion User Photos (2015) http://mashable.com/2013/09/16/facebook-photo-uploads/
Facebook Photo Facts (2015) http://www.adweek.com/socialtimes/wp-content/uploads/sites/2/2013/11/Superfish-VENN-infographcis_final.jpg
Felzenszwalb P, Oberlin JG (2014) Multiscale fields of patterns. In: Advances in neural information processing systems, pp 82–90
Han J, Ma K-K (2002) Fuzzy color histogram and its use in color image retrieval. IEEE Transactions on Image Processing 11(8):944–952
Hansen T (2003) A neural model of early vision: contrast, contours, corners and surfaces. University of Ulm Faculty of Computer Science: 35–42
Hansen T, Neumann H (2004) Neural mechanisms for the robust representation of junctions. Neural Computation 16(5):1013–1037
Heikkilä M, Pietikäinen M, Schmid C (2009) Description of interest regions with local binary patterns. Pattern recognition 42(3):425–436
Huang J, Kumar SR, Mitra M, Zhu W-J, Zabih R (1997) Image indexing using color correlograms. In: Proceedings of the IEEE Computer Society Conference on Computer vision and pattern recognition, 1997, pp 762–768
Huang S, Wang W, Zhang H (2014) Retrieving images using saliency detection and graph matching. In: IEEE International Conference on Image Processing (ICIP) 2014, pp 3087–3091
INRIA Holidays dataset (2015) https://lear.inrialpes.fr/~jegou/data.php. Accessed 10 July 2015
Jégou H, Douze M, Schmid C, Pérez P (2010) Aggregating local descriptors into a compact image representation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, pp 3304–3311
Jégou H, Douze M, Schmid C (2008) Hamming embedding and weak geometry consistency for large scale image search-extended version
Kasutani E, Yamada A (2001) The MPEG-7 color layout descriptor: a compact image feature description for high-speed image/video segment retrieval. In: Proceedings of IEEE International Conference on Image Processing, 2001. pp 674–677
Li J, Wang JZ (2008) Real-time computerized annotation of pictures. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(6):985–1002
Lin R-J, Lin W-S (2014) A computational visual saliency model based on statistics and machine learning. Journal of vision 14(9):1
Liu G-H, Li Z-Y, Zhang L, Xu Y (2011) Image retrieval based on micro-structure descriptor. Pattern Recognition 44(9):2123–2133
Liu G-H, Yang J-Y (2013) Content-based image retrieval using color difference histogram. Pattern Recognition 46(1):188–198
Liu G-H, Zhang L, Hou Y-K, Li Z-Y, Yang J-Y (2010) Image retrieval based on multi-texton histogram. Pattern Recognition 43(7):2380–2389
Lu G, Teng S (1999) A novel image retrieval technique based on vector quantization. In: Proceedings of International Conference on Computational Intelligence for Modeling, Control and Automation, 1999. pp 36–41
Manjunath BS, Ma W-Y (1996) Texture features for browsing and retrieval of image data. IEEE Transactions on Pattern Analysis and Machine Intelligence 18(8):837–842
Manjunath BS, Ohm J-R, Vasudevan VV, Yamada A (2001) Color and texture descriptors. IEEE Transactions on Circuits and Systems for Video Technology, 11(6):703–715
Messing DS, Van Beek P, Errico JH (2001) The mpeg-7 colour structure descriptor: image description using colour and local spatial information. In: Proceedings of the International Conference on Image Processing, 2001, pp 670–673
Mu Y, Yan S, Liu Y, Huang T, Zhou B (2008) Discriminative local binary patterns for human detection in personal album. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, 2008, pp 1–8
Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7):971–987
Olshausen BA, Field DJ (2004) Sparse coding of sensory inputs. Current opinion in neurobiology 14(4):481–487
Ortega M, Rui Y, Chakrabarti K, Porkaew K, Mehrotra S, Huang TS (1998) Supporting ranked boolean similarity queries in MARS. IEEE Transactions on Knowledge and Data Engineering 10(6):905–925
Pass G, Zabih R, Miller J (1997) Comparing images using color coherence vectors. In: Proceedings of the fourth ACM international conference on Multimedia, 1997. ACM, pp 65–73
Portilla J, Simoncelli EP (2000) A parametric texture model based on joint statistics of complex wavelet coefficients. International Journal of Computer Vision 40(1):49–70
Rahimi M, Moghaddam ME (2013) A content-based image retrieval system based on Color Ton Distribution descriptors. SIViP 1–14. doi:10.1007/s11760-013-0506-6
Smeulders AW, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12):1349–1380
Subrahmanyam M, Maheshwari R, Balasubramanian R (2012) Expert system design using wavelet and color vocabulary trees for image retrieval. Expert Systems with Applications 39(5):5104–5114
Subrahmanyam M, Wu QJ, Maheshwari R, Balasubramanian R (2013) Modified color motif co-occurrence matrix for image indexing and retrieval. Computers & Electrical Engineering 39(3):762–774
Takala V, Ahonen T, Pietikäinen M (2005) Block-based methods for image retrieval using local binary patterns. In: Image analysis. Springer, pp 882–891
Terzić K, Rodrigues JM, du Buf JH (2015) BIMP: A real-time biological model of multi-scale keypoint detection in V1. Neurocomputing 150:227–237
van Ginneken B, ter Haar Romeny BM (2003) Multi-scale texture classification from generalized locally orderless images. Pattern Recognition 36(4):899–911
Vinje WE, Gallant JL (2000) Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287(5456):1273–1276
Vipparthi SK, Murala S, Nagar SK (2015) Dual directional multi-motif XOR patterns: A new feature descriptor for image indexing and retrieval. Optik-International Journal for Light and Electron Optics 126(15):1467–1473
Vipparthi SK, Nagar S (2014) Expert image retrieval system using directional local motif XoR patterns. Expert Systems with Applications 41(17):8016–8026
Wang X, Wang Z (2013) A novel method for image retrieval based on structure elements’ descriptor. Journal of Visual Communication and Image Representation 24(1):63–74
Yang M, Kpalma K, Ronsin J (2008) A survey of shape feature extraction techniques. Pattern recognition:43–90
Yao C-H, Chen S-Y (2003) Retrieval of translated, rotated and scaled color textures. Pattern Recognition 36(4):913–929
Zhang D, Lu G (2004) Review of shape representation and description techniques. Pattern recognition 37(1):1–19
Zhang J, Tan T (2002) Brief review of invariant texture analysis methods. Pattern recognition 35(3):735–747
Acknowledgments
This research is supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2013R1A1A2012904).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ahmad, J., Sajjad, M., Rho, S. et al. Multi-scale local structure patterns histogram for describing visual contents in social image retrieval systems. Multimed Tools Appl 75, 12669–12692 (2016). https://doi.org/10.1007/s11042-016-3436-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-016-3436-9