Abstract
Browsing, searching and retrieving images from large databases based on low level color or texture visual features have been widely studied in recent years but are also often limited in terms of usefulness. In this paper, we propose a new framework that allows users to effectively browse and search in large image database based on their segmentation-based descriptive content and, more precisely, based on the geometrical layout and shapes of the different objects detected and segmented in the scene. This descriptive information, provided at a higher level of abstraction, can be a significant and complementary information which helps the user to browse through the collection in an intuitive and efficient manner. In addition, we study and discuss various ways and tools for efficiently clustering or for retrieving a specific subset or class of images in terms of segmentation-based descriptive content which can also be used to efficiently summarize the content of the image database. Experiments conducted on the Berkeley Segmentation Datasets show that this new framework can be effective in supporting image browsing and retrieval tasks.
Similar content being viewed by others
Notes
A region is a set of connected pixels belonging to the same class and a class, a set of pixels possessing similar textural characteristics.
Our approach is tolerant to different types of image degradation (e.g., noise, blur, distortions) insofar as the segmentation method is able to give a segmentation map which is robust enough for these types of degradation.
It is worth mentioning that we could also exploit all the set of ground truth segmentations for each image by computing an average VoI distance computed across all the existing ground truth segmentations.
Source code (in C++ language) of our algorithm with the set of clustering and mapping results for each clustering strategy are publicly available at the following http address http://www.iro.umontreal.ca/~mignotte/ResearchMaterial/scvoi.html
At this stage, it is important to recall that Google search image will not seek in a specific database (with 300 images as the BSDS300), but on the whole web image database and therefore it will have more choice to refine its search process which will be more accurate and efficient.
This map has also been estimated on 3 other image databases, namely; 1) the Weizmann database (1 & 2 objects) (200 images), 2) the Microsoft Research Cambridge Object Recognition Image Database (MSRC) (591 images), 3) The Stanford Background Dataset (DAGS) (715 images). The references of these image databases and the obtained visualization maps are publicly available (with the source code of our algorithm) at the following http address: www.iro.umontreal.ca/~mignotte/ResearchMaterial/scvoi.html
References
Alvarez C, Id-Oumohmed A, Mignotte M, Nie J-Y (2004) Toward cross-language and cross-media image retrieval. In: 5th Workshop on Cross Langage Evaluation Forum, CLEF 2004 Lecture notes in Computer science, Multilingual information access for text, speech and images, vol 3491, Bath, United Kingdom, pp 676–688
Borg I, Groenen P (2005) Modern Multidimensional Scaling: Theory and Applications. Springer
Banks S (1990) Signal processing image processing and pattern recognition. Prentice Hall
Bartolini I, Ciaccia P, Patella M (2006) Adaptively browsing image databases with PIBE. Multimed Tools Appl 31(3):269–286
Cayton L, Dasgupta S (2006) Robust euclidean embedding. In: Proceedings of the twentythird International Conference on Machine learning. ACM Press, pp 169–176
Ghosh S, Pfeiffer J, Mulligan J (2009) A general framework for reconciling multiple weak segmentations of an image. In: Proceedings of the Workshop on Applications of Computer Vision, (WACV’09), Utah, USA, pp 1–8
Heesch D (2008) A survey of browsing models for content based image retrieval. Multimed Tools Appl 40(2):261–284
Id-Oumohmed A, Mignotte M, Nie J-Y (2005) Semantic-based cross-media image retrieval. In: 3rd International Conference on Advances in Pattern Recognition, ICAPR’05, Lecture Notes in Computer Science, volume LNCS 3686, Pattern Recognition and Data Mining, Proceedings Part 2, Bath, United Kingdom (UK), pp 414–423
Jiang Y, Zhou Z-H (2004) SOM ensemble-based image segmentation. Neural Process Lett 20(3):171–178
Keuchel J, Kuttel D (2006) Efficient combination of probabilistic sampling approximations for robust image segmentation. In: 28th Annual Symposium of the German Association for Pattern Recognition. DAGM-Symposium, Lecture Notes in Computer Science, Berlin, Germany, pp 41–50
Lloyd SP (1982) Least squares quantization in PCM. IEEE Trans Inform Theory 28(2):129–136
Leelanupab T, Feng Y, Stathopoulos V, Jose JM (2009) A simulated user study of image browsing using high-level classification. In: Semantic Multimedia, 4th International Conference on Semantic and Digital Media Technologies, SAMT 2009, Graz, Austria, December 2-4, 2009, Proceedings, pp 3–15
Meila M (2005) Comparing clusterings - an axiomatic view. In: Proceedings of the 2005 22nd International Conference on Machine Learning (ICML’05), Bonn, Germany, pp 577–584
Meila M (2007) Comparing clusterings–an information based distance. J Multivar Anal 98(5):873–895
Mignotte M (2008) Segmentation by fusion of histogram-based K-means clusters in different color spaces. IEEE Trans Image Process 17(5):780–787
Mignotte M (2010) A label field fusion Bayesian model and its penalized maximum Rand estimator for image segmentation. IEEE Trans Image Process 19(6):1610–1624
Mignotte M (2014) A label field fusion model with a variation of information estimator for image segmentation. Fusion Information
Ma Y, Derksen H, Hong W, Wright J (2007) Segmentation of multivariate mixed data via lossy coding and compression. IEEE Trans Pattern Anal Mach Intell 29(9):1546–1562
Mignotte M (2011) A de-texturing and spatially constrained K-means approach for image segmentation. Pattern Recogn lett 32(2):359–367
Mignotte M (2011) MDS-based multiresolution nonlinear dimensionality reduction model for color image segmentation. IEEE Trans Neural Netw 22(3):447–460
Mignotte M (2014) A non-stationary MRF model for image segmentation from a soft boundary map. Pattern Anal Appl 17(1):129–139
Mignotte M (2004) Nonparametric multiscale energy-based model and its application in some imagery problems. IEEE Trans Pattern Anal Mach Intell 26(2):184–197
Mignotte M (2010) A multiresolution markovian fusion model for the color visualization of hyperspectral images. IEEE Trans Geosci Remote Sens 48(12):4236–4247
Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the 8th International Conference on Computer Vision (ICCV’01), vol 2, pp 416–423
Rachid H, Mignotte M (2009) A hierarchical graph-based Markovian clustering approach for the unsupervised segmentation of textured color images. In: Proceedings of the 16th IEEE International Conference on Image Processing (ICIP’09), pp 1365–1368
Schaefer G (2012) Interactive browsing of image repositories - (invited paper). In: Computer Vision and Graphics - International Conference, ICCVG 2012, Warsaw, Poland, September 24-26, 2012. Proceedings, pp 236–244
Schaefer G (2010) A next generation browsing environment for large image repositories. Multimed Tools Appl 47(1):105–120
Torgerson W (1952) Multidimensional scaling: I. theory and method. Psychometrika 17:401–419
Urban J, Jose J, Van Rijsbergen C (2006) An adaptive technique for content-based image retrieval. Multimed Tools Appl 31(1):1–28
Vega-Pons S, Ruiz-Shulcloper J (2011) A survey of clustering ensemble algorithms. Int J Pattern Recogn Artif Intell, IJPRAI 25(3):337–372
Wattuya P, Rothaus K, Prani J-S, Jiang X (2008) A random walker based approach to combining multiple segmentations. In: Proceedings of the 19th International Conference on Pattern Recognition (ICPR’08), Florida, USA, pp 1–4
Yang AY, Wright J, Sastry S, Ma Y (2008) Unsupervised segmentation of natural images via lossy data compression. Comput Vis Image Underst 110(2):212–225
Young G, Householder A (1938) Discussion of a set of points in terms of their mutual distances. Psychometrika 3
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Khlif, A., Mignotte, M. Segmentation data visualizing and clustering. Multimed Tools Appl 76, 1531–1552 (2017). https://doi.org/10.1007/s11042-015-3148-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-015-3148-6