Query by Semantic Example
A solution to the problem of image retrieval based on query-by-semantic-example (QBSE) is presented. QBSE extends the idea of query-by-example to the domain of semantic image representations. A semantic vocabulary is first defined, and a semantic retrieval system is trained to label each image with the posterior probability of appearance of each concept in the vocabulary. The resulting vector is interpreted as the projection of the image onto a semantic probability simplex, where a suitable similarity function is defined. Queries are specified by example images, which are projected onto the probability simplex. The database images whose projections on the simplex are closer to that of the query are declared its closest neighbors. Experimental evaluation indicates that 1) QBSE significantly outperforms the traditional query-by-visual-example paradigm when the concepts in the query image are known to the retrieval system, and 2) has equivalent performance even in the worst case scenario of queries composed by unknown concepts.
Unable to display preview. Download preview PDF.
- 2.Blei, D., Jordan, M.I.: Modeling annotated data. In: Proceedings of the 26th Intl. ACM SIGIR Conf., pp. 127–134 (2003)Google Scholar
- 3.Lavrenko, V., Manmatha, R., Jeon, J.: A model for learning the semantics of pictures. In: NIPS (2003)Google Scholar
- 4.Picard, R.: Digital Libraries: Meeting Place for High-Level and Low-Level Vision. In: Li, S., Teoh, E.-K., Mital, D., Wang, H. (eds.) ACCV 1995. LNCS, vol. 1035. Springer, Heidelberg (1996)Google Scholar
- 5.Feng, S.L., Manmatha, R., Lavrenko, V.: Multiple bernoulli relevance models for image and video annotation. In: CVPR (2004)Google Scholar
- 6.Smeulders, A., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval: the end of the early years. PAMI 22(12), 1349–1380 (2000)Google Scholar
- 7.Vasconcelos, N.: Image Indexing with Mixture Hierarchies. In: CVPR, Kawai, Hawaii (2001)Google Scholar
- 9.Carneiro, G., Vasconcelos, N.: Formulating Semantics Image Annotation as a Supervised Learning Problem. In: CVPR, San Diego (2005)Google Scholar
- 10.Vasconcelos, N.: Minimum Probability of Error Image Retrieval. IEEE Transactions on Signal Processing 52(8) (2004)Google Scholar