AMR 2012: Adaptive Multimedia Retrieval: Semantics, Context, and Adaptation pp 116-129 | Cite as
Representativeness and Diversity in Photos via Crowd-Sourced Media Analysis
Abstract
In this paper we address the problem of user-adapted image retrieval. First, we provide a survey of the performance of the existing social media retrieval platforms and highlight their limitations. In this context, we propose a hybrid, two step, machine and human automated media analysis approach. It aims to improve retrieval relevance by selecting a small number of representative and diverse images from a noisy set of candidate images (e.g. the case of Internet media). In the machine analysis step, to ensure representativeness, images are re-ranked according to the similarity to the “most common” image in the set. Further, to ensure also the diversity of the results, images are clustered and the best ranked images among the most representative in each cluster are retained. The human analysis step aims to bridge further inherent descriptor semantic gap. The retained images are further refined via crowd-sourcing which adapts the results to human. The method was validated in the context of the retrieval of images with monuments using a data set of more than 25.000 images retrieved from various social image search platforms.
Keywords
Image Retrieval Relevance Feedback Image Search Automate Image Analysis Query ObjectNotes
Acknowledgments
This research is partially supported by the CUbRIK project, an IP funded within the FP7/2007–2013 under grant agreement n287704 and by the Romanian Sectoral Operational Programme Human Resources Development 2007–2013 through the Financial Agreement POSDRU/89/1.5/S/62557.
References
- 1.Bartolini, I., Ciaccia, P.: Multi-dimensional keyword-based image annotation and search. In: ACM International Workshop on Keyword Search on Structured Data, New York, USA, pp. 1–6 (2010)Google Scholar
- 2.Kennedy, L.S., Naaman, M.: Generating diverse and representative image search results for landmarks. In: International Conference on World Wide Web, New York, NY, USA, pp. 297–306 (2008)Google Scholar
- 3.Popescu, A., Moëllic, P.A., Kanellos, I., Landais, R.: Lightweight web image reranking. In: ACM International Conference on Multimedia, New York, NY, USA, pp. 657–660 (2009)Google Scholar
- 4.Fergus, R., Perona, P., Zisserman, A.: A visual category filter for Google images. In: Pajdla, T., Matas, J.G. (eds.) ECCV 2004. LNCS, vol. 3021, pp. 242–256. Springer, Heidelberg (2004)CrossRefGoogle Scholar
- 5.Nguyen, N.V., Ogier, J.M., Tabbone, S., Boucher, A.: Text retrieval relevance feedback techniques for bag of words model in CBIR. In: International Conference on Machine Learning and Pattern Recognition (2009)Google Scholar
- 6.Larson, M., Rae, A., Demarty, C.H., Kofler, C., Metze, F., Troncy, R., Mezaris, V., Jones, G.J. In: MediaEval 2011 Workshop at Interspeech 2011, vol. 807, CEUR-WS.org, 1–2 September 2011Google Scholar
- 7.Crandall, D.J., Backstrom, L., Huttenlocher, D., Kleinberg, J.: Mapping the world’s photos. In: International Conference on World Wide Web, New York, NY, USA, pp. 761–770 (2009)Google Scholar
- 8.Hays, J., Efros, A.A.: Im2gps: estimating geographic information from a single image. In: IEEE International Conference on Computer Vision and Pattern Recognition (2008)Google Scholar
- 9.Van de Weijer, J., Schmid, C., Verbeek, J., Larlus, D.: Learning color names for real-world applications. IEEE Trans. Image Process. 18(7), 1512–1523 (2009)MathSciNetCrossRefGoogle Scholar