MMM 2015: MultiMedia Modeling pp 403-414 | Cite as
Robust User Community-Aware Landmark Photo Retrieval
Abstract
Given a query photo characterizing a location-aware landmark shot by a user, landmark retrieval is about returning a set of photos ordered in their similarities to the photo. Existing studies on landmark retrieval focus on exploiting location-aware visual features or attributes to conduct a matching process between candidate images and a query image. However, these approaches are based on a hypothesis that a landmark of interest is well-captured and distinctive enough to be distinguished from others. In fact, distinctive landmarks may be biasedly taken due to bad viewpoints or angles. This will discourage the recognition results if a biased query photo is issued. In this paper, we present a novel approach towards landmark retrieval by exploiting the dimension of user community. Our approach in this system consists of three steps. First, we extract communities based on user interest which can characterize a group of users in terms of their social media activities such as user-generated contents/comments. Then, a group of photos that are recommended by the community to which the query user belongs, together with the query photo, can constitute a set of multiple queries. Finally, a pattern mining algorithm is presented to discover regular landmark-specific patterns from this multi-query set. These patterns can faithfully represent the characteristics of a landmark of interest. Experiments conducted on benchmarks are conducted to show the effectiveness of our approach.
Keywords
Landmark Photo Retrieval User Community Query ExpansionPreview
Unable to display preview. Download preview PDF.
References
- 1.Arandjelovic, R., Zisserman, A.: Three things everyone should know to improve object retrieval. In: CVPR (2012)Google Scholar
- 2.Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet distribution. J. Mach. Learn. Res. 3, 993–1022 (2003)MATHGoogle Scholar
- 3.Cheng, Z., Ren, J., Shen, J., Miao, H.: Building a large scale test collection for effective benchmarking of mobile landmark search. In: Li, S., El Saddik, A., Wang, M., Mei, T., Sebe, N., Yan, S., Hong, R., Gurrin, C. (eds.) MMM 2013, Part II. LNCS, vol. 7733, pp. 36–46. Springer, Heidelberg (2013)CrossRefGoogle Scholar
- 4.Chum, O., Philbin, J., Sivic, J., Isard, M., Zisserman, A.: Total recall: automatic query expansion with a generative feature model for object retrieval. In: ICCV (2007)Google Scholar
- 5.Clauset, A., Newman, M., Moore, C.: Finding community structure in very large networks. Physical Review E 70(6), 066111 (2004)Google Scholar
- 6.Crandall, D., Backstrom, L., Huttenlocher, D., Kleinberg, J.: Mapping the world’s photos. In: WWW (2009)Google Scholar
- 7.Doersch, C., Singh, S., Gupta, A., Sivic, J., Efros, A.A.: What makes paris look like paris? ACM Trans. Graph. 31(4), 101 (2012)CrossRefGoogle Scholar
- 8.Fang, Q., Sang, J., Xu, C.: Giant: Geo-informative attributes for location recognition and exploration. In: ACM Multimedia (2013)Google Scholar
- 9.Fernando, B., Tuytelaars, T.: Mining multiple queries for image retrieval: on-the-fly learning of an object-specific mid-level representation. In: ICCV (2013)Google Scholar
- 10.Gao, Y., Wang, M., Zha, Z.-J., Shen, J., Li, X., Wu, X.: Visual-textual joint relevance learning for tag-based social image search. IEEE Trans. Image Processing 22(1), 363–376 (2013)CrossRefMathSciNetGoogle Scholar
- 11.Grunwald, P.D.: The minimum description length principle. The MIT press (2007)Google Scholar
- 12.Hays, J., Efros, A.A.: im2gps: estimating geographic information from a single image. In: CVPR (2008)Google Scholar
- 13.Li, J., Qian, X., Tang, Y.Y., Yang, L., Mei, T.: Gps estimation for places of interest from social users’ uploaded photos. IEEE Trans. Multimedia 15(8), 2058–2071 (2013)CrossRefGoogle Scholar
- 14.Li, Q., Gu, Y., Qian, X.: Lcmkl: latent-communtiy and multi-kernel learning based image annotation. In: CIKM (2013)Google Scholar
- 15.Li, X., Snoek, C.G.M.: Classifying tag relevance with relevant positive and negative examples. In: ACM Multimedia (2013)Google Scholar
- 16.Liu, Q., Yang, Y., Wang, X., Cao, L.: Quality assessment on user generated image for mobile search application. In: Li, S., El Saddik, A., Wang, M., Mei, T., Sebe, N., Yan, S., Hong, R., Gurrin, C. (eds.) MMM 2013, Part II. LNCS, vol. 7733, pp. 1–11. Springer, Heidelberg (2013)CrossRefGoogle Scholar
- 17.Perdoch, O.C.M., Matas, J.: Efficient representation of local geometry for large scale object retrieval. In: CVPR (2009)Google Scholar
- 18.Palsetia, D., Patwary, M.M.A., Zhang, K., Lee, K., Moran, C., Xie, Y., Honbo, D., Agrawal, A., Keng Liao, W., Choudhary, A.: User-interest based community extraction in social networks. In: SNA-KDD Workshop (2012)Google Scholar
- 19.Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Lost in quantization: improving particular object retrieval in large scale image databases. In: CVPR (2008)Google Scholar
- 20.Vreeken, J., Leeuwen, M., Siebes, A.: Krimp: mining itemsets that compress. Data Min. Knowl. Discov. 23, 169–241 (2011)CrossRefMATHMathSciNetGoogle Scholar
- 21.Wang, Y., Lin, X., Wu, L.: Exploiting correlation consensus: Towards subspace clustering for multi-modal data. In: ACM Multimedia (2014)Google Scholar
- 22.Wang, Y., Lin, X., Zhang, Q.: Towards metric fusion on multi-view data: a cross-view based graph random walk approach. In: ACM CIKM (2013)Google Scholar
- 23.Wang, Y., Lin, X., Zhang, Q., Wu, L.: Shifting hypergraphs by probabilistic voting. In: Tseng, V.S., Ho, T.B., Zhou, Z.-H., Chen, A.L.P., Kao, H.-Y. (eds.) PAKDD 2014, Part II. LNCS, vol. 8444, pp. 234–246. Springer, Heidelberg (2014)CrossRefGoogle Scholar
- 24.Wang, Y., Pei, J., Lin, X., Zhang, Q., Zhang, W.: An iterative fusion approach to graph-based semi-supervised learning from multiple views. In: Tseng, V.S., Ho, T.B., Zhou, Z.-H., Chen, A.L.P., Kao, H.-Y. (eds.) PAKDD 2014, Part II. LNCS, vol. 8444, pp. 162–173. Springer, Heidelberg (2014)CrossRefGoogle Scholar
- 25.Wu, L., Cao, X.: Geolocation estimation from two shadow trajectories. In: CVPR (2010)Google Scholar
- 26.Wu, L., Cao, X., Foroosh, H.: Camera calibration and geo-location estimation from two shadow trajectories. Computer Vision and Image Understanding 114(8), 915–927 (2010)CrossRefGoogle Scholar
- 27.Wu, L., Wang, Y., Shepherd, J.: Efficient image and tag co-ranking: A bregman divergence optimization method. In: ACM Multimedia (2013)Google Scholar
- 28.Xue, Y., Qian, X., Zhang, B.: Mobile image retrieval using multi-photos as query. In: ICME (2013)Google Scholar
- 29.Yuan, Z., Sang, J., Liu, Y., Xu, C.: Latent feature learning in social media network. In: ACM Multimedia (2013)Google Scholar
- 30.Zheng, Y., Zhao, M., Song, Y., Adam, H., Buddemeier, U., Bissacco, A., Brucher, F., Chua, T.-S., Neven, H.: Tour the world: building a web-scale landmark recognition engine. In: CVPR (2009)Google Scholar
- 31.Zhu, J., Hoi, S.C.H., Lyu, M.R., Yan, S.: Near-duplicate keyframe retrieval by nonrigid image matching. In: ACM Multimedia (2008)Google Scholar