Multimedia Systems

, Volume 22, Issue 4, pp 395–404 | Cite as

Geo-location driven image tagging via cross-domain learning

  • Weizhi Nie
  • Anan LiuEmail author
  • Zhongyang Wang
  • Yuting Su
Special Issue Paper


With the rapid development of location-based social network, more and more multimedia data are uploaded by users. These data always include large-scale of independent information with both textual and visual contents. To bridge the semantic gap in between, we propose a novel cross-domain learning method for automatic image annotation with geo-location information. First, we propose the topic model-based method for popular concept extraction to adaptively construct cross-domain datasets. Then these concepts are utilized to collect the visual correlation information from Flickr. Finally, we leverage cross-domain learning method for model learning. The comparison experiments on cross-domain datasets are conducted to demonstrate the superiority of the proposed method.


Location-based social network Image annotation Cross-domain data Machine learning Social media 



I would like to express my deep gratitude to Prof. Tat-Seng Chua and the NeXT group in National University of Singapore for helpful discussion. This work was supported in part by the National Natural Science Foundation of China (61100124, 21106095, 61170239, and 61202168), the Grant of Elite Scholar Program of Tianjin University, the Grant of Introducing Talents to Tianjin Normal University (5RL123), the Grant of Introduction of One Thousand High-level Talents in Three Years in Tianjin.


  1. 1.
    Gao, Y., Tang, J., Hong, R., Dai, Q., Chua, T.-S., Jain, R.: W2go: a travel guidance system by automatic landmark ranking. In: ACM Multimedia, pp. 123–132 (2010)Google Scholar
  2. 2.
    Ji, R., Duan, L.-Y., Chen, J., Yao, H., Yuan, J., Rui, Y., Gao, W.: Location discriminative vocabulary coding for mobile landmark search. Int. J. Comput. Vis. 96(3), 290–314 (2012)CrossRefzbMATHGoogle Scholar
  3. 3.
    Gao, Y., Wang, F., Luan, H., Chua, T.: Brand data gathering from live social media streams. In: ACM Conference on Multimedia Retrieval (2014)Google Scholar
  4. 4.
    Wang, H., Huang, H., Ding, C.H.Q.: Image annotation using bi-relational graph of images and semantic labels. In: CVPR, pp. 793–800 (2011)Google Scholar
  5. 5.
    Sang, J., Xu, C., Liu, J.: User-aware image tag refinement via ternary semantic analysis. IEEE Trans. Multimed. 14(3–2), 883–895 (2012)CrossRefGoogle Scholar
  6. 6.
    Gao, Y., Wang, M., Luan, H., Shen, J., Yan, S., Tao, D.: Tag-based social image search with visual-text joint hypergraph learning. In: ACM Multimedia, pp. 1517–1520 (2011)Google Scholar
  7. 7.
    Belani, A.: Vandalism detection in wikipedia: a bag-of-words classifier approach. In: CoRR, vol. abs/1001.0700 (2010)Google Scholar
  8. 8.
    Yang, J., Yan, R., Hauptmann, A.: Cross-domain video concept detection using adaptive svms. In: ACM Multimedia (2007)Google Scholar
  9. 9.
    Jiang, Y., Wang, J., Chang, S., Ngo, C.: Domain adaptive semantic diffusion for large scale context-based video annotation. In: ICCV (2009)Google Scholar
  10. 10.
    III, H.D.: Frustratingly easy domain adaptation. In: CoRR, vol. abs/0907.1815 (2009)Google Scholar
  11. 11.
    Wu, P., Dietterich, T.: Improving svm accuracy by training on auxiliary data sources. In: ICML (2004)Google Scholar
  12. 12.
    Jiang, W., Zavesky, E., Chang, S., Loui, A.: Cross-domain learning methods for high-level visual concept classification. In: ICIP (2008)Google Scholar
  13. 13.
    Roy, S.D., Mei, T., Zeng, W., Li, S.: Towards cross-domain learning for social video popularity prediction. IEEE Trans. Multimed. 15(6):1 (2013)Google Scholar
  14. 14.
    Huang, J., Smola, A., Gretton, A., Borgwardt, K., Schölkopf, B.: Correcting sample selection bias by unlabeled data. In: NIPS (2006)Google Scholar
  15. 15.
    Storkey, A., Sugiyama, M.: Mixture regression for covariate shift. In: NIPS (2006)Google Scholar
  16. 16.
    Fang, Z., Zhang, Z.M.: Discriminative feature selection for multi-view cross-domain learning. In: CIKM, pp. 1321–1330 (2013)Google Scholar
  17. 17.
    Chen, L., Duan, L., Tsang, I.W., Xu, D.: Efficient discriminative learning of class hierarchy for many class prediction. In: ACCV, vol. 1, pp. 274–288 (2012)Google Scholar
  18. 18.
    Bruzzone, L., Marconcini, M.: Domain adaptation problems: a dasvm classification technique and a circular validation strategy. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 770–787 (2010)CrossRefGoogle Scholar
  19. 19.
    Bruzzone, L., Chi, M., Marconcini, M.: Transductive svms for semisupervised classification of hyperspectral data. In: IGARSS, p. 4 (2005)Google Scholar
  20. 20.
    Yuan, Y., Wu, F., Shao, J., Zhuang, Y.: Image annotation by semi-supervised cross-domain learning with group sparsity. J. Visual Commun. Image Rep. 24(2), 95–102 (2013)CrossRefGoogle Scholar
  21. 21.
    Si, S., Tao, D., Wang, M., Chan, K.: Social image annotation via cross-domain subspace learning. Multimed. Tools Appl. 56(1), 91–108 (2012)CrossRefGoogle Scholar
  22. 22.
    Federico, L., Nestor, D., Oscar, C.: Smitag: a social network for semantic annotation of medical images. In: CLEI (2012)Google Scholar
  23. 23.
    Si, S., Tao, D., Chan, K.: Cross-domain web image annotation. In: ICDM Workshops, pp. 184–189 (2009)Google Scholar
  24. 24.
    Denoyer, L., Gallinari, P.: A ranking based model for automatic image annotation in a social network. In: ICWSM (2010)Google Scholar
  25. 25.
    Han, Y., Wu, F., Zhuang, Y.: Multi-label image annotation by structural grouping sparsity. In: Social Media Modeling and Computing, pp. 97–118 (2011)Google Scholar
  26. 26.
    Joshi, D., Luo, J., Yu, J., Lei, P., Gallagher, A.C.: Using geotags to derive rich tag-clouds for image annotation. In: Social Media Modeling and Computing, pp. 239–256 (2011)Google Scholar
  27. 27.
    Gao, Y., Wang, M., Zha, Z., Shen, J., Li, X., Wu, X.: Visual-textual joint relevance learning for tag-based social image search. IEEE Trans. Image Process. 22(1), 363–373 (2013)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Blei, D., Ng, A., Jordan, M.: Latent dirichlet allocation. In: NIPS (2001)Google Scholar
  29. 29.
    Balamurali, A., Mukherjee, S., Malu, A., Bhattacharyya, P.: Leveraging sentiment to compute word similarity. In: CoRR (2012)Google Scholar
  30. 30.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D.A., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. PAMI 32(9), 1627–1645 (2010)CrossRefGoogle Scholar
  31. 31.
    Viola, P.A., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)CrossRefGoogle Scholar
  32. 32.
    Sang, J., Xu, C.: Right buddy makes the difference: an early exploration of social relation analysis in multimedia applications. In: ACM Multimedia, pp. 19–28 (2012)Google Scholar
  33. 33.
    Ji, R., Yao, H., Liu, W., Sun, X., Tian, Q.: Task-dependent visual-codebook compression. IEEE Trans. Image Process. 21(4), 2282–2293 (2012)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: CVPR (2009)Google Scholar
  35. 35.
    Sanromà, G., Alquézar, R., Serratosa, F.: A new graph matching method for point-set correspondence using the em algorithm and softassign. Comput. Vis. Image Underst. 116(2), 292–304 (2012)CrossRefGoogle Scholar
  36. 36.
    Pan, S.J., Kwok, J., Yang, Q.: Transfer learning via dimensionality reduction. In: AAAI (2008)Google Scholar
  37. 37.
    Taylor, M.E., Stone, P.: Cross-domain transfer for reinforcement learning. In: ICML, pp. 879–886 (2007)Google Scholar
  38. 38.
    Duan, L., Tsang, I., Xu, D.: Domain transfer multiple kernel learning. IEEE Trans. Pattern Anal. Mach. Intell. 34(3), 465–479 (2012)CrossRefGoogle Scholar
  39. 39.
    Lanckriet, G., Cristianini, N., Bartlett, P., Ghaoui, L., Jordan, M.: Learning the kernel matrix with semidefinite programming. J. Mach. Learn. Res. 5, 27–72 (2004)MathSciNetzbMATHGoogle Scholar
  40. 40.
    Liu, X., Wang, L., Yin, J., Liu, L.: Incorporation of radius-info can be simple with simplemkl. Neurocomputing 89, 30–38 (2012)CrossRefGoogle Scholar
  41. 41.
    Sonnenburg, S., Rätsch, G., Schäfer, C., Schölkopf, B.: Large scale multiple kernel learning. J. Mach. Learn. Res 7, 1531–1565 (2006)MathSciNetzbMATHGoogle Scholar
  42. 42.
    Ji, R., Gao, Y., Zhong, B., Yao, H., Tian, Q.: Mining flickr landmarks by modeling reconstruction sparsity. TOMCCAP 7, 31 (2011)Google Scholar
  43. 43.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR, vol. 1, pp. 886–893 (2005)Google Scholar
  44. 44.
    Mäenpää, T., Ojala, T., Pietikäinen, M., Soriano, M.: Robust texture classification by subsets of local binary patterns. In: ICPR (2000)Google Scholar
  45. 45.
    Zhang, W., Lu, Y., Xue, X., Fan, J.: Automatic image annotation with weakly labeled dataset. In: ACM Multimedia, pp. 1185–1188 (2011)Google Scholar
  46. 46.
    Liu, X., Gao, Y., Ji, R., Chang, S., Huang, T. S.: Localizing web videos from heterogeneous images. In: AAAI (late-breaking developments) (2013)Google Scholar
  47. 47.
    Gao, Y., Dai, Q.: Clip based video summarization and ranking. In: CIVR, pp. 135–140 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Weizhi Nie
    • 1
  • Anan Liu
    • 1
    Email author
  • Zhongyang Wang
    • 1
  • Yuting Su
    • 1
  1. 1.The department of Electronics Information EngineeringTianjin UniversityTianjinChina

Personalised recommendations