Personalized Annotation for Mobile Photos Based on User’s Social Circle

  • Yanhui Hong
  • Tiandi Chen
  • Kang Zhang
  • Lifeng SunEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9516)


For mobile photos annotation, users are more interested in the context information behind the photos. The user’s social circle can provide valuable information for it. However, the accompanying textual information of social network is sparse and ambiguous in nature. In this paper, we propose a personalized annotation framework for mobile photos leveraging the user’s social circle. To address the unreliability problem of social network, we present an algorithm to generate reliable tags for social photos before assigning tags to the user’s unlabeled photos. In the tag generation stage, a multi-modality hierarchical clustering algorithm is performed to detect social events. Besides, we use “Album” instead of individual photo as the basic unit for clustering. Finally, we employ a weighted nearest neighbor model for label propagation. We evaluate our framework on a large-scale, real-world dataset from Renren, the largest Facebook-like social network in China. Our evaluation results show promising results of our proposed framework.


Personalized annotation Social network Event detection 



This work was part-funded by 973 Program under Grant No. 2011CB302206, National Natural Science Foundation of China under Grant No. 61272231, 61472204, Beijing Key Laboratory of Networked Multimedia.


  1. 1.
    Becker, H., Naaman, M., Gravano, L.: Learning similarity metrics for event identification in social media. In: Proceedings of the Third ACM International Conference on Web Search and Data Mining, pp. 291–300. ACM (2010)Google Scholar
  2. 2.
    Cao, L., Luo, J., Kautz, H., Huang, T.S.: Image annotation within the context of personal photo collections using hierarchical event and scene models. IEEE Trans. Multimedia 11(2), 208–219 (2009)CrossRefGoogle Scholar
  3. 3.
    Cheng, A.J., Lin, F.E., Kuo, Y.H., Hsu, W.H.: Gps, compass, or camera? Investigating effective mobile sensors for automatic search-based image annotation. In: Proceedings of the International Conference on Multimedia, pp. 815–818. ACM (2010)Google Scholar
  4. 4.
    Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image retrieval: Ideas, influences, and trends of the new age. ACM Comput. Surv. (CSUR) 40(2), 5 (2008)CrossRefGoogle Scholar
  5. 5.
    Gallagher, A.C., Neustaedter, C.G., Cao, L., Luo, J., Chen, T.: Image annotation using personal calendars as context. In: Proceedings of the 16th ACM International Conference on Multimedia, pp. 681–684. ACM (2008)Google Scholar
  6. 6.
    Guillaumin, M., Mensink, T., Verbeek, J., Schmid, C.: Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 309–316. IEEE (2009)Google Scholar
  7. 7.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  8. 8.
    Lei, C., Liu, D.: Image annotation via social diffusion analysis with common interests. In: 2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp. 1–6. IEEE (2014)Google Scholar
  9. 9.
    Li, X., Gavves, E., Snoek, C.G., Worring, M., Smeulders, A.W.: Personalizing automated image annotation using cross-entropy. In: Proceedings of the 19th ACM International Conference on Multimedia, pp. 233–242. ACM (2011)Google Scholar
  10. 10.
    Liu, J., Li, M., Liu, Q., Lu, H., Ma, S.: Image annotation via graph learning. Pattern Recogn. 42(2), 218–228 (2009)zbMATHCrossRefGoogle Scholar
  11. 11.
    Liu, J., Li, Z., Tang, J., Jiang, Y., Lu, H.: Personalized geo-specific tag recommendation for photos on social websites. IEEE Trans. Multimedia 16(3), 588–600 (2014)CrossRefGoogle Scholar
  12. 12.
    Liu, X., Qian, X., Lu, D., Hou, X., Wang, L.: Personalized tag recommendation for flickr users. In: 2014 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2014)Google Scholar
  13. 13.
    Makadia, A., Pavlovic, V., Kumar, S.: A new baseline for image annotation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 316–329. Springer, Heidelberg (2008) CrossRefGoogle Scholar
  14. 14.
    McAuley, J., Leskovec, J.: Image labeling on a network: using social-network metadata for image classification. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part IV. LNCS, vol. 7575, pp. 828–841. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  15. 15.
    Niu, Z., Hua, G., Gao, X., Tian, Q.: Semi-supervised relational topic model for weakly annotated image recognition in social media. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4233–4240. IEEE (2014)Google Scholar
  16. 16.
    Petkos, G., Papadopoulos, S., Kompatsiaris, Y.: Social event detection using multimodal clustering and integrating supervisory signals. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, p. 23. ACM (2012)Google Scholar
  17. 17.
    Reuter, T., Papadopoulos, S., Mezaris, V., Cimiano, P.: Reseed: social event detection dataset. In: Proceedings of the 5th ACM Multimedia Systems Conference, pp. 35–40. ACM (2014)Google Scholar
  18. 18.
    Rodriguez, A., Laio, A.: Clustering by fast search and find of density peaks. Science 344(6191), 1492–1496 (2014)CrossRefGoogle Scholar
  19. 19.
    Tang, J., Hong, R., Yan, S., Chua, T.S., Qi, G.J., Jain, R.: Image annotation by k nn-sparse graph-based label propagation over noisily tagged web images. ACM Trans. Intell. Syst. Technol. (TIST) 2(2), 14 (2011)Google Scholar
  20. 20.
    Ulges, A., Worring, M., Breuel, T.: Learning visual contexts for image annotation from flickr groups. IEEE Trans. Multimedia 13(2), 330–341 (2011)CrossRefGoogle Scholar
  21. 21.
    Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., Gong, Y.: Locality-constrained linear coding for image classification. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3360–3367. IEEE (2010)Google Scholar
  22. 22.
    Wu, Z., Aizawa, K.: Building friend wall for local photo repository by using social attribute annotation. J. Multimedia 9(1), 4–13 (2014)CrossRefGoogle Scholar
  23. 23.
    Zhang, D., Islam, M.M., Lu, G.: A review on automatic image annotation techniques. Pattern Recogn. 45(1), 346–362 (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Yanhui Hong
    • 1
  • Tiandi Chen
    • 1
  • Kang Zhang
    • 1
  • Lifeng Sun
    • 1
    Email author
  1. 1.Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and TechnologyTsinghua UniversityBeijingChina

Personalised recommendations