Geotag Propagation with User Trust Modeling

Part of the Computer Communications and Networks book series (CCN)


The amount of information that people share on social networks is constantly increasing. People also comment, annotate, and tag their own content (videos, photos, notes, etc.), as well as the content of others. In many cases, the content is tagged manually. One way to make this time-consuming manual tagging process more efficient is to propagate tags from a small set of tagged images to the larger set of untagged images automatically. In such a scenario, however, a wrong or a spam tag can damage the integrity and reliability of the automated propagation system. Users may make mistakes in tagging, or irrelevant tags and content may be added maliciously for advertisement or self-promotion. Therefore, a certain mechanism insuring the trustworthiness of users or published content is needed. In this chapter, we discuss several image retrieval methods based on tags, various approaches to trust modeling and spam protection in social networks, and trust modeling in geotagging systems. We then consider a specific example of automated geotag propagation system that adopts a user trust model. The tag propagation in images relies on the similarity between image content (famous landmarks) and its context (associated geotags). For each tagged image, similar untagged images are found by the robust graph-based object duplicate detection, and the known tags are propagated accordingly. The user trust value is estimated based on a social feedback from the users of the photo-sharing system, and only tags from trusted users are propagated. This approach demonstrates that a practical tagging system significantly benefits from the intelligent combination of efficient propagation algorithm and a user-centered trust model.


Training Image Scale Invariant Feature Transform Legitimate User User Trust Famous Landmark 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported by the Swiss National Foundation for Scientific Research in the framework of NCCR Interactive Multimodal Information Management (IM2), the Swiss National Science Foundation Grant “Multimedia Security” (number 200020-113709), and partially supported by the European Network of Excellence PetaMedia (FP7/2007-2011).


  1. 1.
    Ballard, D.H.: Generalizing the Hough transform to detect arbitrary shapes. Pattern Recogn. 13(2), 111–122 (1981)zbMATHCrossRefGoogle Scholar
  2. 2.
    Barnett, E.: 3.4 billion photographs on Google+ in 100 days. Scholar
  3. 3.
    Budanitsky, A., Hirst, G.: Evaluating WordNet-based measures of lexical semantic relatedness. Comput. Linguist. 32(1), 13–47 (2006)zbMATHCrossRefGoogle Scholar
  4. 4.
    Cilibrasi, R.L., Vitanyi, P.M.B.: The Google similarity distance. IEEE Trans. Knowl. Data Eng. 19(3), 370–383 (2007)CrossRefGoogle Scholar
  5. 5.
    Fetterly, D., Manasse, M., Najork, M.: Spam, damn spam, and statistics: using statistical analysis to locate spam web pages. In: Proceedings of the ACM WebDB, Paris, pp. 1–6 (2004)Google Scholar
  6. 6.
    Gammeter, S., Bossard, L., Quack, T., van Gool, L.: I know what you did last summer: object level auto-annotation of holiday snaps. In: Proceedings of the ICCV, Kyoto, pp. 614–621 (2009)Google Scholar
  7. 7.
    Gyongyi, Z., Garcia-Molina, H., Pedersen, J.: Combating web spam with TrustRank. In: Proceedings of the VLDB, Toronto, pp. 576–587 (2004)Google Scholar
  8. 8.
    Hays, J., Efros, A.A.: im2gps: Estimating geographic information from a single image. In: Proceedings of the IEEE CVPR, Anchorage, pp. 1–8 (2008)Google Scholar
  9. 9.
    Heymann, P., Koutrika, G., Garcia-Molina, H.: Fighting spam on social web sites: a survey of approaches and future challenges. IEEE Internet Comput. 11(6), 36–45 (2007)CrossRefGoogle Scholar
  10. 10.
    Hollenstein, L., Purves, R.: Exploring place through user-generated content: using Flickr to describe city cores. J. Spat. Inf. Sci. 1–29 (2010)Google Scholar
  11. 11.
    International Press Telecommunications Council: IPTC Photo Metadata Standard, IPTC Core 1.1 and IPTC Extension 1.1. Technical report (2009)Google Scholar
  12. 12.
    Ivanov, I., Vajda, P., Lee, J.S., Ebrahimi, T.: In tags we trust: trust modeling in social tagging of multimedia content. IEEE Signal Proc. Mag. 29(2), 98–107 (2012)CrossRefGoogle Scholar
  13. 13.
    Ivanov, I., Vajda, P., Lee, J.S., Goldmann, L., Ebrahimi, T.: Geotag propagation in social networks based on user trust model. MTAP 56(1), 155–177 (2012)Google Scholar
  14. 14.
    Jøsang, A., Ismail, R., Boyd, C.: A survey of trust and reputation systems for online service provision. Decision Support Syst. 43(2), 618–644 (2007)CrossRefGoogle Scholar
  15. 15.
    Kennedy, L.S., Chang, S.F., Kozintsev, I.V.: To search or to label?: predicting the performance of search-based automatic image classifiers. In: Proceedings of the ACM MIR, Santa Barbara, pp. 249–258 (2006)Google Scholar
  16. 16.
    Kennedy, L.S., Naaman, M.: Generating diverse and representative image search results for landmarks. In: Proceedings of the WWW, Beijing, pp. 297–306 (2008)Google Scholar
  17. 17.
    Kessler, S.: Mashable Infographics – Facebook Photos by the Numbers.
  18. 18.
    Kleinberg, J.M.: Authoritative sources in a hyperlinked environment. JACM 46(5), 604–632 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Koutrika, G., Effendi, F.A., Gyöngyi, Z., Heymann, P., Garcia-Molina, H.: Combating spam in tagging systems: an evaluation. ACM TWEB 2(4), 22:1–22:34 (2008)Google Scholar
  20. 20.
    Krause, B., Schmitz, C., Hotho, A., G., S.: The anti-social tagger: detecting spam in social bookmarking systems. In: Proceedings of the ACM AIRWeb, Beijing, pp. 61–68 (2008)Google Scholar
  21. 21.
    Krestel, R., Chen, L.: Using co-occurence of tags and resources to identify spammers. In: Proceedings of the ECML PKDD, Antwerp, pp. 38–46 (2008)Google Scholar
  22. 22.
    Liu, K., Fang, B., Zhang, Y.: Detecting tag spam in social tagging systems with collaborative knowledge. In: Proceedings of the IEEE FSKD, Tianjin, pp. 427–431 (2009)Google Scholar
  23. 23.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  24. 24.
    Luo, J., Joshi, D., Yu, J., Gallagher, A.: Geotagging in multimedia and computer vision – a survey. MTAP 51(1), 187–211 (2011)Google Scholar
  25. 25.
    Markines, B., Cattuto, C., Menczer, F.: Social spam detection. In: Proceedings of the ACM AIRWeb, Madrid, pp. 41–48 (2009)Google Scholar
  26. 26.
    Marlow, C., Naaman, M., Boyd, D., Davis, M.: Ht06, tagging paper, taxonomy, flickr, academic article, to read. In: Proceedings of the ACM HT, Odense, pp. 31–40 (2006)Google Scholar
  27. 27.
    Marti, S., Garcia-Molina, H.: Taxonomy of trust: Categorizing P2P reputation systems. Comput. Netw. 50(4), 472–484 (2006)zbMATHCrossRefGoogle Scholar
  28. 28.
    Mikolajczyk, K., Schmid, C.: An affine invariant interest point detector. In: Proceedings of the ECCV, Copenhagen, pp. 128–142 (2002)Google Scholar
  29. 29.
    Mori, G., Malik, J.: Recognizing objects in adversarial clutter: breaking a visual CAPTCHA. In: Proceedings of the IEEE CVPR, Madison, pp. I–134–I–141 (2003)Google Scholar
  30. 30.
    Nister, D., Stewenius, H.: Robust scalable recognition with a vocabulary tree. In: Proceedings of the IEEE CVPR, New York, pp. 2161–2168 (2006)Google Scholar
  31. 31.
    Noll, M.G., Yeung, C.A., Gibbins, N., Meinel, C., Shadbolt, N.: Telling experts from spammers: expertise ranking in folksonomies. In: Proceedings of the ACM SIGIR, Boston, pp. 612–619 (2009)Google Scholar
  32. 32.
    Parr, B.: Mashable Infographics – Facebook by the Numbers.
  33. 33.
    Quack, T., Leibe, B., Van Gool, L.: World-scale mining of objects and events from community photo collections. In: Proceedings of the IEEE CIVR, Niagara Falls, pp. 47–56 (2008)Google Scholar
  34. 34.
    Sahami, M., Dumais, S., Heckerman, D., Horvitz, E.: A bayesian approach to filtering junk e-mail. Technical report, WS-98-05 (1998)Google Scholar
  35. 35.
    Technical Standardization Committee on AV & IT Storage Systems and Equipment: Exchangeable image file format for digital still cameras: Exif Version 2.2. Technical report, JEITA CP-3451 (2002)Google Scholar
  36. 36.
    Thomason, A.: Blog spam: A review. In: Proceedings of the CEAS, Mountain View (2007)Google Scholar
  37. 37.
    Vajda, P., Goldmann, L., Ebrahimi, T.: Analysis of the limits of graph-based object duplicate detection. In: Proceedings of the Symposium on Multimedia, San Diego, pp. 600–605 (2009)Google Scholar
  38. 38.
    Vajda, P., Ivanov, I., Goldmann, L., Lee, J.S., Ebrahimi, T.: Robust duplicate detection of 2D and 3D objects. IJMDEM 1(3), 19–40 (2010)Google Scholar
  39. 39.
    von Ahn, L., Blum, M., Hopper, N.J., Langford, J.: CAPTCHA: using hard AI problems for security. In: Proceedings of the Eurocrypt, Warsaw, pp. 294–311 (2003)Google Scholar
  40. 40.
    von Ahn, L., Maurer, B., Mcmillen, C., Abraham, D., Blum, M.: reCAPTCHA: Human-based character recognition via web security measures. Science 321(5895), 1465–1468 (2008)Google Scholar
  41. 41.
    Whitby, A., Jøsang, A., Indulska, J.: Filtering out unfair ratings in bayesian reputation systems. In: Proceedings of the IEEE AAMAS, New York, pp. 106–117 (2004)Google Scholar
  42. 42.
    Wikimedia Foundation Inc.: Wikipedia–Flickr.
  43. 43.
    Wu, C.T., Cheng, K.T., Zhu, Q., Wu, Y.L.: Using visual features for anti-spam filtering. In: Proceedings of the IEEE ICIP, Genoa, vol. 3, pp. III – 509–512 (2005)Google Scholar
  44. 44.
    Xu, Z., Fu, Y., Mao, J., Su, D.: Towards the semantic web: collaborative tag suggestions. In: Proceedings of the ACM WWW, Santa Barbara (2006)Google Scholar
  45. 45.
    Yahoo! Inc.: Flickr – Tags.
  46. 46.
    Yang, Y., Sun, Y.L., Kay, S., Yang, Q.: Defending online reputation systems against collaborative unfair raters through signal modeling and trust. In: Proceedings of the ACM SAC, Honolulu, pp. 1308–1315 (2009)Google Scholar
  47. 47.
    Zheng, Y.T., Zhao, M., Song, Y., Adam, H., Buddemeier, U., Bissacco, A., Brucher, F., Chua, T., Neven, H.: Tour the World: building a web-scale landmark recognition engine. In: Proceedings of the IEEE CVPR, Miami, pp. 1085–1092 (2009)Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  1. 1.Multimedia Signal Processing Group (MMSPG)École Polytechnique Fédérale de Lausanne (EPFL)LausanneSwitzerland
  2. 2.School of Integrated TechnologyYonsei UniversityIncheonSouth Korea

Personalised recommendations