Skip to main content
SpringerLink
Log in

Using Geotags to Derive Rich Tag-Clouds for Image Annotation

  • Chapter
Social Media Modeling and Computing

Abstract

Geotagging has become popular for many multimedia applications. In this chapter, we present an integrated and intuitive system for location-driven tag suggestion, in the form of tag-clouds, for geotagged photos. Potential tags from multiple sources are extracted and weighted. Sources include points of interest (POI) tags from a public Geographic Names Information System (GNIS) database, community tags from Flickr® pictures, and personal tags shared through users’ own, family, and friends’ photo collections. To increase the effectiveness of GNIS POI tags, bags of place-name tags are first retrieved, clustered, and then re-ranked using a combined tf-idf and spatial distance criteria. The community tags from photos taken in the vicinity of the input geotagged photo are ranked according to distance and visual similarity to the input photo. Personal tags from other personally related photos inherently carry a significant weight due more to their high relevance than to both the generic place-name tags and community tags, and are ranked by weights that decay over time and distance differences. Finally, a rich set of the most relevant location-driven tags is presented to the user in the form of individual tag clouds under the three mentioned source categories. The tag clouds act as intuitive suggestions for tagging an input image. We also discuss quantitative and qualitative findings from a user study that we conducted. Evaluation has revealed the respective benefits of the three categories toward the effectiveness of the integrated tag suggestion system.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahern, S., Davis, M., Eckles, D., King, S., Naaman, M., Nair, R., Spasojevic, M., Yang, J.: Zonetag: Designing context aware mobile media capture to increase participation. In: Proceedings of Workshop on Pervasive Image Capture and Sharing (2006)

    Google Scholar 

  2. Ames, M., Naaman, M.: Why we tag: Motivations for annotation in mobile and online media. In: Proceedings of ACM SIGCHI Conference on Human Factors in Computing Systems (2007)

    Google Scholar 

  3. Cao, L., Luo, J., Kautz, H., Huang, T.: Annotating collections of geotagged photos using hierarchical event and scene models. In: Proceedings of IEEE CVPR (2008)

    Google Scholar 

  4. Cao, L., Luo, J., Huang, T.S.: Annotating photo collections by label propagation according to multiple proximity cues. In: Proceedings of ACM Multimedia (2008)

    Google Scholar 

  5. Cao, L., Yu, J., Luo, J., Huang, T.S.: Enhancing semantic and geographic annotation of Web images via logistic canonical correlation regression. In: Proceedings of ACM Multimedia (2009)

    Google Scholar 

  6. Comaniciu, D., Meer, P.: Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  7. Crandall, D., Backstrom, L., Huttenlocher, D., Kleinberg, J.: Mapping the world’s photos. In: Proceedings of World Wide Web Conference (2009)

    Google Scholar 

  8. Cristani, M., Perina, A., Castellani, U., Murino, V.: Geo-located image analysis using latent representations. In: Proceedings of IEEE CVPR (2008)

    Google Scholar 

  9. Divvala, S., Hoiem, D., Hays, J., Efros, A., Hebert, M.: An empirical study of context in object detection. In: Proceedings of IEEE CVPR (2009)

    Google Scholar 

  10. Dubinko, M., Kumar, R., Magnani, Novak J., Raghavan, P., Tomkins, A.: Visualizing tags over time. In: Proceedings of World Wide Web Conference (2006)

    Google Scholar 

  11. Hays, J., Efros, A.: IM2GPS: Estimating geographic information from a single image. In: Proceedings of IEEE CVPR (2008)

    Google Scholar 

  12. Jacobs, N., Satkin, S., Roman, N., Speyer, R., Pless, R.: Geolocating static cameras. In: Proceedings of IEEE International Conference on Computer Vision (2007)

    Google Scholar 

  13. Jain, V., Singhal, A., Luo, J.: Selective hidden random fields: Exploiting domain specific saliency for event classification. In: Proceedings of IEEE CVPR (2008)

    Google Scholar 

  14. Joshi, D., Luo, J.: Inferring generic activities and events from image content and bags of geo-tags. In: Proceedings of ACM CIVR (2008)

    Google Scholar 

  15. Joshi, D., Gallagher, A., Yu, J., Luo, J.: Inferring photographic location using geotagged web images. Multimed. Tools Appl. J. (2010)

    Google Scholar 

  16. Joshi, D., Luo, J., Yu, J., Lei, P., Gallagher, A.: Rich location-driven tag cloud suggestions based on public, community, and personal sources. In: Proceedings of ACM Int. Workshop on Connected Media Mining (2010)

    Google Scholar 

  17. Kennedy, L., Naaman, M., Ahern, S., Nair, R., Rattenbury, T.: How Flickr helps us make sense of the world: Context and content in community-contributed media collections. In: Proceedings of ACM Multimedia (2007)

    Google Scholar 

  18. Kennedy, L., Slaney, M., Weinberger, K.: Reliable tags using image similarity: Mining specificity and expertise from large-scale multimedia databases. In: ACM Workshop on Web-Scale Multimedia Corpus (2009)

    Google Scholar 

  19. Kleban, J., Moxley, E., Xu, J., Manjunath, B.S.: Global annotation on georeferenced photographs. In: Proceedings of ACM CIVR (2009)

    Google Scholar 

  20. Kosecka, J., Zhang, W.: Video compass. In: Proceedings of European Conference on Computer Vision (ECCV) (2002)

    Google Scholar 

  21. Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: Proceedings of IEEE CVPR (2006)

    Google Scholar 

  22. Liao, L., Fox, D., Kautz, H.: Extracting places and activities from GPS traces using hierarchical conditional random fields. Int. J. Robot. Res. (2007)

    Google Scholar 

  23. Li, L.-J., Fei-Fei, L.: What, where and who? Classifying event by scene and object recognition. In: Proceedings of IEEE ICCV (2007)

    Google Scholar 

  24. Luo, J., Boutell, M., Brown, C.: Pictures are not taken in a vacuum: An overview of exploiting context for semantic scene content understanding. IEEE Signal Process. Mag. 23(2), 101–114 (2006)

    Article  Google Scholar 

  25. Luo, J., Yu, J., Joshi, D., Hao, W.: Event recognition: viewing the world with a third eye. In: Proceedings of ACM Multimedia (2008)

    Google Scholar 

  26. Moxley, E., Kleban, J., Manjunath, B.S.: SpiritTagger: A geo-aware tag suggestion tool mined from Flickr. In: Proceedings of ACM Multimedia Information Retrieval (MIR) (2008)

    Google Scholar 

  27. O’Hare, N., Smeaton, A.: Context-aware person identification in personal photo collections. IEEE Trans. Multimed. (2009)

    Google Scholar 

  28. Quack, T., Leibe, B., Van Gool, L.: World-scale mining of objects and events from community photo collections. In: Proceedings of CIVR (2008)

    Google Scholar 

  29. Torralba, A., Fergus, R., Freeman, W.T.: Tiny images. Technical Report MIT-CSAIL-TR-2007-024 (2007)

    Google Scholar 

  30. Toyama, K., Logan, R., Roseway, A.: Geographic location tags on digital images. In: Proceedings of ACM Multimedia (2003)

    Google Scholar 

  31. Tsai, C.-M., Qamra, A., Chang, E.: Extent: Inferring image metadata from context and content. In: Proceedings of IEEE ICME (2005)

    Google Scholar 

  32. Wei, X.-Y., Jiang, Y.-G., Ngo, C.-W.: Exploring inter-concept relationship with context space for semantic video indexing. In: Proceedings of ACM CIVR (2009)

    Google Scholar 

  33. Wolf, L., Bileschi, S.: A critical view of context. Int. J. Comput. Vis. 68(1), 43–52 (2006)

    Article  Google Scholar 

  34. Yu, J., Luo, J.: Leveraging probabilistic season and location context models for scene understanding. In: Proceedings of ACM CIVR (2008)

    Google Scholar 

  35. Yuan, J., Luo, J., Kautz, H., Wu, Y.: Mining GPS traces and visual words for event classification. In: Proceedings of ACM Multimedia Information Retrieval (MIR) (2008)

    Google Scholar 

  36. Zheng, V.W., Zheng, Y., Xie, X., Yang, Q.: Collaborative location and activity recommendations with GPS history data. In: Proceedings of World Wide Web Conference (2010)

    Google Scholar 

  37. Zheng, Y.-T., Zhao, M., Song, Y., Adam, H., Buddemeier, U., Bissacco, A., Brucher, F., Chua, T.-S., Neven, H.: Tour the world: Building a webscale landmark recognition engine. In: Proceedings of IEEE CVPR (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Corporate Research and Engineering, Eastman Kodak Company, Rochester, USA

    Dhiraj Joshi, Jiebo Luo, Jie Yu, Phoury Lei & Andrew Gallagher

Authors
  1. Dhiraj Joshi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiebo Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Phoury Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Andrew Gallagher
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dhiraj Joshi .

Editor information

Editors and Affiliations

  1. School of Computer Engineering, Nanyang Technological University, Singapore, 639798, Singapore

    Steven C.H. Hoi

  2. Kodak Research Laboratories, Lake Avenue 1999, Rochester, 14650, New York, USA

    Jiebo Luo

  3. Media Informatics and Multimedia Systems, University of Oldenburg, Escherweg 2, Oldenburg, 26121, Germany

    Susanne Boll

  4. School of Computer Engineering, Nanyang Technological University, Singapore, 639798, Singapore

    Dong Xu

  5. Dept. Computer Science and Engineering, Michigan State University, Engineering Building 3115, East Lansing, 48824, Michigan, USA

    Rong Jin

  6. Dept. Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong/PR China

    Irwin King

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag London Limited

About this chapter

Cite this chapter

Joshi, D., Luo, J., Yu, J., Lei, P., Gallagher, A. (2011). Using Geotags to Derive Rich Tag-Clouds for Image Annotation. In: Hoi, S., Luo, J., Boll, S., Xu, D., Jin, R., King, I. (eds) Social Media Modeling and Computing. Springer, London. https://doi.org/10.1007/978-0-85729-436-4_11

Download citation

  • DOI: https://doi.org/10.1007/978-0-85729-436-4_11

  • Publisher Name: Springer, London

  • Print ISBN: 978-0-85729-435-7

  • Online ISBN: 978-0-85729-436-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation