Multimedia Tools and Applications

, Volume 70, Issue 1, pp 89–118 | Cite as

Collaborative event annotation in tagged photo collections

  • Christos Zigkolis
  • Symeon Papadopoulos
  • George Filippou
  • Yiannis Kompatsiaris
  • Athena Vakali


Events constitute a significant means of multimedia content organization and sharing. Despite the recent interest in detecting events and annotating media content in an event-centric way, there is currently insufficient support for managing events in large-scale content collections and limited understanding of the event annotation process. To this end, this paper presents CrEve, a collaborative event annotation framework which uses content found in social media sites with the prime objective to facilitate the annotation of large media corpora with event information. The proposed annotation framework could significantly benefit social media research due to the proliferation of event-related user-contributed content. We demonstrate that, compared to a standard “browse-and-annotate” interface, CrEve leads to a 19% increase in the coverage of the generated ground truth in a large-scale annotation experiment. Furthermore, the paper discusses the results of a user study that quantifies the performance of CrEve and the contribution of different event dimensions in the event annotation process. The study confirms the prevalence of spatio-temporal queries as the prime option of discovering event-related content in a large collection. In addition, textual queries and social cues (content contributor) were also found to be significant as event search dimensions. Finally, it demonstrates the potential of employing automatic photo clustering methods with the goal of facilitating event annotation.


Event authoring Multimedia annotation Ground truth generation 



Christos Zigkolis’s work has been co-financed by the European Union (European Social Fund—ESF) and Greek national funds through the Operational Program “Education and Lifelong Learning” of the National Strategic Reference Framework (NSRF)—Research Funding Program: Heracleitus II. Investing in knowledge society through the European Social Fund. The work of Symeon Papadopoulos was supported by the GLOCAL and SocialSensor projects, partially funded by the European Commission, under contract numbers FP7-248984 and FP7-287975.


  1. 1.
    Ames M, Naanman M (2007) Why we tag: motivations for annotation in mobile and online media. In: Proceedings of the SIGCHI conference on human factors in computing systems CHI ’07, pp 971–980Google Scholar
  2. 2.
    Appan P, Sundaram H (2004) Networked multimedia event exploration. In: Proceedings of the 12th annual ACM international conference on multimedia (MULTIMEDIA ’04). ACM, New York, NY, USA, pp 40–47CrossRefGoogle Scholar
  3. 3.
    Appan P, Shevade B, Sundaram, H, Birchfield, D (2005) Interfaces for networked media exploration and collaborative annotation. In: Proceedings of the 10th international conference on intelligent user interfaces (IUI ’05). ACM, New York, NY, USA, pp 106–113Google Scholar
  4. 4.
    Begelman G, Keller P, Smadja F (2006) Automatic tag clustering: improving search and exploration in the tag space. In: Proc. of the collaborative web tagging workshop at WWW 06Google Scholar
  5. 5.
    Burke R (2002) Hybrid recommender systems: survey and experiments. User Model User-Adapt Interact 12(4):331–370CrossRefzbMATHGoogle Scholar
  6. 6.
    Chua TS, Tang J, Hong R, Li H, Luo Z, Zheng YT (2009) NUS-WIDE: a real-world web image database from National University of Singapore. In: ACM international conference on image and video retrieval, Greece, 8–10 Jul 2009Google Scholar
  7. 7.
    Cooper M, Foote J, Girgensohn A, Wilcox L (2005) Temporal event clustering for digital photo collections. ACM transactions on multimedia computing, communications and applications, vol 1, pp 269–288, 3 Aug 2005Google Scholar
  8. 8.
    Golder S, Huberman A. (2006) The structure of collaborative tagging systems, 2006. HP Labs Technical Report available at
  9. 9.
    Huiskes MJ, Lew MS (2008) The MIR Flickr retrieval evaluation. In: ACM international conference on multimedia information retrieval (MIR’08), Vancouver, CanadaGoogle Scholar
  10. 10.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91110CrossRefGoogle Scholar
  11. 11.
    Liu X, Troncy R, Huet B (2011) Finding media illustrating events. In: ACM international conference on multimedia retrieval (ICMR 2011), Trento, ItalyGoogle Scholar
  12. 12.
    Marlow C, Naaman M, Boyd D, Davis M (2006) HT06, tagging paper, taxonomy, Flickr, academic article, to read. In: Proceedings of the seventeenth conference on hypertext and hypermedia (HYPERTEXT ’06). ACM, New York, NY, USA, pp 31–40CrossRefGoogle Scholar
  13. 13.
    Mirkovic M, Culibrk D, Papadopoulos S, Zigkolis C, Kompatsiaris Y, McArdle G, Crnojevic V (2011) A comparative study of spatial, temporal and content-based patterns emerging in YouTube and Flickr. In: International conference on Computational Aspects of Social Networks (CASoN), pp 189–194Google Scholar
  14. 14.
    Naaman M, Harada S, Wang Q, Garcia-Molina H, Paepcke A (2004) Context data in geo-referenced digital photo collections. In: Proceedings of the 12th annual ACM international conference on multimedia (MULTIMEDIA ’04)Google Scholar
  15. 15.
    Papadopoulos S, Zigkolis C, Kompatsiaris Y, Vakali A (2011) Cluster-based landmark and event detection on tagged photo collections. IEEE Multimedia Mag 18(1):52–63CrossRefGoogle Scholar
  16. 16.
    Papadopoulos S, Zigkolis C, Kapiris S, Kompatsiaris Y, Vakali A (2010) ClustTour: city exploration by use of hybrid photo clustering. In: Technical demonstration session of ACM multimedia 2010 (Florence, 2010)Google Scholar
  17. 17.
    Papadopoulos S, Troncy R, Mezaris V, Huet B, Kompatsiaris I (2011) Social event detection at MediaEval 2011: challenges, dataset and evaluation. In: MediaEval workshop, Pisa, Italy, 1–2 Sept 2011Google Scholar
  18. 18.
    Papadopoulos S, Zigkolis C, Kompatsiaris Y, Vakali A (2011) CERTH @ MediaEval 2011 SED task. In: Proceedings of MediaEval workshop, Pisa, Italy, 1–2 Sept 2011Google Scholar
  19. 19.
    Pu P, Chen L, Kumar P (2008) Evaluating product search and recommender systems for E-commerce environments, vol 8, 1–2 Jun 2008, pp 1–27Google Scholar
  20. 20.
    Rattenbury T, Good N, Naaman M (2007) Towards automatic extraction of event and place semantics from Flickr tags. In: Proceedings of the 30th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR ’07)Google Scholar
  21. 21.
    Quack T, Leibe B, Van Gool L (2008) World-scale mining of objects and events from community photo collections. In: Proceedings of the 2008 international conference on content-based image and video retrieval (CIVR ’08). ACM, New York, NY, USA, pp 47–56CrossRefGoogle Scholar
  22. 22.
    Sayyadi H, Hurst M, Maykov A (2009) Event detection and tracking in social streams. In: Proceedings of international AAAI conference on weblogs and social media. AAAI PressGoogle Scholar
  23. 23.
    Scherp A, Saathoff C, Franz T, Staab S (2012) A core ontology on events for representing occurrences in the real world. Multimedia Tools and Applications 58(2):293–331CrossRefGoogle Scholar
  24. 24.
    Shaw R, Troncy R, Hardman L (2009) LODE: linking open descriptions of events. In: 4th Asian Semantic Web Conference (ASWC’2009). Shanghai, ChinaGoogle Scholar
  25. 25.
    Shneiderman B, Bederson BB, Drucker SM (2006) Find that photo!: interface strategies to annotate, browse, and share. Communications of the ACM, vol 49, pp 69–71, 4 Apr 2006Google Scholar
  26. 26.
    Sigurbjörnsson B, van Zwol R (2008) Flickr tag recommendation based on collective knowledge. In: Proceedings of the 17th international conference on World Wide Web (WWW ’08), pp 327–336Google Scholar
  27. 27.
    Smeaton AF, Over P, Kraaij W (2006) Evaluation campaigns and TRECVid. In: Proceedings of the 8th ACM international workshop on multimedia information retrieval, Santa Barbara, California, USA, 26–27 Oct 2006. MIR ’06. ACM Press, New York, NY, pp 321–330Google Scholar
  28. 28.
    Suh B, Bederson BB (2007) Semi-automatic photo annotation strategies using event based clustering and clothing based person recognition. Interact Comput 19(4):524–544CrossRefGoogle Scholar
  29. 29.
    Troncy R, Malocha B, Fialho A (2010) Linking events with media I-SEMANTICS’10. In: 6th international conference on semantic systems, colocated with “In the open track of the linked data triplification challenge, Graz, Austria, 1–3 Sept 2010Google Scholar
  30. 30.
    Van de Sande, KE, Gevers T, Snoek CG (2009) Evaluating color descriptors for object and scene recognition. IEEE Trans Pattern Anal Mach Intell 99Google Scholar
  31. 31.
    Von Ahn, L, Dabbish, L (2004) Labeling images with a computer game. In: Proceedings of the SIGCHI conference on Human factors in computing systems (CHI ’04)Google Scholar
  32. 32.
    Vander-wal T (2007) Explaining and showing broad and narrow folksonomies.
  33. 33.
    Westermann U, Jain R (2007) Toward a common event model for multimedia applications. IEEE MultiMedia Mag 14(1):19–29CrossRefGoogle Scholar
  34. 34.
    Wu X, Lu Y-J, Peng Q, Ngo C-W (2011) Mining event structures from web videos. IEEE MultiMedia Mag 18(1):38–51CrossRefGoogle Scholar
  35. 35.
    Xie L, Sundaram H, Campbell M (2008) Event mining in multimedia streams. Proc IEEE 96(4):623–647CrossRefGoogle Scholar
  36. 36.
    Zsombori V, Frantzis M, Guimaraes RL, Ursu MF, Cesar P, Kegel I, Craigie R, Bulterman DCA (2011) Automatic generation of video narratives from shared UGC. In: Proceedings of the 22nd ACM conference on hypertext and hypermedia, pp 325–334Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Christos Zigkolis
    • 1
    • 2
  • Symeon Papadopoulos
    • 1
    • 2
  • George Filippou
    • 2
  • Yiannis Kompatsiaris
    • 1
  • Athena Vakali
    • 2
  1. 1.Information Technologies InstituteCERTHThessalonikiGreece
  2. 2.Department of InformaticsAUTHThessalonikiGreece

Personalised recommendations