PolemicTweet: Video Annotation and Analysis through Tagged Tweets

  • Samuel Huron
  • Petra Isenberg
  • Jean Daniel Fekete
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8118)


We present PolemicTweet a system with an encompassing, economic, and engaging approach to video tagging and analysis. Annotating and tagging videos manually is a boring and time-consuming process. Yet, in the last couple of years the audiences of events—such as academic conferences—have begun to produce unexploited metadata in the form of micropost activities. With PolemicTweet we explore the use of tagged microposts for both video annotation and browsing aid. PolemicTweet is a system 1) to crowd source conference video tagging with structured sentiment metadata, 2) to engage audiences in a tagging process, and 3) to visualize these annotations for browsing and analyzing a video. We describe the the system and its components as well as the results from a one-year live deployment in 27 different events.


Backchannel Video annotation Crowdsourcing Video analysis Live tagging 


  1. 1.
    TED: Ideas worth spreading, (retrieved January 01, 2013)
  2. 2.
    Allen, C.: The use of video in organizational studies. SIGCHI Bull. 21, 115–117 (1989)CrossRefGoogle Scholar
  3. 3.
    Hauptmann, A.G.: Lessons for the future from a decade of informedia video analysis research. In: Leow, W.-K., Lew, M., Chua, T.-S., Ma, W.-Y., Chaisorn, L., Bakker, E.M. (eds.) CIVR 2005. LNCS, vol. 3568, pp. 1–10. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  4. 4.
    Roschelle, J., Goldman, S.: VideoNoter: A productivity tool for video data analysis. Journal of Behavior Research Methods 23, 219–224 (1991)CrossRefGoogle Scholar
  5. 5.
    Mackay, W.E.: EVA: An experimental video annotator for symbolic analysis of video data. SIGCHI Bull. 21, 68–71 (1989)CrossRefGoogle Scholar
  6. 6.
    Abowd, G.D., Gauger, M., Lachenmann, A.: The Family Video Archive: An annotation and browsing environment for home movies. In: Proc. Workshop on Multimedia Information Retrieval, pp. 1–8. ACM (2003)Google Scholar
  7. 7.
    Diakopoulos, N., Goldenberg, S., Essa, I.: Videolyzer: Quality analysis of online informational video for bloggers and journalists. In: Proc. CHI, pp. 799–808. ACM (2009)Google Scholar
  8. 8.
    Cockburn, A., Dale, T.: CEVA: A tool for collaborative video analysis. In: Proc. GROUP, pp. 47–55. ACM (1997)Google Scholar
  9. 9.
    Schroeter, R., Hunter, J., Kosovic, D.: Vannotea: A collaborative video indexing, annotation and discussion system for broadband networks. In: Proc. of. K-CAP, pp. 1–8. ACM (2011)Google Scholar
  10. 10.
    Engström, A., Perry, M., Juhlin, O.: Amateur vision and recreational orientation: Creating live video together. In: Proc. CSCW, pp. 651–660. ACM (2012)Google Scholar
  11. 11.
    Daniel, G., Chen, M.: Video visualization. In: Proc. VIS, pp. 409–416. IEEE (2003)Google Scholar
  12. 12.
    Ruppert, T., Kohlhammer, J.: A radial visualization tool for depicting hierarchically structured video content. In: Proc. VAST, pp. 251–252. IEEE (2010)Google Scholar
  13. 13.
    Tonomura, Y., Akutsu, A., Otsuji, K., Sadakata, T.: VideoMAP and VideoSpaceIcon: tools for anatomizing video content. In: Proc. INTERACT & CHI, pp. 131–136. ACM (1993)Google Scholar
  14. 14.
    Parry, M.L., Legg, P.A., Chung, D.H.S., Griffiths, I.W., Chen, M.: Hierarchical event selection for video storyboards with a case study on snooker video visualization. IEEE TVCG 17, 1747–1756 (2011)Google Scholar
  15. 15.
    Diakopoulos, N.A., Shamma, D.A.: Characterizing debate performance via aggregated twitter sentiment. In: Proc. of CHI, pp. 1195–1198. ACM (2010)Google Scholar
  16. 16.
    Diakopoulos, N., Naaman, M., Kivran-Swaine, F.: Diamonds in the rough: Social media visual analytics for journalistic inquiry. In: Proc. of Visual Analytics Science and Technology (VAST), pp. 115–122. IEEE (2010)Google Scholar
  17. 17.
    Bernstein, M.S., Brandt, J., Miller, R.C., Karger, D.R.: Crowds in two seconds: Enabling realtime crowd-powered interfaces. In: Proc. UIST, pp. 33–42. ACM (2011)Google Scholar
  18. 18.
    Wu, S.-Y.Y., Thawonmas, R., Chen, K.-T.T.: Video summarization via crowdsourcing. In: Extended Abstracts CHI, pp. 1531–1536. ACM (2011)Google Scholar
  19. 19.
    Snow, R., Connor, B.O., Jurafsky, D., Ng, A.Y., Labs, D., St, C.: Cheap and fast—but is it good? Evaluating non-expert annotations for natural language tasks. In: Proc. EMNLP, pp. 254–263. ACL (2008)Google Scholar
  20. 20.
    Ipeirotis, P.G., Provost, F., Wang, J.: Quality management on Amazon Mechanical Turk. In: Proc. SIGKDD Workshop on Human Computation, p. 1403. ACM (2010)Google Scholar
  21. 21.
    McCarthy, J.F., Boyd, D., Churchill, E.F., Griswold, W.G., Lawley, E., Zaner, M.: Digital backchannels in shared physical spaces: attention, intention and contention. In: Proc. CSCW, pp. 550–553. ACM (2004)Google Scholar
  22. 22.
    Rekimoto, J., Ayatsuka, Y., Uoi, H., Arai, T.: Adding another communication channel to reality: An experience with a chat-augmented conference. In: Proc. CHI. ACM (1998)Google Scholar
  23. 23.
    Harry, D., Green, J., Donath, J.: integrating backchannels in physical space. In: Proc. of CHI, pp. 1361–1370. ACM (2009)Google Scholar
  24. 24.
    McNely, B.: Backchannel persistence and collaborative meaning-making. In: Proc. SIGDOC, pp. 297–304. ACM (2009)Google Scholar
  25. 25.
    Cogdill, S., Fanderclai, T.L., Kilborn, J., Williams, M.G.: Backchannel: Whispering in digital conversation. In: Proc. HICSS, p. 8 (2001)Google Scholar
  26. 26.
    Dörk, M., Gruen, D., Williamson, C., Carpendale, S.: A Visual Backchannel for Large-Scale Events. IEEE TVCG 16, 1129–1138 (2010)Google Scholar
  27. 27.
    Marcus, A., Bernstein, M.S., Badar, O., Karger, D.R., Madden, S., Miller, R.C.: Twitinfo: Aggregating and visualizing microblogs for event exploration. In: Proc. CHI, pp. 227–236. ACM (2011)Google Scholar
  28. 28.
    Bry, F., Gehlen-Baum, V., Pohl, A.: Promoting awareness and participation in large class lectures: The digital backchannel backstage. In: Proc. IADIS, pp. 27–34. Emerald Group Publishing Limited (2011)Google Scholar
  29. 29.
    McCarthy, J.F., Boyd, D.: Digital backchannels in shared physical spaces: experiences at an academic conference. In: Extended Abstracts CHI, pp. 1641–1644. ACM (2005)Google Scholar
  30. 30.
    Sopan, A., Rey, P., Butler, B., Shneiderman, B.: Monitoring Academic Conferences: Real-time Visualization and Retrospective Analysis of Backchannel Conversations. In: Proc. SOCINFO (2012)Google Scholar
  31. 31.
    Pang, B., Lee, L.: Opinion mining and sentiment analysis. Journal of  Foundations and Trends in Information Retrieval - FTIR, 135 (2008)Google Scholar
  32. 32.
    Jiang, L., Yu, M., Zhou, M., Liu, X., Zhao, T.: Target-dependent twitter sentiment classification. In: Proc. ACL HLT, pp. 151–160 (2011)Google Scholar
  33. 33.
    Speriosu, M., Sudan, N., Upadhyay, S., Baldridge, J.: Twitter polarity classification with label propagation over lexical links and the follower graph. In: Proc. HLT, pp. 53–63. ACM (2011)Google Scholar
  34. 34.
    Huron, S., Fekete, J.-D.: User-Supplied Sentiments in Tweets. In: 2nd Workshop on Interactive Visual Text Analysis. IEEE (2012)Google Scholar
  35. 35.
    Crowdflower: Senti crowdflower, (retrieved January 01, 2013)
  36. 36.
    Quinn, A.J., Bederson, B.B.: Human computation: a survey and taxonomy of a growing field. In: Proc. CHI, pp. 1403–1412. ACM (2011)Google Scholar
  37. 37.
    Médiamétrie-eStat: La fréquentation des sites Internet français Février (2012), (retrieved January 01, 2013)

Copyright information

© IFIP International Federation for Information Processing 2013

Authors and Affiliations

  • Samuel Huron
    • 1
  • Petra Isenberg
    • 1
  • Jean Daniel Fekete
    • 1
  1. 1.INRIA-Team AvizUniversité Paris-SudOrsay CedexFrance

Personalised recommendations