Social Video Retrieval: Research Methods in Controlling, Sharing, and Editing of Web Video

  • Konstantinos Chorianopoulos
  • David A. Shamma
  • Lyndon Kennedy
Part of the Computer Communications and Networks book series (CCN)


Content-based video retrieval has been a very efficient technique with new video content, but it has not fully explored the increasingly dynamic interactions between users and content. We present a comprehensive survey on user-based techniques and instrumentation for social video retrieval researchers. Community-based approaches suggest there is much to learn about an unstructured video just by analyzing the dynamics of how it is being used. In particular, we explore three pillars of online user activity with video content: (1) Seeking patterns within a video is linked to interesting video segments, (2) sharing patterns between users indicates that there is a correlation between social activity and popularity of a video, and (3) editing of live events is automated through the synchronization of audio across multiple viewpoints of the same event. Moreover, we present three complementary research methods in social video retrieval: experimental replication of user activity data and signal analysis, data mining and prediction on natural user activity data, and hybrid techniques that combine robust content-based approaches with crowd sourcing of user-generated content. Finally, we suggest further research directions in the combination of richer user and content modeling, because it provides an attractive solution to the personalization, navigation, and social consumption of videos.


Video Clip Video Content Video Segment Video Retrieval Video User 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Baecker, R., Rosenthal, A.J., Friedlander, N., Smith, E., Cohen, A.: A multimedia system for authoring motion pictures. In: Proceedings of the Fourth ACM International Conference on Multimedia, MULTIMEDIA ’96, pp. 31–42. ACM, New York (1996). doi:10.1145/244130.244142.
  2. 2.
    Carlier, A., Charvillat, V., Ooi, W.T., Grigoras, R., Morin, G.: Crowdsourced automatic zoom and scroll for video retargeting. In: Proceedings of the International Conference on Multimedia, MM ’10, pp. 201–210. ACM, New York (2010). doi:10.1145/1873951.1873962.
  3. 3.
    Cesar, P., Chorianopoulos, K.: The evolution of tv systems, content, and users toward interactivity. Found. Trends Hum.-Comput. Interact. 2(4), 373–95 (2009). doi:10.1561/1100000008.
  4. 4.
    Cha, M., Kwak, H., Rodriguez, P., Ahn, Y.Y., Moon, S.: I tube, you tube, everybody tubes: analyzing the world’s largest user generated content video system. In: Proceedings of the 7th ACM SIGCOMM Conference on Internet Measurement, IMC ’07, pp. 1–14. ACM, New York (2007). doi:
  5. 5.
    Cheng, X., Dale, C., Liu, J.: Statistics and social network of youtube videos. In: Quality of Service, 2008. IWQoS 2008, 16th International Workshop on, pp. 229–238 (2008). doi:10.1109/IWQOS.2008.32Google Scholar
  6. 6.
    Chorianopoulos, K., Leftheriotis, I., Gkonela, C.: Socialskip: pragmatic understanding within web video. In: Proceddings of the 9th International Interactive Conference on Interactive Television, EuroITV ’11, pp. 25–28. ACM, New York (2011). doi:
  7. 7.
    Christel, M.G., Hauptmann, A.G., Wactlar, H.D., Ng, T.D.: Collages as dynamic summaries for news video. In: Proceedings of the Tenth ACM International Conference on Multimedia, MULTIMEDIA ’02, pp. 561–569. ACM, New York (2002). doi:10.1145/641007.641120.
  8. 8.
    Crane, R., Sornette, D.: Viral, quality, and junk videos on youtube: separating content from noise in an information-rich environment. In: Proceedings of AAAI Symposium on Social Information Processing, Menlo Park (2008)Google Scholar
  9. 9.
    Cunningham, S.J., Nichols, D.M.: How people find videos. In: Proceedings of the 8th ACM/IEEE-CS Joint Conference on Digital Libraries, JCDL ’08, pp. 201–210. ACM, New York (2008). doi:10.1145/1378889.1378924.
  10. 10.
    Davis, M.: Human-computer interaction. In: Baecker, R.M., Grudin, J., Buxton, W.A.S., Greenberg, S. (eds.) Readings in Human-Computer Interaction: Toward the Year 2000, Chap. Media Streams: An Iconic Visual Language for Video Representation, pp. 854–866. Morgan Kaufmann, San Francisco (1995).
  11. 11.
    Drucker, S.M., Glatzer, A., De Mar, S., Wong, C.: Smartskip: consumer level browsing and skipping of digital video content. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Changing Our World, Changing Ourselves, CHI ’02, pp. 219–226. ACM, New York (2002). doi:10.1145/503376.503416.
  12. 12.
    Girgensohn, A., Boreczky, J., Wilcox, L.: Keyframe-based user interfaces for digital video. Computer 34(9), 61–67 (2001). doi:10.1109/2.947093. Google Scholar
  13. 13.
    Haitsma, J., Kalker, T.: A highly robust audio fingerprinting system with an efficient search strategy. J. New Music Res. 32(2), 211–221 (2003). doi:10.1076/jnmr. Scholar
  14. 14.
    Kennedy, L., Naaman, M.: Less talk, more rock: automated organization of community-contributed collections of concert videos. In: Proceedings of the 18th International Conference on World Wide Web, WWW ’09, pp. 311–320. ACM, New York (2009). doi:
  15. 15.
    Levy, P.: Collective Intelligence: Mankind’s Emerging World in Cyberspace. Perseus Books, Cambridge (1997)Google Scholar
  16. 16.
    Li, F.C., Gupta, A., Sanocki, E., He, L.W., Rui, Y.: Browsing digital video. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’00, pp. 169–176. ACM, New York (2000). doi:10.1145/332040.332425.
  17. 17.
    Lienhart, R., Pfeiffer, S., Effelsberg, W.: Video abstracting. Commun. ACM 40(12), 54–62 (1997). doi:10.1145/265563.265572.
  18. 18.
    Liu, J., Dolan, P., Pedersen, E.R.: Personalized news recommendation based on click behavior. In: Proceedings of the 15th International Conference on Intelligent User Interfaces, IUI ’10, pp. 31–40. ACM, New York (2010). doi:10.1145/1719970.1719976.
  19. 19.
    Ma, Y.F., Lu, L., Zhang, H.J., Li, M.: A user attention model for video summarization. In: Proceedings of the Tenth ACM International Conference on Multimedia, MULTIMEDIA ’02, pp. 533–542. ACM, New York (2002). doi:10.1145/641007.641116.
  20. 20.
    Mitra, S., Agrawal, M., Yadav, A., Carlsson, N., Eager, D., Mahanti, A.: Characterizing web-based video sharing workloads. ACM Trans. Web 5(2), 8:1–8:27 (2011). doi:10.1145/ 1961659.1961662.
  21. 21.
    Money, A.G., Agius, H.: Video summarisation: a conceptual framework and survey of the state of the art. J. Vis. Commun. Image Represent. 19(2), 121–143 (2008). doi:10.1016/ j.jvcir.2007.04.002.
  22. 22.
    Naci, S.U., Hanjalic, A.: Intelligent browsing of concert videos. In: Proceedings of the 15th International Conference on Multimedia, MULTIMEDIA ’07, pp. 150–151. ACM, New York (2007). doi:10.1145/1291233.1291264.
  23. 23.
    Nair, R., Reid, N., Davis, M.: Photo loi: browsing multi-user photo collections. In: Proceedings of the 13th Annual ACM International Conference on Multimedia, MULTIMEDIA ’05, pp. 223–224. ACM, New York (2005). doi:10.1145/1101149.1101187.
  24. 24.
    Olsen, D.R., Moon, B.: Video summarization based on user interaction. In: Proceedings of the 9th International Interactive Conference on Interactive Television, EuroITV ’11, pp. 115–122. ACM, New York (2011). doi:10.1145/2000119.2000142.
  25. 25.
    Peng, W.T., Chu, W.T., Chang, C.H., Chou, C.N., Huang, W.J., Chang, W.Y., Hung, Y.P.: Editing by viewing: automatic home video summarization by viewing behavior analysis. Multimed. IEEE Trans. 13(3), 539–550 (2011). doi:10.1109/TMM.2011.2131638CrossRefGoogle Scholar
  26. 26.
    Segaran, T.: Programming Collective Intelligence: Building Smart Web 2.0 Applications, 1st edn. O’Reilly, Beijing/Sebastapol (2007).\&path=ASIN/0596529325
  27. 27.
    Shamma, D.A., Kennedy, L., Churchill, E.F.: Tweet the debates: understanding community annotation of uncollected sources. In: WSM ’09: Proceedings of the International Workshop on Workshop on Social Media. ACM, Beijing (2009)Google Scholar
  28. 28.
    Shamma, D.A., Kennedy, L., Churchill, E.F.: Viral actions: predicting video view counts using synchronous sharing behaviors. In: ICWSM 11: Proceedings of the International Conference on Weblogs and Social Media Data. AAAI, Barcelona (2011)Google Scholar
  29. 29.
    Shaw, R., Davis, M.: Toward emergent representations for video. In: MULTIMEDIA ’05: Proceedings of the 13th Annual ACM International Conference on Multimedia, pp. 431–434. ACM, New York (2005). doi:
  30. 30.
    Shaw, R., Schmitz, P.: Community annotation and remix: a research platform and pilot deployment. In: HCM ’06: Proceedings of the 1st ACM International Workshop on Human-Centered Multimedia, pp. 89–98. ACM, New York (2006). doi:
  31. 31.
    Snoek, C., Worring, M., Smeulders, A., Freiburg, B.: The role of visual content and style for concert video indexing. In: Multimedia and Expo, 2007 IEEE International Conference on, pp. 252–255. IEEE, Washington, DC (2007)Google Scholar
  32. 32.
    Truong, B.T., Venkatesh, S.: Video abstraction: a systematic review and classification. ACM Trans. Multimed. Comput. Commun. Appl. 3(1) (2007). doi:10.1145/1198302.1198305. Google Scholar
  33. 33.
    Wang, A.: The shazam music recognition service. Commun. ACM 49(8), 44–48 (2006). doi:10.1145/1145287.1145312, Google Scholar
  34. 34.
    Yan, R., Hauptmann, A.G.: A review of text and image retrieval approaches for broadcast news video. Inf. Retr. 10, 445–484 (2007). doi:10.1007/s10791-007-9031-yCrossRefGoogle Scholar
  35. 35.
    Yew, J., Shamma, D.A., Churchill, E.F.: Knowing funny: genre perception and categorization in social video sharing. In: Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems, CHI ’11, pp. 297–306. ACM, New York (2011). doi:
  36. 36.
    Yu, B., Ma, W.Y., Nahrstedt, K., Zhang, H.J.: Video summarization based on user log enhanced link analysis. In: Proceedings of the Eleventh ACM International Conference on Multimedia, MULTIMEDIA ’03, pp. 382–391. ACM, New York (2003). doi:10.1145/957013.957095.
  37. 37.
    Zhang, D., Guo, B., Yu, Z.: The emergence of social and community intelligence. Computer 44(7), 21–28 (2011). doi:10.1109/MC.2011.65CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Konstantinos Chorianopoulos
    • 1
  • David A. Shamma
    • 2
  • Lyndon Kennedy
    • 2
  1. 1.Ionian UniversityMarousiGreece
  2. 2.Yahoo! ResearchSanta ClaraUSA

Personalised recommendations