Skip to main content
SpringerLink
Log in

Discrimination of media moments and media intervals: sticker-based watch-and-comment annotation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper we discuss the problem of how to discriminate moments of interest on videos or live broadcast shows. The primary contribution is a system which allows users to personalize their programs with previously created media stickers—pieces of content that may be temporarily attached to the original video. We present the system’s architecture and implementation, which offer users operators to transparently annotate videos while watching them. We offered a soccer fan the opportunity to add stickers to the video while watching a live match: the user reported both enjoying and being comfortable using the stickers during the match—relevant results even though the experience was not fully representative.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. E.g. http://www.gsmfans.com.br/index.php?topic=602.0 currently offers ring tones from 39 Brazilian teams.

  2. E.g. http://www.ecvitoria.com.br/site/papeldeparede/default.jsp offers wallpapers of the Vitoria Sport Club.

  3. http://www.international-television.org/tv_audience_measurement_research_boards_and_institutes.html

  4. http://www.bbc.co.uk/iplayer

  5. http://www.hulu.com

  6. http://www.boxee.tv

  7. An example is http://www.creative.com/mylivecam/livecentral/downloads.aspx

  8. http://www.zeroconf.org

References

  1. ABNT-2007: ABNT NBR 15606-2 Associação Brasileira de Normas Técnicas. Digital Terrestrial Television Standard 06: Data Codification and Transmission Specifications for Digital Broadcasting, Part 2 - GINGA-NCL: XML Application Language for Application Coding. http://www.abnt.org.br/imagens/Normalizacao_TV_Digital/ABNTNBR15606-2_2007Ing_2008.pdf (version 2008)

  2. Allen JF, Hayes PJ (1990) Moments and points in an interval-based temporal logic. Comput Intell 5(4):225–238. doi:10.1111/j.1467-8640.1989.tb00329.x

    Google Scholar 

  3. Athanasiadis E, Mitropoulos S (2010) A distributed platform for personalized advertising in digital interactive TV environments. J Syst Softw 83:1453–1469. doi:10.1016/j.jss.2010.02.040

    Article  Google Scholar 

  4. Blanco-Fernández Y, Arias JJP, Gil-Solla A, Cabrer MR, Nores ML, Duque JG, Vilas AF, Redondo RPD, Muñoz JB (2008) An MHP framework to provide intelligent personalized recommendations about digital TV contents. Softw Pract Exp 38(9):925–960

    Article  Google Scholar 

  5. Blanco-Fernández Y, Pazos-Arias JJ, Gil-Solla A, Ramos-Cabrer M, López-Nores M (2008) Zaptv: Personalized user-generated content for handheld devices in dvb-h mobile networks. In: Proceedings of the European conference on interactive television (EUROITV’08) (LNCS 5866). Springer, Berlin, pp 193–203. doi:10.1007/978-3-540-69478-6_26

    Google Scholar 

  6. Bulterman DCA, Jansen AJ, César P, Mullender S, Hyche E, DeMeglio M, Quint J, Kawamura H, Weck D, Pañeda XG, Melendi D, Cruz-Lara S, Hanclik M, Zucker DF, Michel T (2008) Synchronized Multimedia Integration Language (SMIL 3.0). http://www.w3.org/TR/SMIL3/

  7. Cattelan RG, Santos FS, Goularte R, Teixeira CAC, Pimentel MdGC (2008) Watch-and-comment as a paradigm toward ubiquitous interactive video editing. ACM Trans Multimed Comput Comm Appl (ACM TOMCCAP) 4(4):1–24

    Article  Google Scholar 

  8. César P, Bulterman DCA, Jansen AJ (2006) The ambulant annotator: empowering viewer-side enrichment of multimedia content. In: Proceedings of the ACM symposium on document engineering (DOCENG’06). ACM, New York, pp 186–187 doi:10.1145/1166160.1166209

    Chapter  Google Scholar 

  9. César P, Bulterman DCA, Jansen AJ (2008) Usages of the secondary screen in an interactive television environment: Control, enrich, share, and transfer television content. In: Proceedings of the European conference on interactive television (EUROITV’08) (LNCS 5866), pp 168–177

  10. César P, Bulterman DCA, Jansen AJ (2009) Leveraging user impact: an architecture for secondary screens usage in interactive television. Multimedia Syst 15(3):127–142

    Article  Google Scholar 

  11. César P, Bulterman DCA, Jansen AJ, Geerts D, Knoche H, Seager W (2009) Fragment, tag, enrich, and send: Enhancing social sharing of video. ACM Trans Multimed Comput Comm Appl (ACM TOMCCAP) 5(19):1–19:27. doi:10.1145/1556134.1556136

    Article  Google Scholar 

  12. Chi MC, Yeh CH, Chen MJ (2009) Robust region-of-interest determination based on user attention model through visual rhythm analysis. IEEE Trans Circuits Syst Video Technol 19:1025–1038. doi:10.1109/TCSVT.2009.2022822

    Article  Google Scholar 

  13. Coppens T, Trappeniers L, Godon M (2004) AmigoTV: Towards a social TV experience. In: Proceedings of the European conference on interactive television (EUROITV’04)

  14. Costa M, Correia N, Guimarães N (2002) Annotations as multiple perspectives of video content. In: Proceedings of the ACM international conference on multimedia (MULTIMEDIA’02), pp 283–286. doi:10.1145/641007.641065

  15. Costa RMdR, Moreno MF, Rodrigues RF, Soares LFG (2006) Live editing of hypermedia documents. In: Proceedings of the ACM symposium on document engineering (DOCENG’06). ACM, New York, pp 165–172. doi:10.1145/1166160.1166202

    Chapter  Google Scholar 

  16. de Ávila PM, Zorzo SD (2009) Recommender TV - A Personalized TV Guide System Compliant with Ginga. In: Proceedings of the international conference on security and cryptography (SIGMAP’09), part of the international joint conference on e-business and telecommunications (ICETE’09), pp 149–156

  17. de Freitas GB, Teixeira CAC (2009) Ubiquitous services in home networks offered through digital TV. In: Proceedings of the ACM symposium on applied computing (SAC’09). ACM, New York, pp 1834–1838. doi:10.1145/1529282.1529691

    Chapter  Google Scholar 

  18. De Lucena VF, Filho JEC, Viana NS, Maia O (2009) A home automation proposal built on the ginga digital TV middleware and the OSGi framework. IEEE Trans Consum Electron 55(3):1254–1262

    Article  Google Scholar 

  19. Deigmoeller J, Itagaki T, Stoll G, Just N (2010) An approach to intelligently crop and scale video for broadcast applications. In: Proceedings of the ACM symposium on applied computing (SAC’10). ACM, New York, pp 1911–1918. doi:10.1145/1774088.1774493

    Google Scholar 

  20. Di Massa R, Montagnuolo M, Messina A (2010) Implicit news recommendation based on user interest models and multimodal content analysis. In: Proceedings of the international workshop on automated information extraction in media production (AIEMPro’10). ACM, New York, pp 33–38.doi:10.1145/1877850.1877861

    Google Scholar 

  21. Dimitrova N, Janevski A, Li D, Zimmerman J (2003) Who’s that actor?: the Infosip TV agent. In: Proceedings of the ACM SIGMM workshop on experiential telepresence (ETP’03). ACM, New York, pp 76–79. doi:10.1145/982484.982499

    Chapter  Google Scholar 

  22. Drucker SM, Glatzer A, De Mar S, Wong C (2002) Smartskip: consumer level browsing and skipping of digital video content. In: Proceedings of the ACM conference on human factors in computing systems (CHI’02). ACM, New York, pp 219–226. doi:10.1145/503376.503416

    Google Scholar 

  23. Furini M, Geraci F, Montangero M, Pellegrini M (2010) Stimo: Still and moving video storyboard for the web scenario. Multimedia Tools and Applications 46:47–69. doi:10.1007/s11042-009-0307-7

    Article  Google Scholar 

  24. Gao Y, Dai QH (2008) Clip based video summarization and ranking. In: Proceedings of the international conference on content-based image and video retrieval (CIVR’08). ACM, New York, pp 135–140. doi:10.1145/1386352.1386375

    Chapter  Google Scholar 

  25. Goularte R, Cattelan RG, Camacho-Guerrero JA, Inácio VR Jr, Pimentel MdGC (2004) Interactive multimedia annotations: enriching and extending content. In: Proceedings of the ACM symposium on document engineering (DOCENG’04), pp 84–86. doi:10.1145/1030397.1030414

  26. Hickson I (2011) HTML5 a vocabulary and associated APIs for HTML and XHTML—W3C Working Draft 13 January 2011. http://www.w3.org/TR/html5/

  27. Hölbling G, Rabl T, Coquil D, Kosch H (2008) Interactive TV services on mobile devices. IEEE Multimed 15(2):72–76. doi:10.1109/MMUL.2008.34

    Article  Google Scholar 

  28. Hsu SH, Wen MH, Lin HC, Lee CC, Lee CH (2007) AIMED–a personalized TV recommendation system. In: Proceedings of the European conference on interactive television (EUROITV’07) (LNCS 4417), pp 166–174

  29. Huang EM, Harboe G, Tullio J, Novak A, Massey N, Metcalf CJ, Romano G (2009) Of social television comes home: a field study of communication choices and practices in TV-based text and voice chat. In: Proceedings of the ACM international conference on human factors in computing systems (CHI’09), pp 585–594

  30. Huet B, Jiten J, Merialdo B (2005) Personalization of hyperlinked video in interactive television. In: Proceedings of the IEEE conference on multimedia and expo, p 4. doi:10.1109/ICME.2005.1521460

  31. ISO (1996) Information technology—generic coding of moving pictures and associated audio information: digital storage media command & control. ISO/IEC 13818-6

  32. Jansen J, César P, Bulterman DC (2010) A model for editing operations on active temporal multimedia documents. In: Proceedings of the ACM symposium on document engineering (DOCENG’10). ACM, New York, pp 87–96. doi:10.1145/1860559.1860579

    Google Scholar 

  33. Laiola Guimarães R, César P, Bulterman DC (2010) Creating and sharing personalized time-based annotations of videos on the web. In: Proceedings of the ACM symposium on document engineering (DOCENG’10). ACM, New York, pp 27–36. doi:10.1145/1860559.1860567

    Google Scholar 

  34. Liu Z, Gibbon DC, Drucker H, Basso A (2008) Content personalization and adaptation for three-screen services. In: Proceedings of the international conference on content-based image and video retrieval (CIVR ’08). ACM, New York, pp 635–644. doi:10.1145/1386352.1386444

    Chapter  Google Scholar 

  35. Liu Z, Zavesky E, Shahraray B, Gibbon D, Basso A (2008) Brief and high-interest video summary generation: evaluating the at&t labs rushes summarizations. In: Proceedings of the ACM TRECVid video summarization workshop (TVS’08). ACM, New York, pp 21–25. doi:10.1145/1463563.1463565

    Chapter  Google Scholar 

  36. Lynn SG, Olsen DR Jr, Partridge BG (2009) Time warp football. In: Proceedings of the European conference on interactive television (EUROITV’09). ACM, New York, pp 77–86. doi:10.1145/1542084.1542098

    Chapter  Google Scholar 

  37. Miyahara M, Aoki M, Takiguchi T, Ariki Y (2008) Tagging video contents with positive/negative interest based on user’s facial expression. In: Satoh S, Nack F, Etoh M (eds) Advances in multimedia modeling (LNCS 4903). Springer, Berlin, pp 210–219

    Chapter  Google Scholar 

  38. Motti VG, Fagá R Jr, Catellan RG, Pimentel MdGC, Teixeira CA (2009) Collaborative synchronous video annotation via the watch-and-comment paradigm. In: Proceedings of the European Conference on Interactive Television (EUROITV’09). ACM, New York, pp 67–76. doi:10.1145/1542084.1542097

    Chapter  Google Scholar 

  39. Nathan M, Harrison C, Yarosh S, Terveen LG, Stead L, Amento B (2008) CollaboraTV: making television viewing social again. In: Proceedings of the international conference on designing interactive user experiences for TV and video (UXTV’08). ACM, New York, pp 85–94

    Chapter  Google Scholar 

  40. Nitta N, Takahashi Y, Babaguchi N (2009) Automatic personalized video abstraction for sports videos using metadata. Multimedia Tools and Applications 41(1):1–25

    Article  Google Scholar 

  41. Patel M, Gossweiler R, Sahami M, Blackburn J, Brown D, Knight A (2008) Google TV search: dual-wielding search and discovery in a large-scale product. In: Proceedings of the international conference on designing interactive user experiences for TV and video (UXTV’08). ACM, New York, pp 95–104. doi:10.1145/1453805.1453826

    Chapter  Google Scholar 

  42. Petersen MK, Butkus A (2008) Modeling emotional context from latent semantics. In: Proceedings of the international conference on designing interactive user experiences for TV and video (UXTV’08). ACM, New York, pp 63–66. doi:10.1145/1453805.1453819

    Chapter  Google Scholar 

  43. Pimentel MdGC, Cattelan RG, Melo EL, Teixeira CA (2008) End-user editing of interactive multimedia documents. In: Proceedings of the ACM symposium on document engineering (DOCENG’08). ACM, New York, pp 298–301. doi:10.1145/1410140.1410204

    Google Scholar 

  44. Pimentel MdGC, Cattelan RG, Freitas G, Melo EL, Teixeira CAC (2009) Watch-and-comment as an approach to collaborative annotate points of interest in video and interactive-TV programs. In: Marcus A, Roibás AC, Sala R (eds) Mobile TV: customizing content and experience. Springer, Berlin, pp 349–366

    Google Scholar 

  45. Pimentel MdGC, Cattelan RG, Melo EL, Prado AF, Teixeira CAC (2010) End-user live editing of iTV programmes. Int J Adv Media Comm 4(1):78–103. doi:10.1504/IJAMC.2010.030007

    Article  Google Scholar 

  46. Ramos G, Balakrishnan R (2003) Fluid interaction techniques for the control and annotation of digital video. In: Proceedings of the ACM symposium on user interface software and technology (UIST’03), pp 105–114. doi:10.1145/964696.964708

  47. Rogge B, Bekaert J, de Walle R (2004) Timing issues in multimedia formats: review of the principles and comparison of existing formats. IEEE Trans Multimedia 6(6):910–924

    Article  Google Scholar 

  48. Schulzrinne H, Rao A, Lanphier R (1998) Real Time Streaming Protocol (RTSP). http://www.ietf.org/rfc/rfc2326.txt

  49. Soares LFG, de Souza Filho (2007) Interactive television in Brazil: system software and the digital divide. In: Proceedings of the European conference on interactive television (EUROITV’07) (LNCS 4417). Springer, Berlin, pp 41–44

    Google Scholar 

  50. Soares LFG, Rodrigues RF, Moreno MF (2007) Ginga-NCL: the declarative environment of the Brazilian digital TV system. J Braz Comput Soc 12(4):37–46

    Article  Google Scholar 

  51. Soares LF, Rodrigues RF, Cerqueira R, Barbosa SD (2010) Variable and state handling in NCL. Multimedia Tools and Applications 50:465–489. doi:10.1007/s11042-010-0478-2

    Article  Google Scholar 

  52. Teixeira CAC, Freitas G, Pimentel MdGC (2010) Distributed discrimination of media moments and media intervals: a watch-and-comment approach. In: Proceedings of the ACM symposium on applied computing (SAC’10), pp 1929–1935

  53. Tjondronegoro D, Chen YPP, Pham B (2004) Integrating highlights for more complete sports video summarization. IEEE MultiMed 11:22–37. doi:10.1109/MMUL.2004.28

    Article  Google Scholar 

  54. Troncy R, Mannens E, Pfeiffer S, Deursen DV (2010) Media fragments URI 1.0—W3C working draft, 8 December 2010. http://www.w3.org/2008/WebVideo/Fragments/WD-media-fragments-spec/

  55. Vandermolen H, Wiegering A, Sommers M, Levine SJ, Franco A (2009) Identifying events of interest within video content. http://www.freepatentsonline.com/7624416.html US Patent(7624416)

  56. Vuorimaa P, Bulterman D, César, P (2008) SMIL timesheets 1.0—W3C working draft, 10 January 2008. http://www.w3.org/TR/2008/WD-timesheets-20080110/

  57. Wei Y, Bhandarkar SM, Li K (2009) Client-centered multimedia content adaptation. ACM Trans Multimed Comput Comm Appl (ACM TOMCCAP) 5(22):1–22:26. doi:10.1145/1556134.1556139

    Article  Google Scholar 

  58. Xie L, Sundaram H, Campbell M (2008) Event mining in multimedia streams. Proc IEEE 96: 623–647

    Article  Google Scholar 

  59. Yamamoto M, Nitta N, Babaguchi N (2006) Estimating intervals of interest during TV viewing for automatic personal preference acquisition. In: Zhuang Y, Yang S, Rui Y, He Q (eds) Advances in multimedia information processing (PCM’06) (LNCS 4261). Springer, Berlin, pp 615–623

    Chapter  Google Scholar 

Download references

Acknowledgements

We thank the several agencies which provide funds to our research: FAPESP, CAPES, CNPq, FINEP, and MCT/CTIC. We also thank Rudinei Goularte, Renan G. Cattelan, Luiz F. G. Soares, Dick C. A. Bulterman and Pablo S. César for many suggestions and challenging questions. We also thank the anonymous reviewers for their suggestions and advice.

Author information

Authors and Affiliations

  1. Departamento de Computação, Universidade Federal de São Carlos, São Carlos, Brazil

    Cesar A. C. Teixeira, Erick L. Melo & Giliard B. Freitas

  2. Departamento de Ciências de Computação, Universidade Federal da Bahia, Salvador, Bahia, Brazil

    Celso A. S. Santos

  3. Departamento de Ciências de Computação, Instituto de Ciências Matemáticas e de Computação, Universidade de São Paulo, São Paulo, Brazil

    Maria da Graça C. Pimentel

Authors
  1. Cesar A. C. Teixeira
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Erick L. Melo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Giliard B. Freitas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Celso A. S. Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Maria da Graça C. Pimentel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maria da Graça C. Pimentel.

Additional information

This work has been supported in Brazil by grants from CAPES, CNPq, FAPESP, FINEP and MCT.

Appendix: NCL documents used in experiment

Appendix: NCL documents used in experiment

The NCL program in Listing 2, equivalent to the one presented in Listing 1, corresponds to a sticker presenting the Corinthians logo and anthem for 15 s.

The NCL code in Listing 3 is equivalent to the one used in the original broadcast.

The NCL program in Listing 4, edited on the fly on the set-top-box, extends the program in Listing 3 to allow a user to add the sticker (Listing 2) while watching the match. The presentation of the sticker is activated when the user presses the red button in the remote control, and may be canceled by the user pressing the blue button.

It is interesting to observe than the new document is simple, as it should be, and that the temporal consistency is maintained, as expected [32].

Rights and permissions

Reprints and permissions

About this article

Cite this article

Teixeira, C.A.C., Melo, E.L., Freitas, G.B. et al. Discrimination of media moments and media intervals: sticker-based watch-and-comment annotation. Multimed Tools Appl 61, 675–696 (2012). https://doi.org/10.1007/s11042-011-0846-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-011-0846-6

Keywords

Search

Navigation