Multimedia Tools and Applications

, Volume 77, Issue 18, pp 24313–24331 | Cite as

A video engine supported by social buzz to automatically create TV summaries

  • Pedro Almeida
  • Jorge Ferraz de Abreu
  • Rita Oliveira
  • Diogo Gomes


Viewers post a lot of TV program-related information on social networks while they are watching TV, especially during its key moments. Therefore, this social buzz has the potential to be used as an automatic editorial criterion. Having this premise in consideration, this paper reports on the nowUP solution, a service developed with the main goal of automatically creating TV summaries of popular television programs (like football matches, talent or reality shows) based on the Twitter activity and integrating a part of that activity in the TV show summary. A data-mining engine continuously processes the activity of this social network looking for tweets associated with the TV shows. Based on the program metadata it indexes the twitter activity; correlates tweets; and creates clusters of peaks, being the relevant clusters associated with the highlights of the TV show. With this, the video engine automatically creates a full video summary (an edited sequence of TV highlights) and publishes it in an online platform and on a Catch-up TV service. The paper reports on the nowUP development and on the results of its evaluation, namely comparing its outputs with official editorial/professional video summaries. The results show that the solution was very successful in achieving the project main goal and users want to have access to this type of social buzz-based video summaries. The nowUP solution also promises potential gains in the value chain of TV producers and broadcasters.


TV summary Highlights Twitter activity Evaluation 



The authors would like to acknowledge the remaining partners of the nowUP project (Institute of Telecommunications of the University of Aveiro and Altice Labs).


  1. 1.
    Abel F, Gao Q, Houben GJ, Tao K (2011) Semantic enrichment of twitter posts for user profile construction on the social web. Proceedings of the 8th extended semantic web conference on The semanic web: research and applications - Volume Part II (ESWC'11). Springer-Verlag, Berlin, 375–389Google Scholar
  2. 2.
    Abreu J, Almeida P, Teles B, Reis M (2013) Viewer behaviors and practices in the (new) Television Environment. Proceedings of the 11th European conference on Interactive TV and video (EuroITV '13). ACM, New York, 5–12. doi: 10.1145/2465958.2465970Google Scholar
  3. 3.
    Abreu J, Nogueira J, Becker V (2016) Survey of Catch-up TV and Other Time-Shift Services: A Comprehensive Analysis and Taxonomy of Linear and Nonlinear Television. Telecommun Syst 64:57–74. CrossRefGoogle Scholar
  4. 4.
    Alonso O, Shiells K (2013) Timelines as summaries of popular scheduled events. Proceedings of the 22nd International World Wide Web Conference Committee (IW3C2):1037–1044.
  5. 5.
    Beuker I (2012) The Social EPG: The Next Step Towards Social TV? Viral Blog. Accessed 21 April 2017Google Scholar
  6. 6.
    Bradley M, Lang P (1994) Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. Journal Behav. Ther. Exp. Psychiatry, Mar; 25(1):49–59.
  7. 7.
    Capon G (2014) Twitter Amplify - Harmonize the two-screen experience. The Guardian. Accessed 21 April 2017
  8. 8.
    Cremonesi P et al. (2013) TV program detection in tweets. Proceedings of the 11th European Conference on Interactive TV and Video (EuroITV '13). 45–54. doi: 10.1145/2465958.2465960Google Scholar
  9. 9.
    Crisci A, Grasso V, Nesi P et al (2017) Predicting TV programme audience by using twitter based metrics. Multimed Tools Appl, Published Online:1–30.
  10. 10.
    Doman K et al. (2014) Event Detection based on Twitter Enthusiasm Degree for Generating a Sports Highlight Video. Proceedings of 22nd ACM International Conference on Multimedia (ACM-MM2014), pp.949–952. doi:
  11. 11.
    Dumont E, Quénot G (2012) Automatic Story Segmentation for TV News Video Using Multiple Modalities. Int J Digit Multimed Broadcast 2012:11. Google Scholar
  12. 12.
    Flick U (2009) An introduction to qualitative research (4th ed.). Sage Publications, LondonGoogle Scholar
  13. 13.
    Forlines C, Peker K, Divakaran A (2006) Subjective assessment of consumer video summarization. SPIE Proceedings - Special Session: Evaluating Video Summarization, Browsing, and Retrieval Techniques 6073.
  14. 14.
    Genc Y, Sakamoto Y, Nickerson J (2011) Discovering context: classifying tweets through a semantic transform based on wikipedia. Proceedings of the 6th international conference on Foundations of augmented cognition: directing the future of adaptive systems (FAC'11). Springer-Verlag, Berlin, 484–492Google Scholar
  15. 15.
    Jiang W, Cotton C, Loui A (2011) Automatic consumer video summarization by audio and visual analysis. Proceedings of International Conference on Multimedia and Expo (ICME):1–6.
  16. 16.
    Kantar Media (2015) Who's Tweeting about TV in the UK? Kantar Media. Accessed 21 April 2017
  17. 17.
    Kim H-G et al (2008) Real-Time Highlight Detection in Baseball Video for TVs with Time-shift Function. IEEE Transactions on Consumer Electronics, Volume 54(2):831–838. CrossRefGoogle Scholar
  18. 18.
    Marlow S et al. (2002) Audio processing for automatic TV sports program highlights detection. Proceedings of Irish Signals and Systems Conference (ISSC 2002), 25–26.
  19. 19.
    Nielsen Social (2017) Social TV Analytics & Solutions. Nielsen Social. Accessed 21 April 2017
  20. 20.
    Nielsen Social (2017) TV Season in review: the top social moments of the 2016–17 season. Nielsen Social. Accessed 30 Aug 2017
  21. 21.
    Nielsen Social (2017) What We Do. Nielsen Social. Accessed 21 April 2017
  22. 22.
    Otani M, Nakashima Y, Sato T et al (2017) Video summarization using textual descriptions for authoring video blogs. Multimed Tools Appl 76(9):12097–12115. CrossRefGoogle Scholar
  23. 23.
    Outhwaite W, Turner S (2007) The Sage Handbook of Social Science Methodology, 1st ed., Sage PublicationsGoogle Scholar
  24. 24.
    Ozdikis O, Senkul P, Oguztuzun H (2012) Semantic expansion of hashtags for enhanced event detection in Twitter. Proceedings of the International Conference Advances in Social Networks Analysis and Mining (ASONAM). IEEE, 20–24. doi: 10.1109/ASONAM.2012.14Google Scholar
  25. 25.
    Sgarbi E, Borges D (2005) Structure in Soccer Videos: Detecting and Classifying Highlights for Automatic Summarization. In: Sanfeliu A, Cortés ML (eds) Progress In Pattern Recognition, Image Analysis And Applications, Ciarp 2005, LNCS, vol 3773. Springer, Heidelberg, pp 691–700. Google Scholar
  26. 26.
    Tellyo (2017) Media sharing. Tellyo. Accessed 21 April 2017
  27. 27.
    Twitter Biz (2016) Twitter Amplify will create enormous value for broadcasters and brands. Twitter Business. Accessed 11 Jan 2016
  28. 28.
    Ulanoff L (2015) Twitter experiments with TV Timelines. Mashable. Accessed 21 April 2017
  29. 29.
    Viacom (2013) When Networks Network - TV Gets Social. Viacom Inc. Accessed 11 Jan 2016
  30. 30.
    Vilaça A, Antunes M, Gomes D (2015) TV-Pulse: Detecting TV highlights in Social Networks. Proceedings of ConfTele 2015 - 10th Conference on Telecommunications, Aveiro, PortugalGoogle Scholar
  31. 31.
    Vilaça A, Antunes M, Gomes D (2015) TV-Pulse: Improvements on detecting TV highlights in Social Networks using metadata and semantic similarity. Proceedings of 14th Conferência sobre Redes de Computadores, Évora, PortugalGoogle Scholar
  32. 32.
    WildMoka (2014) Moments Capture. WildMoka Website. Accessed 11 Jan 2016
  33. 33.
    WildMoka (2014) Moments Share. WildMoka Website. Accessed 11 Jan 2016
  34. 34.
    WildMoka (2014) Platform. WildMoka Website. Accessed 11 Jan 2016
  35. 35.
    WildMoka (2014) Press Release – CANAL+ Group selects Moments Share. WildMoka Website. Accessed 11 Jan 2016
  36. 36.
    WildMoka (2014) Moments Replay. WildMoka Website. htttp:// Accessed 11 Jan 2016
  37. 37.
    Yang J, Huang Y, Tsai C et al. (2009) An Automatic Multimedia Content Summarization System for Video Recommendation. Educational Technol & Soc, 12 (1), 49–61.
  38. 38.
    Yang J, Li J, Liu S (2017) A novel technique applied to the economic investigation of recommender system. Multimed Tools Appl, Published Online:1–16.
  39. 39.
    Zhang J, Li X, Nie W et al (2017) Automatic report generation based on multi-modal information. Multimed Tools Appl 76(9):12005–12015. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Pedro Almeida
    • 1
  • Jorge Ferraz de Abreu
    • 1
  • Rita Oliveira
    • 1
  • Diogo Gomes
    • 2
  1. 1.Digimedia, Department of Communication and ArtUniversity of AveiroAveiroPortugal
  2. 2.Institute of Telecommunications, Department of Electronics, Telecommunications and InformaticsUniversity of AveiroAveiroPortugal

Personalised recommendations