Skip to main content
SpringerLink
Log in

A semantic parliamentary multimedia approach for retrieval of video clips with content understanding

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Digital videos of parliamentary activity play an important role in enhancing transparency and accountability for open e-government. The rapid growth in these videos and the lack of semantic annotations and relationships between video and knowledge resources make it increasingly difficult to find accurate video clips with contextual information for content understanding. To overcome this problem, we highlight the need for building multimedia systems based on a semantic vision. With this aim, we focus on (1) how to address the knowledge representation for automatic extraction of contextual information for video content understanding; (2) how to link the parliamentary knowledge structures within video resources to provide accurate video clips retrieval; and (3) how to perform semantic annotation on video resources. The methodology applied is focused on a systematic approach that uses techniques from ontology engineering. This approach is based on the definition of two models: the semantic model and the reference architecture. The semantic model is composed of a reference ontology and a semantic video annotation framework. The ontology provides the support for video content understanding and the semantic vocabulary for annotating video resources. The video annotation framework is based on an RDF-powered semantic video annotation to effectively relate low- and mid-level visual features, corresponding to speakers’ interventions, to high-level parliamentary concepts. To evaluate the proposed system, a prototype for the Canary Islands Parliament (Spain) has been carried out. The results show how semantic enhancement is a key enabler for improved video retrieval on parliamentary multimedia content.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://xmlns.com/foaf/spec/.

  2. http://rdfs.org/sioc/spec/.

  3. http://lov.okfn.org/dataset/lov/vocabs/kdo.

  4. http://swrl.stanford.edu/ontologies/built-ins/3.3/temporal.owl.

  5. http://www.parcan.es/video/canales.py.

References

  1. Arndt R, Troncy R, Staab S, Hardman L, Vacura M (2007) COMM: Designing a Well-Founded Multimedia Ontology for the Web. In International Semantic Web Conference, Lecture Notes in Computer Science, Vol. 4825, pp. 30-43

  2. Aydinhlar, M., Yazici, A.: Semi-automatic video annotation tool. In: Gelenbe, E., Lent, R. (eds.) Computer and Information Sciences III, pp. 303–310 (2013)

  3. Bertini, M., Del Bimbo, A., Torniai, C., Cucchiara, R., Grana, C.: MOM: multimedia ontology manager. A framework for automatic annotation and semantic retrieval of video sequences. In ACM Multimedia, pp. 787–788 (2006)

  4. Bertini, M., d’Amico, G., Ferracani, A., Meoni, M., Serra, G.: Sirio, Orione and Pan: an integrated web system for ontology-based video search and annotation. In: ACM International Conference on Multimedia, pp. 1625–1628 (2010)

  5. Bizer, C., Heath, T., Berners-Lee, T.: Linked data—the story so far. Int. J. Semant. Web Inf. Syst. 5(3), 1–22 (2009)

    Article  Google Scholar 

  6. Breuker, J.: Constructing a legal core ontology: LRI-core. In: Proc. of the workshop on ontologies and their applications, pp. 115–126 (2004)

  7. Bohlken, W., Neumann, B., Hotz, L., Koopmann, P.: Ontology-based realtime activity monitoring using beam search. Lect. Notes Comput. Sci. 6962, 112–121 (2011)

    Article  Google Scholar 

  8. Burger, T., Simperl, E.: A conceptual model for publishing multimedia content on the semantic web. In SAMT conference 2009. Lect. Notes Comput. Sci. 5887, 101–113 (2009)

    Article  Google Scholar 

  9. Casellas, N., Casanovas, P., Vallbé, J., Poblet, M., Bláquez, M., Contreras, J., López-Cobo, J.M., Benjamins, V.R.: Semantic enhancement for legal information retrieval: Iuriservice performance. In: Proc of the 12th ACM International Conference on Artificial Intelligence and Law, pp. 49–57 (2007)

  10. Engel, A.: Verification, Validation and testing of engineered system. Wiley, Hoboken (2010)

    Book  Google Scholar 

  11. Eurovoc. EU’s multilingual thesaurus. http://eurovoc.europa.eu/drupal/ (2014). Accessed Nov 2017

  12. Global Centre for ICT in Parliament United Nations Department of Economic and Social Affairs. World e-Parliament Report 2012. In: Inter-Parliamentary Union Press, by the Division for Public Administration and Development Management of the United Nations (2012)

  13. Gómez-Romero, J., Patricio, M.A., García, J., Molina, J.M.: Ontology-based context representation and reasoning for object tracking and scene interpretation in video. Expert Syst. Appl. 38, 7494–7510 (2010)

    Article  Google Scholar 

  14. Haslhofer, B., Jochum, W., King, R., Sadilek, C., Schellner, K.: The LEMO annotation framework: weaving multimedia annotations with the web. Int. J. Digit. Libr. 10(1), 15–32 (2009)

    Article  Google Scholar 

  15. Heggland, J.: Ontolog: temporal annotation using ad hoc ontologies and application profiles. Lect. Notes Comput. Sci. 2458, 118–128 (2002)

    Article  MATH  Google Scholar 

  16. Hunter, J., Newmarch, J.: An indexing, browsing, search and retrieval system for audiovisual libraries. Lect. Notes Comput. Sci. 1696, 76–91 (1999)

    Article  Google Scholar 

  17. Juric, D., Hollink, L., Houben, G.: Bringing parliamentary debates to the semantic web. In: Proceedings of the Workshop on detection, representation, and exploitation of events in the semantic web (2012)

  18. Kralingen, R.W., Visser, P.R., Bench-Capon, T.J., van Den Herik, T.: A principled approach to developing legal knowledge systems. Int. J. Hum. Comput. Stud. 51(6), 1121–1154 (1999)

    Google Scholar 

  19. Li, Y., Wald, M., Wills, G.: Applying linked data in multimedia annotations. Int. J. Semant. Comput. 6(3), 203–237 (2012)

    Article  Google Scholar 

  20. Lombardo, V., Pizzo, A.: Ontology-based visualization of characters’ intentions. Lect. Notes Comput. Sci. 8832, 176–187 (2014)

    Article  Google Scholar 

  21. Mannens, E., Derusen, D., Troncy, R., Pfeiffer, S., Parker, C., Lafon, Y., Jansen, J., Haunsenblas, M., Vand de Wall, R.: A URI-based approach for addressing fragments of media resources on the Web. Multimedia Tools Appl. 59, 691–715 (2012). https://doi.org/10.1007/s11042-010-0683-z

    Article  Google Scholar 

  22. Marín-Reyes, P., Lorenzo-Navarro, J., Castrillón-Santana, M., Sánchez-Nielsen, E.: Shot classification and keyframe detection for vision based speakers diarization in parliamentary debates. Lect Notes Artif Intell 9868, 48–57 (2016)

    Google Scholar 

  23. Nixon, L., Bauer, M., Bara, C., Kurt, T., Pereira, J.: ConnectME: semantic tools for enriching online video with web content. In: 8th International Conference on Semantic Systems, pp. 55–62 (2012)

  24. Ontomedia. http://ontomedia.de (2006). Accessed Nov 2017

  25. OpenLink Software. Virtuoso Universal Server. http://virtuoso.openlinksw.com/ (2014). Accessed Nov 2017

  26. Poppe, C., Martens, G., De Potter, P., Van de Walle, R.: Semantic web technologies for video surveillance metadata. Multimedia Tools Appl. 56(3), 439–467 (2012)

    Article  Google Scholar 

  27. Protégé. http://protege.standford.edu/ (2016). Accessed Nov 2017

  28. Scherp, A., Eibing, D., Saathoff, C.: A Method for integrating multimedia metadata standards and metadata formats with the multimedia ontology. Int. J. Semant. Comput. 6(1), 20–25 (2012)

    Article  Google Scholar 

  29. Schroeter, R., Hunter, J., Kowsovic, D.: Vannotea: a collaborative video indexing, annotation and discussion system for broadband networks. In: Proceedings of the second international conference on knowledge capture, pp. 1–8 (2003)

  30. Sikos, L.F., Powers, D.M.W.: Knowledge-driven video information retrieval with LOD: from semi-structured t structured video metadata. In: Exploiting semantic annotations in information retrieval, pp. 35–37 (2015)

  31. Sikos, L.: RDF-powered semantic video annotation tools with concept mapping to Linked Data for next-generation video indexing: a comprehensive review. Multimedia Tools Appl. 76, 14437–14460 (2017)

    Article  Google Scholar 

  32. Suárez-Figueroa, M.C., Gómez-Pérez, A., Fernández-López, M.: The NeOn methodology for ontology engineering. Ontology Engineering in a Networked World, pp. 9–34. Springer, Berlin (2012)

    Book  Google Scholar 

  33. Studer, R., Benjamins, R., Fensel, D.: Knowledge engineering: principles and methods. IEEE Trans. Data and Knowl. Eng. 25(1–2), 161–197 (1998)

    Article  MATH  Google Scholar 

  34. Yu, H., Pedrinaci, C., Dietze, S., Domingue, J.: Using linked data to annotate and search educational video resources for supporting distance learning. IEEE Trans. Learn. Technol. 5(2), 130–142 (2012)

    Article  Google Scholar 

  35. Valente, A.: Legal knowledge engineering, frontiers in artificial intelligence and applications, vol. 30. IOS Press, The Netherlands (1995). ISBN 978-90-5199-230-4

    Google Scholar 

  36. Wowza media systems. http://www.wowza.com (2017). Accessed Nov 2017

  37. W3C. W3C media annotation working group. ontology for media resources 1.0. http://www.w3.org/TR/mediaont-10/ (2012). Accessed Nov 2017

  38. W3C. W3C Semantic Web. Activity Resource Description Framework 1.1 (RDF) http://www.w3.org/RDF/ (2014a). Accessed Nov 2017

  39. W3C. W3C Semantic Web activity. RDF Schema 1.1. http://www.w3.org/TR/rdf-schema/ (2014b). Accessed Nov 2017

  40. W3C. Web ontology language. http://www.w3.org/TR/owl-features/ (2004). Accessed Nov 2017

  41. W3C. Semantic web activity. SPARQL query language for RDF. http://www.w3.org/TR/rdf-sparql-query/ (2008). Accessed Nov 2017

Download references

Acknowledgements

This work has been supported by the EC (FEDER) and the Spanish MINECO with the I + D + I contract number: TIN2016-78919-R. The authors appreciate the reviewers’ comments and their effort to improve this paper.

Author information

Authors and Affiliations

  1. Departamento de Ingeniería Informáticay de Sistemas-Universidad de La Laguna, 38271, Santa Cruz de Tenerife, Spain

    Elena Sánchez-Nielsen & Francisco Chávez-Gutiérrez

  2. Parlamento de Canarias, Teobaldo Power 7, 38002, Santa Cruz de Tenerife, Spain

    Francisco Chávez-Gutiérrez

  3. SIANI-Universidad de Las Palmas de Gran Canaria, 35017, Gran Canaria, Spain

    Javier Lorenzo-Navarro

Authors
  1. Elena Sánchez-Nielsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francisco Chávez-Gutiérrez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Javier Lorenzo-Navarro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elena Sánchez-Nielsen.

Additional information

Communicated by T. Plagemann.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sánchez-Nielsen, E., Chávez-Gutiérrez, F. & Lorenzo-Navarro, J. A semantic parliamentary multimedia approach for retrieval of video clips with content understanding. Multimedia Systems 25, 337–354 (2019). https://doi.org/10.1007/s00530-019-00610-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-019-00610-2

Keywords

Search

Navigation