Skip to main content
SpringerLink
Log in

Contribution of NLP to the Content Indexing of Multimedia Documents

  • Conference paper
Image and Video Retrieval (CIVR 2004)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 3115))

Included in the following conference series:

Abstract

This paper describes the role natural language processing (NLP) can play for multimedia applications. As an example of such an application, we present an approach dealing with the conceptual indexing of soccer videos which the help of structured information automatically extracted by NLP tools from multiple sources of information relating to video content, consisting in a rich range of textual and transcribed sources covering soccer games. This work has been investigated and developed in the EU funded project MUMIS. As a second example of such an application, we describe briefly ongoing work in the context of the Esperonto project dealing with upgrading the actual web towards the Semantic Web (SW), including the automatic semantic indexing of web pages containing a combination of text and images.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Adani, N., Bugatti, A., Leonardi, R., Migliorati, P.: Semantic description of multimedia documents: the Mpeg-7 approach. In: Proceedings of the Conference on Content-Based Multimedia Indexing, CBMI 2001, Brescia (2001)

    Google Scholar 

  2. André, E.: Natural Language in Multimedia/Multimodal Systems. In: Mitkov, R. (ed.) Handbook of Computational Linguistics, Oxford (2000)

    Google Scholar 

  3. André, E.: The Generation of Multimedia Presentations. In: Handbook of Natural Language Processing, Marcel Dekker, New York (2000)

    Google Scholar 

  4. Assfalg, J., Bertini, M., Colombo, C., Del Bimbo, A.: Semantic annotations of sports videos. In: Proceedings of the Conference on Content-Based Multimedia Indexing, CBMI 2001, Brescia (2001)

    Google Scholar 

  5. Cunningham, H.: Information Extraction: A user Guide, Research Report CS-99-07, Department of Computer Science, May 1999. University of Sheffield (1999)

    Google Scholar 

  6. Day, N.: MPEG-7 Applications: Multimedia Search and Retrieval. In: Proceedings of the First International Workshop on Multimedia Annotation, MMA 2001 (2001)

    Google Scholar 

  7. Declerck, T., Wittenburg, P., Cunningham, H.: The Automatic Generation of Formal Annotations in a Multimedia Indexing and Searching Environment. In: Proceedings of the Workshop on Human Language Technology and Knowledge Management, ACL 2001 (2001)

    Google Scholar 

  8. Declerck, T.: A set of tools for integrating linguistic and non-linguistic information. In: Proceedings of SAAKM 2002, ECAI 2002, Lyon (2002)

    Google Scholar 

  9. Djoerd, H., de Jong, F., Netter, K. (eds.): 14th Twente Workshop on Language Technology, Language Technology in Multimedia Information Retrieval, TWLT 14, Enschede, Universiteit Twente (1998)

    Google Scholar 

  10. Johnston, M.: Unification-based Multimodal Parsing. In: Proceedings of the 17th International Conference on Computational Linguistics, COLING 1998 (1998)

    Google Scholar 

  11. de Jong, F., Gauvin, J., Hiemstra, D., Netter, K.: Language-Based Multimedia Information Retrieval. In: Proceedings of the 6th Conference on Recherche d’Information Assistee par Ordinateur, RIAO-2000, 2000. IndexingWorkshop (CBMI 2001) (2001)

    Google Scholar 

  12. Krieger, H.-U., Schaefer, U.: TDL – a type description language for constraint-based grammars. In: Proceedings of the 15th International Conference on Computational Linguistics, COLING 1994 (1994)

    Google Scholar 

  13. Maybury, M.: Multimedia Interaction for the New Millenium. In: Proceedings of Eurospeech 1999 (1999)

    Google Scholar 

  14. McKeown, K.: Text generation. Cambridge University Press, Cambridge (1985)

    Book  Google Scholar 

  15. Merlino, A., Morey, D., Maybury, M.: Broadcast News Navigation using Story Segments. In: ACM International Multimedia Conference (1997)

    Google Scholar 

  16. Miller, G.A.: WordNet: A Lexical Database for English. Communications of the ACM 11 (1995)

    Google Scholar 

  17. Moore, J., Paris, C.: Planning Text for Advisory Dialogues. In: Proceedings of the 27th ACL, Vancouver (1989)

    Google Scholar 

  18. Sixth Message Understanding Conference (MUC-6). Morgan Kaufmann, San Francisco (1995)

    Google Scholar 

  19. Seventh Message Understanding Conference (MUC-7) SAIC Information Extraction (1998), http://www.muc.saic.com/

  20. Naphade Milid, R., Huang, T.S.: Recognizing high-level concepts for video indexing. In: Proceedings of the Conference on Content-Based Multimedia Indexing, CBMI 2001, Brescia (2001); Extraction and Navigation System. In: Proceedings of the 6th Conference on Recherche d’Information Assistee par Ordinateur, RIAO 2000 (2000)

    Google Scholar 

  21. Saggion, H., Cunningham, H., Bontcheva, K., Maynard, D., Ursu, C., Hamza, O., Wilks, Y.: Access to Multimedia Information through Multisource and Multilanguage Information Extraction. In: Andersson, B., Bergholtz, M., Johannesson, P. (eds.) NLDB 2002. LNCS, vol. 2553, pp. 160–171. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  22. Salembier, P.: An overview of Mpeg-7 multimedia description schemes and of future visual information challenges for content-based indexing. In: Proceedings of the Conference on Content-Based Multimedia Indexing, CBMI 2001, Brescia (2001)

    Google Scholar 

  23. Salway, A.: Talking Pictures: Indexing and Representing Video with Collateral Texts. In: Hiemstra, D., de Jong, F., Netter, K. (eds.) Language Technology in Multimedia Information Retrieval (Proceedings of the 14th Twente Workshop on Language Technology, TWLT 14), Enschede, Universiteit Twente (1998)

    Google Scholar 

  24. Staab, S., Maedche, A., Handschuh, S.: An Annotation Framework for the Semantic Web. In: The First International Workshop on Multimedia Annotation, Tokyo, Japan (2001)

    Google Scholar 

  25. Wester, M., Kessens, J.M., Strik, H.: Goal-directed ASR in a Multimedia Indexing and Searching Environment (MUMIS). In: Proceedings of the 7th International Conference on Spoken Language Processsing (ICLSP 2002) (2002)

    Google Scholar 

  26. EUR: http://www.foyer.de/euromedia/

  27. GDA: http://www.csl.sony.co.jp/person/nagao/gda/

  28. INF: http://www.informedia.cs.cmu.edu/

  29. ISI: http://www.wins.uva.nl/research/isis/isisNS.html

  30. ISLE: http://www.ilc.pi.cnr.it/EAGLES/ISLE_Home_Page.htm

  31. NSF: http://www.nsf.gov/od/lpa/news/press/pr9714.htm

  32. OLI: http://twentyone.tpd.tno.nl/olive

  33. POP: http://twentyone.tpd.tno.nl/popeye

  34. SUR: http://www-rocq.inria.fr/nastar/MM98/node1.html

  35. THI: http://www.dcs.shef.ac.uk/research/groups/spandh/projects/thisl

  36. UMA: http://ciir.cs.umass.edu/research/

  37. UNL: http://www.ias.unu.edu/research_prog/science_technology_universalnetwork_languge.html

  38. VIR: http://www.virage.com/

  39. COL: http://www.cs.columbia.edu/hjing/sumDemo

  40. Veltkamp, R., Tanase, M.: Content-based Image Retrieval Systems: a survey. Technical report UU-CS-2000-34, Utrecht University (2000)

    Google Scholar 

  41. Chang, S.F., Chen, W., Meng, H.J., Sundaram, H., Zhong, D.: A Fully Automated Content-based Video Search Engine Supporting Spatio Temporal Queries. IEEE Transactions on Circuits and Systems for Video Technology (1998)

    Google Scholar 

  42. Netter, K.: Pop-Eye and OLIVE. Human Language as the Medium for Cross-lingual Multimedia Information Retrieval. Technical report, Language Technology Lab. DFKI GmbH (1998)

    Google Scholar 

  43. Sable, C., Hatzivassiloglou, V.: Text-based approaches for the categorization of images. In: Proceedings of ECDL (1999)

    Google Scholar 

  44. Srihari, R.K.: Automatic Indexing and Content-Based Retrieval of Captioned Images, Computer 28/9 (1995)

    Google Scholar 

  45. Gong, Y., Sin, L.T., Chuan, C.H., Zhang, H., Sakauchi, M.: Automatic Parsing of TV Soccer Programs. In: Proceedings of the International Conference on Multimedia Computing and Systems (IEEE) (1995)

    Google Scholar 

  46. Steinbiss, V., Ney, H., Haeb-Umbach, R., Tran, B.-H., Essen, U., Kneser, R., Oerder, M., Meier, H.-G., Aubert, X., Dugast, C., Geller, D.: The Philips Research System for Large-Vocabulary Continuous-Speech Recognition. In: Proc. of Eurospeech 1993 (1993)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Saarland University and DFKI GmbH, Stuhlsatzenhausweg 3, D66123, Saarbruecken, Germany

    Thierry Declerck, Jan Kuper, Horacio Saggion, Anna Samiotou, Peter Wittenburg & Jesus Contreras

Authors
  1. Thierry Declerck
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Kuper
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Horacio Saggion
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anna Samiotou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peter Wittenburg
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jesus Contreras
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Computing, Mathematical and Information Sciences, University of Brighton, UK

    Peter Enser

  2. Informatics and Telematics Institute, Centre for Research and Technology-Hellas, 57001, Thessaloniki, Greece

    Yiannis Kompatsiaris

  3. Centre for Digital Video Processing, Adaptive Information Cluster, Dublin City University, Ireland

    Noel E. O’Connor

  4. Dublin City University, Dublin, Ireland

    Alan F. Smeaton

  5. ISLA lab, Informatics Institute, University of Amsterdam, The Netherlands

    Arnold W. M. Smeulders

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Declerck, T., Kuper, J., Saggion, H., Samiotou, A., Wittenburg, P., Contreras, J. (2004). Contribution of NLP to the Content Indexing of Multimedia Documents. In: Enser, P., Kompatsiaris, Y., O’Connor, N.E., Smeaton, A.F., Smeulders, A.W.M. (eds) Image and Video Retrieval. CIVR 2004. Lecture Notes in Computer Science, vol 3115. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-27814-6_71

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-27814-6_71

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-22539-3

  • Online ISBN: 978-3-540-27814-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation