Promoting free Dialog Video Corpora: The IFADV Corpus Example

  • R. J. J. H. van Son
  • Wieneke Wesseling
  • Eric Sanders
  • Henk van den Heuvel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5509)


Research into spoken language has become more visual over the years. Both fundamental and applied research have progressively included gestures, gaze, and facial expression. Corpora of multi-modal conversational speech are rare and frequently difficult to use due to privacy and copyright restrictions. In contrast, Free-and-Libre corpora would allow anyone to add incremental annotations and improvement, distributing the cost of construction and maintenance. A freely available annotated corpus is presented with high quality video recordings of face-to-face conversational speech. An effort has been made to remove copyright and use restrictions. Annotations have been processed to RDBMS tables that allow SQL queries and direct connections to statistical software. A few simple examples are presented to illustrate the use of a databases of annotated speech. From our experiences we would like to advocate the formulation of “best practises” for both legal handling and database storage of recordings and annotations.


Word Alignment Conversational Speech Language Corpus Functional Label Copyright Restriction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Benson, D.A., Karsch-Mizrachi, I., Lipman, D.J., Ostell, J., Wheeler, D.L.: Genbank. Nucleic Acids Research 35(Database-Issue), 21–25 (2007)CrossRefGoogle Scholar
  2. 2.
    Kolbitsch, J., Maurer, H.: The transformation of the web: How emerging communities shape the information we consume. Journal of Universal Computer Science 12(2), 187–213 (2006)Google Scholar
  3. 3.
    Lerner, J., Tirole, J.: Some simple economics of open source. Journal of Industrial Economics 50, 197–234 (2002)CrossRefGoogle Scholar
  4. 4.
    Ciffolilli, A.: The economics of open source hijacking and declining quality of digital information resources: A case for copyleft. Development and Comp. Systems 0404008, EconWPA (April 2004)Google Scholar
  5. 5.
    Rullani, F.: Dragging developers towards the core. how the free/libre/open source software community enhances developers’ contribution. LEM Papers Series 2006/22, Laboratory of Economics and Management (LEM), Sant’Anna School of Advanced Studies, Pisa, Italy (September 2006)Google Scholar
  6. 6.
    ELRA: European Language Resources Association: Catalogue of Language Resources (2004–2007),
  7. 7.
    LDC: The Language Data Consortium Corpus Catalog (1992–2007),
  8. 8.
    HLT-Agency: Centrale voor Taal- en Spraaktechnologie, TST-centrale (2007),
  9. 9.
    MAPtask: HCRC Map Task Corpus (1992–2007),
  10. 10.
    Blache, P., Rauzy, S., Ferré, G.: An XML Coding Scheme for Multimodal Corpus Annotation. In: Proceedings of Corpus Linguistics (2007)Google Scholar
  11. 11.
    Bertrand, R.: Corpus d’interactions dilogales, CID (2007),
  12. 12.
  13. 13.
    CGN: The Spoken Dutch Corpus project (2006),
  14. 14.
    SMIL: W3C Synchronized Multimedia Integration Language (2008),
  15. 15.
    Ide, N., Romary, L.: Outline of the international standard linguistic annotation framework. In: Proceedings of the ACL 2003 workshop on Linguistic annotation, Morristown, NJ, USA, Association for Computational Linguistics, pp. 1–5 (2003)Google Scholar
  16. 16.
    Schmidt, T., Chiarcos, C., Lehmberg, T., Rehm, G., Witt, A., Hinrichs, E.: Avoiding data graveyards: From heterogeneous data collected in multiple research projects to sustainable linguistic resources. In: Proceedings of the E-MELD 2006 Workshop on Digital Language Documentation: Tools and Standards: The State of the Art, Lansing, Michigan (2006)Google Scholar
  17. 17.
    Rehm, G., Witt, A., Hinrichs, E., Reis, M.: Sustainability of annotated resources in linguistics. In: Proceedings of Digital Humanities 2008, Oulu, Finland, pp. 27–29 (2008)Google Scholar
  18. 18.
    IDABC: European Interoperability Framework for Pan-European eGovernmentservices (2004),
  19. 19.
    Ken Coar: The Open Source Definition, Annotated (2006),
  20. 20.
    Schmidt, T., Duncan, S., Ehmer, O., Hoyt, J., Kipp, M., Loehr, D., Magnusson, M., Rose, T., Sloetjes, H.: An exchange format for multimodal annotations. In: (ELRA), E.L.R.A. (ed.) Proceedings of the Sixth International Language Resources and Evaluation (LREC 2008), Marrakech, Morocco (May 2008)Google Scholar
  21. 21.
    Carletta, J., Isard, A., Isard, S., Kowtko, J., Doherty-Sneddon, G., Anderson, A.: The reliability of a dialogue structure coding scheme. Computational Linguistics 23, 13–31 (1997)Google Scholar
  22. 22.
    Core, M., Allen, J.: Coding dialogs with the damsl annotation scheme. In: AAAI Fall Symposium on Communicative Action in Humans and Machines, pp. 28–35 (1997)Google Scholar
  23. 23.
    ELAN: ELAN is a professional tool for the creation of complex annotations on video and audio resources (2002–2007),
  24. 24.
    Caspers, J.: Local speech melody as a limiting factor in the turn-taking system in dutch. Journal of Phonetics 31(2), 251–276 (2003)CrossRefGoogle Scholar
  25. 25.
    Wesseling, W., van Son, R.J.J.H.: Early Preparation of Experimentally Elicited Minimal Responses. In: Proceedings of the 6th SIGdial Workshop on Discourse and Dialogue, pp. 11–18 (2005)Google Scholar
  26. 26.
    Mengel, A., Heid, U.: Enhancing reusability of speech corpora by hyperlinked query output. In: Proceedings of EUROSPEECH 1999, Budapest, pp. 2703–2706 (1999)Google Scholar
  27. 27.
    Cassidy, S.: Compiling multi-tiered speech databases into the relational model: Experiments with the EMU system. In: Proceedings of EUROSPEECH 1999, Budapest, pp. 2239–2242 (1999)Google Scholar
  28. 28.
    Van Son, R., Binnenpoorte, D., van den Heuvel, H., Pols, L.: The IFA corpus: a phonemically segmented Dutch Open Source speech database. In: Proceedings of EUROSPEECH 2001, Aalborg, pp. 2051–2054 (2001)Google Scholar
  29. 29.
    Van Son, R., Pols, L.: Structure and access of the open source IFA Corpus. In: Proceedings of the IRCS workshop on Linguistic Databases, Philadelphia, pp. 245–253 (2001)Google Scholar
  30. 30.
    R Core Team: The R Project for Statistical Computing (1998–2008),
  31. 31.
    IMDI: ISLE Meta Data Initiative (1999–2007),
  32. 32.
    WIPO: Berne Convention for the Protection of Literary and Artistic Works (1979),
  33. 33.
    WIPO: 5: International Treaties and Conventions on Intellectual Property. In: WIPO Handbook on Intellectual Property: Policy, Law and Use, 2nd edn., pp. 237–364. WIPO (2004), (Date of access: March 2008)
  34. 34.
    Maurer, S.M., Hugenholtz, P.B., Onsrud, H.J.: Europe’s database experiment. Science 294, 789–790 (2001)Google Scholar
  35. 35.
    Kienle, H.M., German, D., Tilley, S., Müller, H.A.: Intellectual property aspects of web publishing. In: SIGDOC 2004: Proceedings of the 22nd annual international conference on Design of communication, pp. 136–144. ACM, New York (2004)Google Scholar
  36. 36.
    EC: First evaluation of Directive 96/9/EC on the legal protection of databases, DG Internal Market and Services Working Paper (2005),
  37. 37.
    FSF: GNU General Public License, version 2 (1991),
  38. 38.
    IDABC: European Union Public Licence, EUPL v.1.0 (2008),
  39. 39.
    ten Bosch, L., Oostdijk, N., Boves, L.: On temporal aspects of turn taking in conversational dialogues. Speech Communication 47(1-2), 80–86 (2005)CrossRefGoogle Scholar
  40. 40.
    : Community cleverness required. Nature 455(7209), 1 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • R. J. J. H. van Son
    • 1
  • Wieneke Wesseling
    • 1
  • Eric Sanders
    • 2
  • Henk van den Heuvel
    • 2
  1. 1.ACLC/IFAUniversity of AmsterdamThe Netherlands
  2. 2.SPEX/CLSTRadboud University NijmegenThe Netherlands

Personalised recommendations