ANNODIS and Related Projects: Case Studies on the Annotation of Discourse Structure



In this paper we report on the efforts of three projects to annotate texts and dialogues with discourse structure. We provide a theoretical discussion of various alternatives and then present our approach to discourse structure annotation, along with some applications of the resources that we have developed.


Discourse structures Multiple annotation levels Bottom-up/Top-down discourse analysis Topical chains Enumerative structures Discourse relations 


  1. 1.
    Afantenos, S.D., Asher, N.: Testing SDRT’s right frontier. In: Proceedings of the 23rd International Conference on Computational Linguistics (COLING 2010), pp. 1–9 (2010)Google Scholar
  2. 2.
    Afantenos, S.D., Denis, P., Muller, P., Danlos, L.: Learning recursive segments for discourse parsing. In: Proceedings of LREC 2010 (2010)Google Scholar
  3. 3.
    Afantenos, S., Asher, N., Benamara, F., Bras, M., Fabre, C., Ho-Dac, L.M., Le Draoulec, A., Muller, P., Péry-Woodley, M. P., Prévot, L., Rebeyrolles, J., Tanguy, L., Vergez-Couret, M., Vieu, L.: An empirical resource for discovering cognitive principles of discourse organisation: the ANNODIS corpus. In: Calzolari, N., Choukri, K., Declerck, T., Doǧan, M.U., Maegaard, B., Mariani, J., Odijk, J., Piperidis, S. (eds.) Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), European Language Resources Association (ELRA). Istanbul, Turkey (2012)Google Scholar
  4. 4.
    Asher, N.: Reference to Abstract Objects in Discourse. Kluwer, The Netherlands (1993)CrossRefGoogle Scholar
  5. 5.
    Asher, N.: Lexical Meaning in Context: A Web of Words. Cambridge University Press, Cambridge (2011)CrossRefGoogle Scholar
  6. 6.
    Asher, N., Lascarides, A.: Logics of conversation. In: Studies in Natural Language Processing. Cambridge University Press, Cambridge (2003)Google Scholar
  7. 7.
    Asher, N., Hardt, D., Busquets, J.: Discourse parallelism, ellipsis and ambiguity. J. Semant. 18(1), (2001)Google Scholar
  8. 8.
    Asher, N., Benamara, F., Mathieu, Y.Y.: Distilling opinion in discourse: a preliminary study. In: Proceedings of Computational Linguistics (CoLing), pp. 7–10 (2008)Google Scholar
  9. 9.
    Atallah, C.: Analyse de relations de discours causales en corpus: étude empirique et caractérisation théorique. Ph.D. thesis, Université de Toulouse, Toulouse (2014)Google Scholar
  10. 10.
    Baldridge, J., Asher, N., Hunter, J.: Annotation for and Robust Parsing of Discourse Structure on Unrestricted Texts. Zeitschrift fur. Sprachwissenschaft 26, 213–239 (2007)Google Scholar
  11. 11.
    Benamara, F, Asher, N, Mathieu, Y, Popescu, V., Chardon, B.: Evaluation in discourse: a corpus-based study. In: Dialogue and Discourse (2015). (in press)Google Scholar
  12. 12.
    Biber, D.: Variation Across Speech and Writing. Cambridge University Press, Cambridge (1988)CrossRefGoogle Scholar
  13. 13.
    Bourigault, D.: Un analyseur syntaxique opérationnel : SYNTEX. Université de Toulouse, Mémoire d’HDR (2007)Google Scholar
  14. 14.
    Bras, M.: French adverb d’abord and discourse structure. In: Aurnague, M., Larrazabal, J-M., Korta, K. (eds.) Language, Representation and Reasoning. Memorial Volume to Isabel Gomez Txurruka, pp. 77–102. Presses Universitaires du Pays Basque, Bilbao (2007)Google Scholar
  15. 15.
    Bras, M., Le Draoulec, A., Vieu, L.: French adverbial Puis between temporal structure and discourse structure. In: Bras, M., Vieu, L. (eds.) Semantic and Pragmatic Issues in Dialogue: Experimenting with Current Theories. CRISPI, vol. 9, pp. 109–146. Elsevier, Amsterdam (2001)Google Scholar
  16. 16.
    Bras, M., Le Draoulec, A., Asher, N.: A formal analysis of the French temporal connective alors. Oslo Stud Lang 1, 149–170 (2009)Google Scholar
  17. 17.
    Carletta, J., Isard, S., Doherty-Sneddon, G.: HCRC Dialogue Structure Coding Manual. HCRC Publications, The University of Edinburgh (1996)Google Scholar
  18. 18.
    Chafe, W.L.: Discourse Consciousness and Time: The Flow and Displacement of Conscious Experience in Speaking and Writing. University of Chicago Press, Chicago (1994)Google Scholar
  19. 19.
    Chardon, B., Benamara, F., Mathieu, Y.Y., Popescu, V., Asher, N.: Measuring the effect of discourse structure on sentiment analysis. In: CICLing, pp. 25–37 (2013a)Google Scholar
  20. 20.
    Chardon, B., Benamara, F., Mathieu, Y. Y., Popescu, V., Asher, N.: Sentiment composition using a parabolic model. In: Proceedings of the 10th International Conference on Computational Semantics (IWCS 2013), pp. 47–58 (2013b)Google Scholar
  21. 21.
    Charolles, M.: L’encadrement du discours - Univers, champs, domaines et espace. Cahiers de recherche linguistique 6, 1–73 (1997)Google Scholar
  22. 22.
    Charolles, M., Le Draoulec, A., Péry-Woodley, M.-P., Sarda, L.: Temporal and spatial dimensions of discourse organisation. J. Fr. Lang. Stud. 15(2), 203–218 (2005)Google Scholar
  23. 23.
    Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20(1), 37–46 (1960)CrossRefGoogle Scholar
  24. 24.
    Colléter, M., Fabre, C., Ho-Dac, L.-M., Péry-Woodley, M.-P., Rebeyrolle, J., Tanguy, L.: La ressource ANNODIS multi-échelle : guide d’annotation et bonus. Technical report 20. Carnets de grammaires, CLLE-ERSS (2012)Google Scholar
  25. 25.
    Cornish, F.: Anaphora. Discourse and Understanding. Evidence from English and French. Clarendon Press, Oxford (1999)Google Scholar
  26. 26.
    Danlos, L.: Strong generative capacity of RST, SDRT and discourse dependency DAGSs. Pages 69–95 of: Benz, A., Kuhnlein, P. (eds.) Constraints in Discourse. John Benjamins, Amsterdam (2008)Google Scholar
  27. 27.
    Egg, M., Redeker, G.: How complex is discourse structure? In: Calzolari, N., Choucri, K., Maegaard, B., Mariani, J., Odijk, J., Piperidis, S., Rosner, M., Tapias, D. (eds.) Proceedings of LREC’10. ELRA (2010)Google Scholar
  28. 28.
    Enkvist, N.E.: Connexity, interpretability, universes of discourse, and text worlds. In: Allén, S. (ed.) Possible Worlds in Humanities, Arts and Sciences, pp. 162–186. Walter de Gruyter, Berlin (1989)Google Scholar
  29. 29.
    Feng, V.W., Hirst, G.: Text-level discourse parsing with rich linguistic features. IN: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, vol. 1: Long Papers), pp. 60–68. Association for Computational Linguistics, Jeju Island, Korea (2012)Google Scholar
  30. 30.
    Forbes, K., Miltsakaki, E., Prasad, R., Sarkar, A., Joshi, A.K., Webber, B.L.: D-LTAG system: discourse parsing with a lexicalized tree-adjoining grammar. J. Logic Lang. Inf. 12(3), 261–279 (2003)CrossRefGoogle Scholar
  31. 31.
    Francis, G.: Labelling discourse: an aspect of nominal-group lexical cohesion. In: Coulthard, M. (ed.) Advances in Written Text Analysis, pp. 83–101. Routledge, London (1994)Google Scholar
  32. 32.
    Fries, P.: Themes method of development and texts. In: Hasan, R., Fries, P. (eds.) On Subject and Theme: A Discourse Functional Perspective, pp. 317–359. John Benjamins, Amsterdam (1995)CrossRefGoogle Scholar
  33. 33.
    Goutsos, D.: A model of sequential relations in expository test. Text 16(4), 501–533 (1996)CrossRefGoogle Scholar
  34. 34.
    Grosz, B., Sidner, C.: Attention, intentions and the structure of discourse. Comput. Linguist. 12, 175–204 (1986)Google Scholar
  35. 35.
    Halliday, M.A.K.: Text as semantic choice in social contexts. In: van Dijk, T., Petöfi, J.S. (eds.) Grammars and Descriptions, pp. 176–226. Walter de Gruyter, Berlin (1977)Google Scholar
  36. 36.
    Halliday, M.A.K.: An Introduction to Functional Grammar, 2nd edn. Arnold, London (1985)Google Scholar
  37. 37.
    Halliday, M.A.K., Hasan, R.: Cohesion in English. Longman, London (1976)Google Scholar
  38. 38.
    Hempel, S., Degand, L.: sequencers in different text genres: academic writing, journalese and fiction. J. Pragmat. 40, 676–693 (2008)Google Scholar
  39. 39.
    Hernault, H., Prendinger, H., duVerle, D.A., Ishizuka, M.: HILDA: a discourse parser using support vector machine classification. Dialogue Discourse 1(3), 1–33 (2010)CrossRefGoogle Scholar
  40. 40.
    Hitzeman, J., Moens, M., Grover, C.: Algorithms for analyzing the temporal structure of discourse. In: Proceedings of the 7th Meeting of the European Chapter of the Association for Computational Linguistics, pp. 253–260 (1995)Google Scholar
  41. 41.
    Ho-Dac, L-M., Péry-Woodley, M-P.: A data-driven study of temporal adverbials as discourse segmentation markers. Discours 4 (2009)Google Scholar
  42. 42.
    Ho-Dac, L.-M., Péry-Woodley, M.-P, Tanguy, L.: Anatomie des structures énumératives. In: Actes de TALN, (ed.) 2010. Université de Montréal, for ATALA, Montréal (2010)Google Scholar
  43. 43.
    Ho-Dac, L.-M., Fabre, Cécile, Péry-Woodley, M.-P., Rebeyrolle, J., Tanguy, L.: On the signalling of multi-level discourse structures. Discours 10 (2012)Google Scholar
  44. 44.
    Hobbs, J.R.: Coherence and coreference. Cognit. Sci. 3(1), 67–90 (1979)CrossRefGoogle Scholar
  45. 45.
    Hovy, E.H.: Parsimonious and profligate approaches to the question of discourse structure relations. In: Proceedings of the Fifth International Workshop on Natural Language Generation, pp. 128–136 (1990)Google Scholar
  46. 46.
    Joty, S., Carenini, G., Ng, R.: A novel discriminative framework for sentence-level discourse analysis. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Association for Computational Linguistics, Jeju Island, Korea (2012)Google Scholar
  47. 47.
    Kamp, H., Reyle, U.: From Discourse to Logic: Introduction to Modeltheoretic Semantics of Natural Language. Formal Logic and Discourse Representation Theory. Kluwer Academic Publishers, The Netherlands (1993)Google Scholar
  48. 48.
    Landis, J.R., Koch, G.G.: The measurement of observer agreement for categorical data. Biometrics 33(1), 159–174 (1977)CrossRefGoogle Scholar
  49. 49.
    Lascarides, A., Asher, N.: Temporal interpretation, discourse relations and commonsense entailment. Linguist. Philos. 16(5), 437–493 (1993)CrossRefGoogle Scholar
  50. 50.
    Mann, W., Thompson, S.: Rhetorical structure theory: a theory of text organization. Technical report, Information Science Institute (1987)Google Scholar
  51. 51.
    Marcu, D.: Building up rhetorical structure trees. Proceedings of the thirteenth national conference on Artificial intelligence. AAAI’96, vol. 2, pp. 1069–1074. AAAI press, California (1996)Google Scholar
  52. 52.
    Mathet, Y., Widlöcher, A.: La plate-forme GLOZZ : environnement d’annotation et d’exploration de corpus. In: Actes de TALN, (ed.) 2009. LIPN, for ATALA, Senlis (2009)Google Scholar
  53. 53.
    Maudet, N., Muller, P., Prévot, L.: Social constraints on rhetorical relations in dialogue. In: Sidner, C., Harpur, J., Benz, A., Kühnlein, P. (eds.) Proceedings of the Workshop on Constraints in Discourse, pp. 133–139 (2006)Google Scholar
  54. 54.
    Muller, P., Prévot, L.: An empirical study of acknowledgment structures. In: Proceedings of Diabruck 2003, 7th Workshop on the Semantics and Pragmatics of Dialogue, (Sept 4th–6th) (2003)Google Scholar
  55. 55.
    Muller, P., Prévot, L.: The rhetorical attachment of questions and answers. In: Korta, K., Garmendia, J. (eds.) Meaning, Intentions, and Argumentation. (CSLI-LN) Center for the Study of Language and Information - Lecture Notes, vol. 186. University of Chicago press, Chicago (2008).
  56. 56.
    Muller, P., Afantenos, S., Denis, P., Asher, N.: Constrained decoding for text-level discourse parsing. In: Proceedings of COLING (2012a)Google Scholar
  57. 57.
    Muller, P., Vergez, M., Prévot, L, Asher, N, Benamara, F., Bras, M., Le Draoulec, A., Vieu, L.: Manuel d’annotation en relations de discours du projet Annodis. Technical report 21. CLLE (2012b)Google Scholar
  58. 58.
    Polanyi, L.: A formal model of the structure of discourse. J. Pragmat. 12, 601–638 (1988)CrossRefGoogle Scholar
  59. 59.
    Polanyi, L., Culy, C., van den Berg, M., Thione, G.L., Ahn, D.: A rule based approach to discourse parsing. In: Strube, M., Sidner, C. (eds.) Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue, pp. 108–117. Association for Computational Linguistics, Cambridge (2004)Google Scholar
  60. 60.
    Power, R., Scott, D., Bouayad-Agha, N.: Document structure. Comput. Linguist. 2(29), 211–260 (2003)CrossRefGoogle Scholar
  61. 61.
    Prasad, R., Dinesh, N., Lee, A., Miltsakaki, E., Robaldo, L., Joshi, A., Webber, B.: The penn discourse TreeBank 2.0. In: Calzolari, N., Choukri, K., Maegaard, B., Mariani, J., Odjik, J., Piperidis, S., Tapias, D. (eds.) Proceedings of the Sixth International Language Resources and Evaluation (LREC’08). European Language Resources Association (ELRA), Marrakech, Morocco (2008).
  62. 62.
    Prévot, L., Vieu, L., Asher, N.: Une formalisation plus précise pour une annotation moins confuse: la relation d’Élaboration d’entité. J. Fr. Lang. Stud. 19(2), 207–228 (2009)CrossRefGoogle Scholar
  63. 63.
    Roze, C.: Vers une algèbre des relations de discours. Ph.D. thesis, Université Paris 7 (2013)Google Scholar
  64. 64.
    Sagae, K.: Analysis of discourse structure with syntactic dependencies and data-driven shift-reduce parsing. Proceedings of the 11th International Conference on Parsing Technologies. IWPT ’09, pp. 81–84. Association for Computational Linguistics, Stroudsburg (2009)Google Scholar
  65. 65.
    Somasundaran, S.: Discourse-level relations for Opinion Analysis. Ph.D. thesis, University of Pittsburgh (2010)Google Scholar
  66. 66.
    Subba, R., Di Eugenio, B.: An effective discourse parser that uses rich linguistic information. In: Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 566–574. Association for Computational Linguistics, Boulder, Colorado (2009)Google Scholar
  67. 67.
    Trnavac, R., Taboada, M.: The contribution of nonveridical rhetorical relations to evaluation in discourse. Lang. Sci. 34(3), 301–318 (2010)CrossRefGoogle Scholar
  68. 68.
    Turney, P.D.: Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews. In: Proceedings of Annual Meeting of the Association for Computational Linguistics (2002)Google Scholar
  69. 69.
    Venant, A., Asher, N., Muller, P., Denis, P., Afantenos, S.: Expressivity and comparison of models of discourse structure. In: Proceedings of the SIGDIAL 2013 Conference, pp. 2–11. Association for Computational Linguistics (2013a)Google Scholar
  70. 70.
    Vergez-Couret, M.: Etude en corpus des réalisations linguistiques de la relation d’Elaboration. Ph.D. thesis, Université de Toulouse, Toulouse (2010)Google Scholar
  71. 71.
    Webber, B., Egg, M., Kordoni, V.: Discourse structure and language technology. Nat. Lang. Eng. 18(4), 437–490 (2012)CrossRefGoogle Scholar
  72. 72.
    Wiebe, J., Riloff, E.: Creating subjective and objective sentence classifiers from unannotated texts. In: Proceedings of the International Conference on Intelligent Text Processing and Computational Linguistics (CICLing). Lecture Notes in Computer Science, vol. 3406, pp. 486–497 (2005)Google Scholar
  73. 73.
    Wolf, F., Gibson, E.: Representing discourse coherence: a corpus based study. Comput. Linguist. 31(2), 249–287 (2005)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2017

Authors and Affiliations

  1. 1.Toulouse UniversityToulouseFrance

Personalised recommendations