Advertisement

Modeling and Evaluating Summaries Using Complex Networks

  • Thiago Alexandre Salgueiro Pardo
  • Lucas Antiqueira
  • Maria das Graças Volpe Nunes
  • Osvaldo N. OliveiraJr.
  • Luciano da Fontoura Costa
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3960)

Abstract

This paper presents a summary evaluation method based on a complex network measure. We show how to model summaries as complex networks and establish a possible correlation between summary quality and the measure known as dynamics of the network growth. It is a generic and language independent method that enables easy and fast comparative evaluation of summaries. We evaluate our approach using manually produced summaries and automatic summaries produced by three automatic text summarizers for the Brazilian Portuguese language. The results are in agreement with human intuition and showed to be statistically significant.

Keywords

Complex Network Natural Language Processing Word Association Source Text Text Quality 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Antiqueira, L., Nunes, M.G.V., Oliveira Jr., O.N., Costa, L.F.: Modelando Textos como Redes Complexas. In: Anais do III (ed.) Workshop em Tecnologia da Informação e da Linguagem Humana – TIL. São Leopoldo-RS, Brazil, July 22-26 (2005a)Google Scholar
  2. Antiqueira, L., Nunes, M.G.V., Oliveira Jr., O.N., Costa, L.F.: Complex networks in the assessment of text quality. physics/0504033 (2005b)Google Scholar
  3. Barabási, A.L.: Linked: How Everything Is Connected to Everything Else and What It Means for Business, Science, and Everyday Life. Plume, New York (2003)Google Scholar
  4. Cancho, R.F., Capocci, A., Caldarelli, G.: Spectral methods cluster words of the same class in a syntactic dependency network. cond-mat/0504165 (2005)Google Scholar
  5. Casella, J., Berger, R.L.: Statistical Inference, Duxbury, Belmont, California (2001)Google Scholar
  6. Costa, L.F.: Whats in a name? International Journal of Modern Physics C 15, 371–379 (2004)CrossRefGoogle Scholar
  7. Dorogovtsev, S.N., Mendes, J.F.F.: Evolution of networks. Advances in Physics 51(4), 1079–1187 (2002)CrossRefGoogle Scholar
  8. Erkan, G., Radev, D.R.: Lexrank: Graph-based centrality as salience in text summarization. Journal of Artificial Intelligence Research – JAIR 22, 457–479 (2004)Google Scholar
  9. Kupiec, J., Pedersen, J., Chen, F.: A trainable document summarizer. In: The Proceedings of the 18th ACM-SIGIR Conference on Research & Development in Information Retrieval, pp. 68–73 (1995)Google Scholar
  10. Lin, C.-Y., Hovy, E.H.: Automatic Evaluation of Summaries Using N-gram Cooccurrence Statistics. In: The Proceedings of Language Technology Conference – HLT, Edmonton, Canada, May 27 - June 1 (2003)Google Scholar
  11. Luhn, H.: The automatic creation of literature abstracts. IBM Journal of Research and Development 2, 159–165 (1958)CrossRefGoogle Scholar
  12. Mani, I.: Automatic Summarization. John Benjamin’s Publishing Company (2001)Google Scholar
  13. Mihalcea, R.: Language Independent Extractive Summarization. In: The Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, Ann Arbor, Michigan (2005)Google Scholar
  14. Miller, G.A.: Wordnet: a dictionary browser. In: The Proceedings of the First International Conference on Information in Data, University of Waterloo, Waterloo (1985)Google Scholar
  15. Módolo, M.: SuPor: um Ambiente para a Exploração de Métodos Extrativos para a Sumarização Automática de Textos em Português. Master thesis. Departamento de Computação, UFSCar (2003)Google Scholar
  16. Motter, A.E., Moura, A.P.S., Lai, Y.C., Dasgupta, P.: Topology of the conceptual network of language. Physical Review E 65, 65102 (2002)CrossRefGoogle Scholar
  17. Newman, M.E.J.: The structure and function of complex networks. SIAM Review 45, 167–256 (2003)CrossRefGoogle Scholar
  18. Pardo, T.A.S., Rino, L.H.M.: TeMário: Um Corpus para Sumarização Automática de Textos. NILC technical report. NILC-TR-03-09. São Carlos-SP, October, 13p (2003)Google Scholar
  19. Pardo, T.A.S., Rino, L.H.M.: Descrição do GEI - Gerador de Extratos Ideais para o Português do Brasil. NILC technical report. NILC-TR-04-07. São Carlos-SP, August, 10p (2004)Google Scholar
  20. Pardo, T.A.S., Rino, L.H.M., Nunes, M.G.V.: GistSumm: A Summarization Tool Based on a New Extractive Method. In: Mamede, N.J., Baptista, J., Trancoso, I., Nunes, M.d.G.V. (eds.) PROPOR 2003. LNCS, vol. 2721, pp. 210–218. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  21. Rino, L.H.M., Pardo, T.A.S., Silla Jr., C.N., Kaestner, C.A., Pombo, M.: A Comparison of Automatic Summarization Systems for Brazilian Portuguese Texts. In: Bazzan, A.L.C., Labidi, S. (eds.) SBIA 2004. LNCS (LNAI), vol. 3171, pp. 235–244. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  22. Santos Jr., E., Mohamed, A.A., Zhao, Q.: Automatic Evaluation of Summaries Using Document Graphs. In: The Proceedings of the Workshop on Text Summarization Branches Out, pp. 66–73 (2004)Google Scholar
  23. Salton, G., Buckley, C.: Term-weighting approaches in automatic text retrieval. Information Processing and Management 24, 513–523 (1988)CrossRefGoogle Scholar
  24. Sigman, M., Cecchi, G.A.: Global Organization of the Wordnet Lexicon. In: The Proceedings of the National Academy of Sciences, vol. 99, pp. 1742–1747 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Thiago Alexandre Salgueiro Pardo
    • 1
  • Lucas Antiqueira
    • 1
  • Maria das Graças Volpe Nunes
    • 1
  • Osvaldo N. OliveiraJr.
    • 1
    • 2
  • Luciano da Fontoura Costa
    • 2
  1. 1.Núcleo Interinstitucional de Lingüística Computacional (NILC), CP 668 – ICMC-USPSão CarlosBrasil
  2. 2.Instituto de Física de São Carlos, CP 369 – IFSC-USPSão CarlosBrasil

Personalised recommendations