Text Segmentation Using Context Overlap

  • Radim Řehůřek
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4874)

Abstract

In this paper we propose features desirable of linear text segmentation algorithms for the Information Retrieval domain, with emphasis on improving high similarity search of heterogeneous texts. We proceed to describe a robust purely statistical method, based on context overlap exploitation, that exhibits these desired features. Experimental results are presented, along with comparison to other existing algorithms.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Choi, F.Y.Y.: Advances in domain independent linear text segmentation. In: Proc. of the North American Chapter of the Association of Computational Linguistic, Seattle, pp. 26–33 (2000)Google Scholar
  2. 2.
    Choi, F.Y.Y., Wiemer-Hastings, P., Moore, J.: Latent semantic analysis for text segmentation. In: Proc. of the 6th Conference on Empirical Methods in Natural Language Processing, pp. 109–117 (2001)Google Scholar
  3. 3.
    Foltz, P.W.: Latent Semantic Analysis for text-based research. Behavior Research Methods, Instruments and Computers 28, 197–202 (1996)Google Scholar
  4. 4.
    Hearst, M.A.: Multi-paragraph segmentation of expository text. In: Proc. of the ACL 1994, Las Crces, NM (1994)Google Scholar
  5. 5.
    Min-Yen, K., Klavans, J.L., McKeown, K.R.: Linear segmentation and segment significance. In: Proc. of WVLC-6, Montreal, pp. 197–205 (1998)Google Scholar
  6. 6.
    Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401(675), 788–791 (1999)CrossRefGoogle Scholar
  7. 7.
    Pevzner, L., Hearst, M.: A critique and improvement of an evaluation metric for text segmentation. Computational Linguistics 28(1), 19–36 (2002)CrossRefGoogle Scholar
  8. 8.
  9. 9.
    Reynar, J.C.: Topic segmentation: Algorithms and applications. Ph.D. thesis, Computer and Information Science, University of Pennsylvania (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Radim Řehůřek
    • 1
  1. 1.Faculty of Informatics, Masaryk University 

Personalised recommendations