LRD: Latent Relation Discovery for Vector Space Expansion and Information Retrieval
In this paper, we propose a text mining method called LRD (latent relation discovery), which extends the traditional vector space model of document representation in order to improve information retrieval (IR) on documents and document clustering. Our LRD method extracts terms and entities, such as person, organization, or project names, and discovers relationships between them by taking into account their co-occurrence in textual corpora. Given a target entity, LRD discovers other entities closely related to the target effectively and efficiently. With respect to such relatedness, a measure of relation strength between entities is defined. LRD uses relation strength to enhance the vector space model, and uses the enhanced vector space model for query based IR on documents and clustering documents in order to discover complex relationships among terms and entities. Our experiments on a standard dataset for query based IR shows that our LRD method performed significantly better than traditional vector space model and other five standard statistical methods for vector expansion.
KeywordsMutual Information Information Retrieval Relation Strength Vector Space Model Textual Corpus
Unable to display preview. Download preview PDF.
- 1.Castillo, G., Sierra, G., McNaught, J.: An improved algorithm for semantic clustering. In: Proc. of 1st International Symposium on Information and Communication Technologies, Dublin, Ireland. ACM International Conference Proceeding Series, pp. 304–309 (2003)Google Scholar
- 4.Ding, C.H.Q.: A probabilistic model for dimensionality reduction in information retrieval and filtering. In: Proc. of the 1st SIAM Computational Information Retrieval Workshop, Raleigh, NC (2000)Google Scholar
- 6.Hotho, A., Stumme, G.: Conceptual clustering of text clusters. In: Proc. of the Fachgruppentreffen Maschinelles Lernen (FGML), Hannover, Germany, pp. 37–45 (2002)Google Scholar
- 7.Ikehara, S., Murakami, J., Kimoto, Y., Araki, T.: Vector space model based on semantic attributes of words. In: Proc. of the Pacific Association for Computational Linguistics (PACLING), Kitakyushu, Japan (2001)Google Scholar
- 9.Gonçalves, A., Uren, V., Kern, V., Pacheco, R.: Mining Knowledge from Textual Databases: An Approach using Ontology-based Context Vectors. In: Proc. of the International Conference on Artificial Intelligence and Applications (AIA 2005), Innsbruck, Austria, pp. 66–71 (2005)Google Scholar
- 12.Church, K., Hanks, P.: Word association norms, mutual information, and lexicography. Computational Linguistics 16(1), 22–29 (1990)Google Scholar