A Method for Pinpoint Clustering of Web Pages with Pseudo-Clique Search
This paper presents a method for Pinpoint Clustering of web pages. We try to find useful clusters of web pages which are significant in the sense that their contents are similar to ones of higher-ranked pages. Since we are usually careless of lower-ranked pages, they are unconditionally discarded even if their contents are similar to some pages with high ranks. Such hidden pages together with significant higher-ranked pages are extracted as a cluster. As the result, our clusters can provide new valuable information for users.
In order to obtain such clusters, we first extract semantic correlations among terms by applying Singular Value Decomposition (SVD) to the term-document matrix generated from a corpus. Based on the correlations, we can evaluate potential similarities among web pages to be clustered. The set of web pages is represented as a weighted graph G based on the similarities and their ranks. Our clusters can be found as pseudo-cliques in G. An algorithm for finding Top-N weighted pseudo-cliques is presented. Our experimental result shows that a quite valuable cluster can be actually extracted according to our method.
We also discuss an idea for improvement on meanings of clusters. With the help of Formal Concept Analysis, our clusters, called FC-based clusters, can be provided with clear meanings. Our preliminary experimentation shows that the extended method would be a promising approach to finding meaningful clusters.
KeywordsSingular Value Decomposition Semantic Similarity Formal Concept Maximal Clique Information Retrieval System
Unable to display preview. Download preview PDF.
- 1.Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank Citation Ranking: Bringing Order to the Web (1999), http://dbpubs.stanford.edu/pub/1999-66
- 3.Strang, G.: Introduction to Linear Algebra, 3rd edn. Wellesley-Cambridge Press (2003)Google Scholar
- 4.Kita, K., Tsuda, K., Shishibori, M.: Information Retrieval Algorithms. Kyoritsu Shuppan (2002) (in Japanese)Google Scholar
- 9.Satoh, K.: A Method for Generating Data Abstraction Based on Optimal Clique Search. Master’s Thesis, Graduate School of Eng., Hokkaido Univ. (March 2003) (in Japanese)Google Scholar
- 10.Masuda, S.: Analysis of Ascidian Gene Expression Data by Clique Search. Master’s Thesis, Graduate School of Eng., Hokkaido Univ. (March 2005) (in Japanese)Google Scholar
- 11.Shi, B.: Top-N Clique Search of Web Pages. Master’s Thesis, Graduate School of Eng., Hokkaido Univ. (March 2005) (in Japanese)Google Scholar
- 14.Okubo, Y., Haraguchi, M.: Finding Top-N Pseudo-Cliques in Simple Graph. In: Proceedings of the 9th World Multiconference on Systemics, Cybernetics and Informatics - WMSCI 2005, vol. III, pp. 215–220 (2005)Google Scholar