Chapter

Knowledge Discovery in Databases: PKDD 2006

Volume 4213 of the series Lecture Notes in Computer Science pp 30-41

Clustering Scientific Literature Using Sparse Citation Graph Analysis

  • Levent BolelliAffiliated withCarnegie Mellon UniversityDepartment of Computer Science and Engineering, The Pennsylvania State University
  • , Seyda ErtekinAffiliated withCarnegie Mellon UniversityDepartment of Computer Science and Engineering, The Pennsylvania State University
  • , C. Lee GilesAffiliated withCarnegie Mellon UniversityDepartment of Computer Science and Engineering, The Pennsylvania State UniversityCollege of Information Sciences and Technology, The Pennsylvania State University

* Final gross prices may vary according to local VAT.

Get Access

Abstract

It is well known that connectivity analysis of linked documents provides significant information about the structure of the document space for unsupervised learning tasks. However, the ability to identify distinct clusters of documents based on link graph analysis is proportional to the density of the graph and depends on the availability of the linking and/or linked documents in the collection. In this paper, we present an information theoretic approach towards measuring the significance of individual words based on the underlying link structure of the document collection. This enables us to generate a non-uniform weight distribution of the feature space which is used to augment the original corpus-based document similarities. The experimental results on the collection of scientific literature show that our method achieves better separation of distinct groups of documents, yielding improved clustering solutions.