Chapter

Knowledge Discovery in Databases: PKDD 2007

Volume 4702 of the series Lecture Notes in Computer Science pp 152-163

Site-Independent Template-Block Detection

  • Aleksander KołczAffiliated withMicrosoft Live Labs, Redmond WA
  • , Wen-tau YihAffiliated withMicrosoft Research, Redmond WA

* Final gross prices may vary according to local VAT.

Get Access

Abstract

Detection of template and noise blocks in web pages is an important step in improving the performance of information retrieval and content extraction. Of the many approaches proposed, most rely on the assumption of operating within the confines of a single website or require expensive hand-labeling of relevant and non-relevant blocks for model induction. This reduces their applicability, since in many practical scenarios template blocks need to be detected in arbitrary web pages, with no prior knowledge of the site structure. In this work we propose to bridge these two approaches by using within-site template discovery techniques to drive the induction of a site-independent template detector. Our approach eliminates the need for human annotation and produces highly effective models. Experimental results demonstrate the usefulness of the proposed methodology for the important applications of keyword extraction, with relative performance gain as high as 20%.