A Two-Phase Sampling Technique to Improve the Accuracy of Text Similarities in the Categorisation of Hidden Web Databases

  • Yih-Ling Hedley
  • Muhammad Younas
  • Anne James
  • Mark Sanderson
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3306)

Abstract

The larger amount of high quality and specialised information on the Web is stored in document databases, which is not indexed by general-purpose search engines such as Google and Yahoo. Such information is dynamically generated as a result of submitting queries to databases – which are referred to as Hidden Web databases. This paper presents a Two-Phase Sampling (2PS) technique that detects Web page templates from the randomly sampled documents of a database. It generates terms and frequencies that summarise the database content with improved accuracy. We then utilise such statistics to improve the accuracy of text similarity computation in categorisation. Experimental results show that 2PS effectively eliminates terms contained in Web page templates, and generates terms and frequencies with improved accuracy. We also demonstrate that 2PS improves the accuracy of text similarity computation required in the process of database categorisation.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Yih-Ling Hedley
    • 1
  • Muhammad Younas
    • 1
  • Anne James
    • 1
  • Mark Sanderson
    • 2
  1. 1.School of Mathematical and Information SciencesCoventry UniversityCoventryUK
  2. 2.Department of Information StudiesUniversity of SheffieldSheffieldUK

Personalised recommendations