Abstract
The larger amount of high quality and specialised information on the Web is stored in document databases, which is not indexed by general-purpose search engines such as Google and Yahoo. Such information is dynamically generated as a result of submitting queries to databases – which are referred to as Hidden Web databases. This paper presents a Two-Phase Sampling (2PS) technique that detects Web page templates from the randomly sampled documents of a database. It generates terms and frequencies that summarise the database content with improved accuracy. We then utilise such statistics to improve the accuracy of text similarity computation in categorisation. Experimental results show that 2PS effectively eliminates terms contained in Web page templates, and generates terms and frequencies with improved accuracy. We also demonstrate that 2PS improves the accuracy of text similarity computation required in the process of database categorisation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bergman, M.K.: The Deep Web: Surfacing Hidden Value. Appeared in The Journal of Electronic Publishing from the University of Michigan (2001), http://www.press.umich.edu/jep/07-01/bergman.html (retrieved August 10, 2004)
Callan, J., Connell, M.: Query-Based Sampling of Text Databases. ACM Transactions on Information Systems 19(2), 97–130 (2001)
Fravano, L., Change, K., Garcia-Molina, H., Paepcke, A.: STARTS Stanford Proposal for Internet Meta-Searching. In: Proceedings of the ACM-SIGMOD International Conference on Management of Data (1997)
Gravano, L., Ipeirotis, P.G., Sahami, M.: QProber: A System for Automatic Classification of Hidden-Web Databases. ACM Transactions on Information Systems (TOIS) 21(1) (2003)
Heß, M., Drobnik, O.: Clustering Specialised Web-databases by Exploiting Hyperlinks. In: Proceedings of the Second Asian Digital Library Conference (1999)
Hedley, Y.L., Younas, M., James, A., Sanderson, M.: Query-Related Data Extraction of Hidden Web Documents. In: Proceedings of SIGIR (2004)
Lin, K.I., Chen, H.: Automatic Information Discovery from the Invisible Web. In: International Conference on Information Technology: Coding and Computing (2002)
Meng, W., Wang, W., Sun, H., Yu, C.: Concept Hierarchy Based Text Database Categorization. International Journal on Knowledge and Information Systems 4(2), 132–150 (2002)
Salton, G., McGill, M.: Introduction to Modern Information Retrieval. McCraw-Hill, New York (1983)
Sugiura, A., Etzioni, O.: Query Routing for Web Search Engines: Architecture and Experiment. In: 9th WWW Conference (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hedley, YL., Younas, M., James, A., Sanderson, M. (2004). A Two-Phase Sampling Technique to Improve the Accuracy of Text Similarities in the Categorisation of Hidden Web Databases. In: Zhou, X., Su, S., Papazoglou, M.P., Orlowska, M.E., Jeffery, K. (eds) Web Information Systems – WISE 2004. WISE 2004. Lecture Notes in Computer Science, vol 3306. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30480-7_54
Download citation
DOI: https://doi.org/10.1007/978-3-540-30480-7_54
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23894-2
Online ISBN: 978-3-540-30480-7
eBook Packages: Springer Book Archive