Abstract
This paper proposes an enhanced method of Web information extraction by exploiting general phenomena that Web pages in a site tend to have common structures and dynamic Web pages contain multiple data blocks with repeating structural patterns. By considering this kind of regularity in dynamic Web pages, we develop a data block extraction system which basically adopts a supervised learning mechanism with training and extraction phases. In the training phase, the user selects and specifies a data block and the extraction rules for the block are generated. During this phase, the block is defined with the HTML DOM-tree path to the block and the tag sequence of the block. In the extraction phase, the rules are applied to the target pages to extract those blocks that have similar structure as the user-defined block. A series of experiments are performed to evaluate the user-defined data block extraction method for a number of well-known Web sites with dynamic Web pages, and the result of evaluation is satisfactory with high precision and recall measures.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Chang, C., Ko, S.: OLERA: Semisupervised Web-Data Exraction with Visual Support. IEEE Intelligent Systems 19(6), 56–64 (2004)
Crescenzi, V., Mecca, G., Merialdo, P.: RoadRunner: Towards Automatic Data Extraction from Large Web Sites, VLDB 2001, pp. 109–118 (2001)
Gusfield, D.: Algorithms on Strings, Trees, and Sequences, pp. 215–234. Cambridge University Press, Cambridge (1997)
Liu, B., Grossman, R., Zhai, Y.: Mining Web Pages for Data Records. IEEE Intellgent Systems, 49–55 (2004)
Liu, C., Pu, H.W.: An XML-enabled Wrapper Construction System for Web Information Sources. In: Proc. of the 1th Intl. Conf. on Data Engineering, pp. 611–621 (2000)
Sahuguet, A., Azavant, F.: Building Intelligent Web Applications Using Lightweight Wrappers. Data and Knowledge Engineering 36(3), 283–316 (2001)
Shi, Z., Milio, E., Zincir-Heywood, N.: Post-supervised Template Induction for Information Extraction from Lists and Tables in Dynamic Web Sources. Kluwer Academic Publishers, Dordrecht (2003)
Document Object Model (DOM), http://www.w3.org/DOM/
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Choi, C., Kang, J., Choi, J. (2007). Extraction of User-Defined Data Blocks Using the Regularity of Dynamic Web Pages. In: Huang, DS., Heutte, L., Loog, M. (eds) Advanced Intelligent Computing Theories and Applications. With Aspects of Theoretical and Methodological Issues. ICIC 2007. Lecture Notes in Computer Science, vol 4681. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74171-8_13
Download citation
DOI: https://doi.org/10.1007/978-3-540-74171-8_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74170-1
Online ISBN: 978-3-540-74171-8
eBook Packages: Computer ScienceComputer Science (R0)