Little Knowledge Rules the Web: Domain-Centric Result Page Extraction

  • Tim Furche
  • Georg Gottlob
  • Giovanni Grasso
  • Giorgio Orsi
  • Christian Schallhart
  • Cheng Wang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6902)

Abstract

Web extraction is the task of turning unstructured HTML into structured data. Previous approaches rely exclusively on detecting repeated structures in result pages. These approaches trade intensive user interaction for precision.

In this paper, we introduce the Amber (“Adaptable Model-based Extraction of Result Pages”) system that replaces the human interaction with a domain ontology applicable to all sites of a domain. It models domain knowledge about (1) records and attributes of the domain, (2) low-level (textual) representations of these concepts, and (3) constraints linking representations to records and attributes. Parametrized with these constraints, otherwise domain-independent heuristics exploit the repeated structure of result pages to derive attributes and records. Amber is implemented in logical rules to allow an explicit formulation of the heuristics and easy adaptation to different domains.

We apply Amber to the UK real estate domain where we achieve near perfect accuracy on a representative sample of 50 agency websites.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Baumgartner, R., Flesca, S., Gottlob, G.: Visual Web Information Extraction with Lixto. In: VLDB (2001)Google Scholar
  2. 2.
    Chang, C.-H., Kayed, M., Girgis, M.R., Shaalan, K.F.: A survey of web information extraction systems. TKDE 18(10) (2006)Google Scholar
  3. 3.
    Hsu, C., Dung, M.: Generating finite-state transducers for semistructured data extraction from the web. IS 23(8) (1998)Google Scholar
  4. 4.
    Kayed, M., Chang, C.-H.: FiVaTech: Page-Level Web Data Extraction from Template Pages. TKDE 22(2) (2010)Google Scholar
  5. 5.
    Kushmerick, N., Weld, D.S., Doorenbos, R.: Wrapper Induction for Information Extraction. In: VLDB (1997)Google Scholar
  6. 6.
    Laender, A.H.F., Ribeiro-Neto, B.A., da Silva, A.S., Teixeira, J.S.: A brief survey of web data extraction tools. SIGMOD Rec. 31(2) (2002)Google Scholar
  7. 7.
    Liu, W., Meng, X., Meng, W.: Vision-based Web Data Records Extraction. In: WebDB (2006)Google Scholar
  8. 8.
    Senellart, P., Mittal, A., Muschick, D., Gilleron, R., Tommasi, M.: Automatic wrapper induction from hidden-web sources with domain knowledge. In: WIDM (2008)Google Scholar
  9. 9.
    Simon, K., Lausen, G.: ViPER: Augmenting Automatic Information Extraction with visual Perceptions. In: CIKM (2005)Google Scholar
  10. 10.
    Su, W., Wang, J., Lochovsky, F.H.: ODE: Ontology-Assisted Data Extraction. TODS, vol. 34(2) (2009)Google Scholar
  11. 11.
    Wang, J., Chen, C., Wang, C., Pei, J., Bu, J., Guan, Z., Zhang, W.V.: Can we learn a template-independent wrapper for news article extraction from a single training site?. In: KDD (2009)Google Scholar
  12. 12.
    Wang, J., Lochovsky, F.H.: Data extraction and label assignment for Web databases. In: WWW (2003)Google Scholar
  13. 13.
    Zhai, Y., Liu, B.: Structured Data Extraction from the Web Based on Partial Tree Alignment. TKDE 18(12) (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Tim Furche
    • 1
  • Georg Gottlob
    • 1
  • Giovanni Grasso
    • 1
  • Giorgio Orsi
    • 1
  • Christian Schallhart
    • 1
  • Cheng Wang
    • 1
  1. 1.Department of Computer ScienceUniversity of OxfordUK

Personalised recommendations