Advertisement

Information Extraction from Webpages Based on DOM Distances

  • Carlos Castillo
  • Héctor Valero
  • José Guadalupe Ramos
  • Josep Silva
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7182)

Abstract

Retrieving information from Internet is a difficult task as it is demonstrated by the lack of real-time tools able to extract information from webpages. The main cause is that most webpages in Internet are implemented using plain (X)HTML which is a language that lacks structured semantic information. For this reason much of the efforts in this area have been directed to the development of techniques for URLs extraction. This field has produced good results implemented by modern search engines. But, contrarily, extracting information from a single webpage has produced poor results or very limited tools. In this work we define a novel technique for information extraction from single webpages or collections of interconnected webpages. This technique is based on DOM distances to retrieve information. This allows the technique to work with any webpage and, thus, to retrieve information online. Our implementation and experiments demonstrate the usefulness of the technique.

Keywords

Resource Description Framework Information Extraction Relevant Node Domain Distance Document Object Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Dalvi, B., Cohen, W.W., Callan, J.: Websets: Extracting sets of entities from the web using unsupervised information extraction. Technical report, Carnegie Mellon School of computer Science (2011)Google Scholar
  2. 2.
    Kushmerick, N., Weld, D.S., Doorenbos, R.: Wrapper induction for information extraction. In: Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence (IJCAI 1997) (1997)Google Scholar
  3. 3.
    Cohen, W.W., Hurst, M., Jensen, L.S.: A flexible learning system for wrapping tables and lists in html documents. In: Proceedings of the international World Wide Web conference (WWW 2002), pp. 232–241 (2002)Google Scholar
  4. 4.
    Lee, P.Y., Hui, S.C., Fong, A.C.M.: Neural networks for web content filtering. IEEE Intelligent Systems 17(5), 48–57 (2002)CrossRefGoogle Scholar
  5. 5.
    Anti-Porn Parental Controls Software. Porn Filtering (March 2010), http://www.tueagles.com/anti-porn/
  6. 6.
    Kang, B.-Y., Kim, H.-G.: Web page filtering for domain ontology with the context of concept. IEICE - Trans. Inf. Syst. E90, D859–D862 (2007)CrossRefGoogle Scholar
  7. 7.
    Henzinger, M.: The Past, Present and Future of Web Information Retrieval. In: Proceedings of the 23th ACM Symposium on Principles of Database Systems (2004)Google Scholar
  8. 8.
    W3C Consortium. Resource Description Framework (RDF), www.w3.org/RDF
  9. 9.
    W3C Consortium. Web Ontology Language (OWL), www.w3.org/2004/OWL
  10. 10.
    Microformats.org. The Official Microformats Site (2009), http://microformats.org
  11. 11.
    Khare, R., Çelik, T.: Microformats: a Pragmatic Path to the Semantic Web. In: Proceedings of the 15h International Conference on World Wide Web, pp. 865–866 (2006)Google Scholar
  12. 12.
    Khare, R.: Microformats: The Next (Small) Thing on the Semantic Web? IEEE Internet Computing 10(1), 68–75 (2006)CrossRefGoogle Scholar
  13. 13.
    Gupta, S., et al.: Automating Content Extraction of HTML Documents. World Wide Archive 8(2), 179–224 (2005)CrossRefGoogle Scholar
  14. 14.
    Li, P., Liu, M., Lin, Y., Lai, Y.: Accelerating Web Content Filtering by the Early Decision Algorithm. IEICE Transactions on Information and Systems E91-D, 251–257 (2008)CrossRefGoogle Scholar
  15. 15.
    W3C Consortium, Document Object Model (DOM), www.w3.org/DOM
  16. 16.
    Baeza-Yates, R., Castillo, C.: Crawling the Infinite Web: Five Levels Are Enough. In: Leonardi, S. (ed.) WAW 2004. LNCS, vol. 3243, pp. 156–167. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  17. 17.
    Micarelli, A., Gasparetti, F.: Adaptative Focused Crawling. In: The Adaptative Web, pp. 231–262 (2007)Google Scholar
  18. 18.
    Nielsen, J.: Designing Web Usability: The Practice of Simplicity. New Riders Publishing, Indianapolis (2010) ISBN 1-56205-810-XGoogle Scholar
  19. 19.
    Zhang, J.: Visualization for Information Retrieval. The Information Retrieval Series. Springer, Heidelberg (2007) ISBN 3-54075-1475Google Scholar
  20. 20.
    Hearst, M.A.: TileBars: Visualization of Term Distribution Information. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, Denver, CO, pp. 59–66 (May 1995)Google Scholar
  21. 21.
    Gottron, T.: Evaluating Content Extraction on HTML Documents. In: Proceedings of the 2nd International Conference on Internet Technologies and Applications, pp. 123–132 (2007)Google Scholar
  22. 22.
    Apache Foundation. The Apache crawler Nutch (2010), http://nutch.apache.org

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Carlos Castillo
    • 1
  • Héctor Valero
    • 1
  • José Guadalupe Ramos
    • 2
  • Josep Silva
    • 1
  1. 1.Universidad Politécnica de ValenciaValenciaSpain
  2. 2.Instituto Tecnológico de La PiedadLa PiedadMéxico

Personalised recommendations