Preloading Browsers for Optimizing Automatic Access to Hidden Web: A Ranking-Based Repository Solution

  • Justo Hidalgo
  • Alberto Pan
  • José Losada
  • Manuel Álvarez
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4152)


As Web applications grow in terms of quantity and quality, different vertical solutions could make use of them as an important source of information. Nevertheless, obtaining information from web sources becomes a challenging issue because of their complex access due to the hypertext browsing paradigm, and HTML’s semistructured format. Web Automation middleware navigates through web links and fills web forms in an automatic way, so to extract information from the Hidden Web. The main optimization parameter is the time required to navigate through the intermediate pages that lead to the desired results. This work proposes a technique which focuses on improving the browsing time by storing information from previous queries, and using it to preload an adequate subset of the navigational sequence on a specific browser, before the next sequence is launched. It also takes into account the most commonly used sequences, being the ones to be preloaded more often.


Time Section Result Page Query Type Query Form Sample Query 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arasu, A., Garcia-Molina, H.: Extracting Structured Data from Web Pages. In: Proceedings of the ACM SIGMOD international conference on Management of data (2003)Google Scholar
  2. 2.
    Bergman, M.K.: The Deep Web. Surfacing Hidden Value,
  3. 3.
    Garret, J.J.: Ajax: A New Approach to Web Applications,
  4. 4.
    Hidalgo, J., Pan, A., Losada, J., Álvarez, M.: Adding Physical Optimization to Cost Models in Information Mediators. In: 2005 IEEE Conference on e-Business Engineering (2005)Google Scholar
  5. 5.
    Hidalgo, J., Pan, A., Losada, J., Álvarez, M., Viña, A.: Building the Architecture of a Statistics-based Query Optimization Solution for Heterogeneous Mediators. In: 6th International Conference on Information Integration and Web-based Applications & Services (2004)Google Scholar
  6. 6.
    Knoblock, C.A., Lerman, K., Minton, S., Muslea, I.: Accurately and Reliably Extracting Data from the Web: A Machine Learning Approach. Bulletin of the IEEE Computer Society Technical Committee on Data Enginnering (1999)Google Scholar
  7. 7.
    Kushmerick, N., Weld, D.S., Doorembos, R.: Wrapper induction for information extraction. In: Proceedings of the fifteenth International Joint Conference on Artificial Intelligence (1997)Google Scholar
  8. 8.
    Laender, A.H.F., Ribeiro-Neto, B.A., Soares da Silva, A., Teixeira, J.S.: A Brief Survey of Web Data Extraction Tools. ACM SIGMOD Record 31(2) (2002)Google Scholar
  9. 9.
    Pan, A., et al.: Semi-Automatic Wrapper Generation for Commercial Web Sources. In: Proceedings of IFIP WG8.1 Working Conference on Engineering Information Systems in the Internet Context (2002)Google Scholar
  10. 10.
    Pan, A., Raposo, J., Álvarez, M., Montoto, P., Orjales, V., Hidalgo, J., Ardao, L., Molano, A., Viña, A.: The DENODO Data Integration Platform. In: 28th International Conference on Very Large Databases (2002)Google Scholar
  11. 11.
    Raghavan, S., García-Molina, H.: Crawling the Hidden Web. In: Proceedings of the 27th International Conference on Very Large Databases (2001)Google Scholar
  12. 12.
    Raposo, J., Pan, A., Álvarez, M., Hidalgo, J.: Automatically Generating Labeled Examples for Web Wrapper Maintenance. In: Proceedings of the 2005 IEEE/WIC/ACM International Conference on Web Intelligence (2005)Google Scholar
  13. 13.
    Raposo, J., Pan, A., Álvarez, M., Viña, A.: Automatic Wrapper Maintenance for Semi-Structured Web Sources Using Results from Previous Queries. In: Proceedings of the 2005 ACM Symposium on Applied Computing (2005)Google Scholar
  14. 14.
    Wiederhold, G.: Mediators in the Architecture of Future Information Systems. IEEE Computer Society Press, Los Alamitos (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Justo Hidalgo
    • 1
  • Alberto Pan
    • 2
  • José Losada
    • 1
  • Manuel Álvarez
    • 2
  1. 1.Denodo Technologies, Inc.MadridSpain
  2. 2.Department of Information and Communications TechnologiesUniversity of A CoruñaSpain

Personalised recommendations