Optimization of Automatic Navigation to Hidden Web Pages by Ranking-Based Browser Preloading

  • Justo Hidalgo
  • José Losada
  • Manuel Álvarez
  • Alberto Pan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4055)


Web applications have become an invaluable source of information for many different vertical solutions, but their complex navigation and semistructured format make their information difficult to retrieve. Web Automation and Extraction systems are able to navigate through web links and to fill web forms automatically in order to get information not directly accessible by a URL. In these systems, the main optimization parameter is the time required to navigate through the intermediate pages which lead to the desired final pages. This paper proposes a series of techniques and algorithms that improves this parameter by basically storing historical information from previous queries, and using it to make the browser manager preload an adequate subset of the whole navigational sequence on a specific browser, before the following query is executed. These techniques also handle which sequences are the most common, thus being the ones which are preloaded more often.


Time Section Result Page Query Type Query Form Sample Query 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arasu, A., Garcia-Molina, H.: Extracting Structured Data from Web Pages. In: Proceedings of the ACM SIGMOD international conference on Management of data (2003)Google Scholar
  2. 2.
    Bergman, M.K.: The Deep Web. Surfacing Hidden Value,
  3. 3.
    Garret, J.J.: Ajax: A New Approach to Web Applications,
  4. 4.
    Hidalgo, J., Pan, A., Losada, J., Álvarez, M.: Adding Physical Optimization to Cost Models in Information Mediators. In: IEEE Conference on e-Business Engineering (2005)Google Scholar
  5. 5.
    Hidalgo, J., Pan, A., Losada, J., Álvarez, M., Viña, A.: Building the Architecture of a Statistics-based Query Optimization Solution for Heterogeneous Mediators. In: 6th International Conference on Information Integration and Web-based Applications & Services, p. 1 (2004)Google Scholar
  6. 6.
    Knoblock, C.A., Lerman, K., Minton, S., Muslea, I.: Accurately and Reliably Extracting Data from the Web: A Machine Learning Approach. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering (1999)Google Scholar
  7. 7.
    Kushmerick, N., Weld, D.S., Doorembos, R.: Wrapper induction for information extraction. In: Proceedings of the fifteenth International Joint Conference on Artificial Intelligence (1997)Google Scholar
  8. 8.
    Laender, A.H.F., Ribeiro-Neto, B.A., Soares da Silva, A., Teixeira, J.S.: A Brief Survey of Web Data Extraction Tools. ACM SIGMOD Record 31(2) (2002)Google Scholar
  9. 9.
    Pan, A., et al.: Semi-Automatic Wrapper Generation for Commercial Web Sources. In: Proceedings of IFIP WG8.1 Working Conference on Engineering Information Systems in the Internet Context (2002)Google Scholar
  10. 10.
    Pan, A., Raposo, J., Álvarez, M., Montoto, P., Orjales, V., Hidalgo, J., Ardao, L., Molano, A., Viña, A.: The DENODO Data Integration Platform. In: 28th International Conference on Very Large Databases (2002)Google Scholar
  11. 11.
    Raghavan, S., García-Molina, H.: Crawling the Hidden Web. In: Proceedings of the 27th International Conference on Very Large Databases (2001)Google Scholar
  12. 12.
    Wiederhold, G.: Mediators in the Architecture of Future Information Systems. IEEE Computer (March 1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Justo Hidalgo
    • 1
  • José Losada
    • 1
  • Manuel Álvarez
    • 2
  • Alberto Pan
    • 2
  1. 1.Denodo Technologies, Inc.MadridSpain
  2. 2.Department of Information and Communications Technologies.University of A CoruñaSpain

Personalised recommendations