Efficiently Updating Cost Repository Values for Query Optimization on Web Data Sources in a Mediator/Wrapper Environment

  • Justo Hidalgo
  • Alberto Pan
  • Manuel Álvarez
  • Jaime Guerrero
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4032)


Optimizing accesses to sources in a mediator/wrapper environment is a critical need. Due to a variety of reasons, relational-based optimization techniques are of no use when having to handle HTTP-based web sources, so new approaches which take into account client/server communication costs must be devised. This paper describes a cost model that stores values from a complete set of web source-focused parameters obtained by the web wrappers, by using a novel updating technique that handles the values measured by the wrappers in previous query executions, and generates a new model instance in each new iteration with an efficient processing cost. This instance allows rapid value updates caused by changes of the server quality or bandwidth, so typical in this context. The results of these techniques are demonstrated both theoretically and by means of an implementation showing how performance improves in real-world web sources when compared to classical approaches.


Cost Model Cost Propagation Query Optimization Query Execution Query Plan 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Adali, S., Candan, K., Papakonstantinou, Y., Subrahmanian, V.S.: Query Caching and Optimization in distributed mediator systems. In: ACM SIGMOD Conference on Management of Data (1996)Google Scholar
  2. 2.
    Citeseer Scientific Literature Digital Library, http://citeseer.ist.psu.edu
  3. 3.
    Du, W., Krishnamurthy, R., Shan, M.: Query Optimization in a Heterogeneous DMBS. In: 18th International Conference on Very Large Databases (1992)Google Scholar
  4. 4.
    Google Scholar Search Engine, http://scholar.google.com
  5. 5.
    Hidalgo, J., Pan, A., Losada, J., Álvarez, M.: Adding Physical Optimization to Cost Models in Information Mediators. In: 2005 IEEE Conference on e-Business Engineering (2005)Google Scholar
  6. 6.
    Hidalgo, J., Pan, A., Losada, J., Álvarez, M., Viña, A.: Building the Architecture of a Statistics-based Query Optimization Solution for Heterogeneous Mediators. In: 6th International Conference on Information Integration and Web-based Applications & Services (2004)Google Scholar
  7. 7.
    Ives, Z., Florescu, D., Friedman, M., Levy, A., Weld, D.S.: An Adaptive Query Execution System for Data Integration. In: 24th International Conference on Very Large Databases (1999)Google Scholar
  8. 8.
    Naacke, H., Gardarin, G., Tomasic, A.: Leveraging mediator cost models with heterogeneous data sources. In: ICDE (1998)Google Scholar
  9. 9.
    Ozcan, R., Haas, L.: Cost Models DO Matter: Providing Cost Information for Diverse Data Sources in a Federated System. In: The VLDB Conference (1999)Google Scholar
  10. 10.
    Pan, A., Raposo, J., Álvarez, M., Montoto, P., Orjales, V., Hidalgo, J., Ardao, L., Molano, A., Viña, A.: The DENODO Data Integration Platform. In: 28th International Conference on Very Large Databases (2002)Google Scholar
  11. 11.
    Papakonstantinou, Y., Gupta, A., Haas, L.: Capabilities-Based Query Rewriting in Mediator Systems. In: Fourth International Conference on Parallel and Distributed Information Systems (1996)Google Scholar
  12. 12.
    Rahal, A., Zhu, Q., Larson, P.-A.: Evolutionary Techniques for Updating Query Cost Models in a Dynamic Multidatabase Environment. The VLDB Journal – The International Journal on Very Large Data Bases 13(2) (2004)Google Scholar
  13. 13.
    Raposo, J., Pan, A., Hidalgo, J., Viña, A.: The WARGO System: Semi-automatic Wrapper Generation in presence of Complex Data Access Modes. In: 2nd International Workshop on Web Based Collaboration (2002)Google Scholar
  14. 14.
    Wiederhold, G.: Mediators in the Architecture of Future Information Systems. IEEE Computer, 38–49 (March 1992)Google Scholar
  15. 15.
    XIWT. Cross-Industry Working Team. Internet Service Performance: Data Analysis and Visualization (July 2000), http://www.xiwt.org/documents/IPERF-paper2.pdf
  16. 16.
    Zadorozhny, V., Raschid, L., Vidal, M.E.: Efficient Evaluation of Queries in a Mediator for WebSources. In: ACM SIGMOD International Conference on Management of Data (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Justo Hidalgo
    • 1
  • Alberto Pan
    • 2
  • Manuel Álvarez
    • 2
  • Jaime Guerrero
    • 2
  1. 1.Denodo Technologies, Inc.MadridSpain
  2. 2.Department of Information and Communications TechnologiesUniversity of A CoruñaSpain

Personalised recommendations