Advertisement

Benchmarking Query Execution Robustness

  • Janet L. Wiener
  • Harumi Kuno
  • Goetz Graefe
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5895)

Abstract

Benchmarks that focus on running queries on a well-tuned database system ignore a long-standing problem: adverse runtime conditions can cause database system performance to vary widely and unexpectedly. When the query execution engine does not exhibit resilience to these adverse conditions, addressing the resultant performance problems can contribute significantly to the total cost of ownership for a database system in over-provisioning, lost efficiency, and increased human administrative costs. For example, focused human effort may be needed to manually invoke workload management actions or fine-tune the optimization of specific queries.

We believe a benchmark is needed to measure query execution robustness, that is, how adverse or unexpected conditions impact the performance of a database system. We offer a preliminary analysis of barriers to query execution robustness and propose some metrics for quantifying the impact of those barriers. We present and analyze results from preliminary tests on four real database systems and discuss how these results could be used to increase the robustness of query processing in each case. Finally, we outline how our efforts could be expanded into a benchmark to quantify query execution robustness.

Keywords

robust query processing robust query execution data warehouses operational business intelligence 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Babu, S., Bizarro, P., DeWitt, D.: Proactive re-optimization with rio. In: SIGMOD 2005: Proceedings of the 2005 ACM SIGMOD international conference on Management of data, pp. 936–938. ACM, New York (2005)CrossRefGoogle Scholar
  2. 2.
    Bizarro, P., Babu, S., DeWitt, D., Widom, J.: Content-based routing: different plans for different data. In: VLDB 2005: Proceedings of the 31st international conference on Very large data bases, pp. 757–768. VLDB Endowment (2005)Google Scholar
  3. 3.
    Cole, R.L., Graefe, G.: Optimization of dynamic query evaluation plans. In: SIGMOD 1994: Proceedings of the 1994 ACM SIGMOD international conference on Management of data, pp. 150–160. ACM, New York (1994)CrossRefGoogle Scholar
  4. 4.
    El Gebaly, K., Aboulnaga, A.: Robustness in automatic physical database design. In: EDBT (2008)Google Scholar
  5. 5.
    Graefe, G.: Query evaluation techniques for large databases. ACM Comput. Surv. 25(2), 73–169 (1993)CrossRefGoogle Scholar
  6. 6.
    Graefe, G., Kuno, H.A., Wiener, J.L.: Visualizing the robustness of query execution. In: CIDR (2009)Google Scholar
  7. 7.
    Gupta, A., Davis, K.C., Grommon-Litton, J.: Performance comparison of property map and bitmap indexing. In: DOLAP 2002: Proceedings of the 5th ACM international workshop on Data Warehousing and OLAP, pp. 65–71. ACM, New York (2002)CrossRefGoogle Scholar
  8. 8.
    Harish, D., Darera, P.N., Haritsa, J.R.: On the production of anorexic plan diagrams. In: VLDB 2007: Proceedings of the 33rd international conference on Very large data bases, pp. 1081–1092. VLDB Endowment (2007)Google Scholar
  9. 9.
    Harish, D., Darera, P.N., Haritsa, J.R.: Identifying robust plans through plan diagram reduction. In: VLDB, pp. 1124–1140 (2008)Google Scholar
  10. 10.
    Ioannidis, Y.E., Ng, R.T., Shim, K., Sellis, T.K.: Parametric query optimization. VLDB Journal 6(2), 132–151 (1997)CrossRefGoogle Scholar
  11. 11.
    Krompass, S., Kuno, H., Dayal, U., Kemper, A.: Dynamic workload management for very large data warehouses: Juggling feathers and bowling balls. In: VLDB (2007)Google Scholar
  12. 12.
    Markl, V., Lohman, G.: Learning table access cardinalities with LEO. In: SIGMOD 2002: Proceedings of the 2002 ACM SIGMOD international conference on Management of data, pp. 613–613. ACM, New York (2002)CrossRefGoogle Scholar
  13. 13.
    Markl, V., Raman, V., Simmen, D., Lohman, G., Pirahesh, H., Cilimdzic, M.: Robust query processing through progressive optimization. In: SIGMOD 2004: Proceedings of the 2004 ACM SIGMOD international conference on Management of data, pp. 659–670. ACM, New York (2004)CrossRefGoogle Scholar
  14. 14.
    Othayoth, R., Poess, M.: The Making of TPC-DS. In: Proc. of the 32nd Intl. Conf. on Very Large Data Bases (VLDB), pp. 1049–1058 (2006)Google Scholar
  15. 15.
    Poess, M., Floyd, C.: New tpc benchmarks for decision support and web commerce. SIGMOD Rec. 29(4) (2000)Google Scholar
  16. 16.
    Reddy, N., Haritsa, J.R.: Analyzing plan diagrams of database query optimizers. In: VLDB 2005: Proceedings of the 31st international conference on Very large data bases, pp. 1228–1239. VLDB Endowment (2005)Google Scholar
  17. 17.
    Sarda, P., Haritsa, J.R.: Green query optimization: taming query optimization overheads through plan recycling. In: VLDB 2004: Proceedings of the Thirtieth international conference on Very large data bases, pp. 1333–1336. VLDB Endowment (2004)Google Scholar
  18. 18.
    Schneider, D.A., DeWitt, D.J.: A performance evaluation of four parallel join algorithms in a shared-nothing multiprocessor environment. SIGMOD Rec. 18(2), 110–121 (1989)CrossRefGoogle Scholar
  19. 19.
    Stillger, M., Lohman, G.M., Markl, V., Kandil, M.: LEO - db2’s learning optimizer. In: VLDB 2001: Proceedings of the 27th International Conference on Very Large Data Bases, pp. 19–28. Morgan Kaufmann Publishers Inc., San Francisco (2001)Google Scholar
  20. 20.
    Zhang, N., Haas, P.J., Josifovski, V., Lohman, G.M., Zhang, C.: Statistical learning techniques for costing XML queries. In: VLDB 2005: Proceedings of the 31st international conference on Very large data bases, pp. 289–300. VLDB Endowment (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Janet L. Wiener
    • 1
  • Harumi Kuno
    • 1
  • Goetz Graefe
    • 1
  1. 1.Hewlett-Packard LabsPalo AltoUSA

Personalised recommendations