Generating Shifting Workloads to Benchmark Adaptability in Relational Database Systems

  • Tilmann Rabl
  • Andreas Lang
  • Thomas Hackl
  • Bernhard Sick
  • Harald Kosch
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5895)


A large body of research concerns the adaptability of database systems. Many commercial systems already contain autonomic processes that adapt configurations as well as data structures and data organization. Yet there is virtually no possibility for a just measurement of the quality of such optimizations. While standard benchmarks have been developed that simulate real-world database applications very precisely, none of them considers variations in workloads produced by human factors. Today’s benchmarks test the performance of database systems by measuring peak performance on homogeneous request streams. Nevertheless, in systems with user interaction access patterns are constantly shifting. We present a benchmark that simulates a web information system with interaction of large user groups. It is based on the analysis of a real online eLearning management system with 15,000 users. The benchmark considers the temporal dependency of user interaction. Main focus is to measure the adaptability of a database management system according to shifting workloads. We will give details on our design approach that uses sophisticated pattern analysis and data mining techniques.


Benchmarking Adaptability Polynomial Approximation Time Series Generation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Zilio, D.C., Rao, J., Lightstone, S., Lohman, G.M., Storm, A.J., Garcia-Arellano, C., Fadden, S.: Db2 design advisor: Integrated automatic physical database design. In: VLDB 2004: Proceedings of the Thirtieth International Conference on Very Large Data Bases, pp. 1087–1097. Morgan Kaufmann, San Francisco (2004)Google Scholar
  2. 2.
    Dageville, B., Das, D., Dias, K., Yagoub, K., Zaït, M., Ziauddin, M.: Automatic sql tuning in oracle 10g. In: VDLB 2004: Proceedings of the Thirtieth International Conference on Very Large Data Bases, pp. 1098–1109. Morgan Kaufmann, San Francisco (2004)Google Scholar
  3. 3.
    Agrawal, S., Chaudhuri, S., Kollár, L., Marathe, A.P., Narasayya, V.R., Syamala, M.: Database tuning advisor for microsoft sql server 2005. In: VDLB 2004: Proceedings of the Thirtieth International Conference on Very Large Data Bases, pp. 1110–1121. Morgan Kaufmann, San Francisco (2004)Google Scholar
  4. 4.
    Bruno, N., Chaudhuri, S.: An online approach to physical design tuning. In: ICDE 2007: Proceedings of the 23rd International Conference on Data Engineering, pp. 826–835. IEEE, Los Alamitos (2007)CrossRefGoogle Scholar
  5. 5.
    Wiese, D., Rabinovitch, G., Reichert, M., Arenswald, S.: Autonomic tuning expert: a framework for best-practice oriented autonomic database tuning. In: CASCON 2008: Proceedings of the 2008 conference of the center for advanced studies on collaborative research, pp. 27–41. ACM, New York (2008)CrossRefGoogle Scholar
  6. 6.
    Nambiar, R.O., Poess, M.: The making of tpc-ds. In: VLDB 2006: Proceedings of the 32nd international conference on Very large data bases, pp. 1049–1058 (2006)Google Scholar
  7. 7.
    Poess, M.: Controlled sql query evolution for decision support benchmarks. In: WSOP 2007: Proceedings of the 6th International Workshop on Software and Performance, pp. 38–41. ACM, New York (2007)CrossRefGoogle Scholar
  8. 8.
    Hsu, W.W., Smith, A.J., Young, H.C.: Characteristics of production database workloads and the tpc benchmarks. IBM Systems Journal 40(3), 781–802 (2001)CrossRefGoogle Scholar
  9. 9.
    Agrawal, S., Chu, E., Narasayya, V.: Automatic physical design tuning: Workload as a sequence. In: SIGMOD 2006: Proceedings of the 2006 ACM SIGMOD international conference on Management of data, pp. 683–694. ACM, New York (2006)CrossRefGoogle Scholar
  10. 10.
    Holze, M., Ritter, N.: Autonomic databases: Detection of workload shifts with n-gram-models. In: Atzeni, P., Caplinskas, A., Jaakkola, H. (eds.) ADBIS 2008. LNCS, vol. 5207, pp. 127–142. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  11. 11.
    Consens, M.P., Barbosa, D., Teisanu, A.M., Mignet, L.: Goals and benchmarks for autonomic configuration recommenders. In: SIGMOD 2005: Proceedings of the 2005 ACM SIGMOD International Conference on Management of Data, pp. 239–250. ACM, New York (2005)CrossRefGoogle Scholar
  12. 12.
    Rabl, T., Pfeffer, M., Kosch, H.: Dynamic allocation in a self-scaling cluster database. Concurrency and Computation: Practice and Experience 20(17), 2025–2038 (2007)CrossRefGoogle Scholar
  13. 13.
    Mitzenmacher, M.: A brief history of generative models for power law and lognormal distributions. Internet Mathematics 1(2), 226–251 (2004)zbMATHMathSciNetGoogle Scholar
  14. 14.
    Chen, P.P.S.: The entity-relationship model — toward a unified view of data. ACM Transactions on Database Systems 1(1), 9–36 (1976)CrossRefGoogle Scholar
  15. 15.
    Stephens, J.M., Poess, M.: Mudd: a multi-dimensional data generator. In: WOSP 2004: Proceedings of the 4th international workshop on Software and performance, pp. 104–109. ACM, New York (2004)CrossRefGoogle Scholar
  16. 16.
    Fuchs, E., Gruber, C., Reitmaier, T., Sick, B.: Processing short-term and long-term information with a combination of polynomial approximation techniques and time-delay neural networks. IEEE Transactions on Neural Networks (2009) (accepted – to appear)Google Scholar
  17. 17.
    Fuchs, E., Gruber, T., Nitschke, J., Sick, B.: On-line motif detection in time series with SwiftMotif. Pattern Recognition 42(11), 3015–3031 (2009)zbMATHCrossRefGoogle Scholar
  18. 18.
    Elhay, S., Golub, G.H., Kautsky, J.: Updating and downdating of orthogonal polynomials with data fitting applications. SIAM Journal on Matrix Analysis and Applications 12(2), 327–353 (1991)zbMATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Fuchs, E.: On discrete polynomial least-squares approximation in moving time windows. In: Gautschi, W., Golub, G., Opfer, G. (eds.) Applications and Computation of Orthogonal Polynomials. International Series of Numerical Mathematics, vol. 131, pp. 93–107. Birkhäuser, Basel (1999); (Proceedings of the Conference at the Mathematical Research Institute Oberwolfach, Germany, March 22-28 1998)Google Scholar
  20. 20.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)zbMATHGoogle Scholar
  21. 21.
    Blackburn, S.M., McKinley, K.S., Garner, R., Hoffmann, C., Khan, A.M., Bentzur, R., Diwan, A., Feinberg, D., Frampton, D., Guyer, S.Z., Hirzel, M., Hosking, A.L., Jump, M., Lee, H., Moss, J.E.B., Phansalkar, A., Stefanovic, D., Van Drunen, T., von Dincklage, D., Wiedermann, B.: Wake up and smell the coffee: evaluation methodology for the 21st century. Communications of the ACM 51(8), 83–89 (2008)CrossRefGoogle Scholar
  22. 22.
    Bruno, N.: A critical look at the tab benchmark for physical design tools. SIGMOD Record 36(4), 7–12 (2007)CrossRefGoogle Scholar
  23. 23.
    Poess, M., Nambiar, R.O., Walrath, D.: Why you should run tpc-ds: A workload analysis. In: VLDB 2007: Proceedings of the 33rd international conference on Very large data bases, pp. 1138–1149. VLDB Endowment (2007)Google Scholar
  24. 24.
    Poess, M., Nambiar, R.O.: Energy cost, the key challenge of today’s data centers: A power consumption analysis of tpc-c results. Proceedings of VLDB Endowment 1(2), 1229–1240 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Tilmann Rabl
    • 1
  • Andreas Lang
    • 1
  • Thomas Hackl
    • 2
  • Bernhard Sick
    • 3
  • Harald Kosch
    • 1
  1. 1.Chair of Distributed Information Systems 
  2. 2.InteLeC-Zentrum 
  3. 3.Computationally Intelligent Systems GroupUniversity of PassauGermany

Personalised recommendations