The VLDB Journal

, Volume 19, Issue 2, pp 203–230 | Cite as

A framework for testing DBMS features

  • Eric Lo
  • Carsten Binnig
  • Donald Kossmann
  • M. Tamer Özsu
  • Wing-Kai Hon
Regular Paper


Testing a specific feature of a DBMS requires controlling the inputs and outputs of the operators in the query execution plan. However, that is practically difficult to achieve because the inputs/outputs of a query depend on the content of the test database. In this paper, we propose a framework to test DBMS features. The framework includes a database generator called QAGen so that the generated test databases are able to meet the test requirements defined on the test queries. The framework also includes a set of tools to automate test case constructions and test executions. A wide range of DBMS feature testing tasks can be facilitated by the proposed framework.


Database testing Data generation Symbolic query processing Symbolic execution 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ammann, P., Offutt, J.: Using formal methods to derive test frames in category-partition testing. In: Annual Conference on Computer Assurance, pp. 69–80 (1994)Google Scholar
  2. 2.
    Binnig, C., Kossmann, D., Lo, E.: Reverse query processing. In: ICDE (2007)Google Scholar
  3. 3.
    Binnig, C., Kossmann, D., Lo, E., Özsu, M.T.: QAGen: generating query-aware test databases. In: SIGMOD, pp. 341–352 (2007)Google Scholar
  4. 4.
    Bruno, N., Chaudhuri, S.: Flexible database generators. In VLDB, pp. 1097–1107 (2005)Google Scholar
  5. 5.
    Bruno, N., Chaudhuri, S., Thomas, D.: Generating Queries with Cardinality Constraints for DBMS Testing. TKDE (2006)Google Scholar
  6. 6.
    Chaudhuri, S., Narasayya, V.: TPC-D data generation with skew. Accessible at (1997)
  7. 7.
    Chordia, S., D ettinger, E., Triou, E.: Different query verification approaches used to test entity sql. In: DBTest, p. 7 (2008)Google Scholar
  8. 8.
    Clarke, E.M., Grumberg, O., Peled, D.A.: Model checking (2000)Google Scholar
  9. 9.
    Codd E.F.: A relational model of data for large shared data banks. Commun. ACM 13(6), 377–387 (1970)zbMATHCrossRefGoogle Scholar
  10. 10.
    Cohen D.M., Dalal S.R., Fredman M.L., Patton G.C.: The aetg system: an approach to testing based on combinatiorial design. IEEE TSE 23(7), 437–444 (1997)Google Scholar
  11. 11.
    Cohen, M.B., Gibbons, P.B., Mugridge, W.B., Colbourn, C.J.: Constructing test suites for interaction testing. In: ICSE, pp. 38–48 (2003)Google Scholar
  12. 12.
    Cook, B., Kroening, D., Sharygina, N.: Cogent: Accurate theorem proving for program verification. In: CAV, pp. 296–300 (2005)Google Scholar
  13. 13.
    DTM Data Generator.
  14. 14.
    Elhemali, M., Giakoumakis, L.: Unit-testing query transformation rules. In: DBTest, p. 3 (2008)Google Scholar
  15. 15.
    Elkan, C.: A decision procedure for conjunctive query disjointness. In: PODS (1989)Google Scholar
  16. 16.
    Ganski, R.A., Wong, H.K.T.: Optimization of nested SQL queries revisited. In: SIGMOD, pp. 23–33 (1987)Google Scholar
  17. 17.
    Garey, M.R., Johnson, D.S.: Computers and Intractability; A Guide to the Theory of NP-Completeness (1990)Google Scholar
  18. 18.
    Graefe G.: Query evaluation techniques for large databases. ACM Comput. Surv. 25(2), 73–170 (1993)CrossRefGoogle Scholar
  19. 19.
    Gray, J., Sundaresan, P., Englert, S., Baclawski, K., Weinberger, P.J.: Quickly generating billion-record synthetic databases. In: SIGMOD, pp. 243–252 (1994)Google Scholar
  20. 20.
    Grindal M., Offutt J., Andler S.F.: Combination testing strategies: a survey. Softw. Test. Verif. Reliab. 15(3), 167–199 (2005)CrossRefGoogle Scholar
  21. 21.
    Houkjær, K., Torp, K., Wind, R.: Simple and realistic data generation. In: VLDB, pp. 1243–1246 (2006)Google Scholar
  22. 22.
    Ibarra O.H., Kim C.E.: Fast approximation algorithms for the knapsack and sum of subset problems. J. ACM 22(4), 463–468 (1975)zbMATHCrossRefMathSciNetGoogle Scholar
  23. 23.
  24. 24.
    Kellerer H., Mansini R., Pferschy U., Speranza M.G.: An efficient fully polynomial approximation scheme for the subset-sum problem. J. Comput. Syst. Sci. 66(2), 349–370 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  25. 25.
    King J.C.: Symbolic execution and program testing. Commun. ACM 19(7), 385–394 (1976)zbMATHCrossRefGoogle Scholar
  26. 26.
    Klug A.: Calculating constraints on relational expression. TODS 5(3), 260–290 (1980)zbMATHCrossRefGoogle Scholar
  27. 27.
    Kuijpers B.: Introduction to constraint databases. SIGMOD Rec. 31(3), 35–36 (2002)CrossRefGoogle Scholar
  28. 28.
    Lo, E., Binnig, C., Kossmann, D., Ozsu, M.T., Hon, W.-K.: A Framework for Testing DBMS Features. Technical Report. Hong Kong Polytechnic University (2008)Google Scholar
  29. 29.
    Mannila, H., Räihä, K.-J.: Test data for relational queries. In: PODS, pp. 217–223 (1986)Google Scholar
  30. 30.
    Mishra, C., Koudas, N., Zuzarte, C.: Generating targeted queries for database testing. In: SIGMOD, pp. 499–510 (2008)Google Scholar
  31. 31.
    Ostrand, T.J., Balcer, M.J.: The category-partition method for specifying and generating fuctional tests. Commun. ACM, 31(6) (1988)Google Scholar
  32. 32.
    Poess, M., Stephens, J.M.: Generating thousand benchmark queries in seconds. In: VLDB, pp. 1045–1053 (2004)Google Scholar
  33. 33.
    Przydatek B.: A fast approximation algorithm for the subset-sum problem. Int. Trans. Oper. Res. 9(4), 437–459 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  34. 34.
    Slutz, D.R.: Massive Stochastic Testing of SQL. In: VLDB, pp. 618–622 (1998)Google Scholar
  35. 35.
    Stephens, J.M., Poess, M.: Mudd: a multi-dimensional data generator. In: WOSP, pp. 104–109 (2004)Google Scholar
  36. 36.
    Williams, A.W., Probert, R.L.: A measure for component interaction test coverage. In: AICCSA, pp. 304–312 (2001)Google Scholar
  37. 37.
    Zipf, G.: Human Behaviour and the Principle of Least Effort (1949)Google Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  • Eric Lo
    • 1
  • Carsten Binnig
    • 2
  • Donald Kossmann
    • 2
  • M. Tamer Özsu
    • 3
  • Wing-Kai Hon
    • 4
  1. 1.Department of ComputingThe Hong Kong Polytechnic UniversityHung HomHong Kong
  2. 2.ETH ZurichZurichSwitzerland
  3. 3.David R. Cheriton School of Computer ScienceUniversity of WaterlooWaterlooCanada
  4. 4.Department of Computer ScienceNational Tsing Hua UniversityHsinchuTaiwan

Personalised recommendations