Advertisement

Selectivity Estimation of High Dimensional Window Queries via Clustering

  • Christian Böhm
  • Hans-Peter Kriegel
  • Peer Kröger
  • Petra Linhart
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3633)

Abstract

Query optimization is an important functionality of modern database systems and often based on estimating the selectivity of queries before actually executing them. Well-known techniques for estimating the result set size of a query are sampling and histogram-based solutions. Sampling-based approaches heavily depend on the size of the drawn sample which causes a trade-off between the quality of the estimation and the time in which the estimation can be executed for large data sets. Histogram-based techniques eliminate this problem but are limited to low-dimensional data sets. They either assume that all attributes are independent which is rarely true for real-world data or else get very inefficient for high-dimensional data. In this paper we present the first multivariate parametric method for estimating the selectivity of window queries for large and high-dimensional data sets. We use clustering to compress the data by generating a precise model of the data using multivariate Gaussian distributions. Additionally, we show efficient techniques to evaluate a window query against the Gaussian distributions we generated. Our experimental evaluation shows that this approach is significantly more efficient for multidimensional data than all previous approaches.

Keywords

Storage Cost Query Optimization Window Query Selectivity Estimation High Dimensional Data Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Faloutsos, C., Barber, R., Flickner, M., Hafner, J., Niblack, W., Petkovic, D., Equitz, W.: Efficient and Effective Querying by Image Content. Journal of Intelligent Information Systems 3, 231–262 (1994)CrossRefGoogle Scholar
  2. 2.
    Mehrotra, R., Gary, J.: Feature-Based Retrieval of Similar Shapes. In: Proc. 9th Int. Conf. on Data Engineering, Vienna, Austria, pp. 108–115 (1993)Google Scholar
  3. 3.
    Shoichet, B.K., Bodian, D.L., Kuntz, I.D.: Molecular Docking Using Shape Descriptors. Journal of Computational Chemistry 13, 380–397 (1992)CrossRefGoogle Scholar
  4. 4.
    Berchtold, S., Keim, D.A., Kriegel, H.P.: The X-Tree: An Index Structure for High-Dimensional Data. In: Proc. 22nd Int. Conf. on Very Large Databases, VLDB 1996 (1996)Google Scholar
  5. 5.
    Lin, K.I., Jagadish, H.V., Faloutsos, C.: The TV-tree an index structure for high-dimensional data. VLDB Journal: Very Large Data Bases 3, 517–542 (1994)CrossRefGoogle Scholar
  6. 6.
    Weber, R., Schek, H.J., Blott, S.: A Quantitative Analysis and Performance Study for Similarity-Search Methods in High-Dimensional Spaces. In: Proc. 24th Int. Conf. on Very Large Databases (VLDB 1998), pp. 194–205 (1998)Google Scholar
  7. 7.
    McQueen, J.: Some Methods for Classification and Analysis of Multivariate Observations. In: 5th Berkeley Symp. Math. Statist. Prob., vol. 1, pp. 281–297 (1967)Google Scholar
  8. 8.
    Sibson, R.: SLINK: An Optimally Efficient Algorithm for the Single-Link Cluster Method. The Computer Journal 16, 30–34 (1973)CrossRefMathSciNetGoogle Scholar
  9. 9.
    Ester, M., Kriegel, H.P., Sander, J., Xu, X.: A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. In: Proc. 2nd Int. Conf. on Knowledge Discovery and Data Mining (KDD 1996), Portland, OR, pp. 291–316 (1996)Google Scholar
  10. 10.
    Ankerst, M., Breunig, M.M., Kriegel, H.P., Sander, J.: OPTICS: Ordering Points to Identify the Clustering Structure. In: Proc. ACM Int. Conf. on Management of Data, SIGMOD 1999 (1999)Google Scholar
  11. 11.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, Series B 39, 1–31 (1977)zbMATHMathSciNetGoogle Scholar
  12. 12.
    Selinger, P.G., Astrahan, M.M., Chamberlin, D.D., Lorie, R.A., Price, T.G.: Access Path Selection in a Relational Database Management System. In: Proc. ACM Int. Conf. on Management of Data, SIGMOD 1979 (1979)Google Scholar
  13. 13.
    Piatetsky-Shapiro, G., Connell, C.: Accurate estimation of the number of tuples satisfying a condition. In: Proc. ACM Int. Conf. on Management of Data, SIGMOD 1984 (1984)Google Scholar
  14. 14.
    Muralikrishna, M., De Witt, D.J.: Equi-Depth Histograms For Estimating Selectivity Factors For Muli-Dimensional Queries. In: Proc. ACM Int. Conf. on Management of Data, SIGMOD 1988 (1988)Google Scholar
  15. 15.
    Poosala, V., Ioannidis, Y.E.: Selectivity Estimation without the Attribute Value Independence Assumption. In: Proc. 23rd Int. Conf. on Very Large Databases, VLDB 1997 (1997)Google Scholar
  16. 16.
    Bruno, N., Chaudhuri, S., Gravan, L.: STHoles: a Multidimensional Workload-aware Histogram. In: Proc. ACM Int. Conf. on Management of Data, SIGMOD 2001 (2001)Google Scholar
  17. 17.
    Matias, Y., Vitter, J.S., Wang, M.: Wavelet-Based Histograms for Selectivity Estimation. In: Proc. ACM Int. Conf. on Management of Data (SIGMOD 1998), pp. 448–459 (1998)Google Scholar
  18. 18.
    Lipton, R., Naughton, J.: Query size estimation by adaptive sampling. In: Proc. ACM Symp. on Principles of Database Systems, PODS 1990 (1990)Google Scholar
  19. 19.
    Lipton, R., Naughton, J., Schneider, D.: Practical selectivity estimation through adaptive sampling. In: Proc. ACM Int. Conf. on Management of Data, SIGMOD 1990 (1990)Google Scholar
  20. 20.
    Hou, W.C., Ozsoyoglu, G., Dodgu, E.: Error-constrained Count Query: Evaluation in Relational Databases. In: Proc. ACM Int. Conf. on Management of Data, SIGMOD 1991 (1991)Google Scholar
  21. 21.
    Chen, C.M., Roussopoulos, N.: Adaptive Selectivity Estimation Using Query Feedback. In: Proc. ACM Int. Conf. on Management of Data, SIGMOD 1994 (1994)Google Scholar
  22. 22.
    Han, J., Kamber, M.: Data Mining: Concepts and Techniques. Academic Press, London (2001)Google Scholar
  23. 23.
    Fayyad, U., Reina, C., Bradley, P.: Initialization of Iterative Refinement Clustering Algorithms. In: Proc. 4th Int. Conf. on Knowledge Discovery and Data Mining, KDD 1998 (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Christian Böhm
    • 1
  • Hans-Peter Kriegel
    • 1
  • Peer Kröger
    • 1
  • Petra Linhart
    • 1
  1. 1.Institute for Computer ScienceUniversity of MunichGermany

Personalised recommendations