Advertisement

Optimizing I/O Costs of Multi-dimensional Queries Using Bitmap Indices

  • Doron Rotem
  • Kurt Stockinger
  • Kesheng Wu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3588)

Abstract

Bitmap indices are efficient data structures for processing complex, multi-dimensional queries in data warehouse applications and scientific data analysis. For high-cardinality attributes, a common approach is to build bitmap indices with binning. This technique partitions the attribute values into a number of ranges, called bins, and uses bitmap vectors to represent bins (attribute ranges) rather than distinct values. In order to yield exact query answers, parts of the original data values have to be read from disk for checking against the query constraint. This process is referred to as candidate check and usually dominates the total query processing time.

In this paper we study several strategies for optimizing the candidate check cost for multi-dimensional queries. We present an efficient candidate check algorithm based on attribute value distribution, query distribution as well as query selectivity with respect to each dimension. We also show that re-ordering the dimensions during query evaluation can be used to reduce I/O costs. We tested our algorithm on data with various attribute value distributions and query distributions. Our approach shows a significant improvement over traditional binning strategies for bitmap indices.

Keywords

Range Query Query Evaluation Bitmap Index Query Selectivity Query Distribution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Chan, C.Y., Ioannidis, Y.E.: An Efficient Bitmap Encoding Scheme for Selection Queries. In: SIGMOD, Philadelphia, Pennsylvania, USA, June 1999. ACM Press, New York (1999)Google Scholar
  2. 2.
    Chaudhuri, S., Dayal, U.: An Overview of Data wharehousing and OLAP Technology. ACM SIGMOD Record 26(1), 65–74 (1997)CrossRefGoogle Scholar
  3. 3.
    Guha, S., Koudas, N., Srivastava, D.: Fast Algorithms For Hierarchical Range Histogram Construction. In: PODS 2002, Madison, Wisconsin, USA, June 2002. ACM Press, New York (2002)Google Scholar
  4. 4.
    Johnson, T.: Performance Measurements of Compressed Bitmap Indices. In: International Conference on Very Large Data Bases, Edinburgh, Scotland, September 1999. Morgan Kaufmann, San Francisco (1999)Google Scholar
  5. 5.
    Koudas, N.: Space Efficient Bitmap Indexing. In: International Conference on Information and Knowledge Management, McLean, Virginia, USA, November 2000, ACM Press, New York (2000)Google Scholar
  6. 6.
    Koudas, N., Muthukrishnan, S., Srivastava, D.: Optimal Histograms for Hierarchical Range Queries. In: PODS, Dallas, Texas, USA. ACM Press, New York (2000)Google Scholar
  7. 7.
    Mohr, A.E.: Bit Allocation in Sub-linear Time and the Multiple-Choice Knapsack Problem. In: Data Compression Conference, Snao Bird, Utah, USA, March 2002. IEEE Computer Society Press, Los Alamitos (2002)Google Scholar
  8. 8.
    O’Neil, P.: Model 204 Architecture and Performance. In: 2nd International Workshop in High Performance Transaction Systems, Asilomar, California, USA. Springer, Heidelberg (1987)Google Scholar
  9. 9.
    O’Neil, P., Quass, D.: Improved Query Performance with Variant Indexes. In: Proceedings ACM SIGMOD International Conference on Management of Data, Tucson, Arizona, USA, May 1997. ACM Press, New York (1997)Google Scholar
  10. 10.
    Rotem, D., Stockinger, K., Wu, K.: Efficient Binning for Bitmap Indices on High-Cardinality Attributes. Technical report, Berkeley Lab. (November 2004)Google Scholar
  11. 11.
    Stockinger, K., Wu, K., Shoshani, A.: Evaluation Strategies for Bitmap Indices with Binning. In: Galindo, F., Takizawa, M., Traunmüller, R. (eds.) DEXA 2004. LNCS, vol. 3180, pp. 120–129. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  12. 12.
    Wu, K., Otoo, E.J., Shoshani, A.: On the Performance of Bitmap Indices for High Cardinality Attributes. In: International Conference on Very Large Data Bases, Toronto, Canada, September 2004. Morgan Kaufmann, San Francisco (2004)Google Scholar
  13. 13.
    Wu, M.-C., Buchmann, A.P.: Encoded Bitmap Indexing for Data Warehouses. In: International Conference on Data Engineering, Orlando, Florida, USA, February 1998. IEEE Computer Society Press, Los Alamitos (1998)Google Scholar
  14. 14.
    Wu, K.-L., Yu, P.S.: Range-Based Bitmap Indexing for High-Cardinality Attributes with Skew. Technical report, IBM Watson Research Center (May 1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Doron Rotem
    • 1
  • Kurt Stockinger
    • 1
  • Kesheng Wu
    • 1
  1. 1.Lawrence Berkeley National LaboratoryUniversity of CaliforniaBerkeleyUSA

Personalised recommendations