SSDBM 2008: Scientific and Statistical Database Management pp 517-524 | Cite as
Analysis of Basic Data Reordering Techniques
Abstract
Data reordering techniques are applied to improve the space and time efficiency of storage and query systems in various scientific and commercial applications. Run-length encoding is a prominent approach of compression in many areas, whose performance is significantly enhanced by achieving longer and fewer “runs” through data reordering. In this paper we theoretically study two reordering techniques, namely lexicographical order and Gray code order. We analyze these two methods in the context of bitmap indexes, which are known to have high query performances. We take into account the two commonly used bitmap encodings: equality and range. Our analysis indicates that, when we have all the possible data tuples, both ordering methods perform the same with equality encoding. However, Gray code achieves better compression with range encoding. Experimental results are provided to validate the theoretical analysis.
Keywords
Full Data Range Query Lexicographic Order Gray Code Compression PerformancePreview
Unable to display preview. Download preview PDF.
References
- 1.Antoshenkov, G.: Byte-aligned bitmap compression. In: Data Compression Conference, Nashua, NH. Oracle Corp. (1995)Google Scholar
- 2.Antoshenkov, G., Ziauddin, M.: Query processing and optimization in oracle rdb. The VLDB Journal 5(4), 229–237 (1996)CrossRefGoogle Scholar
- 3.Chan, C.Y., Ioannidis, Y.E.: Bitmap index design and evaluation. In: Proceedings of the 1998 ACM SIGMOD international conference on Management of data, pp. 355–366. ACM Press, New York (1998)CrossRefGoogle Scholar
- 4.Informix. Decision support indexing for enterprise datawarehouse, http://www.informix.com/informix/corpinfo/-zines/whiteidx.htm
- 5.Johnson, D., Krishnan, S., Chhugani, J., Kumar, S., Venkatasubramanian, S.: Compressing large boolean matrices using reordering techniques. In: VLDB 2004 (2004)Google Scholar
- 6.Chen, J., Wu, K., Koegler, W., Shoshani, A.: Using bitmap index for interactive exploration of large datasets. In: Proceedings of SSDBM (2003)Google Scholar
- 7.O’Neil, P.: Informix and Indexing Support for Data Warehouses. Database Programming and Design 10, 38–43 (1997)Google Scholar
- 8.O’Neil, P., Quass, D.: Improved query performance with variant indexes. In: Proceedings of the 1997 ACM SIGMOD international conference on Management of data, pp. 38–49. ACM Press, New York (1997)CrossRefGoogle Scholar
- 9.Pinar, A., Tao, T., Ferhatosmanoglu, H.: Compressing bitmap indices by data reorganization. In: ICDE, pp. 310–321 (2005)Google Scholar
- 10.Salomon, D.: Data Compression: The Complete Reference, 3rd edn (2004)Google Scholar
- 11.Stockinger, K., Shalf, J., Bethel, W., Wu, K.: Dex: Increasing the capability of scientific data analysis pipelines by using efficient bitmap indices to accelerate scientific visualization. In: Proceedings of SSDBM (2005)Google Scholar
- 12.Wu, K., Otoo, E.J., Shoshani, A.: Optimizing bitmap indices with efficient compression. ACM Trans. Database Syst. 31(1), 1–38 (2006)CrossRefGoogle Scholar