Abstract
Bitmap indices are widely used in commercial databases for processing complex queries, due to their efficient use of bit-wise operations. Bitmap indices apply natively to relational and linear datasets, with distinct separation of the columns or attributes, but do not perform well on multidimensional array scientific data.
We propose a new method for multidimensional array indexing that considers the spatial component of multidimensional arrays. The hierarchical indexing method is based on sparse n-dimensional trees for dimension partitioning, and bitmap indexing with adaptive binning for attribute partitioning. This indexing performs well on range queries involving both dimension and attribute constraints, as it prunes the search space early. Moreover, the indexing is easily extensible to membership queries.
The indexing method was implemented on top of a state of the art bitmap indexing library Fastbit, using tables partitioned along any subset of the data dimensions. We show that the hierarchical bitmap index outperforms conventional bitmap indexing, where an auxiliary attribute is required for each dimension. Furthermore, the adaptive binning significantly reduces the amount of bins and therefore memory requirements.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Antoshenkov, G.: Byte-aligned bitmap compression. In: Proceedings of the Data Compression Conference. DCC 1995, p. 476. IEEE (1995)
Baumann, P., Dehmel, A., Furtado, P., Ritsch, R., Widmann, N.: The multidimensional database system RasDaMan. In: ACM SIGMOD Record, vol. 27, pp. 575–577. ACM (1998)
Beckmann, N., Kriegel, H.P., Schneider, R., Seeger, B.: The R*-tree: an efficient and robust access method for points and rectangles, vol. 19. ACM (1990)
Chan, C.Y., Ioannidis, Y.E.: Bitmap index design and evaluation. In: ACM SIGMOD Record, vol. 27, pp. 355–366. ACM (1998)
Chan, C., Ioannidis, Y.: An efficient bitmap encoding scheme for selection queries. ACM SIGMOD Record (1999). http://dl.acm.org/citation.cfm?id=304201
Chou, J., et al.: Parallel index and query for large scale data analysis. In: Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, p. 30. ACM (2011)
Gosink, L., Shalf, J., Stockinger, K., Wu, K., Bethel, W.: HDF5-FastQuery: accelerating complex queries on HDF datasets using fast bitmap indices. In: 18th International Conference on Scientific and Statistical Database Management, pp. 149–158. IEEE (2006)
Guttman, A.: R-trees: a dynamic index structure for spatial searching, vol. 14. ACM (1984)
Lawder, J.K., King, P.J.H.: Querying multi-dimensional data indexed using the Hilbert space-filling curve. ACM SIGMOD Rec. 30(1), 19–24 (2001)
Lungu, T., Callahan, P.S.: QuikSCAT science data product user’s manual: overview and geophysical data products. D-18053-Rev A, version 3, p. 91 (2006)
Nagarkar, P., Candan, K.S., Bhat, A.: Compressed spatial hierarchical bitmap (cSHB) indexes for efficiently processing spatial range query workloads. Proc. VLDB Endow. 8(12), 1382–1393 (2015)
Park, K.: A hierarchical binary quadtree index for spatial queries. Wirel. Netw. 25(4), 1913–1929 (2018). https://doi.org/10.1007/s11276-018-1661-z
SeaPAC: Rapidscat level 2b ocean wind vectors in 12.5km slice composites version 1.1. In: NASA Physical Oceanography DAAC (2015). https://doi.org/10.5067/RSX12-L2B11
Sinha, R.R., Winslett, M.: Multi-resolution bitmap indexes for scientific data. ACM Trans. Database Syst. (TODS) 32(3), 16 (2007)
Stockinger, K.: Bitmap indices for speeding up high-dimensional data analysis. In: Hameurlain, A., Cicchetti, R., Traunmüller, R. (eds.) DEXA 2002. LNCS, vol. 2453, pp. 881–890. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-46146-9_87
Stonebraker, M., Brown, P., Zhang, D., Becla, J.: SciDB: a database management system for applications with complex analytics. Computing in Science and Engineering 15(3), 54–62 (2013). https://doi.org/10.1109/MCSE.2013.19
Su, Y., Wang, Y., Agrawal, G.: In-situ bitmaps generation and efficient data analysis based on bitmaps. In: Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing, pp. 61–72. ACM (2015)
Wang, Y., Su, Y., Agrawal, G.: A novel approach for approximate aggregations over arrays. In: Proceedings of the 27th International Conference on Scientific and Statistical Database Management, p. 4. ACM (2015)
Wang, Y., Su, Y., Agrawal, G., Liu, T.: SciSD: novel subgroup discovery over scientific datasets using bitmap indices. In: Proceedings of Ohio State CSE Technical report (2015)
Wu, K., et al.: FastBit: interactively searching massive data. J. Phys. Conf. Seri. 180, 012053 (2009). IOP Publishing
Wu, K., Otoo, E.J., Shoshani, A.: Optimizing bitmap indices with efficient compression. ACM Trans. Database Syst. (TODS) 31(1), 1–38 (2006)
Wu, K.L., Yu, P.S.: Range-based bitmap indexing for high cardinality attributes with skew. In: Proceedings of the Twenty-Second Annual International COMPSAC 1998, pp. 61–66. IEEE (1998)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Krčál, L., Ho, SS., Holub, J. (2022). Hierarchical Bitmap Indexing for Range Queries on Multidimensional Arrays. In: Bhattacharya, A., et al. Database Systems for Advanced Applications. DASFAA 2022. Lecture Notes in Computer Science, vol 13245. Springer, Cham. https://doi.org/10.1007/978-3-031-00123-9_40
Download citation
DOI: https://doi.org/10.1007/978-3-031-00123-9_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-00122-2
Online ISBN: 978-3-031-00123-9
eBook Packages: Computer ScienceComputer Science (R0)