Memory-Efficient Storing of Timestamps for Spatio-Temporal Data Management in Columnar In-Memory Databases

Abstract

Vast amounts of spatio-temporal data are continuously accumulated through the wide distribution of location-acquisition technologies. Concerning the increased performance requirements of spatio-temporal data mining applications, in-memory database systems are used to store and process the data. As DRAM capacities are limited and expensive, the efficient utilization of the available resources is necessary. In contrast to storing the positions of moving objects, there is less focus on optimized storage concepts for the temporal component. However, it has a significant impact on the memory footprint and the overall system performance. Especially for columnar databases, the memory-efficient storing of timestamps is challenging as numerous compression approaches are optimized for contradicting data characteristics (e.g., low number of distinct values, sequences of equal values). In this paper, we present and compare different data layouts for columnar in-memory databases to store timestamps. Additionally, we propose an optimized approach for range queries with standard access ranges that uses multiple columns. We evaluate the memory consumption and performance of different compression techniques for specific access patterns. Based on the results, we introduce a workload-aware heuristic approach for the selection of performance and cost balancing data layouts. Further, we demonstrate that workload-driven optimizations for timestamps can significantly reduce the data footprint and increase the performance of spatio-temporal data management.