Abstract
The paper deals with indexing of a complex type data stream stored in a database. We present a novel indexing schema and framework referred to as ReTIn (Real-Time Indexing), the objective of which is to allow indexing of complex data arriving as a stream to a database with respect to soft real-time constraints met with some level of confidence for the maximum duration of insert and select operations. The idea of ReTIn is a combination of a sequential access to the most recent data and an index-based access to less recent data stored in the database. The collection of statistics makes balancing of indexed and unindexed parts of the database efficient. We have implemented ReTIn using PostgreSQL DBMS and its GIN index. Experimental results presented in the paper demonstrate some properties and advantages of our approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
B. Babcock, S. Babu, M. Datar, R. Motwani, and J. Widom, “Models and issues in data stream systems,” in Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, Madison, Wisconsin, 2002, pp. 1–16.
T.-W. Kuo and K.-Y. Lam, “Real-time Database Systems: An Overview of System Characteristics and Issues,” in Real-Time Database Systems, 2001, pp. 3–8.
A. Arasu, S. Babu, and J. Widom, “The CQL continuous query language: semantic foundations and query execution,” The VLDB Journal, vol. 15, no. 2, pp. 121–142, 2006.
J. Krämer and B. Seeger, “Semantics and implementation of continuous sliding window queries over data streams,” ACM Trans. Database Syst., vol. 34, no. 1, pp. 1–49, 2009.
M.-J. Hsieh, M.-S. Chen, and P. S. Yu, “Approximate Query Processing in Cube Streams,” IEEE Transactions on Knowledge and Data Engineering, vol. 19, pp. 1557–1570, 2007.
C. C. Aggarwal, Data Streams: Models and Algorithms. Springer US, 2007.
J. Feng, Y. Wang, J. Yao, and T. Watanabe, “Multi-Granularity Aggregation Index for Data Stream,” in Cyberworlds, International Conference on, Los Alamitos, CA, USA, 2008, pp. 767–771.
G. No, S. Servigne, and R. Laurini, “The Po-tree: a Real-time Spatiotemporal Data Indexing Structure,” in Developments in Spatial Data Handling, Springer Berlin Heidelberg, 2005, pp. 259–270.
P. Chmelar, A. Lanik, and J. Mlich, “SUNAR: Surveillance Network Augmented by Retrieval,” in ACIVS 2010, 2010, pp. 155–166.
B. Adelberg, H. Garcia-Molina, and B. Kao, “Applying update streams in a soft real-time database system,” SIGMOD Rec., vol. 24, no. 2, pp. 245–256, 1995.
T. Chiueh and L. Huang, “Efficient Real-Time Index Updates in Text Retrieval Systems,” EXPERIMENTAL COMPUTER SYSTEMS LAB, DEPARTMENT OF COMPUTER SCIENCE, STATE UNIVERSITY OF NEW, 1999.
Q. Zhu, B. Dunkel, N. Soparkar, S. Chen, B. Schiefer, and T. Lai, “A piggyback method to collect statistics for query optimization in database management systems,” in Proceedings of the 1998 conference of the Centre for Advanced Studies on Collaborative research, Toronto, Ontario, Canada, 1998, p. 25.
“Global Surface Summary of the Day - GSOD.” NOAA.
Acknowledgments
This work has been supported by the research project Security-Oriented Research in Information Technology CEZ MSM0021630528, grant VG20102015006 of the Ministry of the Interior of the Czech Republic, the European Regional Development Fund in the IT4Innovations Centre of Excellence (CZ.1.05/1.1.00/02.0070) and with a financial support from the Czech Republic state budget through the Ministry of Industry and Trade.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Chmelar, P., Drozd, M., Sebek, M., Zendulka, J. (2015). Real-Time Indexing of Complex Data Streams. In: Sobh, T., Elleithy, K. (eds) Innovations and Advances in Computing, Informatics, Systems Sciences, Networking and Engineering. Lecture Notes in Electrical Engineering, vol 313. Springer, Cham. https://doi.org/10.1007/978-3-319-06773-5_50
Download citation
DOI: https://doi.org/10.1007/978-3-319-06773-5_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-06772-8
Online ISBN: 978-3-319-06773-5
eBook Packages: EngineeringEngineering (R0)