Abstract
Large-scale network monitoring systems require efficient storage and consolidation of measurement data. Relational databases and popular tools such as the Round-Robin Database show their limitations when handling a large number of time series. This is because data access time greatly increases with the cardinality of data and number of measurements. The result is that monitoring systems are forced to store very few metrics at low frequency in order to grant data access within acceptable time boundaries.
This paper describes a novel compressed time series database named tsdb whose goal is to allow large time series to be stored and consolidated in realtime with limited disk space usage. The validation has demonstrated the advantage of tsdb over traditional approaches, and has shown that tsdb is suitable for handling a large number of time series.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Box, G., Jenkins, G.: Time Series Analysis: Forecasting and Control. Prentice Hall PTR (1994) ISBN: 0130607746
StumbleUpon, OpenTSDB: Open Time Series Database, http://opentsdb.net
Oetiker, T.: RRDtool: Round Robin Database Tool, http://oss.oetiker.ch/rrdtool/
Deri, L., et al.: Monitoring Networks Using ntop. In: Proc. of IM 2001 (May 2001)
Vlachos, M., Kozat, S., Yu, P.S.: Optimal Distance Bounds for Fast Search on Compressed Time-series Query Logs. TWEB 4(2) (2010)
Salomon, D.: Data Compression: The Complete Reference. Springer, Heidelberg (2000)
Rice, R.F., Plaunt, R.: Adaptive Variable-Length Coding for Efficient Compression of Spacecraft Television Data. IEEE Transactions on Communications 16(9), 889–897 (1971)
Harizopoulos, S., Liang, V., Abadi, D.J., Madden, S.: Performance Tradeoffs in Read-Optimized Databases. In: Proc. of VLDB (2006)
Anh, V.N., Moffat, A.: Inverted Index Compression Using Word-Aligned Binary Codes. Journal of Information Retrieval 8(1), 151–166 (2005)
Zukowski, M., et al.: Super-Scalar RAMCPU Cache Compression. In: Proc. of International Conference on Data Engineering, ICDE (2006)
Abadi, D., Madden, S., Ferreira, M.: Integrating Compression and Execution in Column-Oriented Database Systems. In: Proc. of 32nd ACM SIGMOD International Conference on Management of Data (2006)
Reinhold, L.M.: QuickLZ (2011), http://www.quicklz.com
Olson, M., Bostic, K., Seltzer, M.: Berkeley DB. In: Proc. of Usenix Annual Technical Conference (1999)
Mullins, C.S.: Database Administration: The Complete Guide to Practices and Procedures (2002) ISBN: 0-201-741296
Rafiei, D., Mendelzon, A.: Similarity-based queries for time series data. In: Proc. of the ACM SIGMOD (1997)
Brillinger, D.R.: Time Series: Data Analysis and Theory. Society for Industrial and Applied Mathematics (2001) ISBN-10: 0898715016
Krishnamurthy, S., et al.: TelegraphCQ: An Architectural Status Report. IEEE Data Engineering Bulletin 26(1) (March 2003)
Zhao, X.: High Performance Algorithms for Multiple Streaming Time Series, Ph.D. Dissertation, New York University (2006)
Shieh, J., Keogh, E.: iSAX: Indexing and Mining Terabyte Sized Time Series. In: Proc. of ACM SIGKDD (2008)
Tiwari: Professional NoSQL. John Wiley and Sons (2011) ISBN: 047094224X
Zaitsev, P.: Why MySQL could be slow with large tables? (June 2006), http://www.mysqlperformanceblog.com
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 IFIP International Federation for Information Processing
About this paper
Cite this paper
Deri, L., Mainardi, S., Fusco, F. (2012). tsdb: A Compressed Database for Time Series. In: Pescapè, A., Salgarelli, L., Dimitropoulos, X. (eds) Traffic Monitoring and Analysis. TMA 2012. Lecture Notes in Computer Science, vol 7189. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28534-9_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-28534-9_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28533-2
Online ISBN: 978-3-642-28534-9
eBook Packages: Computer ScienceComputer Science (R0)