Abstract
With the increasing use of sensor and IoT technologies, sensor stream data is generated and consumed at an unprecedented scale. Traditional storage mechanisms represented by relational database systems become more and more difficult to adapt to the store, query, update and other operations of large-scale sensor stream data. This, in turn, has led to the emergence of a new kind of complementary non-relational data store subsumed under the term time series database (TSDB). However, the heterogeneity and diversity of numerous TSDBs impede the well-informed comparison and selection for a given application context. A thorough survey shows that current benchmarks for TSDBs are few and they still need improvement in workload implementation based on real business requirements, data generator based on real-world data and fine-grained performance metrics. How to implement a benchmarking tool for TSDBs according to different tradeoffs in IoT scenarios becomes a key challenge, which will be addressed in this paper. Firstly, we propose a benchmarking platform TS_Store_Test, which integrates five well-known TSDBs using the micro-services mechanism. Meanwhile, we integrated and extend Prometheus to capture the performance metrics in a refined manner. Based on TS_Store_Test, the execution efficiency of some workloads from technical and business perspectives is tested using the real hydrological sensor data. Experimental results demonstrate the usability and scalability of TS_Store_Test, and also show the performance differences of different TSDBs for sensor stream data. Finally, TS_Store_Test is compared with other NoSQL benchmarking suits.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Qin, X., Wang, H., Du, X., Wang, S.: Big data analysis-competition and symbiosis of RDBMS and MapReduce. J. Softw. 23(1), 32–45 (2012)
Dunning, T., Friedman, E.: Time Series Databases-New Ways to Store and Access Data. O’Reilly Media, Sebastopol (2015)
IoTDB Homepage. http://tsfile.cn/index. Accessed 21 Apr 2019
Druid Homepage. http://druid.io/. Accessed 21 Apr 2019
Riak TS Homepage. http://basho.com/products/riak-ts/. Accessed 21 Apr 2019
Davoudian, A., Chen, L., Liu, M.: Survey on NoSQL stores. ACM Comput. Surv. 51(2), 1–43 (2018)
Gessert, F., Wingerath, W., Friedrich, S., Ritter, N.: NoSQL database systems: a survey and decision guidance. Comput. Sci. Res. Dev. 32, 353–365 (2016)
Lourenço, J., Cabral, B., Carreiro, P., Vieira, M., Bernardino, J.: Choosing the right NoSQL database for the job: a quality attribute evaluation. J. Big Data 2, 1–26 (2015)
Han, R., John, L.K., Zhan, J.: Benchmarking big data systems: a review. IEEE Trans. Serv. Comput. 11(3), 580–595 (2018)
Zhou, X., Qin, X., Wang, Q.: Big data benchmarks: state-of-art and trends. J. Comput. Appl. 35(4), 1137–1142 (2015)
Qian, W., Xia, F., Zhou, M., Jin, C., Zhou, A.: Challenges and progress of big data management system benchmarks. Big Data Res. 1, 1–15 (2015)
Gregg, B.: Systems Performance: Enterprise and the Cloud. Prentice Hall, Ann Arbor (2013)
Kai, J.: Research on reliable-oriented adapation on microservice system. Shanghai University, Shanghai (2016)
Prometheus Homepage. https://prometheus.io/. Accessed 21 Apr 2019
Jensen, S.K., Pedersen, T.B., Thomsen, C.: Time series management systems: a survey. IEEE Trans. Knowl. Data Eng. 29(11), 2581–2600 (2017)
Wlodarczyk, T.W.: Overview of time series storage and processing in a cloud environment. In: 4th IEEE International Conference on Cloud Computing Technology and Science, pp. 625–628. IEEE Computer Society, Taipei (2012)
Bader, A., Kopp, O., Michael, F.: Survey and comparison of open source time series databases. In: Mitschang, B., et al. (eds.) BTW 2017. LNI, pp. 249–268. Gesellschaft für Informatik, Bonn (2017)
Gandini, A., Gribaudo, M., Knottenbelt, W.J., Osman, R., Piazzolla, P.: Performance evaluation of NoSQL databases. In: Horváth, A., Wolter, K. (eds.) EPEW 2014. LNCS, vol. 8721, pp. 16–29. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10885-8_2
Matallah, H., Belalem, G., Bouamrane, K.: Experimental comparative study of NoSQL databases: HBase versus MongoDB by YCSB. Comput. Syst. Sci. Eng. 32(4), 307–317 (2017)
Patil, S., et al.: YCSB++: benchmarking and performance de-bugging advanced features in scalable table stores. In: SOCC 2011, Article No. 9. ACM, Cascais (2011)
Alabdulkarim, Y., Barahmand, S., Ghandeharizadeh, S.: BG: a scalable benchmark for interactive social networking actions. Future Gener. Comput. Syst. 85, 29–38 (2018)
Ferdman, M., et al.: Clearing the clouds: a study of emerging scale-out workloads on modern hardware. In: International Conference Architectural Support for Programming Languages and Operating Systems, ASPLOS 2012, pp. 37–48. ACM, London (2012)
Zhan, J.F., et al.: BigDataBench: an open-source big data benchmark suite. Chin. J. Comput. 39(1), 196–210 (2016)
MongoDB Homepage. https://www.mongodb.com/. Accessed 21 Apr 2019
HBase Homepage. https://hbase.apache.org/. Accessed 21 Apr 2019
Pungilă, C., Fortiş, T., Aritoni, O.: Benchmarking database systems for the requirements of sensor readings. IETE Tech. Rev. 26(5), 342–349 (2009)
Shah, S.M., Wei, R., Kolovos, D.S., Rose, L.M., Paige, R.F., Barmpis, K.: A framework to benchmark NoSQL data stores for large-scale model persistence. In: Dingel, J., Schulte, W., Ramos, I., Abrahão, S., Insfran, E. (eds.) MODELS 2014. LNCS, vol. 8767, pp. 586–601. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11653-2_36
QuasarDB Homepage. https://www.quasardb.net/. Accessed 21 Apr 2019
Timescale Homepage. https://www.timescale.com/. Accessed 21 Apr 2019
InfluxData Homepage. https://www.influxdata.com/. Accessed 21 Apr 2019
AXIBASE Homepage. https://axibase.com/products/axibase-time-series-database/. Accessed 21 Apr 2019
OpenTSDB Homepage. http://opentsdb.net/. Accessed 21 Apr 2019
kdb+ Homepage. https://kx.com/. Accessed 21 Apr 2019
KairosDB Homepage. http://kairosdb.github.io/. Accessed 21 Apr 2019
SiteWhere Homepage. https://github.com/sitewhere/sitewhere. Accessed 21 Apr 2019
Dunning, T., Friedman, E.: Streaming Architecture: New Designs Using Apache Kafka and MapR Streams. O’Reilly Media, Sebastopol (2016)
Lu, R., Wu, G., Xie, B., Hu, J.: Stream bench: towards benchmarking modern distributed stream computing frameworks. In: IEEE/ACM 7th International Conference of Utility and Cloud Computing, pp. 69–78. IEEE, London (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Ye, F., Liu, Z., Zhu, S., Zhang, P., Chen, Y. (2020). Research of Benchmarking and Selection for TSDB. In: Wen, S., Zomaya, A., Yang, L.T. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2019. Lecture Notes in Computer Science(), vol 11945. Springer, Cham. https://doi.org/10.1007/978-3-030-38961-1_54
Download citation
DOI: https://doi.org/10.1007/978-3-030-38961-1_54
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-38960-4
Online ISBN: 978-3-030-38961-1
eBook Packages: Computer ScienceComputer Science (R0)