Advertisement

UniBench: A Benchmark for Multi-model Database Management Systems

  • Chao Zhang
  • Jiaheng LuEmail author
  • Pengfei Xu
  • Yuxing Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11135)

Abstract

Unlike traditional database management systems which are organized around a single data model, a multi-model database (MMDB) utilizes a single, integrated back-end to support multiple data models, such as document, graph, relational, and key-value. As more and more platforms are proposed to deal with multi-model data, it becomes crucial to establish a benchmark for evaluating the performance and usability of MMDBs. Previous benchmarks, however, are inadequate for such scenario because they lack a comprehensive consideration for multiple models of data. In this paper, we present a benchmark, called UniBench, with the goal of facilitating a holistic and rigorous evaluation of MMDBs. UniBench consists of a mixed data model, a synthetic multi-model data generator, and a set of core workloads. Specifically, the data model simulates an emerging application: Social Commerce, a Web-based application combining E-commerce and social media. The data generator provides diverse data format including JSON, XML, key-value, tabular, and graph. The workloads are comprised of a set of multi-model queries and transactions, aiming to cover essential aspects of multi-model data management. We implemented all workloads on ArangoDB and OrientDB to illustrate the feasibility of our proposed benchmarking system and show the learned lessons through the evaluation of these two multi-model databases. The source code and data of this benchmark can be downloaded at http://udbms.cs.helsinki.fi/bench/.

Notes

Acknowledgment

This work is partially supported by Academy of Finland (310321), China Scholarship (CSC) and CIMO Fellowship.

References

  1. 1.
    ArangoDB: Multi-model NoSQL database (2018). https://www.arangodb.com/
  2. 2.
    Carey, M.J., DeWitt, D.J., Naughton, J.F.: The 007 benchmark. In: ACM SIGMOD, pp. 12–21 (1993)CrossRefGoogle Scholar
  3. 3.
    Chen, Y., et al.: A study of SQL-on-Hadoop systems. In: Big Data Benchmarks, Performance Optimization, and Emerging Hardware, pp. 154–166 (2014)Google Scholar
  4. 4.
    Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with YCSB. In: ACM SoCC, pp. 143–154 (2010)Google Scholar
  5. 5.
    DeWitt, D.J.: The Wisconsin benchmark: past, present, and future. In: The Benchmark Handbook, pp. 119–165 (1991)Google Scholar
  6. 6.
    Erling, O., et al.: The LDBC social network benchmark: interactive workload. In: SIGMOD (2015)Google Scholar
  7. 7.
    Fader, P.S.: Customer-base analysis with discrete-time transaction data. Ph.D. thesis, University of Auckland (2004)Google Scholar
  8. 8.
    Fader, P.S., Hardie, B.G., Lee, K.L.: RFM and CLV: using ISO-value curves for customer base analysis. J. Mark. Res. 42(4), 415–430 (2005)CrossRefGoogle Scholar
  9. 9.
    Feinberg, D., Adrian, M., Heudecker, N., Ronthal, A.M., Palanca, T.: Gartner magic quadrant for operational database management systems, 12 October 2015Google Scholar
  10. 10.
    Ghazal, A., et al.: BigBench: towards an industry standard benchmark for big data analytics. In: ACM SIGMOD (2013)Google Scholar
  11. 11.
    Gupta, S., et al.: Modeling customer lifetime value. J. Serv. Res. 9(2), 139–155 (2006)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Huang, Z., Benyoucef, M.: From e-commerce to social commerce: a close look at design features. ECRA 12, 246–259 (2013)Google Scholar
  13. 13.
    Lehmann, J., et al.: DBPedia - a large-scale, multilingual knowledge base extracted from Wikipedia. Semant. Web 6(2), 167–195 (2015)Google Scholar
  14. 14.
    Leskovec, J., Adamic, L.A., Huberman, B.A.: The dynamics of viral marketing. TWEB 1(1), 5 (2007)CrossRefGoogle Scholar
  15. 15.
    Lu, J.: Benchmarking holistic approaches to XML tree pattern query processing. In: DASFAA Workshops, pp. 170–178 (2010)CrossRefGoogle Scholar
  16. 16.
    Lu, J.: Towards benchmarking multi-model databases. In: CIDR (2017)Google Scholar
  17. 17.
    Lu, J., Holubová, I.: Multi-model data management: what’s new and what’s next? In: EDBT (2017)Google Scholar
  18. 18.
    Oliveira, F.R., del Val Cura, L.M.: Performance evaluation of NoSQL multi-model data stores in polyglot persistence applications. In: IDEAS, pp. 230–235 (2016)Google Scholar
  19. 19.
    OrientDB: Multi-model & graph database. http://orientdb.com/orientdb/
  20. 20.
    Pluciennik, E., Zgorzalek, K.: The Multi-model databases - a review. In: BDAS, pp. 141–152 (2017)Google Scholar
  21. 21.
    Poess, M., Rabl, T., Jacobsen, H., Caufield, B.: TPC-DI: the first industry benchmark for data integration. PVLDB 7(13), 1367–1378 (2014)Google Scholar
  22. 22.
    Prat, A., Averbuch, A.: Benchmark design for navigational pattern matching benchmarking (2015). http://ldbcouncil.org/sites/default/files/LDBC_D3.3.34.pdf
  23. 23.
    Schmidt, A., Waas, F., Kersten, M.L., Carey, M.J., Manolescu, I., Busse, R.: XMark: a benchmark for XML data management. In: VLDB, pp. 974–985 (2002)CrossRefGoogle Scholar
  24. 24.
    Stonebraker, M.: The case for polystores (2015). http://wp.sigmod.org/?p=1629
  25. 25.
    Transaction Processing Performance Council: TPC Benchmark C (Revision 5.11) (2010)Google Scholar
  26. 26.
    Wadsworth, E.: Buy’til you die-a walkthrough (2012)Google Scholar
  27. 27.
    Zhang, K.Z.: Consumer behavior in social commerce: a literature review. Decis. Support Syst. 86, 95–108 (2016)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Chao Zhang
    • 1
  • Jiaheng Lu
    • 1
    Email author
  • Pengfei Xu
    • 1
  • Yuxing Chen
    • 1
  1. 1.Department of Computer ScienceUniversity of HelsinkiHelsinkiFinland

Personalised recommendations