Skip to main content
SpringerLink
Log in

Opportunities for optimism in contended main-memory multicore transactions

  • Special Issue Paper
  • Published:
The VLDB Journal Aims and scope Submit manuscript

Abstract

Main-memory multicore transactional systems have achieved excellent performance using single-version optimistic concurrency control (OCC), especially on uncontended workloads. Nevertheless, systems based on other concurrency control protocols, such as hybrid OCC/ locking and variations on multiversion concurrency control (MVCC), are reported to outperform the best OCC systems, especially with increasing contention. This paper shows that implementation choices unrelated to concurrency control can explain some of these performance differences. Our evaluation shows the strengths and weaknesses of OCC, MVCC, and TicToc concurrency control under varying workloads and contention levels, and the importance of several implementation choices called basis factors. Given sensible basis factor choices, OCC performance does not collapse on high-contention TPC-C. We also present two optimization techniques, deferred updates and timestamp splitting, that can dramatically improve the high-contention performance of both OCC and MVCC. These techniques are known, but we apply them in a new context and highlight their potency: when combined, they lead to performance gains of \(4.74\times \) for MVCC and \(5.01\times \) for OCC in a TPC-C workload.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. The distinction between commit and read bounds matters only for MSTO, which executes declared read-only transactions at timestamp \( wts _ g \). OSTO and TSTO never execute transactions in the past and could alternatively free objects sooner, when \( wts _ g > fts \).

  2. Since Cicada’s basis factor choices are good, we doubt changes in basis factors would dramatically alter its performance.

  3. MVCC can remove some scalability bottlenecks involving read-only transactions since declared read-only transactions can always commit.

  4. This constrains the use of deferred updates for operations that can overflow. For example, an increment operation can be encoded as a deferred update only if the increment can be performed for any value without error. This is possible for bignums, floating-point numbers, and fixed-size integers with modular or clipped arithmetic, as well as for values that are constrained by external factors, but it is not typically true for fixed-size integers with signaling overflow.

  5. It is important to preserve \(w. updater \) in case of concurrent access by other transactions.

References

  1. Abramson, N.: The Aloha system: Another alternative for computer communications. In: Proceedings of the November 17–19, 1970. In: Fall Joint Computer Conference, AFIPS ’70 (Fall), pp. 281–285. ACM (1970)

  2. Agrawal, R., Carey, M.J., Livny, M.: Concurrency control performance modeling: alternatives and implications. ACM Trans. Database Syst. (TODS) 12(4), 609–654 (1987)

    Article  Google Scholar 

  3. Badrinath, B., Ramamritham, K.: Semantics-based concurrency control: beyond commutativity. ACM Trans. Database Syst. (TODS) 17(1), 163–199 (1992)

    Article  Google Scholar 

  4. Bernstein, P.A., Goodman, N.: Multiversion concurrency control-theory and algorithms. ACM Trans. Database Syst. (TODS) 8(4), 465–483 (1983)

    Article  MathSciNet  Google Scholar 

  5. Cachopo, J., Rito-Silva, A.: Versioned boxes as the basis for memory transactions. Sci. Comp. Programm. 63(2), 172–185 (2006)

    Article  MathSciNet  Google Scholar 

  6. Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with YCSB. In: Proceedings of the 1st ACM Symposium on Cloud Computing, SOCC ’10, pp. 143–154. ACM (2010)

  7. Diaconu, C., Freedman, C., Ismert, E., Larson, P.A., Mittal, P., Stonecipher, R., Verma, N., Zwilling, M.: Hekaton: SQL Server’s memory-optimized OLTP engine. In: Proceedings of the 2013 International Conference on Management of Data, SIGMOD ’13, pp. 1243–1254. ACM (2013)

  8. Dice, D., Shalev, O., Shavit, N.: Transactional locking II. In: Proceedings of the 20th International Symposium on Distributed Computing, DISC ’06, pp. 194–208. Springer (2006)

  9. Difallah, D.E., Pavlo, A., Curino, C., Cudre-Mauroux, P.: OLTP-bench: an extensible testbed for benchmarking relational databases. PVLDB 7(4), 277–288 (2013)

    Google Scholar 

  10. Ding, B., Kot, L., Gehrke, J.: Improving optimistic concurrency control through transaction batching and operation reordering. PVLDB 12(2), 169–182 (2018)

    Google Scholar 

  11. Dragojević, A., Guerraoui, R., Kapalka, M.: Stretching transactional memory. In: Proceedings of the 30th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’09, pp. 155–165. ACM (2009)

  12. Dragojević, A., Narayanan, D., Hodson, O., Castro, M.: FaRM: Fast remote memory. In: Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation, NSDI ’14, pp. 401–414. ACM (2014)

  13. Durner, D., Leis, V., Neumann, T.: On the impact of memory allocation on high-performance query processing. In: Proceedings of the 15th International Workshop on Data Management on New Hardware, DaMoN ’19. ACM (2019). https://doi.org/10.1145/3329785.3329918

  14. Faleiro, J.M., Abadi, D.J.: Rethinking serializable multiversion concurrency control. PVLDB 8(11), 1190–1201 (2015)

    Google Scholar 

  15. Fernandes, S., Cachopo, J.: A scalable and efficient commit algorithm for the JVSTM. In: Proceedings of the 5th ACM SIGPLAN Workshop on Transactional Computing (2010)

  16. Guerraoui, R., Herlihy, M., Pochon, B.: Toward a theory of transactional contention managers. In: Proceedings of the 24th annual ACM Symposium on Principles of Distributed Computing, PODC ’05, pp. 258–264. ACM (2005)

  17. Held, G., Stonebraker, M., Wong, E.: INGRES: a relational data base system. In: Proceedings of the May 19–22, 1975, national computer conference and exposition, pp. 409–416. ACM (1975)

  18. Héman, S., Zukowski, M., Nes, N.J., Sidirourgos, L., Boncz, P.: Positional update handling in column stores. In: Proceedings of the 2010 International Conference on Management of Data, SIGMOD ’10, pp. 543–554. ACM (2010)

  19. Herlihy, M., Koskinen, E.: Transactional boosting: A methodology for highly-concurrent transactional objects. In: Proceedings of the 13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP ’08, pp. 207–216. ACM (2008)

  20. Herlihy, M., Moss, J.E.B.: Transactional memory: Architectural support for lock-free data structures. In: Proceedings of the 20th Annual International Symposium on Computer Architecture, ISCA ’93, pp. 289–300. ACM, New York, NY, USA (1993). https://doi.org/10.1145/165123.165164

  21. Herman, N., Inala, J.P., Huang, Y., Tsai, L., Kohler, E., Liskov, B., Shrira, L.: Type-aware transactions for faster concurrent code. In: Proceedings of the 11th European Conference on Computer Systems, EuroSys ’16. ACM (2016)

  22. Huang, Y., Qian, W., Kohler, E., Liskov, B., Shrira, L.: Opportunities for optimism in contended main-memory multicore transactions. PVLDB 13(5), 629–642 (2020). https://doi.org/10.14778/3377369.3377373

    Article  Google Scholar 

  23. Jannen, W., Yuan, J., Zhan, Y., Akshintala, A., Esmet, J., Jiao, Y., Mittal, A., Pandey, P., Reddy, P., Walsh, L., et al.: BetrFS: A right-optimized write-optimized file system. In: 13th USENIX Conference on File and Storage Technologies, FAST ’15, pp. 301–315. ACM (2015)

  24. Kallman, R., Kimura, H., Natkins, J., Pavlo, A., Rasin, A., Zdonik, S., Jones, E.P.C., Madden, S., Stonebraker, M., Zhang, Y., Hugg, J., Abadi, D.J.: A high-performance, distributed main memory transaction processing system. PVLDB 1(2), 1496–1499 (2008). https://doi.org/10.14778/1454159.1454211

    Article  Google Scholar 

  25. Kim, K., Wang, T., Johnson, R., Pandis, I.: ERMIA: Fast memory-optimized database system for heterogeneous workloads. In: Proceedings of the 2016 International Conference on Management of Data, SIGMOD ’16, pp. 1675–1687. ACM (2016)

  26. Kimura, H.: FOEDUS: OLTP engine for a thousand cores and NVRAM. In: Proceedings of the 2015 International Conference on Management of Data, SIGMOD ’15, pp. 691–706. ACM (2015)

  27. Korth, H.F.: Locking primitives in a database system. J. ACM (JACM) 30(1), 55–79 (1983)

    Article  MathSciNet  Google Scholar 

  28. Kung, H.T., Robinson, J.T.: On optimistic methods for concurrency control. ACM Trans. Database Syst. (TODS) 6(2), 213–226 (1981)

    Article  Google Scholar 

  29. Lamb, A., Fuller, M., Varadarajan, R., Tran, N., Vandiver, B., Doshi, L., Bear, C.: The Vertica analytic database: C-Store 7 years later. PVLDB 5(12), 1790–1801 (2012)

    Google Scholar 

  30. Leis, V., Kemper, A., Neumann, T.: Exploiting hardware transactional memory in main-memory databases. In: IEEE 30th International Conference on Data Engineering, Chicago, ICDE 2014, IL, USA, March 31–April 4, 2014, pp. 580–591 (2014). https://doi.org/10.1109/ICDE.2014.6816683

  31. Lim, H.: Line comment in experiment script (run_exp.py). Available at https://github.com/efficient/cicada-exp-sigmod2017/blob/5a4db37750d1dc787f71f22b425ace82a18f6011/run_exp.py#L859 (2017)

  32. Lim, H., Kaminsky, M., Andersen, D.G.: Cicada: Dependably fast multi-core in-memory transactions. In: Proceedings of the 2017 International Conference on Management of Data, SIGMOD ’17, pp. 21–35. ACM (2017)

  33. Maabreh, K.S., Al-Hamami, A.: Increasing database concurrency control based on attribute level locking. In: 2008 International Conference on Electronic Design, pp. 1–4. IEEE (2008)

  34. Mao, Y., Kohler, E., Morris, R.T.: Cache craftiness for fast multicore key-value storage. In: Proceedings of the 7th European Conference on Computer Systems, EuroSys ’12, pp. 183–196. ACM (2012)

  35. McKenney, P.E., Boyd-Wickizer, S.: RCU usage in the Linux kernel: one decade later. Tech. rep. (2012)

  36. Mellor-Crummey, J.M., Scott, M.L.: Algorithms for scalable synchronization on shared-memory multiprocessors. ACM Trans. Comp. Syst. (TOCS) 9(1), 21–65 (1991)

    Article  Google Scholar 

  37. Mu, S., Angel, S., Shasha, D.: Deferred runtime pipelining for contentious multicore software transactions. In: Proceedings of the 14th European Conference on Computer Systems, EuroSys ’19, pp. 40:1–40:16. ACM (2019)

  38. Narula, N., Cutler, C., Kohler, E., Morris, R.: Phase reconciliation for contended in-memory transactions. In: Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation, OSDI ’14, pp. 511–524. ACM (2014)

  39. Navathe, S., Ceri, S., Wiederhold, G., Dou, J.: Vertical partitioning algorithms for database design. ACM Trans. Database Syst. (TODS) 9(4), 680–710 (1984)

    Article  Google Scholar 

  40. OW2 Consortium: RUBiS. Available at https://rubis.ow2.org/

  41. Rampant Pixels: rpmalloc—rampant pixels memory allocator. Available at https://github.com/rampantpixels/rpmalloc (2019)

  42. Reed, D.P.: Naming and synchronization in a decentralized computer system. Ph.D. thesis, Massachusetts Institute of Technology (1978)

  43. Schwarz, P.M., Spector, A.Z.: Synchronizing shared abstract types. ACM Trans. Comput. Syst. 2(3), 223–250 (1984)

  44. Shapiro, M., Preguiça, N., Baquero, C., Zawirski, M.: Conflict-free replicated data types. In: Symposium on Self-Stabilizing Systems, pp. 386–400. Springer (2011)

  45. Shasha, D., Llirbat, F., Simon, E., Valduriez, P.: Transaction chopping: algorithms and performance studies. ACM Trans. Database Syst. (TODS) 20(3), 325–363 (1995)

    Article  Google Scholar 

  46. Shavit, N., Touitou, D.: Software transactional memory. In: Proceedings of the Fourteenth Annual ACM Symposium on Principles of Distributed Computing, Ottawa, Ontario, Canada, August 20–23, 1995, pp. 204–213. ACM (1995). https://doi.org/10.1145/224964.224987

  47. Spiegelman, A., Golan-Gueta, G., Keidar, I.: Transactional data structure libraries. In: Proceedings the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’16. ACM (2016)

  48. Stonebraker, M., Abadi, D.J., Batkin, A., Chen, X., Cherniack, M., Ferreira, M., Lau, E., Lin, A., Madden, S., ONeil, E., et al.: C-Store: a column-oriented DBMS. PVLDB pp. 553–564 (2005)

  49. Tang, D., Jiang, H., Elmore, A.J.: Adaptive concurrency control: Despite the looking glass, one concurrency control does not fit all. In: The 8th Biennial Conference on Innovative Data Systems Research, CIDR ’17 (2017)

  50. Transaction Processing Performance Council: TPC benchmark C. Available at http://www.tpc.org/tpcc/

  51. Transaction Processing Performance Council: TPC benchmark C standard specification, revision 5.11. Available at http://www.tpc.org/tpc_documents_current_versions/pdf/tpc-c_v5.11.0.pdf (2010)

  52. Tu, S., Zheng, W., Kohler, E., Liskov, B., Madden, S.: Speedy transactions in multicore in-memory databases. In: Proceedings of the 24th ACM Symposium on Operating Systems Principles, SOSP ’13, pp. 18–32. ACM (2013)

  53. Wang, T., Kimura, H.: Mostly-optimistic concurrency control for highly contended dynamic workloads on a thousand cores. PVLDB 10(2), 49–60 (2016)

    Google Scholar 

  54. Wang, Z., Mu, S., Cui, Y., Yi, H., Chen, H., Li, J.: Scaling multicore databases via constrained parallel execution. In: Proceedings of the 2016 International Conference on Management of Data, SIGMOD ’16, pp. 1643–1658. ACM (2016)

  55. Wang, Z., Qian, H., Li, J., Chen, H.: Using restricted transactional memory to build a scalable in-memory database. In: Proceedings of the 9th European Conference on Computer Systems, EuroSys ’14, pp. 26:1–26:15. ACM (2014)

  56. Wei, X., Shi, J., Chen, Y., Chen, R., Chen, H.: Fast in-memory transaction processing using RDMA and HTM. In: Proceedings of the 25th Symposium on Operating Systems Principles, SOSP ’15, pp. 87–104. ACM (2015). https://doi.org/10.1145/2815400.2815419

  57. Weihl, W.E.: Commutativity-based concurrency control for abstract data types. IEEE Trans. Comp. 37(12), 1488–1505 (1988)

    Article  MathSciNet  Google Scholar 

  58. Wu, Y., Arulraj, J., Lin, J., Xian, R., Pavlo, A.: An empirical evaluation of in-memory multi-version concurrency control. PVLDB 10(7), 781–792 (2017)

    Google Scholar 

  59. Yu, X., Bezerra, G., Pavlo, A., Devadas, S., Stonebraker, M.: Staring into the abyss: an evaluation of concurrency control with one thousand cores. PVLDB 8(3), 209–220 (2014)

    Google Scholar 

  60. Yu, X., Pavlo, A., Sanchez, D., Devadas, S.: TicToc: Time traveling optimistic concurrency control. In: Proceedings of the 2016 International Conference on Management of Data, SIGMOD ’16, pp. 1629–1642. ACM (2016)

  61. Zheng, W., Tu, S., Kohler, E., Liskov, B.: Fast databases with fast durability and recovery through multicore parallelism. In: Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation, OSDI ’14, pp. 465–477. ACM (2014)

Download references

Acknowledgements

Part of the work on basis factors was presented by Yihe Huang at the Student Research Competition at the 27th ACM Symposium on Operating Systems Principles (SRC @ SOSP 2019). We also thank the the AWS Cloud Credits for Research Program for providing us compute infrastructure. This work was funded through NSF awards CNS-1704376, CNS-1513416, CNS-1513447, and CNS-1513471. We’re grateful to Stratos Idreos, Andy Pavlo, and Peter Alvaro for thoughtful comments on earlier drafts. Thanks also to anonymous reviewers of the work.

Author information

Authors and Affiliations

  1. Harvard University, Cambridge, MA, USA

    Yihe Huang, William Qian & Eddie Kohler

  2. MIT, Cambridge, MA, USA

    Barbara Liskov

  3. Brandeis University, Waltham, MA, USA

    Liuba Shrira

Authors
  1. Yihe Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. William Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eddie Kohler
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Barbara Liskov
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Liuba Shrira
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eddie Kohler.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, Y., Qian, W., Kohler, E. et al. Opportunities for optimism in contended main-memory multicore transactions. The VLDB Journal 31, 1239–1261 (2022). https://doi.org/10.1007/s00778-021-00719-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00778-021-00719-9

Search

Navigation