Skip to main content
SpringerLink
Log in

Efficiently making (almost) any concurrency control mechanism serializable

  • Regular Paper
  • Published:
The VLDB Journal Aims and scope Submit manuscript

An Erratum to this article was published on 23 June 2017

This article has been updated

Abstract

Concurrency control (CC) algorithms must trade off strictness for performance. In particular, serializable CC schemes generally pay higher cost to prevent anomalies, both in runtime overhead such as the maintenance of lock tables and in efforts wasted by aborting transactions. We propose the serial safety net (SSN), a serializability-enforcing certifier which can be applied on top of various CC schemes that offer higher performance but admit anomalies, such as snapshot isolation and read committed. The underlying CC mechanism retains control of scheduling and transactional accesses, while SSN tracks the resulting dependencies. At commit time, SSN performs a validation test by examining only direct dependencies of the committing transaction to determine whether it can commit safely or must abort to avoid a potential dependency cycle. SSN performs robustly for a variety of workloads. It maintains the characteristics of the underlying CC without biasing toward a certain type of transactions, though the underlying CC scheme might. Besides traditional OLTP workloads, SSN also efficiently handles heterogeneous workloads which include a significant portion of long, read-mostly transactions. SSN can avoid tracking the vast majority of reads (thus reducing the overhead of serializability certification) and still produce serializable executions with little overhead. The dependency tracking and validation tests can be done efficiently, fully parallel and latch-free, for multi-version systems on modern hardware with substantial core count and large main memory. We demonstrate the efficiency, accuracy and robustness of SSN using extensive simulations and an implementation that overlays snapshot isolation in ERMIA, a memory-optimized OLTP engine that supports multiple CC schemes. Evaluation results confirm that SSN is a promising approach to serializability with robust performance and low overhead for various workloads.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Change history

  • 23 June 2017

    An erratum to this article has been published.

Notes

  1. Phantom protection, as we will discuss later in Sect. 6, is another necessary, but largely orthogonal issue.

  2. There are many formulations such as [4] and [41], and the presentation with this form of dependency definition is in [1].

  3. We ignore self loops, since our model excludes them. In reality, transactions should be allowed to read their own writes.

  4. SSI allows (c) if the leftmost transaction is read-only and sufficiently old, but rejects (f) if a (harmless) forward anti-dependency edge joins T with its predecessor.

  5. For example, PostgreSQL maintains the equivalent of v.cstamp and v.prev. In each version, SSN takes an extra of 16 bytes for sstamp and pstamp, assuming 8-byte stamps. For one million versions, SSN needs in total less than 16 MB of extra memory. This is likely tolerable in today’s systems with abundant memory and storage.

  6. Storing TID (before the overwriter finalizes) in the overwritten version’s sstamp also eases the removal of updated versions from the read set (see the end of Sect. 4.1.3). When iterating the read set, the updater T simply skips versions whose sstamp points to T’s own TID.

  7. There exists other viable approaches other than spinning for U’s sstamp to become stable; e.g., T could deduce U’s low watermark by helping with U’s pre-commit phase by iterating over U’s read set. In experiments, we use spinning due to its simplicity and lightweightness.

  8. Code and scripts available at http://github.com/ermia-db/ssn-simulator.

  9. For example, consider \(T_1 \xleftarrow { w{:}r } {} T_n \xleftarrow { r{:}w } {} \cdots T_i \cdots \xleftarrow { r{:}w } {} T_1\), where each \(T_i\) begins just before \(T_{i-1}\) commits.

  10. Code available at http://github.com/ermia-db/ermia.

  11. Our current implementation accepts a user-defined, workload-specific threshold. Self-tuning it as workload changes is future work.

References

  1. Adya, A.: Weak Consistency: A Generalized Theory and Optimistic Implementations for Distributed Transactions. Ph.D. thesis, MIT (1999)

  2. Alomari, M., Fekete, A., Röhm, U.: Performance of program modification techniques that ensure serializable executions with snapshot isolation DBMS. Inf. Syst. 40, 84–101 (2014)

    Article  Google Scholar 

  3. Berenson, H., Bernstein, P.A., Gray, J., Melton, J., O’Neil, E.J., O’Neil, P.E.: A critique of ANSI SQL isolation levels. In: SIGMOD, pp. 1–10 (1995)

  4. Bernstein, P.A., Shipman, D.W., Wong, W.S.: Formal aspects of serializability in database concurrency control. IEEE Trans. Softw. Eng. 5(3), 203–216 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  5. Cahill, M.J., Röhm, U., Fekete, A.D.: Serializable isolation for snapshot databases. ACM TODS 34(4), 20:1–20:42 (2009)

    Article  Google Scholar 

  6. Casanova, M., Bernstein, P.: General purpose schedulers for database systems. Acta Inf. 14, 195–220 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  7. Chen, S., Ailamaki, A., Athanassoulis, M., Gibbons, P.B., Johnson, R., Pandis, I., Stoica, R.: TPC-E vs. TPC-C: characterizing the new TPC-E benchmark via an I/O comparison study. SIGMOD Rec. 39(3), 5–10 (2011)

    Article  Google Scholar 

  8. DeBrabant, J., Pavlo, A., Tu, S., Stonebraker, M., Zdonik, S.: Anti-caching: a new approach to database management system architecture. PVLDB 6(14), 1942–1953 (2013)

    Google Scholar 

  9. Eswaran, K.P., Gray, J., Lorie, R.A., Traiger, I.L.: The notions of consistency and predicate locks in a database system. Commun. ACM 19(11), 624–633 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  10. Faleiro, J.M., Abadi, D.J.: Rethinking serializable multiversion concurrency control. PVLDB 8(11), 1190–1201 (2015)

    Google Scholar 

  11. Fekete, A., Liarokapis, D., O’Neil, E., O’Neil, P., Shasha, D.: Making snapshot isolation serializable. ACM TODS 30(2), 492–528 (2005)

    Article  Google Scholar 

  12. Fekete, A., O’Neil, E., O’Neil, P.: A read-only transaction anomaly under snapshot isolation. SIGMOD Rec. 33(3), 12–14 (2004)

    Article  Google Scholar 

  13. Graefe, G.: Hierarchical locking in b-tree indexes. In: BTW, pp. 18–42 (2007)

  14. Graefe, G.: A survey of B-tree locking techniques. ACM TODS 35(3), 16:1–16:26 (2010)

    Article  Google Scholar 

  15. Gray, J.: Notes on Data Base Operating Systems. Advanced Course: Operating Systems. Springer, Berlin (1978)

  16. Gray, J., Lorie, R.A., Putzolu, G.R., Traiger, I.L.: Granularity of locks and degrees of consistency in a shared data base. In: IFIP Working Conference on Modelling in DBMS (1976)

  17. Hadzilacos, T.: Serialization graph algorithms for multiversion concurrency control. In: PODS, pp. 135–141 (1988)

  18. Intel Corporation: Intel 64 and ia-32 architectures software developer’s manual (2015)

  19. Johnson, R., Kim, K., Wang, T., Pandis, I.: Robust concurrency control in main-memory DBMS: what main memory giveth, the application taketh away. In: IMDM, pp. 57–59 (2014)

  20. Johnson, R., Pandis, I., Hardavellas, N., Ailamaki, A., Falsafi, B.: Shore-MT: a scalable storage manager for the multicore era. In: EDBT (2009)

  21. Jordan, J.R., Banerjee, J., Batman, R.B.: Precision locks. In: SIGMOD, pp. 143–147 (1981)

  22. Jorwekar, S., Fekete, A., Ramamritham, K., Sudarshan, S.: Automating the detection of snapshot isolation anomalies. In: PVLDB, pp. 1263–1274 (2007)

  23. Jung, H., Han, H., Fekete, A., Röhm, U., Yeom, H.Y.: Performance of serializable snapshot isolation on multicore servers. Database Syst. Adv. Appl. LNCS 7826, 416–430 (2013)

    Google Scholar 

  24. Kemper, A., Neumann, T.: HyPer: A hybrid OLTP&OLAP main memory database system based on virtual memory snapshots. In: ICDE, pp. 195–206 (2011)

  25. Kim, K., Wang, T., Johnson, R., Pandis, I.: ERMIA: fast memory-optimized database system for heterogeneous workloads. In: SIGMOD (2016)

  26. Kimura, H.: FOEDUS: OLTP engine for a thousand cores and NVRAM. In: SIGMOD, pp. 691–706 (2015)

  27. Kung, H.T., Robinson, J.T.: On optimistic methods for concurrency control. ACM TODS 6(2), 213–226 (1981)

    Article  Google Scholar 

  28. Larson, P.A., Blanas, S., Diaconu, C., Freedman, C., Patel, J.M., Zwilling, M.: High-performance concurrency control mechanisms for main-memory databases. In: PVLDB (2011)

  29. Lee, V.C., Lam, K.W.: Conflict free transaction scheduling using serialization graph for real-time databases. J. Syst. Softw. 55(1), 57–65 (2000)

    Article  Google Scholar 

  30. Levandoski, J., Lomet, D., Sengupta, S., Stutsman, R., Wang, R.: Multi-version range concurrency control in Deuteronomy. In: PVLDB, pp. 2146–2157 (2015)

  31. Levandoski, J.J., Lomet, D., Mokbel, M.F., Zhao, K.K.: Deuteronomy: transaction support for cloud data. In: CIDR (2011)

  32. Lomet, D., Fekete, A., Wang, R., Ward, P.: Multi-version concurrency via timestamp range conflict management. In: ICDE, pp. 714–725 (2012)

  33. Lomet, D.B.: Key range locking strategies for improved concurrency. In: PVLDB, pp. 655–664 (1993)

  34. Lomet, D.B., Sengupta, S., Levandoski, J.J.: The bw-tree: a b-tree for new hardware platforms. In: ICDE, pp. 302–313 (2013)

  35. Mao, Y., Kohler, E., Morris, R.T.: Cache craftiness for fast multicore key-value storage. In: EuroSys, pp. 183–196 (2012)

  36. Microsoft: Hot it works: SQL Server Super-latching/Sub-latches. Official Team Web Log for Microsoft Customer and Support (CSS) SQL Support (2009). http://blogs.msdn.microsoft.com/psssql/2009/01/28/hot-it-works-sql-server-superlatching-sub-latches/

  37. Mohan, C.: ARIES/KVL: A key-value locking method for concurrency control of multiaction transactions operating on B-Tree indexes. In: PVLDB, pp. 392–405 (1990)

  38. Neumann, T., Mühlbauer, T., Kemper, A.: Fast serializable multi-version concurrency control for main-memory database systems. In: SIGMOD, pp. 677–689 (2015)

  39. PostgreSQL Global Development Group: Chapter 13. Concurrency Control. PostgreSQL 9.4.1 Documentation (2015)

  40. Pandis, I., Johnson, R., Hardavellas, N., Ailamaki, A.: Data-oriented transaction execution. PVLDB 3(1), 928–939 (2010)

    Google Scholar 

  41. Papadimitriou, C.H.: The serializability of concurrent database updates. J. ACM 26(4), 631–653 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  42. Ports, D.R.K., Grittner, K.: Serializable snapshot isolation in PostgreSQL. PVLDB 5(12), 1850–1861 (2012)

    Google Scholar 

  43. Ren, K., Thomson, A., Abadi, D.J.: Lightweight locking for main memory database systems. In: PVLDB, 145–156 (2013)

  44. Revilak, S., O’Neil, P.E., O’Neil, E.J.: Precisely serializable snapshot isolation (PSSI). In: ICDE (2011)

  45. Sadoghi, M., Ross, K.A., Canim, M., Bhattacharjee, B.: Making updates disk-I/O friendly using SSDs. PVLDB 6(11), 997–1008 (2013)

    Google Scholar 

  46. Thomasian, A.: A performance comparison of locking methods with limited wait depth. IEEE Trans. Knowl. Data Eng. 9(3), 421–434 (1997)

    Article  Google Scholar 

  47. Transaction Processing Performance Council: Standard specification. TPC Benchmark E Version 1.12.0 (2010)

  48. Transaction Processing Performance Council: Standard Specification. TPC Benchmark C Version 5.11 (2010)

  49. Tu, S., Zheng, W., Kohler, E., Liskov, B., Madden, S.: Speedy transactions in multicore in-memory databases. In: SOSP, pp. 18–32 (2013)

  50. Wang, T., Johnson, R., Fekete, A., Pandis, I.: The serial safety net: efficient concurrency control on modern hardware. In: DaMoN, pp. 8:1–8:8 (2015)

  51. Yuan, Y., Wang, K., Lee, R., Ding, X., Xing, J., Blanas, S., Zhang, X.: BCC: Reducing false aborts in optimistic concurrency control with low cost for in-memory databases. In: PVLDB (2016)

Download references

Acknowledgements

The authors gratefully acknowledge Goetz Graefe and Harumi Kuno for their invaluable suggestions and input they provided during the development of SSN, as well as Kangnyeon Kim for his help building the prototype. We also thank the anonymous reviewers for their comments and suggestions that greatly improved this paper. Funding was provided by ‘Hewlett Packard Enterprise’.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Toronto, Toronto, ON, Canada

    Tianzheng Wang

  2. LogicBlox, Atlanta, GA, USA

    Ryan Johnson

  3. School of Information Technologies, University of Sydney, Sydney, NSW, Australia

    Alan Fekete

  4. Amazon Web Services, Palo Alto, CA, USA

    Ippokratis Pandis

Authors
  1. Tianzheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ryan Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alan Fekete
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ippokratis Pandis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tianzheng Wang.

Additional information

The original version of this article was revised:The original article uses incorrect versions of Figures 6, 7, 8, 9 that do not match their descriptions and they have been corrected.

An erratum to this article is available at https://doi.org/10.1007/s00778-017-0471-8.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, T., Johnson, R., Fekete, A. et al. Efficiently making (almost) any concurrency control mechanism serializable. The VLDB Journal 26, 537–562 (2017). https://doi.org/10.1007/s00778-017-0463-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00778-017-0463-8

Keywords

Search

Navigation