Skip to main content

Banyan: Coordination-Free Distributed Transactions over Mergeable Types

  • 332 Accesses

Part of the Lecture Notes in Computer Science book series (LNPSE,volume 12470)

Abstract

Programming loosely connected distributed applications is a challenging endeavour. Loosely connected distributed applications such as geo-distributed stores and intermittently reachable IoT devices cannot afford to coordinate among all of the replicas in order to ensure data consistency due to prohibitive latency costs and the impossibility of coordination if availability is to be ensured. Thus, the state of the replicas evolves independently, making it difficult to develop correct applications. Existing solutions to this problem limit the data types that can be used in these applications, which neither offer the ability to compose them to construct more complex data types nor offer transactions.

In this paper, we describe Banyan, a distributed programming model for developing loosely connected distributed applications. Data types in Banyan are equipped with a three-way merge function à la Git to handle conflicts. Banyan provides isolated transactions for grouping together individual operations which do not require coordination among different replicas. We instantiate Banyan over Cassandra, an off-the-shelf industrial-strength distributed store. Several benchmarks, including a distributed build-cache, illustrates the effectiveness of the approach.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-64437-6_12
  • Chapter length: 20 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   69.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-64437-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   89.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.
Fig. 12.

References

  1. Abadi, D.: Consistency tradeoffs in modern distributed database system design: CAP is only part of the story. Computer 45(2), 37–42 (2012). https://doi.org/10.1109/MC.2012.33

    CrossRef  Google Scholar 

  2. Amazon DynamoDB: Fast and flexible NoSQL database service for any scale (2020). https://aws.amazon.com/dynamodb/

  3. Apache Cassandra: The right choice when you need scalability and high availability without compromising performance (2020). https://cassandra.apache.org/

  4. Azure CosmosDB: Build or modernise scalable, high-performance apps (2020). https://azure.microsoft.com/en-in/services/cosmos-db/

  5. Bailis, P., Davidson, A., Fekete, A., Ghodsi, A., Hellerstein, J.M., Stoica, I.: Highly available transactions: virtues and limitations. Proc. VLDB Endow. 7(3), 181–192 (2013). https://doi.org/10.14778/2732232.2732237

  6. Bailis, P., Ghodsi, A., Hellerstein, J.M., Stoica, I.: Bolt-on causal consistency. In: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, SIGMOD 2013, pp. 761–772 (2013). https://doi.org/10.1145/2463676.2465279

  7. Bazel: A fast, scalable, multi-language build system (2020). https://bazel.build/

  8. Berenson, H., Bernstein, P., Gray, J., Melton, J., O’Neil, E., O’Neil, P.: A critique of ANSI SQL isolation levels. SIGMOD Rec. 24(2), 1–10 (1995). https://doi.org/10.1145/568271.223785

    CrossRef  Google Scholar 

  9. Bernstein, P.A., Shipman, D.W., Wong, W.S.: Formal aspects of serializability in database concurrency control. IEEE Trans. Softw. Eng. 5(3), 203–216 (1979). https://doi.org/10.1109/TSE.1979.234182

    MathSciNet  CrossRef  MATH  Google Scholar 

  10. Brutschy, L., Dimitrov, D., Müller, P., Vechev, M.: Serializability for eventual consistency: criterion, analysis, and applications. In: Proceedings of the 44th ACM SIGPLAN Symposium on POPL, POPL 2017, pp. 458–472. (2017). https://doi.org/10.1145/3009837.3009895

  11. Burckhardt, S., Gotsman, A., Yang, H., Zawirski, M.: Replicated data types: specification, verification. Optimality. SIGPLAN Not. 49(1), 271–284 (2014). https://doi.org/10.1145/2578855.2535848

    CrossRef  MATH  Google Scholar 

  12. Burckhardt, S., Leijen, D., Fähndrich, M., Sagiv, M.: Eventually consistent transactions. In: Seidl, H. (ed.) ESOP 2012. LNCS, vol. 7211, pp. 67–86. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28869-2_4

    CrossRef  Google Scholar 

  13. Crain, T., Shapiro, M.: Designing a causally consistent protocol for geo-distributed partial replication. In: Proceedings of the First Workshop on Principles and Practice of Consistency for Distributed Data, PaPoC 2015 (2015). https://doi.org/10.1145/2745947.2745953

  14. Crooks, N., Pu, Y., Estrada, N., Gupta, T., Alvisi, L., Clement, A.: TARDiS: a branch-and-merge approach to weak consistency. In: Proceedings of the 2016 International Conference on Management of Data, SIGMOD 2016, pp. 1615–1628 (2016). https://doi.org/10.1145/2882903.2882951

  15. Driscoll, J.R., Sarnak, N., Sleator, D.D., Tarjan, R.E.: Making data structures persistent. In: Proceedings of the Eighteenth Annual ACM Symposium on Theory of Computing, STOC 1986, pp. 109–121 (1986). https://doi.org/10.1145/12130.12142

  16. Farinier, B., Gazagnaire, T., Madhavapeddy, A.: Mergeable persistent data structures. In: Vingt-sixièmes Journées Francophones des Langages Applicatifs (JFLA 2015) (2015)

    Google Scholar 

  17. Gilbert, S., Lynch, N.: Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News 33(2), 51–59 (2002). https://doi.org/10.1145/564585.564601

    CrossRef  Google Scholar 

  18. Git: A distributed version control system (2020). https://git-scm.com/

  19. Gradle: An open-source build automation tool (2020). https://gradle.org/

  20. Herlihy, M.P., Wing, J.M.: Linearizability: a correctness condition for concurrent objects. ACM Trans. Program. Lang. Syst. 12(3), 463–492 (1990). https://doi.org/10.1145/78969.78972

    CrossRef  Google Scholar 

  21. Irmin: A distributed database built on the principles of Git (2020). https://irmin.org/

  22. Kaki, G., Nagar, K., Najafzadeh, M., Jagannathan, S.: Alone together: compositional reasoning and inference for weak isolation. Proc. ACM Program. Lang. 2(POPL) (2017). https://doi.org/10.1145/3158115

  23. Kaki, G., Priya, S., Sivaramakrishnan, K., Jagannathan, S.: Mergeable replicated data types. Proc. ACM Program. Lang. 3(OOPSLA) (2019). https://doi.org/10.1145/3360580

  24. Kermarrec, A.M., van Steen, M.: Gossiping in distributed systems. SIGOPS Oper. Syst. Rev. 41(5), 2–7 (2007). https://doi.org/10.1145/1317379.1317381

    CrossRef  Google Scholar 

  25. Kraska, T., Pang, G., Franklin, M.J., Madden, S., Fekete, A.: MDCC: multi-data center consistency. In: Proceedings of the 8th ACM European Conference on Computer Systems, EuroSys 2013, pp. 113–126 (2013). https://doi.org/10.1145/2465351.2465363

  26. Lakshman, A., Malik, P.: Cassandra: a decentralized structured storage system. SIGOPS Oper. Syst. Rev. 44(2), 35–40 (2010). https://doi.org/10.1145/1773912.1773922

    CrossRef  Google Scholar 

  27. Lamport, L.: Paxos made simple. ACM SIGACT News (Distrib. Comput. Column) 32(4), 51–58 (2001). https://www.microsoft.com/en-us/research/publication/paxos-made-simple/. (Whole Number 121, December 2001)

  28. Li, C., Porto, D., Clement, A., Gehrke, J., Preguiça, N., Rodrigues, R.: Making geo-replicated systems fast as possible, consistent when necessary. In: Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation, OSDI 2012, pp. 265–278 (2012)

    Google Scholar 

  29. Lloyd, W., Freedman, M.J., Kaminsky, M., Andersen, D.G.: Don’t settle for eventual: scalable causal consistency for wide-area storage with COPS. In: Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, SOSP 2011, pp. 401–416 (2011). https://doi.org/10.1145/2043556.2043593

  30. Lopes, P., et al.: Antidote SQL: relaxed when possible, strict when necessary. CoRR abs/1902.03576 (2019). http://arxiv.org/abs/1902.03576

  31. Milano, M., Myers, A.C.: MixT: a language for mixing consistency in geodistributed transactions. In: Proceedings of the 39th ACM SIGPLAN Conference on PLDI, pp. 226–241 (2018). https://doi.org/10.1145/3192366.3192375

  32. Riak: Enterprise NoSQL Database (2020). https://riak.com/

  33. Shapiro, M., Preguiça, N., Baquero, C., Zawirski, M.: Conflict-free replicated data types. In: Défago, X., Petit, F., Villain, V. (eds.) SSS 2011. LNCS, vol. 6976, pp. 386–400. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24550-3_29

    CrossRef  MATH  Google Scholar 

  34. Sivaramakrishnan, K., Kaki, G., Jagannathan, S.: Declarative programming over eventually consistent data stores. In: Proceedings of the 36th ACM SIGPLAN Conference on PLDI, pp. 413–424 (2015). https://doi.org/10.1145/2737924.2737981

  35. Sovran, Y., Power, R., Aguilera, M.K., Li, J.: Transactional storage for geo-replicated systems. In: Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, SOSP 2011, pp. 385–400 (2011). https://doi.org/10.1145/2043556.2043592

  36. Viotti, P., Vukolić, M.: Consistency in non-transactional distributed storage systems. ACM Comput. Surv. 49(1) (2016). https://doi.org/10.1145/2926965

Download references

Acknowledgements

Parts of this research were funded by grants from the Tezos Foundation.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shashank Shekhar Dubey .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Dubey, S.S., Sivaramakrishnan, K.C., Gazagnaire, T., Madhavapeddy, A. (2020). Banyan: Coordination-Free Distributed Transactions over Mergeable Types. In: Oliveira, B.C.d.S. (eds) Programming Languages and Systems. APLAS 2020. Lecture Notes in Computer Science(), vol 12470. Springer, Cham. https://doi.org/10.1007/978-3-030-64437-6_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-64437-6_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-64436-9

  • Online ISBN: 978-3-030-64437-6

  • eBook Packages: Computer ScienceComputer Science (R0)