Encyclopedia of Big Data Technologies

Living Edition
| Editors: Sherif Sakr, Albert Zomaya

Conflict-Free Replicated Data Types CRDTs

Living reference work entry
DOI: https://doi.org/10.1007/978-3-319-63962-8_185-1

Definition

A conflict-free replicated data type (CRDT) is an abstract data type, with a well-defined interface, designed to be replicated at multiple processes and exhibiting the following properties: (i) any replica can be modified without coordinating with other replicas and (ii) when any two replicas have received the same set of updates, they reach the same state, deterministically, by adopting mathematically sound rules to guarantee state convergence.

Overview

Internet-scale distributed systems often replicate data at multiple geographic locations to provide low latency and high availability, despite outages and network failures. To this end, these systems must accept updates at any replica and propagate these updates asynchronously to the other replicas. This approach allows replicas to temporarily diverge and requires a mechanism for merging concurrent updates into a common state. CRDTs provide a principled approach to address this issue.

As any abstract data type, a CRDT...

This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

This work was partially supported by NOVA LINCS (UID/CEC/04516/2013), EU H2020 LightKone project (732505), and SMILES line in project TEC4Growth (NORTE-01-0145-FEDER-000020).

References

  1. Akkoorath DD, Tomsic AZ, Bravo M, Li Z, Crain T, Bieniusa A, Preguiça N, Shapiro M (2016) Cure: strong semantics meets high availability and low latency. In: Proceedings of the 2016 IEEE 36th international conference on distributed computing systems (ICDCS), pp 405–414.  https://doi.org/10.1109/ICDCS.2016.98
  2. Almeida PS, Baquero C (2013) Scalable eventually consistent counters over unreliable networks. CoRR abs/1307.3207. http://arxiv.org/abs/1307.3207, 1307.3207
  3. Almeida PS, Shoker A, Baquero C (2018) Delta state replicated data types. J Parallel Distrib Comput 111:162–173. https://doi.org/10.1016/j.jpdc.2017.08.003CrossRefGoogle Scholar
  4. Balegas V, Duarte S, Ferreira C, Rodrigues R, Preguiça NM, Najafzadeh M, Shapiro M (2015a) Putting consistency back into eventual consistency. In: Réveillère L, Harris T, Herlihy M (eds) Proceedings of the tenth European conference on computer systems, EuroSys 2015, Bordeaux. ACM, pp 6:1–6:16. https://doi.org/10.1145/2741948.2741972
  5. Balegas V, Serra D, Duarte S, Ferreira C, Shapiro M, Rodrigues R, Preguiça NM (2015b) Extending eventually consistent cloud databases for enforcing numeric invariants. In: 34th IEEE symposium on reliable distributed systems, SRDS 2015, Montreal. IEEE Computer Society, pp 31–36.  https://doi.org/10.1109/SRDS.2015.32
  6. Baquero C, Almeida PS, Shoker A (2014) Making operation-based CRDTs operation-based. In: Proceedings of the first workshop on principles and practice of eventual consistency, PaPEC’14. ACM, New York, pp 7:1–7:2. https://doi.org/10.1145/2596631.2596632
  7. Biyikoglu C (2017) Under the hood: Redis CRDTs (conflict-free replicated data types). Online https://goo.gl/tGqU7h. Accessed 24 Nov 2017
  8. Brewer E (2010) On a certain freedom: exploring the CAP space, invited talk at PODC 2010, ZurichGoogle Scholar
  9. Brown R, Cribbs S, Meiklejohn C, Elliott S (2014) Riak DT map: a Composable, convergent replicated dictionary. In: Proceedings of the first workshop on principles and practice of eventual consistency, PaPEC’14. ACM, New York, pp 1:1–1:1. https://doi.org/10.1145/2596631.2596633
  10. Burckhardt S, Gotsman A, Yang H, Zawirski M (2014) Replicated data types: specification, verification, optimality. In: Proceedings of the 41st ACM SIGPLAN-SIGACT symposium on principles of programming languages, POPL’14. ACM, New York, pp 271–284. https://doi.org/10.1145/2535838.2535848
  11. Cabrita G, Preguiça N (2017) Non-uniform replication. In: Proceedings of the 21th international conference on principles of distributed systems, OPODIS 2017, Schloss Dagstuhl – Leibniz-Zentrum fuer Informatik, LIPIcsGoogle Scholar
  12. Charron-Bost B (1991) Concerning the size of logical clocks in distributed systems. Inf Process Lett 39(1):11–16. https://doi.org/10.1016/0020-0190(91)90055-MMathSciNetCrossRefGoogle Scholar
  13. Enes V (2017) Efficient Synchronization of State-based CRDTs. Master’s thesis, Universidade do Minho. http://vitorenesduarte.github.io/page/other/msc-thesis.pdf
  14. Gilbert S, Lynch N (2002) Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News 33(2):51–59. https://doi.org/10.1145/564585.564601CrossRefGoogle Scholar
  15. Gomes VBF, Kleppmann M, Mulligan DP, Beresford AR (2017) Verifying strong eventual consistency in distributed systems. Proc ACM Program Lang 1(OOPSLA):109:1–109:28. https://doi.org/10.1145/3133933CrossRefGoogle Scholar
  16. Gonçalves RJT, Almeida PS, Baquero C, Fonte V (2017) DottedDB: anti-entropy without Merkle trees, deletes without tombstones. In: Proceedings of the 2017 IEEE 36th symposium on reliable distributed systems (SRDS), pp 194–203.  https://doi.org/10.1109/SRDS.2017.28
  17. Gotsman A, Yang H, Ferreira C, Najafzadeh M, Shapiro M (2016) ’cause i’m strong enough: reasoning about consistency choices in distributed systems. In: Proceedings of the 43rd annual ACM SIGPLAN-SIGACT symposium on principles of programming languages, POPL’16. ACM, New York, pp 371–384. https://doi.org/10.1145/2837614.2837625
  18. Kleppmann M, Beresford AR (2017) A conflict-free replicated JSON datatype. IEEE Trans Parallel Distrib Syst 28(10):2733–2746.  https://doi.org/10.1109/TPDS.2017.2697382CrossRefGoogle Scholar
  19. Kulkarni SS, Demirbas M, Madappa D, Avva B, Leone M (2014) Logical physical clocks. In: Aguilera MK, Querzoni L, Shapiro M (eds) Principles of distributed systems – 18th international conference, OPODIS 2014, Cortina d’Ampezzo. Proceedings. Lecture notes in computer science, vol 8878. Springer, pp 17–32. https://doi.org/10.1007/978-3-319-14472-6_2Google Scholar
  20. Lamport L (1978) Time, clocks, and the ordering of events in a distributed system. Commun ACM 21(7):558–565. https://doi.org/10.1145/359545.359563CrossRefGoogle Scholar
  21. Lamport L (1994) The temporal logic of actions. ACM Trans Program Lang Syst 16(3):872–923. https://doi.org/10.1145/177492.177726CrossRefGoogle Scholar
  22. Li C, Porto D, Clement A, Gehrke J, Preguiça N, Rodrigues R (2012) Making geo-replicated systems fast as possible, consistent when necessary. In: Proceedings of the 10th USENIX conference on operating systems design and implementation, OSDI’12. USENIX Association, Berkeley, pp 265–278. http://dl.acm.org/citation.cfm?id=2387880.2387906
  23. Malkhi D, Terry DB (2007) Concise version vectors in winfs. Distrib Comput 20(3):209–219. https://doi.org/10.1007/s00446-007-0044-yCrossRefGoogle Scholar
  24. O’Neil PE (1986) The escrow transactional method. ACM Trans Database Syst 11(4):405–430. https://doi.org/10.1145/7239.7265CrossRefGoogle Scholar
  25. Preguiça N, Marques JM, Shapiro M, Letia M (2009) A commutative replicated data type for cooperative editing. In: Proceedings of the 2009 29th IEEE international conference on distributed computing systems, ICDCS’09. IEEE Computer Society, Washington, DC, pp 395–403.  https://doi.org/10.1109/ICDCS.2009.20
  26. Preguiça NM, Zawirski M, Bieniusa A, Duarte S, Balegas V, Baquero C, Shapiro M (2014) Swiftcloud: fault-tolerant geo-replication integrated all the way to the client machine. In: 33rd IEEE international symposium on reliable distributed systems workshops, SRDS workshops 2014, Nara. IEEE Computer Society, pp 30–33.  https://doi.org/10.1109/SRDSW.2014.33
  27. Roh HG, Jeon M, Kim JS, Lee J (2011) Replicated abstract data types: building blocks for collaborative applications. J Parallel Distrib Comput 71(3):354–368. https://doi.org/10.1016/j.jpdc.2010.12.006CrossRefGoogle Scholar
  28. Shapiro M, Preguiça N, Baquero C, Zawirski M (2011) Conflict-free replicated data types. In: Proceedings of the 13th international conference on stabilization, safety, and security of distributed systems, SSS’11. Springer, Berlin/Heidelberg, pp 386–400. http://dl.acm.org/citation.cfm?id=2050613.2050642CrossRefGoogle Scholar
  29. Sovran Y, Power R, Aguilera MK, Li J (2011) Transactional storage for geo-replicated systems. In: Proceedings of the twenty-third ACM symposium on operating systems principles, SOSP’11. ACM, New York, pp 385–400. https://doi.org/10.1145/2043556.2043592CrossRefGoogle Scholar
  30. Terry DB, Demers AJ, Petersen K, Spreitzer M, Theimer M, Welch BB (1994) Session guarantees for weakly consistent replicated data. In: Proceedings of the third international conference on parallel and distributed information systems, PDIS’94, Austin. IEEE Computer Society, pp 140–149.  https://doi.org/10.1109/PDIS.1994.331722CrossRefGoogle Scholar
  31. van der Linde A, Leitão JA, Preguiça N (2016) Δ-CRDTs: making δ-CRDTs delta-based. In: Proceedings of the 2nd workshop on the principles and practice of consistency for distributed data, PaPoC’16. ACM, New York, pp 12:1–12:4. https://doi.org/10.1145/2911151.2911163
  32. Weiss S, Urso P, Molli P (2009) Logoot: a scalable optimistic replication algorithm for collaborative editing on p2p networks. In: Proceedings of the 2009 29th IEEE international conference on distributed computing systems, ICDCS’09. IEEE Computer Society, Washington, DC, pp 404–412.  https://doi.org/10.1109/ICDCS.2009.75CrossRefGoogle Scholar
  33. Weiss S, Urso P, Molli P (2010) Logoot-undo: distributed collaborative editing system on p2p networks. IEEE Trans Parallel Distrib Syst 21(8):1162–1174.  https://doi.org/10.1109/TPDS.2009.173CrossRefGoogle Scholar
  34. Zawirski M, Baquero C, Bieniusa A, Preguiça N, Shapiro M (2016) Eventually consistent register revisited. In: Proceedings of the 2nd workshop on the principles and practice of consistency for distributed data, PaPoC’16. ACM, New York, pp 9:1–9:3. https://doi.org/10.1145/2911151.2911157
  35. Zeller P (2017) Testing properties of weakly consistent programs with repliss. In: Proceedings of the 3rd international workshop on principles and practice of consistency for distributed data, PaPoC’17. ACM, New York, pp 3:1–3:5. https://doi.org/10.1145/3064889.3064893, https://dl.acm.org/authorize?N37605
  36. Zeller P, Bieniusa A, Poetzsch-Heffter A (2014) Formal specification and verification of CRDTs. In: Formal techniques for distributed objects, FORTE 2014. Lecture notes in computer science. Springer, pp 33–48CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.NOVA LINCS & DI, FCT, Universidade NOVA de LisboaCaparicaPortugal
  2. 2.HASLab/INESC TEC & Universidade do MinhoBragaPortugal
  3. 3.Sorbonne Université, LIP6 & INRIAParisFrance

Section editors and affiliations

  • Mohammad Sadoghi

There are no affiliations available