Distributed Protocol Combinators

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11372)


Distributed systems are hard to get right, model, test, debug, and teach. Their textbook definitions, typically given in a form of replicated state machines, are concise, yet prone to introducing programming errors if naïvely translated into runnable implementations.

In this work, we present Distributed Protocol Combinators (DPC), a declarative programming framework that aims to bridge the gap between specifications and runnable implementations of distributed systems, and facilitate their modeling, testing, and execution. DPC builds on the ideas from the state-of-the art logics for compositional systems verification. The contribution of DPC is a novel family of program-level primitives, which facilitates construction of larger distributed systems from smaller components, streamlining the usage of the most common asynchronous message-passing communication patterns, and providing machinery for testing and user-friendly dynamic verification of systems. This paper describes the main ideas behind the design of the framework and presents its implementation in Haskell. We introduce DPC through a series of characteristic examples and showcase it on a number of distributed protocols from the literature.



We thank PADL’19 referees for their many helpful suggestions. The authors’ work on this project has been supported by the grant by UK Research Institute in Verified Trustworthy Software Systems (VeTSS).


  1. 1.
    Brady,E.: Type-driven development of concurrent communicating systems. Comput. Sci. (AGH), 18(3) (2017)CrossRefGoogle Scholar
  2. 2.
    Chandra, T., Griesemer, R., Redstone, J.: Paxos made live: an engineering perspective. In: PODC (2007)Google Scholar
  3. 3.
    Coq Development Team. The Coq Proof Assistant Reference Manual (2018)Google Scholar
  4. 4.
    Dinsdale-Young, T., Dodds, M., Gardner, P., Parkinson, M.J., Vafeiadis, V.: Concurrent abstract predicates. In: D’Hondt, T. (ed.) ECOOP 2010. LNCS, vol. 6183, pp. 504–528. Springer, Heidelberg (2010). Scholar
  5. 5.
    Dragoi, C., Henzinger, T.A., Zufferey,D.: PSync: a partially synchronous language for fault-tolerant distributed algorithms. In: POPL (2016)Google Scholar
  6. 6.
    García-Pérez, Á., Gotsman, A., Meshman, Y., Sergey, I.: Paxos consensus, deconstructed and abstracted. In: Ahmed, A. (ed.) ESOP 2018. LNCS, vol. 10801, pp. 912–939. Springer, Cham (2018). Scholar
  7. 7.
    Gray, J.N.: Notes on data base operating systems. In: Operating Systems (1978)Google Scholar
  8. 8.
    Hawblitzel, C., et al.: IronFleet: proving practical distributed systems correct. In: SOSP (2015)Google Scholar
  9. 9.
    Killian, C.E., Anderson, J.W., Braud, R., Jhala, R., Vahdat, A.M.: Mace: language support for building distributed systems. In: PLDI (2007)Google Scholar
  10. 10.
    Kleppmann, M.: How to do distributed locking, 08 February 2016.
  11. 11.
    Krogh-Jespersen, M., Timani, A., Ohlenbusch, M.E., Birkedal, L.: Aneris: a logic for node-local, modular reasoning of distributed systems (2018, unpublished draft)Google Scholar
  12. 12.
    Lamport, L.: The part-time parliament. ACM Trans. Comput. Syst. 16(2), 133–169 (1998)CrossRefGoogle Scholar
  13. 13.
    Lamport, L.: Paxos made simple (2001)Google Scholar
  14. 14.
    Alford, M.W., et al.: Formal foundation for specification and verification. In: Paul, M., Siegert, H.J. (eds.) Distributed Systems: Methods and Tools for Specification. An Advanced Course. LNCS, vol. 190, pp. 203–285. Springer, Heidelberg (1985). Scholar
  15. 15.
    Lampson, B.W.: How to build a highly available system using consensus. In: WDAG (1996)Google Scholar
  16. 16.
    Leonini, L., Riviere, E., Felber, P.: SPLAY: distributed systems evaluation made simple (or how to turn ideas into live systems in a breeze). In: NSDI (2009)Google Scholar
  17. 17.
    Liang, S., Hudak, P., Jones, M.P.: Monad transformers and modular interpreters. In: POPL (1995)Google Scholar
  18. 18.
    Liu, Y.A., Stoller, S.D., Lin, B., Gorbovitski, M.: From clarity to efficiency for distributed algorithms. In: OOPSLA (2012)Google Scholar
  19. 19.
    Nanevski, A., Morrisett, G., Shinnar, A., Govereau, P., Birkedal, L.: Ynot: dependent types for imperative programs. In: ICFP (2008)Google Scholar
  20. 20.
    Newcombe, C., Rath, T., Zhang, F., Munteanu, B., Brooker, M., Deardeuff, M.: How Amazon web services uses formal methods. Commun. ACM 58(4), 66–73 (2015)CrossRefGoogle Scholar
  21. 21.
    O’Hearn, P., Reynolds, J., Yang, H.: Local reasoning about programs that alter data structures. In: Fribourg, L. (ed.) CSL 2001. LNCS, vol. 2142, pp. 1–19. Springer, Heidelberg (2001). Scholar
  22. 22.
    Pîrlea, G., Sergey, I.: Mechanising blockchain consensus. In: CPP (2018)Google Scholar
  23. 23.
    I. Sergey, A. Nanevski, and A. Banerjee. Mechanized verification of fine-grained concurrent programs. In PLDI, 2015Google Scholar
  24. 24.
    Sergey, I., Wilcox, J.R., Tatlock, Z.: Programming and proving with distributed protocols. In: PACMPL(POPL), vol. 2 (2018)Google Scholar
  25. 25.
    van Renesse, R., Altinbuken, D.: Paxos made moderately complex. ACM Comp. Surv. 47(3), 42 (2015)Google Scholar
  26. 26.
    Wilcox, J.R., Sergey, I., Tatlock, Z.: Programming language abstractions for modularly verified distributed systems. In: SNAPL (2017)Google Scholar
  27. 27.
    Wilcox, J.R., et al.: Verdi: a framework for implementing and formally verifying distributed systems. In: PLDI (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Aarhus UniversityAarhusDenmark
  2. 2.Yale-NUS College and NUS School of ComputingSingaporeSingapore

Personalised recommendations