Decentralized Message Ordering for Publish/Subscribe Systems

  • Cristian Lumezanu
  • Neil Spring
  • Bobby Bhattacharjee
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4290)


We describe a method to order messages across groups in a publish/subscribe system without centralized control or large vector timestamps. We show that our scheme is practical—little state is required; that it is scalable—the maximum message load is limited by receivers; and that it performs well—the paths messages traverse to be ordered are not made much longer than necessary. Our insight is that only messages to groups that overlap in membership can be observed to arrive out of order: sequencing messages to these groups is sufficient to provide a consistent order, and when publishers subscribe to the groups to which they send, this message order is a causal order.


Sequence Number Sequencing Graph Causal Order Zipf Distribution Sequencing Node 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bharambe, A.R., Rao, S., Seshan, S.: Mercury: A scalable publish-subscribe system for Internet games. In: NetGames (2002)Google Scholar
  2. 2.
    Hu, S.Y., Liao, G.M.: Scalable peer-to-peer networked virtual environment. In: chang Feng, W. (ed.) NETGAMES, pp. 129–133. ACM Press, New York (2004)Google Scholar
  3. 3.
    Morse, K.: Interest management in large-scale distributed simulations. Technical report, UC Irvine (1996)Google Scholar
  4. 4.
    Défago, X., Schiper, A., Urbán, P.: Total order broadcast and multicast algorithms: Taxonomy and survey. In: ACM Computing Surveys (2004)Google Scholar
  5. 5.
    Lamport, L.: Time, clocks and the ordering of events in a distributed system. Communications of the ACM (1978)Google Scholar
  6. 6.
    Schiper, A., Eggli, J., Sandoz, A.: A new algorithm to implement causal ordering. In: Bermond, J.-C., Raynal, M. (eds.) WDAG 1989. LNCS, vol. 392. Springer, Heidelberg (1989)CrossRefGoogle Scholar
  7. 7.
    Peterson, L.L., Buchholz, N.C., Schlichting, R.D.: Preserving and using context information in interprocess communication. ACM TOCS 7(3), 217–246 (1989)CrossRefGoogle Scholar
  8. 8.
    Dolev, D., Dwork, C., Stockmeyer, L.: Early delivery totally ordered multicast in asynchronous environments. In: 23rd Int’l Symposium on Fault-Tolerant Computing (FTCS-23) (1993)Google Scholar
  9. 9.
    Birman, K.P., Joseph, T.A.: Reliable communication in the presence of failures. ACM TOCS 5(1), 47–76 (1987)CrossRefGoogle Scholar
  10. 10.
    Amir, Y., Moser, L.E., Melliar-Smith, P.M., Agarwal, D.A., Ciarfella, P.: The Totem single-ring ordering and membership protocol. ACM TOCS 13(4), 311–342 (1995)CrossRefGoogle Scholar
  11. 11.
    Cristian, F.: Asynchronous atomic broadcast. IBM Technical Disclosure Bulletin (1991)Google Scholar
  12. 12.
    Rajagopalan, B., McKinley, P.: A token-based protocol for reliable, ordered multicast communication. In: 8th Symposium on Reliable Distributed Systems (SRDS) (1989)Google Scholar
  13. 13.
    Kaashoek, M.F., Tanenbaum, A.S.: An evaluation of the Amoeba group communication system. In: ICDCS (1996)Google Scholar
  14. 14.
    Garcia-Molina, H., Spauster, A.: Ordered and reliable multicast communication. ACM TOCS 9(3), 242–271 (1991)CrossRefGoogle Scholar
  15. 15.
    Schiper, A., Birman, K., Stephenson, P.: Lightweight causal and atomic group multicast. ACM TOCS 9(3), 272–314 (1991)CrossRefGoogle Scholar
  16. 16.
    Chang, J.M., Maxemchuk, N.F.: Reliable broadcast protocols. ACM TOCS 2(3), 251–273 (1984)CrossRefGoogle Scholar
  17. 17.
    Whetten, B., Montgomery, T., Kaplan, S.M.: A high performance totally ordered multicast protocol. In: Birman, K.P., Mattern, F., Schiper, A. (eds.) Dagstuhl Seminar 1994. LNCS, vol. 938. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  18. 18.
    Gautier, L., Diot, C.: Design and evaluation of mimaze, a multi-player game on the Internet. In: IEEE Int’l Conference on Multimedia Computing and Systems (1998)Google Scholar
  19. 19.
    Ishibashi, Y., Tasaka, S., Tachibana, Y.: A media synchronization scheme with causality control in networked environments. In: IEEE LCN (1999)Google Scholar
  20. 20.
    Ishibashi, Y., Tasaka, S., Tachibana, Y.: Adaptive causality and media synchronization control for networked multimedia applications. In: IEEE ICC (2001)Google Scholar
  21. 21.
    Iimura, T., Hazeyama, H., Kadobayashi, Y.: Zoned federation of game servers: a peer-to-peer approach to scalable multi-player online games. In: NETGAMES (2004)Google Scholar
  22. 22.
    Jia, X.: A total ordering multicast protocol using propagation trees. IEEE Trans. Parallel Distrib. Syst. 6(6), 617–627 (1995)CrossRefGoogle Scholar
  23. 23.
    Ezhilchelvan, P.D., Macedo, R.A., Shrivastava, S.K.: Newtop: a fault-tolerant group communication protocol. In: ICDCS (1995)Google Scholar
  24. 24.
    Aguilera, M.K., Strom, R.E.: Efficient atomic broadcast using deterministic merge. In: PODC (2000)Google Scholar
  25. 25.
    Carzaniga, A., Rosenblum, D.S., Wolf, A.L.: Design and evaluation of a wide-area event notification service. ACM TOCS 19(3), 332–383 (2001)CrossRefGoogle Scholar
  26. 26.
    Aguilera, M.K., Strom, R.E., Sturman, D.C., Astley, M., Chandra, T.D.: Matching events in a content-based subscription system. In: PODC, pp. 53–61 (1999)Google Scholar
  27. 27.
    Pietzuch, P., Bacon, J.: Hermes: A distributed event-based middleware architecture. In: 1st International Workshop on Distributed Event-Based Systems (DEBS 2002) (2002)Google Scholar
  28. 28.
    Castro, M., Druschel, P., Kermarrec, A.M., Rowstron, A.I.: Scribe: A large-scale and decentralized application-level multicast infrastructure. IEEE Journal of Selected Areas in Communication (2002)Google Scholar
  29. 29.
    Zegura, E., Calvert, K., Bhattacharjee, S.: How to model an internetwork. In: IEEE Infocom (1996)Google Scholar
  30. 30.
    Wolman, A., Voelker, G.M., Sharma, N., Cardwell, N., Karlin, A.R., Levy, H.M.: On the scale and performance of cooperative web proxy caching. In: SOSP (1999)Google Scholar
  31. 31.
    Breslau, L., Cao, P., Fan, L., Phillips, G., Shenker, S.: Web caching and zipf-like distributions: Evidence and implications. In: INFOCOM (1999)Google Scholar
  32. 32.
    Chu, Y.H., Rao, S.G., Zhang, H.: A case for end system multicast. In: ACM Sigmetrics (2000)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2006

Authors and Affiliations

  • Cristian Lumezanu
    • 1
  • Neil Spring
    • 1
  • Bobby Bhattacharjee
    • 1
  1. 1.University of MarylandCollege ParkUSA

Personalised recommendations