A high performance totally ordered multicast protocol

  • Brian Whetten
  • Todd Montgomery
  • Simon Kaplan
Group Communication
Part of the Lecture Notes in Computer Science book series (LNCS, volume 938)


This paper presents the Reliable Multicast Protocol (RMP). RMP provides a totally ordered, reliable, atomic multicast service on top of an unreliable multicast datagram service such as IP Multicasting. RMP is fully and symmetrically distributed so that no site bears an undue portion of the communication load. RMP provides a wide range of guarantees, from unreliable delivery to totally ordered delivery, to K-resilient, majority resilient, and totally resilient atomic delivery. These QoS guarantees may be selected on a per packet basis. RMP provides many communication options, including virtual synchrony, a publisher/subscriber model of message delivery, a client/server model of delivery, an implicit naming service, mutually exclusive handlers for messages, and mutually exclusive locks.

It has commonly been held that a large performance penalty must be paid in order to implement total ordering—RMP discounts this. On SparcStation5's on a 1250 KB/sec Ethernet, RMP provides totally ordered packet delivery to one destination at 1070 KB/sec throughput and with 4.0 ms packet latency. The performance stays roughly constant independent of the number of destinations. For two or more destinations on a LAN, RMP provides higher throughput than any protocol that does not use multicast or broadcast.


Data Packet Congestion Control Aggregate Throughput Multicast Packet Positive Acknowledgment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [ADKM91]
    Y. Amir, D. Dolev, S. Kramer and D. Malki. “Transis: A Communication Subsystem for High Availability.” Proceedings, 22nd Annual International Symposium on Fault Tolerant Computing (July 1992).Google Scholar
  2. [AFM92]
    S. Armstrong, A. Freier, K. Marzullo. “Multicast Transport Protocol”. RFC1301. (February, 1992).Google Scholar
  3. [AMSM92]
    D. A. Agarwal, P. M. Melliar-Smith, and L. E. Moser. “Totem: A protocol for message ordering in a wide-area network.” Proceedings of the First ISMM International Conference on Computer Communications and Networks (San Diego, CA, June 1992). pp. 1–5.Google Scholar
  4. [ARP93]
    R. Aiello, E. Pagani, G. P. Rossi, “Design of a Reliable Multicast Protocol”. Proceedings of IEEE INFOCOM '93 (San Francisco, March 1993). pp. 75–81.Google Scholar
  5. [APR93]
    R. Aiello, G. P. Rossi, E. Pagani. “Casual Ordering in Reliable Group Communications.” Proceedings of ACM SIGCOMM '93 (San Francisco, Sept. 1993). pp. 106–115.Google Scholar
  6. [BLNS82]
    A. Birrell, R. Levin, R. Needham, M. Schroeder. “Grapevine: An exercise in distributed computing.” Communications of the ACM, 25, 4, April 1982. pp. 260–274.Google Scholar
  7. [BSS91]
    K. Birman, A. Schiper, P. Stephenson. “Lightweight Causal and Atomic Group Multicast.” ACM Transactions on Computer Systems. 9, 3 (Aug. 1991). pp. 272–314.Google Scholar
  8. [BiCl94]
    K. Birman, T. Clark. “Performance of the Isis Distributed Computing Toolkit.” Technical Report TR-94-1432, Dept. of Computer Science, Cornell University.Google Scholar
  9. [Birman93]
    K. Birman. “The Process Group Approach to Reliable Distributed Computing.” Communications of the ACM. December, 1993, 36, 12. pp. 37–53.Google Scholar
  10. [CaMo94]
    J. Callahan, T. Montgomery. “A Decentralized Software Bus based on IP Multicasting.” Proceedings of Third Workshop on Enabling Technologies: Infrastructure For Collaborative Enterprises, Morgantown, WV, April 17–19, 1994, pp. 65–69.Google Scholar
  11. [Carroll93]
    Alan Carroll. “ConversationBuilder: A Collaborative Erector Set.” Ph.D. Thesis, Department of Computer Science, University of Illinois, 1993.Google Scholar
  12. [ChMa83]
    J. M. Chang and N. F. Maxemchuk. “A Broadcast Protocol for Broadcast Networks.” Proceedings of GLOBCOM (Dec. 1983).Google Scholar
  13. [Chang84]
    J. M. Chang. “Simplifying Distributed Database Systems Design by Using a Broadcast Network.” Proceedings of SIGMOD (June 1984). pp. 223–233.Google Scholar
  14. [ChMa84]
    J. M. Chang and N. F. Maxemchuk. “Reliable Broadcast Protocols.” ACM Transactions on Computer Systems. 2, 3 (Aug. 1984). pp. 251–273.Google Scholar
  15. [ChZw85]
    D.R. Cheriton and W. Zwaenepoel. “Distributed Process Groups in the V-Kernel.” ACM Transactions on Computer Systems. 3, 2 (May 1985). pp. 77–107.Google Scholar
  16. [CrPa88]
    J. Crowcroft, K. Paliwoda. “A Multicast Transport Protocol”. Proceedings of ACM SIGCOMM '88. pp. 247–256.Google Scholar
  17. [Deering89]
    S. Deering. “Host Extensions for IP Multicasting”. STD 5. RFC1112. (August 1989).Google Scholar
  18. [DKM93]
    D. Dolev, S. Kramer, D. Malki. “Early Delivery Totally Ordered Multicast in Asynchronous Environments.” 23rd Annual International Symposium on Fault-Tolerant Computing (FTCS). (Toulouse, France, June, 1993). pp. 544–553.Google Scholar
  19. [Jacob88]
    V. Jacobson. “Congestion Avoidance and Control.” Proceedings of ACM SIGCOMM '88 Symp (Sept. 1988). pp. 314–329.Google Scholar
  20. [JHC94]
    P. Jain, N. Hutchinson, and S. Chanson. “A Framework or the Non-Monolithic Implementation of Protocols in the x-kernel.” Proceedings of USENIX High Speed Networking (August 1994). pp. 13–30.Google Scholar
  21. [KTHB89]
    M. F. Kaashoek, A. S. Tanenbaum, S. F. Hummel, H. E. Bal. “An Efficient Reliable Broadcast Protocol.” Operating Systems Review. 23, 4 (Oct. 1989), pp. 5–19.Google Scholar
  22. [Lamp78]
    L. Lamport. “Time, Clocks, and the Ordering of Events in a Distributed System.” Communications of the ACM. 21, 7Google Scholar
  23. [LuGl90]
    S. W. Luan and V. D. Gligor. “A Fault Tolerant Protocol For Atomic Broadcast.” IEEE Transactions on Parallel and Distributed Systems. 1, 3 (July 1990). pp. 271–285.Google Scholar
  24. [MaBe93]
    C. Maeda and B. Bershad. “Protocol Service Decomposition for High-Performance Networking.” Proceedings of 14th ACM Symposium on Operating Systems Principles, December 1993.Google Scholar
  25. [MaCh84]
    N. F. Maxemchuk and J. M. Chang. “Analysis of the Messages Transmitted in a Broadcast Protocol.” Proceedings of the International Computer Conference (Amsterdam, May 1984). pp. 1263–1267.Google Scholar
  26. [MeBo76]
    R. M. Metcalf. and D. R. Boggs, “Ethernet: Distributed Packet Switching for Local Computer Networks.” Communications of the ACM. 19, 7 (July 1976). pp. 395–404.Google Scholar
  27. [MMA90]
    P. M. Meillar-Smith, L. E. Moser, V. Agrawala. “Broadcast Protocols for Distributed Systems.” IEEE Transactions on Parallel and Distributed Systems. 1, 1 (Jan. 1990). pp. 17–25.Google Scholar
  28. [PEA94]
    S. Pejhan, A. Eleftheriadis, D. Anastassiou. “Distributed Multicast Address Management in the Global Internet.” Submitted to IEEE Journal on Selected Areas in Communication on March 1, 1994.Google Scholar
  29. [PBS89]
    L. L. Peterson, N. C. Buchholz, R.D. Schlichting. “Preserving and Using Context Information in Interprocess Communication”. ACM Transactions on Computer Systems 7, 3 (Aug. 1989). pp. 217–246.Google Scholar
  30. [Purtilo85]
    J. Purtilo. “Polylith: An Environment to Support Management of Tool Interfaces.” ACM SIGPLAN Symposium on Language Issues in Programming Environments, Seattle, WA, June 25–28, 1985. pp. 12–18.Google Scholar
  31. [RaLi93]
    K. Ravindran and X. T. Lin. “Structural Complexity and Execution Efficiency of Distributed Application Protocols.” Proceedings of ACM SIGCOMM '93. (San Francisco Sept. 1993). pp. 160–169.Google Scholar
  32. [TNML93]
    C. Thekkath, T. Nguyen, E. Moy, and E. Lazowska. “Implementing Network Protocols at User Level.” IEEE/ACM Transactions on Networking, 1(5), 1993. pp. 554–565.Google Scholar
  33. [Verissmo92]
    Verissmo. “xAMp: A Multi-primitive Group Communications Service. Proceedings of the 11th Symposium on Reliable Distributed Computing.Google Scholar
  34. [WhStFe94]
    B. Whetten, S. Steinberg, and D. Ferrari. “The Packet Starvation Effect in CSMA/CD LANs and a Solution”. Proceedings of IEEE Local Computer Networks Conference (Minneapolis, Minnesota, Oct. 1994).Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • Brian Whetten
    • 1
  • Todd Montgomery
    • 2
  • Simon Kaplan
    • 3
  1. 1.University of California at BerkeleyUSA
  2. 2.West Virginia UniversityUSA
  3. 3.University of Illinois at Champaign-UrbanaUSA

Personalised recommendations