An efficient and scalable framework for content-based publish/subscribe systems

Article
First Online:
Received:
Accepted:

Abstract

Challenges for content-based publish/sub scribe systems include efficient subscription management and event matching, load balancing, and efficient and scalable event delivery. This paper presents an efficient and scalable framework for content-based publish/subscribe systems. We propose using K-D trees to dynamically partition and organize subscriptions, thereby preserving subscription locality, minimizing event matching load and ensuring load balance across nodes. We propose an efficient event delivery mechanism that cleverly exploits embedded trees in distributed hash tables to disseminate events. We show that the latency of event publication and delivery is low. The event delivery mechanism can deliver events to a large number of subscribers at low latency and overhead, consuming modest bandwidth.

Keywords

Distributed hash tables Content-based publish/subscribe Subscription management K-D trees Event matching Event delivery 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

We thank Adair Dingle for her valuable comments and inspiring discussions on the first draft of this paper. We also thank anonymous reviewers for their constructive feedbacks which helped improve the paper.

References

  1. 1.
    Fabret F, Jacobsen HA, Llirbat F, Pereira J, Ross KA, Shasha D (2001) Filtering algorithms and implementation for very fast publish/subscribe systems. In: Proceedings of the 2001 ACM SIGMOD, vol. 30. Santa Barbara, CA, pp 115–126Google Scholar
  2. 2.
    Segall B, Arnold D (1997) Elvin has left the building: a publish/subscribe notification service with quenching. In: Proceedings of AUUG. Brisbane, Australia, pp 243–255, SeptemberGoogle Scholar
  3. 3.
    Triantafillou P, Economides A (2004) Subscription summarization: a new paradigm for efficient publish/subscribe systems. In: Proceedings of the 24th IEEE ICDCS. Tokyo, Japan, pp 562–571, MarchGoogle Scholar
  4. 4.
    Carzaniga A, Wolf AL (2003) Forwarding in a content-based network. In: Proceedings of ACM SIGCOMM. Karlsruhe, Germany, pp 163–174, AugustGoogle Scholar
  5. 5.
    Carzaniga A, Rutherford MJ, Wolf AL (2004) A routing scheme for content-based networking. In: Proceedings of IEEE INFOCOM. Hongkong, China, pp 918–928, MarchGoogle Scholar
  6. 6.
    Cao F, Singh JP (2005) MEDYM: match-early and dynamic multicast for content-based publish-subscribe service networks. In: Proceedings of the 4th international workshop on distributed event-based systems. Washington, DC, pp 370–376Google Scholar
  7. 7.
    Gupta A, Sahin OD, Agrawal D, Abbadi AE (2004) Meghdoot: content-based publish/subscribe over P2P networks. In: ACM/IFIP/USENIX 5th international middleware conference. Toronto, Ontario, Canada, OctoberGoogle Scholar
  8. 8.
    Stoica I, Morris R, Karger D, Kaashoek M, Balakrishnan H (2001) Chord: a scalable peer-to-peer lookup service for internet applications. In: Proceedings of ACM SIGCOMM. San Diego, CA, pp 149–160, AugustGoogle Scholar
  9. 9.
    Rowstron A, Druschel P (2001) Pastry: scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In: Proceedings of the 18th IFIP/ACM international conference on distributed system platforms (Middleware). Heidelberg, Germany, pp 329–350, NovemberGoogle Scholar
  10. 10.
    Zhao BY, Kubiatowicz JD, Joseph AD (2001) Tapestry: an infrastructure for fault-tolerance wide-area location and routing. Tech. Rep. UCB/CSD-01-1141, Computer Science Division, University of California, Berkeley, AprilGoogle Scholar
  11. 11.
    Ratnasamy S, Francis P, Handley M, Karp R, Shenker S (2001) A scalable content-addressable network. In: Proceedings of ACM SIGCOMM. San Diego, CA, pp 161–172, AugustGoogle Scholar
  12. 12.
    Aguilera MK, Strom RE, Sturman DC, Astley M, Chandra TD (1999) Matching events in a content-based subscription system. In: Proceedings of the 8th ACM symposium on principles of distributed computing (PODC). Atlanta, GA, pp 53–61, MayGoogle Scholar
  13. 13.
    Li G, Hou S, Jacobsen H-A (2005) A unified approach to routing, covering and merging in publish/subscribe systems based on modified binary decision diagrams. In: Proceedings of international onference on distributed computing systems (ICDCS), Columbus, OH, JuneGoogle Scholar
  14. 14.
    Banavar G, Chandra T, Mukherjee B, Nagarajarao J, Strom RE, Sturman DC (1999) An efficient multicast protocol for content-based publish-subscribe systems. In: Proceedings of the 19th IEEE ICDCS. Washington, DC, pp 262–272, JuneGoogle Scholar
  15. 15.
    Carzaniga A, Rosenblum DS, Wolf AL (2001) Design and evaluation of a wide-area event notification service. ACM Trans Comput Syst 19(3):332–383CrossRefGoogle Scholar
  16. 16.
    Banavar G, Chandra T, Mukherjee B, Nagarajarao J, Strom RE, Sturman DC (1999) An efficient multicast protocol for content-based publish-subscribe systems. In: Proceedings of the 19th IEEE international conference on distributed computing systems(ICDCS). Austin, TX, pp 262–272, MayGoogle Scholar
  17. 17.
    Cao F, Singh JP (2004) Efficient event routing in content-basedpublish/subscribe service networks. In: Proceedings of INFOCOM, vol. 2. Hong Kong, China, pp 929–940, MarchGoogle Scholar
  18. 18.
    Zhang R, Hu YC (2005) HYPER: a hybrid approach to efficient content-based publish/subscribe. In: Proceedings of international conference on distributed computing systems (ICDCS). Columbus, OH, JuneGoogle Scholar
  19. 19.
    Rowstron AIT, Kermarrec A-M, Castro M, Druschel P (2001) SCRIBE: the design of a large-scale event notification infrastructure. In: Proceedings of the 3rd international networked group communication, pp 30–43Google Scholar
  20. 20.
    Zhuang SQ, Zhao BY, Joseph AD, Katz RH, Kubiatowicz J (2001) Bayeux: an architecture for scalable and fault-tolerant wide-area data dissemination. In: Proceedings of the eleventh international workshop on network and operating system support for digital audio and video (NOSSDAV). Port Jefferson, New York, pp 11–20, JuneGoogle Scholar
  21. 21.
    Castro M, Druschel P, Kermarrec A-M, Nandi A, Rowstron A, Singh A (2003) Splitstream: high-bandwidth multicast in cooperative environments. In: Proceedings of the 19th ACM symposium on operating systems principles (SOSP). Bolton Landing, NY, OctoberGoogle Scholar
  22. 22.
    Tam D, Azimi R, Jacobsen H-A (2003) Building content-based publish/subscribe systems with distributed hash tables. In: Proceedings of the international workshop on databases, information systems and peer-to-peer computing. Berlin, Germany, SeptemberGoogle Scholar
  23. 23.
    Terpstra WW, Behnel S, Fiege L, Zeidler A, Buchmann AP (2003) A peer-to-peer approach to content-based publish/subscribe. In: Proceedings of the second international workshop on distributed event-based systems (DEBS). San Diego, CA, JuneGoogle Scholar
  24. 24.
    Triantafillou P, Aekaterinidis I (2004) Content-based publish-subscribe over structured P2P networks. In: Proceedings of the third international workshop on distributed event-based systems (DEBS). Edinburgh, Scotland, UK, pp 104–109, MayGoogle Scholar
  25. 25.
    Perng G, Wang C, Reiter MK (2004) Providing content-based services in a peer-to-peer environment. In: Proceedings of the third international workshop on distributed event-based systems (DEBS). Edinburgh, Scotland, UK, pp 74–79, MayGoogle Scholar
  26. 26.
    Choi Y, Park K, Park D (2004) HOMED: a peer-to-peer overlay architecture for large-scale content-based publish/subscribe systems. In: Proceedings of the third international workshop on distributed event-based systems (DEBS). Edinburgh, Scotland, UK, pp 20–25, MayGoogle Scholar
  27. 27.
    Zhu Y, Hu Y (2007) Ferry: an P2P-based architecture for content-based publish/subscribe services. IEEE Trans Parallel Distrib Syst 18(5):672–685CrossRefGoogle Scholar
  28. 28.
    Bentley JL (1975) Multidimensional binary search trees used for associative searching. Commun ACM 18(9):509–517MATHCrossRefMathSciNetGoogle Scholar
  29. 29.
    Zhang C, Krishnamurthy A, Wang RY (2005) Brushwood: distributed trees in peer-to-peer systems. In: Proceedings of 4th international workshop on peer-to-peer systems (IPTPS), Ithaca, NY, FebruaryGoogle Scholar
  30. 30.
    Chawathe Y, Ramabhadran S, Ratnasamy S, LaMarca A, Shenker S, Hellerstein J (2005) A case study in building layered DHT applications. In: Proceedings of SIGCOMM. Philadelphia, PA, pp 97–108, AugustGoogle Scholar
  31. 31.
    Wang Y-M, Qiu L, Achlioptas D, Das G, Larson P, Wang HJ (2002) Subscription partitioning and routing in content-based publish/subscribe systems. In: Proceedings of the 16th international symposium on distributed computing (DISC). Toulouse, France, OctoberGoogle Scholar

Copyright information

© Springer Science + Business Media, LLC 2008

Authors and Affiliations

  1. 1.Department of CSSESeattle UniversitySeattleUSA
  2. 2.Computer Science and Computer Engineering DepartmentUniversity of ArkansasFayettevilleUSA

Personalised recommendations