DDSS: A Low-Overhead Distributed Data Sharing Substrate for Cluster-Based Data-Centers over Modern Interconnects

  • Karthikeyan Vaidyanathan
  • Sundeep Narravula
  • Dhabaleswar K. Panda
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4297)


Information-sharing is a key aspect of distributed applications such as database servers and web servers. Information-sharing also assists services such as caching, reconfiguration, etc. In the past, information-sharing has been implemented using ad-hoc messaging protocols which often incur high overheads and are not very scalable. This paper presents a new design for a scalable and a low-overhead Distributed Data Sharing Substrate (DDSS). DDSS is designed to support efficient data management and coherence models by leveraging the features of modern interconnects. It is implemented over the OpenFabrics interface and portable across multiple interconnects including iWARP-capable networks in LAN/WAN environments. Experimental evaluations with networks like InfiniBand and iWARP-capable Ammasso through data-center services show an order of magnitude performance improvement and the load resilient nature of the substrate. Application-level evaluations with Distributed STORM achieves close to 19% performance improvement over traditional implementation, while evaluations with check-pointing application suggest that DDSS is highly scalable.


Remote Node Coherence Model Atomic Operation Cooperative Cache Free Block 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Shah, H.V., Minturn, D.B., Foong, A., McAlpine, G.L., Madukkarumukumana, R.S., Regnier, G.J.: CSP: A Novel System Architecture for Scalable Internet and Communication Services. In: 3rd USENIX Symposium on Internet Technologies and Systems (2001)Google Scholar
  2. 2.
    Tang, C., Chen, D., Dwarkadas, S., Scott, M.: Integrating Remote Invocation and Distributed Shared State (2004)Google Scholar
  3. 3.
    InfiniBand Trade Association,
  4. 4.
    OpenFabrics Alliance: OpenFabrics,
  5. 5.
    Shah, H.V., Pinkerton, J., Recio, R., Culley, P.: DDP over Reliable Transports (2002)Google Scholar
  6. 6.
    Ammasso, inc.,
  7. 7.
    Narravula, S., Balaji, P., Vaidyanathan, K., Krishnamoorthy, S., Wu, J., Panda, D.K.: Supporting Strong Coherency for Active Caches in Data-Centers in InfiniBand. SAN (2004)Google Scholar
  8. 8.
    Zhang, Y., Zheng, W.: User-level communication based cooperative caching. In: ACM SIGOPS Operating Systems (2003)Google Scholar
  9. 9.
    Narravula, S., Jin, H.W., Vaidyanathan, K., Panda, D.K.: Designing Efficient Cooperative Caching Schemes for Data-Centers over RDMA-enabled Networks. In: CCGRID (2005)Google Scholar
  10. 10.
    Chen, D., Tang, C., Sanders, B., Dwarkadas, S., Scott, M.: Exploiting high-level coherence information to optimize distributed shared state. In: Proc. of the 9th ACM Symp. on Principles and Practice of Parallel Programming (2003)Google Scholar
  11. 11.
    Vaidyanathan, K., Jin, H.W., Panda, D.K.: Exploiting RDMA Operations for Providing Efficient Fine-Grained Resource Monitoring in Cluster-based Servers. In: Workshop on Remote Direct Memory Access (RDMA), RAIT, Barcelona, Spain (2006)Google Scholar
  12. 12.
    The STORM Project at OSU BMI,
  13. 13.
    Vaidyanathan, K., Narravula, S., Panda, D.K.: Soft Shared State Primitives for Multi-Tier Data-Center Services. Technical Report OSU-CISRC-1/06-TR06, OSU (2006)Google Scholar
  14. 14.
    Balaji, P., Vaidyanathan, K., Narravula, S., Savitha, K., Jin, H.W., Panda, D.K.: Exploiting Remote Memory Operations to Design Efficient Reconfiguration for Shared Data-Centers. In: Workshop on RAIT, San Diego, CA (2004)Google Scholar
  15. 15.
    Chen, D., Dwarkadas, S., Parthasarathy, S., Pinheiro, E., Scott, M.L.: InterWeave: A Middleware System for Distributed Shared State. In: Dwarkadas, S. (ed.) LCR 2000. LNCS, vol. 1915, pp. 207–220. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  16. 16.
    Carter, J., Ranganathan, A., Susarla, S.: Khazana: An Infrastructure for Building Distributed Services. In: ICDCS (1998)Google Scholar
  17. 17.
    Parthasarathy, S., Dwarkadas, S.: InterAct: Virtual Sharing for Interactive Client-Server Application. In: Workshop on Languages, Compilers, and Systems for Computers (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Karthikeyan Vaidyanathan
    • 1
  • Sundeep Narravula
    • 1
  • Dhabaleswar K. Panda
    • 1
  1. 1.Department of Computer Science and EngineeringThe Ohio State University 

Personalised recommendations