Transparent Information Dissemination

  • Amol Nayate
  • Mike Dahlin
  • Arun Iyengar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3231)


This paper describes Transparent Replication through Invalidation and Prefetching (TRIP), a self tuning data replication middleware system that enables transparent replication of large-scale information dissemination services. The TRIP middleware is a key building block for constructing information dissemination services, a class of services where updates occur at an origin server and reads occur at a number of replicas; examples information dissemination services include content distribution networks such as Akamai [1] and IBM’s Sport and Event replication system [2]. Furthermore, the TRIP middleware can be used to build key parts of general applications that distribute content such as file systems, distributed databases, and publish-subscribe systems.

Our data replication middleware supports transparent replication by providing two crucial properties: (1) sequential consistency to avoid introducing anomalous behavior to increasingly complex services and (2) self-tuning transmission of updates to maximize performance and availability given available system resources. Our analysis of simulations and our evaluation of a prototype support the hypothesis that it is feasible to provide transparent replication for dissemination services. For example, in simulations, our system’s performance is a factor of three to four faster than a demand-based middleware system for a wide range of configurations.


  1. 1.
    Turbo-charging dynamic web data with akamai edgesuite. Akamai White Paper (2001)Google Scholar
  2. 2.
    Challenger, J., Dantzig, P., Iyengar, A.: A scalable and highly available system for serving dynamic data at frequently accessed web sites. In: ACM/IEEE, Supercomputing 1998 (1998)Google Scholar
  3. 3.
    Kokku, R., Yalagandula, P., Venkatramani, A., Dahlin, M.: Nps: A non-interfering deployable web prefetching system. In: 4th USENIX Symposium on Internet Technologies and Systems (2003)Google Scholar
  4. 4.
    Lamport, L.: How to make a multiprocessor computer that correctly executes multiprocess programs. IEEE Transactions on Computers C-28, 690–691 (1979)CrossRefMATHGoogle Scholar
  5. 5.
    Frigo, M., Luchangco, V.: Computation-Centric Memory Models. In: Tenth Annual ACM Symposium on Parallel Algorithms and Architectures (1998)Google Scholar
  6. 6.
    Hill, M.: Multiprocessors should support simple memory consistency models. In: IEEE Computer (1998)Google Scholar
  7. 7.
    Brewer, E.: Lessons from giant-scale services. IEEE Internet Computing (2001)Google Scholar
  8. 8.
    Lipton, R., Sandberg, J.: PRAM: A scalable shared memory. Technical Report CS-TR-180-88, Princeton (1988)Google Scholar
  9. 9.
    Gao, L., Dahlin, M., Nayate, A., Zheng, J., Iyengar, A.: Application specific data replication for edge services. In: International World Wide Web Conference (2003)Google Scholar
  10. 10.
    Sydney olympic games web site, – site is no longer available (2000)
  11. 11.
    Mazières, D.: A toolkit for user-level file systems. In: 2001 USENIX Technical Conference, pp. 261–274 (2001)Google Scholar
  12. 12.
    Awadallah, A., Rosenblum, M.: The vMatrix: A network of virtual machine monitors for dynamic content distribution. In: Internat. Workshop on Web Caching and Content Distribution (2002)Google Scholar
  13. 13.
    Vahdat, A., Dahlin, M., Anderson, T., Aggarwal, A.: Active Naming: Flexible Location and Transport of Wide-Area Resources. In: 2nd USENIX Symposium on Internet Technologies and Systems (1999)Google Scholar
  14. 14.
    Whitaker, A., Shaw, M., Gribble, S.: Denali: Lightweight virtual machines for distributed and networked applications. In: 2002 USENIX Technical Conference (2002)Google Scholar
  15. 15.
    Adve, S., Gharachorloo, K.: Shared memory consistency models: A tutorial. IEEE Computer 29, 66–76 (1996)CrossRefGoogle Scholar
  16. 16.
    Burns, R., Rees, R., Long, D.: Consistency and locking for distributing updates to web servers using a file system. In: Workshop on Performance and Architecture of Web Servers (2000)Google Scholar
  17. 17.
    Lamport, L.: Time, clocks, and the ordering of events in a distributed system. Comm. of the ACM 21 (1978)Google Scholar
  18. 18.
    Nayate, A., Dahlin, M., Iyengar, A.: Integrating Prefetching and Invalidation for Transparent Replication of Dissemination Services. Technical Report TR-03-44, University of Texas at Austin (2003)Google Scholar
  19. 19.
    Terry, B., Demers, A., Petersen, K., Spreitzer, M.J., Theimer, M., Welch, B.: Session guarantees for weakly consistent replicated data. In: International Conference on Parallel and Distributed Information Systems, pp. 140–149 (1994)Google Scholar
  20. 20.
    Duchamp, D.: Prefetching Hyperlinks. In: 2nd USENIX Symposium on Internet Technologies and Systems (1999)Google Scholar
  21. 21.
    Gwertzman, J., Seltzer, M.: The case for geographical pushcaching. In: HOTOS 1995, pp. 51–55 (1995)Google Scholar
  22. 22.
    Padmanabhan, V., Mogul, J.: Using Predictive Prefetching to Improve World Wide Web Latency. In: ACM SIGCOMM Conference, pp. 22–36 (1996)Google Scholar
  23. 23.
    Venkataramani, A., Yalagandula, P., Kokku, R., Sharif, S., Dahlin, M.: Potential costs and benefits of long-term prefetching for content-distribution. In: Web Caching and Content Distribution Workshop (2001)Google Scholar
  24. 24.
    Venkataramani, A., Kokku, R., Dahlin, M.: TCP-Nice: A mechanism for background transfers. In: OSDI 2002 (2002)Google Scholar
  25. 25.
    Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Misinter, L., Leach, P., Berners-Lee, T.: Hypertext Transfer Protocol – HTTP/1.1. RFC 2616, IETF (1999)Google Scholar
  26. 26.
    Howard, J., Kazar, M., Menees, S., Nichols, D., Satyanarayanan, M., Sidebotham, R., West, M.: Scale and Performance in a Distributed File System. ACM Trans. on Computer Systems 6, 51–81 (1988)CrossRefGoogle Scholar
  27. 27.
    Mills, D.: Network time protocol (version 3) specification, implementation and analysis. Technical report, IETF (1992)Google Scholar
  28. 28.
    Yalagandula, P., Dahlin, M.: SDIMS: A scalable distributed information management system. Technical Report TR-03-47, University of Texas Dept. of CS (2003)Google Scholar
  29. 29.
    MQSeries: An introduction to messaging and queueing. IBM Corporation GC33-0805-01 (1995)Google Scholar
  30. 30.
    Chen, M., LaPaugh, A., Singh, J.P.: Content distribution for publish/subscribe services. In: Proceedings of the International Middleware Conference (2003)Google Scholar
  31. 31.
    White, B., Lepreau, J., Stoller, L., Ricci, R., Guruprasad, S., Newbold, M., Hibler, M., Barb, C., Joglekar, A.: An integrated experimental environment for distributed systems and networks. In: 5th Symp on Operating Systems Design and Impl. (2002)Google Scholar
  32. 32.
    Chankhunthod, A., Danzig, P., Neerdaels, C., Schwartz, M., Worrell, K.: A Hierarchical Internet Object Cache. In: 1996 USENIX Technical Conference (1996)Google Scholar
  33. 33.
    Petersen, K., Spreitzer, M., Terry, D., Theimer, M., Demers, A.: Flexible Update Propagation for Weakly Consistent Replication. In: 16th ACM Symposium on Operating Systems Principles (1997)Google Scholar
  34. 34.
    Davison, B.D., Liberatore, V.: Pushing politely: ImprovingWeb responsiveness one packet at a time (extended abstract). Performance Evaluation Review 28, 43–49 (2000)CrossRefGoogle Scholar
  35. 35.
    Li, D., Cheriton, D.R.: Scalable web caching of frequently updated objects using reliable multicast. In: Proceedings of the 1999 Usenix Symposium on Internet Technologies and Systems, USITS 1999 (1999)Google Scholar
  36. 36.
    Worrell, K.: Invalidation in Large Scale Network Object Caches. Master’s thesis, University of Colorado, Boulder (1994)Google Scholar
  37. 37.
    Yin, J., Alvisi, L., Dahlin, M., Lin, C.: Volume Leases to Support Consistency in Large-Scale Systems. IEEE Transactions on Knowledge and Data Engineering 11, 563–576 (1999)CrossRefGoogle Scholar
  38. 38.
    Yin, J., Alvisi, L., Dahlin, M., Iyengar, A.: Engineering web cache consistency. ACM Transactions on Internet Technologies 2 (2002)Google Scholar
  39. 39.
    Bradley, A., Bestavros, A.: Basis token consistency: Supporting strong web cache consistency. In: GLOBECOMM (2003)Google Scholar
  40. 40.
    Challenger, J., Dantzig, P., Iyengar, A., Squillante, M., Zhang, L.: Efficiently serving dynamic data at highly accessed web sites. IEEE/ACM Transactions on Networking 12, 233–246 (2004)CrossRefGoogle Scholar
  41. 41.
    Fei, Z.: A novel approach to managing consistency in content distribution networks. In: Internat. Workshop on Web Caching and Content Distribution (2001)Google Scholar
  42. 42.
    Acharya, S., Franklin, M., Zdonik, S.: Balancing push and pull for data broadcast. In: Proceedings of the 1997 ACM SIGMOD international conference on Management of data, pp. 183–194. ACM Press, New York (1997)CrossRefGoogle Scholar
  43. 43.
    Ladin, R., Liskov, B., Shrira, L., Ghemawat, S.: Providing high availability using lazy replication. ACM Trans. on Computer Systems 10, 360–361 (1992)CrossRefGoogle Scholar
  44. 44.
    Yu, H., Vahdat, A.: The costs and limits of availability for replicated services. In: 18th ACM Symposium on Operating Systems Principles (2001)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2004

Authors and Affiliations

  • Amol Nayate
    • 1
  • Mike Dahlin
    • 1
  • Arun Iyengar
    • 2
  1. 1.University of Texas at AustinAustinUSA
  2. 2.IBM TJ Watson Research CenterYorktown HeightsUSA

Personalised recommendations