Scaling Microblogging Services with Divergent Traffic Demands

  • Tianyin Xu
  • Yang Chen
  • Lei Jiao
  • Ben Y. Zhao
  • Pan Hui
  • Xiaoming Fu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7049)


Today’s microblogging services such as Twitter have long outgrown their initial designs as SMS-based social networks. Instead, a massive and steadily-growing user population of more than 100 million is using Twitter for everything from capturing the mood of the country to detecting earthquakes and Internet service failures. It is unsurprising that the traditional centralized client-server architecture has not scaled with user demands, leading to server overload and significant impairment of availability. In this paper, we argue that the divergence in usage models of microblogging services can be best addressed using complementary mechanisms, one that provides reliable messages between friends, and another that delivers events from popular celebrities and media outlets to their thousands or even millions of followers. We present Cuckoo, a new microblogging system that offloads processing and bandwidth costs away from a small centralized server base while ensuring reliable message delivery. We use a 20-day Twitter availability measurement to guide our design, and trace-driven emulation of 30,000 Twitter users to evaluate our Cuckoo prototype. Compared to the centralized approach, Cuckoo achieves 30-50% server bandwidth savings and 50-60% CPU load reduction, while guaranteeing reliable message delivery.


Server Cloud Media User Overlay Network Online Social Network Twitter User 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1. ‘Twitters’ Beat Media in Reporting China Earthquake (2008)Google Scholar
  2. 2.
    Buchegger, S., Schiöberg, D., Vu, L.-H., Datta, A.: PeerSoN: P2P Social Networking – Early Experiences and Insights. In: Proc. of SNS (2009)Google Scholar
  3. 3.
    Castro, M., Costa, M., Rowstron, A.: Debunking Some Myths about Structured and Unstructured Overlays. In: Proc. of NSDI (2005)Google Scholar
  4. 4.
    Cha, M., Haddadi, H., Benevenuto, F., Gummadi, K.P.: Measuring User Influence in Twitter: The Million Follower Fallacy. In: Proc. of ICWSM (2010)Google Scholar
  5. 5.
    CNET News. Twitter Crippled by Denial-of-Service Attack (2009)Google Scholar
  6. 6.
    Cutillo, L.A., Molva, R., Strufe, T.: Safebook: A Privacy Preserving Online Social Network Leveraging on Real-Life Trust. IEEE Communication Magazine 47(12), 94–101 (2009)CrossRefGoogle Scholar
  7. 7.
    Eugster, P.T., Guerraoui, R., Kermarrec, A.M., Massoulié, L.: From Epidemics to Distributed Computing. IEEE Computer 37, 60–67 (2004)CrossRefGoogle Scholar
  8. 8. San Francisco Twitter Users Shocked to Lose All Their Followers (2010)Google Scholar
  9. 9. Twitter Predicts Box-Office Sales Better Than a Prediction Market (2010)Google Scholar
  10. 10.
    Frey, D., Guerraoui, R., Kermarrec, A.-M., Koldehofe, B., Mogensen, M., Monod, M., Quéma, V.: Heterogeneous Gossip. In: Bacon, J.M., Cooper, B.F. (eds.) Middleware 2009. LNCS, vol. 5896, pp. 42–61. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  11. 11.
    Ghosh, S., Korlam, G., Ganguly, N.: The Effects of Restrictions on Number of Connections in OSNs: A Case-Study on Twitter. In: Proc. of WOSN (2010)Google Scholar
  12. 12. Twitter Election Predictions Are More Accurate Than YouGov (2010)Google Scholar
  13. 13.
    Gyarmati, L., Trinh, T.A.: Measuring User Behavior in Online Social Networks. IEEE Network Magazine, Special Issue on Online Social Networks 24(5), 26–31 (2010)Google Scholar
  14. 14.
    Java, A., Song, X., Finin, T., Tseng, B.: Why We Twitter: Understanding Microblogging Usage and Communities. In: Proc. of WEBKDD/SNA-KDD (2007)Google Scholar
  15. 15.
    Kermarrec, A.-M., Massoulié, L., Ganesh, A.: Probabilistic Reliable Dissemination in Large-Scale Systems. IEEE Transactions on Parallel and Distributed Systems 14(3), 248–258 (2003)CrossRefGoogle Scholar
  16. 16.
    Krishnamurthy, B., Gill, P., Arlitt, M.: A Few Chirps about Twitter. In: Proc. of WOSN (2008)Google Scholar
  17. 17.
    Kwak, H., Lee, C., Park, H., Moon, S.: What is Twitter, a Social Network or a News Media? In: Proc. of WWW (2010)Google Scholar
  18. 18.
    Mislove, A., Post, A., Druschel, P., Gummadi, K.P.: Ostra: Leveraging Trust to Thwart Unwanted Communication. In: Proc. of NSDI (2008)Google Scholar
  19. 19. Twitter CEO: 40 Percent of All Tweets Created on Mobile Devices (2011)Google Scholar
  20. 20.
    Motoyama, M., Meeder, B., Levchenko, K., Voelker, G.M., Savage, S.: Measuring Online Service Availability Using Twitter. In: Proc. of WOSN (2010)Google Scholar
  21. 21.
    New York Times. Sports Fans Break Records on Twitter (2010)Google Scholar
  22. 22.
    Ramasubramanian, V., Peterson, R., Sirer, E.G.: Corona: A High Performance Publish-Subscribe System for the World Wide Web. In: Proc. of NSDI (2006)Google Scholar
  23. 23.
    Reuters News. Twitter Snags over 100 Million Users, Eyes Money-Making (2010)Google Scholar
  24. 24.
    Rowstron, A., Druschel, P.: Pastry: Scalable, Decentralized Object Location and Routing for Large-Scale Peer-to-Peer Systems. In: Liu, H. (ed.) Middleware 2001. LNCS, vol. 2218, pp. 329–351. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  25. 25.
    Pingdom, R.: Twitter Growing Pains Cause Lots of Downtime in 2007 (2007)Google Scholar
  26. 26.
    Sandler, D., Mislove, A., Post, A., Druschel, P.: FeedTree: Sharing Web Micronews with Peer-to-Peer Event Notification. In: van Renesse, R. (ed.) IPTPS 2005. LNCS, vol. 3640, pp. 141–151. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  27. 27.
    Sandler, D.R., Wallach, D.S.: Birds of a FETHR: Open, Decentralized Micropublishing. In: Proc. of IPTPS (2009)Google Scholar
  28. 28.
    Shakimov, A., Varshavsky, A., Cox, L.P., Cáceres, R.: Privacy, Cost, and Availability Tradeoffs in Decentralized OSNs. In: Proc. of WOSN (2009)Google Scholar
  29. 29.
    Twitter Blog. Big Goals, Big Game, Big Records (2010)Google Scholar
  30. 30.
    Twitter Blog. What’s Happening with Twitter? (2010)Google Scholar
  31. 31.
    Twitter Engineering Blog. A Perfect Storm.....of Whales (2010)Google Scholar
  32. 32.
    Wang, F., Xiong, Y., Liu, J.: mTreebone: A Collaborative Tree-Mesh Overlay Network for Multicast Video Streaming. IEEE Transactions on Parallel and Distributed Systems 21(3), 379–392 (2010)CrossRefGoogle Scholar
  33. 33.
    Xu, T., Chen, Y., Fu, X., Hui, P.: Twittering by Cuckoo – Decentralized and Socio-Aware Online Microblogging Services. In: SIGCOMM Demo (2010)Google Scholar
  34. 34.
    Xu, T., Chen, Y., Jiao, L., Zhao, B.Y., Hui, P., Fu, X.: Cuckoo: Scaling Microblogging Services with Divergent Traffic Demands. Technical Report IFI-TB-2011-01, Univ. of Goettingen (2011)Google Scholar
  35. 35.
    Xu, T., Chen, Y., Zhao, J., Fu, X.: Cuckoo: Towards Decentralized, Socio-Aware Online Microblogging Services and Data Measurement. In: Proc. of HotPlanet (2010)Google Scholar
  36. 36.
    Zhao, B.Y., Huang, L., Stribling, J., Rhea, S.C., Joseph, A.D., Kubiatowicz, J.D.: Tapestry: A Resilient Global-Scale Overlay for Service Deployment. IEEE Journal on Selected Areas in Communications 22(1), 41–53 (2004)CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2011

Authors and Affiliations

  • Tianyin Xu
    • 1
  • Yang Chen
    • 1
  • Lei Jiao
    • 1
  • Ben Y. Zhao
    • 2
  • Pan Hui
    • 3
  • Xiaoming Fu
    • 1
  1. 1.University of GoettingenGermany
  2. 2.U.C. Santa BarbaraUSA
  3. 3.Deutsche Telekom LaboratoriesGermany

Personalised recommendations