Skip to main content

Advertisement

SpringerLink
  1. Home
  2. Journal of Internet Services and Applications
  3. Article
Challenges in very large distributed systems
Download PDF
Your article has downloaded

Similar articles being viewed by others

Slider with three articles shown per slide. Use the Previous and Next buttons to navigate the slides or the slide controller buttons at the end to navigate through each slide.

Introduction Address by the Program Committee of the Conference “Management of the Development of Large-Scale Systems”

01 May 2022

S. N. Vasil’ev

The evolution of distributed computing systems: from fundamental to new frontiers

30 January 2021

Dominic Lindsay, Sukhpal Singh Gill, … Peter Garraghan

Replication in Distributed Systems: Models, Methods, and Protocols

01 September 2020

A. R. Nasibullin & B. A. Novikov

Dynamo: Handling Scientific Data Across Sites and Storage Media

12 April 2021

Yutaro Iiyama, Benedikt Maier, … Christoph Paus

Rucio: Scientific Data Management

09 August 2019

Martin Barisits, Thomas Beermann, … Tobias Wegner

Interactive checks for coordination avoidance

05 September 2020

Michael Whittaker & Joseph M. Hellerstein

Heterogeneous Load Balancing using Predictive Load Summarization

28 February 2022

Prasad Velpula, Rajendra Pamula, … Amjan Shaik

BOINC: A Platform for Volunteer Computing

16 November 2019

David P. Anderson

Power measurement techniques for energy-efficient computing: reconciling scalability, resolution, and accuracy

23 April 2018

Thomas Ilsche, Robert Schöne, … Wolfgang E. Nagel

Download PDF
  • SI: FOME - The Future of Middleware
  • Open Access
  • Published: 19 November 2011

Challenges in very large distributed systems

  • Maarten van Steen1,
  • Guillaume Pierre1 &
  • Spyros Voulgaris1 

Journal of Internet Services and Applications volume 3, pages 59–66 (2012)Cite this article

  • 3086 Accesses

  • 9 Citations

  • Metrics details

Abstract

Many modern distributed systems are required to scale in terms of their support for processes, resources, and users. Moreover, a system is often also required to operate across the Internet and across different administrative domains. These scalability requirements lead to a number of well-known challenges in which distribution transparency needs to be traded off against loss of performance. We concentrate on two major challenges for which we claim there is no easy solution. These challenges originate from the fact that users and system are becoming increasingly integrated and are effectively leading us to large-scale socio-technical distributed systems. We identify the design of such integrated systems as one challenge, in particular when it comes to placing humans in the loop as a necessity to proper operation of the system as a whole. As users are so tightly integrated into the overall design, and systems naturally expand through composition, we will be facing problems with respect to long-term management, which we identify as another major challenge.

Download to read the full article text

References

  1. Amazon Web Services (2011) Summary of the Amazon EC2 and Amazon RDS service disruption in the US east region. http://aws.amazon.com/message/65648/

  2. Atzori L, Iera A, Morabito G (2010) The internet of things: a survey. Comput Netw 54(15):2787–2805

    Article  Google Scholar 

  3. Bergstra J, Burgess M (eds) (2007) Handbook of network and system administration. Elsevier, Amsterdam

    Google Scholar 

  4. Chen H, Jiang G, Zhang H, Yoshihira K (2010) A cooperative sampling approach to discovering optimal configurations in large scale computing systems. In: Proc. 29th IEEE international symposium on reliable distributed systems

    Google Scholar 

  5. Dey A (2010) Context-aware computing. In: Krumm J (ed) Ubiquitous computing fundamentals. CRC Press, Boca Raton, pp 321–352

    Google Scholar 

  6. Gilbert S, Lynch N (2002) Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News 33(2):51–59

    Article  Google Scholar 

  7. Huijsman R-J (Aug. 2011) An investigation into evolving distributed systems. Master’s thesis. VU University Amsterdam

  8. Jelasity M, Voulgaris S, Guerraoui R, Kermarrec A-M, van Steen M (2007) Gossip-based peer sampling. ACM Trans Comput Syst 25(3)

  9. Kreitz G, Niemelä F (2010) Spotify—large scale, low latency, P2P music-on-demand streaming. In: Proc. 10th international conference on peer-to-peer computing, Aug. 2010. IEEE Comput Soc, Los Alamitos, pp 266–275

    Google Scholar 

  10. Lewis TG (2009) Network science: theory and practice. Wiley, New York

    Book  Google Scholar 

  11. Madan A, Cebrian M, Moturu S, Farrahi K, Pentland A (2011) Sensing the ‘Health state’ of our society. Technical Report TR-663, MIT, Cambridge, MA

  12. Madden SR, Franklin MJ, Hellerstein JM, Hong W (2005) TinyDB: an acquisitional query processing system for sensor networks. ACM Trans Database Syst 30(1):122–173

    Article  Google Scholar 

  13. McSherry F, Mironov I (2009) Differentially private recommender systems: building privacy into the Netflix prize contenders. In: Proc. 15th international conference knowledge discovery and data mining (KDD), June 2009. ACM, New York, pp 627–637

    Chapter  Google Scholar 

  14. Moran S, Nakata K (2010) Ubiquitous monitoring and user behaviour: a preliminary model. J Ambient Intell Smart Environ 2(1):67–80

    Google Scholar 

  15. Mottola L, Picco GP (2011) Programming wireless sensor networks: fundamental concepts and state of the art. ACM Comput Surv 43(3):19

    Article  Google Scholar 

  16. Newman M (2010) Networks, an introduction. Oxford University Press, Oxford

    Book  Google Scholar 

  17. Ogston E, Bakker A, van Steen M (2006) On the value of random opinions in decentralized recommendation. In: Proc. 6th international conference on distributed applications and interoperable systems. Lecture notes in computer science, vol 4025, pp 84–98. Springer, Berlin

    Chapter  Google Scholar 

  18. Olguín DO, Pentland AS (2010) Sensor-based organisational design and engineering. Int J Organ Des Eng 1(1/2):69–97

    Google Scholar 

  19. Poslad S (2009) Ubiquitous computing: smart devices, environments and interactions. Wiley, New York

    Book  Google Scholar 

  20. Qin F, Tucek J, Sundaresan J, Zhou Y (2005) Rx: treating bugs as allergies—a safe method to survive software failures. In: Proc. 20th symposium on operating system principles, Oct. 2005. ACM, New York, pp 235–248

    Google Scholar 

  21. Ramakrishnan N, Keller BJ, Mirza BJ, Grama AY, Karypis G (2001) Privacy risks in recommender systems. IEEE Internet Comput 5:54–62

    Article  Google Scholar 

  22. Urdaneta G, Pierre G, van Steen M (2011) A survey of DHT security techniques. ACM Comput Surv 43(2)

  23. van Steen M, Pierre G (2010) Replicating for performance: case studies. In: Charron-Bost B, Pedone F, Schiper A (eds) Replication, theory and practice. Lecture notes in computer science, vol 5959. Springer, Berlin, pp 73–89. Chapter 5

    Google Scholar 

  24. Voulgaris S, van Steen M, Iwanicki K (2007) Proactive gossip-based management of semantic overlay networks. Concurr Comput 19(17):2299–2311

    Article  Google Scholar 

  25. Wams J, van Steen M (2004) Unifying user-to-user messaging systems. IEEE Internet Comput 8(2):76–82

    Article  Google Scholar 

  26. Weiser M (1991) The computer for the 21st century. Sci Am September:67–83

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. VU University Amsterdam, Amsterdam, The Netherlands

    Maarten van Steen, Guillaume Pierre & Spyros Voulgaris

Authors
  1. Maarten van Steen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guillaume Pierre
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Spyros Voulgaris
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maarten van Steen.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

van Steen, M., Pierre, G. & Voulgaris, S. Challenges in very large distributed systems. J Internet Serv Appl 3, 59–66 (2012). https://doi.org/10.1007/s13174-011-0043-x

Download citation

  • Received: 26 October 2011

  • Accepted: 09 November 2011

  • Published: 19 November 2011

  • Issue Date: May 2012

  • DOI: https://doi.org/10.1007/s13174-011-0043-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

  • Distributed systems
  • Socio-technical systems
  • Cloud computing
Download PDF

Advertisement

Over 10 million scientific documents at your fingertips

Switch Edition
  • Academic Edition
  • Corporate Edition
  • Home
  • Impressum
  • Legal information
  • Privacy statement
  • California Privacy Statement
  • How we use cookies
  • Manage cookies/Do not sell my data
  • Accessibility
  • FAQ
  • Contact us
  • Affiliate program

Not affiliated

Springer Nature

© 2023 Springer Nature Switzerland AG. Part of Springer Nature.