Skip to main content

Distributed and Decentralized Orchestration of Containers on Edge Clouds


Cloud Computing has been successful in providing substantial amounts of resources to deploy scalable and highly available applications. However, there is a growing necessity of lower latency services and cheap bandwidth access to accommodate the expansion of IoT and other applications that reside at the internet’s edge. The development of community networks and volunteer computing, together with the today’s low cost of compute and storage devices, is making the internet’s edge filled with a large amount of still underutilized resources. Due to this, new computing paradigms like Edge Computing and Fog Computing are emerging. This work presents Caravela a Docker’s container orchestrator that utilizes volunteer edge resources from users to build an Edge Cloud where it is possible to deploy applications using standard Docker containers. Current cloud solutions are mostly tied to a centralized cluster environment deployment. Caravela employs a completely decentralized architecture, resource discovery and scheduling algorithms to cope with (1) the large amount of volunteer devices, volatile environment, (2) wide area networks that connects the devices and (3) nonexistent natural central administration.

This is a preview of subscription content, access via your institution.


  1. 1.

    Anderson, D.P., Cobb, J., Korpela, E., Lebofsky, M., Werthimer, D.: SETI@Home: an experiment in public-resource computing. Commun. ACM 45(11), 56–61 (2002).

    Article  Google Scholar 

  2. 2.

    Apolónia, N., Ferreira, P., Veiga, L.: Trans-social networks for distributed processing. In: Bestak, R., Kencl, L., Li, L.E., Widmer, J., Yin, H. (eds.) Networking 2012 - 11th International IFIP TC 6 Networking Conference, Prague, Czech Republic, May 21-25, 2012, Proceedings, Part I, Lecture Notes in Computer Science, vol. 7289, pp 82–96. Springer (2012).

  3. 3.

    Apolónia, N., Freitag, F., Navarro, L., Girdzijauskas, S.: Socially aware microcloud service overlay optimization in community networks. Softw. Pract. Exp. 50(5), 675–687 (2020).

    Article  Google Scholar 

  4. 4.

    Babaoglu, O., Marzolla, M., Tamburini, M.: Design and implementation of a P2P Cloud system. In: Proceedings of the 27th Annual ACM Symposium on Applied Computing - SAC ’12, p 412 (2012).

  5. 5.

    Baig, R., Roca, R., Freitag, F., Navarro, L.:, a crowdsourced network infrastructure held in common. Comput. Netw. (2015)

  6. 6.

    Benet, J.: IPFS-Content Addressed, Versioned, P2P File System. IPFS-Content Addressed, Versioned, P2P File System (Draft 3). (2014)

  7. 7.

    Bittencourt, L., Immich, R., Sakellariou, R., Fonseca, N., Madeira, E., Curado, M., Villas, L., DaSilva, L., Lee, C., Rana, O.: The internet of things, fog and cloud continuum: Integration and challenges. Int. Things 3-4, 134–155 (2018).

    Article  Google Scholar 

  8. 8.

    Bonomi, F., Milito, R., Zhu, J., Addepalli, S.: Fog computing and its role in the internet of things. In: Proceedings of the first edition of the MCC workshop on Mobile cloud computing - MCC ’12, p 13 (2012).

  9. 9.

    Cardosa, M., Chandra, A.: Resource bundles: Using aggregation for statistical large-scale resource discovery and management. IEEE Trans. Parall. Distribut. Syst. 21(8), 1089–1102 (2010).

    Article  Google Scholar 

  10. 10.

    Castro, P., Ishakian, V., Muthusamy, V., Slominski, A.: The rise of serverless computing. Commun. ACM 62(12), 44–54 (2019).

    Article  Google Scholar 

  11. 11.

    Chang, H., Hari, A., Mukherjee, S., Lakshman, T.V.: Bringing the cloud to the edge. In: Proceedings - IEEE INFOCOM, pp 346–351 (2014).

  12. 12.

    Cisco Systems: Fog Computing and the Internet of Things: Extend the Cloud to Where the Things Are. White Paper p. 6 (2016).

  13. 13.

    Cunsolo, V.D., Distefano, S., Puliafito, A., Scarpa, M. Huang, D.S., Jo, K.H., Lee, H.H., Kang, H.J., Bevilacqua, V. (eds.): Cloud@Home: Bridging the gap between volunteer and cloud computing. Springer, Berlin (2009)

  14. 14.

    Huedo, E., Montero, R.S., Moreno-Vozmediano, R., Vázquez, C., Holer, V., Llorente, I.M.: Opportunistic deployment of distributed edge clouds for latency-critical applications. J. Grid Comput. 19(1), 2 (2021).

    Article  Google Scholar 

  15. 15.

    Kabbinale, A.R., Dimogerontakis, E., Selimi, M., Ali, A., Navarro, L., Sathiaseelan, A., Crowcroft, J.: Blockchain for economically sustainable wireless mesh networks. Concurr. Comput. Pract. Exp. 32(12). (2020)

  16. 16.

    Kargar, S., Mohammad-Khanli, L.: Fractal: An advanced multidimensional range query lookup protocol on nested rings for distributed systems. J. Netw. Comput. Appl. 87, 147–168 (2017).

    Article  Google Scholar 

  17. 17.

    Khan, A.M., Freitag, F., Rodrigues, L.: Current trends and future directions in community edge clouds. In: 2015 IEEE 4Th International Conference on Cloud Networking, Cloudnet 2015, pp 239–241 (2015).

  18. 18.

    Kochovski, P., Stankovski, V., Gec, S., Faticanti, F., Savi, M., Siracusa, D., Kum, S.: Smart contracts for service-level agreements in edge-to-cloud computing. J. Grid Comput. 18 (4), 673–690 (2020).

    Article  Google Scholar 

  19. 19.

    Mayer, P., Klarl, A., Hennicker, R., Puviani, M., Tiezzi, F., Pugliese, R., Keznikl, J., Bure, T.: The autonomic cloud: A vision of voluntary, Peer-2-Peer cloud computing. In: Proceedings - IEEE 7th International Conference on Self-Adaptation and Self-Organizing Systems Workshops, SASOW 2013, pp 89–94 (2014).

  20. 20.

    Mell, P., Grance, T.: The NIST definition of cloud computing recommendations of the national institute of standards and technology. Nist Special Publication 145, 7 (2011).

    Google Scholar 

  21. 21.

    Mohan, N., Kangasharju, J.: Edge-fog cloud: A distributed cloud for Internet of Things computations. 2016 Cloudification of the Internet of Things CIoT 2016. (2017)

  22. 22.

    Nakamoto, S.: Bitcoin: A Peer-to-Peer Electronic Cash System. p. 9. (2008)

  23. 23.

    Peinl, R., Holzschuher, F., Pfitzer, F.: Docker cluster management for the cloud - survey results and own solution. J. Grid Comput. 14(2), 265–282 (2016).

    Article  Google Scholar 

  24. 24.

    Pouwelse, J., Garbacki, P., Epema, D., Sips, H.: The Bittorrent P2P file-sharing system: measurements and analysis. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 3640, pp 205–216. LNCS (2005).

  25. 25.

    Ryden, M., Oh, K., Chandra, A., Weissman, J.: Nebula: Distributed edge cloud for data intensive computing. In: Proceedings - 2014 IEEE International Conference on Cloud Engineering, IC2E 2014, pp 57–66 (2014).

  26. 26.

    Satyanarayanan, M., Bahl, P., Cáceres, R., Davies, N.: The case for VM-based cloudlets in mobile computing. IEEE Pervas. Comput. 8(4), 14–23 (2009).

    Article  Google Scholar 

  27. 27.

    Selimi, M., Cerdà-Alabern, L., Freitag, F., Veiga, L., Sathiaseelan, A., Crowcroft, J.: A lightweight service placement approach for community network micro-clouds. J. Grid Comput. 17(1), 169–189 (2019).

    Article  Google Scholar 

  28. 28.

    Selimi, M., Cerda-Alabern, L., Sanchez-Artigas, M., Freitag, F., Veiga, L.: Practical service placement approach for microservices architecture. In: Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017, pp 401–410 (2017).

  29. 29.

    Shi, W., Cao, J., Zhang, Q., Li, Y., Xu, L.: Edge computing: Vision and challenges. IEEE Int. Things J. 3(5), 637–646 (2016).

    Article  Google Scholar 

  30. 30.

    Singh, S., Chana, I.: A survey on resource scheduling in cloud computing: Issues and challenges. J. Grid Comput. 14(2), 217–264 (2016).

    Article  Google Scholar 

  31. 31.

    Stoica, I., Morris, R., Liben-Nowell, D., Karger, D.R., Kaashoek, M.F., Dabek, F., Balakrishnan, H.: Chord: A scalable peer-to-peer lookup protocol for Internet applications. IEEE/ACM Trans. Netw. 11(1), 17–32 (2003).

    Article  Google Scholar 

  32. 32.

    Vaquero, L.M., Rodero-Merino, L.: Finding your Way in the Fog: Towards a Comprehensive Definition of Fog Computing. ACM SIGCOMM Comput. Commun. Rev. 44(5), 27–32 (2014).

    Article  Google Scholar 

  33. 33.

    Varghese, B., Buyya, R.: Next generation cloud computing: New trends and research directions. Future Gener. Comput. Syst. 79, 849–861 (2018).

    Article  Google Scholar 

  34. 34.

    Verbelen, T., Simoens, P., Turck, F.D., Dhoedt, B.: Cloudlets : Bringing the cloud to the mobile user. In: Proceedings of the third ACM workshop on Mobile cloud computing and services, pp 29–36 (2012).

  35. 35.

    Vishnumurthy, V., Chandrakumar, S., Emin, G.: Karma: a secure economic framework for peer-to-peer resource sharing. In: Workshop on Economics of Peer-to-peer Systems, p 34 (2003)

  36. 36.

    Yang, S., Butt, A.R., Fang, X., Hu, Y.C., Midkiff, S.P.: A fair, secure and trustworthy peer-to-peer based cycle-sharing system. J. Grid Comput. 4(3), 265–286 (2006).

    Article  Google Scholar 

Download references


This work was supported by national funds through Fundação para a Ciência e a Tecnologia with reference PTDC/EEI-COM/30644/2017 and UIDB/50021/2020, and by Instituto Superior de Engenharia de Lisboa and Instituto Politécnico de Lisboa.

Author information



Corresponding author

Correspondence to José Simão.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Caravela (a.k.a Portuguese man o’war) is a colony of multi-cellular organisms that barely survive alone, so they need to work together to function like a single viable animal.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Pires, A., Simão, J. & Veiga, L. Distributed and Decentralized Orchestration of Containers on Edge Clouds. J Grid Computing 19, 36 (2021).

Download citation


  • Edge cloud
  • Cloud computing
  • Volunteer computing
  • Resource scheduling
  • Resource discovery
  • Docker