Abstract
Automated deployment and run-time management of microservices-based applications in cloud computing environments is relatively well studied with several mature solutions. However, managing such applications and tasks in the cloud-to-edge continuum is far from trivial, with no robust, production-level solutions currently available. This paper presents our first attempt to extend an application-level cloud orchestration framework called MiCADO to utilise edge and fog worker nodes. The paper illustrates how MiCADO-Edge can automatically deploy complex sets of interconnected microservices in such multi-layered cloud-to-edge environments. Additionally, it shows how monitoring information can be collected from such services and how complex, user- defined run-time management policies can be enforced on application components running at any layer of the architecture. The implemented solution is demonstrated and evaluated using two realistic case studies from the areas of video processing and secure healthcare data analysis.
References
Gartner forecasts worldwide public cloud revenue to grow 17% in 2020 (2019). https://www.gartner.com/en/newsroom/press-releases/2019-11-13-gartner-forecasts-worldwide-public-cloud-revenue-to-grow-17-percent-in-2020, Accessed 5 Oct 2020
Marston, S., Li, Z., Bandyopadhyay, S., Ghalsasi, A.: Cloud computing - the business perspective. In: 2011 44th Hawaii International Conference on System Sciences, pp 1–11 (2011)
Kubeedge (2020). https://kubeedge.io/en/, Accessed 4 Oct 2020
Project eve (2020). https://www.lfedge.org/projects/eve/, Accessed 4 Oct 2020
Goethals, T., De Turck, F., Volckaert, B.: Fledge: Kubernetes compatible container orchestration on low-resource edge devices. In: Internet of vehicles : technologies and services toward smart cities, 6th International Conference, IOV 2019, Proceedings, pp 174–189. Springer (2020)
Yousefpour, A., Fung, C., Nguyen, T., Kadiyala, K., Jalali, F., Niakanlahiji, A., Kong, J., Jue, J.P.: All one needs to know about fog computing and related edge computing paradigms: A complete survey. J. Syst Architect. 98, 289–330 (2019). https://doi.org/10.1016/j.sysarc.2019.02.009
Mercer, D.: Global connected and iot device forecast update. https://www.strategyanalytics.com/access-services/devices/connected-home/consumer-electronics/reports/report-detail/global-connected-and-iot-device-forecast-update (2019)
Columbus, L.: Roundup of internet of things forecasts and market estimates, 2016. Forbes Magazine. https://www.forbes.com/sites/louiscolumbus/2016/11/27/roundup-of-internet-of-things-forecasts-and-market-estimates-2016/#6a558beb292d (2016)
The growth in connected iot devices is expected to generate 79.4zb of data in 2025, according to a new idc forecast (2019). https://www.idc.com/getdoc.jsp?containerId=prUS45213219
IEEE standard for adoption of openfog reference architecture for fog computing. IEEE Std 1934-2018, pp. 1–176 (2018)
Kubernetes : Production-grade container orchestration (2020). https://kubernetes.io/, Accessed 4 Oct 2020
Docker swarm (2020). https://docs.docker.com/engine/swarm/, Accessed 4 Oct 2020
Apache brooklyn (2020). http://brooklyn.apache.org/, Accessed 4 Oct 2020
Cloudify orchestration platform - multi cloud, cloud native & edge (2020). https://cloudify.co/, Accessed 4 Oct 2020
Cloudiator (2020). http://cloudiator.org/, Accessed 4 Oct 2020
Alien 4 cloud (2020). https://alien4cloud.github.io/, Accessed 4 Oct 2020
Modaclouds multi-cloud devops alliance: Modaclouds releases multi-cloud devops toolbox (2020). http://multiclouddevops.com/, Accessed 4 Oct 2020
Micadoscale (2020). https://micado-scale.eu/, Accessed 4 Oct 2020
Amazon: Aws cloudformation: Speed up cloud provisioning with infrastructure as code. https://aws.amazon.com/cloudformation/, Accessed 18 Oct 2020 (2020)
OpenStack: Openstack orchestration. https://wiki.openstack.org/wiki/Heat, Accessed 18 Oct 2020 (2020)
Azure resource manager (arm) templates (2020). https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/overview, Accessed 19 Oct 2020
Google cloud depyment manager (2020). https://cloud.google.com/deployment-manager, Accessed 19 Oct 2020
Ghobaei-Arani, M., Souri, A., Rahmanian, A.A.: Resource management approaches in fog computing: a comprehensive review. J. Grid Comput, 1–42 (2019)
Digitbrain h2020 project (2020). https://digitbrain.eu/, Accessed 4 Oct 2020
Asclepios eu h2020 project (2020). https://www.asclepios-project.eu/, Accessed 4 Oct 2020
Oasis topology and orchestration specification for cloud applications (2020). www.oasis-open.org/committees/tosca, Accessed 4 Oct 2020
Pierantoni, G., Kiss, T., Terstyanszky, G., DesLauriers, J., Gesmier, G., Dang, H.-V.: Describing and processing topology and quality of service parameters of applications in the cloud. J. Grid Comput., 1–18 (2020)
Cola - cloud orchestration at the level of application, h2020 eu project (2020). https://project-cola.eu/, Accessed 4 Oct 2020
DesLauriers, J, Kiss, T., Ariyattu, R.C., Dang, H.-V., Ullah, A., Bowden, J., Krefting, D., Pierantoni, G., Terstyánszky, G.: Cloud apps to-go: Cloud portability with tosca and micado. Concurrency and Computation: Practice and Experience, Accepted (2020)
Kiss, T., Kacsuk, P., Kovács, J., Rakoczi, B., Hajnal, Á., Farkas, A., Gesmier, G., Terstyánszky, G.: Micado - microservice-based cloud application-level dynamic orchestrator. Fut. Gener. Comput. Syst. 94, 937–946 (2019)
Kiss, T., DesLauriers, J., Gesmier, G., Terstyánszky, G, Pierantoni, G., Oun, O.A., Taylor, S.J.E., Anagnostou, A., Kovács, J.: A cloud-agnostic queuing system to support the implementation of deadline-based application execution policies. Future Gener. Comput. Syst. 101, 99–111 (2019). https://doi.org/10.1016/j.future.2019.05.062
Kovács, J., Kacsuk, P.: Occopus: a multi-cloud orchestrator to deploy and manage complex scientific infrastructures. J. Grid Comput. 16(1), 19–37 (2018)
Terraform (2020). www.terraform.io, Accessed 4 Oct 2020
Docker (2020). www.docker.com, Accessed 4 Oct 2020
Prometheus (2020). https://prometheus.io/, Accessed 4 Oct 2020
Kovács, J.: Supporting programmable autoscaling rules for containers and virtual machines on clouds. J. Grid Comput. 17(4), 813–829 (2019)
Ansible documentation (2020). https://docs.ansible.com/ansible/latest/index.html, Accessed 19 Oct 2020
Micado - autoscaling framework for docker services on cloud (2020). https://github.com/micado-scale/ansible-micado/tree/edge, Accessed 19 Oct 2020
Oasis (2020). https://www.oasis-open.org/, Accessed 30 Oct 2020
Openstack parser (2020). https://github.com/openstack/tosca-parser, Accessed 4 Oct 2020
Kubeedge (2020). https://github.com/kubeedge/kubeedge, Accessed 19 Oct 2020
Wang, N., Matthaiou, M., Nikolopoulos, D.S., Varghese, B.: Dyverse: Dynamic vertical scaling in multi-tenant edge environments. Future Generation Computer Systems (2020)
McChesney, J., Wang, N., Tanwer, A., de Lara, E., Varghese, B.: Defog: fog computing benchmarks. In: Proceedings of the 4th ACM/IEEE Symposium on Edge Computing, pp 47–58 (2019)
Dyverse - dynamic vertical scaling in multi-tenant edge environments (2020). https://github.com/qub-blesson/DYVERSE, Accessed 19 Oct 2020
Real time face detection (fd) demo application (2020). https://github.com/UoW-CPC/DYVERSE, Accessed 19 Oct 2020
Micado tosca adt repository (2020). https://github.com/micado-scale/tosca/tree/develop/ADT/edge-fog, Accessed 19 Oct 2020
Ullah, A.: Towards a novel biologically-inspired cloud elasticity framework. Ph.D. Thesis, University of Stirling, UK (2017)
Costan, V., Devadas, S.: Intel sgx explained. IACR Cryptol. ePrint Arch. 2016(86), 1–118 (2016)
Sweeney, L.: k-anonymity: A model for protecting privacy. Int. J. Uncertain. Fuzz. Knowl.-Based Syst. 10(05), 557–570 (2002)
Beier, M., Jansen, C., Mayer, G., Penzel, T., Rodenbeck, A., Siewert, R., Witt, M., Wu, J., Krefting, D.: Multicenter data sharing for collaboration in sleep medicine. Fut. Gener. Comput. Syst. 67, 466–480 (2017)
Beier, M., Penzel, T., Krefting, D.: A performant web-based visualization, assessment and collaboration tool for multidimensional biosignals. Front. Neuroinform. 13, 65 (2019)
Bakas, A., Michalas, A.: Power range: Forward private multi-client symmetric searchable encryption with range queries support (2020)
Sabt, M., Achemlal, M., Bouabdallah, A.: Trusted execution environment: what it is, and what it is not. In: 2015 IEEE Trustcom/BigDataSE/ISPA, vol. 1, pp 57–64, IEEE (2015)
Asclepios adt repository (2020). https://github.com/micado-scale/tosca/tree/asclepios/ADT/sleep, Accessed 19 Oct 2020
Amazon greengrass (2020). https://aws.amazon.com/greengrass/, Accessed 4 Oct 2020
Microsoft azure iot (2020). https://azure.microsoft.com/en-gb/overview/iot/, Accessed 4 Oct 2020
Google cloud iot (2020). https://cloud.google.com/iot-core, Accessed 4 Oct 2020
Kubernetes federation project (2020). https://github.com/kubernetes-sigs/kubefed, Accessed 4 Oct 2020
Submariner, connected kubernetes overlay networks (2020). https://github.com/submariner-io/submarinerhttps://github.com/submariner-io/submariner, Accessed 4 Oct 2020
Edgex foundry (2020). https://www.edgexfoundry.org, Accessed 4 Oct 2020
Ostberg, P., Byrne, J., Casari, P., Eardley, P., Anta, A.F., Forsman, J., Kennedy, J., Le Duc, T., Marino, M.N., Loomba, R., Lopez Pena, M.A., Veiga, J.L., Lynn, T., Mancuso, V., Svorobej, S., Torneus, A., Wesner, S., Willis, P., Domaschka, J.: Reliable capacity provisioning for distributed cloud/edge/fog computing applications. In: 2017 European Conference on Networks and Communications (EuCNC), pp 1–6 (2017)
Velasquez, K., Abreu, D. P., Gonçalves, D., Bittencourt, L., Curado, M., Monteiro, E., Madeira, E.: Service orchestration in fog environments. In: 2017 IEEE 5th International Conference on Future Internet of Things and Cloud (FiCloud), pp 329–336 (2017)
de Brito, M.S., Hoque, S., Magedanz, T., Steinke, R., Willner, A., Nehls, D., Keils, O., Schreiner, F.: A service orchestration architecture for fog-enabled infrastructures. In: 2017 Second International Conference on Fog and Mobile Edge Computing (FMEC), pp 127–132 (2017)
Alam, M., Rufino, J., Ferreira, J., Ahmed, S. H., Shah, N., Chen, Y.: Orchestration of microservices for iot using docker and edge computing. IEEE Commun. Mag. 56(9), 118–123 (2018)
Villari, M., Celesti, A., Tricomi, G., Galletta, A., Fazio, M.: Deployment orchestration of microservices with geographical constraints for edge computing. In: 2017 IEEE Symposium on Computers and Communications (ISCC), pp 633–638 (2017)
Pahl, C., Helmer, S., Miori, L., Sanin, J., Lee, B.: A container-based edge cloud paas architecture based on raspberry pi clusters. In: 2016 IEEE 4th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW), pp 117–124 (2016aug)
Wang, N., Varghese, B., Matthaiou, M., Nikolopoulos, D.: Enorm: A framework for edge node resource management. IEEE Trans. Serv. Comput. PP (2017)
Zanni, A., Forsstrom, S., Jennehag, U., Bellavista, P.: Elastic provisioning of internet of things services using fog computing: An experience report. In: 2018 6th IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (MobileCloud), pp 17–22 (2018)
Pizzolli, D., Cossu, G., Santoro, D., Capra, L., Dupont, C., Charalampos, D., De Pellegrini, F., Antonelli, F., Cretti, S.: Cloud4iot: A heterogeneous, distributed and autonomic cloud platform for the iot, pp. 476–479 (2016)
Taherizadeh, S., Stankovski, V., Grobelnik, M.: A capillary computing architecture for dynamic internet of things: Orchestration of microservices from edge devices to fog and cloud providers. Sensors 18 (2018)
Yigitoglu, E., Mohamed, M., Liu, L., Ludwig, H.: Foggy: A framework for continuous automated iot application deployment in fog computing. In: 2017 IEEE International Conference on AI Mobile Services (AIMS), pp 38–45 (2017)
Yigitoglu, E., Liu, L., Looper, M., Pu, C.: Distributed orchestration in large-scale iot systems. In: 2017 IEEE International Congress on Internet of Things (ICIOT), pp 58–65 (2017)
Davoli, G., Borsatti, D., Tarchi, D., Cerroni, W.: Forch: An orchestrator for fog computing service deployment. In: 2020 IFIP Networking Conference (Networking), pp 677–678 (2020)
Lertsinsrubtavee, A., Ali, A., Molina-Jimenez, C., Sathiaseelan, A., Crowcroft, J.: Picasso: A lightweight edge computing platform. In: 2017 IEEE 6th International Conference on Cloud Networking (CloudNet), pp 1–7 (2017)
Cloud application management for platforms version 1.1 (2020). http://docs.oasis-open.org/camp/camp-spec/v1.1/camp-spec-v1.1.html, Accessed 1 Feb 2020
Paladi, N., Michalas, A., Dang, H.-V.: Towards secure cloud orchestration for multi-cloud deployments. In: Proceedings of the 5th Workshop on CrossCloud Infrastructures & Platforms, pp 1–6 (2018)
Fiware: The open source platform for our smart digital future (2020). https://www.fiware.org/, Accessed 19 Oct 2020
Acknowledgements
This work was funded by the following projects: ASCLEPIOS – Advanced Secure Cloud Encrypted Platform for Internationally Orchestrated Solutions in Healthcare – project, No. 826093, European Commission (EU H2020); DIGITbrain - Digital twins bringing agility and innovation to manufacturing SMEs, by empowering a network of DIHs with an integrated digital platform that enables Manufacturing as a Service – project, No. 952071, European Commission (EU H2020).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Ullah, A., Dagdeviren, H., Ariyattu, R.C. et al. MiCADO-Edge: Towards an Application-level Orchestrator for the Cloud-to-Edge Computing Continuum. J Grid Computing 19, 47 (2021). https://doi.org/10.1007/s10723-021-09589-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10723-021-09589-5
Keywords
- Application-level orchestration
- Cloud-Fog-Edge ecosystems
- Cloud-to-Edge continuum
- IoT applications orchestration
- Orchestration of microservices
- Deployment and run-time management