Performance and Cost Trade-Off in IaaS Environments: A Scientific Workflow Simulation Environment Case Study

  • Santiago Gómez SáezEmail author
  • Vasilios Andrikopoulos
  • Michael Hahn
  • Dimka Karastoyanova
  • Frank Leymann
  • Marigianna Skouradaki
  • Karolina Vukojevic-Haupt
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 581)


The adoption of the workflow technology in the eScience domain has contributed to the increase of simulation-based applications orchestrating different services in a flexible and error-free manner. The nature of the provisioning and execution of such simulations makes them potential candidates to be migrated and executed in Cloud environments. The wide availability of Infrastructure-as-a-Service (IaaS) Cloud offerings and service providers has contributed to a raise in the number of supporters of partially or completely migrating and running their scientific experiments in the Cloud. Focusing on Scientific Workflow-based Simulation Environments (SWfSE) applications and their corresponding underlying runtime support, in this research work we aim at empirically analyzing and evaluating the impact of migrating such an environment to multiple IaaS infrastructures. More specifically, we focus on the investigation of multiple Cloud providers and their corresponding optimized and non-optimized IaaS offerings with respect to their offered performance, and its impact on the incurred monetary costs when migrating and executing a SWfSE. The experiments show significant performance improvements and reduced monetary costs when executing the simulation environment in off-premise Clouds.


Workflow simulation eScience Iaas Performance evaluation Cost evaluation Cloud migration 



The research leading to these results has received funding from the FP7 EU project ALLOW Ensembles (600792), the German Research Foundation (DFG) within the Cluster of Excellence in Simulation Technology (EXC310), and the German DFG project BenchFlow (DACH Grant Nr. 200021E-145062/1).


  1. 1.
    Leymann, F., Roller, D.: Production Workflow: Concepts and Techniques. Prentice Hall, Upper Saddle River (2000)zbMATHGoogle Scholar
  2. 2.
    Sonntag, M., Karastoyanova, D.: Next generation interactive scientific experimenting based on the workflow technology. In: Alhajj, R., Leung, V., Saif, M., Thring, R. (eds.) Proceedings of the 21st IASTED International Conference on Modelling and Simulation (MS 2010). ACTA Press (2010)Google Scholar
  3. 3.
    Juve, G., Deelman, E., Vahi, K., Mehta, G., Berriman, B., Berman, B., Maechling, P.: Scientific workflow applications on Amazon EC2. In: 2009 5th IEEE International Conference on E-Science Workshops, pp. 59–66 (2009)Google Scholar
  4. 4.
    de Oliveira, D., Ocaña, K., Ogasawara, E.S., Dias, J., Baião, F.A., Mattoso, M.: A performance evaluation of X-Ray crystallography scientific workflow using SciCumulus. In: Liu, L., Parashar, M. (eds.) IEEE CLOUD, pp. 708–715. IEEEGoogle Scholar
  5. 5.
    Vukojevic-Haupt, K., Karastoyanova, D., Leymann, F.: On-demand provisioning of infrastructure, middleware and services for simulation workflows. In: 2013 IEEE 6th International Conference on Service-Oriented Computing and Applications (SOCA), pp. 91–98. IEEE (2013)Google Scholar
  6. 6.
    Zhao, Y., Li, Y., Raicu, I., Lu, S., Lin, C., Zhang, Y., Tian, W., Xue, R.: A service framework for scientific workflow management in the cloud. In: IEEE Transactions on Services Computing, p. 1 (2014)Google Scholar
  7. 7.
    Gómez Sáez, S., Andrikopoulos, V., Leymann, F., Strauch, S.: Design support for performance aware dynamic application (Re-) distribution in the cloud. IEEE Trans. Serv. Comput. 8, 225–239 (2014)CrossRefGoogle Scholar
  8. 8.
    Andrikopoulos, V., Gómez Sáez, S., Leymann, F., Wettinger, J.: Optimal distribution of applications in the cloud. In: Jarke, M., Mylopoulos, J., Quix, C., Rolland, C., Manolopoulos, Y., Mouratidis, H., Horkoff, J. (eds.) CAiSE 2014. LNCS, vol. 8484, pp. 75–90. Springer, Heidelberg (2014)Google Scholar
  9. 9.
    Gómez Sáez, S., Andrikopoulos, V., Hahn, M., Karastoyanova, D., Leymann, F., Skouradaki, M., Vukojevic-Haupt, K.: Performance and cost evaluation for the migration of a scientific workflow infrastructure to the cloud. In: Proceedings of the 5th International Conference on Cloud Computing and Service Science (CLOSER 2015), pp. 1–10. SciTePress (2015)Google Scholar
  10. 10.
    Sonntag, M., Hahn, M., Karastoyanova, D.: Mayflower - explorative modeling of scientific workflows with BPEL. In: Proceedings of the Demo Track of the 10th International Conference on Business Process Management (BPM 2012), CEUR Workshop Proceedings, 2012, pp. 1–5. CEUR Workshop Proceedings (2012)Google Scholar
  11. 11.
    Görlach, K., Sonntag, M., Karastoyanova, D., Leymann, F., Reiter, M.: Conventional workflow technology for scientific simulation. In: Yang, X., Wang, L., Jie, W. (eds.) Guide to e-Science, pp. 323–352. Springer, London (2011)CrossRefGoogle Scholar
  12. 12.
    Sonntag, M., Karastoyanova, D.: Model-as-you-go: an approach for an advanced infrastructure for scientific workflows. J. Grid Comput. 11, 553–583 (2013)CrossRefGoogle Scholar
  13. 13.
    Sonntag, M., Hotta, S., Karastoyanova, D., Molnar, D., Schmauder, S.: Using services and service compositions to enable the distributed execution of legacy simulation applications. In: Abramowicz, W., Llorente, I.M., Surridge, M., Zisman, A., Vayssière, J. (eds.) ServiceWave 2011. LNCS, vol. 6994, pp. 242–253. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  14. 14.
    Binkele, P., Schmauder, S.: An atomistic monte carlo simulation of precipitation in a binary system. Zeitschrift für Metallkunde 94, 858–863 (2003)CrossRefGoogle Scholar
  15. 15.
    Molnar, D., Binkele, P., Hocker, S., Schmauder, S.: Multiscale modelling of nano tensile tests for different Cu-precipitation states in \(\alpha \)-Fe. In: Proceedings of the 5th International Conference on Multiscale Materials Modelling, pp. 235–239 (2010)Google Scholar
  16. 16.
    Walker, E.: The real cost of a CPU hour. IEEE Comput. 42, 35–41 (2009)CrossRefGoogle Scholar
  17. 17.
    Andrikopoulos, V., Song, Z., Leymann, F.: Supporting the migration of applications to the cloud through a decision support system. In: 2013 IEEE Sixth International Conference on Cloud Computing (CLOUD), pp. 565–572. IEEE (2013)Google Scholar
  18. 18.
    Skouradaki, M., Roller, D.H., Frank, L., Ferme, V., Pautasso, C.: On the road to benchmarking BPMN 2.0 workflow engines. In: Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering ICPE 2015, pp. 1–4. ACM (2015)Google Scholar
  19. 19.
    Röck, C., Harrer, S., Wirtz, G.: Performance benchmarking of BPEL engines: a comparison framework, status quo evaluation and challenges. In: 26th International Conference on Software Engineering and Knowledge Engineering (SEKE), Vancouver, Canada, pp. 31–34 (2014)Google Scholar
  20. 20.
    Pathirage, M., Perera, S., Kumara, I., Weerawarana, S.: A multi-tenant architecture for business process executions. In: Proceedings of the 2011 IEEE International Conference on Web Services, ICWS 2011, pp. 121–128. IEEE Computer Society, Washington, DC, USA (2011)Google Scholar
  21. 21.
    Sonntag, M., Hotta, S., Karastoyanova, D., Molnar, D., Schmauder, S.: Workflow-based distributed environment for legacy simulation applications. In: ICSOFT, vol. 1, pp. 91–94 (2011)Google Scholar
  22. 22.
    Strauch, S., Andrikopoulos, V., Bachmann, T., Karastoyanova, D., Passow, S., Vukojevic-Haupt, K.: Decision support for the migration of the application database layer to the cloud. In: 2013 IEEE 5th International Conference on Cloud Computing Technology and Science (CloudCom), vol. 1, pp. 639–646. IEEE (2013)Google Scholar
  23. 23.
    Ostermann, S., Iosup, A., Yigitbasi, N., Prodan, R., Fahringer, T., Epema, D.: A performance analysis of EC2 cloud computing services for scientific computing. In: Avresky, D.R., Diaz, M., Bode, A., Ciciani, B., Dekel, E. (eds.) Cloud Computing. LNICST, vol. 34, pp. 115–134. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  24. 24.
    Juve, G., Chervenak, A., Deelman, E., Bharathi, S., Mehta, G., Vahi, K.: Characterizing and profiling scientific workflows. Future Gener. Comput. Syst. 29, 682–692 (2013)CrossRefGoogle Scholar
  25. 25.
    Calheiros, R.N., Ranjan, R., Beloglazov, A., De Rose, C.A., Buyya, R.: Cloudsim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw. Pract. Experience 41, 23–50 (2011)CrossRefGoogle Scholar
  26. 26.
    Frey, S., Hasselbring, W.: The cloudmig approach: model-based migration of software systems to cloud-optimized applications. Int. J. Adv. Softw. 4, 342–353 (2011)Google Scholar
  27. 27.
    Darsow, A., Karastoyanova, D., Andrikopoulos, V., Leymann, F.: CloudDSF – the cloud decision support framework for application migration. In: Villari, M., Zimmermann, W., Lau, K.-K. (eds.) ESOCC 2014. LNCS, vol. 8745, pp. 1–16. Springer, Heidelberg (2014)Google Scholar
  28. 28.
    Gómez Sáez, S., Andrikopoulos, V., Wessling, F., Marquezan, C.C.: Cloud adaptation and application (Re-) distribution: bridging the two perspectives. In: Proceedings of the First International Workshop on Engineering Cloud Applications and Services (EnCASE 2014), pp. 163–172. IEEE Computer Society Press (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Santiago Gómez Sáez
    • 1
    Email author
  • Vasilios Andrikopoulos
    • 1
  • Michael Hahn
    • 1
  • Dimka Karastoyanova
    • 1
  • Frank Leymann
    • 1
  • Marigianna Skouradaki
    • 1
  • Karolina Vukojevic-Haupt
    • 1
  1. 1.Institute of Architecture of Application SystemsUniversity of StuttgartStuttgartGermany

Personalised recommendations