Advertisement

Dynamic Communication-Aware Scheduling with Uncertainty of Workflow Applications in Clouds

  • Vanessa Miranda
  • Andrei Tchernykh
  • Dzmitry Kliazovich
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 595)

Abstract

Cloud computing has emerged as a new approach to bring computing as a service, in both academia and industry. One of the challenging issues is scientific workflow execution, where the job scheduling problem becomes more complex, especially when communication processes are taken into account. To provide good performance, many algorithms have been designed for distributed environments. However, these algorithms are not adapted to the uncertain and dynamic nature of cloud computing. In this paper, we present a general view on scheduling problems in cloud computing with communication, and compare existed solutions based on three models of cloud applications named CU-DAG, EB-DAG and CA-DAG. We formulate the problem and review several workflow scheduling algorithms. We discuss the main difficulties of using existed application models in the domain of computations on clouds. Finally, we show that our CA-DAG approach, based on separate vertices for computing and communications, and introducing communication awareness, allows us to mitigate uncertainty in a more efficient way.

Keywords

Cloud computing Scheduling Workflow Communication awareness Uncertainty DAG 

Notes

Acknowledgment

This work is partially supported by CONACYT (Consejo Nacional de Ciencia y Tecnología, México), grant no. 178415. The work of D. Dzmitry Kliazovich is partly funded by National Research Fund, Luxembourg in the framework of ECO-CLOUD (C12/IS/3977641) project.

References

  1. 1.
    Robison, S.: HP Shane Robison Executive Viewpoint: The Next Wave: Everything as a Service. http://www.hp.com/hpinfo/execteam/articles/robison. Accessed 30 January 2014
  2. 2.
    CSC: CSC cloud usage index latest report, Computer Sciences Corporation. http://www.csc.com/au/ds/39454/75790-csc_cloud_usage_index_latest_report. Accessed 20 January 2014
  3. 3.
    Zhang, Q., Cheng, L., Boutaba, R.: Cloud computing: state-of-the-art and research challenges. J. Internet Serv. Appl. 1(1), 7–18 (2010)CrossRefGoogle Scholar
  4. 4.
    N. US Department of Commerce, Final Version of NIST Cloud Computing Definition Published. http://www.nist.gov/itl/csd/cloud-102511.cfm. Accessed 20 January 2014
  5. 5.
    Hollinsworth, D.: The workflow reference model. In: Workflow Management Coalition, vol. TC00–1003 (1995)Google Scholar
  6. 6.
    Pandey, S., Wu, L., Guru, S.M., Buyya, R.: A particle swarm optimization-based heuristic for scheduling workflow applications in cloud computing environments. In: 2010 24th IEEE International Conference on Advanced Information Networking and Applications (AINA), pp. 400–407 (2010)Google Scholar
  7. 7.
    Kandula, S., Sengupta, S., Greenberg, A., Patel, P., Chaiken, R.: The nature of data center traffic: measurements & analysis. In: Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement Conference, New York, NY, USA, pp. 202–208 (2009)Google Scholar
  8. 8.
    AbdelBaky, M., Parashar, M., Kim, H., Jordan, K.E., Sachdeva, V., Sexton, J., Jamjoom, H., Shae, Z.Y., Pencheva, G., Tavakoli, T., Wheeler, M.F.: Enabling high-performance computing as a service. Computer 45(10), 72–80 (2012)CrossRefGoogle Scholar
  9. 9.
    Tchernykh, A., Schwiegelsohn, U., Alexandrov, V., Talbi, E.: Towards understanding uncertainty in cloud computing resource provisioning. SPU 2015 - solving problems with uncertainties (3rd Workshop). In: Conjunction with the 15th International Conference on Computational Science (ICCS 2015), Reykjavík, Iceland, 1–3 June 2015. Procedia Computer Science, Elsevier, vol. 51, pp. 1772–1781 (2015)Google Scholar
  10. 10.
    Tychinsky A.: Innovation Management of Companies: Modern Approaches, Algorithms, Experience. Taganrog Institute of Technology, Taganrog (2006). http://www.aup.ru/books/m87/
  11. 11.
    Kliazovich, D., Pecero, J., Tchernykh, A., Bouvry, P., Khan, S., Zomaya, A.: CA-DAG: modeling communication-aware applications for scheduling in cloud computing. J. Grid Comput., 1–17 (2015). Springer, Netherlands Google Scholar
  12. 12.
    Tsafrir, D., Etsion, Y., Feitelson, D.G.: Backfilling using system-generated predictions rather than user runtime estimates. IEEE Trans. Parallel Distrib. Syst. 18(6), 789–803 (2007)CrossRefGoogle Scholar
  13. 13.
    Ramírez-Alcaraz, J.M., Tchernykh, A., Yahyapour, R., Schwiegelshohn, U., Quezada-Pina, A., González-García, J.L., Hirales-Carbajal, A.: Job allocation strategies with user run time estimates for online scheduling in hierarchical Grids. J. Grid Comput. 9(1), 95–116 (2011)CrossRefGoogle Scholar
  14. 14.
    Zitzler, E.: Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications, vol. 63. Shaker, Ithaca (1999) Google Scholar
  15. 15.
    Kliazovich, D., Bouvry, P., Khan, S.U.: DENS: data center energy-efficient network-aware scheduling. Cluster Comput. 16(1), 65–75 (2013)CrossRefGoogle Scholar
  16. 16.
    Liu, K., Jin, H., Chen, J., Liu, X., Yuan, D., Yang, Y.: A compromised-time-cost scheduling algorithm in swindew-c for instance-intensive cost-constrained workflows on a cloud computing platform. Int. J. High Perform. Comput. Appl. 24(4), 445–456 (2010)CrossRefGoogle Scholar
  17. 17.
    Jin, J., Luo, J., Song, A., Dong, F., Xiong, R.: BAR: an efficient data locality driven task scheduling algorithm for cloud computing. In: 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2011), pp. 295–304 (2011)Google Scholar
  18. 18.
    Sonnek, J., Greensky, J., Reutiman, R., Chandra, A.: Starling: minimizing communication overhead in virtualized computing platforms using decentralized affinity-aware migration. In: 39th International Conference on Parallel Processing (ICPP 2010), pp. 228–237 (2010)Google Scholar
  19. 19.
    Pecero, J.E., Trystram, D., Zomaya, A.Y.: A new genetic algorithm for scheduling for large communication delays. In: Sips, H., Epema, D., Lin, H.-X. (eds.) Euro-Par 2009. LNCS, vol. 5704, pp. 241–252. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  20. 20.
    Stage, A., Setzer, T.: Network-aware migration control and scheduling of differentiated virtual machine workloads. In: Proceedings of the 2009 ICSE Workshop on Software Engineering Challenges of Cloud Computing, pp. 9–14. IEEE Computer Society (2009)Google Scholar
  21. 21.
    Sinnen, O., Sousa, L.A.: Communication contention in task scheduling. IEEE Trans. Parallel Distrib. Syst. 16(6), 503–515 (2005)CrossRefGoogle Scholar
  22. 22.
    Volckaert, B., Thysebaert, P., De Leenheer, M., De Turck, F., Dhoedt, B., Demeester, P.: Network aware scheduling in grids. In: Proceedings of the 9th European Conference on Networks and Optical Communifications, p. 9 (2004)Google Scholar
  23. 23.
    Malawski, M., Juve, G., Deelman, E., Nabrzyski, J.: Cost-and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, p. 22 (2012)Google Scholar
  24. 24.
    Yu, J., Buyya, R.: A taxonomy of workflow management systems for grid computing. J. Grid Comput. 3(3–4), 171–200 (2005)CrossRefGoogle Scholar
  25. 25.
    Tchernykh, A., Pecero, J., Barrondo, A., Schaeffer, E.: Adaptive energy efficient scheduling in peer-to-peer desktop grids. Future Gener. Comput. Systems 36, 209–220 (2014)CrossRefGoogle Scholar
  26. 26.
    Tchernykh, A., Lozano, L., Schwiegelshohn, U., Bouvry, P., Pecero, J.E., Nesmachnow, S., Drozdov, A.Y.: Online bi-objective scheduling for IaaS clouds with ensuring quality of service. J. Grid Comput., 1–18 (2015). SpringerGoogle Scholar
  27. 27.
    Carbajal, A.H., Tchernykh, A., Yahyapour, R., Röblitz, T., Ramírez-Alcaraz, J.M., González-García, J.L.: Multiple workflow scheduling strategies with user run time estimates on a grid. J. Grid Comput. 10(2), 325–346 (2012). Springer-Verlag, New York, USACrossRefGoogle Scholar
  28. 28.
    Quezada, A., Tchernykh, A., González, J., Hirales, A., Ramírez, J.-M., Schwiegelshohn, U., Yahyapour, R., Miranda, V.: Adaptive parallel job scheduling with resource admissible allocation on two level hierarchical grids. Future Gener. Comput. Syst. 28(7), 965–976 (2012)CrossRefGoogle Scholar
  29. 29.
    Rodriguez, A., Tchernykh, A., Ecker, K.: Algorithms for dynamic scheduling of unit execution time tasks. Eur. J. Oper. Res. 146(2), 403–416 (2003). Elsevier Science, North-HollandMathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Kianpisheh, S., Jalili, S., Charkari, N.M.: Predicting job wait time in grid environment by applying machine learning methods on historical information. Int. J. Grid Distrib. Comput. 5(3) (2012)Google Scholar
  31. 31.
    Iverson, M.A., Ozguner, F.; Follen, G.J.: Run-time statistical estimation of task execution times for heterogeneous distributed computing. In: Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing, 1996, pp. 263–270 (1996)Google Scholar
  32. 32.
    Ramirez-Velarde, R.V., Rodriguez-Dagnino, R.M.: From commodity computers to high-performance environments: scalability analysis using self-similarity, large deviations and heavy-tails. Concurrency Comput. Pract. Exp. 22, 1494–1515 (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Vanessa Miranda
    • 1
  • Andrei Tchernykh
    • 1
  • Dzmitry Kliazovich
    • 2
  1. 1.CICESE Research CenterEnsenadaMexico
  2. 2.University of LuxembourgLuxembourgLuxembourg

Personalised recommendations