Partitioning and Scheduling Workflows across Multiple Sites with Storage Constraints

  • Weiwei Chen
  • Ewa Deelman
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7204)

Abstract

This paper aims to address the problem of scheduling large workflows onto multiple execution sites with storage constraints. Three heuristics are proposed to first partition the workflow into sub-workflows. Three estimators and two schedulers are then used to schedule sub-workflows to the execution sites. Performance with three real-world workflows shows that this approach is able to satisfy storage constraints and improve the overall runtime by up to 48% over a default whole-workflow scheduling.

Keywords

workflow scheduling partitioning storage constraints 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Taylor, I.J., Deelman, E., et al.: Workflows for e-Science. In: Scientific Workflows for Grids. Springer (2007)Google Scholar
  2. 2.
    Berriman, G.B., et al.: Montage: a grid-enabled engine for delivering custom science-grade mosaics on demand. In: Proc. of SPIE, vol. 5493, pp. 221–232 (2004)Google Scholar
  3. 3.
    Maechling, P., Deelman, E., et al.: SCEC CyberShake Workflows–Automating Probabilistic Seismic Hazard Analysis Calculations. In: Workflows for e-Science. Scientific Workflows for Grids. Springer (2007)Google Scholar
  4. 4.
    USC Epigenome Center, http://epigenome.usc.edu
  5. 5.
    Deelman, E., et al.: Pegasus: A framework for mapping complex scientific workflows onto distributed systems. Sci. Program 13, 219–237 (2005)Google Scholar
  6. 6.
    Litzkow, M., et al.: Condor–A Hunter of Idle Workstations. In: ICDCS (June 1988)Google Scholar
  7. 7.
    Blythe, J., et al.: Task Scheduling Strategies for Workflow-Based Applications in Grids. In: CCGrid (2005)Google Scholar
  8. 8.
    Topcuoglu, H., et al.: Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing. IEEE TPDS 13(3) (March 2002)Google Scholar
  9. 9.
    Duan, R., et al.: Run-time Optimisation of Grid Workflow Applications. In: 7th IEEE/ACM Intl. Conf. on Grid Computing (September 2005)Google Scholar
  10. 10.
    Sonmez, O.O.: Application-Oriented Scheduling in Multicluster Grids (June 2010), http://www.pds.ewi.tudelft.nl/~homedirsonmez/research.htm
  11. 11.
    Singh, G., et al.: Optimizing Workflow Data Footprint. Special issue of the Scientific Programming Journal dedicated to Dynamic Computational Workflows: Discovery, Optimisation and Scheduling (2007)Google Scholar
  12. 12.
    Papadimitriou, C.H., et al.: Combinatorial Optimization: Algorithms and Complexity, Dover, pp. 120–128 (1998) ISBN 0486402584 Google Scholar
  13. 13.
    Eucalyptus Systems, http://www.eucalyptus.com/
  14. 14.
  15. 15.
    Juve, G., et al.: Scientific Workflow Applications on Amazon EC2, E-Science Workshops, Oxford UK (December 2009)Google Scholar
  16. 16.
    Wieczorek, M., et al.: Scheduling of scientific workflows in the ASKALON grid environment. SIGMOND Record 34(3) (September 2005)Google Scholar
  17. 17.

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Weiwei Chen
    • 1
  • Ewa Deelman
    • 1
  1. 1.Information Sciences InsituteUniversity of Southern CaliforniaMarina del ReyUSA

Personalised recommendations