Mapping Workflows on Grid Resources: Experiments with the Montage Workflow

Conference paper

Abstract

Scientific workflows have received considerable attention in Grid computing. This paper is concerned with the issue of scheduling scientific workflows and, by considering a commonly used astronomy workflow, Montage, investigates the impact of different strategies to schedule the workflow graph. Our experiments suggest that the rather regular and symmetric nature of the Montage graph allows rather simple to implement scheduling heuristics that do not take into account the whole structure of the graph, such as Min-min, to deliver competitive performance in most cases of interest. The results support the view that sophisticated graph scheduling heuristics may not be always a prerequisite for good performance in workflow execution. Instead, mechanisms to deal with uncertainties in execution time may be of comparatively higher importance.

Keywords

Expense Marin Berman Prodan 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    J. Annis, Y. Zhao, J. Voeckler, M. Wilde, S. Kent, and I. Foster. Applying Chimera virtual data concepts to cluster finding in the Sloan Sky Survey. In: Supercomputing’02: Proceedings of the 2002 ACM/IEEE Conference on Supercomputing, Los Alamitos, CA, USA, IEEE Computer Scciety Press, 2002, pp. 1-14. Google Scholar
  2. 2.
    G. B. Berilman, J. C. Good, A. C. Laity, et at. Montage: A Grid Enabled Image Mosaic Service for the National Virtual Observatory. In Astronomical Data Analysis Software & Systems ADASS) XIII, 2003. Google Scholar
  3. 3.
    5. Bhamthi, A. Chervenak, E. Deelman, G. Mehta, M.-H. Su, and K. Vahi. Characterization of Scientific Workflows. In Proceedings of the 3rd Workshop on Wor4flows in Support of Large- Scale Science (WORKS 2008), November 2008. Google Scholar
  4. 4.
    J. Blythe, S. Jam, E. Deelman, Y. Gil, K. Vahi, A. Mandal, and K. Kennedy. Task Scheduling Strategies for Workflow-based Applications in Grids. In IEEE International Symposium on Cluster Computing and the Grid (CCGrid 2005), 2005. Google Scholar
  5. 5.
    T. D. Bmun, H.J. Siegel, N. Beck, L. L. Boloni, M. Maheswaran,A. I. Reuther,J. P. Robertson, M. D. Theys, and B. Yao. A comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. Journal of Parallel and Distributed Computing, vol. 61, pp. 810-837, 2001. Google Scholar
  6. 6.
    L. Breslau, D. Estrin, K. Fall, S. Floyd, J. Heidemann, A. Helmy, P. Huang, S. McCanne, K. Varadhan, Y. Xu, and H. Yu. Advances in network simulation. Computer, vol.33(5), pp.S9- 67, May 2000. See also http://www.isi.edu/nsnamlns. Google Scholar
  7. 7.
    L.-C. Canon, E. Jeannot, R. Sakellariou, and W. Zheng. Comparative evaluation of the Robustness of DAG Scheduling Heuristics. In Grid Computing: Achievements and Prospects (eds: S. Gorlatch, P. Fragopoulou, T. Priol), Spffnger, 2008, pp. 73-84. An extended version is available as CoreGRID Technical Report TR-0120, December 2007. Google Scholar
  8. 8.
    H. Casanova, A. Legrand, D. Zagorodnov, and F. Berman. Heuristics for scheduling parameter sweep applications in Gild environments. In Proceedings of the 9th Heterogeneous Computing Works lwp (HCW’OO), 2000, pp. 349-363. Google Scholar
  9. 9.
    E. Deelman, D. Gannon, M. Shields, and I. Taylor. Workflows and c-Science: An overview of workflow system features and capabilities. Future Generation Computer Systems, 25, 2009, pp. 528-540. Google Scholar
  10. 10.
    E. Deelman, G. Singh, M. H. Su,J. Blythe,Y. Gi1,C. Kesselman, G. Mehta, K. Vahi, G. B. Berilman, J. Good, A. Laity, J. C. Jacob, and D. S. Katz. Pegasus: A framework for mapping complex scientific workflows onto distributed systems. Scientific Programning, 13(3), 2005, pp. 219-237. Google Scholar
  11. 11.
    E Dong and S. G. Aid. PFAS: A Resource-Performance-Fluctuation-Aware Workflow Scheduling Aigorithm for Gild Computing. In Proceedings of IPDPS 2007, 2007. Google Scholar
  12. 12.
    R. Huang, H. Casanova, and A. A. Chien. Using Virtual Grids to Simplify Application Scheduling. In Proceedings of IPDPS 2006, 2006. Google Scholar
  13. 13.
    Y.-K. Kwok and I. Ahmad. Static scheduling algorithms for allocating directed task graphs to multiprocessors. ACM Computing Surveys, 31(4), December 1999, pp. 406-471. Google Scholar
  14. 14.
    K. Lee, N. W. Paton, R. Sakellañou, E. Deelman, A. A. A. Fernandes, and G. Mehta. Adaptive Workflow Processing and Execution in Pegasus. In 3rd International Workshop on Wor4flow Management and Applications in Grid Environments (WaGeO8) (in Proceedings of the Third International Conference on Grid and Pervasive Computing Symposia/Workshops, May 25-28 2008, Kunrning, China), 2008, pp. 99106. Google Scholar
  15. 15.
    M. M. Lopez, E. Heymann, and M. A. Senar. Analysis of Dynamic Heuristics for Workflow Scheduling on Grid Systems. In Proceedings of the 5th International Symposium on Parallel and Distributed Computing (ISPDC), 2006, pp. 199-207. Google Scholar
  16. 16.
    A. Mandal, K. Kennedy, C. Koelbel, G. Mann, J. Mellor-Crunimey, B. Liu, and L. Johnsson. Scheduling Strategies for Mapping Application Workflows onto the Grid. In IEEE International Symposium on High Performance Distributed Computing (HPDC 2005), 2005. Google Scholar
  17. 17.
    Montage. An Astronomical Image Mosaic Engine. http://montage.ipac.caltech.edu/ Google Scholar
  18. 18.
    M. L. Pinedo. Scheduling: Theory, Algorithms, and Systems. Springer, 2008. Google Scholar
  19. 19.
    A. Ramakrishnan, G. Singh, H. Zhao, E. Deehrnin, R. Sakellariou, K. Vahi, K. Blackburn, D. Meyers, and M. Sainidi. Scheduling Data-Intensive Workflows onto Stomge-Constrained Distributed Resources. In Proceedings of the 7th IEEE International Symposium on Cluster Computing and the Grid (CCGrid’07), 2007, pp. 401-409. Google Scholar
  20. 20.
    R. Sakellariou and H. Zhao. A Hybrid Heuristic for DAG Scheduling on Heterogeneous Systems. In Proceedings of the 13th Heterogeneous Computing Workshop (HCW’04), Santa Fe, New Mexico, USA, on April 26, 2004. Google Scholar
  21. 21.
    R. Sakellariou and H. Zhao. A low-cost rescheduling policy for efficient mapping of work- flows on grid systems. Scientific Programming, 12(4), December 2004. Google Scholar
  22. 22.
    I. J. Taylor, E. Deelman, D. B. Gannon, and M. Schields. Workfiows for c-Science. Scientific Wor4flows for Gridy. Springer, 2007. Google Scholar
  23. 23.
    H. Topcuoglu, S. Hariri, and M.-Y. Wu. Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Transactions on Parallel and Distributed Systems, 13(3), Maich 2002, pp. 260-274. Google Scholar
  24. 24.
    M. Wieczorek, R. Pmdan, and T. Fahringer. Scheduling of Scientific Workflows in the ASKALON Grid Environment. In SIGMOD Record, volume 34(3), September 2005. Google Scholar
  25. 25.
    M. Wieczorek, R. Prodan, and T. Fahffngen Comparison of Workflow Scheduling Stmtegies on the Grid. In Proceedings of the Second Grid Resource Management Workshop (GRMW’2005), Spffnger, LNCS 3911,2006, pp.792-800. Google Scholar
  26. 26.
    J. Yu and R. Buyya. A Taxonomy of Scientific Workflow Systems for Gild Computing. In SIGMOD Record, volume 34(3), September 2005. Google Scholar
  27. 27.
    J. Yu, R. Buyya, and K. Ramamohanamo. Workflow Scheduling Algorithms for Grid Computing. In Studies in Computational Intelligence, volume 146, Springer, 2008, pp. 173-214. Google Scholar
  28. 28.
    H. Zhao and R. Sakellariou. An experimental investigation into the rankfunction of the heterogeneous earliest finish time scheduling algoffthm. In Euro-Par 2003, Spffnger-Verlag, LNCS 2790,2003. Google Scholar
  29. 29.
    Y. Zhao, J. Dobson, I. Foster, L. Moreau, and M. Wilde. A notation and system for expressing and executing cleanly typed workflows on messy scientific data. In SIGMOD Record, volume 34(3), September 2005, pp. 37-43. Google Scholar

Copyright information

© Springer US 2010

Authors and Affiliations

  1. 1.School of Computer ScienceUniversity of ManchesterManchesterUnited Kingdom
  2. 2.USC Information Sciences InstituteMarina Del ReyUSA

Personalised recommendations