Batch-Scheduling Dags for Internet-Based Computing
The process of scheduling computations for Internet-based computing presents challenges not encountered with more traditional computing platforms. The looser coupling among participating computers makes it harder to utilize remote clients well, and raises the specter of a kind of “gridlock” that ensues when a computation stalls because no new tasks are eligible for execution. This paper studies the problem of scheduling computation-dags in a manner that renders tasks eligible for execution at the maximum possible rate. Earlier work has developed a framework for such scheduling when a new task is allocated to a remote client as soon as it returns the results from an earlier task. The proof in that work that many dags cannot be scheduled optimally within this paradigm signaled the need for a companion theory that addresses the scheduling problem for all computation-dags. A new, batched, scheduling paradigm for Internet-based computing is developed in this work. Although optimal batched schedules always exist, computing such a schedule is NP-Hard, even for bipartite dags. In response, a polynomial-time algorithm is developed for producing optimal batched schedules for a rich family of dags obtained by “composing” tree-structured building-block dags. Finally, a fast heuristic schedule is developed for “expansive” dags.
KeywordsSource Node Sink Node Rich Family Composition Operation Pebble Game
Unable to display preview. Download preview PDF.
- 1.Buyya, R., Abramson, D., Giddy, J.: A case for economy Grid architecture for service oriented Grid computing. In: 10th Heterogeneous Computing Wkshp (2001)Google Scholar
- 2.Cirne, W., Marzullo, K.: The Computational Co-Op: gathering clusters into a metacomputer. In: 13th Intl. Parallel Processing Symp., pp. 160–166 (1999)Google Scholar
- 4.Foster, I., Kesselman, C.: The Grid: Blueprint for a New Computing Infrastructure, 2nd edn. Morgan-Kaufmann, San Francisco (2004)Google Scholar
- 5.Foster, I., Kesselman, C., Tuecke, S.: The anatomy of the Grid: enabling scalable virtual organizations. Intl. J. Supercomputer Applications (2001)Google Scholar
- 6.Gao, L., Malewicz, G.: Internet computing of tasks with dependencies using unreliable workers. In: 8th Intl. Conf. on Principles of Distributed Systems, pp. 315–325 (2004)Google Scholar
- 7.Kondo, D., Casanova, H., Wing, E., Berman, F.:: Models and scheduling guidelines for global computing applications. Intl. Parallel and Distr. Processing Symp (2002)Google Scholar
- 8.Korpela, E., Werthimer, D., Anderson, D., Cobb, J., Lebofsky, M.: SETI@home: massively distributed computing for SETI. In: Dubois, P.F. (ed.) Computing in Sci. and Engr., IEEE Computer Soc. Press, Los Alamitos (2000)Google Scholar
- 9.Malewicz, G.: Parallel Scheduling of Complex Dags under Uncertainty. 17th ACM Symposium on Parallelism in Algorithms and Architectures (2005) (to appear)Google Scholar
- 10.Malewicz, G.: Implementation and Experiments with an Algorithm for Parallel Scheduling of Complex Dags under Uncertainty (2005) (submitted for publication)Google Scholar
- 11.Malewicz, G., Rosenberg, A.L., Yurkewych, M.: On Scheduling Complex Dags for Internet-Based Computing. IEEE Intl. Parallel and Distr. Processing Symp., 66 (2005)Google Scholar
- 12.Paterson, M.S., Hewitt, C.E.: Comparative schematology. In: Project MAC Conf. on Concurrent Systems and Parallel Computation, pp. 119–127. ACM Press, New York (1970)Google Scholar
- 16.Sun, X.-H., Wu, M.: GHS: A performance prediction and task scheduling system for Grid computing. In: IEEE Intl. Parallel and Distributed Processing Symp. (2003)Google Scholar