Abstract
The process of scheduling computations for Internet-based computing presents challenges not encountered with more traditional computing platforms. The looser coupling among participating computers makes it harder to utilize remote clients well, and raises the specter of a kind of “gridlock” that ensues when a computation stalls because no new tasks are eligible for execution. This paper studies the problem of scheduling computation-dags in a manner that renders tasks eligible for execution at the maximum possible rate. Earlier work has developed a framework for such scheduling when a new task is allocated to a remote client as soon as it returns the results from an earlier task. The proof in that work that many dags cannot be scheduled optimally within this paradigm signaled the need for a companion theory that addresses the scheduling problem for all computation-dags. A new, batched, scheduling paradigm for Internet-based computing is developed in this work. Although optimal batched schedules always exist, computing such a schedule is NP-Hard, even for bipartite dags. In response, a polynomial-time algorithm is developed for producing optimal batched schedules for a rich family of dags obtained by “composing” tree-structured building-block dags. Finally, a fast heuristic schedule is developed for “expansive” dags.
A portion of the research of G. Malewicz was done while visiting the Univ. of Massachusetts Amherst. The research of A. Rosenberg was supported in part by NSF Grant CCF-0342417.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Buyya, R., Abramson, D., Giddy, J.: A case for economy Grid architecture for service oriented Grid computing. In: 10th Heterogeneous Computing Wkshp (2001)
Cirne, W., Marzullo, K.: The Computational Co-Op: gathering clusters into a metacomputer. In: 13th Intl. Parallel Processing Symp., pp. 160–166 (1999)
Cook, S.A.: An observation on time-storage tradeoff. J. Comp. Syst. Scis. 9, 308–316 (1974)
Foster, I., Kesselman, C.: The Grid: Blueprint for a New Computing Infrastructure, 2nd edn. Morgan-Kaufmann, San Francisco (2004)
Foster, I., Kesselman, C., Tuecke, S.: The anatomy of the Grid: enabling scalable virtual organizations. Intl. J. Supercomputer Applications (2001)
Gao, L., Malewicz, G.: Internet computing of tasks with dependencies using unreliable workers. In: 8th Intl. Conf. on Principles of Distributed Systems, pp. 315–325 (2004)
Kondo, D., Casanova, H., Wing, E., Berman, F.:: Models and scheduling guidelines for global computing applications. Intl. Parallel and Distr. Processing Symp (2002)
Korpela, E., Werthimer, D., Anderson, D., Cobb, J., Lebofsky, M.: SETI@home: massively distributed computing for SETI. In: Dubois, P.F. (ed.) Computing in Sci. and Engr., IEEE Computer Soc. Press, Los Alamitos (2000)
Malewicz, G.: Parallel Scheduling of Complex Dags under Uncertainty. 17th ACM Symposium on Parallelism in Algorithms and Architectures (2005) (to appear)
Malewicz, G.: Implementation and Experiments with an Algorithm for Parallel Scheduling of Complex Dags under Uncertainty (2005) (submitted for publication)
Malewicz, G., Rosenberg, A.L., Yurkewych, M.: On Scheduling Complex Dags for Internet-Based Computing. IEEE Intl. Parallel and Distr. Processing Symp., 66 (2005)
Paterson, M.S., Hewitt, C.E.: Comparative schematology. In: Project MAC Conf. on Concurrent Systems and Parallel Computation, pp. 119–127. ACM Press, New York (1970)
Rosenberg, A.L.: On scheduling mesh-structured computations for Internet-based computing. IEEE Trans. Comput. 53, 1176–1186 (2004)
Rosenberg, A.L., Sudborough, I.H.: Bandwidth and pebbling. Computing 31, 115–139 (1983)
Rosenberg, A.L., Yurkewych, M.: Guidelines for scheduling some common computation-dags for Internet-based computing. IEEE Trans. Comput. 54, 428–438 (2005)
Sun, X.-H., Wu, M.: GHS: A performance prediction and task scheduling system for Grid computing. In: IEEE Intl. Parallel and Distributed Processing Symp. (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Malewicz, G., Rosenberg, A.L. (2005). Batch-Scheduling Dags for Internet-Based Computing. In: Cunha, J.C., Medeiros, P.D. (eds) Euro-Par 2005 Parallel Processing. Euro-Par 2005. Lecture Notes in Computer Science, vol 3648. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11549468_31
Download citation
DOI: https://doi.org/10.1007/11549468_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28700-1
Online ISBN: 978-3-540-31925-2
eBook Packages: Computer ScienceComputer Science (R0)