Journal of Grid Computing

, Volume 11, Issue 3, pp 341–360

JETS: Language and System Support for Many-Parallel-Task Workflows

  • Justin M. Wozniak
  • Michael Wilde
  • Daniel S. Katz
Article
  • 123 Downloads

Abstract

Many-task computing is a well-established paradigm for implementing loosely coupled applications (tasks) on large-scale computing systems. However, few of the model’s existing implementations provide efficient, low-latency support for executing tasks that are tightly coupled multiprocessing applications. Thus, a vast array of parallel applications cannot readily be used effectively within many-task workloads. In this work, we present JETS, a middleware component that provides high performance support for many-parallel-task computing (MPTC). JETS is based on a highly concurrent approach to parallel task dispatch and on new capabilities now available in the MPICH2 MPI implementation and the ZeptoOS Linux operating system. JETS represents an advance over the few known examples of multilevel many-parallel-task scheduling systems: it more efficiently schedules and launches many short-duration parallel application invocations; it overcomes the challenges of coupling the user processes of each multiprocessing application invocation via the messaging fabric; and it concurrently manages many application executions in various stages. We report here on the JETS architecture and its performance on both synthetic benchmarks and an MPTC application in molecular dynamics.

Keywords

MPI MTC MPTC Swift JETS NAMD Workflow 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Justin M. Wozniak
    • 1
  • Michael Wilde
    • 1
  • Daniel S. Katz
    • 1
  1. 1.Argonne National LaboratoryArgonneUSA

Personalised recommendations