Advertisement

Autonomous Resource-Aware Scheduling of Large-Scale Media Workflows

  • Stein Desmet
  • Bruno Volckaert
  • Filip De Turck
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6155)

Abstract

The media processing and distribution industry generally requires considerable resources to be able to execute the various tasks and workflows that constitute their business processes. The latter processes are often tied to critical constraints such as strict deadlines. A key issue herein is how to efficiently use the available computational, storage and network resources to be able to cope with the high work load. Optimizing resource usage is not only vital to scalability, but also to the level of QoS (e.g. responsiveness or prioritization) that can be provided. We designed an autonomous platform for scheduling and workflow-to-resource assignment, taking into account the different requirements and constraints. This paper presents the workflow scheduling algorithms, which consider the state and characteristics of the resources (computational, network and storage). The performance of these algorithms is presented in detail in the context of a European media processing and distribution use-case.

Keywords

Computational Resource Schedule Algorithm Data Resource Network Link Execution Path 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    IBBT: Independent films in progress (IFIP) (2008-2010), http://www.ibbt.be/en/project/ifip
  2. 2.
    Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness (Series of Books in the Mathematical Sciences). W. H. Freeman, New York (January 1979)Google Scholar
  3. 3.
    Yu, J., Buyya, R., Ramamohanarao, K.: Workflow scheduling algorithms for grid computing. In: Metaheuristics for Scheduling in Distributed Computing Environments. Springer, Heidelberg (2008)Google Scholar
  4. 4.
    Dong, F., Akl, S.G.: Scheduling algorithms for grid computing: State of the art and open problems. Technical report, School of Computing, Queen’s Univeristy (2006)Google Scholar
  5. 5.
    Kwok, Y.K., Ahmad, I.: Static scheduling algorithms for allocating directed task graphs to multiprocessors. ACM Comput. Surv. 31(4), 406–471 (1999)CrossRefGoogle Scholar
  6. 6.
    Andries, L.: Facing media traffic challenges. Broadcast Engineering (Febuary 2010)Google Scholar
  7. 7.
    IBBT: Grid enabled infrastructure for service oriented high definition media applications (GEISHA), http://www.ibbt.be/en/project/geisha
  8. 8.
    Volckaert, B., Wauters, T., De Leenheer, M., Thysebaert, P., De Turck, F., Dhoedt, B., Demeester, P.: Gridification of collaborative audiovisual organizations through the mediagrid framework. Future Gener. Comput. Syst. 24(5), 371–389 (2008)CrossRefGoogle Scholar
  9. 9.
    Harmer, T.: Gridcast–a next generation broadcast infrastructure? Cluster Computing 10(3), 277–285 (2007)CrossRefGoogle Scholar
  10. 10.
    Walsh, A.E.: The media grid: A public utility for digital media. In: 8th International Symposium on Spatial Media (ISSM) jointly held with IEEE International Conference on Computer and Information Technology, IEEE CIT (2007)Google Scholar
  11. 11.
    Basu, S., Adhikari, S., Kumar, R., Yan, Y., Hochmuth, R., Blaho, B.E.: mmgrid: Distributed resource management infrastructure for multimedia applications. In: IPDPS 2003: Proceedings of the 17th International Symposium on Parallel and Distributed Processing, Washington, DC, USA, pp. 88–1. IEEE Computer Society, Los Alamitos (2003)Google Scholar
  12. 12.
    Seinstra, F.J., Geusebroek, J.M., Koelma, D., Snoek, C.G.M., Worring, M., Smeulders, A.W.M.: High-performance distributed video content analysis with parallel-horus. IEEE MultiMedia 14(4), 64–75 (2007)CrossRefGoogle Scholar
  13. 13.
    Chrétienne, P., Coffman, E.G., Lenstra, J.K., Liu, Z. (eds.): Scheduling Theory and its Applications. John Wiley and Sons, Chichester (1995)zbMATHGoogle Scholar
  14. 14.
    Ardagna, D., Pernici, B.: Adaptive service composition in flexible processes 33, 369–384 (June 2007)Google Scholar
  15. 15.
    Alhusaini, A.H., Prasanna, V.K., Raghavendra, C.S.: A unified resource scheduling framework for heterogeneous computing environments. In: Proc. Eighth Heterogeneous Computing Workshop (HCW 1999), April 12, pp. 156–165 (1999)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2010

Authors and Affiliations

  • Stein Desmet
    • 1
  • Bruno Volckaert
    • 1
  • Filip De Turck
    • 1
  1. 1.Department of Information Technology(INTEC) - IBCNGhent University - IBBTGentBelgium

Personalised recommendations