Advertisement

Metascheduling Strategies in Distributed Computing with Non-dedicated Resources

  • Victor ToporkovEmail author
  • Alexey Tselishchev
  • Dmitry Yemelyanov
  • Petr Potekhin
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 307)

Abstract

In this chapter, we address problems of efficient computing in distributed systems with non-dedicated resources including utility Grid. There are global job flows from external users along with resource owner’s local tasks upon resource non-dedication condition. Competition for resource reservation between independent users, local and global job flows substantially complicates scheduling and the requirement to provide the necessary quality of service. A metascheduling concept, justified in this work, assumes a complex combination of job flow dispatching and application-level scheduling methods for parallel jobs, as well as resource sharing and consumption policies established in virtual organizations and based on economic principles.

Keywords

Distributed computing economic scheduling resource management co-allocation slot job task batch 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Garg, S.K., Buyya, R., Siegel, H.J.: Scheduling Parallel Applications on Utility Grids: Time and Cost Trade-off Management. In: 32nd Australasian Computer Science Conference (ACSC 2009), Wellington, New Zealand, pp. 151–159 (2009)Google Scholar
  2. 2.
    Degabriele, J.P., Pym, D.: Economic Aspects of a Utility Computing Service, Trusted Systems Laboratory HP Laboratories Bristol HPL-2007-101. Technical Report, pp. 1–23 (July 3, 2007)Google Scholar
  3. 3.
    Garg, S.K., Yeo, C.S., Anandasivam, A., Buyya, R.: Environment-conscious Scheduling of HPC Applications on Distributed Cloud-oriented Data Centers. J. Parallel and Distributed Computing 71(6), 732–749 (2011)CrossRefzbMATHGoogle Scholar
  4. 4.
    Tesauro, G., Bredin, J.L.: Strategic Sequential Bidding in Auctions Using Dynamic Programming. In: 1st International Joint Conference on Autonomous Agents and Multiagent Systems, Part 2, pp. 591–598. ACM, New York (2002)CrossRefGoogle Scholar
  5. 5.
    Thain, D., Tannenbaum, T., Livny, M.: Distributed Computing in Practice: the Condor Experience. J. Concurrency and Computation: Practice and Experience 17(2-4), 323–356 (2004)CrossRefGoogle Scholar
  6. 6.
    Berman, F.: High-performance Schedulers. In: Foster, I., Kesselman, C. (eds.) The Grid: Blueprint for a New Computing Infrastructure, pp. 279–309. Morgan Kaufmann, San Francisco (1999)Google Scholar
  7. 7.
    Yang, Y., Raadt, K., Casanova, H.: Multiround Algorithms for Scheduling Divisible Loads. IEEE Trans. Parallel and Distributed Systems 16(8), 1092–1102 (2005)CrossRefGoogle Scholar
  8. 8.
    Natrajan, A., Humphrey, M.A., Grimshaw, A.S.: Grid Resource Management in Legion. In: Nabrzyski, J., Schopf, J.M., Weglarz, J. (eds.) Grid Resource Management. State of the Art and Future Trends, pp. 145–160. Kluwer Academic Publishers, Boston (2003)Google Scholar
  9. 9.
    Beiriger, J., Johnson, W., Bivens, H.: Constructing the ASCI Grid. In: 9th IEEE Symposium on High Performance Distributed Computing, pp. 193–200. IEEE Press, New York (2000)Google Scholar
  10. 10.
    Frey, J., Foster, I., Livny, M.: Condor-G: A Computation Management Agent for Multi-institutional Grids. In: 10th International Symposium on High-Performance Distributed Computing, pp. 55–66. IEEE Press, New York (2001)Google Scholar
  11. 11.
    Abramson, D., Giddy, J., Kotler, L.: High Performance Parametric Modeling with Nimrod/G: Killer Application for the Global Grid? In: International Parallel and Distributed Processing Symposium, pp. 520–528. IEEE Press, New York (2000)Google Scholar
  12. 12.
    Foster, I., Kesselman, C., Tuecke, S.: The Anatomy of the Grid: Enabling Scalable Virtual Organizations. Int. J. of High Performance Computing Applications 15(3), 200–222 (2001)CrossRefGoogle Scholar
  13. 13.
    Ranganathan, K., Foster, I.: Decoupling Computation and Data Scheduling in Distributed Data-intensive Applications. In: 11th IEEE International Symposium on High Performance Distributed Computing, pp. 376–381. IEEE Press, New York (2002)Google Scholar
  14. 14.
    Kurowski, K., Nabrzyski, J., Oleksiak, A., Weglarz, J.: Multicriteria Aspects of Grid Resource Management. In: Nabrzyski, J., Schopf, J.M., Weglarz, J. (eds.) Grid Resource Management. State of the art and Future Trends, pp. 271–293. Kluwer Academic Publishers, Boston (2003)Google Scholar
  15. 15.
    Garg, S.K., Konugurthi, P., Buyya, R.: A Linear Programming-driven Genetic Algorithm for Meta-scheduling on Utility Grids. J. Par., Emergent and Distr. Systems 26, 493–517 (2011)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Buyya, R., Abramson, D., Giddy, J.: Economic Models for Resource Management and Scheduling in Grid Computing. J. Concurrency and Computation 14(5), 1507–1542 (2002)CrossRefzbMATHGoogle Scholar
  17. 17.
    Ernemann, C., Hamscher, V., Yahyapour, R.: Economic Scheduling in Grid Computing. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2002. LNCS, vol. 2537, pp. 128–152. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  18. 18.
    Lee, Y.C., Wang, C., Zomaya, A.Y., Zhou, B.B.: Profit-driven Scheduling for Cloud Services with Data Access Awareness. J. Par. and Distr. Computing 72(4), 591–602 (2012)CrossRefGoogle Scholar
  19. 19.
    Toporkov, V.V.: Job and Application-Level Scheduling in Distributed Computing. Ubiquitous Computing and Communication J. Applied Computing 4(3), 559–570 (2009)MathSciNetGoogle Scholar
  20. 20.
    Toporkov, V.V., Toporkova, A., Tselishchev, A., Yemelyanov, D.: Job and Application-Level Scheduling: an Integrated Approach for Achieving Quality of Service in Distributed Computing. In: 4th International Conference on Dependability of Computer Systems, pp. 202–209. IEEE CS Press, Los Alamitos (2009)Google Scholar
  21. 21.
    Toporkov, V.: Application-Level and Job-Flow Scheduling: an Approach for Achieving Quality of Service in Distributed Computing. In: Malyshkin, V. (ed.) PaCT 2009. LNCS, vol. 5698, pp. 350–359. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  22. 22.
    Aida, K., Casanova, H.: Scheduling Mixed-parallel Applications with Advance Reservations. In: 17th IEEE Int. Symposium on HPDC, pp. 65–74. IEEE CS Press, New York (2008)Google Scholar
  23. 23.
    Ando, S., Aida, K.: Evaluation of Scheduling Algorithms for Advance Reservations. Information Processing Society of Japan SIG Notes HPC-113, 37–42 (2007)Google Scholar
  24. 24.
    Elmroth, E., Tordsson, J.: A Standards-based Grid Resource Brokering Service Supporting Advance Reservations, Coallocation and Cross-Grid Interoperability. J. of Concurrency and Computation 25(18), 2298–2335 (2009)CrossRefGoogle Scholar
  25. 25.
    Toporkov, V., Toporkova, A., Bobchenkov, A., Yemelyanov, D.: Resource Selection Algorithms for Economic Scheduling in Distributed Systems. Procedia Computer Science 4, 2267–2276 (2011)CrossRefGoogle Scholar
  26. 26.
    Bailey Lee, C., Schwartzman, Y., Hardy, J., Snavely, A.: Are User Runtime Estimates Inherently Inaccurate? In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2004. LNCS, vol. 3277, pp. 253–263. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  27. 27.
    Toporkov, V., Tselishchev, A., Yemelyanov, D., Bobchenkov, A.: Dependable Strategies for Job-flows Dispatching and Scheduling in Virtual Organizations of Distributed Computing Environments. In: Zamojski, W., Mazurkiewicz, J., Sugier, J., Walkowiak, T., Kacprzyk, J. (eds.) Complex Systems and Dependability. AISC, vol. 170, pp. 289–304. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  28. 28.
  29. 29.
    Toporkov, V., Toporkova, A., Tselishchev, A., Yemelyanov, D.: Slot Selection Algorithms for Economic Scheduling in Distributed Computing with High QoS Rates. In: Zamojski, W., Mazurkiewicz, J., Sugier, J., Walkowiak, T., Kacprzyk, J. (eds.) New Results in Dependability & Comput. Syst. AISC, vol. 224, pp. 459–468. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  30. 30.
    Azzedin, F., Maheswaran, M., Arnason, N.: A Synchronous Co-allocation Mechanism for Grid Computing Systems. Cluster Computing 7, 39–49 (2004)CrossRefGoogle Scholar
  31. 31.
    Castillo, C., Rouskas, G.N., Harfoush, K.: Resource Co-allocation for Large-scale Distributed Environments. In: 18th ACM International Symposium on High Performance Distributed Compuing, pp. 137–150. ACM, New York (2009)Google Scholar
  32. 32.
    Takefusa, A., Nakada, H., Kudoh, T., Tanaka, Y.: An Advance Reservation-based Co-allocation Algorithm for Distributed Computers and Network Bandwidth on QoS-guaranteed Grids. In: Frachtenberg, E., Schwiegelshohn, U. (eds.) JSSPP 2010. LNCS, vol. 6253, pp. 16–34. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  33. 33.
    Blanco, H., Guirado, F., Lérida, J.L., Albornoz, V.M.: MIP Model Scheduling for Multi-clusters. In: Caragiannis, I., Alexander, M., Badia, R.M., Cannataro, M., Costan, A., Danelutto, M., Desprez, F., Krammer, B., Sahuquillo, J., Scott, S.L., Weidendorfer, J. (eds.) Euro-Par Workshops 2012. LNCS, vol. 7640, pp. 196–206. Springer, Heidelberg (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Victor Toporkov
    • 1
    Email author
  • Alexey Tselishchev
    • 2
  • Dmitry Yemelyanov
    • 1
  • Petr Potekhin
    • 1
  1. 1.National Research University “MPEI”MoscowRussia
  2. 2.CERN (European Organization for Nuclear Research)Genève 23Switzerland

Personalised recommendations