Advertisement

Moldable Job Scheduling for HPC as a Service

  • Kuo-Chan Huang
  • Tse-Chi Huang
  • Mu-Jung Tsai
  • Hsi-Ya Chang
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 276)

Abstract

As cloud computing emerges and gains acceptance, more and more software applications of various domains are transforming into the SaaS model. Recently, the concept of HPC as a Service (HPCaaS) was proposed to bring the traditional high performance computing field into the era of cloud computing. One of its goals aims to allow users to get easier access to HPC facilities and applications. This paper deals with related job submission and scheduling issues to achieve such goal. Traditional HPC users in supercomputing centers are required to specify the amount of processors to use upon job submission. However, we think this requirement might not be necessary for HPCaaS users since most modern parallel jobs are moldable and they usually could not know how to choose an appropriate amount of processors to allow their jobs to finish earlier. Therefore, we propose a moldable job scheduling approach which relieves HPC users’ burden of selecting an appropriate number of processors and can achieve even better system performance than existing job scheduling methods. The experimental results indicate that our approach can achieve up to 75% performance improvement than the traditional rigid processor allocation method and 3% improvement than previous moldable job scheduling methods.

Keywords

moldable job HPC as a Service processor allocation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    AbdelBaky, M., Parashar, M., Kim, H., JordanKirk, E.J., Sachdeva, V., Sexton, J., Jamjoom, H., Shae, Z.Y., Pencheva, G., Tavakoli, R., Wheeler, M.F.: Enabling High Performance Computing as a Service. IEEE Computer 45, 72–80 (2012)CrossRefGoogle Scholar
  2. 2.
    Feitelson, D.G., Weil, A.M.: Utilization and Predictability in Scheduling the IBM SP2 with Backfilling. In: 12th Int’l Parallel Processing Symp., pp. 542–546 (April 1998)Google Scholar
  3. 3.
    Gibbons, R.: A Historical Application Profiler for Use by Parallel Schedulers. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1997 and JSSPP 1997. LNCS, vol. 1291, pp. 58–77. Springer, Heidelberg (1997)Google Scholar
  4. 4.
    Lifka, D.: The ANL/IBM SP Scheduling System. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1995 and JSSPP 1995. LNCS, vol. 949, pp. 295–303. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  5. 5.
    Mu’alem, A.W., Feitelson, D.G.: Utilization, Predictability, Workloads, and User Runtime Estimate in Scheduling the IBM SP2 with Backfilling. IEEE Transactions on Parallel and Distributed Systems 12(6), 529–543 (2001)CrossRefGoogle Scholar
  6. 6.
    Downey, A.B.: A Model for Speedup of Parallel Programs. UC Berkeley EECS Technical Report, No. UCB/CSD-97-933 (January 1997)Google Scholar
  7. 7.
    Downey, A.B.: A Parallel Workload Model and Its Implications for Processor Allocation. In: The 6th International Symposium on High Performance Distributed Computing (1997)Google Scholar
  8. 8.
    Srinivasan, S., Krishnamoorthy, S., Sadayappan, P.: A Robust Scheduling Strategy for Moldable Scheduling of Parallel Jobs. In: 5th IEEE International Conference on Cluster Computing, pp. 92–99 (2003)Google Scholar
  9. 9.
    Srinivasan, S., Subramani, V., Kettimuthu, R., Holenarsipur, P., Sadayappan, P.: Effective Selection of Partition Sizes for Moldable Scheduling of Parallel Jobs. In: Sahni, S.K., Prasanna, V.K., Shukla, U. (eds.) HiPC 2002. LNCS, vol. 2552, pp. 174–183. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  10. 10.
    Sun, H., Cao, Y., Hsu, W.J.: Efficient Adaptive Scheduling of Multiprocessors with Stable Parallelism Feedback. IEEE Transactions on Parallel and Distributed System 22(4) (April 2011)Google Scholar
  11. 11.
    Huang, K.C.: Performance Evaluation of Adaptive Processor Allocation Policies for Moldable Parallel Batch Jobs. In: 3th Workshop on Grid Technologies and Applications (2006)Google Scholar
  12. 12.
  13. 13.
    Feitelson, D.G.: A Survey of Scheduling in Multiprogrammed Parallel Systems, Research Report RC 19790 (87657), IBM T. J. Watson Research Center (October 1994)Google Scholar
  14. 14.
    Feitelson, D.G., Rudolph, L., Schweigelshohn, U., Sevcik, K., Wong, P.: Theory and Practice in Parallel Job Scheduling. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1997 and JSSPP 1997. LNCS, vol. 1291, pp. 1–34. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  15. 15.
    Kleinrock, L., Huang, J.H.: On parallel processing systems: Amdahl’s law generalized and some results on optimal design. IEEE Trans. Softw. Eng. 18(5) (1992)Google Scholar
  16. 16.
    Huang, K.C., Huang, T.C., Tung, T.H., Shih, P.Z.: Effective Processor Allocation for Moldable Jobs with Application Speedup Model. In: Proceedings of the International Computer Symposium, ICS 2012, Taiwan (2012)Google Scholar
  17. 17.
    The Message Passing Interface (MPI) standard, http://www.mcs.anl.gov/research/projects/mpi/

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Kuo-Chan Huang
    • 1
  • Tse-Chi Huang
    • 1
  • Mu-Jung Tsai
    • 1
  • Hsi-Ya Chang
    • 2
  1. 1.Department of Computer ScienceNational Taichung University of EducationTaichungTaiwan
  2. 2.National Center for High-Performance ComputingHsinchuTaiwan

Personalised recommendations