Decentralized Preemptive Scheduling Across Heterogeneous Multi-core Grid Resources

  • Arun BalasubramanianEmail author
  • Alan Sussman
  • Norman Sadeh
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8429)


The recent advent of multi-core computing environments increases the heterogeneity of grid resources and the complexity of managing them, making efficient load balancing challenging. In an environment where jobs are submitted regularly into a grid which is already executing several jobs, it becomes important to provide low job turn-around times and high throughput for the users. Typically, the grids employ a First Come First Serve (FCFS) method of executing the jobs in the queue which results in suboptimal turn-around times and wait times for most jobs. Hence a conventional FCFS scheduling strategy does not suffice to reduce the average wait times across all jobs. In this paper, we propose new decentralized preemptive scheduling strategies that backfill jobs locally and dynamically migrate waiting jobs across nodes to leverage residual resources, while guaranteeing (on a best effort basis) bounded turn-around and waiting times for all jobs. The methods attempt to maximize total throughput and minimize average waiting time while balancing load across available grid resources. Experimental results for both intra-node and internode scheduling via simulation show that our scheduling schemes perform considerably better than the conventional FCFS approach of a distributed or a centralized scheduler.


Distributed systems Scheduling Preemptive scheduling Performance Load balancing Heterogeneous processors Grid computing 



We appreciate the comments received from anonymous reviewers of the JSSPP 2013 workshop. They pointed out some key issues that has led us to do further research on this topic. We thank Manjunath Gopinath, Bin Liu, Sarat Babu Eruvuru, Bhavani Bhaskar and Abhishek Prasad for their participation in discussions and their feedback on this idea.


  1. 1.
    Al-Azzoni, I., Down, D.G.: Dynamic scheduling for heterogeneous desktop grids. J. Parallel Distrib. Comput. 70(12), 1231–1240 (2010)CrossRefzbMATHGoogle Scholar
  2. 2.
    Bach, M.J.: The Design of the UNIX Operating System, Chapter 8 - Process Scheduling and Time. Prentice Hall, Upper Saddle River (1986)Google Scholar
  3. 3.
    Berman, F., Wolski, R., Figueira, S., Schopf, J., Shao, G.: Application level scheduling on distributed heterogenous networks. In: Proceedings of the 1996 ACM/IEEE Conference on Supercomputing. ACM/IEEE (1996)Google Scholar
  4. 4.
    Zhou, D., Lo, V.: Wave scheduler: scheduling for faster turnaround time in peer-based desktop grid systems. In: Feitelson, D.G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2005. LNCS, vol. 3834, pp. 194–218. Springer, Heidelberg (2005) CrossRefGoogle Scholar
  5. 5.
    Zhou, D., Lo, V.: Wavegrid: a scalable fast-turnaround heterogeneous peer-based desktop grid system. In: Proceedings of the 20th International Parallel and Distributed Processing Symposium (IPDPS2006), April 2006. IEEE Computer Society Press (2006)Google Scholar
  6. 6.
    Harchol-Balter, M., Schroeder, B., Bansal, N., Agrawal, M.: Size-based scheduling to improve web performance. ACM Trans. Comput. Syst. 21(2), 207–233 (2003)CrossRefGoogle Scholar
  7. 7.
    Jiménez, V.J., Vilanova, L., Gelado, I., Gil, M., Fursin, G.G., Navarro, N.: Predictive runtime code scheduling for heterogeneous architectures. In: Seznec, A., Emer, J., O’Boyle, M., Martonosi, M., Ungerer, T. (eds.) HiPEAC 2009. LNCS, vol. 5409, pp. 19–33. Springer, Heidelberg (2009) CrossRefGoogle Scholar
  8. 8.
    Kim, J.S., Keleher, P., Marsh, M., Bhattacharjee, B., Sussman, A.: Using content-addressable networks for load balancing in desktop grids. In: Proceedings of the 16th IEEE International Symposium on High Performance, Distributed Computing (HPDC-16), June 2007 (2007)Google Scholar
  9. 9.
    Lee, J., Keleher, P., Sussman, A.: Decentralized dynamic scheduling across heterogeneous multi-core desktop grids. In: Proceedings of the 19th International Heterogeneity in Computing Workshop (HCW2010), April 2010. IEEE Computer Society Press (2010)Google Scholar
  10. 10.
    Lee, J., Keleher, P., Sussman, A.: Supporting computing element heterogeneity in p2p grids. In: Proceedings of the IEEE Cluster 2011 Conference, September 2011. IEEE Computer Society Press (2011)Google Scholar
  11. 11.
    Li, C., Ding, C., Shen, K.: Quantifying the cost of context switch. In: Proceedings of the 2007 Workshop on Experimental Computer Science ExpCS ’07. ACM, New York (2007)Google Scholar
  12. 12.
    Litzkow, M., Livny, M., Mutka, M.: Condor-a hunter of idle workstations. In: 8th International Conference on Distributed, Computing Systems, pp. 104–111 (1988)Google Scholar
  13. 13.
    Niu, S., Zhai, J., Ma, X., Liu, M., Zhai, Y., Chen, W., Zheng, W.: Employing checkpoint to improve job scheduling in large-scale systems. In: Cirne, W., Desai, N., Frachtenberg, E., Schwiegelshohn, U. (eds.) JSSPP 2012. LNCS, vol. 7698, pp. 36–55. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  14. 14.
    Raman, R., Livny, M., Solomon, M.: Matchmaking: distributed resource management for high throughput computing. In: Proceedings of the 7th International Symposium on High Performance, Distributed Computing, July 1998, pp. 140–146 (1998)Google Scholar
  15. 15.
    Moore, S.: Multicore is bad news for super computers. IEEE Spectrum. 45(11), 15 (2008)CrossRefGoogle Scholar
  16. 16.
    Snell, Q.O., Clement, M.J., Jackson, D.B.: Preemption based backfill. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2002. LNCS, vol. 2537, pp. 24–37. Springer, Heidelberg (2002) CrossRefGoogle Scholar
  17. 17.
    Ratnasamy, S., Francis, P., Handley, M., Karp, R., Shenker, S.: A scalable content addressable network. In: Proceedings of the ACM SIGCOMM Conference, August 2001 (2001)Google Scholar
  18. 18.
    Stallings, W.: Operating Systems: Internals and Design Principles, 4th edn. Prentice Hall, Upper Saddle River (2001). ISBN: 0-13-031999-6Google Scholar
  19. 19.
    Tanenbaum, A.S.: Modern Operating Systems, 3rd edn. Pearson Education, Upper Saddle River (2008). ISBN: 0-13-600663-9Google Scholar
  20. 20.
    Thompson, K.: UNIX implementation. Bell Syst. Tech. J. 57, 1931–1946 (1978)CrossRefGoogle Scholar
  21. 21.
    Leinberger, W., Karypis, G., Kumar, V.: Job scheduling in the presence of multiple resource requirements. In: Supercomputing ’99: Proceedings of the 1999 ACM/IEEE Conference on Supercomputing (CDROM), p. 47. ACM, NewYork (1999)Google Scholar
  22. 22.
    Leinberger, W., Karypis, G., Kumar, V., Biswas, R.: Load balancing across near-homogeneous multi-resource servers. In: Proceedings of the 9th Heterogeneous Computing Workshop, appears with the Proceedings of IPDPS 2000, pp. 60–71 (2000)Google Scholar
  23. 23.
    Xhafa, F., Abraham, A.: Computational models and heuristic methods for grid scheduling problems. Future Gener. Comput. Syst. 26(4), 608–621 (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Arun Balasubramanian
    • 1
    Email author
  • Alan Sussman
    • 2
  • Norman Sadeh
    • 1
  1. 1.Institute for Software ResearchCarnegie Mellon UniversityPittsburghUSA
  2. 2.Department of Computer ScienceUniversity of MarylandCollege ParkUSA

Personalised recommendations