On the Heterogeneity Bias of Cost Matrices When Assessing Scheduling Algorithms

  • Louis-Claude Canon
  • Laurent Philippe
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9233)


Assessing the performance of scheduling heuristics through simulation requires to generate synthetic instances of tasks and machines with well-identified properties. Carefully controlling these properties is mandatory to avoid any bias. We consider the scheduling problem consisting of allocating independent sequential tasks on unrelated processors while minimizing the maximum execution time. In this problem, the instance is a cost matrix that specifies the execution cost of any task on any machine. This paper proposes a measure for quantifying the heterogeneity properties of a cost matrix. An analysis of two classical methods used in the literature reveals a bias in previous studies. A new method is proposed to generate instances with given heterogeneity properties and it is shown that they have a significant impact on several heuristics.


Schedule Problem Cost Matrix Execution Cost Schedule Heuristic Instance Property 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Al-Qawasmeh, A., Pasricha, S., Maciejewski, A., Siegel, H.: Power and thermal-aware workload allocation in heterogeneous data centers. TC 64(2), 477–491 (2013)MathSciNetGoogle Scholar
  2. 2.
    Al-Qawasmeh, A., Maciejewski, A., Roberts, R.G., Siegel, H.: Characterizing task-machine affinity in heterogeneous computing environments. In: IPDPSW (2011)Google Scholar
  3. 3.
    Al-Qawasmeh, A., Maciejewski, A., Siegel, H.: Characterizing heterogeneous computing environments using singular value decomposition. In: IPDPSW (2010)Google Scholar
  4. 4.
    Al-Qawasmeh, A., Maciejewski, A., Wang, H., Smith, J., Siegel, H., Potter, J.: Statistical measures for quantifying task and machine heterogeneities. J. Supercomput. 57(1), 34–50 (2011)CrossRefGoogle Scholar
  5. 5.
    Ali, S.: A comparative study of dynamic mapping heuristics for a class of independent tasks onto heterogeneous computing systems. Ph.D. thesis, Purdue University (1999)Google Scholar
  6. 6.
    Ali, S., Siegel, H., Maheswaran, M., Hensgen, D.: Task execution time modeling for heterogeneous computing systems. In: HCW, pp. 185–199. IEEE (2000)Google Scholar
  7. 7.
    Ali, S., Siegel, H., Maheswaran, M., Hensgen, D., Ali, S.: Representing task and machine heterogeneities for heterogeneous computing systems. Tamkang J. Sci. Eng. 3(3), 195–208 (2000)Google Scholar
  8. 8.
    Armstrong, Jr. R.K.: Investigation of effect of different run-time distributions on SmartNet performance. Technical report, DTIC Document (1997)Google Scholar
  9. 9.
    Bardsiri, A.K., Hashemi, S.M.: A comparative study on seven static mapping heuristics for grid scheduling problem. IJSEIA 6(4), 247–256 (2012)Google Scholar
  10. 10.
    Braun, T.D., Siegel, H., Beck, N., Bölöni, L.L., Maheswaran, M., et al.: A comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. JPDC 61(6), 810–837 (2001)Google Scholar
  11. 11.
    Canon, L.C., Philippe, L.: On the heterogeneity bias of cost matrices when assessing scheduling algorithms. Technical report, FEMTO-ST, February 2015Google Scholar
  12. 12.
    Casanova, H., Legrand, A., Zagorodnov, D., Berman, F.: Heuristics for scheduling parameter sweep applications in grid environments. In: HCW, pp. 349–363 (2000)Google Scholar
  13. 13.
    Diaz, C.O., Pecero, J.E., Bouvry, P.: Scalable, low complexity, and fast greedy scheduling heuristics for highly heterogeneous distributed computing systems. J. Supercomput. 67(3), 837–853 (2014)CrossRefGoogle Scholar
  14. 14.
    Friese, R., Khemka, B., Maciejewski, A., Siegel, H.J., Koenig, G., et al.: An analysis framework for investigating the trade-offs between system performance and energy consumption in a heterogeneous computing environment. In: IPDPSW (2013)Google Scholar
  15. 15.
    Ghosh, S., Henderson, S.G.: Behavior of the norta method for correlated random vector generation as the dimension increases. ACM TOMACS 13(3), 276–294 (2003)CrossRefGoogle Scholar
  16. 16.
    Graham, R.L.: Bounds on multiprocessing timing anomalies. J. Appl. Math. 17(2), 416–429 (1969)zbMATHGoogle Scholar
  17. 17.
    Graham, R.L., Lawler, E.L., Lenstra, J.K., Kan, A.H.G.R.: Optimization and approximation in deterministic sequencing and scheduling: a survey. Ann. Discrete Math. 5, 287–326 (1979)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Khemka, B., Friese, R., Pasricha, S., Maciejewski, A., Siegel, H., et al.: Utility maximizing dynamic resource management in an oversubscribed energy-constrained heterogeneous computing system. In: Sustainable Computing: Informatics and Systems (2014)Google Scholar
  19. 19.
    Leung, J.Y.T. (ed.): Handbook of Scheduling: Algorithms, Models, and Performance Analysis. Chapman & Hall/CCR, London (2004)Google Scholar
  20. 20.
    Luo, P., Lü, K., Shi, Z.: A revisit of fast greedy heuristics for mapping a class of independent tasks onto heterogeneous computing systems. JPDC 67(6), 695–714 (2007)zbMATHGoogle Scholar
  21. 21.
    Saltelli, A., Chan, K., Scott, E.M.: Sensitivity analysis. Wiley, New York (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.FEMTO-ST/CNRS – Université de Franche-Comté/UBFCBesançonFrance

Personalised recommendations