Modeling Parallel System Workloads with Temporal Locality

  • Tran Ngoc Minh
  • Lex Wolters
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5798)

Abstract

In parallel systems, similar jobs tend to arrive within bursty periods. This fact leads to the existence of the locality phenomenon, a persistent similarity between nearby jobs, in real parallel computer workloads. This important phenomenon deserves to be taken into account and used as a characteristic of any workload model. Regrettably, this property has received little if any attention of researchers and synthetic workloads used for performance evaluation to date often do not have locality. With respect to this research trend, Feitelson has suggested a general repetition approach to model locality in synthetic workloads [6]. Using this approach, Li et al. recently introduced a new method for modeling temporal locality in workload attributes such as run time and memory [14]. However, with the assumption that each job in the synthetic workload requires a single processor, the parallelism has not been taken into account in their study. In this paper, we propose a new model for parallel computer workloads based on their result. In our research, we firstly improve their model to control locality of a run time process better and then model the parallelism. The key idea for modeling the parallelism is to control the cross-correlation between the run time and the number of processors. Experimental results show that not only the cross-correlation is controlled well by our model, but also the marginal distribution can be fitted nicely. Furthermore, the locality feature is also obtained in our model.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Song, B., Ernemann, C., Yahyapour, R.: Parallel computer workload modeling with markov chains. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2004. LNCS, vol. 3277, pp. 47–62. Springer, Heidelberg (2005)Google Scholar
  2. 2.
    Ernemann, C., Song, B., Yahyapour, R.: Scaling of workload traces. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 166–182. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  3. 3.
    Fraley, C., Raftery, A.E.: Model-Based Clustering, Discriminant Analysis, and Density Estimation. Journal of the American Statistical Association 97, 611–631 (2002)MATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Lifka, D.A.: The ANL/IBM SP Scheduling System. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1995 and JSSPP 1995. LNCS, vol. 949, pp. 295–303. Springer, Heidelberg (1995)Google Scholar
  5. 5.
    Jackson, D.B., Snell, Q., Clement, M.J.: Core algorithms of the maui scheduler. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 2001. LNCS, vol. 2221, pp. 87–102. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  6. 6.
    Feitelson, D.G.: Locality of Sampling and Diversity in Parallel System Workloads. In: Proceedings of 21st ACM International Conference on Supercomputing. ACM Press, USA (2007)Google Scholar
  7. 7.
    Feitelson, D.G.: Workload Modeling for Computer Systems Performance Evaluation. Book Draft, Version 0.18 (2008)Google Scholar
  8. 8.
  9. 9.
    Grid Workloads Archive, http://gwa.ewi.t-udelft.nl/
  10. 10.
  11. 11.
    Li, H.: Long Range Dependent Job Arrival Process and Its Implications in Grid Environments. In: Proceedings of MetroGrid Workshop, 1st International Conference on Networks for Grid Applications. ACM Press, France (2007)Google Scholar
  12. 12.
    Li, H., Groep, D., Wolters, L.: An Evaluation of Learning and Heuristic Techniques for Application Run Time Predictions. In: Proceedings of 11th Annual Conference of the Advance School for Computing and Imaging (ASCI), Netherlands (2005)Google Scholar
  13. 13.
    Li, H., Groep, D., Wolters, L.: Workload characteristics of a multi-cluster supercomputer. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2004. LNCS, vol. 3277, pp. 176–193. Springer, Heidelberg (2005)Google Scholar
  14. 14.
    Li, H., Muskulus, M., Wolters, L.: Modeling Correlated Workloads by Combining Model Based Clustering and a Localized Sampling Algorithm. In: Proceedings of 21st ACM International Conference on Supercomputing. ACM Press, USA (2007)Google Scholar
  15. 15.
    Mohamed, H., Epema, D.: The Design and Implementation of the KOALA Co-Allocating Grid Scheduler. In: Sloot, P.M.A., Hoekstra, A.G., Priol, T., Reinefeld, A., Bubak, M. (eds.) EGC 2005. LNCS, vol. 3470, pp. 640–650. Springer, Heidelberg (2005)Google Scholar
  16. 16.
    Wan, M., Moore, R., Kremenek, G., Steube, K.: A Batch Scheduler for the Intel Paragon with a Non-Contiguous Node Allocation Algorithm. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1996 and JSSPP 1996. LNCS, vol. 1162, pp. 48–64. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  17. 17.
  18. 18.
  19. 19.
  20. 20.
    Denning, P.J.: The Locality Principle. Communications of ACM 48, 19–24 (2005)CrossRefGoogle Scholar
  21. 21.
  22. 22.
    Gibbons, R.: A Historical Application Profiler for Use by Parallel Schedulers. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1997 and JSSPP 1997. LNCS, vol. 1291, pp. 58–77. Springer, Heidelberg (1997)Google Scholar
  23. 23.
    Riedi, R.H., Crouse, M.S., Ribeiro, V.J., Baraniuk, R.G.: A Multifractal Wavelet Model with Application to Network Traffic. Journal of IEEE Transactions on Information Theory 45(4), 992–1018 (1999)MATHCrossRefMathSciNetGoogle Scholar
  24. 24.
    Lublin, U., Feitelson, D.G.: The Workload on Parallel Supercomputers: Modeling the Characteristics of Rigid Jobs. Journal of Parallel and Distributed Computing 63 (2003)Google Scholar
  25. 25.
    Lo, V., Mache, J., Windisch, K.: A comparative study of real workload traces and synthetic workload models for parallel job scheduling. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1998, SPDP-WS 1998, and JSSPP 1998. LNCS, vol. 1459, pp. 25–46. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  26. 26.
    Smith, W., Foster, I., Taylor, V.: Predicting application run times using historical information. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS-WS 1998, SPDP-WS 1998, and JSSPP 1998. LNCS, vol. 1459, pp. 122–142. Springer, Heidelberg (1998)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Tran Ngoc Minh
    • 1
  • Lex Wolters
    • 1
  1. 1.Leiden Institute of Advanced Computer ScienceLeiden UniversityLeidenThe Netherlands

Personalised recommendations