Skip to main content
SpringerLink
Log in

A historical application profiler for use by parallel schedulers

  • Conference paper
  • First Online:
Job Scheduling Strategies for Parallel Processing (JSSPP 1997)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1291))

Included in the following conference series:

Abstract

Scheduling algorithms that use application and system knowledge have been shown to be more effective at scheduling parallel jobs on a multiprocessor than algorithms that do not. This paper focuses on obtaining such information for use by a scheduler in a network of workstations environment.

The log files from three parallel systems are examined to determine both how to categorize parallel jobs for storage in a job database and what job information would be useful to a schedules. A Historical Profiler is proposed that stores information about programs and users, and manipulates this information to provide schedulers with execution time predictions. Several preemptive and non-preemptive versions of the FCFS, EASY and Least Work First scheduling algorithms are compared to evaluate the utility of the profiler. It is found that both preemption and the use of application execution time predictions obtained from the Historical Profiler lead to improved performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. G. Amdahl. Validity of the single-processor approach to achieving large-scale computing capabilities. In Proceedings of the 1967 AFIPS Conference, volume 30, AMPS Press, pages 483–485, 1967.

    Google Scholar 

  2. S.V. Anastasiadis and K.C. Sevcik. Parallel application scheduling on networks of workstations. To appear in: Journal of Parallel and Distributed Computing, June 1997.

    Google Scholar 

  3. T.B. Brecht and K. Guha. Using parallel program characteristics in dynamic processor allocation policies. Performance Evaluation, 27(8):519–539, October 1996.

    Google Scholar 

  4. M. Calzarossa and G. Serazzi. A characterization of the variation in time of workload arrival patterns. IEEE Transactions on Computers, C-34(2):156–162, February 1985.

    Google Scholar 

  5. A.C. Dusseau, R.H. Arpaci, and D.E. Culler. Effective distributed scheduling of parallel workloads. In Proceedings of the 1996 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, pages 25–36, 1996.

    Google Scholar 

  6. L. W. Dowdy. On the partitioning of multiprocessor systems. Technical Report Technical Report 88-06, Vanderbilt University, March 1988.

    Google Scholar 

  7. N.R. Draper and H. Smith. Applied Regression Analysis, 2nd ed. John Wiley and Sons, Toronto, 1981.

    Google Scholar 

  8. D.G. Feitelson and B. Nitzberg. Job characteristics of a production parallel scientific workload on the NASA Ames iPSC/ 860. In Proceedings of IPPS '95 Workshop on Job Scheduling Strategies for Parallel Processing, pages 215–227, April 1995.

    Google Scholar 

  9. R.B. Gibbons. A historical profiler for use by parallel schedulers. M. Sc. thesis, University of Toronto, Toronto, Ontario, Canada, 1997.

    Google Scholar 

  10. D. Ghosal, G. Serazzi, and S. K. Tripathi. The processor working set and its use in scheduling multiprocessor systems. IEEE Transactions on Software Engineering, 17(5):443–453, May 1991.

    Article  Google Scholar 

  11. S. Hotovy. Workload evolution on the Cornell Theory Center IBM SP2. In Proceedings of IPPS '96 Workshop on Job Scheduling Strategies for Parallel Processing, pages 15–22, April 1996.

    Google Scholar 

  12. S. Hotovy, D. Scheider, and T. O'Donnell. Analysis of the early workload on the Cornell Theory Center IBM SP2. In Proceedings of the 1996 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, pages 272–273, May 1996.

    Google Scholar 

  13. M. Kumar. Measuring parallelism in computation-intensive scien tific/engineering applications. IEEE Transactions on Computing, 37(9):1088–1098, September 1988.

    Article  Google Scholar 

  14. D.A. Lifka. The ANL/IBM SP scheduling system. In Proceedings of IPPS '95 Workshop on Job Scheduling Strategies for Parallel Processing, pages 187–191, April 1995.

    Google Scholar 

  15. LSF Users's Guide. Platform Computing Corporation, 5001 Yonge St, Suite 1401, North York, ONT, Canada M2N 6P6, 1996.

    Google Scholar 

  16. S. Majumdar, D.L. Eager, and R.B. Bunt. Scheduling in multiprogrammed parallel systems. In Proceedings of the 1988 ACM SIGMETRICS Conference on Measurement and Modelling of Computer Systems, pages 104–113, May 1990.

    Google Scholar 

  17. S. Majumdar, D.L. Eager, and R.B. Bunt. Characterization of programs for scheduling in multiprogrammed parallel systems. Performance Evaluation, 13(2):109–130, February 1991.

    Article  MathSciNet  Google Scholar 

  18. T.D. Nguyen, R. Vaswani, and J. Zahorjan. Parallel application characterization for multiprocessor scheduling policy design. In Proceedings of IPPS '96 Workshop on Job Scheduling Strategies for Parallel Processing, pages 105–118, April 1996.

    Google Scholar 

  19. T.D. Nguyen, R. Vaswani, and J. Zahorjan. Using runtime measured workload characteristics in parallel processor scheduling. In Proceedings of IPPS '96 Workshop on Job Scheduling Strategies for Parallel Processing, pages 93–104, April 1996.

    Google Scholar 

  20. E.W. Parsons. Using Resource Requirements in Multiprogrammed Multiprocessor Scheduling. Ph. D. thesis, University of Toronto, Toronto, Ontario, Canada, 1997.

    Google Scholar 

  21. J. Pasquale, B. Bittel, and D. Kraiman. A static and dynamic workload characterization study of the San Diego Supercomputer Center Cray X-MP. In Proceedings of the 1991 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, pages 218–219, 1991.

    Google Scholar 

  22. K.H. Park and L.W. Dowdy. Dynamic partitioning of multiprocessor systems. International Journal of Parallel Programming, 18(2):91–120, February 1989.

    Article  MathSciNet  Google Scholar 

  23. E.W. Parsons and K.C. Sevcik. Multiprocessor scheduling for high-variability service time distributions. In Proceedings of IPPS '95 Workshop on Job Scheduling Strategies for Parallel Processing, pages 76–88, April 1995.

    Google Scholar 

  24. E.W. Parsons and K.C. Sevcik. Benefits of speedup knowledge in memory-constrained multiprocessor scheduling. Performance Evaluation, 27(8):253–272, October 1996.

    Google Scholar 

  25. K.C. Sevcik. Characterizations of parallelism in applications and their use in scheduling. In Proceedings of the 1989 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, pages 171–180, May 1989.

    Google Scholar 

  26. K. C. Sevcik. Application scheduling and processor allocation in multiprogrammed parallel processing systems. Performance Evaluation, 19:107–140, 1994.

    Article  Google Scholar 

  27. C.S. Wu. Processor scheduling in multiprogrammed shared memory numa multiprocessors. M. Sc. thesis, Department of Computer Science, University of Toronto, Toronto, Ontario, Canada, October 1993.

    Google Scholar 

  28. S. Zhou, X. Zheng, J. Wang, and P. Delisle. Utopia: a load sharing facility for large, heterogenous distributed computer systems. Software: Practice And Experience, 23(12):1305–1336, December 1993.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of British Columbia, 201-2366 Main Mall, V6T 1Z4, Vancouver, B.C., Canada

    Richard Gibbons

Authors
  1. Richard Gibbons
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Dror G. Feitelson Larry Rudolph

Rights and permissions

Reprints and permissions

Copyright information

© 1997 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Gibbons, R. (1997). A historical application profiler for use by parallel schedulers. In: Feitelson, D.G., Rudolph, L. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP 1997. Lecture Notes in Computer Science, vol 1291. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-63574-2_16

Download citation

  • DOI: https://doi.org/10.1007/3-540-63574-2_16

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-63574-1

  • Online ISBN: 978-3-540-69599-8

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation