Are User Runtime Estimates Inherently Inaccurate?

  • Cynthia Bailey Lee
  • Yael Schwartzman
  • Jennifer Hardy
  • Allan Snavely
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3277)


Computer system batch schedulers typically require information from the user upon job submission, including a runtime estimate. Inaccuracy of these runtime estimates, relative to the actual runtime of the job, has been well documented and is a perennial problem mentioned in the job scheduling literature. Typically users provide these estimates under circumstances where their job will be killed after the provided amount of time elapses. Also, users may be unaware of the potential benefits of providing accurate estimates, such as increased likelihood of backfilling. This study examines user behavior when the threat of job killing is removed, and when a tangible reward for accuracy is provided. We show that under these conditions, about half of users provide an improved estimate, but there is not a substantial improvement in the overall average accuracy.


Kill Time Actual Runtime Runtime Estimate Scheduler Performance Average Slowdown 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Cirne, W., Berman, F.: A comprehensive model of the supercomputer workload. In: Proceedings of IEEE 4th AnnualWorkshop on Job Scheduling Strategies for Parallel Processing, Cambridge, MA (2001)Google Scholar
  2. 2.
    Mu’alem, W., Feitelson, D.G.: Utilization, Predictability, Workloads, and User Runtime Estimates in Scheduling the IBM SP2 with Backfilling. IEEE Trans. Parallel and Distributed Systems, 12(6) (June 2001)Google Scholar
  3. 3.
    Srinivasan, S., Kettimuthu, R., Subramani, V., Sadayappan, P.: Characterization of Backfilling Strategies for Parallel Job Scheduling. In: Proceedings of 2002 International Workshops on Parallel Processing (August 2002)Google Scholar
  4. 4.
    Chiang, S.-H., Arpaci-Dusseau, A., Vernon, M.K.: The Impact of More Accurate Requested Runtimes on Production Job Scheduling Performance. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) Proceedings of the 4th Workshop on Workload Characterization (July 2002)Google Scholar
  5. 5.
    Lawson, B.G., Smirni, E.: Multiple-Queue Backfilling Scheduling with Priorities and Reservations for Parallel Systems. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2002. LNCS, vol. 2537, pp. 72–87. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  6. 6.
    Stoica, I., Abdel-Wahab, H., Pothen, A.: A Microeconomic Scheduler for Parallel Computers. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS 1995 Workshop: Job Scheduling Strategies for Parallel Processing (April 1995)Google Scholar
  7. 7.
    Ward Jr., W.A., Mahood, C.L., West, J.E.: Scheduling Jobs on Parallel Systems Using a Relaxed Backfill Strategy. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2002. LNCS, vol. 2537, pp. 88–102. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  8. 8.
    Horizon, B.: National Partner for Advanced Computing Infastructure (NPACI).,
  9. 9.
    Jones, J.P., Nitzberg, B.: Scheduling for Parallel Supercomputing: A Historical Perspective of Achievable Utilization. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1999, IPPS-WS 1999, and SPDP-WS 1999. LNCS, vol. 1659, p. 1. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  10. 10.
    Kruger, J., Dunning, D.: Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology 77(6) (December 1999)Google Scholar
  11. 11.
    Lovallo, D., Kahneman, D.: Delusions of Success. Harvard Business Review 81(7) (July 2003)Google Scholar
  12. 12.
    Buehler, R.: Planning, personality, and prediction: The role of future focus in optimistic time predictions. Organizational Behavior and Human Decision Processes 92(1/2) (September-November 2003)Google Scholar
  13. 13.
    Perkovic, D., Keleher, P.: Randomization, Speculation, and Adaptation in Batch Schedulers. In: Proceedings of Supercomputing 2000 (November 2000)Google Scholar
  14. 14.
    Zotkin, D., Keleher, P.: Sloppiness as a Virtue: Job-Length Estimation and Performance in Backfilling Schedulers. In: Proceedings of the 8th HPDC, Redondo Beach, CA (August 1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Cynthia Bailey Lee
    • 1
    • 2
  • Yael Schwartzman
    • 1
  • Jennifer Hardy
    • 1
  • Allan Snavely
    • 1
    • 2
  1. 1.San Diego Supercomputer CenterUniversity of California, San DiegoLa JollaUSA
  2. 2.Department of Computer Science and EngineeringUniversity of California, San DiegoLa JollaUSA

Personalised recommendations