Are User Runtime Estimates Inherently Inaccurate?
Computer system batch schedulers typically require information from the user upon job submission, including a runtime estimate. Inaccuracy of these runtime estimates, relative to the actual runtime of the job, has been well documented and is a perennial problem mentioned in the job scheduling literature. Typically users provide these estimates under circumstances where their job will be killed after the provided amount of time elapses. Also, users may be unaware of the potential benefits of providing accurate estimates, such as increased likelihood of backfilling. This study examines user behavior when the threat of job killing is removed, and when a tangible reward for accuracy is provided. We show that under these conditions, about half of users provide an improved estimate, but there is not a substantial improvement in the overall average accuracy.
KeywordsKill Time Actual Runtime Runtime Estimate Scheduler Performance Average Slowdown
- 1.Cirne, W., Berman, F.: A comprehensive model of the supercomputer workload. In: Proceedings of IEEE 4th AnnualWorkshop on Job Scheduling Strategies for Parallel Processing, Cambridge, MA (2001)Google Scholar
- 2.Mu’alem, W., Feitelson, D.G.: Utilization, Predictability, Workloads, and User Runtime Estimates in Scheduling the IBM SP2 with Backfilling. IEEE Trans. Parallel and Distributed Systems, 12(6) (June 2001)Google Scholar
- 3.Srinivasan, S., Kettimuthu, R., Subramani, V., Sadayappan, P.: Characterization of Backfilling Strategies for Parallel Job Scheduling. In: Proceedings of 2002 International Workshops on Parallel Processing (August 2002)Google Scholar
- 4.Chiang, S.-H., Arpaci-Dusseau, A., Vernon, M.K.: The Impact of More Accurate Requested Runtimes on Production Job Scheduling Performance. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) Proceedings of the 4th Workshop on Workload Characterization (July 2002)Google Scholar
- 6.Stoica, I., Abdel-Wahab, H., Pothen, A.: A Microeconomic Scheduler for Parallel Computers. In: Feitelson, D.G., Rudolph, L. (eds.) IPPS 1995 Workshop: Job Scheduling Strategies for Parallel Processing (April 1995)Google Scholar
- 8.Horizon, B.: National Partner for Advanced Computing Infastructure (NPACI)., http://www.npaci.edu/Horizon/
- 10.Kruger, J., Dunning, D.: Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology 77(6) (December 1999)Google Scholar
- 11.Lovallo, D., Kahneman, D.: Delusions of Success. Harvard Business Review 81(7) (July 2003)Google Scholar
- 12.Buehler, R.: Planning, personality, and prediction: The role of future focus in optimistic time predictions. Organizational Behavior and Human Decision Processes 92(1/2) (September-November 2003)Google Scholar
- 13.Perkovic, D., Keleher, P.: Randomization, Speculation, and Adaptation in Batch Schedulers. In: Proceedings of Supercomputing 2000 (November 2000)Google Scholar
- 14.Zotkin, D., Keleher, P.: Sloppiness as a Virtue: Job-Length Estimation and Performance in Backfilling Schedulers. In: Proceedings of the 8th HPDC, Redondo Beach, CA (August 1999)Google Scholar