Estimating the Expected Cost of Function Evaluation Strategies
We propose a sampling-based method to estimate the expected cost of a given strategy that evaluates a given Boolean function. In general, computing the exact expected cost of a strategy that evaluates a Boolean function obtained by some algorithm may take exponential time. Consequently, it may not be possible to assess the quality of the solutions obtained by different algorithms in an efficient manner. We demonstrate the effectiveness of the estimation method in random instances for algorithms developed for certain functions where the expected cost can be computed in polynomial time. We show that the absolute percentage errors are very small even for samples of moderate size. We propose that in order to compare strategies obtained by different algorithms, it is practically sufficient to compare the estimates when the exact computation of the expected cost is not possible.
KeywordsFunction evaluation Sequential testing Cost estimation Monte Carlo methods
The authors gratefully acknowledge the support provided by TUBITAK 1001 programme, project number 113M478.
- Boros, E., & Ünlüyurt, T. (2000). Sequential testing of series parallel systems of small depth. In Laguna & Velarde (Eds.), Computing tools for modeling, optimization and simulation (pp. 39–74). Boston: Kluwer Academic Publishers.Google Scholar
- Crama, Y., & Hammer, P. L. (2011). Boolean functions: Theory, algorithms and applications. Cambridge University Press.Google Scholar
- Mitten, L. G. (1960). An analytic solution to the least cost testing sequence problem. Journal of Industrial Engineering 17.Google Scholar