Inferential Performance Assessment of Stochastic Optimisers and the Attainment Function
- 3.7k Downloads
The performance of stochastic optimisers can be assessed experimentally on given problems by performing multiple optimisation runs, and analysing the results. Since an optimiser may be viewed as an estimator for the (Pareto) minimum of a (vector) function, stochastic optimiser performance is discussed in the light of the criteria applicable to more usual statistical estimators. Multiobjective optimisers are shown to deviate considerably from standard point estimators, and to require special statistical methodology. The attainment function is formulated, and related results from random closed-set theory are presented, which cast the attainment function as a mean-like measure for the outcomes of multiobjective optimisers. Finally, a covariance-measure is defined, which should bring additional insight into the stochastic behaviour of multiobjective optimisers. Computational issues and directions for further work are discussed at the end of the paper.
KeywordsPareto Front Stochastic Behaviour Objective Vector Multiobjective Genetic Algorithm Empirical Cumulative Distribution Function
Unable to display preview. Download preview PDF.
- Cressie, N. A. C. (1993). Statistics for Spatial Data. Wiley Series in Probability and Mathematical Statistics. John Wiley & Sons, New York, revised edition.Google Scholar
- Fonseca, C. M. and Fleming, P. J. (1996). On the performance assessment and comparison of stochastic multiobjective optimizers. In Voigt, H.-M., Ebeling, W., Rechenberg, I., and Schwefel, H.-P., editors, Parallel Problem Solving from Nature-PPSN IV, number 1141 in Lecture Notes in Computer Science, pages 584–593. Springer Verlag, BeCrossRefGoogle Scholar
- Goutsias, J. (1998). Modeling random shapes: An introduction to random closed set theory. Technical Report JHU/ECE 90-12, Department of Electrical and Computer Engineering, Image Analysis and Comunnications Laboratory, The John Hopkins University, Baltimore, MD 21218.Google Scholar
- Hoos, H. and Stutzle, T. (1998). Evaluating Las Vegas algorithms-pitfalls and remedies. In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence, pages 238–245.Google Scholar
- Kendall, D. G. (1974). Foundations of a theory of random sets. In Harding, E. F. and Kendall, D. G., editors, Stochastic Geometry. A Tribute to the Memory of Rollo Davidson, pages 322–376. John Wiley & Sons, New York.Google Scholar
- Knowles, J. D. and Corne, D. W. (2000). Approximating the nondominated front using the Pareto Archived Evolution Strategy. IEEE Transactions on Evolutionary Computation, 8(2):149–172.Google Scholar
- Shaw, K. J., Fonseca, C. M., Nortcliffe, A. L., Thompson, M., Love, J., and Fleming, P. J. (1999). Assessing the performance of multiobjective genetic algorithms for optimization of a batch process scheduling problem. In Proceedings of the Congress on Evolutionary Computation (CEC99), volume 1, pages 37–45, Washington DC.Google Scholar
- Van Veldhuizen, D. and Lamont, G. B. (2000). On measuring multiobjective evolutionary algorithm performance. In Proceedings of the 2000 Congress on Evolutionary Computation, pages 204–211.Google Scholar
- Zitzler, E. (1999). Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications. PhD thesis, Computer Engineering and Net-works Laboratory, Swiss Federal Institute of Technology Zurich.Google Scholar
- Zitzler, E., Deb, K., and Thiele, L. (1999). Comparison of multiobjective evolutionary algorithms: Empirical results (Revised version). Technical Report 70, Computer Engineering and Networks Laboratory, Swiss Federal Institute of Technology Zurich.Google Scholar