Skip to main content
SpringerLink
Log in

Probabilistic analysis of optimization algorithms-some aspects from a practical point of view

  • Published:
Acta Applicandae Mathematica Aims and scope Submit manuscript

Abstract

In this paper the utility and the difficulties of probabilistic analysis for optimization algorithms are discussed. Such an analysis is expected to deliver valuable criteria-better than the worst-case complexity-for the efficiency of an algorithm in practice.

The author has done much work of that kind in the field of linear programming. Based on that experience he gives some insight into the general principles for such an approach. He reports on some typical and representative attempts to analyze algorithms, resp. problems, of linear and combinatorial optimization. For each case he describes the problem, the stochastic model under consideration, the algorithm, the results, and tries to give a brief idea of the way these results could be obtained. He concludes with a discussion of some drawbacks and difficulties in that field of research. Among these are the strong sensibility with respect to the chosen model, the restriction of results to the asymptotic case, the restriction to somehow inefficient algorithms, etc. These points are the reasons why probabilistic analysis is of limited value for practice today. On the other hand, they show which principal problems should be attacked in the future to obtain the desired utility.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Adler I., Karp R., and Shamir R.: A family of simplex variants solving an m x d linear program in expected number of pivot steps depending on d only, University of California, Computer Science Division, Berkeley, December 1983.

    Google Scholar 

  2. Adler I., Karp R., and Shamir R.: A simplex variant solving an m x d linear program in O(min2, d 2)) expected number of pivot steps, University of California, Computer Science Division, Berkeley, 1983.

    Google Scholar 

  3. Adler I. and Megiddo N.: A simplex algorithm where the average number of steps is bounded between two quadratic functions of the smaller dimension, Dept. of Industrial Engineering and Operations Research, University of California, Berkeley, California, December 1983.

    Google Scholar 

  4. Adler, I.: The expected number of Pivots needed to solve parametric linear programs and the efficiency of the self-dual-simplex method, University of California, Dept. of IE and OR, 1983.

  5. Aho A. V., Hopcroft J. E., and Ullmann J. D.: The Design and Analysis of Computer Algorithms, Addison-Wesley, Reading, Mass, 1974.

    Google Scholar 

  6. Artstein Z. and Hart S.: Law of large numbers for random sets and allocation processes, Math. Operations Res. 6 (1981), 485–492.

    Google Scholar 

  7. Bachem A. and Euler R.: Recent trends in combinatorial optimization, OR-Spektrum 6 (1984), 1–22.

    Google Scholar 

  8. Benveniste R.: Evaluating computational efficiency: a stochastic approach, Mathematical Programming 22 (1982), 261–287.

    Google Scholar 

  9. Berenguer S. E. and Smith R. L.: The expected number of extreme points of a random linear program, Mathematical Programming Study 35 (1985), 1–6.

    Google Scholar 

  10. Blair C.: Random linear programs with many variables and few constraints, College of Commerce and Business Administration, University of Illinois at Urbana, Champaign, Aprill 1983.

    Google Scholar 

  11. Borgwardt, K. H.: Untersuchungen zur Asymptotik der mittleren Schrittzahl von Simplexverfahren in der linearen Optimierung, Dissertation, Universität Kaiserslautern, 1977.

  12. Borgwardt K. H.: Some distribution-independent results about the asymptotic order of the average number of pivot steps of the simplex method, Math. Operations Res. 7 (1982), 441–462.

    Google Scholar 

  13. Borgwardt K. H.: The average number of pivot steps required by the simplex method is polynomial, Zeit. Operations Res. 26 (1982), 157–177.

    Google Scholar 

  14. Borgwardt, K. H.: The Simplex Method — A Probabilistic Analysis, Springer Verlag (1986).

  15. Brown, G. G. and Schubert, B. O.: On random binary trees, Math. Operations Res. No. 1 (1984), 43–65.

  16. Burkard R. E. and Fincke U.: On random quadratic bottleneck assignment problems, Mathematical Programming 23 (1982), 227–232.

    Google Scholar 

  17. Burkard R. E. and Fincke U.: The asymptotic probabilistic behaviour of quadratic sum assignment problems, Zeit. Operations Res. 27 (1983), 73–81.

    Google Scholar 

  18. Burkard R. and Fincke U.: Probabilistic asymptotic properties of some combinatorial optimization problems, Discrete Appl. Math. 12 (1985), 21–29.

    Google Scholar 

  19. Carson J. S. and Law A. M.: A note on SPIRA'S algorithm for the all-pairs shortest-path problem, SIAM J. Computation 6 (1977), 696–699.

    Google Scholar 

  20. D'Atri G. and Puech C.: Probabilistic analysis of the subset-sum problem, Discrete Appl. Math. 4 (1982), 329–334.

    Google Scholar 

  21. Dempster M. A. H., Fisher M. L., Jansen L., Lageweg B. J., Lenstra J. K., and Rinnooy Kan A. H. G.: Analysis of heuristics for stochastic programming: Results for Hierarchical Scheduling Problems 8 (1983), 525–537.

    Google Scholar 

  22. Dijkstra E. W.: A note on two problems in connexion with graphs, Numerische Mathematik 1 (1959), 269–271.

    Google Scholar 

  23. Erdös P. and Renyi A.: On random graphs I, Publ. Math. Debrecan 6 (1959), 290–297.

    Google Scholar 

  24. Erdös P.: Graph theory and probability I, Canadian J. Math. 11 (1959), 34–38.

    Google Scholar 

  25. Erdös P.: Graph theory and probability II, Canadian J. Math. 13 (1961), 346–352.

    Google Scholar 

  26. Fisher M. L. and Hochbaum D. S.: Probabilistic analysis of the planar K-MEDIAN problem, Math. Operations Res. 5 (1980), 27–34.

    Google Scholar 

  27. Fisher M. L. and Krieger Abba M.: Analysis of a linearization heuristic for single-machine scheduling to maximize profit, Mathematical Programming 28 (1984), 218–225.

    Google Scholar 

  28. Fox B. L., Lenstra J. K., Rinnooy Kan A. H. G. and Schrage L. E.: Branching from the largest upper bound: folklore and facts, Report 7722, Erasmus University, Rotterdam, 1977.

    Google Scholar 

  29. Frenk J. B. G., Van Houweninge M. and Rinnooy Kan A. H. G.: Asymptotic properties of the quadratic assignment problem, Math. Operation Res. 10 (1982), 100–116.

    Google Scholar 

  30. Frieze A. M.: On the exact solution of random travelling salesman problems with medium size integer coefficients, Dept. of Comp. Science & Statistics, Queen Mary College, London, England, 1985.

    Google Scholar 

  31. Gale D.: How to solve linear inequalities, Am. Math. Monthly 76 (1969), 589–599.

    Google Scholar 

  32. Garey M. R. and Johnson D. S.: Computers and Intractability: A Guide to the Theory of NP-Completeness, Freeman, San Francisco, 1979.

    Google Scholar 

  33. Gazmuri P. G.: Probabilistic analysis of a machine scheduling problem Math. Operations Res. 10 (1985), 328–339.

    Google Scholar 

  34. Grötschel M., Lovász L., and Schrijver A.: The ellipsoid method and its consequences in combinatorial optimization, Combinatorica 1 (1981), 169–197.

    Google Scholar 

  35. Grötschel, M.: Developments in combinatorial optimization, Universität Augsburg, Preprint No. 26, 1984.

  36. Grötschel, M.: Operations research I, Skripten zur Vorlesung SS 1985, Universität Augsburg, 1985.

  37. Haimovich M.: The simplex algorithm is very good! — On the expected number of pivot steps and related properties of random linear programs, Columbia University, New York, 1983.

    Google Scholar 

  38. Halton J. H. and Terada R.: A fast algorithm for the Euclidean travelling salesman problem optimal with probability one, SIAM J. Computation 11 (1982), 28–46.

    Google Scholar 

  39. Hochbaum D. S. and Shmoys D. B.: A best possible heuristic for the k-center problem, Math. Operations Res. 10 (1985), 180–184.

    Google Scholar 

  40. Karp R. M. and Steele J. M.: Probabilistic analysis of heuristics, in Lawler E. G., Lenstra J. K., Rinnooy Kan A. H. G., and Schmoys D. (eds.), The Travelling Salesman Problem, John Wiley, New York, 1985, pp. 181–205.

    Google Scholar 

  41. Karp R. M.: The probabilistic analysis of some combinatorial search algorithms, in J. F. Traub (ed.), Algorithms and Complexity, Academic Press, New York, 1976, pp. 1–19.

    Google Scholar 

  42. Kemp R.: Fundamentals of the Average Case Analysis of Particular Algorithms, Teubner, John Wiley, New York, 1984.

    Google Scholar 

  43. Klee V.: Combinatorial optimization: what is the state of the art? Math. Operations Res. 5 (1980), 1–26.

    Google Scholar 

  44. Klee, V. and Minty, G. J.: How good is the simplex algorithm? In O. Shisha (ed.) Inequalities III, Academic Press, New York, pp. 159–175.

  45. Korte B. and Hausmann D.: An analysis of the Greedy Heuristic for independence systems, Ann. Discrete Math. 2 (1978), 65–74.

    Google Scholar 

  46. Lawler E. L.: Fast approximation algorithms for knapsack problems, Math. Operations Res. 4 (1979), 339–356.

    Google Scholar 

  47. Lenstra J. K., Rinnooy Kan A. H. G., and Van Emde Boas P.: An appraisal of computational complexity for operations researchers, Erasmus University, Rotterdam, Econometric Institute, Report 8211/0, 1982.

    Google Scholar 

  48. Lenstra J. K. and Rinnooy Kan A. H. G. U.: Computational complexity of discrete optimization problems, Erasmus University, Rotterdam, Report 7727/0, 1977.

    Google Scholar 

  49. Lieberherr K.: Probabilistic combinatiorial optimization, Lecture Notes in Computer Sci. 118 (1981), 423–432.

    Google Scholar 

  50. Lindberg P. O. and Olafsson S.: On the lengths of simplex paths: the assignment case, Royal Institute of Technology, Stockholm, TRITA-MAI 1980–29, 1980.

    Google Scholar 

  51. Lindberg P. O. and Olafsson S.: On the lengths of simplex paths: the transportation case, Royal Institute of Technology, Stockholm, TRITA-MAI 1983–6, 1983.

    Google Scholar 

  52. Lifschitz V.: The efficiency of an algorithm of integer programming: a probabilistic analysis, Proc. AMS 79 (1980), 72–76.

    Google Scholar 

  53. Loulou R.: Tight bounds and probabilistic analysis of two heuristics for parallel processor scheduling, Math. Operations Res. 9 (1984), 142–150.

    Google Scholar 

  54. May J. H. and Smith R. L.: Random polytopes: their definition, generation and aggregate properties, Mathematical Programming 24 (1982), 39–54.

    Google Scholar 

  55. Meanti M., Rinnooy Kan A. H. G., Stougie L., and Vercellis C.: A probabilistic analysis of the multiknapsack value function, Report 8435, Erasmus University, Rotterdam, Econometric Institute, 1984.

    Google Scholar 

  56. Nicholson T. A. J.: Finding the shortest route between two points in a network, Computing J. 9 (1966), 275–280.

    Google Scholar 

  57. Palmer E. M.: Graphical Evolution: An Introduction to the Theory of Random Graphs, John Wiley New York, 1985.

    Google Scholar 

  58. Posa L.: Hamiltonian circuits in random graphs, Discrete Math. 14 (1976), 359–364.

    Google Scholar 

  59. Shamir R.: The efficiency of the simplex-method: a survey, Dept. of Industrial Engineering and Operations Research, University of California, Berkeley, 1984.

    Google Scholar 

  60. Smale, S.: The problem of the average speed of the simplex method, Proc. 11th Internat. Symp. Math. Programming, Universität Bonn, August 1982, pp. 530–539.

  61. Smale S: On the average speed of the simplex method, Mathematical Programming 27 (1983), 241–262.

    Google Scholar 

  62. Spira P. M.: A new algorithm for finding all shortest paths in a graph of positive arcs in average time O(n 2 log2 n), SIAM J. Computation 2 (1973), 28–32.

    Google Scholar 

  63. Stein D. M.: An asymptotic, probabilistic analysis of a routing problem, Math. Operations Res. 3 (1978), 89–101.

    Google Scholar 

  64. Tinhofer, G.: Probabilistische Ansätze in der Optimierung, Berichte der Mathematisch-Statistischen Sektion in der Forschungsgesellschaft Joanneum, Bericht No. 230 (1984).

  65. Todd M. J.: Polynomial expected behaviour of a pivoting algorithm for linear complimentarity and linear programming problems, Technical Report No. 595, School of Operations Research and Industrial Engineering. Cornell University, New York, 1983.

    Google Scholar 

  66. Walkup D. W.: On the expected value of a random assignment problem, SIAM J. Computation 8 (1979), 440–442.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut für Mathematik der Universität Augsburg, Memmingerstrasse 6, D-8900, Augsberg, West Germany

    K. H. Borgwardt

Authors
  1. K. H. Borgwardt
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Borgwardt, K.H. Probabilistic analysis of optimization algorithms-some aspects from a practical point of view. Acta Appl Math 10, 171–210 (1987). https://doi.org/10.1007/BF00046618

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00046618

AMS subject classifications

Key words

Search

Navigation