Journal of Global Optimization

, Volume 3, Issue 2, pp 171–192 | Cite as

Improving Hit-and-Run for global optimization

  • Zelda B. Zabinsky
  • Robert L. Smith
  • J. Fred McDonald
  • H. Edwin Romeijn
  • David E. Kaufman


Improving Hit-and-Run is a random search algorithm for global optimization that at each iteration generates a candidate point for improvement that is uniformly distributed along a randomly chosen direction within the feasible region. The candidate point is accepted as the next iterate if it offers an improvement over the current iterate. We show that for positive definite quadratic programs, the expected number of function evaluations needed to arbitrarily well approximate the optimal solution is at most O(n5/2) wheren is the dimension of the problem. Improving Hit-and-Run when applied to global optimization problems can therefore be expected to converge polynomially fast as it approaches the global optimum.

Key words

Random search Monte Carlo optimization algorithm complexity global optimization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    M. Abramowitz and I. A. Stegun, eds. (1961),Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables (National Bureau of Standards, Applied Mathematics Series 55, June 1964).Google Scholar
  2. 2.
    C. J. P. Bélisle, H. E. Romeijn, and R. L. Smith (1993), Hit-and-Run Algorithms for Generating Multivariate Distributions, to appear inMathematics of Operations Research.Google Scholar
  3. 3.
    C. J. P. Bélisle, H. E. Romeijn, and R. L. Smith (1990), Hide-and-Seek: A Simulated Annealing Algorithm for Global Optimization, Technical Report 90-25, Department of Industrial and Operations Engineering, The University of Michigan, Ann Arbor, September 1990.Google Scholar
  4. 4.
    H. C. P. Berbee, C. G. E. Boender, A. H. G. Rinnooy Kan, C. L. Scheffer, R. L. Smith, and J. Teigen (1987), Hit-and-Run Algorithms for the Identification of Nonredundant Linear Inequalities,Mathematical Programming 37 184–207.Google Scholar
  5. 5.
    A. Boneh (1983), A Probabilistic Algorithm for Identifying Redundancy by a Random Feasible Point Generator (RFPG), in M. H. Karwan, V. Lotfi, J. Teigen, and S. Zionts, eds.,Redundancy in Mathematical Programming (Springer-Verlag, Berlin).Google Scholar
  6. 6.
    H. Cramér (1946),Mathematical Methods of Statistics (Princeton University Press).Google Scholar
  7. 7.
    L. C. W. Dixon and G. P. Szegö, eds. (1975),Towards Global Optimization (North-Holland, Amsterdam).Google Scholar
  8. 8.
    L. C. W. Dixon and G. P. Szegö, eds. (1978),Towards Global Optimization 2 (North-Holland, Amsterdam).Google Scholar
  9. 9.
    W. Feller (1971),An Introduction to Probability Theory and Its Applications, Volume 2, 2nd Edition (John Wiley and Sons, New York).Google Scholar
  10. 10.
    I. S. Gradshteyn and I. M. Ryzhik, eds. (1980), translated by Alan Jeffrey,Table of Integrals, Series, and Products (Academic Press, New York).Google Scholar
  11. 11.
    M. H. Karwan, V. Lotfl, J. Teigen, and S. Zionts, eds. (1983),Redundancy in Mathematical Programming (Springer-Verlag, Berlin), 108–134.Google Scholar
  12. 12.
    D. E. Knuth (1969),The Art of Computer Programming, Vol. 2 (Addison-Wesley, Reading, Massachusetts), p. 116.Google Scholar
  13. 13.
    J. P. Lawrence III and K. Steiglitz (1972), Randomized Pattern Search,IEEE Transactions On Computers C-21, 382–385.Google Scholar
  14. 14.
    K. G. Murty (1983),Linear Programming (John Wiley and Sons, New York).Google Scholar
  15. 15.
    V. A. Mutseniyeks and L. Rastrigin (1964), Extremal Control of Continuous Multi-Parameter Systems by the Method of Random Search,Engineering Cybernetics 1, 82–90.Google Scholar
  16. 16.
    N. R. Patel, R. L. Smith, and Z. B. Zabinsky (1988), Pure Adaptive Search In Monte Carlo Optimization,Mathematical Programming 43, 317–328.Google Scholar
  17. 17.
    L. A. Rastrigin (1960), Extremal Control by the Method of Random Scanning,Automation and Remote Control 21, 891–896.Google Scholar
  18. 18.
    L. A. Rastrigin (1963), The Convergence of the Random Method in the Extremal Control of a Many-Parameter System,Automation and Remote Control 24, 1337–1342.Google Scholar
  19. 19.
    S. M. Ross (1983),Stochastic Processes (John Wiley and Sons, New York).Google Scholar
  20. 20.
    L. E. Scales (1985)Introduction to Non-Linear Optimization (Macmillan).Google Scholar
  21. 21.
    G. Schrack and N. Borowski (1972), An Experimental Comparison of Three Random Searches, in F. Lootsma, ed.,Numerical Methods for Nonlinear Optimization (Academic Press, London), pp. 137–147.Google Scholar
  22. 22.
    G. Schrack and M. Choit (1976), Optimized Relative Step Size Random Searches,Mathematical Programming 10, 270–276.Google Scholar
  23. 23.
    M. A. Schumer and K. Steiglitz (1968), Adaptive Step Size Random Search,IEEE Transactions On Automatic Control AC-13, 270–276.Google Scholar
  24. 24.
    R. L. Smith (1984), Efficient Monte Carlo Procedures for Generating Points Uniformly Distributed over Bounded Regions,Operations Research 32, 1296–1308.Google Scholar
  25. 25.
    F. J. Solis and R. J.-B. Wets (1981), Minimization by Random Search Techniques,Mathematics of Operations Research 6, 19–30.Google Scholar
  26. 26.
    A. Sommerfeld (1949),Partial Differential Equations in Physics (Academic Press, New York).Google Scholar
  27. 27.
    Z. B. Zabinsky (1985),Computational Complexity of Adaptive Algorithms in Monte Carlo Optimization (Ph.D. Dissertation from The University of Michigan, Ann Arbor MI).Google Scholar
  28. 28.
    Z. B. Zabinsky and R. L. Smith (1992), Pure Adaptive Search in Global Optimization,Mathematical Programming 53, 323–338.Google Scholar
  29. 29.
    Z. B. Zabinsky, D. L. Graesser, M. E. Tuttle, and G. I. Kim (1992), Global Optimization of Composite Laminates Using Improving Hit-and-Run, in C. A. Floudas and P. M. Pardalos, eds.,Recent Advances in Global Optimization, Princeton University Press.Google Scholar

Copyright information

© Kluwer Academic Publishers 1993

Authors and Affiliations

  • Zelda B. Zabinsky
    • 1
  • Robert L. Smith
    • 2
  • J. Fred McDonald
    • 3
  • H. Edwin Romeijn
    • 4
  • David E. Kaufman
    • 2
  1. 1.Industrial Engineering Program, FU-20University of WashingtonSeattleUSA
  2. 2.Department of Industrial and Operations EngineeringThe University of MichiganAnn ArborUSA
  3. 3.Department of Mathematics and StatisticsUniversity of WindsorWindsorCanada
  4. 4.Department of Operations Research & Tinbergen InstituteErasmus University RotterdamDR RotterdamThe Netherlands

Personalised recommendations