BIT Numerical Mathematics

, Volume 17, Issue 2, pp 160–169 | Cite as

Optimization of functions whose values are subject to small errors

  • Torkel Glad
  • Allen Goldstein


In this paper we consider the minimization of a function whose values can only be obtained with an error. For the case when the error has certain statistical properties this problem has been investigated by Kiefer and Wolfowitz (1) and Kushner (2, 3). Kushner has shown that a certain class of algorithms converge to a stationary point with probability one. Here a different approach is used. The error is assumed to have an upper bound and it is shown that a stationary point can be obtained to within a certain accuracy, dependent on the magnitude of the error. Our results are related to works concerning roundoff errors for one dimensional optimization (4) and solution of nonlinear equations (5). The algorithm we use can be regarded as an extension of the methods used in (6), (8) and (9).


Computational Mathematic Stationary Point Nonlinear Equation Small Error Roundoff Error 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    J. Kiefer and J. Wolfowitz,Stochastic Estimation of the Maximum of a Regression Function. Ann. Math. Stat., 23, pp. 462–466, 1952.Google Scholar
  2. 2.
    H. J. Kushner,Stochastic Approximation Algorithms for the Local Optimization of Functions with Non-unique Stationary Points. I.E.E.E. Trans. Autom. Control, AC-17, pp. 646–654, 1972.Google Scholar
  3. 3.
    H. J. Kushner and T. Gavin,Extensions of Kerten's Adaptive Stochastic Approximation Method, Ann. Math. Stat., 1, pp. 851–861, 1973.Google Scholar
  4. 4.
    R. P. Brent,Algorithms for Minimization without Derivatives. Prentice Hall, 1973.Google Scholar
  5. 5.
    P. Lancaster,Error Analysis for the Newton-Raphson Method. Num. Math., 9, pp. 55–68, 1966.Google Scholar
  6. 6.
    A. Goldstein and J. Price,An effective algorithm for minimization, Num. Math., 10, pp. 184–189, 1967.Google Scholar
  7. 7.
    W. Murray,Second Derivative Methods. In E. Murray, ed., Numerical Methods for Unconstrained Optimization, Academic Press, 1972.Google Scholar
  8. 8.
    R. Mifflin,A Superlinearly Convergent Algorithm for Minimization without Evaluating Derivatives, Math. Progr. 9, pp. 100–117, 1975.Google Scholar
  9. 9.
    D. Winfield,Function Minimization by Interpolation in a Data Table, J. Inst. Math. Appl., 12, pp. 339–347, 1973.Google Scholar
  10. 10.
    L. V. Kantorovich and G. P. Akilov,Functional Analysis in Normed Spaces, McMillan, New York, 1964.Google Scholar

Copyright information

© BIT Foundations 1977

Authors and Affiliations

  • Torkel Glad
    • 1
    • 2
  • Allen Goldstein
    • 1
    • 2
  1. 1.Department of Automatic ControlLund Institute of TechnologyLund 7Sweden
  2. 2.Department of MathematicsUniversity of WashingtonSeattleUSA

Personalised recommendations