Abstract
Based on the NEWUOA algorithm, a new derivative-free algorithm is developed, named LCOBYQA. The main aim of the algorithm is to find a minimizer \(x^{*} \in\mathbb{R}^{n}\) of a non-linear function, whose derivatives are unavailable, subject to linear inequality constraints. The algorithm is based on the model of the given function constructed from a set of interpolation points. LCOBYQA is iterative, at each iteration it constructs a quadratic approximation (model) of the objective function that satisfies interpolation conditions, and leaves some freedom in the model. The remaining freedom is resolved by minimizing the Frobenius norm of the change to the second derivative matrix of the model. The model is then minimized by a trust-region subproblem using the conjugate gradient method for a new iterate. At times the new iterate is found from a model iteration, designed to improve the geometry of the interpolation points. Numerical results are presented which show that LCOBYQA works well and is very competing against available model-based derivative-free algorithms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Buckley, A.G., Jenning, L.S.: Test functions for unconstrained minimization. Technical report, CS-3, Dalhousie University, Canada (1989)
Conn, A., Gould, N., Toint, Ph.: Trust Region Methods. SIAM, Philadelphia (2000)
Conn, A., Sheinberg, K., Toint, Ph.: An algorithm using quadratic interpolation for unconstrained derivative-free optimization. In: Pillo, G.Di., Giannessi, F. (eds.) Nonlinear Optimization and Application, pp. 27–47. Plenum, New York (1996)
Conn, A., Sheinberg, K., Toint, Ph.: A derivative free optimization (DFO) algorithm. Report 98(11) (1998)
Dong, S.: Methods for Constrained Optimization. Springer, Berlin (2002). 18: project
Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley, New York (1987)
Frank, V., Bersini, H.B.: CONDOR, a new parallel constrained extension of Powell’s UOBYQA algorithm: experimental results and comparison with DFO algorithm. J. Comput. Appl. Math. 181, 157–175 (2005)
Gay, D.M., David, M.: In: Lecture Note in Mathematics, vol. 1066, pp. 74–105. Springer, Berlin (1984)
Gill, P.E., Murray, W., Wright, M.H.: In: Inertia-Controlling Methods for General Quadratic Programming, vol. 33, pp. 1–36. SIAM, Philadelphia (1991)
Gumma, E.: A derivative-free algorithm for linearly constrained optimization problems. Ph.D., University of Khartoum, Sudan (2011)
Hock, W., Schittkowski, K.: In: Test Example for Nonlinear Programming Codes. Lecture Notes en Economics and Mathematical Systems, vol. 187 (1981)
Igor, G., Nash, G., Sofer, A.: In: Linear and Nonlinear Programming. SIAM, Philadelphia (2009)
Ju, Z., Wang, C., Jiguo, J.: Combining trust region and line search algorithm for equality constrained optimization. Appl. Math. Comput. 14(1–2), 123–136 (2004)
Powell, M.J.D.: A direct search optimization method that model the objective function and constrained functions by linear interpolation. In: Gomez, S., Hennart, J.P. (eds.) Advances in Optimization and Numerical Analysis, Proceeding of the Sixth Workshop on Optimization and Numerical Analysis, Oaxaca, Mexico, vol. 275, pp. 15–67 (1994)
Powell, M.J.D.: UOBYQA: unconstrained optimization by quadratic approximation. Math. Program. 92, 555–582 (2002)
Powell, M.J.D.: Least Frobenius norm updating for quadratic models that satisfy interpolation conditions. Math. Program. 100, 183–215 (2004)
Powell, M.J.D.: The NEWUOA software for unconstrained optimization without derivative. In: Di pillo, G., Roma, M. (eds.) IMA in Large Scale Nonlinear Optimization. Springer, New York (2006)
Powell, M.J.D.: Development of NEWUOA for minimization without derivatives. IMA J. Numer. Anal. 28, 649–664 (2008)
Powell, M.J.D.: The BOBYQA algorithm for bound constrained optimization without derivative. Technical report, 2009/NA06, CMS University of Cambridge (2009)
Toint, Ph.: Some numerical results using a sparse matrix updating formula in unconstrained optimization. Math. Comput. 32, 839–859 (1987)
Winfield, D.: Function and Functional Optimization by Interpolation in Data Table. Ph.D., Harvard University, Cambridge, USA (1969)
Winfield, D.: Function minimization by interpolation in data table. J. Inst. Math. Appl. 12, 339–347 (1973)
Acknowledgements
Parts of this work was done during the first author’s two visits to African Institute for Mathematical Sciences (AIMS), South Africa. He is very grateful to all AIMS staff. Special thanks to Professors Fritz Hanhe and Barry Green the previous and current directors of AIMS, respectively, for their excellent hospitality and facilities that supported the research. Also, he would like to thank Professor M.J.D. Powell, an emeritus Professor at the Centre for Mathematical Sciences, Department of Applied Mathematics and Theoretical Physics, University of Cambridge, for his encouragement and help during his Ph.D. work.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gumma, E.A.E., Hashim, M.H.A. & Ali, M.M. A derivative-free algorithm for linearly constrained optimization problems. Comput Optim Appl 57, 599–621 (2014). https://doi.org/10.1007/s10589-013-9607-y
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-013-9607-y