Abstract
In this paper, we propose a model-hybrid approach for nonlinear optimization that employs both trust region method and quasi-Newton method, which can avoid possibly resolve the trust region subproblem if the trial step is not acceptable. In particular, unlike the traditional trust region methods, the new approach does not use a single approximate model from beginning to the end, but instead employs quadratic model or conic model at every iteration adaptively. We show that the new algorithm preserves the strong convergence properties of trust region methods. Numerical results are also presented.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Deng, N.Y., Xiao, Y., Zhou, F.J.: Nonmonotonic trust region algorithm. J. Optim. Theory. Appl. 76(2), 259–285 (1993)
Mo, J.T., Liu, C.Y., Yan, S.C.: A nonmonotone trust region method based on nonincreasing technique of weighted average of the successive function values. J. Comput. Appl. Math. 209, 97–108 (2007)
Gu, N.Z., Mo, J.T.: Incorporating nonmonotone strategies into the trust region method for unconstrained optimization. Appl. Math. Comput. 55, 2158–2172 (2008)
Liu, J.H., Ma, C.F.: A nonmonotone trust region method with new inexact line search for unconstrained optimization. Numer. Algorithm. (2012). doi:10.1007/s11075-012-9652-0
Zhao, X., Wang, X.Y.: A nonmonotone self-adaptive trust region algorithm with line search based on the new conic model. J. Taiyuan Univ. Sci. Tech. 31(1), 68–71 (2010)
Nocedal, J., Yuan, Y.X.: Combining Trust-Region and Line-Search Techniques, Optimization Technology Center mar OTC 98/04 (1998)
Gertz, E.M.: A quasi-Newton trust-region method. Math. Program 100, 447–470 (2004)
Qu, S.J., Zhang, K.C., Wang, F.S.: A global optimization using linear relaxation for generalized geometric programming. Eur. J. Oper. Res. 190, 345–356 (2008)
Wang, F.S., Zhang, K.C.: A hybrid algorithm for nonlinear minimax problems. Ann. Oper. Res. 164, 167–191 (2008)
Davidon, W.C.: Conic approximations and colinear scaling for optimizers. SIAM J. Numer. Anal. 17(2), 268–281 (1980)
Sorensen, D.C.: The q-superlinear convergence of a colinear scaling algorithm for unconstrained optimization. SIAM J. Numer. Anal. 17(1), 84–114 (1980)
Ariyawansa, K.A.: Deriving collinear scaling algorithms as extension of quasi-Newton methods and the local convergence of DFP and BFGS related collinear scaling algorithms. Math. Program 49, 23–48 (1990)
Sheng, S.: Interpolation by conic model for unconstrained optimization. Computing 54, 83–98 (1995)
Sun, W.Y.: Nonquadratic model optimization methods. Asia Pac. J. Oper. Res. 13, 43–63 (1996)
Gourgeon, H., Nocedal, J.: A conic algorithm for optimization. SIAM J. Sci. Comput. 6, 253–267 (1985)
Di, S., Sun, W.Y.: A trust region method for conic model to solve unconstrained optimization. Optim. Methods Softw. 6, 237–263 (1996)
Xu, C.X., Yang, X.Y.: Convergence of conic quasi-Newton trust region menthods for unconstrained minimization. Mathematica Applicata 11, 71–76 (1998)
Fu, J.H., Sun, W.Y., Sampaio, R.J.B.: An adaptive approach of conic trust-region method for unconstrained optimization problems. J. Appl. Math. Comput. 19, 165–177 (2005)
Wang, J.Y., Ni, Q.: An algorithm for solving new trust region subproblem with conic model. Sci. China Ser. A 51, 461–473 (2008)
Grandinetti, L.: Some investigations in a new algorithm for nonlinear optimization based on conic models of the objective function. J. Optim. Theory Appl. 43, 1–21 (1984)
Powell, M.J.D.: A new algorithm for unconstrained optimization. In: Rosen, J.B., Mangasarian, O.L., Ritter, K. (eds.) Nonlinear Programming, pp. 31–66. Academic, New York (1970)
Zhou, Q.Y., Zhang, C.: A new nonmonotone adaptive trust region method based on simple quadratic models. J. Appl. Math. Comput. 40, 111–123 (2012)
Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust-Region Methods. SIAM Publications, Philadelphia (2000)
Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, New York (2006)
Powell, M.J.D.: Convergence properties of a class of minimization algorithms. In: Mangasarian, O.L., Meyer, R.R., Robinson, S.M. (eds.) Nonlinear Programming, vol. 2, p. 1C27. Academic, New York (1975)
Dennis, J.E., Mei, H.H.W.: Two new unconstrained optimization algorithms which use function and gradient values. J. Optim. Theory Appl. 28, 453–482 (1979)
Shultz, G.A., Schnabel, R.B., Byrd, R.H.: A family of trust-region based algorithm for unconstrained minimization with strong global convergence. SIAM J. Numer. Anal. 22, 47–67 (1985)
Byrd, R.H., Schnabel, R.B., Shultz, G.A.: Approximation solution of the trust region problem by minimization over two-dimensional subspace. Math. Program 40, 247–263 (1988)
Steihaug, T.: The conjugate gradient method and trust-regions in large scale optimization. SIAM J. Numer. Anal. 20, 626–627 (1983)
Toint, P.L.: Towards an efficient sparsity exploiting Newton method for minimization. In: Duff, I.S. (ed.) Sparse Matrices and Their Uses, pp. 57–88. Academic, London (1981)
More, J.J., Sorensen, D.C.: Computing a trust-region step. SIAM J. Sci. Stat. Comput. 4, 553–572 (1983)
More, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7, 17–41 (1981)
Dolan, E.D., More, J.J.: Benchmarking optimization software with performance profiles. Math. Program 91, 201–213 (2002)
Author information
Authors and Affiliations
Corresponding author
Additional information
This work is supported by the National Natural Science Foundation of China (11171250); The Natural Science Foundation of Shanxi Province of China (2011011002-2)
Rights and permissions
About this article
Cite this article
Wang, FS., Jian, JB. & Wang, CL. A model-hybrid approach for unconstrained optimization problems. Numer Algor 66, 741–759 (2014). https://doi.org/10.1007/s11075-013-9757-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-013-9757-0