Complexity analysis of interior point algorithms for nonLipschitz and nonconvex minimization
 Wei Bian,
 Xiaojun Chen,
 Yinyu Ye
 … show all 3 hide
Rent the article at a discount
Rent now* Final gross prices may vary according to local VAT.
Get AccessAbstract
We propose a first order interior point algorithm for a class of nonLipschitz and nonconvex minimization problems with box constraints, which arise from applications in variable selection and regularized optimization. The objective functions of these problems are continuously differentiable typically at interior points of the feasible set. Our first order algorithm is easy to implement and the objective function value is reduced monotonically along the iteration points. We show that the worstcase iteration complexity for finding an \(\epsilon \) scaled first order stationary point is \(O(\epsilon ^{2})\) . Furthermore, we develop a second order interior point algorithm using the Hessian matrix, and solve a quadratic program with a ball constraint at each iteration. Although the second order interior point algorithm costs more computational time than that of the first order algorithm in each iteration, its worstcase iteration complexity for finding an \(\epsilon \) scaled second order stationary point is reduced to \(O(\epsilon ^{3/2})\) . Note that an \(\epsilon \) scaled second order stationary point must also be an \(\epsilon \) scaled first order stationary point.
 Bian, W, Chen, X (2013) Worstcase complexity of smoothing quadratic regularization methods for nonLipschitzian optimization. SIAM J. Optim. 28: pp. 17181741 CrossRef
 Bruckstein, AM, Donoho, DL, Elad, M (2009) From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev. 51: pp. 3481 CrossRef
 Cartis, C, Gould, NIM, Toint, PhL (2011) On the evaluation complexity of composite function minimization with applications to nonconvex nonlinear programming. SIAM J. Optim. 21: pp. 17211739 CrossRef
 Cartis, C, Gould, NIM, Toint, PhL (2011) Adaptive cubic regularisation methods for unconstrained optimization, Part I: motivation, convergence and numerical results. Math. Program. 127: pp. 245295 CrossRef
 Cartis, C, Gould, NIM, Toint, PhL (2011) Adaptive cubic regularisation methods for unconstrained optimization, Part II: worstcase function and derivativeevaluation complexity. Math. Program. 130: pp. 295319 CrossRef
 Cartis, C, Gould, NIM, Toint, PhL (2012) An adaptive cubic regularization algorithm for nonconvex optimization with convex constraints and its functionevaluation complexity. IMA J. Numer. Anal. 32: pp. 16611695 CrossRef
 Cartis, C, Gould, NIM, Toint, PhL (2012) Complexity bounds for secondorder optimality in unconstrained optimization. J. Complex. 28: pp. 93108 CrossRef
 Chen, X (2012) Smoothing methods for nonsmooth, novonvex minimization. Math. Program. 134: pp. 7199 CrossRef
 Chen, X., Ge, D., Wang, Z., Ye, Y.: Complexity of unconstrained \(L_2\)  \(L_p\) minimization. Math. Program. 143, 371–383 (2014)
 Chen, X, Ng, M, Zhang, C (2012) Nonconvex $$l_p$$ l p regularization and box constrained model for image restoration. IEEE Trans. Image Process. 21: pp. 47094721 CrossRef
 Chen, X, Niu, L, Yuan, Y (2013) Optimality conditions and smoothing trust region Newton method for nonLipschitz optimization. SIAM J. Optim. 23: pp. 15281552 CrossRef
 Chen, X, Xu, F, Ye, Y (2010) Lower bound theory of nonzero entries in solutions of $$l_2$$ l 2  $$l_{p}$$ l p minimization. SIAM J. Sci. Comput. 32: pp. 28322852 CrossRef
 Chen, X, Zhou, W (2010) Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 3: pp. 765790 CrossRef
 Fan, J (1997) Comments on ’Wavelets in stastics: a review’ by A. Antoniadis. J. Ital. Stat. Soc. 6: pp. 131138 CrossRef
 Garmanjani, R., Vicente, L.N.: Smoothing and worst case complexity for directsearch methods in nonsmooth optimization. IMA J. Numer. Anal. 33, 1008–1028 (2013)
 Ge, D, Jiang, X, Ye, Y (2011) A note on the complexity of $$L_p$$ L p minimization. Math. Program. 21: pp. 17211739
 Gratton, S, Sartenaer, A, Toint, PhL (2008) Recursive trustregion methods for multiscale nonlinear optimization. SIAM J. Optim. 19: pp. 414444 CrossRef
 Griewank, A.: The Modification of Newton’s Method for Unconstrained Optimization by Bounding Cubic Terms. Technical report NA/12 (1981). University of Cambridge, UK Department of Applied Mathematics and Theoretical Physics (1981)
 Hastie, T, Tibshirani, R, Friedman, J (2009) The Elements of Statistical Learning Data Mining, Inference, and Prediction. Springer, New York
 Huang, J, Horowitz, JL, Ma, S (2008) Asymptotic properties of bridge estimators in sparse highdimensional regression models. Ann. Stat. 36: pp. 587613 CrossRef
 Nesterov, Y (2004) Introductory Lectures on Convex Optimization, Applied Optimization. Kluwer, Dordrecht CrossRef
 Nesterov, Y, Polyak, BT (2006) Cubic regularization of Newton’s method and its global performance. Math. Program. 108: pp. 177205 CrossRef
 Nesterov, Y (2008) Accelerating the cubic regularization of Newton’s method on convex problems. Math. Program. 112: pp. 159181 CrossRef
 Nikolova, M, Ng, MK, Zhang, S, Ching, WK (2008) Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 1: pp. 225 CrossRef
 Tibshirani, R (1996) Shrinkage and selection via the Lasso. J. R. Stat. Soc. Ser. B 58: pp. 267288
 Vavasis, S.A., Zippel, R.: Proving Polynomial Time for SphereConstrained Quadratic Programming, Technical Report 901182, Department of Computer Science. Cornell University, Ithaca, NY (1990)
 Vavasis, SA (1991) Nonlinear Optimization: Complexity Issues. Oxford Sciences, New York
 Ye, Y (1997) Interior Point Algorithms: Theory and Analysis. Wiley, New York CrossRef
 Ye, Y (1998) On the complexity of approximating a KKT point of quadratic programming. Math. Program. 80: pp. 195211 CrossRef
 Ye, Y A new complexity result on minimization of a quadratic function with a sphere constraint. In: Floudas, C, Pardalos, PM eds. (1992) Recent Advances in Global Optimization. Princeton University Press, Princeton
 Zhang, C.H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38, 894–942 (2010)
 Title
 Complexity analysis of interior point algorithms for nonLipschitz and nonconvex minimization
 Journal

Mathematical Programming
Volume 149, Issue 12 , pp 301327
 Cover Date
 20150201
 DOI
 10.1007/s1010701407535
 Print ISSN
 00255610
 Online ISSN
 14364646
 Publisher
 Springer Berlin Heidelberg
 Additional Links
 Topics
 Keywords

 Constrained nonLipschitz optimization
 Complexity analysis
 Interior point method
 First order algorithm
 Second order algorithm
 90C30
 90C26
 65K05
 49M37
 Industry Sectors
 Authors

 Wei Bian ^{(1)}
 Xiaojun Chen ^{(2)}
 Yinyu Ye ^{(3)}
 Author Affiliations

 1. Department of Mathematics, Harbin Institute of Technology, Harbin, China
 2. Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China
 3. Department of Management Science and Engineering, Stanford University, Stanford, CA , 94305, USA