Identifying Active Manifolds in Regularization Problems

  • W. L. Hare
Part of the Springer Optimization and Its Applications book series (SOIA, volume 49)


In 2009, Tseng and Yun [Math. Programming (Ser. B) 117, 387–423 (2009)], showed that the regularization problem of minimizing f(x) +  | ​ | x | ​ | 1, where f is a \({\mathcal{C}}^{2}\) function and | ​ | x | ​ | 1 is the l 1 norm of x, can be approached by minimizing the sum of a quadratic approximation of f and the l 1 norm. We consider a generalization of this problem, in which the l 1 norm is replaced by a more general nonsmooth function that contains an underlying smooth substructure. In particular, we consider the problem
$${ \min }_{x}\{f(x) + P(x)\},$$
where f is \({\mathcal{C}}^{2}\) and P is prox-regular and partly smooth with respect to an active manifold \(\mathcal{M}\) (the l 1 norm satisfies these conditions.) We reexamine Tseng and Yun’s algorithm in terms of active set identification, showing that their method will correctly identify the active manifold in a finite number of iterations. That is, after a finite number of iterations, all future iterates x k will satisfy \({x}^{k} \in \mathcal{M}\). Furthermore, we confirm a conjecture of Tseng that, regardless of what technique is used to solve the original problem, the subproblem \({p}^{k} =\mathrm{{ argmin}}_{p}\{\langle \nabla f({x}^{k}),p\rangle + \frac{r} {2}\vert {x}^{k} - p{\vert }^{2} + P(p)\}\) will correctly identify the active manifold in a finite number of iterations.


Nonconvex optimization Active constraint identification Prox-regular Partly smooth 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hare, W.L.: A proximal method for identifying active manifolds. Comput. Optim. Appl. 43, 295–306 (2009)MathSciNetMATHCrossRefGoogle Scholar
  2. 2.
    Hare, W.L., Lewis, A.S.: Identifying active constraints via partial smoothness and prox-regularity. J. Convex Anal. 11, 251–266 (2004)MathSciNetMATHGoogle Scholar
  3. 3.
    Hare, W.L., Poliquin, R.A.: Prox-regularity and stability of the proximal mapping. J. Convex Anal 14, 589–606 (2007)MathSciNetMATHGoogle Scholar
  4. 4.
    Lewis, A.S.: Active sets, nonsmoothness, and sensitivity. SIAM J. Optim. 13, 702–725 (2003)CrossRefGoogle Scholar
  5. 5.
    Poliquin, R.A., Rockafellar, R.T.: Prox-regular functions in variational analysis. Trans. Amer. Math. Soc. 348, 1805–1838 (1996)MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Rockafellar, R.T., Wets, R.J-B: Variational Analysis. Springer, Berlin (1998)Google Scholar
  7. 7.
    Tseng, P., Yun, S.: A coordinate gradient descent method for nonsmooth separable minimization. Math. Programming (Ser. B) 117, 387–423 (2009)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Department of Mathematics and StatisticsKelownaCanada

Personalised recommendations