Mathematical Programming

, Volume 146, Issue 1–2, pp 409–436

Distance majorization and its applications

Full Length Paper Series A

Abstract

The problem of minimizing a continuously differentiable convex function over an intersection of closed convex sets is ubiquitous in applied mathematics. It is particularly interesting when it is easy to project onto each separate set, but nontrivial to project onto their intersection. Algorithms based on Newton’s method such as the interior point method are viable for small to medium-scale problems. However, modern applications in statistics, engineering, and machine learning are posing problems with potentially tens of thousands of parameters or more. We revisit this convex programming problem and propose an algorithm that scales well with dimensionality. Our proposal is an instance of a sequential unconstrained minimization technique and revolves around three ideas: the majorization-minimization principle, the classical penalty method for constrained optimization, and quasi-Newton acceleration of fixed-point algorithms. The performance of our distance majorization algorithms is illustrated in several applications.

Keywords

Constrained optimization Majorization-minimization (MM) Sequential unconstrained minimization Projection 

Mathematics Subject Classification (2000)

65K05 90C25 90C30 62J02 

Copyright information

© Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2013

Authors and Affiliations

  1. 1.Department of Human GeneticsUniversity of CaliforniaLos AngelesUSA
  2. 2.Department of StatisticsNorth Carolina State UniversityRaleighUSA
  3. 3.Departments of Biomathematics, Human Genetics, and StatisticsUniversity of CaliforniaLos AngelesUSA

Personalised recommendations