Mathematical Programming

, Volume 143, Issue 1–2, pp 339–356 | Cite as

MM algorithms for geometric and signomial programming

Full Length Paper Series A


This paper derives new algorithms for signomial programming, a generalization of geometric programming. The algorithms are based on a generic principle for optimization called the MM algorithm. In this setting, one can apply the geometric-arithmetic mean inequality and a supporting hyperplane inequality to create a surrogate function with parameters separated. Thus, unconstrained signomial programming reduces to a sequence of one-dimensional minimization problems. Simple examples demonstrate that the MM algorithm derived can converge to a boundary point or to one point of a continuum of minimum points. Conditions under which the minimum point is unique or occurs in the interior of parameter space are proved for geometric programming. Convergence to an interior point occurs at a linear rate. Finally, the MM framework easily accommodates equality and inequality constraints of signomial type. For the most important special case, constrained quadratic programming, the MM algorithm involves very simple updates.


Arithmetic-geometric mean inequality Geometric programming Global convergence MM algorithm Linearly constrained quadratic programming Parameter separation Penalty method Signomial programming 

Mathematics Subject Classification (2000)

90C25 26D07 


  1. 1.
    Bertsekas, D.P.: Nonlinear Programming. Athena Scientific (1999)Google Scholar
  2. 2.
    Borwein, J.M., Lewis, A.S.: Convex Analysis and Nonlinear Optimization: Theory and Examples. Springer, New York (2000)CrossRefGoogle Scholar
  3. 3.
    Boyd, S., Kim, S.J., Vandenberghe, L., Hassibi, A.: A tutorial on geometric programming. Optim. Eng. 8, 67–127 (2007)CrossRefMATHMathSciNetGoogle Scholar
  4. 4.
    Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)CrossRefMATHGoogle Scholar
  5. 5.
    del Mar Hershenson, M., Boyd, S.P., Lee, T.H.: Optimal design of a CMOS op-amp via geometric programming. IEEE Trans Comput. Aided Des. 20, 1–21 (2001)CrossRefGoogle Scholar
  6. 6.
    Ecker, J.G.: Geometric programming: methods, computations and applications. SIAM Rev. 22, 338–362 (1980)CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    Feigin, P.D., Passy, U.: The geometric programming dual to the extinction probability problem in simple branching processes. Ann. Prob. 9, 498–503 (1981)CrossRefMATHMathSciNetGoogle Scholar
  8. 8.
    Hoffman, K.: Analysis in Euclidean Space. Prentice-Hall, Englewood Cliffs (1975)MATHGoogle Scholar
  9. 9.
    Hunter, D.R., Lange, K.: A tutorial on MM algorithms. Am. Stat. 58, 30–37 (2004)CrossRefMathSciNetGoogle Scholar
  10. 10.
    Lange, K.: Optimization. Springer, New York (2004)CrossRefMATHGoogle Scholar
  11. 11.
    Lange, K., Hunter, D.R., Yang, I.: Optimization transfer using surrogate objective functions (with discussion). J Comput Graph. Stat. 9, 1–59 (2000)MathSciNetGoogle Scholar
  12. 12.
    Mazumdar, M., Jefferson, T.R.: Maximum likelihood estimates for multinomial probabilities via geometric programming. Biometrika 70, 257–261 (1983)CrossRefMATHMathSciNetGoogle Scholar
  13. 13.
    Nocedal, J., Wright, S.J.: Numer. Optim. Springer, Berlin (1999)CrossRefGoogle Scholar
  14. 14.
    Passy, U., Wilde, D.J.: A geometric programming algorithm for solving chemical equilibrium problems. SIAM J Appl Math. 16, 363–373 (1968)CrossRefGoogle Scholar
  15. 15.
    Peressini, A.L., Sullivan, F.E., Uhl, J.J. Jr.: The Mathematics of Nonlinear Programming. Springer, New York (1988)Google Scholar
  16. 16.
    Peterson, E.L.: Geometric programming. SIAM Rev. 18, 338–362 (1976)CrossRefGoogle Scholar
  17. 17.
    Ruszczynski, A.: Optimization. Princeton University Press, Princeton (2006)MATHGoogle Scholar
  18. 18.
    Sha, F., Saul, L.K., Lee, D.D.: Multiplicative updates for nonnegative quadratic programming in support vector machines. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems, vol. 15, pp. 1065–1073. MIT Press, CambridgeGoogle Scholar
  19. 19.
    Steele, J.M.: The Cauchy-Schwarz Master Class: An Introduction to the Art of Inequalities. Cambridge University Press and the Mathematical Association of America, Cambridge (2004)CrossRefGoogle Scholar
  20. 20.
    Wang, Y., Zhang, K., Shen, P.: A new type of condensation curvilinear path algorithm for unconstrained generalized geometric programming. Math. Comput. Model. 35, 1209–1219 (2002)CrossRefMATHMathSciNetGoogle Scholar
  21. 21.
    Zhou, H., Alexander, D., Lange, K.L.: A quasi-Newton acceleration method for high-dimensional optimization algorithms. Stat. Comput. 21, 173–261 (2011)CrossRefMathSciNetGoogle Scholar
  22. 22.
    Zhou, H., Lange, K.L.: MM algorithms for some discrete multivariate distributions. J. Comput. Graph. Stat 19(3), 645–665 (2010)CrossRefMathSciNetGoogle Scholar
  23. 23.
    Zhou, H., Lange, K.L.: A fast procedure for calculating importance weights in bootstrap sampling. Comput. Stat. Data Anal. 55, 26–33 (2011)CrossRefMATHMathSciNetGoogle Scholar
  24. 24.
    Zhou, H., Lange, K.L.: Path following in the exact penalty method of convex programming. arXiv:1201.3593 (2011)Google Scholar
  25. 25.
    Zhou, H., Lange, K.L., Suchard, M.A.: Graphical processing units and high-dimensional optimization. Stat. Sci. 25(3), 311–324 (2010)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2012

Authors and Affiliations

  1. 1.Departments of Biomathematics, Human Genetics, and StatisticsUniversity of CaliforniaLos AngelesUSA
  2. 2.Department of StatisticsNorth Carolina State UniversityRaleighUSA

Personalised recommendations