Optimization and Engineering

, Volume 17, Issue 4, pp 897–918 | Cite as

Data fitting with geometric-programming-compatible softmax functions

  • Warren HoburgEmail author
  • Philippe Kirschen
  • Pieter Abbeel


Motivated by practical applications in engineering, this article considers the problem of approximating a set of data with a function that is compatible with geometric programming (GP). Starting with well-established methods for fitting max-affine functions, it is shown that improved fits can be obtained using an extended function class based on the softmax of a set of affine functions. The softmax is generalized in two steps, with the most expressive function class using an implicit representation that allows fitting algorithms to locally tune softness. Each of the proposed function classes is directly compatible with the posynomial constraint forms in GP. Max-monomial fitting and posynomial fitting are shown to correspond to fitting special cases of the proposed implicit softmax function class. The fitting problem is formulated as a nonlinear least squares regression, solved locally using a Levenberg–Marquardt algorithm. Practical implementation considerations are discussed. The article concludes with numerical examples from aerospace engineering and electrical engineering.


Convex optimization Convex regression Geometric programming 



The authors thank Aude Hofleitner, Timothy Hunter, and several anonymous reviewers for their thorough and insightful comments on the draft. This work was supported by a National Science Foundation Graduate Research Fellowship.


  1. Agarwal S, Mierle K (2010) Others: ceres solver.
  2. Babakhani A, Lavaei J, Doyle J, Hajimiri A (2010) Finding globally optimum solutions in antenna optimization problems. In: IEEE international symposium on antennas and propagationGoogle Scholar
  3. Beightler CS, Phillips DT (1976) Applied geometric programming. Wiley, New YorkzbMATHGoogle Scholar
  4. Boyd S, Kim SJ, Vandenberghe L, Hassibi A (2007) A tutorial on geometric programming. Optim Eng 8(1):67–127MathSciNetCrossRefzbMATHGoogle Scholar
  5. Boyd S, Vandenberghe L (2004) Convex Optim. Cambridge University Press, New YorkCrossRefzbMATHGoogle Scholar
  6. Boyd SP, Kim SJ, Patil DD, Horowitz MA (2005) Digital circuit optimization via geometric programming. Op Res 53:899–932MathSciNetCrossRefzbMATHGoogle Scholar
  7. Chiang M (2005) Geometric programming for communication systems. Commun Inf Theory 2:1–154. doi:10.1516/0100000005.
  8. Daems W, Gielen G, Sansen W (2003) Simulation-based generation of posynomial performance models for the sizing of analog integrated circuits. IEEE Trans Comput Aided Des Integr Circuits Syst 22(5):517–534CrossRefGoogle Scholar
  9. Drela M (2000) Xfoil subsonic airfoil development system. Open source software.
  10. Duffin RJ, Peterson EL, Zener C (1967) Geometric programming: theory and application. Wiley, New YorkzbMATHGoogle Scholar
  11. Hannah L, Dunson D (2012) Ensemble methods for convex regression with applications to geometric programming based circuit design. arXiv preprint.
  12. Hannah LA, Dunson DB (2011) Multivariate convex regression with adaptive partitioning. arXiv preprint.
  13. Hoburg W, Abbeel P (2014) Geometric programming for aircraft design optimization. AIAA J 52:2414–2426CrossRefGoogle Scholar
  14. Kasamsetty K, Ketkar M, Sapatnekar SS (2000) A new class of convex functions for delay modeling and its application to the transistor sizing problem [cmos gates]. IEEE Trans Comput Aided Des Integr Circuits Syst 19(7):779–788CrossRefGoogle Scholar
  15. Kim J, Vandenberghe L, Yang CKK (2010) Convex piecewise-linear modeling method for circuit optimization via geometric programming. IEEE Trans Comput Aided Des Integr Circuits Syst 29(11):1823–1827. doi: 10.1109/TCAD.2010.2053151 CrossRefGoogle Scholar
  16. Li X, Gopalakrishnan P, Xu Y, Pileggi T (2004) Robust analog/rf circuit design with projection-based posynomial modeling. In: Proceedings of the 2004 IEEE/ACM international conference on computer-aided design, pp 855–862. IEEE computer societyGoogle Scholar
  17. Magnani A, Boyd SP (2009) Convex piecewise-linear fitting. Optim Eng 10:1–17. doi: 10.1007/s11081-008-9045-3 MathSciNetCrossRefzbMATHGoogle Scholar
  18. Marquardt DW (1963) An algorithm for least-squares estimation of nonlinear parameters. J Soc Ind Appl Math 11(2):431–441.
  19. Mehrotra S (1992) On the implementation of a primal-dual interior point method. SIAM J Optim 2(4):575–601MathSciNetCrossRefzbMATHGoogle Scholar
  20. Moré JJ (1978) The Levenberg–Marquardt algorithm: implementation and theory. In: Watson GA (ed) Numerical analysis. Springer, Berlin, pp 105–116CrossRefGoogle Scholar
  21. Nesterov Y, Nemirovsky A (1994) Interior-point polynomial methods in convex programming, volume 13 of studies in applied mathematics. SIAM, PhiladelphiaGoogle Scholar
  22. Nocedal J, Wright SJ (2006) Numerical optimization. Springer, New YorkzbMATHGoogle Scholar
  23. Oliveros J, Cabrera D, Roa E, Van Noije W (2008) An improved and automated design tool for the optimization of cmos otas using geometric programming. In: Proceedings of the 21st annual symposium on integrated circuits and system design, pp 146–151. ACMGoogle Scholar
  24. Press WH, Teukolsky SA, Vetterling WT, Flannery BP (2007) Numerical recipes 3rd edition: the art of scientific computing. Cambridge University Press, CambridgezbMATHGoogle Scholar
  25. Wilde D (1978) Globally optimal design. Wiley, New York.
  26. Ypma TJ (1995) Historical development of the Newton–Raphson method. SIAM Rev 37(4):531–551MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Warren Hoburg
    • 1
    Email author
  • Philippe Kirschen
    • 1
  • Pieter Abbeel
    • 2
  1. 1.Department of Aeronautics and AstronauticsMassachusetts Institute of TechnologyCambridgeUSA
  2. 2.Computer Science DepartmentUniversity of CaliforniaBerkeleyUSA

Personalised recommendations