Skip to main content
SpringerLink
Log in

Data fitting with geometric-programming-compatible softmax functions

  • Published:
Optimization and Engineering Aims and scope Submit manuscript

Abstract

Motivated by practical applications in engineering, this article considers the problem of approximating a set of data with a function that is compatible with geometric programming (GP). Starting with well-established methods for fitting max-affine functions, it is shown that improved fits can be obtained using an extended function class based on the softmax of a set of affine functions. The softmax is generalized in two steps, with the most expressive function class using an implicit representation that allows fitting algorithms to locally tune softness. Each of the proposed function classes is directly compatible with the posynomial constraint forms in GP. Max-monomial fitting and posynomial fitting are shown to correspond to fitting special cases of the proposed implicit softmax function class. The fitting problem is formulated as a nonlinear least squares regression, solved locally using a Levenberg–Marquardt algorithm. Practical implementation considerations are discussed. The article concludes with numerical examples from aerospace engineering and electrical engineering.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. The “GP” acronym is overloaded, referring both to geometric programs—the class of optimization problem discussed in this article—and geometric programming—the practice of using such programs to model and solve optimization problems.

  2. A function \(f: {\mathcal {X}} \rightarrow {\mathbb {R}}\) is convex if its domain \({\mathcal {X}}\) is a convex set and the property \(f(\theta {\mathbf {x}_1} + (1-\theta ) {\mathbf {x}_2}) \le \theta f({\mathbf {x}_1}) + (1-\theta ) f({\mathbf {x}_2})\) holds for all \(0 \le \theta \le 1\) and for all \({\mathbf {x}_1}, {\mathbf {x}_2} \in {\mathcal {X}}\).

  3. It is also possible to achieve GP-compatible fits via multiple functions, resulting in a set of constraints.

  4. Equation (20) can also be interpreted as a generalized posynomial (Boyd et al. 2007; Kasamsetty et al. 2000) constraint on w, \(w = \left( \sum _{k=1}^K e^{\alpha b_k} \prod _{i=1}^n u_i^{\alpha a_{ik}} \right) ^{\frac{1}{\alpha }}\).

References

  • Agarwal S, Mierle K (2010) Others: ceres solver. http://ceres-solver.org

  • Babakhani A, Lavaei J, Doyle J, Hajimiri A (2010) Finding globally optimum solutions in antenna optimization problems. In: IEEE international symposium on antennas and propagation

  • Beightler CS, Phillips DT (1976) Applied geometric programming. Wiley, New York

    MATH  Google Scholar 

  • Boyd S, Kim SJ, Vandenberghe L, Hassibi A (2007) A tutorial on geometric programming. Optim Eng 8(1):67–127

    Article  MathSciNet  MATH  Google Scholar 

  • Boyd S, Vandenberghe L (2004) Convex Optim. Cambridge University Press, New York

    Book  MATH  Google Scholar 

  • Boyd SP, Kim SJ, Patil DD, Horowitz MA (2005) Digital circuit optimization via geometric programming. Op Res 53:899–932

    Article  MathSciNet  MATH  Google Scholar 

  • Chiang M (2005) Geometric programming for communication systems. Commun Inf Theory 2:1–154. doi:10.1516/0100000005. http://portal.acm.org/citation.cfm?id=1166381.1166382

  • Daems W, Gielen G, Sansen W (2003) Simulation-based generation of posynomial performance models for the sizing of analog integrated circuits. IEEE Trans Comput Aided Des Integr Circuits Syst 22(5):517–534

    Article  Google Scholar 

  • Drela M (2000) Xfoil subsonic airfoil development system. Open source software. http://web.mit.edu/drela/Public/web/xfoil/

  • Duffin RJ, Peterson EL, Zener C (1967) Geometric programming: theory and application. Wiley, New York

    MATH  Google Scholar 

  • Hannah L, Dunson D (2012) Ensemble methods for convex regression with applications to geometric programming based circuit design. arXiv preprint. http://arxiv.org/abs/1206.4645

  • Hannah LA, Dunson DB (2011) Multivariate convex regression with adaptive partitioning. arXiv preprint. http://arxiv.org/abs/1105.1924

  • Hoburg W, Abbeel P (2014) Geometric programming for aircraft design optimization. AIAA J 52:2414–2426

    Article  Google Scholar 

  • Kasamsetty K, Ketkar M, Sapatnekar SS (2000) A new class of convex functions for delay modeling and its application to the transistor sizing problem [cmos gates]. IEEE Trans Comput Aided Des Integr Circuits Syst 19(7):779–788

    Article  Google Scholar 

  • Kim J, Vandenberghe L, Yang CKK (2010) Convex piecewise-linear modeling method for circuit optimization via geometric programming. IEEE Trans Comput Aided Des Integr Circuits Syst 29(11):1823–1827. doi:10.1109/TCAD.2010.2053151

    Article  Google Scholar 

  • Li X, Gopalakrishnan P, Xu Y, Pileggi T (2004) Robust analog/rf circuit design with projection-based posynomial modeling. In: Proceedings of the 2004 IEEE/ACM international conference on computer-aided design, pp 855–862. IEEE computer society

  • Magnani A, Boyd SP (2009) Convex piecewise-linear fitting. Optim Eng 10:1–17. doi:10.1007/s11081-008-9045-3

    Article  MathSciNet  MATH  Google Scholar 

  • Marquardt DW (1963) An algorithm for least-squares estimation of nonlinear parameters. J Soc Ind Appl Math 11(2):431–441. http://www.jstor.org/stable/2098941

  • Mehrotra S (1992) On the implementation of a primal-dual interior point method. SIAM J Optim 2(4):575–601

    Article  MathSciNet  MATH  Google Scholar 

  • Moré JJ (1978) The Levenberg–Marquardt algorithm: implementation and theory. In: Watson GA (ed) Numerical analysis. Springer, Berlin, pp 105–116

    Chapter  Google Scholar 

  • Nesterov Y, Nemirovsky A (1994) Interior-point polynomial methods in convex programming, volume 13 of studies in applied mathematics. SIAM, Philadelphia

    Google Scholar 

  • Nocedal J, Wright SJ (2006) Numerical optimization. Springer, New York

    MATH  Google Scholar 

  • Oliveros J, Cabrera D, Roa E, Van Noije W (2008) An improved and automated design tool for the optimization of cmos otas using geometric programming. In: Proceedings of the 21st annual symposium on integrated circuits and system design, pp 146–151. ACM

  • Press WH, Teukolsky SA, Vetterling WT, Flannery BP (2007) Numerical recipes 3rd edition: the art of scientific computing. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  • Wilde D (1978) Globally optimal design. Wiley, New York. http://books.google.com/books?id=XYBRAAAAMAAJ

  • Ypma TJ (1995) Historical development of the Newton–Raphson method. SIAM Rev 37(4):531–551

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

The authors thank Aude Hofleitner, Timothy Hunter, and several anonymous reviewers for their thorough and insightful comments on the draft. This work was supported by a National Science Foundation Graduate Research Fellowship.

Author information

Authors and Affiliations

  1. Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA, 02139, USA

    Warren Hoburg & Philippe Kirschen

  2. Computer Science Department, University of California, Berkeley, CA, 94720, USA

    Pieter Abbeel

Authors
  1. Warren Hoburg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Philippe Kirschen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pieter Abbeel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Warren Hoburg.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hoburg, W., Kirschen, P. & Abbeel, P. Data fitting with geometric-programming-compatible softmax functions. Optim Eng 17, 897–918 (2016). https://doi.org/10.1007/s11081-016-9332-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11081-016-9332-3

Keywords

Search

Navigation