Skip to main content
SpringerLink
Log in

Loss functions for finite sets

Computational Optimization and Applications volume 84, pages 421–447 (2023)Cite this article

Abstract

This paper studies loss functions for finite sets. For a given finite set S, we give sum-of-square type loss functions of minimum degree. When S is the vertex set of a standard simplex, we show such loss functions have no spurious minimizers (i.e., every local minimizer is a global one). Up to transformations, we give similar loss functions without spurious minimizers for general finite sets. When S is approximately given by a sample set T, we show how to get loss functions by solving a quadratic optimization problem. Numerical experiments and applications are given to show the efficiency of these loss functions.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2

Data availability statement

We do not analyse or generate any datasets, because our work proceeds within a theoretical and mathematical approach.

Notes

  1. A local minimizer that is not a global minimizer is called a spurious minimizer.

References

  1. Babbush, R., Denchev, V., Ding, N., et al.: Construction of non-convex polynomial loss functions for training a binary classifier with quantum annealing. Preprint (2014). arXiv:1406.4203

  2. Barron, J.T.: A general and adaptive robust loss function. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019)

  3. Beyhaghi, P., Alimo, R., Bewley, T.: A derivative-free optimization algorithm for the efficient minimization of functions obtained via statistical averaging. Comput. Optim. Appl. 76(1), 1–31 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  4. Cheng, D., Gong, Y., Zhou, S. et al.: Person re-identification by multi-channel parts-based CNN with improved triplet loss function. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)

  5. Christoffersen, P., Jacobs, K.: The importance of the loss function in option valuation. J. Financ. Econ. 72(2), 291–318 (2004)

    Article  Google Scholar 

  6. Corless, R.M., Gianni, P.M., Trager, B.M.: A reordered Schur factorization method for zero-dimensional polynomial systems with multiple roots. In: Proceedings of the International Symposium on Symbolic and Algebraic Computation, Maui, Hawaii, pp. 133–140 (1977)

  7. Cox, D., Little, J., OShea, D.: Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra. Springer (2013)

  8. Fan, J., Nie, J., Zhou, A.: Tensor eigenvalue complementarity problems. Math. Program. 170(2), 507–539 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  9. Gonzalez, S., Miikkulainen, R.: Optimizing loss functions through multi-variate Taylor polynomial parameterization. In: Proceedings of the Genetic and Evolutionary Computation Conference (2021)

  10. Guo, B., Nie, J., Yang, Z.: Learning diagonal Gaussian mixture models and incomplete tensor decompositions. Vietnam J. Math. 50(2), 421–446 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  11. Henrion, D., Lasserre, J., Lofberg, J.: GloptiPoly 3: moments, optimization and semidefinite programming. Optim. Methods Softw. 24, 761–779 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  12. Huber, P.J.: Robust estimation of a location parameter. In: Kotz S., Johnson N.L. (eds.) Breakthroughs in Statistics. Springer Series in Statistics (Perspectives in Statistics). Springer, New York (1992).https://doi.org/10.1007/978-1-4612-4380-9_35

  13. Ichihara, H.: Optimal control for polynomial systems using matrix sum of squares relaxations. IEEE Trans. Autom. Control 54(5), 1048–1053 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  14. Ito, Y., Fujimoto, K.: On optimal control with polynomial cost functions for linear systems with time-invariant stochastic parameters. In: American Control Conference (ACC). IEEE (2021)

  15. Jagerman, D.L.: Some properties of the Erlang loss function. Bell Syst. Tech. J. 53(3), 525–551 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  16. Kelley, C.T.: Iterative Methods for Linear and Nonlinear Equations, Frontiers in Applied Mathematics, vol. 16. SIAM, Philadelphia (1995)

    Book  Google Scholar 

  17. Ko, Y.H., Kim, K.J., Jun, C.H.: A new loss function-based method for multiresponse optimization. J. Qual. Technol. 37(1), 50–59 (2005)

    Article  Google Scholar 

  18. Lasserre, J.B.: Global optimization with polynomials and the problem of moments. SIAM J. Optim. 11, 796–817 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  19. Lasserre, J.B.: An Introduction to Polynomial and Semi-Algebraic Optimization. Cambridge University Press (2015)

  20. Lasserre, J.B.: The moment-SOS hierarchy. In: Sirakov, B., Ney de Souza, P., Viana, M. (eds.) Proceedings of the International Congress of Mathematicians (ICM 2018), vol. 3, pp. 3761–3784. World Scientific (2019)

  21. Laszka, A., Szeszlér, D., Buttyán, L.: Linear loss function for the network blocking game: an efficient model for measuring network robustness and link criticality. In: International Conference on Decision and Game Theory for Security. Springer, Berlin, Heidelberg (2012)

  22. Lasserre, J.B.: Homogeneous polynomials and spurious local minima on the unit sphere. Optim. Lett. (2021). https://doi.org/10.1007/s11590-021-01811-3

    Article  MATH  Google Scholar 

  23. Laurent, M.: Sums of squares, moment matrices and optimization over polynomials. In: Emerging Applications of Algebraic Geometry of IMA Volumes in Mathematics and its Applications, vol. 149, pp. 157–270. Springer (2009)

  24. Laurent, M.: Optimization over polynomials: selected topics. In: Jang, S.Y., Kim, Y.R., Lee, D.-W., Yie, I. (eds.) Proceedings of the International Congress of Mathematicians, pp. 843–869 (2014)

  25. Leung, B.P.K., Spiring, F.A.: The inverted beta loss function: properties and applications. IIE Trans. 34(12), 1101–1109 (2002)

    Article  Google Scholar 

  26. Li, Z., Cai, J., Wei, K.: Toward the optimal construction of a loss function without spurious local minima for solving quadratic equations. IEEE Trans. Inf. Theory 66(5), 3242–3260 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  27. More, J.J.: The Levenberg–Marquardt algorithm: implementation and theory. In: Watson, G.A. (ed.) Lecture Notes in Mathematics 630: Numerical Analysis, pp. 105–116. Springer, Berlin (1978)

    Google Scholar 

  28. Nie, J.: The hierarchy of local minimums in polynomial optimization. Math. Program. 151(2), 555–583 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  29. Nie, J., Yang, Z., Zhou, G.: The saddle point problem of polynomials. Found. Comput. Math. 22(4), 1–37 (2021)

    MathSciNet  Google Scholar 

  30. Nie, J.: Generating polynomials and symmetric tensor decompositions. Found. Comput. Math. 17, 423–465 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  31. Nie, J.: Low rank symmetric tensor approximations. SIAM J. Matrix Anal. Appl. 38(4), 1517–1540 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  32. Schorfheide, F.: Loss function-based evaluation of DSGE models. J. Appl. Economet. 15(6), 645–670 (2000)

    Article  Google Scholar 

  33. Sturmfels, B.: Solving systems of polynomial equations. In: CBMS Regional Conference Series in Mathematics, vol. 97. AMS, Providence (2002)

  34. Sturm, J.: Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones. Optim. Methods Softw. 11, 625–653 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  35. Sudre, C.H., Li, W., Vercauteren, T., et al.: Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pp. 240–248. Springer, Cham (2017)

  36. Syed, M.N., Pardalos, P.M., Principe, J.C.: On the optimization properties of the correntropic loss function in data analysis. Optim. Lett. 8(3), 823–839 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  37. Wang, Q., Ma, Y., Zhao, K., Tian, Y.: A comprehensive survey of loss functions in machine learning. Ann. Data Sci. (2020). https://doi.org/10.1007/s40745-020-00253-5

    Article  Google Scholar 

  38. Wu, Z., Shamsuzzaman, M., Pan, E.S.: Optimization design of control charts based on Taguchi’s loss function and random process shifts. Int. J. Prod. Res. 42(2), 379–390 (2004)

    Article  MATH  Google Scholar 

  39. Yuan, Y.X.: Recent advances in numerical methods for nonlinear equations and nonlinear least squares. Numer. Algebra Control Optim. 1, 15–34 (2011)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Funding

The authors are partially supported by the NSF Grant DMS-2110780.

Author information

Authors and Affiliations

  1. Department of Mathematics, University of California San Diego, 9500 Gilman Drive, La Jolla, CA, 92093, USA

    Jiawang Nie

  2. Department of Mathematics, Texas A &M University, College Station, TX, 77843-3368, USA

    Suhan Zhong

Authors
  1. Jiawang Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Suhan Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Suhan Zhong.

Ethics declarations

Conflict of interest

They have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nie, J., Zhong, S. Loss functions for finite sets. Comput Optim Appl 84, 421–447 (2023). https://doi.org/10.1007/s10589-022-00420-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-022-00420-9

Keywords

Mathematics Subject Classification

search

Navigation