Skip to main content
SpringerLink
Log in

Distributionally robust optimization using optimal transport for Gaussian mixture models

  • Research Article
  • Published:
Optimization and Engineering Aims and scope Submit manuscript

Abstract

Distributionally robust optimization (DRO) is an increasingly popular approach for optimization under uncertainty when the probability distribution of the uncertain parameter is unknown. Well-explored DRO approaches in literature, such as Wasserstein DRO, do not make any specific assumptions on the nature of the candidate distributions considered in the ambiguity set. However, in many practical applications, the uncertain parameter may be sourced from a distribution that can be well modeled as a Gaussian Mixture Model (GMM) whose components represent the different subpopulations the uncertain parameter may belong to. In this work, we propose a new DRO method based on an ambiguity set constructed around a GMM. The proposed DRO approach is illustrated on a numerical example as well as a portfolio optimization case study for uncertainty sourced from various distributions. The results obtained from the proposed DRO approach are compared with those from Wasserstein DRO, and are shown to be superior in quality with respect to out-of-sample performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Aragam B, Dan C, Ravikumar P, Xing EP (2018) Identifiability of nonparametric mixture models and bayes optimal clustering. arXiv preprint arXiv:1802.04397

  • Artzner P, Delbaen F, Eber JM, Heath D (1999) Coherent measures of risk. Math Finance 9(3):203–228

    Article  MathSciNet  MATH  Google Scholar 

  • Bayraksan G, Love DK (2015) Data-driven stochastic programming using phi-divergences. In: The operations research revolution. INFORMS, pp 1–19

  • Ben-Tal A, Nemirovski A (1998) Robust convex optimization. Math Oper Res 23(4):769–805

    Article  MathSciNet  MATH  Google Scholar 

  • Benamou JD (2003) Numerical resolution of an “unbalanced" mass transport problem. ESAIM Math Model Numer Anal 37(5):851–868

    Article  MathSciNet  MATH  Google Scholar 

  • Bertsimas D, Kallus N (2020) From predictive to prescriptive analytics. Manag Sci 66(3):1025–1044

    Article  Google Scholar 

  • Bishop CM, Nasrabadi NM (2006) Pattern recognition and machine learning. Springer, New York

    Google Scholar 

  • Blanchet J, Murthy K, Zhang F (2022) Optimal transport-based distributionally robust optimization: structural properties and iterative schemes. Math Oper Res 47(2):1500–1529

    Article  MathSciNet  MATH  Google Scholar 

  • Blondel M, Seguy V, Rolet A (2018) Smooth and sparse optimal transport. In: International conference on artificial intelligence and statistics. PMLR, pp 880–889

  • Caffarelli LA, McCann RJ (2010) Free boundaries in optimal transport and Monge-Ampere obstacle problems. Ann Math pp 673–730

  • Chen Y, Georgiou TT, Tannenbaum A (2018) Optimal transport for gaussian mixture models. IEEE Access 7:6269–6278

    Article  Google Scholar 

  • Chen Z, Kuhn D, Wiesemann W (2022) Data-driven chance constrained programs over Wasserstein balls. Oper Res. https://doi.org/10.1287/opre.2022.2330

    Article  MATH  Google Scholar 

  • Chizat L, Peyré G, Schmitzer B, Vialard FX (2018) Scaling algorithms for unbalanced optimal transport problems. Math Comput 87(314):2563–2609

    Article  MathSciNet  MATH  Google Scholar 

  • Clason C, Lorenz DA, Mahler H, Wirth B (2021) Entropic regularization of continuous optimal transport problems. J Math Anal Appl 494(1):124432

    Article  MathSciNet  MATH  Google Scholar 

  • Cuturi M (2013) Sinkhorn distances: lightspeed computation of optimal transport. In: Advances in neural information processing systems, vol 26

  • Dantzig GB (1955) Linear programming under uncertainty. Manag Sci 1(3–4):197–206

    Article  MathSciNet  MATH  Google Scholar 

  • Delage E, Ye Y (2010) Distributionally robust optimization under moment uncertainty with application to data-driven problems. Oper Res 58(3):595–612

    Article  MathSciNet  MATH  Google Scholar 

  • Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B (Methodol) 39(1):1–22

    MathSciNet  MATH  Google Scholar 

  • Gao R, Kleywegt A (2022) Distributionally robust stochastic optimization with Wasserstein distance. Math Oper Res 48(2023):603–655

    MathSciNet  Google Scholar 

  • Ghaoui LE, Oks M, Oustry F (2003) Worst-case value-at-risk and robust portfolio optimization: a conic programming approach. Oper Res 51(4):543–556

    Article  MathSciNet  MATH  Google Scholar 

  • Goh J, Sim M (2010) Distributionally robust optimization and its tractable approximations. Oper Res 58(4–part–1):902–917

    Article  MathSciNet  MATH  Google Scholar 

  • Goldfarb D, Iyengar G (2003) Robust portfolio selection problems. Math Oper Res 28(1):1–38

    Article  MathSciNet  MATH  Google Scholar 

  • Grunwald PD, Dawid AP (2004) Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory. arXiv: math/0410076

  • Haasler I, Singh R, Zhang Q, Karlsson J, Chen Y (2021) Multi-marginal optimal transport and probabilistic graphical models. IEEE Trans Inf Theory 67(7):4647–4668

    Article  MathSciNet  MATH  Google Scholar 

  • Hanasusanto GA, Kuhn D (2013) Robust data-driven dynamic programming. In: Advances in neural information processing systems, vol 26

  • Hota AR, Cherukuri A, Lygeros J (2019) Data-driven chance constrained optimization under Wasserstein ambiguity sets. In: 2019 American control conference (ACC). IEEE, pp 1501–1506

  • Jiang R, Guan Y (2018) Risk-averse two-stage stochastic program with distributional ambiguity. Oper Res 66(5):1390–1405

    Article  MathSciNet  MATH  Google Scholar 

  • Kaut M, Stein W (2003) Evaluation of scenario-generation methods for stochastic programming. Humboldt-Universität zu Berlin, Mathematisch-Naturwissenschaftliche Fakultät

  • Keith AJ, Ahner DK (2021) A survey of decision making and optimization under uncertainty. Ann Oper Res 300(2):319–353

    Article  MathSciNet  MATH  Google Scholar 

  • Lévy B (2015) A numerical algorithm for \( l_ \{2\}\) semi-discrete optimal transport in 3d. ESAIM Math Model Numer Anal Modélisation Math Anal Numérique 49(6):1693–1715

    Article  MATH  Google Scholar 

  • Li JYM, Mao T (2022) A general wasserstein framework for data-driven distributionally robust optimization: Tractability and applications. arXiv preprint arXiv:2207.09403

  • Liu H, Qiu J, Zhao J (2022) A data-driven scheduling model of virtual power plant using Wasserstein distributionally robust optimization. Int J Electr Power Energy Syst 137:107801

    Article  Google Scholar 

  • McLachlan GJ, Lee SX, Rathnayake SI (2019) Finite mixture models. Ann Rev Stat Appl 6:355–378

    Article  MathSciNet  Google Scholar 

  • Mehrotra S, Zhang H (2014) Models and algorithms for distributionally robust least squares problems. Math Program 146(1–2):123–141

    Article  MathSciNet  MATH  Google Scholar 

  • Mérigot Q (2011) A multiscale approach to optimal transport. Comput Graphics Forum 30(5):1583–1592

    Article  Google Scholar 

  • Mohajerin Esfahani P, Kuhn D (2018) Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations. Math Program 171(1):115–166

    Article  MathSciNet  MATH  Google Scholar 

  • Monge G (1781) Memoir on the theory of cuttings and embankments. Histoire de l’Acad’e mie Royale des Sciences de Paris

  • Natarajan K, Teo CP (2017) On reduced semidefinite programs for second order moment bounds with applications. Math Program 161:487–518

    Article  MathSciNet  MATH  Google Scholar 

  • Nenna L (2016) Numerical methods for multi-marginal optimal transportation. PhD thesis, Université Paris sciences et lettres

  • Ning C, You F (2019) Optimization under uncertainty in the era of big data and deep learning: When machine learning meets mathematical programming. Comput Chem Eng 125:434–448

    Article  Google Scholar 

  • Oliker VI, Prussner LD (1989) On the numerical solution of the equation and its discretizations, i. Numer Math 54(3):271–293

    Article  MATH  Google Scholar 

  • Pass B (2012) On the local structure of optimal measures in the multi-marginal optimal transportation problem. Calc Var Partial Differ Equ 43(3–4):529–536

    Article  MathSciNet  MATH  Google Scholar 

  • Pass B (2015) Multi-marginal optimal transport: theory and applications. ESAIM Math Model Numer Anal Modélisation Math Anal Numérique 49(6):1771–1790

    Article  MathSciNet  MATH  Google Scholar 

  • Pflug G, Wozabal D (2007) Ambiguity in portfolio selection. Quant Finance 7(4):435–442

    Article  MathSciNet  MATH  Google Scholar 

  • Popescu I (2005) A semidefinite programming approach to optimal-moment bounds for convex classes of distributions. Math Oper Res 30(3):632–657

    Article  MathSciNet  MATH  Google Scholar 

  • Rahimian H, Mehrotra S (2019) Distributionally robust optimization: a review. arXiv preprint arXiv:1908.05659

  • Ruszczyński A, Shapiro A (2006) Optimization of convex risk functions. Math Oper Res 31(3):433–452

    Article  MathSciNet  MATH  Google Scholar 

  • Sahinidis NV (2004) Optimization under uncertainty: state-of-the-art and opportunities. Comput Chem Eng 28(6–7):971–983

    Article  Google Scholar 

  • Scarf H (1958) A min max solution of an inventory problem. Studies in the mathematical theory of inventory and production

  • Shafieezadeh Abadeh S, Mohajerin Esfahani PM, Kuhn D (2015) Distributionally robust logistic regression. In: Advances in neural information processing systems, vol 28

  • Shapiro A, Nemirovski A (2005) On complexity of stochastic programming problems. Continuous optimization: current trends and modern applications, pp 111–146

  • Sinkhorn R (1967) Diagonal equivalence to matrices with prescribed row and column sums. Am Math Mon 74(4):402–405

    Article  MathSciNet  MATH  Google Scholar 

  • Van Parys BP, Goulart PJ, Kuhn D (2016) Generalized gauss inequalities via semidefinite programming. Math Program 156:271–302

    Article  MathSciNet  MATH  Google Scholar 

  • Wallace SW, Ziemba WT (2005) Applications of stochastic programming. SIAM, New Delhi

    Book  MATH  Google Scholar 

  • Wiesemann W, Kuhn D, Sim M (2014) Distributionally robust convex optimization. Oper Res 62(6):1358–1376

    Article  MathSciNet  MATH  Google Scholar 

  • Yang SB, Li Z (2022) Kernel distributionally robust chance-constrained process optimization. Comput Chem Eng 165:107953

    Article  Google Scholar 

  • You L, Ma H, Saha TK, Liu G (2021) Gaussian mixture model based distributionally robust optimal power flow with CVaR constraints. arXiv preprint arXiv:2110.13336

  • Yue MC, Kuhn D, Wiesemann W (2022) On linear optimization over Wasserstein balls. Math Program 195(1):1107–1122

    Article  MathSciNet  MATH  Google Scholar 

  • Zhu JJ, Jitkrittum W, Diehl M, Schölkopf B (2021) Kernel distributionally robust optimization: generalized duality theorem and stochastic approximation. In: International conference on artificial intelligence and statistics, PMLR, pp 280–288

Download references

Acknowledgements

The authors gratefully acknowledge the financial support from the Natural Sciences and Engineering Research Council of Canada (NSERC).

Author information

Authors and Affiliations

  1. Department of Chemical and Materials Engineering, University of Alberta, Edmonton, AB, T6G 1H9, Canada

    Sanjula Kammammettu, Shu-Bo Yang & Zukui Li

Authors
  1. Sanjula Kammammettu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shu-Bo Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zukui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zukui Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kammammettu, S., Yang, SB. & Li, Z. Distributionally robust optimization using optimal transport for Gaussian mixture models. Optim Eng (2023). https://doi.org/10.1007/s11081-023-09856-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11081-023-09856-2

Keywords

Search

Navigation