Skip to main content
SpringerLink
Log in

An inexact primal-dual algorithm for semi-infinite programming

  • Original Article
  • Published:
Mathematical Methods of Operations Research Aims and scope Submit manuscript

Abstract

This paper considers an inexact primal-dual algorithm for semi-infinite programming (SIP) for which it provides general error bounds. We create a new prox function for nonnegative measures for the dual update, and it turns out to be a generalization of the Kullback-Leibler divergence. We show that, with a tolerance for small errors (approximation and regularization error), this algorithm achieves an \({\mathcal {O}}(1/\sqrt{K})\) rate of convergence in terms of the optimality gap and constraint violation, where K is the total number of iterations. We then use our general error bounds to analyze the convergence and sample complexity of a specific primal-dual SIP algorithm based on Monte Carlo sampling. Finally, we provide numerical experiments to demonstrate the performance of this algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Aliprantis CD, Border KC (2006) Infinite dimensional analysis. A hitchhiker’s guide. Springer, Berlin

    MATH  Google Scholar 

  • Anstreicher KM, Wolsey LA (2009) Two “well-known” properties of subgradient optimization. Math Program 120(1):213–220

    MathSciNet  MATH  Google Scholar 

  • Betrò B (2004) An accelerated central cutting plane algorithm for linear semi-infinite programming. Math Program 101(3):479–495

    MathSciNet  MATH  Google Scholar 

  • Bhat N, Farias V, Moallemi CC (2012) Non-parametric approximate dynamic programming via the kernel method. In: Advances in neural information processing systems, pp 386–394

  • Bishop CM (2006) Pattern recognition and machine learning. Springer, Berlin

    MATH  Google Scholar 

  • Bonnans JF, Shapiro A (2013) Perturbation analysis of optimization problems. Springer, Berlin

    MATH  Google Scholar 

  • Bruck RE Jr (1977) On the weak convergence of an ergodic iteration for the solution of variational inequalities for monotone operators in hilbert space. J Math Anal Appl 61(1):159–164

    MathSciNet  MATH  Google Scholar 

  • Calafiore G, Campi MC (2005) Uncertain convex programs: randomized solutions and confidence levels. Math Program Ser A 102:25–46

    MathSciNet  MATH  Google Scholar 

  • Campi MC, Garatti S (2008) The exact feasibility of randomized solutions of uncertain convex programs. SIAM J Optim 19(3):1211–1230

    MathSciNet  MATH  Google Scholar 

  • de Farias DP, Van Roy B (2004) On constraint sampling in the linear programming approach to approximate dynamic programming. Math Oper Res 29(3):462–478

    MathSciNet  MATH  Google Scholar 

  • Dentcheva D, Ruszczynski A (2003) Optimization with stochastic dominance constraints. SIAM J Optim 14(2):548–566

    MathSciNet  MATH  Google Scholar 

  • Dentcheva D, Ruszczyński A (2009) Optimization with multivariate stochastic dominance constraints. Math Program 117(1):111–127

    MathSciNet  MATH  Google Scholar 

  • Dentcheva D, Wolfhagen E (2015) Optimization with multivariate stochastic dominance constraints. SIAM J Optim 25(1):564–588

    MathSciNet  MATH  Google Scholar 

  • Duchi J (2016) Introductory lectures on stochastic optimization. Park City Mathematics Institute, Graduate Summer School Lectures

  • Esfahani PM, Sutter T, Lygeros J (2015) Performance bounds for the scenario approach and an extension to a class of non-convex programs. IEEE Tran Autom Control 60(1):46–58

    MathSciNet  MATH  Google Scholar 

  • Fan K (1953) Minimax theorems. Proc Natl Acad Sci 39(1):42–47

    MathSciNet  MATH  Google Scholar 

  • Fonseca I, Leoni G (2007) Modern methods in the calculus of variations: \(Lp\) Spaces. Springer, Berlin

    MATH  Google Scholar 

  • Goberna MA, Lopez MA (2002) Linear semi-infinite programming theory: an updated survey. Eur J Oper Res 143(2):390–405

    MathSciNet  MATH  Google Scholar 

  • Goberna MÁ, López MA (2013) Semi-infinite programming: recent advances, vol 57. Springer, Berlin

    MATH  Google Scholar 

  • Goberna MA, López MA (2017) Recent contributions to linear semi-infinite optimization. 4OR 15(3):221–264

    MathSciNet  MATH  Google Scholar 

  • Gramlich G, Hettich R, Sachs EW (1995) Local convergence of sqp methods in semi-infinite programming. SIAM J Optim 5(3):641–658

    MathSciNet  MATH  Google Scholar 

  • Gribik PR (1979) A central-cutting-plane algorithm for semi-infinite programming problems. In: Semi-infinite Programming. Springer, Berlin, pp 66–82

  • Gustavsson E, Patriksson M, Strömberg A-B (2015) Primal convergence from dual subgradient methods for convex optimization. Math Program 150(2):365–390

    MathSciNet  MATH  Google Scholar 

  • Haskell WB, Shanthikumar JG, Shen ZM (2013) Optimization with a class of multivariate integral stochastic order constraints. Ann Oper Res 206(1):147–162

    MathSciNet  MATH  Google Scholar 

  • Haskell WB, Shanthikumar JG, Shen ZM (2017) Primal-dual algorithms for optimization with stochastic dominance. SIAM J Optim 27(1):34–66

    MathSciNet  MATH  Google Scholar 

  • Hettich R, Kortanek KO (1993) Semi-infinite programming: theory, methods, and applications. SIAM Rev 35(3):380–429

    MathSciNet  MATH  Google Scholar 

  • Hu J, Mello TH, Mehrotra S (2012) Sample average approximation of stochastic dominance constrained programs. Math Program 133(1):171–201

    MathSciNet  MATH  Google Scholar 

  • Ito S, Liu Y, Teo KL (2000) A dual parametrization method for convex semi-infinite programming. Ann Oper Res 98(1):189–213

    MathSciNet  MATH  Google Scholar 

  • Kiwiel KC, Larsson T, Lindberg PO (2007) Lagrangian relaxation via ballstep subgradient methods. Math Oper Res 32(3):669–686

    MathSciNet  MATH  Google Scholar 

  • Kortanek KO, No H (1993) A central cutting plane algorithm for convex semi-infinite programming problems. SIAM J Optim 3(4):901–918

    MathSciNet  MATH  Google Scholar 

  • Lan G, Nemirovski A, Shapiro A (2012) Validation analysis of mirror descent stochastic approximation method. Math Program 134(2):425–458

    MathSciNet  MATH  Google Scholar 

  • Larsson T, Patriksson M, Strömberg A-B (1999) Ergodic, primal convergence in dual subgradient schemes for convex programming. Math Program 86(2):283–312

    MathSciNet  MATH  Google Scholar 

  • Li D-H, Qi L, Tam J, Soon-Yi W (2004) A smoothing newton method for semi-infinite programming. J Glob Optim 30(2–3):169–194

    MathSciNet  MATH  Google Scholar 

  • Ling C, Ni Q, Qi L, Soon-Yi W (2010) A new smoothing newton-type algorithm for semi-infinite programming. J Global Optim 47(1):133–159

    MathSciNet  MATH  Google Scholar 

  • Lin Q, Nadarajah S, Soheili N (2017) Revisiting approximate linear programming using a saddle point based reformulation and root finding solution approach

  • Liu Y, Teo KL (2002) An adaptive dual parametrization algorithm for quadratic semi-infinite programming problems. J Global Optim 24(2):205–217

    MathSciNet  MATH  Google Scholar 

  • Liu Y, Teo KL, Wu S-Y (2004) A new quadratic semi-infinite programming algorithm based on dual parametrization. J Global Optim 29(4):401–413

    MathSciNet  MATH  Google Scholar 

  • López M, Still G (2007) Semi-infinite programming. Eur J Oper Res 180(2):491–518

    MathSciNet  MATH  Google Scholar 

  • Luenberger DG (1997) Optimization by vector space methods. Wiley, New York

    Google Scholar 

  • MacKay DJC (2003) Information theory, inference and learning algorithms. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  • Mahdavi M, Yang T, Jin R (2012) Online stochastic optimization with multiple objectives. arXiv preprint arXiv:1211.6013

  • Mehrotra S, Papp D (2014) A cutting surface algorithm for semi-infinite convex programming with an application to moment robust optimization. SIAM J Optim 24(4):1670–1697

    MathSciNet  MATH  Google Scholar 

  • Mello TH, Mehrotra S (2009) A cutting-surface method for uncertain linear programs with polyhedral stochastic dominance constraints. SIAM J Optim 20(3):1250–1273

    MathSciNet  MATH  Google Scholar 

  • Mohajerin Esfahani P, Sutter T, Kuhn D, Lygeros J (2017) From infinite to finite programs: explicit error bounds with applications to approximate dynamic programming. arXiv preprint arXiv:1701.06379

  • Nedić A, Ozdaglar A (2009) Approximate primal solutions and rate analysis for dual subgradient methods. SIAM J Optim 19(4):1757–1780

    MathSciNet  MATH  Google Scholar 

  • Nedić A, Ozdaglar A (2009) Subgradient methods for saddle-point problems. J Optim Theory Appl 142(1):205–228

    MathSciNet  MATH  Google Scholar 

  • Ni Q, Ling C, Qi L, Teo KL (2006) A truncated projected newton-type algorithm for large-scale semi-infinite programming. SIAM J Optim 16(4):1137–1154

    MathSciNet  MATH  Google Scholar 

  • Noyan N, Rudolf G (2013) Optimization with multivariate conditional value-at-risk constraints. Oper Res 61(4):990–1013

    MathSciNet  MATH  Google Scholar 

  • Noyan N, Rudolf G (2018) Optimization with stochastic preferences based on a general class of scalarization functions. Oper Res 66:463–486

    MathSciNet  Google Scholar 

  • Pinsker MS (1964) Information and information stability of random variables and processes. Holden-Day, San Francisco

    MATH  Google Scholar 

  • Qi L, Wu S-Y, Zhou G (2003) Semismooth newton methods for solving semi-infinite programming problems. J Global Optim 27(2–3):215–232

    MathSciNet  MATH  Google Scholar 

  • Qi L, Ling C, Tong X, Zhou G (2009) A smoothing projected newton-type algorithm for semi-infinite programming. Comput Optim Appl 42(1):1–30

    MathSciNet  MATH  Google Scholar 

  • Reemtsen R (1991) Discretization methods for the solution of semi-infinite programming problems. J Optim Theory Appl 71(1):85–103

    MathSciNet  MATH  Google Scholar 

  • Reemtsen R, Görner S (1998) Numerical methods for semi-infinite programming: a survey. In Semi-infinite programming, Springer, pp 195–275

  • Robert C, Casella G (2004) Monte Carlo statistical methods. Springer, New York

    MATH  Google Scholar 

  • Shapiro A (2009) Semi-infinite programming, duality, discretization and optimality conditions. Optimization 58(2):133–161

    MathSciNet  MATH  Google Scholar 

  • Still G (2001) Discretization in semi-infinite programming: the rate of convergence. Math Program 91(1):53–69

    MathSciNet  MATH  Google Scholar 

  • Zhang L, Wu S-Y, López MA (2010) A new exchange method for convex semi-infinite programming. SIAM J Optim 20(6):2959–2977

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National University of Singapore, Singapore, Singapore

    Bo Wei

  2. Krannert School of Management, Purdue University, West Lafayette, IN, USA

    William B. Haskell

  3. Sino-US Global Logistics Institute, Shanghai Jiao Tong University, Shanghai, P.R. China

    Sixiang Zhao

Authors
  1. Bo Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. William B. Haskell
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sixiang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to William B. Haskell.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, B., Haskell, W.B. & Zhao, S. An inexact primal-dual algorithm for semi-infinite programming. Math Meth Oper Res 91, 501–544 (2020). https://doi.org/10.1007/s00186-019-00698-2

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00186-019-00698-2

Keywords

Search

Navigation