Abstract
We formulate pure characteristics demand models under uncertainties of probability distributions as distributionally robust mathematical programs with stochastic complementarity constraints (DRMP-SCC). For any fixed first-stage variable and a random realization, the second-stage problem of DRMP-SCC is a monotone linear complementarity problem (LCP). To deal with ambiguity of probability distributions of the involved random variables in the stochastic LCP, we use the distributionally robust approach. Moreover, we propose an approximation problem with regularization and discretization to solve DRMP-SCC, which is a two-stage nonconvex-nonconcave minimax optimization problem. We prove the convergence of the approximation problem to DRMP-SCC regarding the optimal solution sets, optimal values and stationary points as the regularization parameter goes to zero and the sample size goes to infinity. Finally, preliminary numerical results for investigating distributional robustness of pure characteristics demand models are reported to illustrate the effectiveness and efficiency of our approaches.
Similar content being viewed by others
Notes
We can generally assume that \(P(\mathrm {d}\xi )=p(\xi )\mathbb {Q}(\mathrm {d}\xi )\) for some nominal probability distribution \(\mathbb {Q}\). We know from Radon-Nikodym theorem (see e.g. [34, Theorem 7.32]) that there exists such a density function \(p(\xi )\) if and only if P is absolutely continuous w.r.t. \(\mathbb {Q}\). Here we neglect \(\mathbb {Q}\) to simplify the notation.
References
Aumann, R.J.: Integrals of set-valued functions. J. Math. Anal. Appl. 12, 1–12 (1965)
Berry, S., Pakes, A.: The pure characteristics demand model. Internat. Econom. Rev. 48, 1193–1225 (2007)
Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer, New York (2013)
Chen, X., Sun, H., Wets, R.J.B.: Regularized mathematical programs with stochastic equilibrium constraints: Estimating structural demand models. SIAM J. Optim. 25, 53–75 (2015)
Chen, X., Xiang, S.: Perturbation bounds of P-matrix linear complementarity problems. SIAM J. Optim. 18, 1250–1265 (2008)
Chen, X., Ye, J.: A class of quadratic programs with linear complementarity constraints. Set-Valued Var. Anal. 17, 113–133 (2009)
Cottle, R.W., Pang, J.S., Stone, R.E.: The Linear Complementarity Problem. SIAM, Philadelphia (1992)
Debreu, G.: Saddle point existence theorems. Cowles Commission Discussion Paper: Mathematicas No. 412 (1952)
Delage, E., Ye, Y.: Distributionally robust optimization under moment uncertainty with application to data-driven problems. Oper. Res. 58, 595–612 (2010)
Esfahani, P.M., Kuhn, D.: Data-driven distributionally robust optimization using the Wasserstein metric: Performance guarantees and tractable reformulations. Math. Program. 171, 115–166 (2018)
Guo, L., Lin, G.H.: Notes on some constraint qualifications for mathematical programs with equilibrium constraints. J. Optim. Theory Appl. 156, 600–616 (2013)
Hanasusanto, G.A., Roitch, V., Kuhn, D., Wiesemann, W.: Ambiguous joint chance constraints under mean and dispersion information. Oper. Res. 65, 751–767 (2017)
Izmailov, A.F., Solodov, M.V.: An active-set Newton method for mathematical programs with complementarity constraints. SIAM J. Optim. 19, 1003–1027 (2008)
Jin, C., Netrapalli, P., Jordan, M.: What is local optimality in nonconvex-nonconcave minimax optimization? In: International Conference on Machine Learning, pp. 4880–4889. PMLR (2020)
Kantorovich, L.V., Rubinshtein, S.: On a space of totally additive functions. Vestnik of the St. Petersburg University: Mathematics 13, 52–59 (1958)
Lin, G.H., Chen, X., Fukushima, M.: Solving stochastic mathematical programs with equilibrium constraints via approximation and smoothing implicit programming with penalization. Math. Program. 116, 343–368 (2009)
Lin, G.H., Fukushima, M.: Stochastic equilibrium problems and stochastic mathematical programs with equilibrium constraints: A survey. Pac. J. Optim. 6, 455–482 (2010)
Liu, Y., Pichler, A., Xu, H.: Discrete approximation and quantification in distributionally robust optimization. Math. Oper. Res. 44, 19–37 (2019)
Luna, J.P., Sagastizábal, C., Solodov, M.: An approximation scheme for a class of risk-averse stochastic equilibrium problems. Math. Program. 157, 451–481 (2016)
Luo, Z.Q., Pang, J.S., Ralph, D.: Mathematical Programs with Equilibrium Constraints. Cambridge University Press, Cambridge (1996)
Mangasarian, O.L.: Error bounds for nondegenerate monotone linear complementarity problems. Math. Program. 48, 437–445 (1990)
Mangasarian, O.L.: Global error bounds for monotone affine variational inequality problems. Linear Algebra Appl. 174, 153–163 (1992)
Mangasarian, O.L., Fromovitz, S.: The Fritz John necessary optimality conditions in the presence of equality and inequality constraints. J. Math. Anal. Appl. 17, 37–47 (1967)
Mangasarian, O.L., Pang, J.S.: Exact penalty for mathematical programs with linear complementarity constraints. Optimization 42, 1–8 (1997)
Mangasarian, O.L., Ren, J.: New improved error bounds for the linear complementarity problem. Math. Program. 66, 241–255 (1994)
Mangasarian, O.L., Shiau, T.H.: Error bounds for monotone linear complementarity problems. Math. Program. 36, 81–89 (1986)
Nouiehed, M., Sanjabi, M., Huang, T., Lee, J.D., Razaviyayn, M.: Solving a class of non-convex min-max games using iterative first order methods. In: Advances in Neural Information Processing Systems, pp. 14934–14942 (2019)
Pang, J.S., Scutari, G.: Nonconvex games with side constraints. SIAM J. Optim. 21, 1491–1522 (2011)
Pang, J.S., Su, C.L., Lee, Y.C.: A constructive approach to estimating pure characteristics demand models with pricing. Oper. Res. 63, 639–659 (2015)
Pflug, G.C., Pichler, A.: Multistage Stochastic Optimization. Springer, Cham (2014)
Razaviyayn, M., Huang, T., Lu, S., Nouiehed, M., Sanjabi, M., Hong, M.: Nonconvex min-max optimization: Applications, challenges, and recent theoretical advances. IEEE Signal Process. Mag. 37, 55–66 (2020)
Rockafellar, R.T., Wets, R.J.B.: Variational Analysis. Springer-Verlag, Berlin (2009)
Scheel, H., Scholtes, S.: Mathematical programs with complementarity constraints: Stationarity, optimality, and sensitivity. Math. Oper. Res. 25, 1–22 (2000)
Shapiro, A., Dentcheva, D., Ruszczyński, A.: Lectures on Stochastic Programming: Modeling and Theory. SIAM, Philadelphia (2014)
Shapiro, A., Xu, H.: Stochastic mathematical programs with equilibrium constraints, modelling and sample average approximation. Optimization 57, 395–418 (2008)
Su, C.L., Judd, K.L.: Constrained optimization approaches to estimation of structural models. Econometrica 80, 2213–2230 (2012)
Villani, C.: Topics in Optimal Transportation. American Mathematical Soc, Providence (2003)
Xie, W.: On distributionally robust chance constrained programs with wasserstein distance. Math. Program. 186, 115–155 (2021)
Xie, W., Ahmed, S.: On deterministic reformulations of distributionally robust joint chance constrained optimization problems. SIAM J. Optim. 28, 1151–1182 (2018)
Xu, H., Liu, Y., Sun, H.: Distributionally robust optimization with matrix moment constraints: Lagrange duality and cutting plane methods. Math. Program. 169, 489–529 (2018)
Xu, H., Ye, J.: Approximating stationary points of stochastic mathematical programs with equilibrium constraints via sample averaging. Set-Valued Var. Anal. 19, 283–309 (2011)
Xu, Y., Yin, W.: A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM J. Imaging Sci. 6, 1758–1789 (2013)
Ye, J., Zhu, D., Zhu, Q.J.: Exact penalization and necessary optimality conditions for generalized bilevel programming problems. SIAM J. Optim. 7, 481–507 (1997)
Ye, Y.: On affine scaling algorithms for nonconvex quadratic programming. Math. Program. 56, 285–300 (1992)
Zhang, J., Xu, H., Zhang, L.: Quantitative stability analysis for distributionally robust optimization with moment constraints. SIAM J. Optim. 26, 1855–1882 (2016)
Acknowledgements
The authors would like to thank the editor and two anonymous referees for their very helpful comments.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This paper is dedicated to the memory of Olvi L. Mangasarian. His contributions to linear complementarity problems have impacted greatly on our research on distributionally robust mathematical programs with stochastic complementarity constraints. Jie Jiang’s work was partly supported by China Postdoctoral Science Foundation (Grant No. 2020M673117) and CAS AMSS-PolyU Joint Laboratory of Applied Mathematics. Xiaojun Chen’s work was partly supported by Hong Kong Research Grant Council PolyU15300219.
Appendix
Appendix
Proof
(The proof of Proposition 1) Denote \(\bar{p}_i=P_0(\varXi _i)\) for \(i=1,\cdots ,k\). We verify that \(\bar{p}=(\bar{p}_1,\cdots ,\bar{p}_k)^\top \in \mathcal {P}_k\) for all sufficiently large k in the following. Since \(\varPsi \) is continuous, we know from mean value theorem of integrals that
for some \(\tilde{\xi }^i\in \varXi _i\), \(i=1,\cdots ,k\). Then
We first consider the case that \(\varXi \) is bounded. For \(\alpha >0\), there exists \(\delta >0\) such that if \(\max _{1\le i\le k}\mathrm {diam}(\varXi _i) <\delta \), then
Since \(\varXi \) is bounded, we can find a sequence \(\{\xi ^k\}_{k=1}^\infty \) such that the corresponding Voronoi tessellation \(\varXi _1,\cdots ,\varXi _k,\cdots \) satisfying
Hence there is \(\bar{k}>0\) such that \(\max _{1\le i\le k}\mathrm {diam}(\varXi _i)<\delta \) for any \(k\ge \bar{k}\).
Then, it knows from (45) that
This, together with Assumption 1, indicates that \(\sum _{i=1}^k \bar{p}_i \varPsi (\xi ^i) \in \varGamma ,\) which implies the nonemptiness of \(\mathcal {P}_k\).
Now we consider the case that \(\varXi \) is unbounded. Let \(\varXi _b:=\{\xi \in \varXi :\left\| \xi \right\| \le b\}\) for \(b>0\). Denote a probability distribution \(\bar{P}_0\) supported on \(\varXi _b\) by
for any measurable \(\varXi _a\subseteq \varXi \), where \(P_0\) is defined in Assumption 1. Note that
We have
Therefore, there exists \(b_0>0\) such that, for any \(b\ge b_0\),
From Assumption 1, we obtain
Due to the boundedness of \(\varXi _b\) and (46), by the same proof for the case that \(\varXi \) is bounded, there exists a \(\bar{k}>0\) such that \(\mathcal {P}_k\) is nonempty for \(k\ge \bar{k}\). \(\square \)
Proof
(The proof of Theorem 7) Since \((x^{k*},\mathbf {y}^{k*},p^{k*})\) is an accumulation point of \((x_\epsilon ^k,\mathbf {y}_\epsilon ^k, p_\epsilon ^k)\) as \(\epsilon \downarrow 0\), there exists a sequence \(\{\epsilon _j\}_{j= 1}^\infty \) with \(\epsilon _j\downarrow 0\) as \(j\rightarrow \infty \), such that \((x_{\epsilon _j}^k,\mathbf {y}_{\epsilon _j}^k,p_{\epsilon _j}^k)\rightarrow (x^{k*},\mathbf {y}^{k*},p^{k*})\) as \(j\rightarrow \infty \). Based on (26), we have
We know from Definitions 3, 4 and Lemma 3 that
where \(\{\lambda ^{k,j}\}_{j=1}^\infty \) and \(\{\mu ^{k,j}\}_{j=1}^\infty \) are two sequences of multipliers, and
Thus, for sufficiently large j, we have
where
Next, we verify the boundedness of sequences \(\{\lambda ^{k,j}\}_{j=1}^\infty \) and \(\{\mu ^{k,j}\}_{j=1}^\infty \). Notice that if \(\{\mu ^{k,j}\}_{j=1}^\infty \) is bounded, from the boundedness of \(\{(x_{\epsilon _j}^k,\mathbf {y}_{\epsilon _j}^k,p_{\epsilon _j}^k)\}_{j=1}^\infty \) and continuous differentiability of G, \(\{\lambda ^{k,j}\}_{j=1}^\infty \) is bounded. Now we assume that \(\{\mu ^{k,j}\}_{j=1}^\infty \) is unbounded. Consider, by dividing \(\left\| \mu ^{k,j}\right\| \), that
which can deduce, according to \(\left\| \mu ^{k,j}\right\| \rightarrow \infty \) as \(j\rightarrow \infty \), that
as \(j\rightarrow \infty \). Since \(\lambda _i^{k,j} =0\) for \(i\in \mathcal {I}_{+0}^k(\mathbf {y}^{k*})\) and \(\mu _i^{k,j} =0\) for \(i\in \mathcal {I}_{0+}^k(\mathbf {y}^{k*})\), we rewrite (48) as
where \(\mathcal {I}_{00}^k(\mathbf {y}^{k*})=\{i:\mathbf {y}^{k*}_i=0, (\mathbf {M} \mathbf {y}^{k*} + \mathbf {q}(x^{k*}))_i=0\}.\)
Then, by MPEC-LICQ at \(\mathbf {y}^{k*}\) for problem (23) with \((\bar{x},\bar{p})=(x^{k*},p^{k*})\) and \(\epsilon =0\), we obtain
as \(k\rightarrow \infty \), which contradicts \(\frac{\mu _i^{k,j}}{\left\| \mu ^{k,j}\right\| }\nrightarrow 0\) for some \(i\in \mathcal {I}_{+0}^k(\mathbf {y}^{k*})\cup \mathcal {I}_{00}^k(\mathbf {y}^{k*})\). Therefore, both \(\{\lambda ^{k,j}\}_{j=1}^\infty \) and \(\{\mu ^{k,j}\}_{j=1}^\infty \) are bounded. Without loss of generality, we assume that \(\lambda ^{k,j}\rightarrow \lambda ^{k*}\) and \(\mu ^{k,j}\rightarrow \mu ^{k*}\) as \(j\rightarrow \infty \). Therefore, by letting \(j\rightarrow \infty \), we have from (47) that
Moreover, for \(\lambda _i^{k,j}\mu _i^{k,j}\ge 0\) for \(i=1,\ldots , mk\), we have \(\lambda _i^{k*}\mu _i^{k*}\ge 0\) for \(i\in \mathcal {I}_{00}^k(\mathbf {y}^{k*})\). This together with (49) means that \((x^{k*},\mathbf {y}^{k*},p^{k*})\) is a block coordinatewise C-stationary point of problem (\(\mathrm {P}_k\)). \(\square \)
Rights and permissions
About this article
Cite this article
Jiang, J., Chen, X. Pure characteristics demand models and distributionally robust mathematical programs with stochastic complementarity constraints. Math. Program. 198, 1449–1484 (2023). https://doi.org/10.1007/s10107-021-01720-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-021-01720-4
Keywords
- Distributionally robust
- Stochastic equilibrium
- Regularization
- Discrete approximation
- Pure characteristics demand