Mathematical Programming

, Volume 174, Issue 1–2, pp 225–251 | Cite as

Statistics with set-valued functions: applications to inverse approximate optimization

  • Anil AswaniEmail author
Full Length Paper Series B


Much of statistics relies upon four key elements: a law of large numbers, a calculus to operationalize stochastic convergence, a central limit theorem, and a framework for constructing local approximations. These elements are well-understood for objects in a vector space (e.g., points or functions); however, much statistical theory does not directly translate to sets because they do not form a vector space. Building on probability theory for random sets, this paper uses variational analysis to develop operational tools for statistics with set-valued functions. These tools are first applied to nonparametric estimation (kernel regression of set-valued functions). The second application is to the problem of inverse approximate optimization, in which approximate solutions (corrupted by noise) to an optimization problem are observed and then used to estimate the amount of suboptimality of the solutions and the parameters of the optimization problem that generated the solutions. We show that previous approaches to this problem are statistically inconsistent when the data is corrupted by noise, whereas our approach is consistent under mild conditions.


Set-valued functions Statistics Inverse optimization 

Mathematics Subject Classification

62F12 62G08 49J53 


  1. 1.
    Ahuja, R., Orlin, J.: Inverse optimization. Oper. Res. 49(5), 771–783 (2001)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Allende, G., Still, G.: Solving bilevel programs with the KKT-approach. Math. Program. 138(1–2), 309 (2013)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Artstein, Z., Vitale, R.: A strong law of large numbers for random compact sets. Ann. Probab. 3(5), 879–882 (1975)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Aswani, A., Bickel, P., Tomlin, C.: Regression on manifolds: estimation of the exterior derivative. Ann. Stat. 39(1), 48–81 (2011)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Aswani, A., Gonzalez, H., Sastry, S., Tomlin, C.: Provably safe and robust learning-based model predictive control. Automatica 49(5), 1216–1226 (2013)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Aswani, A., Kaminsky, P., Mintz, Y., Flowers, E., Fukuoka, Y.: Behavioral modeling in weight loss interventions. Available at SSRN: Accessed 22 Feb 2017 (2016)
  7. 7.
    Aswani, A., Shen, Z.J., Siddiq, A.: Inverse optimization with noisy data. Oper. Res. (2017) (to appear) Google Scholar
  8. 8.
    Aumann, R.J.: Integrals of set-valued functions. J. Math. Anal. Appl. 12(1), 1–12 (1965)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Bahadur, R.R.: Some Limit Theorems in Statistics. SIAM, Philadelphia (1971)zbMATHGoogle Scholar
  10. 10.
    Berge, C.: Topological Spaces: Including a Treatment of Multi-valued Functions, Vector Spaces, and Convexity. Courier Corporation, North Chelmsford (1963)zbMATHGoogle Scholar
  11. 11.
    Bertsimas, D., Gupta, V., Paschalidis, I.: Data-driven estimation in equilibrium using inverse optimization. Math. Program. 153(2), 595–633 (2015)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Bickel, P.: On adaptive estimation. Ann. Stat. 10(3), 647–671 (1982)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Bickel, P., Doksum, K.: Mathematical Statistics: Basic Ideas And Selected Topics, vol. 1, 2nd edn. Pearson Prentice Hall, Upper Saddle River (2006)zbMATHGoogle Scholar
  14. 14.
    Boucheron, S., Lugosi, G., Massart, P.: Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, Oxford (2013)zbMATHGoogle Scholar
  15. 15.
    Castaing, C.: Sur les multi-applications mesurables. Revue française d’informatique et de recherche opérationnelle 1(1), 91–126 (1967)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Chan, T., Craig, T., Lee, T., Sharpe, M.: Generalized inverse multiobjective optimization with application to cancer therapy. Oper. Res. 62(3), 680–695 (2014)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Dempe, S., Dutta, J.: Is bilevel programming a special case of a mathematical program with complementarity constraints? Math. Program. 131(1–2), 37–48 (2012)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Devroye, L., Wise, G.: Detection of abnormal behavior via nonparametric estimation of the support. SIAM J. Appl. Math. 38(3), 480–488 (1980)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Dupacová, J., Wets, R.: Asymptotic behavior of statistical estimators and of optimal solutions of stochastic optimization problems. Ann. Stat. 16(4), 1517–1549 (1988)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Esfahani, P.M., Shafieezadeh-Abadeh, S., Hanasusanto, G.A., Kuhn, D.: Data-driven inverse optimization with incomplete information. arXiv preprint arXiv:1512.05489 (2015)
  21. 21.
    Geffroy, J.: Sur un probleme destimation géométrique. Publ. Inst. Statist. Univ. Paris 13, 191–210 (1964)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Geyer, C.J.: On the asymptotics of constrained M-estimation. Ann. Stat. 22(4), 1993–2010 (1994)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Guntuboyina, A.: Optimal rates of convergence for convex set estimation from support functions. Ann. Stat. 40(1), 385–411 (2012)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Keshavarz, A., Wang, Y., Boyd, S.: Imputing a convex objective function. In: IEEE Multi-Conference on Systems and Control, pp. 613–619 (2011)Google Scholar
  25. 25.
    Knight, K., Fu, W.: Asymptotics for lasso-type estimators. Ann. Stat. 28(5), 1356–1378 (2000)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Korostelëv, A., Simar, L., Tsybakov, A.: Efficient estimation of monotone boundaries. Ann. Stat. 476–489 (1995)Google Scholar
  27. 27.
    Kudo, H.: Dependent experiments and sufficient statistics, vol. 4, pp. 905–927. Natural Science Report. Ochanomizu University (1953)Google Scholar
  28. 28.
    Lachout, P., Liebscher, E., Vogel, S.: Strong convergence of estimators as \(\varepsilon \)n-minimisers of optimisation problemsof optimisation problems. Ann. Inst. Stat. Math. 57(2), 291–313 (2005)zbMATHGoogle Scholar
  29. 29.
    Lovász, L., Vempala, S.: Hit-and-run from a corner. SIAM J. Comput. 35(4), 985–1005 (2006)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Matheron, G.: Random Sets and Integral Geometry. Wiley, Hoboken (1975)zbMATHGoogle Scholar
  31. 31.
    Molchanov, I.: Theory of Random Sets. Springer, Berlin (2006)Google Scholar
  32. 32.
    Nesterov, Y., Nemirovskii, A.: Interior-Point Polynomial Algorithms in Convex Programming. SIAM, Philadelphia (1994)zbMATHGoogle Scholar
  33. 33.
    Noda, K.: Estimation of a regression function by the Parzen kernel-type density estimators. Ann. Inst. Stat. Math. 28(1), 221–234 (1976)MathSciNetzbMATHGoogle Scholar
  34. 34.
    Ouattara, A., Aswani, A.: Duality approach to bilevel programs with a convex lower level. arXiv preprint arXiv:1608.03260 (2016)
  35. 35.
    Outrata, J.: On the numerical solution of a class of Stackelberg problems. Z. Oper. Res. 34(4), 255–277 (1990)MathSciNetzbMATHGoogle Scholar
  36. 36.
    Patschkowski, T., Rohde, A., et al.: Adaptation to lowest density regions with application to support recovery. Ann. Stat. 44(1), 255–287 (2016)MathSciNetzbMATHGoogle Scholar
  37. 37.
    Rényi, A., Sulanke, R.: Über die konvexe hülle von n zufällig gewählten punkten. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 2(1), 75–84 (1963)zbMATHGoogle Scholar
  38. 38.
    Rockafellar, R., Wets, R.: Variational Analysis, 3rd edn. Springer, Berlin (2009)zbMATHGoogle Scholar
  39. 39.
    Royset, J.O.: Approximations and solution estimates in optimization. Math. Program. (2017)Google Scholar
  40. 40.
    Royset, J.O., Wets, R.: Multivariate epi-splines and evolving function identification problems. Set Valued Var. Anal. 24(4), 517–545 (2016)MathSciNetzbMATHGoogle Scholar
  41. 41.
    Royset, J.O., Wets, R.J.B.: Variational theory for optimization under stochastic ambiguity. SIAM J. Optim. 27(2), 1118–1149 (2017)MathSciNetzbMATHGoogle Scholar
  42. 42.
    Salinetti, G., Wets, R.: On the convergence in distribution of measurable multifunctions (random sets) normal integrands, stochastic processes and stochastic infima. Math. Oper. Res. 11(3), 385–419 (1986)MathSciNetzbMATHGoogle Scholar
  43. 43.
    Schneider, R.: Convex Bodies: The Brunn–Minkowski Theory 151. Cambridge University Press, Cambridge (1993)zbMATHGoogle Scholar
  44. 44.
    Schölkopf, B., Platt, J., Shawe-Taylor, J., Smola, A., Williamson, R.: Estimating the support of a high-dimensional distribution. Neural Comput. 13(7), 1443–1471 (2001)zbMATHGoogle Scholar
  45. 45.
    Smith, R.: Efficient Monte Carlo procedures for generating points uniformly distributed over bounded regions. Oper. Res. 32(6), 1296–1308 (1984)MathSciNetzbMATHGoogle Scholar
  46. 46.
    van der Vaart, A.: Asymptotic Statistics. Cambridge University Press, Cambridge (2000)Google Scholar
  47. 47.
    Wald, A.: Note on the consistency of the maximum likelihood estimate. Ann. Math. Stat. 20(4), 595–601 (1949)MathSciNetzbMATHGoogle Scholar
  48. 48.
    Wand, M., Jones, M.: Kernel Smoothing. Taylor & Francis, Abingdon (1994)zbMATHGoogle Scholar
  49. 49.
    Weil, W.: An application of the central limit theorem for Banach-space-valued random variables to the theory of random sets. Probab. Theory Relat. Fields 60(2), 203–208 (1982)MathSciNetzbMATHGoogle Scholar
  50. 50.
    Ye, J., Zhu, D.: Optimality conditions for bilevel programming problems. Optimization 33(1), 9–27 (1995)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society 2018

Authors and Affiliations

  1. 1.Industrial Engineering and Operations ResearchUniversity of CaliforniaBerkeleyUSA

Personalised recommendations