The authors discuss connections between the problems of two-stage stochastic programming, robust decision-making, robust statistical estimation, and machine learning. In the conditions of uncertainty, possible extreme events and outliers, these problems require quantile-based criteria, constraints, and “goodness-of-fit” indicators. The two-stage stochastic optimization (STO) problems with quantile-based criteria can be effectively solved with the iterative stochastic quasigradient (SQG) solution algorithms. The SQG methods provide a new type of machine learning algorithms that can be effectively used for general-type nonsmooth, possibly discontinuous, and nonconvex problems, including quantile regression and neural network training. In general problems of decision-making, feasible solutions and concepts of optimality and robustness are characterized from the context of decision-making situations. Robust machine learning (ML) approaches can be integrated with disciplinary or interdisciplinary decision-making models, e.g., land use, agricultural, energy, etc., for robust decision-making in the conditions of uncertainty, increasing systemic interdependencies, and “unknown risks.”
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Yu. Ermoliev and R. J.-B. Wets (eds.), Numerical Techniques for Stochastic Optimization, Springer-Verlag, Heidelberg (1988).
Y. Ermoliev and L. Hordijk, “Global changes: Facets of robust decisions,” in: K. Marti, Y. Ermoliev, M. Makowski, and G. Pflug (eds.), Coping with Uncertainty: Modeling and Policy Issue, Springer-Verlag, Berlin (2003), pp. 4–28.
Y. Ermoliev, Methods of Stochastic Programming [in Russian], Nauka, Moscow (1976).
Y. Ermoliev, “Stochastic quasigradient methods,” in: P. M. Pardalos (ed.), Encyclopedia of Optimization, Springer-Verlag, New York (2009), pp. 3801–3807.
Y. Ermoliev, “Two-stage stochastic programming: Quasigradient method,” in: P. M. Pardalos (ed.), Encyclopedia of Optimization, Springer-Verlag, New York (2009), pp. 3955–3959.
Y. Ermoliev, “Stochastic quasigradient methods in minimax problems,” in: P. M. Pardalos (ed.), Encyclopedia of Optimization, Springer-Verlag, New York (2009), pp. 3813–3818.
Y. Ermoliev and A. Gaivoronski, “Stochastic quasigradient methods for optimization of discrete event systems,” Annals of Oper. Research, Vol. 39, Iss. 1, 1–39 (1992).
Y. Ermoliev, A. G. Zagorodny, V. L. Bogdanov, T. Ermolieva, P. Havlik, E. Rovenskaya, N. Komendantova, and M. Obersteiner, “Robust food–energy–water–environmental security management: Stochastic quasigradient procedure for linkage of distributed optimization models under asymmetric information and uncertainty,” Cybern. Syst. Analysis, Vol. 58, No. 1, 45–57 (2022). https://doi.org/10.1007/s10559-022-00434-5.
Y. Ermoliev and D. von Winterfeldt, “Systemic risk and security management,” in: Y. Ermoliev, M. Makowski, and K. Marti (eds.), Managing Safety of Heterogeneous Systems, Lecture Notes in Economics and Mathematical Systems, Vol. 658 (2012), pp. 19–49.
T. Ermolieva, P. Havlik, S. Fran, T. Kahi, J. Balkovi, Í. R. Skalsk, Y. Ermoliev, P. S. Knopov, O. M. Borodina, and V. M. Gorbachuk, “A risk-informed decision-making framework for climate change adaptation through robust land use and irrigation planning,” Sustainability, Vol. 14, Iss. 3, 1430 (2022). https://doi.org/10.3390/su14031430.
T. Ermolieva, P. Havl_k, Y. Ermoliev, A. Mosnier, M. Obersteiner, D. Leclere, N. Khabarov, H. Valin, and W. Reuter, “Integrated management of land use systems under systemic risks and security targets: A stochastic global biosphere management model,” J. of Agricultural Economics, Vol. 67, Iss. 3, 584–601 (2016).
O. M. Borodina, S. V. Kyryziuk, O. V. Fraier, Y. M. Ermoliev, T. Y. Ermolieva, P. S. Knopov, and V. M. Horbachuk, “Mathematical modeling of agricultural crop diversification in Ukraine: Scientific approaches and empirical results,” Cybern. Syst. Analysis, Vol. 56, No. 2, 213–222 (2020). https://doi.org/10.1007/s10559-020-00237-6.
J. Gao, X. Xu, G.-Y. Cao, Y. Ermoliev, T. Ermolieva, and E. Rovenskaya, “Strategic decision-support modeling for robust management of the food–energy–water nexus under uncertainty,” J. of Cleaner Production, Vol. 292, 125995 (2021). https://doi.org/10.1016/j.jclepro.2021.125995.
J. P. Ortiz-Partida, T. Kahil, T. Ermolieva, Y. Ermoliev, B. Lane, S. Sandoval-Solis, and Y. Wada, “Atwo-stage stochastic optimization for robust operation of multipurpose reservoirs,” Water Resources Management, Vol. 33, Iss. 11, 3815–3830 (2019). https://doi.org/10.1007/s11269-019-02337-1.
M. Ren, X. Xu, T. Ermolieva, G.-Y. Cao, and Y. Yermoliev, “The optimal technological development path to reduce pollution and restructure iron and steel industry for sustainable transition,” Intern. J. of Sci. and Engineering Investigations, Vol. 7, Iss. 73, 100–105 (2018).
P. Huber, Robust Statistics, John Wiley & Sons, New York–Chichester–Brisbane–Toronto–Singapore (1981).
V. N. Vapnik, The Nature of Statistical Learning Theory, Springer-Verlag, New York (1995).
P. S. Knopov and E. J. Kasitskaya, Empirical Estimates in Stochastic Optimization and Identification, Springer-Verlag, Berlin (2002).
T. Ermolieva, Y. Ermoliev, M. Obersteiner, and E. Rovenskaya, “Two-stage nonsmooth stochastic optimization and iterative stochastic quasigradient procedure for robust estimation, machine learning and decision making,” in: F. S. Roberts and I. A. Sheremet (eds.), Resilience in the Digital Age, Ch. 4, Springer, Cham (2021), pp. 45–74.
Y. Nesterov, Introductory Lectures on Convex Optimization, XVIII, Springer New York, New York (2004).
S. J. Reddi, A. Hefny, S. Sra, B. Poczos, and A. J. Smola, “On variance reduction in stochastic gradient descent and its asynchronous variants,” in: C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett (eds.), Advances in Neural Information Processing Systems 28, Proc. Annual Conf. on Neural Information Processing Systems 2015 (7–12 December 2015, Montreal, Canada), Montreal (2015), pp. 2647–2655.
H. Robbins and S. Monro, “A stochastic approximation method,” The Annals of Mathematical Statistics, Vol. 22, Iss. 3, 400–407 (1951).
B. T. Polyak and A. B. Juditsky, “Acceleration of stochastic approximation by averaging,” SIAM J. on Control and Optimization, Vol. 30, Iss. 4, 838–855 (1992).
F. H. Clarke, Optimization and Nonsmooth Analysis, XIII, John Wiley & Sons New York (1983).
T. Rockafeller, The Theory of Subgradient and its Application to Problems of Optimization: Convex and Nonconvex Functions, Heldermann Verlag, Berlin (1981).
Y. Ermoliev and N. Shor, “On minimization of nondiferentiable functions,” Kibernetika, Vol. 3, No. 1, 101–102 (1967).
Y. Ermoliev and V. Norkin, “On nonsmooth and discontinuous problems of stochastic systems optimization,” Europ. J. Oper. Res., Vol. 101, Iss. 2, 230–244 (1997).
A. Gaivoronski, “Convergence properties of backpropagation for neural nets via theory of stochastic quasigradient methods: Part 1,” Optimization Methods and Software, Vol. 4, Iss. 2, 117–134 (1994).
V. M. Gorbachuk, Y. Ermoliev, T. Ermolieva, and M. S. Dunajevskij, “Quantile-based regression for the assessment of economic and ecological risks,” Proc. 5th Intern. Sci. Conf. on Computational Intelligence (15–20 April 2019, Uzgorod, Ukraine), Uzgorod (2019), pp. 188–190.
Author information
Authors and Affiliations
Corresponding author
Additional information
*The development of robust decision-making, statistical estimation, machine learning, and Big Data analysis problems, respective solution procedures, and case studies, is supported by the joint project between the International Institute for Applied Systems Analysis (IIASA) and National Academy of Sciences of Ukraine (NASU) on “Integrated robust modeling and management of food-energy-water-land use nexus for sustainable development.” The work has received partial support from the Ukrainian National Foundation for Strategic Research, grant No. 2020.02/0121, and project CPEA-LT-2016/10003 jointly with Norwegian University for Science and Technology. The paper contributes to EU PARATUS (CL3-2021-DRS-01-03, SEP-210784020) project on “Promoting disaster preparedness and resilience by co-developing stakeholder support tools for managing systemic risk of compounding disasters.
Translated from Kibernetyka ta Systemnyi Analiz, No. 3, May–June, 2023, pp. 33–47.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ermolieva, T., Ermoliev, Y., Havlik, P. et al. Connections between Robust Statistical Estimation, Robust Decision-Making with Two-Stage Stochastic Optimization, and Robust Machine Learning Problems. Cybern Syst Anal 59, 385–397 (2023). https://doi.org/10.1007/s10559-023-00573-3
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10559-023-00573-3