Skip to main content
SpringerLink
Log in

Solution of Nonconvex Nonsmooth Stochastic Optimization Problems

Cybernetics and Systems Analysis Aims and scope

Abstract

Different classes of nonconvex nonsmooth stochastic optimization problems are analyzed, their generalized differentiability properties and necessary optimality conditions are studied, and a technique for calculating stochastic gradients is developed. For each class of the problems, corresponding solution methods are proposed, in particular, generalizations of the stochastic quasigradient method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price includes VAT (Canada)

Instant access to the full article PDF.

Institutional subscriptions

REFERENCES

  1. J. R. Birge and F. Louveaux, Introduction to Stochastic Programming, Springer, New York (1997).

  2. Yu. M. Ermoliev, Methods of Stochastic Programming [in Russian], Nauka, Moscow (1976).

    Google Scholar 

  3. P. Kall and S.W. Wallace, Stochastic Programming, Willey, Chichester (1994).

    Google Scholar 

  4. Y. S. Kan and A. I. Kibzun, Stochastic Programming Problems with Probability and Quantile Functions, John Willey and Sons, New York (1996).

    Google Scholar 

  5. Yu. Ermoliev and R. J.-B. Wets (eds.), Numerical Techniques for Stochastic Optimization, Springer Series in Computational Mathematics, 10, Springer-Verlag, Berlin (1988).

    Google Scholar 

  6. G.Ch. Pflug, Optimization of Stochastic Models. The Interface between Simulation and Optimization, Kluwer Acad. Publ., Dordrecht (1996).

    Google Scholar 

  7. A. Prekopa, Stochastic Programming, Kluwer Acad. Publ., Dordrecht (1995).

    Google Scholar 

  8. I. V. Yudin, Information Science in Ukraine: Formation, Development, and Problems [in Ukranian], Naukova Dumka, Kiev (1999).

    Google Scholar 

  9. D. B. Yudin, Problems and Methods of Stochastic Programming [in Russian], Sov. Radio, Moscow (1979).

    Google Scholar 

  10. R. T. Rockafeller and S. Uryasev, "Conditional value-at-risk for general loss distributions," J. Bank. Fin., 26, 1443-1471 (2002).

    Google Scholar 

  11. P. S. Knopov and E. J. Kasitskaya, Empirical Estimates in Stochastic Optimization and Identification, Kluwer Acad. Publ., Dordrecht (2002).

    Google Scholar 

  12. Yu. M. Ermoliev and V. I. Norkin, "Normalized convergence in stochastic optimization," Ann. Oper. Res., 30 187-198 (1991).

    Google Scholar 

  13. V. I. Norkin, "Convergence of the empirical mean method in statistics and stochastic programming," Kibern. Sist. Analiz, 2, 107-120 (1992).

    Google Scholar 

  14. Yu. M. Ermoliev, "Stochastic quasigradient methods," in: Y. Ermoliev and R. Wets (eds.), Numerical Techniques for Stochastic Optimization, Springer-Verlag, Berlin (1988), pp. 141-185.

    Google Scholar 

  15. S. P. Uryas'ev, Adaptive Algorithms of Stochastic Optimization and Game Theory [in Russian], Nauka, Moscow (1990).

    Google Scholar 

  16. Yu. Ermoliev and A. Gaivoronski, "Stochastic programming techniques for optimization of discrete event systems," Annals Oper. Res., 39, 120-135 (1992).

    Google Scholar 

  17. Yu. M. Ermoliev, T. Yu. Ermolieva, G. MacDonald, and V. I. Norkin, "Stochastic optimization of insurance portfolios for managing exposure to catastrophic risks," Annals Oper. Res., 99, 207-225 (2000).

    Google Scholar 

  18. Yu. M. Ermoliev, T. Yu. Ermolieva, G. McDonald, and V. I. Norkin, "Problems of insurance of catastrophic risks," Kibern. Sist. Analiz, No. 2, 90-110 (2001).

  19. Yu. M. Ermoliev, V. I. Norkin, and M. A. Keyzer, "Global convergence of the stochastic tatonement process," J. Math. Econ., 34, 173-190 (2000).

    Google Scholar 

  20. Yu. M. Ermoliev and V. I. Norkin, "Nonstationary law of large numbers for dependent random variables and its application in stochastic optimization," Kibern. Sist. Analiz, No. 4, 94-106 (1998).

  21. V. I. Norkin, "Stochastic generalized-differentiable functions in the problem of nonconvex nonsmooth stochastic optimization," Kibernetika, No. 6, 98-102 (1986).

  22. V. S. Mikhalevich, A. M. Gupal, and V. I. Norkin, Methods of Nonconvex Optimization [in Russian], Nauka, Moscow (1987).

    Google Scholar 

  23. V. I. Norkin, "Stochastic Lipschitz functions," Kibernetika, No. 2, 66-71 (1986).

  24. A. M. Gupal, Stochastic Methods of Solution of Nonsmooth Extremum Problems [in Russian], Naukova Dumka, Kiev (1979).

    Google Scholar 

  25. V. Ya. Katkovnik, Linear Estimates and Stochastic Optimization Problems [in Russian], Nauka, Moscow (1976).

    Google Scholar 

  26. Yu. M. Ermoliev and V. I. Norkin, "Stochastic generalized gradient method for nonconvex nonsmooth stochastic optimization," Kibern. Sist. Analiz, No. 2, 50-71 (1998).

  27. A. Gaivoronski, "Implementation of stochastic quasigradient methods," in: Yu. Ermoliev and R. J.-B. Wets (eds.), Numerical Techniques for Stochastic Optimization, Springer-Verlag, Berlin (1988), pp. 313-352.

    Google Scholar 

  28. G.Ch. Pflug, "Step size rules, stopping times and their implementation in stochastic quasigradient algorithms," in: Yu. Ermoliev and R. J.-B. Wets (eds.), Numerical Techniques for Stochastic Optimization, Springer-Verlag, Berlin (1988), pp. 353-372.

    Google Scholar 

  29. Yu. M. Kaniovskii, P. S. Knopov, and Z. V. Nekrylova, Limiting Theorems for Processes of Stochastic Programming [in Russian], Naukova Dumka, Kiev (1980).

    Google Scholar 

  30. A. S. Nemirovskii and D. B. Yudin, Complexity of Problems and Efficiency of Optimization Methods [in Russian], Nauka, Moscow (1979).

    Google Scholar 

  31. B. T. Polyak, Introduction to Optimization [in Russian], Nauka, Moscow (1979).

    Google Scholar 

  32. R. Y. Rubinstein and A. Shapiro, The Optimization of Discrete Event Dynamic Systems by the Score Function Method, Wiley, New York (1993).

    Google Scholar 

  33. Yu. M. Ermoliev and V. I. Norkin, "On nonsmooth and discontinuous problems of stochastic systems optimization," Europ. J. Oper. Res., 101, 230-244 (1997).

    Google Scholar 

  34. F. Mirzoakhmedov, "An optimization problem in queuing theory and a numerical method," Kibernetika, No. 3, 73-75 (1990).

  35. Yu. M. Ermoliev and V. I. Norkin, "Risk and extended expected utility functions: optimization approaches," Interim Report. IR-03-033, Int. Inst. for Appl. Systems Anal., Laxenburg, Austria (2002).

    Google Scholar 

  36. V. I. Norkin and N. V. Roenko, "a-concave functions and measures and their applications," Kibern. Sist. Analiz, No. 6, 77-88 (1991).

  37. Y.G. Ho and X. R. Cao, Discrete Event Dynamic Systems and Perturbation Analysis, Kluwer, Norwell, Mass. (1991).

    Google Scholar 

  38. N. K. Krivulin, "Optimization of complex systems with the use of simulation modeling," Vest. Leningrad. Univ., No. 8, 100-102 (1990).

  39. F. Clark, Optimization and Nonsmooth Analysis [Russian translation], Nauka, Moscow (1988).

    Google Scholar 

  40. P. A. Dorofeev, "On some properties of the generalized gradient method," Zh. Vych. Mat. Mat. Fiz., 25, No. 2, 181-189 (1985).

    Google Scholar 

  41. S. K. Zavriev and A. G. Perevozchikov, "The stochastic method of the generalized gradient descent for solution of minimax problems with apparent variables," Zh. Vych. Mat. Mat. Fiz., 30, No. 4, 491-500 (1990).

    Google Scholar 

  42. E. A. Nurminskii, Numerical Methods for Solution of Stochastic Minimax Problems [in Russian], Naukova Dumka, Kiev (1979).

    Google Scholar 

  43. A. Ruszczynski, "A linearization method for nonsmooth stochastic programming problems," Math. Oper. Res., 12, No. 1, 32-49 (1987).

    Google Scholar 

  44. V. I. Norkin, "The analysis and optimization of probability functions," in: Working Paper WP — 93 6, Int. Inst. for Appl. Syst. Anal. Laxenburg, Austria (1993).

    Google Scholar 

  45. V. I. Norkin, "Nonlocal algorithms of minimization of nondifferentiable functions," Kibernetika, No. 5, 57-60 (1978).

  46. V. I. Norkin, "Generalized-differentiable functions," Kibernetika, No. 1, 9-11 (1980).

  47. R. Mifflin, "Semismooth and semiconvex functions in constrained optimization," SIAM J. Contr. Opt., 15, No. 6, 959-972 (1977).

    Google Scholar 

  48. S. Mirica, "A note on the generalized differentiability of mappings," Nonl. Anal. Theory, Methods, Appl., 4, No. 3, 567-575 (1980).

    Google Scholar 

  49. L. G. Bazhenov, "Finding the stationary points of semiregular functions," Kibern. Sist. Analiz, No. 4, 104-110 (1991).

  50. L. Qi, D. Sun, and G. Zhou, "A new look at smoothing Newton methods for nonlinear complementarity problems and box constrained variational inequalities," Mathematical Programming, 87, No. 1, 1-35 (2000).

    Google Scholar 

  51. I. I. Lyashko, V. F. Emelyanov, and O. K. Boyarchuk, Mathematical Analysis [in Ukrainian], Pt. 1, Vyshcha Shkola, Kiev (1992).

    Google Scholar 

  52. Yu. E. Nesterov, "A method for solving convex programming problem with the convergence rate O (I / k 2)," Dokl. AN SSSR, 269, No. 3, 543-547 (1983).

    Google Scholar 

  53. V. I. Norkin, "Stability of the generalized gradient descent method," Kibernetika, No. 4, 65-72 (1985).

  54. D. Q. Mayne and E. Polak, "Nondifferentiable optimization via adaptive smoothing," J. Opt. Theory Appl., 43, 601-613 (1984).

    Google Scholar 

  55. J. Varga, Optimal Control of Differential and Functional Equations [Russian translation], Nauka, Moscow (1977).

    Google Scholar 

  56. A. M. Gupal and V. I. Norkin, "Algorithm of minimization of discontinuous functions," Kibernetika, No. 2, 73-75 (1977).

  57. Yu. Ermoliev and A. Gaivoronski, "On optimization of discontinuous systems," Working Paper WP 91 49, Int. Inst. for Appl. Syst. Anal., Laxenburg (Austria) (1991).

    Google Scholar 

  58. Yu. M. Ermoliev, V. I. Norkin, and R. Wets, "The minimization of discontinuous functions: mollifier subgradients," Working Paper WP 92 73, Int. Inst. for Appl. Syst. Analysis, Laxenburg (Austria) (1992).

    Google Scholar 

  59. Yu. M. Ermoliev, V. I. Norkin, and R. J.-B. Wets, "The minimization of semi-continuous functions: Mollifier subgradients," SIAM J. Contr. and Opt., No. 1, 149-167 (1995).

  60. Yu. M. Ermoliev and V. I. Norkin, "Constrained optimization of discontinuous systems," in: K. Marti and P. Kall (eds.), Lecture Notes in Economics and Mathematical Systems, 458, Stochastic Programming and Technical Applications, Springer, Berlin (1998), pp. 128-142.

    Google Scholar 

  61. V. I. Norkin, "Stochastic methods of solving nonconvex stochastic programming problems and their application," Author's Abstract of Ph.D. Thesis, Inst. Kibern., Kiev (1998).

    Google Scholar 

  62. V. D. Batukhtin and L. A. Maiboroda, Optimization of Discontinuous Functions [in Russian], Nauka, Moscow (1984).

    Google Scholar 

  63. V. D. Batukhtin and L. A. Maiboroda, Discontinuous Extremum Problems [in Russian], Gippokrat, St. Petersburg (1995).

    Google Scholar 

  64. V. D. Batukhtin, S. I. Bigil'deev, and T. B. Bigil'deeva, "Optimization of summable functions," Kibern. Sist. Analiz, No. 3, 73-89 (2002).

  65. R.T. Rockafellar and R. J.-B. Wets, Variational Analysis, Springer-Verlag, Berlin (1998).

    Google Scholar 

  66. V. I. Norkin, Yu. M. Ermoliev, and A. Ruszczynski, "On optimal allocation of indivisibles under uncertainty," Oper. Res., 46, No. 4, 381-395 (1998).

    Google Scholar 

  67. V. I. Norkin, G. Ch. Pflug, and A. Ruszczynski, "A branch and bound method for stochastic global optimization," Math. Progr., 83, 425-450 (1998).

    Google Scholar 

  68. V. I. Norkin, "Global optimization of probabilities by the stochastic branch and bound method," in: K. Marti and P. Kall (eds.), Lecture Notes in Econ. and Mat. Systems, 458, Stochastic Progr. and Tech. Appl., Springer, Berlin (1998), pp. 186-201.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. International Institute of Applied Systems Analysis, Laxenburg, Austria

    Yu. M. Ermoliev

  2. National Academy of Sciences of Ukraine, Institute of Cybernetics, Kiev, Ukraine

    V. I. Norkin

Authors
  1. Yu. M. Ermoliev
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. V. I. Norkin
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ermoliev, Y.M., Norkin, V.I. Solution of Nonconvex Nonsmooth Stochastic Optimization Problems. Cybernetics and Systems Analysis 39, 701–715 (2003). https://doi.org/10.1023/B:CASA.0000012091.84864.65

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:CASA.0000012091.84864.65

search

Navigation