Skip to main content
SpringerLink
Log in

Computational aspects in applied stochastic control

  • Published:
Computational Economics Aims and scope Submit manuscript

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  • Akian M., 1990, Analyse de l'algorithme multigrille FMGH de résolution d'équation d'Hamilton-Jacobi-Bellman,Analysis and Optimization of Systems, Lecture Notes in Control and Information Science, 144, Springer-Verlag, 113–122

  • Akian M., J. L. Menaldi and A. Sulem, 1993, On an investment- consumption model with transaction costs, INRIA Report 1926

  • Amman, Hans, 1989, Nonlinear control on a vector machine,Parallel Computing,10, 123–127

    Google Scholar 

  • Amman Hans, and D. Kendrick, 1990, A user's guide for DUAL, a program for quadratic linear stochastic control problems, version 3.0, December, (University of Amsterdam and University of Texas at Austin)

  • Amman, Hans and David A. Kendrick, 1993, Nonconvexities in stochastic control models,Center for Economic Research, Working Paper 92-1, Revised July 1993, University of Texas, Austin.

    Google Scholar 

  • Arapostathis A., V. S. Borkar, E. Fernandez-Gaucherand, M. K. Gosh and S. I. Markus, 1993, Discrete time controlled Markov processes with average cost criterion: A survey,SIAM J. on Control and Optimization,31, pp. 282–344.

    Google Scholar 

  • Bar Ilan A. and A. Sulem, 1993, Explicit solution of inventory problem with delivery lags,Math. of Operations Research, Forth.

  • Bellman R., 1957,Dynamic Programming, Princeton University Press, Princeton, N. J.

    Google Scholar 

  • Bellman R., 1957a, A Markovian Decision Process,J. of Math. Mech. 6, 679–684.

    Google Scholar 

  • Bellman R., 1971,Introduction to the Mathematical Theory of Control Processes, New York, Academic Press.

    Google Scholar 

  • Benes V. E., L. A. Shepp and H. S. Witsenhausen, 1980, Some solvable stochastic control problems,Stochastics,4, 39–3.

    Google Scholar 

  • Bensoussan A., 1982,Stochastic Control by Functional Analysis Methods, Amsterdam, North Holland.

    Google Scholar 

  • Bensoussan A., 1988,Perturbation Methods in Optimal Control, New York, Wiley.

    Google Scholar 

  • Bensoussan A., M. Crouhi and J. M. Proth, 1983,Mathematical Theory of Production Planning, North Holland, Amsterdam.

    Google Scholar 

  • Bensoussan A., G. Hurst, B. Naslund, 1974,Management Applications of Modern Control Theory, North Holland, Amsterdam.

    Google Scholar 

  • Bensoussan A., P. Kleindorfer and C. S. Tapiero, (eds), 1978,Applied Optimal Control, North Holland, Amsterdam.

    Google Scholar 

  • Bensoussan A., P. Kleindorfer and C. S. Tapiero, 1980,Applied Stochastic Control in Econometrics and Management Science, North Holland, Amsterdam.

    Google Scholar 

  • Bensoussan A. and J. L. Lions, 1978,Application des Inéquations Variationnelles en Contrôle Stochastique, Dunod, Paris.

    Google Scholar 

  • Bensoussan A. and C.S. Tapiero, 1982, Impulsive control in management: Prospects and applications,Journal of Optimization Theory and Applications,37, 419–442.

    Google Scholar 

  • Bertsekas D. P., S. E. Shreve, 1978,Stochastic Optimal Control: The Discrete Time Case, New York, Academic Press.

    Google Scholar 

  • Bismuth J. M., 1975, Growth and intertemporal allocation of risks,Journal of Economic Theory,10, 239–257.

    Google Scholar 

  • Bismut J. M., 1978, An introductory approach to duality in optimal stochastic control,SIAM Review,20, 62–78.

    Google Scholar 

  • Black F., and M. Scholes, 1973, The pricing of options and corporate liabilites,Journal of Political Economy,81, 673–654.

    Google Scholar 

  • Blackwell D., 1962, Discrete dynamic programming,Annals of Mathematical Stat 33, 719–726.

    Google Scholar 

  • Blackwell D., 1965, Discounted dynamic programming,Annals of Mathematical Stat.,36, 226–235.

    Google Scholar 

  • Chancelier J. Ph., C. Gomez, J. P. Quadrat and A. Sulem, 1987, Automatic study in stochastic control inMathematics and its Applications, IMA, 10, Springer-Verlag, 79–86.

  • Chancelier J. Ph., Gomez, C., Quadrat J. P. and A. Sulem, 1989, An expert system for stochastic control problems: Automatic Report generation,Computer Science in Economics and Management,2, 65–82.

    Google Scholar 

  • Chow G. C., 1975,Analysis and Control of Dynamic Economic Systems, New York, Wiley.

    Google Scholar 

  • Chow G. C., 1981,Econometric Analysis by Control Methods, New York, Wiley.

    Google Scholar 

  • Constantinides G. M., 1986, Capital market equilibrium with transaction costs,Journal of Political Economy,94, no. 4, 842–862.

    Google Scholar 

  • Coomes P. A., PLEM: A computer program for passive learning, stochastic control experiments,Journal of Economic Dynamics and Control,11, 1987, 223.

    Google Scholar 

  • Crandall M. G., H. Ishii, P. L. Lions, 1992, User's guide to viscosity solutions of second order partial differential equations,Bul. Amer. Math. Soc.,27, 1–67.

    Google Scholar 

  • Cruz J. B., 1972,Feedback Systems, New York, Mc Graw Hill.

    Google Scholar 

  • Davis M., A. Norman, 1990, Portfolio selection with transaction costs,Math. of Operations Research,15, 676–713.

    Google Scholar 

  • Davis M. H. A., V. G. Panas and T. Zariphopoulou, 1993, European option pricing with transaction costs,SIAM J. on Control and Optimization,31, 470–493.

    Google Scholar 

  • Denardo E., 1970, On linear programming in a Markov decision problem,Management Science,16, 281–288.

    Google Scholar 

  • Denardo E., and B. Fox, 1968, Multichain Markov renewal programs,SIAM J. on Appl. Math.,16, 468–487.

    Google Scholar 

  • D'Epenoux F., 1960, Sur un problème de production et de stockage dans l'aléatoire,Revue Francaise de Recherche Oper.,14, 3–16.

    Google Scholar 

  • Fitzpatrick B., and W. Fleming, 1991, Numerical methods for an optimal investment consumption model,Math. of Operations Research,16, 823–841.

    Google Scholar 

  • Fleming W. H., 1968, Some problems of stochastic control, in H. Karreman (Ed.),Stochastic Optimization and Control, New York, Wiley, pp. 59–64.

    Google Scholar 

  • Fleming W. H., 1971, Control for small noises intensities,SIAM J. Control 9, no. 3.

    Google Scholar 

  • Fleming W. H., W. Rishel, 1975,Deterministic and Stochastic Optimal Control, New York, Springer Verlag.

    Google Scholar 

  • Fleming W. H. and H. M. Soner, 1993,Controlled Markov Processes and Viscosity Solutions, Springer Verlag, New York.

    Google Scholar 

  • Fleming W. H. and T. Zariphopoulou, 1991, An optimal investment consumption model with borrowing,Math. of Operations Research,16, no. 4, 802–822.

    Google Scholar 

  • Ghellinck G., de, 1960, Les problèmes de décision séquentielle,Cahiers du Centre d'Etudes de Rech. Oper.,2, 161–179.

    Google Scholar 

  • Glynn P. W. and J. L. Sanders, 1986, Monte Carlo optimization of stochastic systems: two new approaches,Mathematics Research center, University of Wisconsin, July.

  • Gomez C., Quadrat J. P., A. Sulem, 1985, Computer algebra as a tool for solving optimal control problems,Applications of Computer Algebra, Kluwer Academic Publishers.

  • Hackbush W., W. Trottenberg, (Eds), 1981,Multigrid Methods and Applications, Lecture Notes in Mathematics, 960, Springer Verlag, Berlin.

    Google Scholar 

  • Haussmann U. G., 1981a, On the adjoint process for optimal control of diffusion processes,SIAM J. Control Optimization,19, 221–2.

    Google Scholar 

  • Haussmann U. G., 1981b, Some examples of optimal stochastic controls,SIAM Rev. 23.

  • Haussmann U.. and J. P. Lepeltier, 1990, On the existence of optimal controls,SIAM J. on Control and Optimization,8, 851–902.

    Google Scholar 

  • Hernandez-Lerma O., and J. B. Lasserre, 1992, Linear programming and average optimality of markov control processes on Borel spaces-unbounded costs,LAAS Report no. 92021, March.

  • Howard R. A., 1960,Dynamic Programming and Markov Processes, Boston, MIT Press.

    Google Scholar 

  • Howard R. A., 1979, Comments on the origin and applications of Markov Decision processes, inDynamic Programming and its Applications, M. Puterman (Ed.), New York, Academic Press, 201–205.

    Google Scholar 

  • Jazwinsky H., 1970,Stochastic Processes and Filtering Theory, Academic Press, New York.

    Google Scholar 

  • Jacobson D. H. and D. Q. Mayne, 1971,Differential Dynamic Programming, New York, American Elsevier.

    Google Scholar 

  • Kalman R. E. and R. S. Bucy, 1961, New results in linear filtering and prediction theory,Trans. ASME Ser. D.J., Basic Eng'g,83, 95–108.

    Google Scholar 

  • Kamien M. I., and N. L. Schwarz, 1981,Dynamic Optimization, The Calculus of Variations and Optimal Control in Economics and Management, North Holland, New York.

    Google Scholar 

  • Kendrick David A. and P. Coomes, 1984,DUAL: A Program for quadratic linear stochastic control problems, Discussion paper 84-15, Center for Economic Research, University of Texas, Austin.

    Google Scholar 

  • Kiefer J. and J. Wolfowitz, 1952, Stochastic estimation of the maximum of a regression function,Annals of Math. Stat.,23, 462–66.

    Google Scholar 

  • Krylov N. V., 1980,Controlled Diffusion Processes, New York, Springer Verlag.

    Google Scholar 

  • Kushner H. J., 1971,Introduction to Stochastic Control, New York, Holt Rinehart and Whinston.

    Google Scholar 

  • Kushner H., 1977,Probabilistic Methods in Stochastic Control and for Elliptic Equations, New York, Academic Press.

    Google Scholar 

  • Kushner H. J., 1990, Numerical methods for stochastic control problems in continuous time,SIAM J. on Control and Optimization,28, 999–1948.

    Google Scholar 

  • Kushner H. J. and P. G. Dupuis, 1993,Numerical Methods for Stochastic Control Problems in Continuous Time, Berlin, Springer Verlag.

    Google Scholar 

  • Leve G. de, 1964a,Generalized Markovian decision processes, part I: Model and Method, Math. Centre Tract 3, Amsterdam.

  • Leve G., de, 1964b,Generalized Markovian decision processes, part II: Probabilistic background, Math. Centre Tract4, Amsterdam.

  • Machina M., 1882, Expected utility analysis without the independence axiom,Econometrica,50, 277–323.

    Google Scholar 

  • MacRae E. C., 1975, An adaptive learning rule for multiperiod decision problems,Econometrica,43, 893.

    Google Scholar 

  • Magill M. J. P., G. M. Constantinides 1976, Portfolio selection with transaction costs,Journal of Economic Theory,13, 245–263.

    Google Scholar 

  • Malliaris A. G. and W. A. Brock, 1982,Stochastic Methods in Economics and Finance, North Holland, Amsterdam.

    Google Scholar 

  • Manne A., 1960, Linear Programming and sequential decisions,Management Science,6, 259–267.

    Google Scholar 

  • Markowitz H. M.,Portfolio Selection. Efficient Diversification of Investments Cowles Foundation Monographs, Yale University Press, New Haven, 1959.

    Google Scholar 

  • Martins L. H., and H. F. Kushner, 1991, Numerical methods for stochastic singularly controlled problems,SIAM J. on Control and Optimization,29, pp. 1443–1475.

    Google Scholar 

  • Matulka J. and R. Neck, OPTCON: An algorithm for the optimal control of nonlinear stochastic models,Annals of Operations Research,37, 1992, 375.

    Google Scholar 

  • McCormick S. F. (Ed.), 1987,Multigrid Methods, SIAM Frontier in Applied Mathematics, vol. 5.

  • Merton R. C., 1971, Optimum consumption and portfolio rules in a continuous time model,J. of Economic Theory,3, 373–413.

    Google Scholar 

  • Neck R. and J. Matulka, 1991, Stochastic optimum control of macroeconomic models using the algorithm OPTCON,Working Paper, Department of Economics, University of Vienna, Austria.

    Google Scholar 

  • Neck R. and J. Matulka, Stochastic control of nonlinear econometric models,Discussion Paper no. 260, Department of Economics, University of Bielefeld, Bielfeld, Germany, 1992.

    Google Scholar 

  • Newman T. G. and P. L. Odell, 1971,The Generation of Random Variates, Griffin's Stat. Monog. New York, Hafner Publ. Co.

    Google Scholar 

  • Norman A. L., First order dual controls,Annals of Economic and Social Measurement,5, 1976, 311.

    Google Scholar 

  • Pekelman D and G. Rausser, 1978, Adaptive control survey: methods and applications, in A. Bensoussan, P. Kleindorfer and C. S. Tapiero, (Eds.),Applied Optimal Control, North Holland, Amsterdam.

    Google Scholar 

  • Pindyck R. S., 1973,Optimal Planning for Economic Stabilization, North Holland, Amsterdam.

    Google Scholar 

  • Quadrat J. P., 1975,Analyse Numérique de L'équation de Bellman Stochastique, INRIA Report, No. 140, Inria, Le Chesnay, France.

    Google Scholar 

  • Quadrat J. P., 1980, Existence de solutions et algorithme de résolution numérique de problèmes stochastiques dégénérées ou non,SIAM J. on Control,18, 199–6.

    Google Scholar 

  • Quadrat J. P., 1989, Optimal stochastic control: numerical methods, inSystems and Control Encyclopedia, Editor in Chief, M.G. Singh, Pergamon Press.

  • Quadrat J. P. and Viot, 1980, Product form and optimal feedback for multi index Markov chains,Allerton Conf. in Optimal Control.

  • Quiggin J., 1982, A theory of anticipated utility,Journal of Economic Behavior and Organization,3, 324–343.

    Google Scholar 

  • Robbins, S. Monro, 1951, A Stochastic approximation method,Annals of Math. Stat., 400–407.

  • Rockafellar R. T. and R. J-B. Wets, 1987, Scenarios and policy aggregation in optimization under uncertainty,Mathematics of Operations Research, IIASA, WP-87, 119, Laxenburg, Austria, Dec. 1987.

  • Rockafellar R. T. and R. J-B. Wets, 1990, Generalized linear quadratic problems of deterministic and stochastic optimal control in discrete time,SIAM J. on Control and Optimization,28, 810–822.

    Google Scholar 

  • Rockafellar R. T. and R. J-B. Wets, 1992, A dual strategy for the implementation of the aggregation principle in decision making under uncertainty,Applied Stochastic Models and Data Analysis,8, 245–255.

    Google Scholar 

  • Ross S., 1983,Introduction to Stochastic Dynamic Programming, New York, Academic Press.

    Google Scholar 

  • Rubenstein R. Y., 1981,Stimulation and the Monte Carlo Methods, Wiley, New York.

    Google Scholar 

  • Sethi S. P. and G. L. Thompson, 1981,Optimal Control Theory Applications to Management Science, Nijhoff, Boston.

    Google Scholar 

  • Shapeley L. S., 1953, Stochastic games,Proc. Nat. Acad. of Sciences,39, 1095–1100.

    Google Scholar 

  • Shreve S. E., H. M. Soner and V. Xu, 1991, Optimal investment and consumption with two bonds and transaction costs,Mathematical Finance,1, no. 3, 58–34.

    Google Scholar 

  • Simon H. A. 1956, Dynamic programming under uncertainty with a quadratic criterion function,Econometrica,24, 74–81.

    Google Scholar 

  • Singh M. G., 1989, (Editor in Chief),Systems and Control Encyclopedia, New York, Pergamon Press.

    Google Scholar 

  • Soner H. M., 1986, Optimal control with state space constraints,SIAM J. on Control and Optimization,24, 552–561.

    Google Scholar 

  • Sulem A., 1986, A solvable one dimensional model of a diffusion inventory system,Math. of Operations Research,11, 125–133.

    Google Scholar 

  • Sulem A., 1986, Explicit solution of a two dimensional deterministic inventory problem,Math. of Operations Research,11, 134–146.

    Google Scholar 

  • Sulem A. and C. S. Tapiero, Supply delays and the inventory stocking policy,International Journal of Production Economics, 1994.

  • Taksar M., M. J. Klass and D. Assaf, 1988, A diffusion model for optimal portfolio selection in the presence of brokerage fees,Math. of Operations Research,13, 277–294.

    Google Scholar 

  • Tapiero C. S., 1977,Managerial Planning: An Optimum and Stochastic Control Approach, New York, Gordon Breach.

    Google Scholar 

  • Tapiero C. S., 1988,Applied Stochastic Models and Control in Management, Amsterdam, North Holland.

    Google Scholar 

  • Tapiero C. S., Stochastic Control: From Theory to Practice,Euro. J. of Operations Research, 1994, Forth.

  • Theil H., 1957, A note on certainty equivalence in dynamic programming,Econometrica,25, 346–349.

    Google Scholar 

  • Tse E., and Y. Bar-Shalom, 1973, An actively adaptive control for linear systems with random parameters,IEEE Trans. on Automatic Control, AC-18, 38–52, February.

    Google Scholar 

  • Van der Wal J., and J. Wessels, 1985, Markov decision processes,Statistica Neerlandica,39, 1985, 219–233.

    Google Scholar 

  • Varaiya P. and R. B. Wets, Stochastic dynamic optimizations: Approaches and Computation, inMathematical Programming: Recent Developments and Applications, Editors M. Iri and K. Tanabe, Kluwer Academic Publishers, 1989, 309–332.

  • Vemuri V. and W. J. Karplus, 1981,Digital Computer Treatment of Partial Differential Equations, Englewood Cliffs, N.J., Prentice Hall.

    Google Scholar 

  • Vial J. P., Computational experience with a primal-dual interior-point method for smooth convex programming,Cahiers de Recherche, Université de Genève, August, 1992.

  • Wets R. J. B., 1989, The aggregation principle in scenarios analysis and stochastic optimization, in S. Wallace,Algorithms and Model Formulations in Mathematical Programming, Springer-Verlag, NATO, ASI, vol. 51, Berlin, 91–113.

    Google Scholar 

  • Whittle P., 1980, Multi-Armed bandits and the Gittins index,J. R. Statist. Soc.,42, 143–149.

    Google Scholar 

  • Whittle P., 1982,Optimization Over Time, vol. 1, John Wiley.

  • Wiener N., 1949,Extrapolation, Interpolation and Smoothing of Stationary Time Series, New York, Wiley.

    Google Scholar 

  • Witsenhausen H. S., 1971, Separation of estimation and control for discrete time systems,Proceedings of the IEEE,59, 1557–1566.

    Google Scholar 

  • Wonham W. M., 1970, Random differential equations in control theory, in A. T. Barucha Reid (Ed.),Probabilistic Methods in Applied Mathematics, vol 2, Academic Press, New York, 131–212.

    Google Scholar 

  • Yang, and H. J. Kushner, 1991, A Monte Carlo method for sensitivity analysis of parametric optimization of nonlinear stochastic systems,SIAM J. on Control, 216–1249.

Download references

Author information

Authors and Affiliations

  1. Departments of Logistics, Information Systems and Decision Sciences, ESSEC, B. P. 105, 95021, Cergy Pontoise

    Charles S. Tapiero

  2. INRIA, Domaine de Voluceau, Rocquencourt, B. P. 105, 78153, Le Chesnay, France

    Agnès Sulem

Authors
  1. Charles S. Tapiero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Agnès Sulem
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tapiero, C.S., Sulem, A. Computational aspects in applied stochastic control. Comput Econ 7, 109–146 (1994). https://doi.org/10.1007/BF01299570

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01299570

Keywords

Search

Navigation