Abstract
We consider a continuous-time model for inventory management with Markov modulated non-stationary demands. We introduce active learning by assuming that the state of the world is unobserved and must be inferred by the manager. We also assume that demands are observed only when they are completely met. We first derive the explicit filtering equations and pass to an equivalent fully observed impulse control problem in terms of the sufficient statistics, the a posteriori probability process and the current inventory level. We then solve this equivalent formulation and directly characterize an optimal inventory policy. We also describe a computational procedure to calculate the value function and the optimal policy and present two numerical illustrations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Allam, S., Dufour, F., & Bertrand, P. (2001). Discrete-time estimation of a Markov chain with marked point process observations. Application to Markovian jump filtering. IEEE Transactions on Automatic Control, 46(6), 903–908. ISSN 0018-9286.
Arjas, E., Haara, P., & Norros, I. (1992). Filtering the histories of a partially observed marked point process. Stochastic Processes and their Applications, 40(2), 225–250. ISSN 0304-4149.
Aviv, Y., & Pazgal, A. (2005). A partially observed Markov decision process for dynamic pricing. Management Science, 51(9), 1400–1416. ISSN 0025-1909.
Azoury, K. S. (1985). Bayes solution to dynamic inventory models under unknown demand distribution. Management Science, 31(9), 1150–1160. ISSN 0025-1909.
Bayraktar, E., & Ludkovski, M. (2009, accepted). Sequential tracking of a hidden Markov chain using point process observations. Stochastic Processes and Their Applications. arXiv:0712.0413.
Bayraktar, E., & Sezer, S. (2006). Quickest detection for a Poisson process with a phase-type change-time distribution (Technical Report). University of Michigan. arXiv:math/0611563.
Bensoussan, A., Çakanyildirim, M., & Sethi, S. P. (2005a). On the optimal control of partially observed inventory systems. Comptes Rendus Mathematique. Academie Des Sciences. Paris, 341(7), 419–426. ISSN 1631-073X.
Bensoussan, A., Liu, R. H., & Sethi, S. P. (2005b). Optimality of an (s,S) policy with compound Poisson and diffusion demands: a quasi-variational inequalities approach. SIAM Journal on Control and Optimization, 44(5), 1650–1676. ISSN 0363-0129 (electronic).
Bensoussan, A., Çakanyildirim, M., & Sethi, S. P. (2007a). A multiperiod newsvendor problem with partially observed demand. Mathematics of Operations Research, 32(2), 322–344. ISSN 0364-765X.
Bensoussan, A., Çakanyildirim, M., & Sethi, S. P. (2007b). Partially observed inventory systems: The case of zero-balance walk. SIAM Journal on Control and Optimization, 46(1), 176–209. ISSN 0363-0129.
Bensoussan, A., Çakanyildirim, M., Minjarez-Sosa, J., Royal, A., & Sethi, S. (2008). Inventory problems with partially observed demands and lost sales. Journal of Optimization Theory and Applications, 136(3), 321–340.
Bertsekas, D. P. (2005). Dynamic programming and optimal control (Vol. I, 3rd ed.). Belmont: Athena Scientific. ISBN 1-886529-26-4.
Beyer, D., & Sethi, S. P. (2005). Average cost optimality in inventory models with Markovian demands and lost sales. In GERAD 25th Anniv. Ser. : Vol. 4. Analysis, control and optimization of complex dynamic systems (pp. 3–23). New York: Springer.
Bremaud, P. (1981). Point processes and queues. New York: Springer.
Costa, O. L. V., & Davis, M. H. A. (1989). Impulse control of piecewise-deterministic processes. Mathematics of Control, Signals and Systems, 2(3), 187–206. ISSN 0932-4194.
Darroch, J. N., & Morris, K. W. (1968). Passage-time generating functions for continuous-time finite Markov chains. Journal of Applied Probability, 5(2), 414–426.
Davis, M. H. A. (1993). Markov models and optimization. London: Chapman & Hall.
Elliott, R. J., & Malcolm, W. P. (2004). Robust M-ary detection filters and smoothers for continuous-time jump Markov systems. IEEE Transactions on Automatic Control, 49(7), 1046–1055. ISSN 0018-9286.
Elliott, R. J., & Malcolm, W. P. (2005). General smoothing formulas for Markov-modulated Poisson observations. IEEE Transactions on Automatic Control, 50(8), 1123–1134. ISSN 0018-9286.
Elliott, R. J., Aggoun, L., & Moore, J. B. (1995). Estimation and control. In Applications of mathematics (New York) : Vol. 29. Hidden Markov models. New York: Springer. ISBN 0-387-94364-1.
Gatarek, D. (1992). Optimality conditions for impulsive control of piecewise-deterministic processes. Mathematics of Control, Signals and Systems, 5(2), 217–232. ISSN 0932-4194.
Hernández-Lerma, O. (1989). Applied mathematical sciences : Vol. 79. Adaptive Markov control processes. New York: Springer. ISBN 0-387-96966-7.
Karlin, S., & Taylor, H. M. (1981). A second course in stochastic processes. New York: Academic Press. ISBN 0-12-398650-8.
Lariviere, M. A., & Porteus, E. L. (1999). Stalking information: Bayesian inventory management with unobserved lost sales. Management Science, 45(3), 346–363. ISSN 0025-1909.
Lovejoy, W. S. (1990). Myopic policies for some inventory models with uncertain demand distributions. Management Science, 36(6), 724–738. ISSN 0025-1909.
Ludkovski, M., & Sezer, S. (2007). Finite horizon decision timing with partially observable Poisson processes (Technical Report). University of California Santa Barbara. This article is available at http://www.pstat.ucsb.edu/faculty/ludkovski/papers.html.
Neuts, M. F. (1989). Structured stochastic matrices of M/G/1 type and their applications. New York: Dekker.
Sethi, S. P., & Cheng, F. (1997). Optimality of (s,S) policies in inventory models with Markovian demand. Operations Research, 45(6), 931–939. ISSN 0030-364X.
Song, J.-S., & Zipkin, P. (1993). Inventory control in a fluctuating demand environment. Operations Research, 41(2), 351–370. ISSN 0030-364X.
Treharne, J. T., & Sox, C. R. (2002). Adaptive inventory control for nonstationary demand and partial information. Management Science, 48(5), 607–624. ISSN 0025-1909.
Zabczyk, J. (1983). Stopping problems in stochastic control. In Proceedings of the international congress of mathematicians (pp. 1425–1437).
Author information
Authors and Affiliations
Corresponding author
Additional information
E. Bayraktar is supported in part by the National Science Foundation.
Rights and permissions
About this article
Cite this article
Bayraktar, E., Ludkovski, M. Inventory management with partially observed nonstationary demand. Ann Oper Res 176, 7–39 (2010). https://doi.org/10.1007/s10479-009-0513-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-009-0513-8