Abstract
We present an overview of approximate Bayesian methods for sequential learning in problems where conjugate Bayesian priors are unsuitable or unavailable. Such problems have numerous applications in simulation optimization, revenue management, e-commerce, and the design of competitive events. We discuss two important computational strategies for learning in such applications, and illustrate each strategy with multiple examples from the recent literature. We also briefly describe conjugate Bayesian models for comparison, and remark on the theoretical challenges of approximate models.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Blei, D.M., Jordan, M.I., Paisley, J.W.: Variational Bayesian inference with stochastic search. In: Proceedings of the 29th International Conference on Machine Learning, pp. 1367–1374 (2012)
Brahma, A., Chakraborty, M., Das, S., Lavoie, A., Magdon-Ismail, M.: A Bayesian market maker. In: Proceedings of the 13th ACM Conference on Electronic Commerce, pp. 215–232 (2012)
Chakraborty, M., Das, S., Lavoie, A., Magdon-Ismail, M., Naamad, Y.: Instructor rating markets. In: Proceedings of the 27th AAAI Conference on Artificial Intelligence, pp. 159–165 (2013)
Chau, M., Fu, M.C., Qu, H., Ryzhov, I.O.: Simulation optimization: a tutorial overview and recent developments in gradient-based methods. In: Tolk, A., Diallo, S.Y., Ryzhov, I.O., Yilmaz, L., Buckley, S., Miller, J.A. (eds.) Proceedings of the 2014 Winter Simulation Conference, pp. 21–35 (2014)
Chhabra, M., Das, S.: Learning the demand curve in posted-price digital goods auctions. In: Proceedings of the 10th International Conference on Autonomous Agents and Multi-Agent Systems, pp. 63–70 (2011)
Chien, Y.T., Fu, K.-S.: On Bayesian learning and stochastic approximation. IEEE Trans. Syst. Sci. Cybern. 3(1), 28–38 (1967)
Cinlar, E.: Probability and Stochastics. Springer, New York (2011)
Dangauthier, P., Herbrich, R., Minka, T.P., Graepel, T.: TrueSkill through time: revisiting the history of chess. In: Platt, J.C., Koller, D., Singer, Y., Roweis, S. (eds.) Advances in Neural Information Processing Systems, Curran Associates, Inc., vol. 20, pp. 337–344 (2007)
Das, S., Magdon-Ismail, M.: Adapting to a market shock: optimal sequential market-making. In: Koller, D., Bengio, Y., Schuurmans, D., Bottou, L., Culotta, R. (eds.) Advances in Neural Information Processing Systems, Curran Associates, Inc., vol. 21, pp. 361–368 (2008)
DeGroot, M.H.: Optimal Statistical Decisions. Wiley, New York (1970)
Engelbert, H.J., Shiryaev, A.N.: On absolute continuity and singularity of probability measures. Banach Cent. Publ. 6, 121–132 (1980)
Garcia-Fernandez, A.F., Svensson, L.: Gaussian MAP filtering using Kalman optimisation. IEEE Trans. Autom. Control, 60(5):1336–1349 (2015)
Gelman, A., Carlin, J., Stern, H., Rubin, D.: Bayesian Data Analysis, 2nd edn. CRC Press, Boca Raton (2004)
Gupta, A., Nagar, D.: Matrix Variate Distributions. Chapman & Hall/CRC, London (2000)
Han, B., Ryzhov, I.O., Defourny, B.: Efficient learning of donor retention strategies for the American Red Cross. In: Pasupathy, R., Kim, S.-H., Tolk, A., Hill, R., Kuhl, M.E. (eds.) Proceedings of the 2013 Winter Simulation Conference, pp. 17–28 (2013)
Herbrich, R., Minka, T.P., Graepel, T.: TrueSkillTM: a Bayesian skill rating system. In: Schölkopf, B., Platt, J.C., Hoffman, T. (eds.) Advances in Neural Information Processing Systems, MIT Press, vol. 19, pp. 569–576 (2006)
Hong, L.J., Nelson, B.L.: A brief introduction to optimization via simulation. In: Rosetti, M., Hill, R., Johansson, B., Dunkin, A., Ingalls, R. (eds.) Proceedings of the 2009 Winter Simulation Conference, pp. 75–85 (2009)
Jaakkola, T.S., Jordan, M.I.: Bayesian parameter estimation via variational methods. Stat. Comput. 10(1), 25–37 (2000)
Kaelbling, L.P.: Learning in Embedded Systems. MIT Press, Cambridge (1993)
Kim, J.-Y.: Limited information likelihood and Bayesian analysis. J. Econom. 107(1), 175–193 (2002)
Kim, S.: Gradient-based simulation optimization. In: Perrone, L.F., Wieland, F.P., Liu, J., Lawson, B.G., Nicol, D.M., Fujimoto, R.M. (eds.) Proceedings of the 2006 Winter Simulation Conference, pp. 159–167 (2006)
Kushner, H.J., Yin, G.: Stochastic Approximation and Recursive Algorithms and Applications, 2nd edn. Springer, Berlin (2003)
Marmidis, G., Lazarou, S., Pyrgioti, E.: Optimal placement of wind turbines in a wind park using Monte Carlo simulation. Renew. Energy 33(7), 1455–1460 (2008)
Minka, T.P.: Bayesian linear regression. Technical report, Microsoft Research (2000)
Minka, T.P.: A family of algorithms for approximate Bayesian inference. Ph.D. thesis, Massachusetts Institute of Technology (2001)
Negoescu, D.M., Frazier, P.I., Powell, W.B.: The knowledge-gradient algorithm for sequencing experiments in drug discovery. INFORMS J. Comput. 23(3), 346–363 (2010)
Pati, D., Bhattacharya, A., Pillai, N.S., Dunson, D.: Posterior contraction in sparse Bayesian factor models for massive covariance matrices. Ann. Stat. 42(3), 1102–1130 (2014)
Powell, W.B.: Approximate Dynamic Programming: Solving the Curses of Dimensionality, 2nd edn. Wiley, New York (2011)
Powell, W.B., Ryzhov, I.O.: Optimal Learning. Wiley, New York (2012)
Qu, H., Ryzhov, I.O., Fu, M.C.: Ranking and selection with unknown correlation structures. In: Laroque, C., Himmelspach, J., Pasupathy, R., Rose, O., Uhrmacher, A.M. (eds.) Proceedings of the 2012 Winter Simulation Conference, pp. 144–155 (2012)
Qu, H., Ryzhov, I.O., Fu, M.C.: Learning logistic demand curves in business-to-business pricing. In: Pasupathy, R., Kim, S.-H., Tolk, A., Hill, R., Kuhl, M.E. (eds.) Proceedings of the 2013 Winter Simulation Conference, pp. 29–40 (2013)
Qu, H., Ryzhov, I.O., Fu, M.C., Ding, Z.: Sequential selection with unknown correlation structures. Operations Research, 63(4):931–948 (2015)
Ryzhov, I.O., Tariq, A., Powell, W.B.: May the best man win: simulation optimization for match-making in e-sports. In: Jain, S., Creasey, R.R., Himmelspach, J., White, K.P., Fu, M.C. (eds.) Proceedings of the 2011 Winter Simulation Conference, pp. 4239–4250 (2011)
Spall, J.C.: Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control. Wiley-Interscience, Hoboken (2005)
Spiegelhalter, D.J., Lauritzen, S.L.: Sequential updating of conditional probabilities on directed graphical structures. Networks 20(5), 579–605 (1990)
Talluri, K.T., Van Ryzin, G.J.: The Theory and Practice of Revenue Management. Springer, New York (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Ryzhov, I.O. (2015). Approximate Bayesian inference for simulation and optimization. In: Defourny, B., Terlaky, T. (eds) Modeling and Optimization: Theory and Applications. Springer Proceedings in Mathematics & Statistics, vol 147. Springer, Cham. https://doi.org/10.1007/978-3-319-23699-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-23699-5_1
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23698-8
Online ISBN: 978-3-319-23699-5
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)