Quantitative Marketing and Economics

, Volume 10, Issue 2, pp 151–196 | Cite as

A practitioner’s guide to Bayesian estimation of discrete choice dynamic programming models

  • Andrew T. ChingEmail author
  • Susumu Imai
  • Masakazu Ishihara
  • Neelam Jain


This paper provides a step-by-step guide to estimating infinite horizon discrete choice dynamic programming (DDP) models using a new Bayesian estimation algorithm (Imai et al., Econometrica 77:1865–1899, 2009a) (IJC). In the conventional nested fixed point algorithm, most of the information obtained in the past iterations remains unused in the current iteration. In contrast, the IJC algorithm extensively uses the computational results obtained from the past iterations to help solve the DDP model at the current iterated parameter values. Consequently, it has the potential to significantly alleviate the computational burden of estimating DDP models. To illustrate this new estimation method, we use a simple dynamic store choice model where stores offer “frequent-buyer” type rewards programs. Our Monte Carlo results demonstrate that the IJC method is able to recover the true parameter values of this model quite precisely. We also show that the IJC method could reduce the estimation time significantly when estimating DDP models with unobserved heterogeneity, especially when the discount factor is close to 1.


Bayesian estimation Dynamic programming Discrete choice models Rewards programs 

JEL Classification

C11 C35 C61 D91 M31 



We thank Martin Burda, Monica Meireles, Matthew Osborne, Peter Rossi, Andrei Strijnev, K. Sudhir, S. Siddarth and two anonymous referees for their helpful comments. We also thank the participants of the UCLA Marketing Camp, SBIES conference, Marketing Science Conference, Marketing Dynamics Conference, UTD-FORMS Conference, Canadian Economic Association Meeting, Econometric Society Meeting and Ph.D. seminars at OSU’s Fisher College of Business, Yale School of Management, University of Groningen, University of Zurich and University of Southern California for their useful feedback. Hyunwoo Lim provided excellent research assistance. All remaining errors are ours. Andrew Ching and Susumu Imai acknowledge the financial support from SSHRC.


  1. Ackerberg, D. A. (2003). Advertising, learning, and consumer choice in experience good markets: An empirical examination. International Economic Review, 44(3), 1007–1040.CrossRefGoogle Scholar
  2. Ackerberg, D. A. (2009). A new use of importance sampling to reduce computational burden in simulation estimation. Quantitative Marketing and Economics, 7(4), 343–376.CrossRefGoogle Scholar
  3. Aguirregabiria, V., & Mira, P. (2002). Swapping the nested fixed point algorithm: A class of estimators for discrete Markov decision models. Econometrica, 70(4), 1519–1543.CrossRefGoogle Scholar
  4. Albert, J. H., & Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association, 88, 669–679.Google Scholar
  5. Allenby, G. M. (1994). An introduction to hierarchical Bayesian modeling. Tutorial Notes, Advanced Research Techniques Forum, American Marketing Association.Google Scholar
  6. Allenby, G. M., & Lenk, P. J. (1994). Modeling household purchase behavior with logistic normal regression. Journal of the American Statistical Association, 89, 1218–1231.Google Scholar
  7. Allenby, G. M., & Rossi, P. E. (2006). Hierarchical Bayes models: A practitioner’s guide. In R. Grover, & M. Vriens (Eds.), The handbook of marketing research. Newbury Park: Sage Publications.Google Scholar
  8. Berry, S. T., Levinshohn, J., & Pakes, A. (1995). Automobile prices in market equilibrium. Econometrica, 63(4), 841–890.CrossRefGoogle Scholar
  9. Brown, M., & Flinn, C. J. (2011). Family law effects on divorce, fertility and child investment. Working paper, Department of Economics, New York University.Google Scholar
  10. Černý, V. (1985). Thermodynamical approach to the travelling salesman problem: An efficient simulation algorithm. Journal of Optimization Theory and Applications, 45(1), 41–51.CrossRefGoogle Scholar
  11. Ching, A. T. (2010). A dynamic oligopoly structural model for the prescription drug market after patent expiration. International Economic Review, 51(4), 1175–1207.CrossRefGoogle Scholar
  12. Ching, A. T., Imai, S., Ishihara, M., & Jain, N. (2009). A dynamic model of consumer learning and forgetting. Work-in-progress, Rotman School of Management, University of Toronto.Google Scholar
  13. Ching, A. T., Imai, S., Ishihara, M., & Jain, N. (2012). Identification of dynamic models of rewards program. Working paper, Rotman School of Management, University of Toronto.Google Scholar
  14. Crawford, G. S., & Shum, M. (2005). Uncertainty and learning in pharmaceutical demand. Econometrica, 73(4), 1137–1174.CrossRefGoogle Scholar
  15. Diermeier, D., Keane, M. P., & Merlo, A. M. (2005). A political economy model of congressional careers. American Economic Review, 95, 347–373.CrossRefGoogle Scholar
  16. Erdem, T., Imai, S., & Keane, M. P. (2003). Brand and quality choice dynamics under price uncertainty. Quantitative Marketing and Economics, 1(1), 5–64.CrossRefGoogle Scholar
  17. Erdem, T., & Keane, M. P. (1996). Decision making under uncertainty: Capturing dynamic brand choice processes in turbulent consumer goods markets. Marketing Science, 15(1), 1–20.CrossRefGoogle Scholar
  18. Geweke, J., Houser, D., & Keane, M. P. (2001). Simulation based inference for dynamic multinomial choice models. In B. H. Baltagi (Ed.), A companion to theoretical econometrics (pp. 466–493). London: Blackwell.Google Scholar
  19. Geweke, J. F., & Keane, M. P. (2000). Bayesian inference for dynamic discrete choice models without the need for dynamic programming. In R. Mariano, T. Schuermann, & M. J. Weeks (Eds.), Simulation based inference and econometrics: Methods and applications. Cambridge: Cambridge University Press.Google Scholar
  20. Gönül, F., & Srinivasan, K. (1996). Estimating the impact of consumer expectations of coupons on purchase behavior: A dynamic structural model. Marketing Science, 15(3), 262–279.CrossRefGoogle Scholar
  21. Hartmann, W. R. (2006). Intertemporal effects of consumption and their implications for demand elasticity estimates. Quantitative Marketing and Economics, 4(4), 325–349.CrossRefGoogle Scholar
  22. Hendel, I., & Nevo, A. (2006). Measuring the implications of sales and consumer inventory behavior. Econometrica, 74(6), 1637–1673.CrossRefGoogle Scholar
  23. Hitsch, G. (2006). An empirical model of optimal dynamic product launch and exit under demand uncertainty. Marketing Science, 25(1), 25–50.CrossRefGoogle Scholar
  24. Hotz, J. V., & Miller, R. (1993). Conditional choice probabilities and the estimation of dynamic models. Review of Economic Studies, 60(3), 497–529.CrossRefGoogle Scholar
  25. Imai, S., Jain, N., & Ching, A. (2009a). Bayesian estimation of dynamic discrete choice models. Econometrica, 77(6), 1865–1899.CrossRefGoogle Scholar
  26. Imai, S., Jain, N., & Ching, A. (2009b). Supplement to ‘Bayesian estimation of dynamic discrete choice models’. Econometrica (Supplementary Material), 77.
  27. Imai, S., & Krishna, K. (2004). Employment, deterrence and crime in a dynamic model. International Economic Review, 45(3), 845–872.CrossRefGoogle Scholar
  28. Ishihara, M. (2011). Dynamic demand for new and used durable goods without physical depreciation. Ph.D. dissertation, Rotman School of Management, University of Toronto.Google Scholar
  29. Keane, M. P., & Wolpin, K. I. (1994). The solution and estimation of discrete choice dynamic programming models by simulation and interpolation: Monte Carlo evidence. Review of Economics and Statistics, 76(4), 648–672.CrossRefGoogle Scholar
  30. Keane, M. P., & Wolpin, K. I. (1997). The career decisions of young men. Journal of Political Economy, 105, 473–521.CrossRefGoogle Scholar
  31. Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220, 671–680.CrossRefGoogle Scholar
  32. Lancaster, T. (1997). Exact structural inference in optimal job search models. Journal of Business and Economic Statistics, 15(2), 165–179.CrossRefGoogle Scholar
  33. McCulloch, R., & Rossi, P. E. (1994). An exact likelihood analysis of the multinomial probit model. Journal of Econometrics, 64, 207–240.CrossRefGoogle Scholar
  34. Norets, A. (2009). Inference in dynamic discrete choice models with serially correlated unobserved state variables. Econometrica, 77(5), 1665–1682.CrossRefGoogle Scholar
  35. Norets, A. (2010). Continuity and differentiability of expected value functions in dynamic discrete choice models. Quantitative Economics, 1(2), 305–322.CrossRefGoogle Scholar
  36. Osborne, M. (2011). Consumer learning, switching costs, and heterogeneity: A structural examination. Quantitative Marketing and Economics, 9(1), 25–70.CrossRefGoogle Scholar
  37. Pantano, J. (2008). Essays in applied microeconomics. Ph.D. dissertation, UCLA.Google Scholar
  38. Roos, J. M. T., Mela, C. F., & Shachar, R. (2011). Hyper-media search and consumption. Working paper, Fuqua School of Business, Duke University.Google Scholar
  39. Rossi, P. E., & Allenby, G. M. (1999). Marketing models of consumer heterogeneity. Journal of Econometrics, 89, 57–78.Google Scholar
  40. Rossi, P. E., Allenby, G. M., & McCulloch, R. (2005). Bayesian statistics and marketing. Chichester: Wiley.CrossRefGoogle Scholar
  41. Rossi, P. E., McCulloch, R., & Allenby, G. M. (1996). The value of purchase history data in target marketing. Marketing Science, 15, 321–340.CrossRefGoogle Scholar
  42. Rust, J. (1987). Optimal replacement of gmc bus engines: An empirical model of Harold Zurcher. Econometrica, 55(5), 999–1033.CrossRefGoogle Scholar
  43. Rust, J. (1988). Maximum likelihood estimation of discrete control processes. SIAM Journal on Control and Optimization, 26(5), 1006–1024.CrossRefGoogle Scholar
  44. Rust, J. (1997). Using randomization to break the curse of dimensionality. Econometrica, 65(3), 487–516.CrossRefGoogle Scholar
  45. Santos, M. S., & Rust, J. (2004). Convergence properties of policy iteration. SIAM Journal on Control and Optimization, 42(6), 2094–2115.CrossRefGoogle Scholar
  46. Silverman, B. W. (1986). Density estimation for statistics and data analysis. London: Chapman and Hall.Google Scholar
  47. Song, I., & Chintagunta, P. K. (2003). A micromodel of new product adoption with heterogeneous and forward looking consumers: Application to the digital camera category. Quantitative Marketing and Economics, 1(4), 371–407.CrossRefGoogle Scholar
  48. Sun, B. (2005). Promotion effect on endogenous consumption. Marketing Science, 24(3), 430–443.CrossRefGoogle Scholar
  49. Train, K. E. (2003). Discrete choice methods with simulation. Cambridge: Cambridge University Press. Available at
  50. Walsh, B. (2004). Markov Chain Monte Carlo and Gibbs sampling. Lecture Notes for EEB 581, University of Arizona.
  51. Yang, B., & Ching, A. (2010). Dynamics of consumer adoption of financial innovation: The case of ATM cards. Working paper, Rotman School of Management, University of Toronto. Available at SSRN:

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Andrew T. Ching
    • 1
    Email author
  • Susumu Imai
    • 2
  • Masakazu Ishihara
    • 3
  • Neelam Jain
    • 4
  1. 1.Rotman School of ManagementUniversity of TorontoTorontoCanada
  2. 2.Department of EconomicsQueen’s UniversityKingstonCanada
  3. 3.Stern School of BusinessNew York UniversityNew YorkUSA
  4. 4.Department of EconomicsCity University LondonLondonUK

Personalised recommendations