# A practitioner’s guide to Bayesian estimation of discrete choice dynamic programming models

- 2.3k Downloads
- 9 Citations

## Abstract

This paper provides a step-by-step guide to estimating infinite horizon discrete choice dynamic programming (DDP) models using a new Bayesian estimation algorithm (Imai et al., Econometrica 77:1865–1899, 2009a) (IJC). In the conventional nested fixed point algorithm, most of the information obtained in the past iterations remains unused in the current iteration. In contrast, the IJC algorithm extensively uses the computational results obtained from the past iterations to help solve the DDP model at the current iterated parameter values. Consequently, it has the potential to significantly alleviate the computational burden of estimating DDP models. To illustrate this new estimation method, we use a simple dynamic store choice model where stores offer “frequent-buyer” type rewards programs. Our Monte Carlo results demonstrate that the IJC method is able to recover the true parameter values of this model quite precisely. We also show that the IJC method could reduce the estimation time significantly when estimating DDP models with unobserved heterogeneity, especially when the discount factor is close to 1.

## Keywords

Bayesian estimation Dynamic programming Discrete choice models Rewards programs## JEL Classification

C11 C35 C61 D91 M31## Notes

### Acknowledgements

We thank Martin Burda, Monica Meireles, Matthew Osborne, Peter Rossi, Andrei Strijnev, K. Sudhir, S. Siddarth and two anonymous referees for their helpful comments. We also thank the participants of the UCLA Marketing Camp, SBIES conference, Marketing Science Conference, Marketing Dynamics Conference, UTD-FORMS Conference, Canadian Economic Association Meeting, Econometric Society Meeting and Ph.D. seminars at OSU’s Fisher College of Business, Yale School of Management, University of Groningen, University of Zurich and University of Southern California for their useful feedback. Hyunwoo Lim provided excellent research assistance. All remaining errors are ours. Andrew Ching and Susumu Imai acknowledge the financial support from SSHRC.

## References

- Ackerberg, D. A. (2003). Advertising, learning, and consumer choice in experience good markets: An empirical examination.
*International Economic Review, 44*(3), 1007–1040.CrossRefGoogle Scholar - Ackerberg, D. A. (2009). A new use of importance sampling to reduce computational burden in simulation estimation.
*Quantitative Marketing and Economics, 7*(4), 343–376.CrossRefGoogle Scholar - Aguirregabiria, V., & Mira, P. (2002). Swapping the nested fixed point algorithm: A class of estimators for discrete Markov decision models.
*Econometrica, 70*(4), 1519–1543.CrossRefGoogle Scholar - Albert, J. H., & Chib, S. (1993). Bayesian analysis of binary and polychotomous response data.
*Journal of the American Statistical Association, 88*, 669–679.Google Scholar - Allenby, G. M. (1994). An introduction to hierarchical Bayesian modeling. Tutorial Notes, Advanced Research Techniques Forum, American Marketing Association.Google Scholar
- Allenby, G. M., & Lenk, P. J. (1994). Modeling household purchase behavior with logistic normal regression.
*Journal of the American Statistical Association, 89*, 1218–1231.Google Scholar - Allenby, G. M., & Rossi, P. E. (2006). Hierarchical Bayes models: A practitioner’s guide. In R. Grover, & M. Vriens (Eds.),
*The handbook of marketing research*. Newbury Park: Sage Publications.Google Scholar - Berry, S. T., Levinshohn, J., & Pakes, A. (1995). Automobile prices in market equilibrium.
*Econometrica, 63*(4), 841–890.CrossRefGoogle Scholar - Brown, M., & Flinn, C. J. (2011). Family law effects on divorce, fertility and child investment. Working paper, Department of Economics, New York University.Google Scholar
- Černý, V. (1985). Thermodynamical approach to the travelling salesman problem: An efficient simulation algorithm.
*Journal of Optimization Theory and Applications, 45*(1), 41–51.CrossRefGoogle Scholar - Ching, A. T. (2010). A dynamic oligopoly structural model for the prescription drug market after patent expiration.
*International Economic Review, 51*(4), 1175–1207.CrossRefGoogle Scholar - Ching, A. T., Imai, S., Ishihara, M., & Jain, N. (2009). A dynamic model of consumer learning and forgetting. Work-in-progress, Rotman School of Management, University of Toronto.Google Scholar
- Ching, A. T., Imai, S., Ishihara, M., & Jain, N. (2012). Identification of dynamic models of rewards program. Working paper, Rotman School of Management, University of Toronto.Google Scholar
- Crawford, G. S., & Shum, M. (2005). Uncertainty and learning in pharmaceutical demand.
*Econometrica, 73*(4), 1137–1174.CrossRefGoogle Scholar - Diermeier, D., Keane, M. P., & Merlo, A. M. (2005). A political economy model of congressional careers.
*American Economic Review, 95*, 347–373.CrossRefGoogle Scholar - Erdem, T., Imai, S., & Keane, M. P. (2003). Brand and quality choice dynamics under price uncertainty.
*Quantitative Marketing and Economics, 1*(1), 5–64.CrossRefGoogle Scholar - Erdem, T., & Keane, M. P. (1996). Decision making under uncertainty: Capturing dynamic brand choice processes in turbulent consumer goods markets.
*Marketing Science, 15*(1), 1–20.CrossRefGoogle Scholar - Geweke, J., Houser, D., & Keane, M. P. (2001). Simulation based inference for dynamic multinomial choice models. In B. H. Baltagi (Ed.),
*A companion to theoretical econometrics*(pp. 466–493). London: Blackwell.Google Scholar - Geweke, J. F., & Keane, M. P. (2000). Bayesian inference for dynamic discrete choice models without the need for dynamic programming. In R. Mariano, T. Schuermann, & M. J. Weeks (Eds.),
*Simulation based inference and econometrics: Methods and applications*. Cambridge: Cambridge University Press.Google Scholar - Gönül, F., & Srinivasan, K. (1996). Estimating the impact of consumer expectations of coupons on purchase behavior: A dynamic structural model.
*Marketing Science, 15*(3), 262–279.CrossRefGoogle Scholar - Hartmann, W. R. (2006). Intertemporal effects of consumption and their implications for demand elasticity estimates.
*Quantitative Marketing and Economics, 4*(4), 325–349.CrossRefGoogle Scholar - Hendel, I., & Nevo, A. (2006). Measuring the implications of sales and consumer inventory behavior.
*Econometrica, 74*(6), 1637–1673.CrossRefGoogle Scholar - Hitsch, G. (2006). An empirical model of optimal dynamic product launch and exit under demand uncertainty.
*Marketing Science, 25*(1), 25–50.CrossRefGoogle Scholar - Hotz, J. V., & Miller, R. (1993). Conditional choice probabilities and the estimation of dynamic models.
*Review of Economic Studies, 60*(3), 497–529.CrossRefGoogle Scholar - Imai, S., Jain, N., & Ching, A. (2009a). Bayesian estimation of dynamic discrete choice models.
*Econometrica, 77*(6), 1865–1899.CrossRefGoogle Scholar - Imai, S., Jain, N., & Ching, A. (2009b). Supplement to ‘Bayesian estimation of dynamic discrete choice models’.
*Econometrica (Supplementary Material), 77*. http://www.econometricsociety.org/ecta/Supmat/5658_proofs.pdf. - Imai, S., & Krishna, K. (2004). Employment, deterrence and crime in a dynamic model.
*International Economic Review, 45*(3), 845–872.CrossRefGoogle Scholar - Ishihara, M. (2011).
*Dynamic demand for new and used durable goods without physical depreciation*. Ph.D. dissertation, Rotman School of Management, University of Toronto.Google Scholar - Keane, M. P., & Wolpin, K. I. (1994). The solution and estimation of discrete choice dynamic programming models by simulation and interpolation: Monte Carlo evidence.
*Review of Economics and Statistics, 76*(4), 648–672.CrossRefGoogle Scholar - Keane, M. P., & Wolpin, K. I. (1997). The career decisions of young men.
*Journal of Political Economy, 105*, 473–521.CrossRefGoogle Scholar - Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing.
*Science, 220*, 671–680.CrossRefGoogle Scholar - Lancaster, T. (1997). Exact structural inference in optimal job search models.
*Journal of Business and Economic Statistics, 15*(2), 165–179.CrossRefGoogle Scholar - McCulloch, R., & Rossi, P. E. (1994). An exact likelihood analysis of the multinomial probit model.
*Journal of Econometrics, 64*, 207–240.CrossRefGoogle Scholar - Norets, A. (2009). Inference in dynamic discrete choice models with serially correlated unobserved state variables.
*Econometrica, 77*(5), 1665–1682.CrossRefGoogle Scholar - Norets, A. (2010). Continuity and differentiability of expected value functions in dynamic discrete choice models.
*Quantitative Economics, 1*(2), 305–322.CrossRefGoogle Scholar - Osborne, M. (2011). Consumer learning, switching costs, and heterogeneity: A structural examination.
*Quantitative Marketing and Economics, 9*(1), 25–70.CrossRefGoogle Scholar - Pantano, J. (2008). Essays in applied microeconomics. Ph.D. dissertation, UCLA.Google Scholar
- Roos, J. M. T., Mela, C. F., & Shachar, R. (2011). Hyper-media search and consumption. Working paper, Fuqua School of Business, Duke University.Google Scholar
- Rossi, P. E., & Allenby, G. M. (1999). Marketing models of consumer heterogeneity.
*Journal of Econometrics, 89*, 57–78.Google Scholar - Rossi, P. E., Allenby, G. M., & McCulloch, R. (2005).
*Bayesian statistics and marketing*. Chichester: Wiley.CrossRefGoogle Scholar - Rossi, P. E., McCulloch, R., & Allenby, G. M. (1996). The value of purchase history data in target marketing.
*Marketing Science, 15*, 321–340.CrossRefGoogle Scholar - Rust, J. (1987). Optimal replacement of gmc bus engines: An empirical model of Harold Zurcher.
*Econometrica, 55*(5), 999–1033.CrossRefGoogle Scholar - Rust, J. (1988). Maximum likelihood estimation of discrete control processes.
*SIAM Journal on Control and Optimization, 26*(5), 1006–1024.CrossRefGoogle Scholar - Rust, J. (1997). Using randomization to break the curse of dimensionality.
*Econometrica, 65*(3), 487–516.CrossRefGoogle Scholar - Santos, M. S., & Rust, J. (2004). Convergence properties of policy iteration.
*SIAM Journal on Control and Optimization, 42*(6), 2094–2115.CrossRefGoogle Scholar - Silverman, B. W. (1986).
*Density estimation for statistics and data analysis*. London: Chapman and Hall.Google Scholar - Song, I., & Chintagunta, P. K. (2003). A micromodel of new product adoption with heterogeneous and forward looking consumers: Application to the digital camera category.
*Quantitative Marketing and Economics, 1*(4), 371–407.CrossRefGoogle Scholar - Sun, B. (2005). Promotion effect on endogenous consumption.
*Marketing Science, 24*(3), 430–443.CrossRefGoogle Scholar - Train, K. E. (2003).
*Discrete choice methods with simulation*. Cambridge: Cambridge University Press. Available at http://elsa.berkeley.edu/books/choice2.html. - Walsh, B. (2004). Markov Chain Monte Carlo and Gibbs sampling. Lecture Notes for EEB 581, University of Arizona. http://nitro.biosci.arizona.edu/courses/EEB581-2004/handouts/Gibbs.pdf.
- Yang, B., & Ching, A. (2010). Dynamics of consumer adoption of financial innovation: The case of ATM cards. Working paper, Rotman School of Management, University of Toronto. Available at SSRN: http://ssrn.com/abstract=1434722.