Abstract
This study reviews estimation methods for the infinite horizon discrete choice dynamic programming models and conducts Monte Carlo experiments. We consider: the maximum likelihood estimator (MLE), the two-step conditional choice probabilities estimator, sequential estimators based on policy iterations mapping under finite dependence, and sequential estimators based on value iteration mappings. Our simulation result shows that the estimation performance of the sequential estimators based on policy iterations and value iteration mappings is largely comparable to the MLE, while they achieve substantial computation gains over the MLE by a factor of 100 for a model with a moderately large state space.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aguirregabiria, V. (1999) “The Dynamics of Markups and Inventories in Retailing Firms”, Review of Economic Studies, Vol. 66, No. 2, pp. 275–308.
—— and P. Mira (2002) “Swapping the Nested Fixed Point Algorithm: A Class of Estimators for Discrete Markov Decision Models”, Econometrica, Vol. 70, No. 4, pp. 1519–1543.
—— and —— (2007) “Sequential Estimation of Dynamic Discrete Games”, Econometrica, Vol. 75, No. 1, pp. 1–53.
—— and —— (2010) “Dynamic Discrete Choice Structural Models: A Survey”, Journal of Econometrics, Vol. 156, No. 1, pp. 38–67.
Altuğ, S. and R. A. Miller (1998) “The Effect of Work Experience on Female Wages and Labour Supply”, Review of Economic Studies, Vol. 65, No. 1, pp. 45–85.
Arcidiacono, P. and P. B. Ellickson (2011) “Practical Methods for Estimation of Dynamic Discrete Choice Models”, Annual Review of Economics, Vol. 3, No. 1, pp. 363–394.
—— and R. A. Miller (2011) “CCP Estimation of Dynamic Discrete Choice Models with Unobserved Heterogeneity”, Econometrica, Vol. 79, No. 6, pp. 1823–1867.
Berkovec, J. and S. Stern (1991) “Job Exit Behavior of Older Men”, Econometrica, Vol. 59, No. 1, pp. 189–210.
Bray, R. L. (2017) “Strong Convergence and Dynamic Economic Models”, Working Paper, Northwestern University, Evanston, IL.
Ching, A. T., S. Imai, M. Ishihara and N. Jain (2012) “A Practitioner’s Guide to Bayesian Estimation of Discrete Choice Dynamic Programming Models”, Quantitative Marketing and Economics, Vol. 10, No. 2, pp. 151–196.
Das, M. (1992) “A Micro-econometric Model of Capital Utilization and Retirement: The Case of the Cement Industry”, Review of Economic Studies, Vol. 59, No. 2, pp. 277–298.
Das, S., M. J. Roberts and J. R. Tybout (2007) “Market Entry Costs, Producer Heterogeneity, and Export Dynamics”, Econometrica, Vol. 75, No. 3, pp. 837–873.
Eckstein, Z. and K. Wolpin (1999) “Why Youth Drop Out of High School: The Impact of Preferences, Opportunities and Abilities”, Econometrica, Vol. 67, No. 6, pp. 1295–1339.
Gayle, G.-L., L. Golan and M. A. Soytas (2015) “What Accounts for the Racial Gap in Time Allocation and Intergenerational Transmission of Human Capital?”, Federal Reserve Bank of St. Louis Working Paper No.: 2015–018, St. Louis, MI.
Gilleskie, D. (1998) “A Dynamic Stochastic Model of Medical Care Use and Work Absence”, Econometrica, Vol. 66, No. 1, pp. 1–45.
Gotz, G. A. and J. J. McCall (1980) “Estimation in Sequential Decisionmaking Models: A Methodological Note”, Economics Letters, Vol. 6, No. 2, pp. 131–136.
Gowrisankaran, G. and M. Rysman (2012) “Dynamics of Consumer Demand for New Durable Goods”, Journal of Political Economy, Vol. 120, No. 6, pp. 1173–1219.
Hotz, J. and R. A. Miller (1993) “Conditional Choice Probabilities and the Estimation of Dynamic Models”, Review of Economic Studies, Vol. 60, No. 3, pp. 497–529.
Imai, S., N. Jain and A. Ching (2009) “Bayesian Estimation of Dynamic Discrete Choice Models”, Econometrica, Vol. 77, No. 6, pp. 1865–1899.
Iskhakov, F., J. Lee, J. Rust, B. Schjerning and K. Seo (2016) “Comment on “Constrained Optimization Approaches to Estimation of Structural Models”, Econometrica, Vol. 84, No. 1, pp. 365–370.
Kasahara, H. (2009) “Temporary Increases in Tariffs and Investment: The Chilean Experience”, Journal of Business & Economic Statistics, Vol. 27, No. 1, pp. 113–127.
—— and B. Lapham (2012) “Productivity and the Decision to Import and Export: Theory and Evidence”, Journal of International Economics, Vol. 89, No. 2, pp. 297–316.
—— and —— (2013) “Productivity and the Decision to Import and Export: Theory and Evidence”, Journal of International Economics, Vol. 89, No. 2, pp. 297–316.
—— and Rodrigue, J. (2008) “Does the Use of Imported Intermediates Increase Productivity? Plant-level Evidence”, Journal of Development Economics, Vol. 87, No. 1, pp. 106–118.
—— and K. Shimotsu (2008) “Pseudo-likelihood Estimation and Bootstrap Inference for Structural Discrete Markov Decision Models”, Journal of Econometrics, Vol. 146, No. 1, pp. 92–106.
—— and —— (2009) “Nonparametric Identification of Mixture Models of Dynamic Discrete Choices”, Econometrica, Vol. 77, No. 1, pp. 135–175.
—— and —— (2011), “Sequential Estimation of Dynamic Programming Models”, Working Paper, University of British Columbia, Vancouver, BC.
—— and —— (2012) “Sequential Estimation of Structural Models with a Point Constraint”, Econometrica, Vol. 80, No. 5, pp. 2303–2319.
Keane, M. P., P. E. Todd and K. I. Wolpin (2011) “The Structural Estimation of Behavioral Models: Discrete Choice Dynamic Programming Methods and Applications”, in O. Ashenfelter and D. Card, eds, Handbook of Labor Economics, Vol. 4A, Amsterdam: Elsevier Science, pp. 331–461.
—— and K. I. Wolpin (1997) “The Career Decisions of Young Men”, Journal of Political Economy, Vol. 105, No. 3, pp. 473–522.
Kennan, J. and J. R. Walker (2011) “The Effect of Expected Income on Individual Migration Decisions”, Econometrica, Vol. 79, No. 1, pp. 211–251.
Miller, R. (1984) “Job Matching and Occupational Choice”, Journal of Political Economy, Vol. 92, No. 6, pp. 1086–1120.
Norets, A. (2009) “Inference in Dynamic Discrete Choice Models With Serially Correlated Unobserved State Variables”, Econometrica, Vol. 77, No. 5, pp. 1665–1682.
Pakes, A. (1986) “Patents as Options: Some Estimates of the Value of Holding European Patent Stocks”, Econometrica, Vol. 54, No. 4, pp. 755–784.
—— (1994) “Dynamic Structural Models, Problems and Prospects”, in C. Sims, Advances in Econometrics, Sixth World Congress, Cambridge: Cambridge University Press.
Puterman, M. L. (1994) Markov Decision Processes, New Jersey: John Wiley & Sons.
Rothwell, G. and J. Rust (1997) “On the Optimal Lifetime of Nuclear Power Plants”, Journal of Business and Economic Statistics, Vol. 15, No. 2, pp. 195–208.
Rust, J. (1987) “Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher”, Econometrica, Vol. 55, No. 5, pp. 999–1033.
—— (1994a) “Estimation of Dynamic Structural Models, Problems and Prospects: Discrete Decision Processes”, in C. Sims, ed, Advances in Econometrics, Sixth World Congress, Vol. 2, Cambridge: Cambridge University Press, pp. 119–170.
—— (1994b) “Structural Estimation of Markov Decision Processes”, in R. E. Engle and D. L. McFadden, eds, Handbook of Econometrics, Vol. 4, Amsterdam: North-Holland, pp. 3081–4143.
—— (2000) “Nested Fixed Point Algorithm Documentation Manual”, Working Paper, Yale University, New Haven, CT.
—— and C. Phelan (1997) “How Social Security and Medicare Affect Retirement Behavior in a World of Incomplete Markets”, Econometrica, Vol. 65, No. 4, pp. 781–832.
Srisuma, S. and O. Linton (2012) “Semiparametric Estimation of Markov Decision Processes with Continuous State Space”, Journal of Econometrics, Vol. 166, No. 2, pp. 320–341.
Su, C.-L. and K. L. Judd (2012) “Constrained Optimization Approaches to Estimation of Structural Models”, Econometrica, Vol. 80, No. 5, pp. 2213–2230.
Wolpin, K. (1984), “An Estimable Dynamic Stochastic Model of Fertility and Child Mortality”, Journal of Political Economy, Vol. 92, No. 5, pp. 852–874.
Yamaguchi, S. (2016) “Effects of Parental Leave Policies on Female Career and Fertility Choices”, Working Paper, McMaster University, Hamilton, ON.
