Computational Economics

, Volume 53, Issue 3, pp 1165–1182 | Cite as

A Practical Approach to Testing Calibration Strategies

  • Yongquan Cao
  • Grey GordonEmail author


A calibration strategy tries to match target moments using a model’s parameters. We propose tests for determining whether this is possible. The tests use moments at random parameter draws to assess whether the target moments are similar to the computed ones (evidence of existence) or appear to be outliers (evidence of non-existence). Our experiments show the tests are effective at detecting both existence and non-existence in a non-linear model. Multiple calibration strategies can be quickly tested using just one set of simulated data. Applying our approach to indirect inference allows for the testing of many auxiliary model specifications simultaneously. Code is provided.


Calibration GMM Indirect inference Existence Misspecification Outlier detection Data mining 

JEL Classification

C13 C51 C52 C80 F34 


  1. Angiulli, F., & Pizzuti, C. (2002). Fast outlier detection in high dimensional spaces (pp. 15–27). Berlin: Springer.Google Scholar
  2. Arellano, C. (2008). Default risk and income fluctuations in emerging economies. American Economic Review, 98(3), 690–712.CrossRefGoogle Scholar
  3. Ben-Gal, I. (2005). Outlier detection. In O. Maimon & L. Rockach (Eds.), Data mining and knowledge discovery handbook (pp. 131–146). Springer.Google Scholar
  4. Billingsley, P. (1995). Probability and measure (3rd ed.). Hoboken, NJ: Wiley.Google Scholar
  5. Bollen, K. A., & Jackman, R. W. (1990). Modern methods of data analysis, chapter regression diagnostics: An expository treatment of outliers and influential cases. Newbury Park, CA: Sage.Google Scholar
  6. Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000). LOF: Identifying density-based local outliers. In Proceedings of the ACM SIGMOD 2000 international conference on management of data (pp. 93–104).Google Scholar
  7. Canova, F., Ferroni, F., & Matthes, C. (2014). Choosing the variables to estimate singular DSGE models. Journal of Applied Econometrics, 29(7), 1099–1117.CrossRefGoogle Scholar
  8. Canova, F., & Sala, L. (2009). Back to square one: Identification issues in DSGE models. Journal of Monetary Economics, 56(4), 431–449.CrossRefGoogle Scholar
  9. Chatterjee, S., & Eyigungor, B. (2012). Maturity, indebtedness, and default risk. American Economic Review, 102(6), 2674–2699.CrossRefGoogle Scholar
  10. Christiano, L. J., & Eichenbaum, M. (1992). Current real-business-cycle theories and aggregate labor-market fluctuations. American Economic Review, 82(3), 430–450.Google Scholar
  11. Enas, G. G., & Choi, S. C. (1986). Choice of the smoothing parameter and efficiency of the k-nearest neighbor classification. Computers & Mathematics with Applications, 12A(2), 235–244.CrossRefGoogle Scholar
  12. Feldman, T., & Sun, Y. (2011). Econometrics and computational economics: an exercise in compatibility. International Journal of Computational Economics and Econometrics, 2(2), 105–114.CrossRefGoogle Scholar
  13. Gallant, A. R., & Tauchen, G. (1996). Which moments to match? Econometric Theory, 12(4), 657–681.CrossRefGoogle Scholar
  14. Gnanadesikan, R. (1997). Methods for statistical data analysis of multivariate observations. New York, NY: Wiley.CrossRefGoogle Scholar
  15. Gordon, G., & Guerron-Quintana, P. (2017). Dynamics of investment, debt, and default. Review of Economic Dynamics. Scholar
  16. Gordon, G., & Qiu, S. (2017). A divide and conquer algorithm for exploiting policy function monotonicity. Quantitative Economics.
  17. Gourieroux, C., Monfort, A., & Renault, E. (1993). Indirect inference. Journal of Applied Econometrics, 8, S85–S118.CrossRefGoogle Scholar
  18. Guerron-Quintana, P. A. (2010). What you match does matter: The effects of data on DSGE estimation. Journal of Applied Econometrics, 25(5), 774–804.CrossRefGoogle Scholar
  19. Hansen, L. P. (1982). Large sample properties of generalized method of moments estimators. Econometrica, 50(4), 1029–1054.CrossRefGoogle Scholar
  20. Hansen, L. P. (2008). Generalized method of moments estimation. In S. N. Durlauf & L. E. Blume (Eds.), The new Palgrave dictionary of economics. Basingstoke: Palgrave Macmillan.Google Scholar
  21. Hansen, L. P., & Heckman, J. J. (1996). The empirical foundations of calibration. Journal of Economic Perspectives, 10(1), 87–104.CrossRefGoogle Scholar
  22. Kleibergen, F. (2005). Testing parameters in GMM without assuming that they are identified. Econometrica, 73(4), 1103–1123.CrossRefGoogle Scholar
  23. Koopmans, T. C., & Reiersol, O. (1950). The identification of structural characteristics. The Annals of Mathematical Statistics, 21(2), 165–181.CrossRefGoogle Scholar
  24. Kriegel, H.-P., Kröger, P., Schubert, E., & Zimek, A. (2009). LoOP: Local outlier probabilities. In Proceedings of the 18th ACM conference on information and knowledge management, CIKM ’09 (pp. 1649–1652). New York, NY, USA: ACM. ISBN 978-1-60558-512-3.Google Scholar
  25. Kydland, F. E., & Prescott, E. C. (1982). Time to build and aggregate fluctuations. Econometrica, 50(6), 1345–70.CrossRefGoogle Scholar
  26. Loftsgaarden, D. O., & Quesenberry, C. P. (1965). A nonparametric estimate of a multivariate density function. Annals of Mathematical Statistics, 36, 1049–1051.CrossRefGoogle Scholar
  27. Penny, K. I., & Jolliffe, I. T. (2001). A comparison of multivariate outlier detection methods for clinical laboratory safety data. The Statistician, 50(3), 295–308.Google Scholar
  28. Ramaswamy, S., Rastogi, R., & Shim, K. (2000). Efficient algorithms for mining outliers from large data sets. In Proceedings of the 2000 ACM SIGMOD international conference on management of data, SIGMOD ’00 (pp. 427–438). New York, NY, USA: ACM.Google Scholar
  29. Sargan, J. D. (1958). The estimation of economic relationships using instrumental variables. Econometrica, 26, 393–415.CrossRefGoogle Scholar
  30. Sargan, J. D. (1959). The estimation of relationships with autocorrelated residuals by the use of instrumental variables. Journal of the Royal Statistical Society: Series B (Methodological), 21(1), 91–105.Google Scholar
  31. Smith, A. A, Jr. (1993). Estimating nonlinear time-series models using simulated vector autoregressions. Journal of Applied Econometrics, 8, S63–S84.CrossRefGoogle Scholar
  32. Stock, J. H., & Wright, J. H. (2000). GMM with weak identification. Econometrica, 68(5), 1055–1096.CrossRefGoogle Scholar
  33. Stock, J. H., Wright, J. H., & Yogo, M. (2002). A survey of weak instruments and weak identification in generalized method of moments. Journal of Business Economics & Statistics, 20(4), 518–529.CrossRefGoogle Scholar
  34. Zhang, K., Hutter, M., & Jin, H. (2009). A new local distance-based outlier detection approach for scattered real-world data (pp. 813–822). Berlin: Springer.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Indiana UniversityBloomingtonUSA

Personalised recommendations