Advertisement

Empirical Economics

, Volume 13, Issue 3–4, pp 155–168 | Cite as

Calibrating histograms with application to economic data

  • D. W. Scott
  • H. -P. Schmitz
Article

Summary

In this paper the problem of automatic calibration of histograms by cross-validation is considered, assuming the true underlying density is continuous with continuous first derivative. The histogram is one of the simpliest semiparametric estimators used by economists, but it is surprisingly difficult to construct histograms with small estimation errors. Cross-validation algorithms attempt to automatically determine histogram bin widths that are nearly optimal with respect to mean integrated squared error. Alternative philosophies and approaches of cross-validation for histograms are presented. It is shown that the classical Sturges' rule performs poorly and that cross-validation is a relatively difficult task. Understanding the performance of cross-validation algorithms in this simple setting should prove valuable when cross-validating other more complex semiparametric procedures.

Key words

Histogram Bin width Cross-validation Automatic bin width selection 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Beach CM, Davidson R (1983) Distribution-free statistical inference with Lorenz curves and income shares. Review of Economic Studies L 723–735Google Scholar
  2. Becker RA, Chambers JM (1984) S: an interactive environment for data analysis and graphics. Wadsworth, Belmont, CaliforniaGoogle Scholar
  3. Bowman AW (1984) An alternative method of cross-validation for the smoothing of density estimates. Biometrika 71:353–360Google Scholar
  4. DIW (1983) “Das Sozio-ökonomische Panel”. Deutsches Institut für Wirtschaftsforschung, BerlinGoogle Scholar
  5. Gibrat R (1930) Une loi des répartitions économiques: t'effet proportionnel. Bulletin Statistique Géneral Francais 19Google Scholar
  6. Härdle W, Scott DW (1988) Economic application of WARP: weighted average of rounded points. Technical Report 88-5, Dept. of Statistics, Rice UniversityGoogle Scholar
  7. Hildenbrand K, Hildenbrand W (1986) On the mean income effect: a data analysis of the U.K. family expenditure survey. In: Hildenbrand W, Mas-Colell A (eds) Contributions to mathematical economics. In honor of Gérard Debreu. North-Holland, Amsterdam, pp 247–268Google Scholar
  8. McDonald JB (1984) Some generalized functions for the size distribution of income. Econometrica 52:647–663Google Scholar
  9. Rudemo M (1982) Empirical choice of histogram and kernel density estimators. Scandinavian Journal of Statistics 9:65–78Google Scholar
  10. Scott DW (1979) On optimal and data-based histograms. Biometrika 66:604–610Google Scholar
  11. Scott DW (1985a) Frequency polygons: theory and application. J American Statistical Association 80:348–354Google Scholar
  12. Scott DW (1985b) Averaged shifted histograms: effective nonparametric density estimators in several dimensions. Ann Statist 13:1024–1040Google Scholar
  13. Scott DW (1986) Choosing smoothing parameters for density estimators. Computer Science and Statistics: Proceedings of the 17th Symposium on the Interface. North-Holland, pp 225–230Google Scholar
  14. Scott DW, Terrell GR (1987) Biased and unbiased cross-validation in density estimation. J American Statistical Association 82:1131–1146Google Scholar
  15. Sendler W (1979) On statistical inference in concentration measurement. Metrika 26:109–122Google Scholar
  16. Silverman BW (1986) Density estimation for statistics and data analysis. Chapman and Hall, LondonGoogle Scholar
  17. Stone CJ (1985) An asymptotically optimal histogram selection rule. In: Olshen R (ed) Proceedings of Berkeley Symp. in Honor of Jerzy Neyman and Jack Kiefer, vol 2. Wadsworth, CA, pp 513–520Google Scholar
  18. Sturges HA (1926) The choice of a class interval. J American Statistical Association 21:65–66Google Scholar
  19. Terrell GR, Scott DW (1985) Oversmoothed nonparametric density estimates. J American Statistical Association 80:209–214Google Scholar

Copyright information

© Physica-Verlag 1988

Authors and Affiliations

  • D. W. Scott
    • 1
  • H. -P. Schmitz
    • 2
  1. 1.Department of StatisticsRice UniversityHoustonUSA
  2. 2.Universität BonnBonnFRG

Personalised recommendations