Robust Statistics

Part of the Selected Works in Probability and Statistics book series (SWPS, volume 13)


This is a short introduction to three papers on robustness, published by Peter Bickel as single author in the period 1975–1984: “One-step Huber estimates in the linear model” (Bickel 1975), “Parametric robustness: small biases can be worthwhile” (Bickel 1984a), and “Robust regression based on infinitesimal neighbourhoods” (Bickel1984b). It was the time when fundamental developments and understanding in robustness took place, and Peter Bickel has made deep contributions in this area. I am trying to place the results of the three papers in a new context of contemporary statistics.


Loss Function Robust Regression Deep Contribution Convergence Rate Optimality Parametric Robustness 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Begun JM, Hall WJ, Huang W-M, Wellner JA (1983) Information and asymptotic efficiency in parametric–nonparametric models. Ann Stat 11(2):432–452CrossRefzbMATHMathSciNetGoogle Scholar
  2. Beran R (1974) Asymptotically efficient adaptive rank estimates in location models. Ann Stat 2:63–74CrossRefzbMATHMathSciNetGoogle Scholar
  3. Bickel P (1975) One-step Huber estimates in the linear model. J Am Stat Assoc 70:428–434CrossRefzbMATHMathSciNetGoogle Scholar
  4. Bickel PJ (1982) On adaptive estimation. Ann Stat 10(3):647–671CrossRefzbMATHMathSciNetGoogle Scholar
  5. Bickel P (1984a) Parametric robustness: small biases can be worthwhile. Ann Stat 12:864–879CrossRefzbMATHMathSciNetGoogle Scholar
  6. Bickel P (1984b) Robust regression based on infinitesimal neighbourhoods. Ann Stat 12:1349–1368CrossRefzbMATHMathSciNetGoogle Scholar
  7. Bickel PJ, Klaassen CAJ (1986) Empirical Bayes estimation in functional and structural models, and uniformly adaptive estimation of location. Adv Appl Math 7(1):55–69CrossRefzbMATHMathSciNetGoogle Scholar
  8. Bickel PJ, Ritov Y (1987) Efficient estimation in the errors in variables model. Ann Stat 15(2):513–540CrossRefzbMATHMathSciNetGoogle Scholar
  9. Bickel PJ, Ritov Y (1988) Estimating integrated squared density derivatives: sharp best order of convergence estimates. Sankhyā Ser A 50(3):381–393zbMATHMathSciNetGoogle Scholar
  10. Bickel PJ, Klaassen CAJ, Ritov Y, Wellner JA (1993) Efficient and adaptive estimation for semiparametric models. Johns Hopkins series in the mathematical sciences. Johns Hopkins University Press, BaltimoreGoogle Scholar
  11. Bickel PJ, Klaassen CAJ, Ritov Y, Wellner JA (1998) Efficient and adaptive estimation for semiparametric models. Springer, New York. Reprint of the 1993 originalGoogle Scholar
  12. Birgé L, Massart P (1993) Rates of convergence for minimum contrast estimators. Probab Theory Relat Fields 97(1–2):113–150CrossRefzbMATHGoogle Scholar
  13. Birgé L, Massart P (1995) Estimation of integral functionals of a density. Ann Stat 23(1):11–29CrossRefzbMATHGoogle Scholar
  14. Bissantz N, Hohage T, Munk A, Ruymgaart F (2007) Convergence rates of general regularization methods for statistical inverse problems and applications. SIAM J Numer Anal 45:2610–2636CrossRefzbMATHMathSciNetGoogle Scholar
  15. Bühlmann P, Yu B (2003) Boosting with the L 2 loss: regression and classification. J Am Stat Assoc 98:324–339CrossRefzbMATHGoogle Scholar
  16. Efron B (1977) The efficiency of Cox’s likelihood function for censored data. J Am Stat Assoc 72(359):557–565CrossRefzbMATHMathSciNetGoogle Scholar
  17. Efron B, Morris C (1971) Limiting the risk of Bayes and empirical Bayes estimators – part I: Bayes case. J Am Stat Assoc 66:807–815zbMATHMathSciNetGoogle Scholar
  18. Friedman J (2001) Greedy function approximation: a gradient boosting machine. Ann Stat 29:1189–1232CrossRefzbMATHGoogle Scholar
  19. Hájek J (1962) Asymptotically most powerful rank-order tests. Ann Math Stat 33:1124–1147CrossRefzbMATHGoogle Scholar
  20. Khan J, Van Aelst S, Zamar R (2007) Robust linear model selection based on least angle regression. J Am Stat Assoc 102:1289–1299CrossRefzbMATHGoogle Scholar
  21. Kiefer J, Wolfowitz J (1956) Consistency of the maximum likelihood estimator in the presence of infinitely many incidental parameters. Ann Math Stat 27:887–906CrossRefzbMATHMathSciNetGoogle Scholar
  22. Klaassen CAJ (1987) Consistent estimation of the influence function of locally asymptotically linear estimators. Ann Stat 15(4):1548–1562CrossRefzbMATHMathSciNetGoogle Scholar
  23. Kosorok MR (2009) What’s so special about semiparametric methods? Sankhyā 71(2, Ser A): 331–353Google Scholar
  24. Laurent B, Massart P (2000) Adaptive estimation of a quadratic functional by model selection. Ann Stat 28(5):1302–1338CrossRefzbMATHMathSciNetGoogle Scholar
  25. Le Cam L (1956) On the asymptotic theory of estimation and testing hypotheses. In: Proceedings of the third Berkeley symposium on mathematical statistics and probability, vol 1. University of California Press, Berkeley, pp 129–156Google Scholar
  26. Lutz R, Kalisch M, Bühlmann P (2008) Robustified L2 boosting. Comput Stat Data Anal 52:3331–3341CrossRefzbMATHGoogle Scholar
  27. Meinshausen N, Bickel P, Rice J (2009) Efficient blind search: optimal power of detection under computational cost constraint. Ann Appl Stat 3:38–60CrossRefzbMATHMathSciNetGoogle Scholar
  28. Murphy SA, van der Vaart AW (1996) Likelihood inference in the errors-in-variables model. J Multivar Anal 59(1):81–108CrossRefzbMATHGoogle Scholar
  29. Neyman J, Scott EL (1948) Consistent estimates based on partially consistent observations. Econometrica 16:1–32CrossRefMathSciNetGoogle Scholar
  30. Pfanzagl J (1990a) Estimation in semiparametric models. Lecture notes in statistics, vol 63. Springer, New York. Some recent developmentsGoogle Scholar
  31. Pfanzagl J (1990b) Large deviation probabilities for certain nonparametric maximum likelihood estimators. Ann Stat 18(4):1868–1877CrossRefzbMATHMathSciNetGoogle Scholar
  32. Pfanzagl J (1993) Incidental versus random nuisance parameters. Ann Stat 21(4):1663–1691CrossRefzbMATHMathSciNetGoogle Scholar
  33. Reiersol O (1950) Identifiability of a linear relation between variables which are subject to error. Econometrica 18:375–389CrossRefMathSciNetGoogle Scholar
  34. Ritov Y, Bickel PJ (1990) Achieving information bounds in non and semiparametric models. Ann Stat 18(2):925–938CrossRefzbMATHMathSciNetGoogle Scholar
  35. Robins J, Tchetgen Tchetgen E, Li L, van der Vaart A (2009) Semiparametric minimax rates. Electron J Stat 3:1305–1321CrossRefzbMATHMathSciNetGoogle Scholar
  36. Schick A (1986) On asymptotically efficient estimation in semiparametric models. Ann Stat 14(3):1139–1151CrossRefzbMATHMathSciNetGoogle Scholar
  37. Stein C (1956) Efficient nonparametric testing and estimation. In: Proceedings of the third Berkeley symposium on mathematical statistics and probability 1954–1955, vol. I. University of California Press, Berkeley/Los Angeles, pp 187–195Google Scholar
  38. Stone CJ (1975) Adaptive maximum likelihood estimators of a location parameter. Ann Stat 3:267–284CrossRefzbMATHGoogle Scholar
  39. Strasser H (1996) Asymptotic efficiency of estimates for models with incidental nuisance parameters. Ann Stat 24(2):879–901CrossRefzbMATHMathSciNetGoogle Scholar
  40. Tchetgen E, Li L, Robins J, van der Vaart A (2008) Minimax estimation of the integral of a power of a density. Stat Probab Lett 78(18):3307–3311CrossRefzbMATHGoogle Scholar
  41. van der Vaart AW (1988) Estimating a real parameter in a class of semiparametric models. Ann Stat 16(4):1450–1474CrossRefzbMATHGoogle Scholar
  42. van der Vaart A (1991) On differentiable functionals. Ann Stat 19(1):178–204CrossRefzbMATHGoogle Scholar
  43. van der Vaart A (1996) Efficient maximum likelihood estimation in semiparametric mixture models. Ann Stat 24(2):862–878CrossRefzbMATHGoogle Scholar
  44. van Eeden C (1970) Efficiency-robust estimation of location. Ann Math Stat 41:172–181CrossRefzbMATHGoogle Scholar
  45. Wellner JA, Klaassen CAJ, Ritov Y (2006) Semiparametric models: a review of progress since BKRW (1993). In: Frontiers in statistics. Imperial College Press, London, pp 25–44Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.ETH ZürichZürichSwitzerland

Personalised recommendations