Statistics and Computing

, Volume 1, Issue 2, pp 105–117 | Cite as

Bayesian analysis of outlier problems using the Gibbs sampler

  • Isabella Verdinelli
  • Larry Wasserman


We consider the Bayesian analysis of outlier models. We show that the Gibbs sampler brings considerable conceptual and computational simplicity to the problem of calculating posterior marginals. Although other techniques for finding posterior marginals are available, the Gibbs sampling approach is notable for its ease of implementation. Allowing the probability of an outlier to be unknown introduces an extra parameter into the model but this turns out to involve only minor modification to the algorithm. We illustrate these ideas using a contaminated Gaussian distribution, at-distribution, a contaminated binomial model and logistic regression.


Contaminated normal Gibbs sampling Monte Carlo outliers posterior marginals 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Abraham, B. and Box, G. E. P. (1978) Linear models and spurious observations.Applied Statistics,27, 131–138.Google Scholar
  2. Becker, R. A., Chambers, J. M. and Wilks, A. R. (1988)The New S Language, Wadsworth, Pacific Grove, California.Google Scholar
  3. Box, G. E. P. and Tiao, G. C. (1968) A Bayesian approach to some outlier problems.Biometrika,55, 119–129.Google Scholar
  4. Carlin, B., Gelfand, A. E. and Smith, A. F. M. (1989) Hierarchical Bayes analysis of change point problems. To appear inApplied Statistics.Google Scholar
  5. Chaloner, K. and Brant, R. (1988) A Bayesian approach to outlier detection and residual analysis.Biometrika,75, 651–660.Google Scholar
  6. Clark, J. P. and Tifft, L. L. (1966) Polygraph and interview validation of self reported deviant behavior.American Sociological Review,31, 516–523.Google Scholar
  7. Copas, J. B. (1988) Binary regression models for contaminated models.Journal of the Royal Statistical Society Series B,50, 225–265.Google Scholar
  8. Dalal, S. R., Fowlkes, E. B. and Hoadley, B. (1969) Risk analysis of the space shuttle: pre-Challenger prediction of failure.Journal of the American Statistical Association,84, 945–957.Google Scholar
  9. Davison, A. C. (1988) Discussion on Copas.Journal of the Royal Statistical Society Series B,50, 258–259.Google Scholar
  10. DeGroot, M. H. (1970)Optimal Statistical Decisions. McGraw-Hill, New York.Google Scholar
  11. DeVroye, L. (1986)Non-uniform Random Variate Generation. Springer-Verlag, New York.Google Scholar
  12. Ekholm, A. and Palmgren, J. (1982) A model for a binary response with misclassification.GLIM-82: Proceedings of the International Conference on Generalized Linear Models, R. Gilchrist (Ed), Springer-Verlag, Heidelberg, pp. 128–43.Google Scholar
  13. Fisher, R. A. (1960)The Design of Experiments (7th edn), Oliver and Boyd, Edinburgh.Google Scholar
  14. Fraser, D. A. S. (1979)Inference and Linear Models, McGraw-Hill, New York.Google Scholar
  15. Freeman, P. R. (1980) On the number of outliers in data from a linear model, inBayesian Statistics, Proceedings of the First International Meeting held in Valencia (Spain), J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith (eds), 347–382.Google Scholar
  16. Gelfand, A. E. and Smith, A. F. M. (1990) Sampling based approaches to calculating marginal densities.Journal of the American Statistical Association.Google Scholar
  17. German, S. and Geman, D. (1984) Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images.IEEE Transactions on Pattern Analysis and Machine Intelligence,6, 721–741.Google Scholar
  18. Gould, L. L. (1969) Who defines delinquency: A comparison of self-reported and officially reported indices of delinquency for three racial groups.Social Problems,16, 325–336.Google Scholar
  19. Guttman, I., Dutter, R. and Freeman, P. (1978) Care and handling of univariate outliers in the general linear model to detect spuriosity—a Bayesian approach.Technometrics,20, 187–194.Google Scholar
  20. Lange, K. L., Little, R. J. A. and Taylor, J. M. G. (1989) Robust statistical modeling using thet distribution.Journal of the American Statistical Association,84, 881–896.Google Scholar
  21. McCullagh, P. and Nelder, J. A. (1989)Generalized Linear Models (2nd edn), Chapman and Hall, London.Google Scholar
  22. O'Hagan, A. (1988) Discussion on Copas.Journal of the Royal Statistical Society Series B,50, 259.Google Scholar
  23. Palmgren, J. and Ekholm, A. (1987) Exponential family nonlinear models for categorical data with error of observation.Appl. Stochast. Model Data Anal.,3, 111–124.Google Scholar
  24. Pettit, L. I. (1988) Bayes methods for outliers in exponential samples.Journal of the Royal Statistical Society Series B,50, 371–380.Google Scholar
  25. Pettit, L. I. and Smith, A. F. M. (1984)Bulletin of the International Statistical Institute: Proceedings of the 44th session, 292–306.Google Scholar
  26. Pettit, L. I. and Smith, A. F. M. (1985) Outliers and influential observations in linear models, inBayesian Statistics 2, J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith (eds), North-Holland, Amsterdam, pp. 473–494.Google Scholar
  27. Pregibon, D. (1981) Logistic regression diagnostics.Ann. Statist. 9, 705–724.Google Scholar
  28. Pregibon, D. (1982) Resistant fits for some commonly used logistic models with medical applications.Biometrics,38, 485–498.Google Scholar
  29. Tanner, M. and Wong, W. (1987) The calculation of posterior distributions by data augmentation (with discussion).Journal of the American Statistical Association,82, 528–550.Google Scholar
  30. Tapia, R. A. and Thompson, J. R. (1978)Nonparametric Probability Density Estimation, Johns Hopkins University Press, Baltimore, Maryland.Google Scholar
  31. Titterington, D. M., Smith, A. F. M. and Makov, U. E. (1985)Statistical Analysis of Finite Mixture Distributions, Wiley, New York.Google Scholar
  32. West, M. (1984) Outlier models and prior distributions in Bayesian linear regression.J. R. Statist. B.,46, 431–439.Google Scholar
  33. West, M. (1987) On scale mixtures of normal distributions.Biometrika,74, 646–648.Google Scholar
  34. Winkler, R. L. and Gaba, A. (1990) Inference with imperfect sampling from a Bernoulli process, inBayesian and Likelihood Methods in Statistics and Econometrics, S. Geisser, J. S. Hodges, S. J. Press and A. Zellner (eds), North-Holland, Amsterdam.Google Scholar
  35. Zeger, S. L. and Karim, M. R. (1991) Generalized linear models with random effects: a Gibbs sampling approach.Journal of the American Statistical Association,86, 79–86.Google Scholar

Copyright information

© Chapman & Hall 1991

Authors and Affiliations

  • Isabella Verdinelli
    • 1
    • 2
  • Larry Wasserman
    • 2
  1. 1.Department of StatisticsUniversity of RomeRomeItaly
  2. 2.Department of StatisticsCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations