Skip to main content
SpringerLink
Log in

Robust nonorthogonal analyses revisited: An update based on trimmed means

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

Three approaches to the analysis of main and interaction effect hypotheses in nonorthogonal designs were compared in a 2×2 design for data that was neither normal in form nor equal in variance. The approaches involved either least squares or robust estimators of central tendency and variability and/or a test statistic that either pools or does not pool sources of variance. Specifically, we compared the ANOVA F test which used trimmed means and Winsorized variances, the Welch-James test with the usual least squares estimators for central tendency and variability and the Welch-James test using trimmed means and Winsorized variances. As hypothesized, we found that the latter approach provided excellent Type I error control, whereas the former two did not.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alexander, R. A., & Govern, D. M. (1994). A new and simpler approximation for ANOVA under variance heterogeneity.Journal of Educational Statistics, 19, 91–101.

    Google Scholar 

  • Algina, J., Oshima, T. C., & Tang, K. L. (1991). Robustness of Yao's, James', and Johansen's tests under variance-covariance heteroscedasticity and nonnormality.Journal of Educational Statistics, 16, 125–140.

    Google Scholar 

  • Behrens, W. V. (1929). Ein beitrag zur fehlerberechnung bei wenigen beobachtungen.Landwirtsch Jahrbucher [An essay on calculating mistakes when having free observations],68, 807–837.

    Google Scholar 

  • Box, G. E. P. (1954). Some theorems on quadratic forms applied in the study of analysis of variance problems, I. Effect of inequality of variance in the one-way classification.Annals of Mathematical Statistics, 25, 290–302.

    Google Scholar 

  • Bradley, J. V. (1978). Robustness?British Journal of Mathematical and Statistical Psychology, 31, 144–152.

    Google Scholar 

  • Brown, M. B., & Forsythe, A. B. (1974). The small sample behavior of some statistics which test the equality of several means.Technometrics, 16, 129–132.

    Google Scholar 

  • Cressie, N. A. C., & Whitford, H. J. (1986). How to use the two samplet-test.Biometrical Journal, 28, 131–148.

    Google Scholar 

  • D'Agostino, R. B., Belanger, A., & D'Agostino, R. B., Jr. (1990). A suggestion for using powerful and informative tests of normality.The American Statistician, 44, 316–321.

    Google Scholar 

  • Fenstad, G. U. (1983). A comparison between U and V tests in the Behrens-Fisher problem.Biometrika, 70, 300–302.

    Google Scholar 

  • Fisher, R. A. (1935).The design of experiments. Edinburgh and London: Oliver & Boyd.

    Google Scholar 

  • Gross, A. M. (1976). Confidence interval robustness with long-tailed symmetric distributions.Journal of the American Statistical Association, 71, 409–416.

    Google Scholar 

  • Hastings, N. A. J., & Peacock, J. B. (1975).Statistical distributions: A handbook for students and practitioners. New York: Wiley.

    Google Scholar 

  • Hoaglin, D. C. (1985). Summarizing shape numerically: The g- and h-distributions. In D. Hoaglin, F. Mosteller, & J. Tukey (Eds.),Exploring data tables, trends, and shapes (pp. 461–513). New York: Wiley.

    Google Scholar 

  • James, G. S. (1951). The comparison of several groups of observations when the ratios of the population variances are unknown.Biometrika, 38, 324–329.

    Google Scholar 

  • James, G. S. (1954). Tests of linear hypotheses in univariate and multivariate analysis when the ratios of the population variances are unknown.Biometrika, 41, 19–43.

    Google Scholar 

  • Johansen, S. (1980). The Welch-James approximation to the distribution of the residual sum of squares in a weighted linear regression.Biometrika, 67, 85–92.

    Google Scholar 

  • Keren, G. (1982). A balanced approach to unbalanced designs. In G. Keren (Ed.),Statistical and methodological issues in psychology and social science research (pp. 155–186). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Keselman, H. J., Carriere, K. C., & Lix, L. M. (1993). Testing repeated measures hypotheses when covariance matrices are heterogeneous.Journal of Educational Statistics, 18, 305–319.

    Google Scholar 

  • Keselman, H. J., Carriere, K. C., & Lix, L. M. (1995). Robust and powerful nonorthogonal analyses.Psychometrika, 60, 395–418.

    Article  Google Scholar 

  • Keselman, H. J., Carriere, K. C., & Lix, L. M. (1996). Errata to “Robust and powerful nonorthogonal analyses”.Psychometrika, 61, 191.

    Article  Google Scholar 

  • Keselman, H. J., Lix, L. M., & Kowalchuk, R. K. (1998). Multiple comparison procedures for trimmed means.Psychological Methods, 3, 123–141.

    Article  Google Scholar 

  • Lewis, C., & Keren, G. (1977). You can't have your cake and eat it too: Some considerations of the error term.Psychological Bulletin, 84, 1150–1154.

    Article  Google Scholar 

  • Lix, L. M., Cribbie, R., & Keselman, H. J. (1996).The analysis of completely randomized univariate designs. Paper presented at the annual meeting of the Psychometric Society, Banff, AB, Canada.

  • Lix, L. M., & Keselman, H. J. (1995).To trim or not to trim: Tests of mean equality under heteroscedasticity and nonnormality. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.

  • Lix, L. M., & Keselman, H. J. (in press). To trim or not to trim: Tests of mean equality under heteroscedasticity and nonnormality,Educational and Psychological Measurement.

  • Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures.Psychological Bulletin, 105, 156–166.

    Article  Google Scholar 

  • Milligan, G. W., Wong, D. S., & Thompson, P. A. (1987). Robustness properties of nonorthogonal analysis of variance.Psychological Bulletin, 101, 464–470.

    Article  Google Scholar 

  • Sawilowsky, S. S., & Blair, R. C. (1992). A more realistic look at the robustness and Type II error probabilities of thet test to departures from population normality.Psychological Bulletin, 111, 352–360.

    Article  Google Scholar 

  • SAS Institute. (1989).SAS/IML software: Usage and reference, version 6 (1st ed.). Cary, NC: Author.

    Google Scholar 

  • Scheffe, H. (1959).The analysis of variance. New York: Wiley.

    Google Scholar 

  • Tukey, J. W. (1960). A survey of sampling from contaminated normal distributions. In I. Olkin, W. Hoeffding, S. Ghurye, W. Madow, & H. Mann (Eds.),Contributions to probability and statistics. Stanford, CA: Stanford University Press.

    Google Scholar 

  • Welch, B. L. (1938). The significance of the difference between two means when the population variances are unequal.Biometrika, 29, 350–362.

    Google Scholar 

  • Welch, B. L. (1947). The generalization of Student's problem when several different population variances are involved.Biometrika, 34, 28–35.

    Google Scholar 

  • Welch, B. L. (1951). On the comparison of several mean values: An alternative approach.Biometrika, 38, 330–336.

    Google Scholar 

  • Wilcox, R. R. (1987). New designs in the analysis of variance.Annual Review of Psychology, 38, 29–60.

    Article  Google Scholar 

  • Wilcox, R. R. (1990). Comparing the means of two independent groups.Biometrical Journal, 32, 771–780.

    Google Scholar 

  • Wilcox, R. R. (1994a). A one-way random effects model for trimmed means.Psychometrika, 59, 289–306.

    Article  Google Scholar 

  • Wilcox, R. R. (1994b). Some results on the Tukey-McLaughlin and Yuen methods for trimmed means when distributions are skewed.Biometrical Journal, 36, 259–273.

    Google Scholar 

  • Wilcox, R. R. (1995a). ANOVA: A paradigm for low power and misleading measures of effect size?Review of Educational Research, 65, 51–77.

    Google Scholar 

  • Wilcox, R. R. (1995b). ANOVA: The practical importance of heteroscedastic methods, using trimmed means versus means, and designing simulation studies.British Journal of Mathematical and Statistical Psychology, 48, 99–114.

    Google Scholar 

  • Wilcox, R. R. (1995c). Three multiple comparison procedures for trimmed means.Biometrical Journal, 37, 643–656.

    Google Scholar 

  • Wilcox, R. R. (1996).Statistics for the social Sciences. New York: Academic Press.

    Google Scholar 

  • Wilcox, R. R., Charlin, V. L., & Thompson, K. L. (1986). New Monte Carlo results on the robustness of the ANOVA F, W and F* statistics.Communications in Statistics, Simulation and Computation, 15(4), 933–943.

    Google Scholar 

  • Yuen, K. K. (1974). The two-sample trimmedt for unequal population variances.Biometrika, 61, 165–170.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, University of Manitoba, R3T 2N2, Winnipeg, Manitoba, Canada

    H. J. Keselman, Rhonda K. Kowalchuk & Lisa M. Lix

Authors
  1. H. J. Keselman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rhonda K. Kowalchuk
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lisa M. Lix
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Financial support for this research was provided by grants to the first author from the National Sciences and Engineering Research Council of Canada (#OGP0015855) and the Social Sciences and Humanities Research Council (#410-95-0006). The authors would like to express their appreciation to the Associate Editor as well as the reviewers who provided valuable comments on an earlier version of this paper.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Keselman, H.J., Kowalchuk, R.K. & Lix, L.M. Robust nonorthogonal analyses revisited: An update based on trimmed means. Psychometrika 63, 145–163 (1998). https://doi.org/10.1007/BF02294772

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02294772

Key words

Search

Navigation