Skip to main content
SpringerLink
Log in

Meta-Analysis

  • Chapter
Advances in Educational and Psychological Testing: Theory and Applications

Part of the book series: Evaluation in Education and Human Services Series ((EEHS,volume 28))

Abstract

Scientists have known for centuries that little can be proven from single studies. The elimination of alternate hypotheses requires the work of many researchers in many studies. Because of the unavoidable presence of sampling error, this problem is magnified when the basic data in the study are statistical. Yet the dominant pattern of review papers in the behavioral and social sciences has been to review each study on an individual basis with little or no attention to the role of sampling error in producing variation in findings between studies. Instead, variation in results from study to study has typically been taken at face value or ascribed to methodological weaknesses in some of the studies. Most reviews conclude that “more research is needed,” although many studies may already be available on the hypothesis in question. The thesis of this chapter is that, in the long run, the only solution to the problem of sampling error inherent in small sample studies is cumulation of results across studies. The subject of the present chapter is methods for conducting such cumulative analyses.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • American Educational Research Association, American Psychological Association, National Council on Measurement in Education (1985). Standards for educational and psychological testing. Washington, DC: American Psychological Association.

    Google Scholar 

  • American Psychological Association, Division of Industrial and Organizational Psychology (Division 14). (1980). Principles for the validation and use of personnel selection procedures, 2nd ed. Berkeley, CA: Author.

    Google Scholar 

  • Anastasi A. (1982). Psychological testing, 5th ed. New York: Macmillan.

    Google Scholar 

  • Bartlett, C. J., Bobko, P., Mosier, S. B., & Hannan, R. (1978). Testing for fairness with a moderated multiple regression strategy: An alternative to differential analysis. Personnel Psychology 31:233–241.

    Article  Google Scholar 

  • Boehm, V. R. (1977). Differential prediction: A methodological artifact? Journal of Applied Psychology 62:146–154.

    Article  Google Scholar 

  • Boster, F. J., & Hunter, J.E. (1979). A causal model of the trial process. Paper presented at the Western Forensic Association.

    Google Scholar 

  • Callender, J. C, & Osburn, H. G. (1980). Development and test of a new model of validity generalization. Journal of Applied Psychology 65:543–558.

    Article  Google Scholar 

  • Cohen, J. (1962). The statistical power of abnormal-social psychological research: a review. Journal of Abnormal and Social Psychology 65:145–153.

    Article  Google Scholar 

  • Cohen, J. (1969). Statistical power analyses for the behavioral sciences. New York: Academic Press.

    Google Scholar 

  • Dillard, J.P., Hunter, J.E., & Burgoon, M. (1984). Sequential requests, persuasive message strategies: A meta-analysis of foot-in-door and door-in-the-face. Human Communication Research 10:461–488.

    Article  Google Scholar 

  • Dye, D. (1982). Validity generalization analysis for data from 16 studies participating in a consortium study. Unpublished manuscript, Department of Psychology, George Washington University, Washington, DC.

    Google Scholar 

  • Fisher, C. D., & Gittelson, R. (1983). A meta-analysis of the correlates of role conflict and ambiguity. Journal of Applied Psychology 68:320–333.

    Article  Google Scholar 

  • Ghiselli, E.E. (1949). The validity of commonly employed occupational tests. University of California Publications in Psychology 5:253–288.

    Google Scholar 

  • Glass, G.V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher 5:3–8.

    Article  Google Scholar 

  • Glass, G.V. (1978). Integrating findings: The meta-analysis of research. In L. Shulman (ed.), Review of research in education, Vol. V. Itasca, IL.: Peacock, pp. 351–379.

    Google Scholar 

  • Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage Publications.

    Google Scholar 

  • Glass, G.V., & Smith, M.L. (1979). Meta-analysis of research on class size and achievement. Educational Evaluation and Policy Analysis 1:2–16.

    Google Scholar 

  • Guzzo, R. A., Jette, R. D., & Katzell, R. A. (1985). The effects of psychologically based intervention programs on worker productivity: A meta-analysis. Personnel Psychology 38:275–292.

    Article  Google Scholar 

  • Hocket, R. D. & Guion, R. M. (1985). A re-evaluation of the absenteeism-job satisfaction relationship. Organizational Behavior and Human Decision Processes 35:340–381.

    Article  Google Scholar 

  • Hunter, J.E. (1979, September). Path analysis: Longitudinal studies and causal analysis in program evaluation. Invited address presented at the American Psychological Association Meeting, New York.

    Google Scholar 

  • Hunter, J.E. (1980). Factor analysis. In P. R. Monge & J.N. Capella (eds.), Multivariate techniques in human communication research. New York: Academic Press.

    Google Scholar 

  • Hunter, J.E., & Gerbing, D.W. (1982). Unidimensional measurement, second order factor analysis, and causal models. In B. Staw & L. L. Cummings (eds.). Research in organizational behavior, Vol. 4. Greenwich, CT: JAI Press, Inc.

    Google Scholar 

  • Hunter, J.E., & Schmidt, F. L. (1978). Differential and single group validity of employment tests by race: A critical analysis of the three recent studies. Journal of Applied Psychology 63:1–11.

    Article  Google Scholar 

  • Hunter, J.E., Schmidt, F.L., & Hunter, R. F. (1979). Differential validity of employment tests by race: A comprehensive review and analysis. Psychological Bulletin 86:721–735.

    Article  Google Scholar 

  • Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982). Meta-analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage Publications.

    Google Scholar 

  • Iaffaldono, M.T., & Muchinsky, P.M. (1985). Job satisfaction and job performance: A meta-analysis. Psychological Bulletin 97:251–273.

    Article  Google Scholar 

  • Jackson, S. E., & Schuler, R. S. (1985). A meta-analysis and conceptual critique of research on role ambiguity and role conflict in work settings. Organizational Behavior and Human Decision Processes 36:16–78.

    Article  Google Scholar 

  • Katzell, R.A., & Dyer, F.J. (1977). Differential validity revived. Journal of Applied Psychology 62:137–145.

    Article  Google Scholar 

  • King, L. M., Hunter, J.E., & Schmidt, F. L. (1980). Halo in a multidimensional forced choice performance evaluation scale. Journal of Applied Psychology 65:507–516.

    Article  Google Scholar 

  • Kraiger, K., & Ford, J. K. (1985). A meta-analysis of race effects in performance ratings. Journal of Applied Psychology 70:56–65.

    Article  Google Scholar 

  • Lent, R. H., Aurbach, H. A., & Levin, L.S. (1971). Research design and validity assessment. Personnel Psychology 24:247–274.

    Article  Google Scholar 

  • Light, R.J., & Smith, P. V. (1971). Accumulating evidence: Procedures for resolving contradictions among different research studies. Harvard Educational Review 41:429–471.

    Google Scholar 

  • Linn, R. L., & Dunbar, S.B. (1982, November). Validity generalization and predictive bias. Paper presented at the Fourth Johns Hopkins University National Symposium on Educational Research, Washington, DC.

    Google Scholar 

  • Linn, R. L., Harnisch, D. L., & Dunbar S. B. (1981). Validity generalization and situational specificity: An analysis of the prediction of first year grades in law school. Applied Psychological Measurement 5:281–289.

    Article  Google Scholar 

  • Loher, B. T., Noe, R. A., Moeller, N., & Fitzgerald, M. P. (1985). A meta-analysis of the relationship of job characteristics to job satisfaction. Journal of Applied Psychology 70:280–289.

    Article  Google Scholar 

  • Mabe, P. A., III, & West, S. G. (1982). Validity of self evaluations of ability: A review and meta-analysis. Journal of Applied Psychology 67:280–296.

    Article  Google Scholar 

  • Maloley et al. v. Department of National Revenue. (1986, February). Canadian Civil Service Appeals Board, Ottawa, Canada.

    Google Scholar 

  • McDaniel, M.A., & Schmidt, F. L. (1985). A meta-analysis of the validity of training and experience ratings in personnel selection. Paper submitted for publication.

    Google Scholar 

  • McEvoy, G.M., & Cascio, W. F. (1985). Strategies for reducing employee turnover: A meta-analysis. Journal of Applied Psychology 70:342–353.

    Article  Google Scholar 

  • McKinney, M.W. (1984). Validity generalization pilot study (Final Report). Submitted to the U.S.E.S. Southern Test Development Field Center, Raleigh, NC.

    Google Scholar 

  • O’Connor, E. J., Wexley, K. N., & Alexander, R. A. (1975). Single group validity: Fact or fallacy? Journal of Applied Psychology 60:352–355.

    Article  Google Scholar 

  • Pearlman, K., Schmidt, F. L., & Hunter, J.E. (1980). Validity generalization results for tests used to predict job proficiency and training success in clerical occupations. Journal of Applied Psychology 65:373–406.

    Article  Google Scholar 

  • Peters, L. H., Harthe, D., & Pohlman, J. (1985). Fiedler’s contingency theory of leadership: An application of the meta-analysis procedures of Schmidt and Hunter. Psychological Bulletin 97:274–285.

    Article  Google Scholar 

  • Peterson, N. G. (1982, October). Investigation of validity generalization in clerical and technical/professional occupations in the insurance industry. Paper presented at the Conference on Validity Generalization, Personnel Testing Council of Southern California, Newport, Beach, CA.

    Google Scholar 

  • Petty, M.M., McGee, G. W., & Cavender, J. W. (1984). A meta-analysis of the relationship between individual job satisfaction and individual performance. Academy of Management Review 9:712–721.

    Google Scholar 

  • Premack, S., & Wanous, J. P. (1985). Meta-analysis of realistic job preview experiments. Journal of Applied Psychology 70:706–719.

    Article  Google Scholar 

  • Rosenthal, R. (1978). Combining results of independent studies. Psychological Bulletin 85:185–193.

    Article  Google Scholar 

  • Rosenthal, R. (1979). The “file drawer problem” and tolerance for null results. Psychological Bulletin 86:638–641.

    Article  Google Scholar 

  • Schmidt, F. L., Berner, J. G., & Hunter, J. E. (1973). Racial differences in validity of employment tests: Reality or illusion? Journal of Applied Psychology 58:5–9.

    Article  Google Scholar 

  • Schmidt, F. L., Gast-Rosenberg, I., & Hunter, J. E. (1980). Validity generalization results for computer programmers. Journal of Applied Psychology 65:635–642.

    Article  Google Scholar 

  • Schmidt, F. L., & Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology 62:529–540.

    Article  Google Scholar 

  • Schmidt, F. L., & Hunter, J. E. (1981). Employment testing: Old theories and new research findings. American Psychologist 36:1128–1137.

    Article  Google Scholar 

  • Schmidt, F.L., Hunter, J.E., & Caplan, J.R. (1981). Validity generalization results for two job groups in the petroleum industry. Journal of Applied Psychology 66:261–273.

    Article  Google Scholar 

  • Schmidt, F. L., Hunter, J.E., & Pearlman, K. (1982). Progress in validity generalization: Comments on Callender & Osburn and further developments. Journal of Applied Psychology 67:835–845.

    Article  Google Scholar 

  • Schmidt, F. L., Hunter, J. E., Pearlman, K., & Shane, G. S. (1979). Further tests of the Schmidt-Hunter Bayesian validity generalization procedure. Personnel Psychology 32:257–281.

    Article  Google Scholar 

  • Schmidt, F. L., Hunter, J. E., & Urry, V. W. (1976). Statistical power in criterion-related validation studies. Journal of Applied Psychology 61:473–485.

    Article  Google Scholar 

  • Schwab, D. P., Olian-Gottlieb, J.D., & Heneman, H. G. (1979). Between subjects expectancy theory research: A statistical review of studies predicting effort and performance. Psychological Bulletin 86:139–147.

    Article  Google Scholar 

  • Smith, M.L., & Glass, G.V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist 32:752–760.

    Article  Google Scholar 

  • Steele, R. P., & Ovalle, N. K. (1984). A review and meta-analysis of research on the relationship between behavioral intentions and employee turnover. Journal of Applied Psychology 69:673–686.

    Article  Google Scholar 

  • Stoffelmeyr, B.E., Dillavou, D., & Hunter, J.E. (1983). Premorbid functioning and recidivism in schizophrenia: A cumulative analysis. Journal of Consulting and Clinical Psychology 51:338–352.

    Article  Google Scholar 

  • Terborg, J.R., & Lee, T.W. (1982). Extension of the Schmidt-Hunter validity generalization procedure to the prediction of absenteeism behavior from knowledge of job satisfaction and organizational commitment. Journal of Applied Psychology 67:280–296.

    Article  Google Scholar 

  • Thorndike, E. L. (1933). The effect of the interval between test and retest on the constancy of the IQ. Journal of Educational Psychology 25:543–549.

    Article  Google Scholar 

  • U.S. Equal Employment Opportunity Commission, U.S. Civil Service Commission, U.S. Department of Labor, & U.S. Department of Justice. (1978). Uniform guidelines on employee selection procedures. Federal Register 43(166):38295–38309.

    Google Scholar 

Download references

Authors
  1. John E. Hunter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frank L. Schmidt
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Ronald K. Hambleton Jac N. Zaal

Rights and permissions

Reprints and permissions

Copyright information

© 1991 Springer Science+Business Media New York

About this chapter

Cite this chapter

Hunter, J.E., Schmidt, F.L. (1991). Meta-Analysis. In: Hambleton, R.K., Zaal, J.N. (eds) Advances in Educational and Psychological Testing: Theory and Applications. Evaluation in Education and Human Services Series, vol 28. Springer, Dordrecht. https://doi.org/10.1007/978-94-009-2195-5_6

Download citation

  • DOI: https://doi.org/10.1007/978-94-009-2195-5_6

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-94-010-7484-1

  • Online ISBN: 978-94-009-2195-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation