Skip to main content
SpringerLink
Log in

The need for an evolving concept of validity in industrial and personnel psychology: Psychometric, legal, and emerging issues

  • Articles
  • Published:
Current Psychology Aims and scope Submit manuscript

Abstract

The Trinitarian view of validity (namely content, criterion-related, and construct) has dominated psychology for almost a half century. In industrial and personnel psychology especially, content and criterion-related (in particular) forms of validity have served as workhorses. Such a limited focus has lead to a stagnation of the area and the inability to incorporate more theory into the research and practice of personnel psychology. Therefore, we review the evolution of the concept of validity in the psychology discipline, with a particular emphasis on industrial and personnel psychology. We then review the legal and statutory history within industrial and personnel psychology that has helped to shape the concept of validity over the years. Finally, we examine some emerging trends in industrial and personnel psychology, including the issue of graduate training, which may have important consequences on how we conceptualize validity in the future in both our research and practice of personnel psychology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aftana, M.S. (1994). On revitalizing the measurement curriculum. American Psychology [comment], 49, 889–890.

    Article  Google Scholar 

  • Aiken, L.S., West, S.G., Sechrest, L., & Reno, R.R. (1990). Graduate training in statistics, methodology, and measurement in psychology. American Psychologist, 45, 721–734.

    Article  Google Scholar 

  • Albemarle Paper Co. v. Moody, 422 U.S. 405 (1975).

  • American Psychological Association (1954). Technical recommendations for psychological tests and diagnostic techniques. Washington, D.C.: American Psychological Association (also in Psychological Bulletin, 51 [2, pt.2]).

    Google Scholar 

  • American Psychological Association (1966). Standards for educational and psychology tests and manuals. Washington, D.C.: Author.

    Google Scholar 

  • American Psychological Association, American Educational Research Association, and National Council on Measurement in Education (APA, AERA, NCME joint committee). (1985). Standards for educational and psychological testing. Washington, D.C.: American Psychological Association.

    Google Scholar 

  • Anastasi, A. (1986). Emerging concepts of test validation. Annual Review of Psychology, 37, 1–15.

    Article  Google Scholar 

  • Anastasi, A., & Urbina, S. (1997). Psychological Testing (7th ed.). Upper Saddle River, NJ: Prentice Hall.

    Google Scholar 

  • Angoff, W.H. (1988). Validity: An evolving concept. In H. Wainer, & H.I. Braun (Eds.) Test validity, (pp. 19–32). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Balma, M.J. (1959). The concept of synthetic validity. Personnel Psychology, 12, 395–396.

    Article  Google Scholar 

  • Biddle, R.E. (1995). Disparate impact reference trilogy for statistics. Labor Law Journal, 46, 651–660.

    Google Scholar 

  • Binel, A., & Simon, T. (1916). The development of intelligence in children (trans. E. S. Kite). Baltimore: Williams & Wilkins.

    Google Scholar 

  • Binning, J.F., & Barrett, G.V. (1989). Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology, 74, 478–494.

    Article  Google Scholar 

  • Black Law Enforcement v. City of Akron, 824 F.2d 475 (1987).

  • Bouman v. Block. 940 F. 2d 1211 (9th Cir., 1991).

  • Bridgman, P.W. (1927). The logic of modern physics. New York: Macmillan

    Google Scholar 

  • Burke, M.J. (1993). Computerized psychological testing: Impacts on measuring predictor constructs and future job behavior. In N. Schmitt, W.C. Borman, and Assoc. (Eds.). Personnel selection in organizations, pp. 203–239. San Francisco, CA: Jossey-Bass Publishers.

    Google Scholar 

  • Campbell, D.T. (1960). Recommendations for APA test standards regarding construct, trait, or discriminant validity. American Psychologist, 15, 546–553.

    Article  Google Scholar 

  • Campbell, D. T., & Fiske, D.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.

    Article  PubMed  Google Scholar 

  • Campbell, J.P. (1990a). Modeling the performance prediction problem in industrial and organizational psychology. In M. D. Dunnette & L. M. Hough (Eds.) Handbook of industrial and organizational psychology (2nd ed., Vol. 1; pp. 687–732). Palo Alto, CA: Consulting Psychologist Press.

    Google Scholar 

  • Campbell, J.P. (1990b). The role of theory in industrial and organizational psychology. In M.D. Dunnette & L. M. Hough (Eds.). Handbook of industrial and organizational psychology (2nd ed., Vol. 1; pp. 39–73). Palo Alto, CA: Consulting Psychologist Press.

    Google Scholar 

  • Civil Rights Act of 1964; 42 U.S.C. 2000e, et seq., sec. 703.

  • Civil Rights Act of 1991; 42 U.S.C. P.L. 102-166; 105 Stat 1071.

  • Clady v. County of Los Angeles, 770 F. 2d 1421 (9th Circ, 1985).

  • Cone, J.D., & Foster, S.L. (1991). Training in measurement: Always the bridesmaid. American Psychologist, 46, 653–654.

    Article  Google Scholar 

  • Colbert v. H-K Corporation, 1971 WL 215 (N.D.Ga. 1971).

  • Cronbach, L.J. (1980). Validity on parole: How can we go straight? New Directions for Testing and Measurement, 5, 99–108.

    Google Scholar 

  • Cronbach, L.J. (1988). Five perspectives on the validity argument. In H. Wainer, & H.I. Braun (Eds.) Test validity (pp. 13–18). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Cronbach, L.J., & Meehl, P.E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.

    Article  PubMed  Google Scholar 

  • Dauben v. Merrell Dow Pharmaceuticals, 509 U.S. 579 (1993).

  • Davis v. City of Dallas, 111 F.2d 205 (1985).

  • Debra P. v. Turlington, 644 F. 2d 397 (5th Cir., 1981).

  • Dothardv. Rawlinson, 433 U.S. 321 (1977).

  • Douglas v. Hampton, CD 313-71 (1975).

  • EEOC v. Atlas Paper Box C. 868 F. 2d 487 (6th Circ.) cert denied 58 U.S. L.W. 3213 (1989).

  • Garrett, H.E. (1937). Statistics in psychology and education. New York: Longmans, Green.

    Google Scholar 

  • Ghiselli, E.E. (1966). The validity of occupational aptitude tests. New York: John Wiley and Sons.

    Google Scholar 

  • Gillespie v. State of Wisconsin, 771 F. 2d 1035 (7th Cir. 1985).

  • Green, B.F. (1988). Construct validity of computer-based tests. In H. Wainer, & H.I. Braun (Eds.) Test validity (pp. 77–86). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Griggs v. Duke Power Company, 401 U.S. 424 (1971).

  • Guardians Association of New York City v. Civil Service Commission of the City of New York, 630 F. 2d 79 (2nd Cir. 1980a).

  • Guardians Association of New York City v. Civil Service Commission of the City of New York, 633 F. 2d 232 (2nd Cir. 1980b).

  • Guardians Association of New York City v. Civil Service Commission of the City of New York, 463 U.S. 582(1983).

  • Guilford, J.P. (1946). New standards for test evaluation. Educational and Psychological Measurement, 6, 427–439.

    Google Scholar 

  • Guion, R.M. (1978a). Scoring of content domain samples: The problem of fairness. Journal of Applied Psychology, 63, 499–506.

    Article  Google Scholar 

  • Guion, R.M. (1978b). “Content validity” in moderation. Personnel Psychology, 31, 205–213.

    Article  Google Scholar 

  • Guion, R.M. (1980). On trinitarian doctrines of validity. Professional Psychology, 11, 385–398.

    Article  Google Scholar 

  • Guion, R.M. (1991). Personnel assessment. In M. D. Dunnette & L. M. Hough (Eds.) Handbook of industrial and organizational psychology (2nd ed., Vol. 2; pp. 327–398). Palo Alto, CA: Consulting Psychologist Press.

    Google Scholar 

  • Hambleton, R.K. (1989). Principles and selected applications of item response theory. In R.L. Linn (Ed.) Educational measurement (3rd ed.). New York: American Council on Education and Macmillan.

    Google Scholar 

  • Harless v. Duck, 619 F.2d 611 (1980).

  • Hollenbeck, J.R., & Whitener, E.M. (1988). Criterion-related validation for small sample contexts: An integrated approach to synthetic validity. Journal of Applied Psychology, 73, 536–544.

    Article  Google Scholar 

  • Hubley, A.M., & Zumbo, B.D. (1996). A dialectic on validity: Where we have been and where we are going. Journal of General Psychology, 123, 207–216.

    Article  Google Scholar 

  • Hunter, J.E., & Schmidt, F.L. (1990). Methods ofmeta-analysis: Correcting for error and bias in research findings. Newbury Park, CA: Sage Publications.

    Google Scholar 

  • In Re Vey, 124 N.J. 534, 591 A.2d 1333 (1991; appeal after remand 272 N.J. supra 199, 639 A.2d 724 (1993); affirmed, 135 N.J. 306, 639 A.2d 718 (1994).

  • Lambert, N.M. (1991). The crisis in measurement literacy in psychology and education. Educational Psychologist, 26, 23–35.

    Article  Google Scholar 

  • Landy, F.L. (1986). Stamp collecting versus science: Validation as hypothesis testing. American Psychologist, 41, 1183–1192.

    Article  Google Scholar 

  • Lees-Haley, P. R., & McDonald, J. J., Jr. (1997). The use (and misuse) of psychological testing in employment litigation. Employee Relations, 23, 37–53.

    Google Scholar 

  • Linn, R. (1994). Performance assessment: Policy, promises and technical measurement standards. Educational Researcher, 23 (9), 4–14.

    Google Scholar 

  • London, M., & Moses, J. L. (1990). The changing roles of the industrial/organizational psychologist: From analyst/technician to change agent/strategist. The Industrial and Organizational Psychologist, 27 (3), 17–26.

    Google Scholar 

  • Mael, F.A. (1991). A conceptual rationale for the domain and attributes of biodata items. Personnel Psychology, 44, 763–792.

    Article  Google Scholar 

  • Mael, F.A., & Hirsch, A.C. (1993). Rainforest empiricism and quasi-rationality: Two approaches to object biodata. Personnel Psychology, 46, 719–739.

    Article  Google Scholar 

  • Meier, S.T. (1993). Revitalizing the measurement curriculum: Four approaches for emphasis in graduate education. American Psychologist, 48, 886–891.

    Article  Google Scholar 

  • Messick, S. (1988). The once and future issues of validity: Assessing the meaning and consequences of measurement. In H. Wainer, & H.I. Braun (Eds.), Test validity (pp. 33–46). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Messick, S. (1989). Validity. In R.L. Linn (Ed.), Educational Measurement (3rd ed., pp. 13–103). New York: American Council on Education/Macmillan.

    Google Scholar 

  • Michlitsch, J. F. (1995). Guidelines related to disparate impact, burden of proof, and the 1991 Civil Rights Act: Analysis of federal court cases up to Wards Cove. Labor Law Journal, 46, 570–582.

    Google Scholar 

  • Mosier, C. I. (1947). A critical examination of the concepts of face validity. Educational and Psychological Measurement, 7, 191–205.

    Article  PubMed  Google Scholar 

  • Moss, P.A. (1992). Shifting conceptions of validity in educational measurement: Implications for performance assessment. Review of Educational Research, 62, 229–258.

    Article  Google Scholar 

  • Motorola, Inc. v. Illinois FEPC, 215 N.E. 2d 286 (111. 1966).

  • Muthén, B. (1988). Some uses of structural equation modeling in validity studies: Extending IRT to external variables. In H. Wainer, & H.I. Braun (Eds.), Test validity, (pp. 213–238). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.

    Google Scholar 

  • Myart v. Motorola, 110 Cong. Rec. 5662.

  • Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill Book Company.

    Google Scholar 

  • Pegues v. Miss. State Employment Service, 488 F. Supp. 239 (E. D. Miss. 1980).

  • Pyburn, K.M., Jr. (1996, March). Are we now, where we were then—The Bell Curve and the Gingrich Revolution—Implications for employment testing litigation. Paper presented at the annual conference of the Personnel Testing Council of Northern California, Sacramento, CA.

  • Rivera v. City of Wichita Falls, 665 F. 2d 531 (5th Cira, 1982).

  • Schmidt, F. L., & Hunter, J. E. (1977). Development of a generalized solution to the problem of validity generalization. Journal of Applied Psychology, 62, 529–540.

    Article  Google Scholar 

  • Schmitt, N., & Landy, F. (1993). The concept of validity. In N. Schmitt, W. Borman, and Associates, Personnel selection in organization (pp. 275–309). San Francisco: Jossey-Bass.

    Google Scholar 

  • Schmitt, N., & Stults, D.M. (1986). Methodology review: Analysis of multitrait-multimethod matrices. Applied Psychological Measurement, 10, 1–22.

    Article  Google Scholar 

  • Shepard, L.A. (1993). Evaluating test validity. In L. Darling-Hammond (Ed.). Review of research in education (Vol. 19), pp. 405–450). Washington, D.C.: American Educational Research Association.

    Google Scholar 

  • Shore, T.H., Shore, L.M., & Thornton, G.C., III (1992). Construct validity of self-and peer-evaluations of performance dimensions in an assessment center. Journal of Applied Psychology, 77, 42–54.

    Article  Google Scholar 

  • Shultz, K.S. (1996). Distinguishing personality and biodata items using confirmatory factor analysis of multitrait-multimethod matrices. Journal of Business and Psychology, 10, 263–288.

    Article  Google Scholar 

  • Skinner, B.F. (1945). The operational analysis of psychological terms. Psychological Review, 52, 270–294.

    Article  Google Scholar 

  • Society for Industrial and Organizational Psychology, Inc. (SIOP) (1987). Principles for the validation and use of personnel selection procedures (3rd ed.). College Park, MD: Author.

    Google Scholar 

  • Spurlock v. United Airlines, 475 F.2d 216 10th Cir. (1972).

  • Tenopyr, M.L. (1977). Content-construct confusion. Personnel Psychology, 30, 47–54.

    Article  Google Scholar 

  • Turban, D.B., Sanders, P.A., Francis, D.J., & Osburn, H.G. (1989). Construct equivalence as an approach to replacing cognitive ability selection tests. Journal of Applied Psychology, 74, 62–71.

    Article  Google Scholar 

  • Tyler, R.W. (1934). Constructing achievements tests. Columbus, OH: Bureau of Educational Research, Ohio State University.

    Google Scholar 

  • Uniform Guidelines on Employee Selection Procedures. (1978). Federal Register, 43 (116), 38296-38308.

    Google Scholar 

  • Vale, C.D. (1991). Computerized adaptive testing. In J.W. Jones, B.D. Steffy, & D.W. Bray (Eds.), Applying psychology in business. Lexington, MA: Lexington Books.

    Google Scholar 

  • Van Aken v. Young, 451 F.Supp 448,454 E.D. Mich. (1982) aff 'd 750 F.2d 43 (6th Cir., 1984).

  • Vance, R.J., MacCallum, R.C., Coovert, M.D., & Hedge, J.W. (1998). Construct validity of multiple job performance measures using confirmatory factor analysis. Journal of Applied Psychology, 73, 74–80.

    Article  Google Scholar 

  • Viswesvaran, C., & Ones, D. S. (1995). Theory testing: Combining psychometric meta-analysis and structural equation modeling. Personnel Psychology, 48, 865–885.

    Article  Google Scholar 

  • Wainer, H., Dorans, N.J., Green, B.F., Mislevy, R.J., Steinberg, L., & Thissen, D. (1990). Future challenges. In H. Wainer (Ed.), Computerized adaptive testing: A primer. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Wards Cove Packing Co. v. Atonio, 109 S. Ct. 2115 (1989).

  • Washington v. Davis, 426 U.S. 229 (1976).

  • Watson v. Fort Worth Bank and Trust, 487 U.S. 977 (1982).

  • Widaman, K.F. (1985). Hierarchically nested covariance models for multitrait-multimethod data. Applied Psychological Measurement, 9, 1–26.

    Article  Google Scholar 

  • Willingham, W.W. (1988). Testing handicapped people—The validity issue. In H. Wainer, & H.I. Braun (Eds.). Test validity (pp. 89–104). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. California State University, San Bernardino, USA

    Kenneth S. Shultz, Matt L. Riggs & Janet L. Kottke

Authors
  1. Kenneth S. Shultz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matt L. Riggs
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Janet L. Kottke
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shultz, K.S., Riggs, M.L. & Kottke, J.L. The need for an evolving concept of validity in industrial and personnel psychology: Psychometric, legal, and emerging issues. Curr Psychol 17, 265–286 (1998). https://doi.org/10.1007/s12144-998-1011-x

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12144-998-1011-x

Keywords

Search

Navigation