In this paper, modern statistics is considered as a branch of psychometrics and the question of how the central problems of statistics can be resolved using psychometric methods is investigated. Theories and methods developed in the fields of test theory, scaling, and factor analysis are related to the principle problems of modern statistical theory and method. Topics surveyed include assessment of probabilities, assessment of utilities, assessment of exchangeability, preposterior analysis, adversary analysis, multiple comparisons, the selection of predictor variables, and full-rank ANOVA. Reference is made to some literature from the field of cognitive psychology to indicate some of the difficulties encountered in probability and utility assessment. Some methods for resolving these difficulties using the Computer-Assisted Data Analysis (CADA) Monitor are described, as is some recent experimental work on utility assessment.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
Similar content being viewed by others
Card, W. I., Rusinkiewicz, M., & Phillips, C. I.Estimation of the utilities of states of health with different visual acuities using a wagering technique. Dijon, France: IF/P TC4 Working Conference on Decision Making and Medical Care, 1975.
Fischhoff, B., Slovic, P., & Lichtenstein, S.Fault trees: Sensitivity of estimated failure probabilities to problem representation (Tech. Rep. PTR-1042-77-8), 1977.
Novick, M. R., Turner, N. J., & Novick, L. R.Experimental studies of CADA-based utility assessment procedures (ONR Tech. Rep. No. 2). August, 1980.
Slovic, P. From Shakespeare to Simon: Speculations—and some evidence—about man's ability to process information.Oregon Research Institute Research Bulletin, 1972,12 (2).
Tversky, A. & Kahneman, D.The framing of decisions and the rationality of choice (ONR Tech. Rep. No. 2). March 1980.
Becker, G. M., DeGroot, M. H. & Marschak, J. Measuring utility by a single-response sequential method.Behavioral Science, 1964,9, 226–232.
Bolker, E. D. A simultaneous axiomatization of utility and subjective probability.Philosophy of Science, 1967,34, 333–340.
Coombs, C. H. Portfolio theory and the measurement of risk. In M. F. Kaplan & S. Schwartz (Eds.),Human judgment and decision processes. New York: Academic Press, Inc., 1975, 63–85.
Dawes, R. M. A case study of graduate admissions: Application of three principles of human decision making.American Psychologist, 1971,26, 180–188.
Dawes, R. M. Predictive models as a guide to preference.IEEE Transactions on Systems, Man, and Cybernetics, 1977,SMC-7, 355–358.
DeFinetti, B.Theory of Probability. London: Wiley, 1974.
Fischhoff, B., Slovic, P. & Lichtenstein, S. Knowing what you want: Measuring labile values. In T. Wallsten (Ed.),Cognitive Processes in Choice and Decision Behavior. Hillsdale, New Jersey: Erlbaum, in press.
Fishburn, P. C. A mixture-set axiomatization of conditional subjective expected utility.Econometrica, 1973,41, 1–25.
Fishburn, P. C.,Decisions and value theory. New York: John Wiley & Sons, 1964.
Fishburn, P. C. Independence in utility theory with whole product sets.Operations Research, 1965,13.
Fishburn, P. C.Utility theory for decision making. New York: John Wiley & Sons, 1970.
Friedman, M. & Savage, L. J. The utility analysis of choice involving risk.Journal of Political Economy, 1948,56, 279–304.
Herstein, I. N. & Milnor, J. An axiomatic approach to measurable utility.Econometrica, 1953,21, 291–297.
Hogarth, R. M. Cognitive processes and the assessment of subjective probability distributions.Journal of the American Statistical Association. 1975,70, 271–289.
Hull, J., Moore, P. G. & Thomas, H. Utility and its measurement.Journal of the Royal Statistical Society, Series A, 1973,136, 226–247.
Jackson, P. H., Novick, M. R. & DeKeyrel, D. F. Adversary preposterior analysis for simple parametric models. In A. Zellner (Ed.),Bayesian analysis in econometrics and statistics—Essays in honor of Harold Jeffreys. Studies in Bayesian econometric (Vol. 1). Amsterdam: North-Holland Publishing Company, 1980.
Jeffrey, R. C. New foundations for Bayesian decision theory. In Y. Bar-Hillel (Ed.),Logic, Methodology and Philosophy of Science. Amsterdam: North-Holland Publishing Co., 1965, 289–300.
Kahneman, D. & Tversky, A. Subjective probability: A judgment of representativeness.Cognitive Psychology, 1972,3, 430–454.
Kahneman, D. & Tversky, A. Prospect theory: An analysis of decision under risk.Econometrica, 1979,47, 263–291.
Keeney, D. & Raiffa, H.Decisions with multiple objectives: Preferences and value tradeoffs. New York: John Wiley & Sons, 1976.
Keeney, R. L., Utility functions for multiattributed consequences.Management Science, 1972,18, 276–287.
Keeney, R. L. Utility independence and preference for multiattributed consequences.Operations Research, 1971,19, 875–893.
Krantz, D. H., Luce, R. D., Suppes, P. & Tversky, A.Foundations of measurement. Volume 1: Additive and polynomial representations. New York: Academic Press, 1971.
Lindley, D. V. A class of utility functions.Annals of Statistics, 1976,4, 1–10.
Lindley, D. V. & Smith, A. F. M. Bayes estimates for the linear model.Journal of the Royal Statistical Society, Series B, 1972,34, 1–41.
Lindley, D. V., Tversky, A. & Brown. On the reconciliation of probability assessments.Journal of the Royal Statistical Society, Series B, 1979,
Luce, R. D.Individual choice behavior: A theoretical analysis. New York: John Wiley & Sons, 1959.
Luce, R. D. & Krantz, D. H. Conditional expected utility.Econometrica, 1971,39, 253–271.
Mosteller, F. & Nogee, P. An experimental measurement of utility.Journal of Political Economy, 1951,59, 371–404.
von Neumann, J. & Morgenstern, O.Theory of games and economic behavior. Princeton, N.J.: Princeton University Press, 1944.
Novick, M. R. A Bayesian approach to the selection of predictor variables. In C. E. Lunneborg (Ed.),Current problems and techniques in multivariate psychology. Seattle, Washington: The University of Washington, 1970.
Novick, M. R. The axioms and principal results of classical test theory.Journal of Mathematical Psychology, 1966,3, 1–18.
Novick, M. R., Chuang, D. & DeKeyrel, D. Local and regional coherence utility assessment procedures.Trabajos de Estadistica, in press.
Novick, Melvin R., Hamer, Robert M., Libby, David D., Chen, James J. & Woodworth, George G.Manual for the Computer-Assisted Data Analyais (CADA) Monitor (1980). Iowa City, Iowa: The University of Iowa, 1980.
Novick, M. R., Jackson, P. H. & Thayer, D. T. Bayesian inference and the classical test theory model: Reliability and true scores.Psychometrika, 1971,36, 261–288.
Novick, M. R. & Lindley, D. V. Fixed-state assessment of utility functions.Journal of the American Statistical Association, 1979,24, 306–311.
Novick, M. R. & Lindley, D. V. The use of more realistic utility functions in educational applications.Journal of Educational Measurement, 1978,15, 181–192.
Pratt, J. W. Risk aversion in the small and in the large.Econometrika, 1964,32, 122–136.
Pratt, J. W., Raiffa, H., & Schlaifer, R.Introduction to statistical decision theory (Prelim. Ed). New York: McGraw-Hill, 1965.
Schlaifer, R.Analysis of decisions under uncertainty. New York: McGraw-Hill, 1969.
Schlaifer, R.Computer programs for elementary decision analysis. Boston: Harvard University, 1971.
Slovic, P., Fischoff, B. & Lichtenstein, S. Behavioral decision theory.Annual Review of Psychology, 1977,28, 1–39.
Slovic, P., Lichtenstein, S. C. & Edwards, W. Boredom induced changes in preferences among bets.American Journal of Psychology, 1965,79, 427–434.
Spetzler, C. S. & Staël von Holstein, von Holstein, C-A. S. Probability encoding in decision analysis.Management Science, 1975,22, 340–358.
Swalm, R. O. Utility theory—insights into risk taking.Harvard Business Review, 1966,44, 123–136.
Torgerson, W. S.Theory and methods of scaling. New York: John Wiley & Sons, Inc., 1958.
Tversky, A. Additivity, utility, and subjective probability.Journal of Mathematical Psychology, 1967,4, 175–201.
Tversky, A. Choice by elimination.Journal of Mathematical Psychology, 1972,9, 341–367.
Tversky, A. Elimination by aspects: A theory of choice.Psychological Review, 1972,29, 281–299.
Tversky, A. On the elicitation of preferences: Descriptive and prescriptive considerations. In D. E. Bell, R. L. Keeney & H. Raiffa (Eds.),Conflicting objectives in decisions. Chichester: John Wiley & Sons, 1977, 209–222.
Tversky, A. & Kahneman, D. Judgment under uncertainty: Heuristics and biases.Science, 1974,185, 1124–1131.
Woodworth, George G. Numerical evaluation of preposterior expectations in the two-parameter normal model, with an application to preposterior consenses analysis. In A. Zellner (Ed.),Bayesian analysis in econometrics and statistics. Amsterdam: North-Holland Publishing Company, 1980.
Woodworth, George G.t for two, or preposterior analysis for two decision makers: Interval estimates for the mean.American Statistican, November, 1976.
1980 Psychometric Society presidential address.
I am indebted to Paul Slovic and David Libby for valuable consultation on the issues discussed in this paper and to Nancy Turner and Laura Novick for assistance in preparation.
Research reported herein was supported under contract number N00014-77-C-0428 from the Office of Naval Research to The University of Iowa, Melvin R. Novick, principal investigator. Opinions expressed herein reflect those of the author and not those of sponsoring agencies.