Abstract
When a decision maker (DM) contracts with an expert to provide information, the nature of the contract can, create incentives for the expert, and it is up to the DM to ensure that the contract provides incentives that align the expert’s and DM’s interests. In this paper, scoring rules (and related functions) are viewed as such contracts and are reinterpreted in terms of agency theory and the theory of revelation games from economics. Although scoring rules have typically been discussed in the literature as devices for eliciting and evaluating subjective probabilities, this study relies on the fact that strictly proper scoring rules reward greater expertise as well as honest revelation. We describe conditions under which a DM can use a strictly proper scoring rule as a contract to give an expert an incentive to gather an amount of information that is optimal from the DM’s perspective. The conditions we consider focus on the expert’s cost structure, and we find that the DM must have substantial knowledge of that cost structure in order to design a specific contract that provides the correct incentives. The model and analysis suggest arguments for hiring and maintaining experts in-house rather than using outside consultants.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cervera, J. L. (1996). Discussion of scoring rules and the evaluation of probabilities,Test, 5:27–31.
Clemen, R. T. andWinkler, R. L. (1985). Limits for the precision and value of information from dependent sources.Operations Research, 33:427–442.
Conroy, R. M. andHughes, J. S. (1987). Delegated information gathering decisions.Accounting Review, 62:50–66.
Cooke, R. M. (1991).Experts in Uncertainty: Expert Opinion and Subjective Probability in Science. Oxford University Press, New York.
De Finetti, B. (1962). Does it make sense to speak of good probability appraisers? InThe Scientist Speculates: An Anthology of Partly Baked Ideas, pp. 357–364. Basic Books, New York.
DeGroot, M. H. andFienberg, S. E. (1982). Assessing probability assessors: Calibration and refinement. In S. Gupta and e. J. O. Berger, eds.,Statistical Decision Theory an Related Topics III, Vol. 1, pp. 291–314. Academic Press, New York.
DeGroot, M. H. andFienberg, S. E. (1983). The comparison and evaluation of forecasters.The Statistician, 32:12–22.
Grossman, S. J. andHart, O. (1983). An analysis of the principal-agent problem.Econometrica, 51:7–46.
Holmstrom, B. (1979). Moral hazard and observability.Bell Journal of Economics, pp. 74–91.
Jensen, M. C. andMeckling, W. H. (1976). Theory of the firm: Managerial behavior, agency costs and ownership structure.Journal of Financial Economics, 3:305–360.
Kadane, J. B. andLarkey, P. D. (1982). Subjective probability and the theory of games.Management Science, 28:113–120.
Kadane, J. B. andWinkler, R. L. (1988). Separating probability elicitation from utilities.Journal of the American Statistical Association, 83:357–363.
Kreps, D. (1990).A Course in Microeconomic Theory, Princeton University Press, Princeton.
LaValle, I. H. (1968). On cash equivalents and information evaluation in decisions under uncertainty.Journal of the American Statistical Association, 63:252–290.
Meyer, M. A. andBooker, J. M. (1991).Eliciting and Analyzing Expert Judgment: A Practical Guide, Academic Press, London.
Morgan, M. G. andHenrion, M. (1990).Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis, Cambridge University Press, Cambridge.
Murphy, A. H. andWinkler, R. L. (1984). Probability forecasting in meteorology.Journal of the American Statistical Association, 79:489–500.
Myerson, R. B. (1977). Incentive compatibility and the bargaining problem,Econometrica, 47:61–73.
Myerson, R. B. (1991).Game Theory: Analysis of Confict. Harvard University Press. Cambridge.
Raiffa, H. andSchlaifer, R. (1961).Applied Statistical Decistion Theory, Graduate School of Business Administration, Harvard University, Cambridge.
Ross, S. (1973). The economic theory of agency: The principal’s problem.American Economic Review, 63:134–139.
Rothschild, M. andStiglitz, J. E. (1970). Increasing risk: I.A. definition.Journal of Economic Theory, 2:225–243.
Savage, L. J. (1971). The elicitation of personal probabilities and expectations.Journal of the American Statistical Association, 66:783–801.
Schervish, M. J. (1989). A general method for comparing probability assessors.Annals of Statistics, 17:1856–1879.
Shuford, E. H., Albert, A., andMassengill, H. E. (1966). Admissible probability measurement procedures.Psychometrica, 31:125–145.
Stael Von Holstein, C. A. S. (1970).Assessment and Evaluation of Subjective Probability Distributions, The Economic Research Institute at the Stockholm School of Economics, Stockholm.
Winkler, R. L. (1967). The quantification of judgment: Some methodological suggestions.Journal of the American Statistical Association, 62:1105–1120.
Winkler, R. L. (1977). Rewarding experitise in probability assessment. In H. Jungermann and G. de Zeeuw, eds.,Decision-Making and Change in Human Affairs, pp. 127–140. D. Reidel, Dordrecht.
Winkler, R. L. (1986). On “good probability appraisers”. In P. K. Goel and A. Zellner, eds.,Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti, pp. 265–278, North-Holland, Amsterdam.
Winkler, R. L. (1996). Scoring rules and the evaluation of probabilities.Test, 5:1–26.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Clemen, R.T. Incentive contrats and strictly proper scoring rules. Test 11, 167–189 (2002). https://doi.org/10.1007/BF02595735
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF02595735