1 Introduction

Several mechanisms, often called proper scoring rules, incentivize an expert to honestly report his subjective probabilities. However, these mechanisms often assume that the expert is risk neutral or that the expert’s preferences over risk are known. Recently, Karni (2009) showed a novel mechanism that induces truth reporting even if the expert’s preferences over risk are not known. Lambert (2011) characterizes the mechanisms that induce truthful-revelation with unknown preferences. In this paper, we describe a very simple principle that can also be used to dispose the assumption that the expert’s preferences are known when beliefs are elicited.

There are two monetary rewards. The expert receives the greater reward with odds proportional to a proper scoring rule. Payments based on this principle delivers incentives for an expert to truthfully reveal what he knows, even if his attitudes toward risk are unknown.

We present two examples to illustrate the use of this principle. In the first example, we consider a traditional proper scoring rules (the Brier Score). In the second example, we consider Prelec (2004) “Truth Serum” which elicits honest opinions in the absence of any data. In both cases, the original mechanism is based on known preferences. We show how to transform these mechanisms, with the principle mentioned above, so that the assumption of risk-neutrality (or known attitudes towards risk) are disposed. While we provide no general results, we hope that these two examples illustrate the basic principle clearly.

2 The principle of eliciting beliefs by paying in chance

An event \(E\) may or may not occur next period. An expert’s (Bob) subjective probability of \(E\) is of interest. The question is how to elicit the Bob’s subjective probability \(p\) of event \(E\). Let’s say that the expert is offered the following monetary rewards

$$\begin{aligned} \begin{array}{l@{\quad }l} S(\hat{p},1)=2-(1-\hat{p})^{2} &{} \mathrm{if\, Bob\, announces\, probability }\hat{p}\,\mathrm{and}\,E\, \mathrm{occurs;} \\ S(\hat{p},0)=2-(\hat{p})^{2} &{} \mathrm{if\, Bob\, announces}\,\hat{p}\,\mathrm{and}\,E\, \mathrm{does\, not\, occur.} \end{array} \end{aligned}$$

This payment scheme \(S\) is a Brier Score, Brier (1950). The critical property of the Brier Score is that it is proper. That is, if the expert maximizes his expected payoff

$$\begin{aligned} \mathop {\max }\limits _{{\hat{p}\varepsilon \left[ {0,1} \right] }} pS\left( {\hat{p},1} \right) + \left( {1 - p} \right) S\left( {\hat{p},0} \right) \end{aligned}$$

then he optimally chooses \(\hat{p}=p.\) De Finetti (1962) describes proper scoring rules as devices that “oblige each participant to express his true feelings, because any departure from his own personal probability results in a diminution of his own average score as he sees it”. That is, a proper score induces a risk-neutral expert to reveal his subjective beliefs. Several proper scoring rules exist [see Bernardo (1979); Good (1997); Hendrickson and Buehler (1971); Kiefer (2010); Lindley (1982); Matheson and Winkler (1976); McCarthy (1956); Savage (1971); Schervish (1989); Van Lenthe (1993); Winkler (1994)].

The main difficulty with these proper scoring rules is that they assume risk-neutrality or, alternatively, that the risk-attitudes of the expert are known [i.e., the utilities of the expert must be known if the rewards \(S(\hat{p},1)\) and \(S(\hat{p},0)\) are interpreted as utils]. If Bob is assumed to be risk-neutral when he is risk-averse, then the Brier Score may induce him to misrepresent his views [see Winkler and Murphy (1970); Kadane and Winkler (1988)]. Hence, it is significant to construct devices that oblige each participant to express his true beliefs, regardless of his attitudes toward risk.

One approach is to learn the experts’ preferences through experiments [see Offerman et al (2009); Jaffray and Karni (1999)]. Another approach was recently proposed by Karni (2009). He constructs a single-stage random scoring rule that induces Bob to honestly reveal his beliefs, regardless of his attitudes toward risk. In fact, Karni (2009) only assumes probabilistic sophistication and dominance [see Machina and Schmeidler (1995)]. So, let \(x\) and \(y\) be two monetary rewards with \(x>y\). Karni (2009) assumes that Bob prefers a lottery that delivers \(x\) dollars with probability \(\mu \) and \(y\) dollars with probability \((1-\mu )\) to a lottery that delivers \(x\) dollars with probability \(\mu ^{\prime }\) and \(y\) dollars with probability \((1-\mu ^{\prime })\) if and only if \(\mu \) is greater than \(\mu ^{\prime }\). That is, Bob prefers a higher chance of a greater monetary reward.

Now consider the following alternative random scoring rule. If Bob announces probability \(\hat{p}\) and \(E\) occurs then Bob receives the lottery

$$\begin{aligned} x \text{ with } \text{ probability } P(\hat{p},1) \text{ and }\,y \text{ with } \text{ probability } \,1-P(\hat{p},1); \end{aligned}$$

If Bob announces probability \(\hat{p}\) and \(E\) does not occur then Bob receives the lottery

$$\begin{aligned} x \text{ with } \text{ probability } P(\hat{p},0) \text{ and }\,y \text{ with } \text{ probability } \,1-P(\hat{p},0)\text{; } \end{aligned}$$

where

$$\begin{aligned} P(\hat{p},1)=\frac{S(\hat{p},1)}{2} \text{ and }\,P(\hat{p},0)=\frac{S(\hat{p} ,0)}{2}. \end{aligned}$$

That is, Bob receives \(x\) with odds proportional to his Brier Score (the division by \(2\) is merely to ensure that \(P(\hat{p},1)\in [0,1]\) and \(P(\hat{p},0)\in [0,1]\)). Thus, assume Bob believes that \(E\) occurs with probability \(p\) and he announces \(\hat{p}\). Then, he obtains \(x\) with probability

$$\begin{aligned} \frac{pS(\hat{p},1)+(1-p)S(\hat{p},0)}{2}. \end{aligned}$$

Thus, if \(S\) is a proper score (as in the case of the Brier Score) then it is optimal for Bob to truthfully reveals his subjective probability \(p\), regardless of his attitudes toward risk. As in Karni’s mechanism, probabilistic sophistication and dominance are the only underlying assumptions.

This example illustrates the principle of eliciting beliefs by paying in chance. The basic procedure is as follows: start with any proper score and normalize it (with some linear, strictly increasing transformation function) so that the score is always between 0 and 1. The higher monetary reward \(x\) is delivered with a probability given by the normalized proper score. So, Bob has a better chance of obtaining a greater reward if his score is higher. It then follows that it is optimal for Bob to truthfully reveals his opinions, regardless of his attitudes toward risk.

Lambert (2011) provides a characterization of scoring rules that induce truthful-revelation with unknown preferences. The random scoring rule described above is subsumed in his characterization [see also Allen (1987); Hossain and Okui (2010); Schlag and van der Weele (2010)].

It is easy to see that the principle of eliciting beliefs by paying in chance can be applied to any bounded, proper score. Like Karni (2009) mechanism, this principle can be extended to many events and many experts, by running separate mechanisms to different experts and events. In addition, it also extends in a straightforward way to the case of multiple data points.

Importantly, this principle can be used in multi-agents, game-theoretic settings. We illustrate this point with Prelec (2004) “Truth Serum”. Consider several experts who face a subjective question such as “Is Picasso the best painter of the \(20^{\text{ th }}\) century?” No objective data is observed on this matter, but each expert has an opinion (say yes or no) on it. The problem is how to induce each expert to announce his honest opinion. Prelec assumes that the opinions of each expert are realizations of a random variable with a common distribution. He designs a Bayesian game in which the action space of expert \(i,\) \(A_{i}\), is to say yes or no and a forecast of fraction of experts who answers yes. A pure strategy, for player \(i\), maps his actual opinions (yes or no) into his action space \(A_{i}\). Player \(i\)’s payoffs is given by a specific utility function

$$\begin{aligned} U_{i}:\prod \limits _{k}A_{k}\longrightarrow \mathfrak R \end{aligned}$$

which he constructs. The key point in his paper is that there exists a Nash equilibrium in which all experts announce their opinions honestly.

As in the case of the Brier Score, if the payoffs given \(U_{i}\) are provided in monetary terms then risk-neutrality is assumed. If the payoffs are assumed to be utils then it is assumed that the expert’s attitudes toward risk are known. However, let \(\tau \) be a linear, strictly increasing function such that, for every player \(i\) and for every action profile \(a\in \prod \nolimits _{k}A_{k}\), \(\tau (U_{i}(a))\) is between \(0\) and \(1\).Footnote 1 Now consider the game in which given an action profile \(a\in \prod \nolimits _{k}A_{k}\), player \(i\) receives \(x\) with probability \(\tau (U_{i}(a))\) and \(y\) with probability \(1-\tau (U_{i}(a))\), where \(x>y\) are monetary rewards. The randomizations of nature determining between \(x\) and \(y\) is independent for each agent and each action profile. It is straightforward to show that the same truth-revealing Nash equilibrium in Prelec’s game remains a Nash equilibrium in this modified game. This follows regardless of the expert’s attitudes toward risks, provided that probabilistic sophistication and dominance are satisfied.

3 Conclusion

A simple way to elicit experts’ opinions when experts’ attitudes toward risk are unknown is to pay the expert in chance: he receives the greater reward with odds based on a proper method of eliciting beliefs that assume risk-neutrality.