Abstract
This paper proposes a methodology to implement probabilistic belief elicitation in continuouschoice games. Representing subjective probabilistic beliefs about a continuous variable as a continuous subjective probability distribution, the methodology involves eliciting partial information about the subjective distribution and fitting a parametric distribution on the elicited data. As an illustration, the methodology is applied to a double auction experiment, where traders’ beliefs about the bidding choices of other market participants are elicited. Elicited subjective beliefs are found to differ from proxies such as Bayesian Nash equilibrium beliefs and empirical beliefs, both in terms of the forecasts of other traders’ bidding choices and in terms of the bestresponse bidding choices prescribed by beliefs. Elicited subjective beliefs help explain observed bidding choices better than BNE beliefs and empirical beliefs. By extending probabilistic belief elicitation beyond discretechoice games to continuouschoice games, the proposed methodology enables to investigate the role of beliefs in a wider range of applications.
This is a preview of subscription content, access via your institution.
Notes
 1.
In a uniformprice double auction (also called static double auction, call auction, or clearinghouse auction) all traders trade at the same time, when the market is ‘called’.
 2.
The equilibrium model for the double auction consists of the BNE.
 3.
Cason and Friedman (1997) conduct a double auction experiment under the trading rules of the model of Rustichini et al. (1994). Most importantly, their design differs from the one of this paper because beliefs data is not elicited. Moreover, private valuations are drawn from the uniform distribution over $[0,4.99], participants play 30 trading rounds, profits accumulate throughout the rounds and participants are paid the accumulated amount. They report analogous evidence that bidding choices deviate from the riskneutral BNE predictions. However, by collecting only bidding choices, they face an identification problem, which I solve, following Manski (2002, 2004), by collecting both choice and beliefs data.
 4.
Thus, in each round a new set of observations of bidding choices, market price and trades is collected.
 5.
The personal values and costs of other buyers and sellers are never revealed. After submitting their choices, players are not allowed to modify them.
 6.
In a market with \(n\) buyers and \(n\) sellers, after sorting all \(2n\) bids and offers as \(\psi _{(1)} \le \psi _{(2)} \le \cdots \le \psi _{(2n)}\), the market price is set at \(p=0.5\psi _{(n)} + 0.5 \psi _{(n+1)}\). \(\psi _{(n)}\) denotes the \(n\)th order statistic among all 2 \(n\) submitted bidding choices.
 7.
The elicitation task is presented after each subject has submitted her bid or offer. Other studies where beliefs are elicited after choices are submitted include Bellemare et al. (2010) and (2012). Eliciting beliefs after choices are submitted has the advantage that choices are unaffected by the belief elicitation but raises the concern that participants state beliefs to rationalize choices. Bellemare et al. (2012) compare the beliefs in a benchmark treatment (with participants who make choices and then state beliefs) to the beliefs in a treatment with observerparticipants (who only state their beliefs but do not make any choice). They find that stated beliefs are not significantly affected by choices.
 8.
If a player does not trade, her initial cash (if a buyer) or the unsold unit of the commodity (if a seller) does not count towards profits.
 9.
The maximum amount a participant could earn in the comprehension quiz was $5. The maximum amount a participant could earn for forecasting was $4. According to the rules, any negative trading profit would be deducted from the total payments. However, no participant received a negative payoff at the end of the session.
 10.
For a discussion on the effects of the size of monetary incentives in belief elicitation see Armantier and Treich (2013).
 11.
\(\zeta _{(n)}\) is the \(n\)th order statistic among the \(2n1\) choices submitted by the other agents.
 12.
For example, the assessment can be affected by the socalled conjunction fallacy: the assessment of the joint probability is higher than the probabilities of the constituent events.
 13.
It is worth pointing out that in games with incomplete information, such as the double auction, beliefs about other players’ actions depend on beliefs about the distribution of players’ preferences (i.e., types, or private valuations) and beliefs about other players’ strategies.
 14.
The choice set is unbounded since a bidding choice can be any nonnegative number, rounded up to the second decimal. Beliefs are a continuous probability distribution defined over a potentially unbounded support. Thus, respondents are not artificially led to believe that other subjects’ choices are restricted to be within a certain upper bound.
 15.
Previous 2player games, in which Player 1 states probabilistic beliefs about randomlymatched Player 2’s choice, use either a probability format (Nyarko and Schotter 2002; CostaGomes and Weizsäcker 2008; Rutström and Wilcox 2009) or a proportion format (ReyBiel 2009; Blanco et al. 2010; Bellemare et al. 2011). The proportion format has been supported by evidence that people are better at working with natural frequencies than with percent probabilities (Hoffrage et al. 2000). In the proportion format, Player 1 states the ‘number of Player 2s out of ...’ that will choose each alternative choice. The base (‘out of ...’), set by the experimenter, is often 100, which facilitates the correspondence between probabilities and proportions. In this paper, several characteristics of the auction experiment make the probability format more appropriate. The auction is a multiplayer game with of 4, 6 or 8 participants, half buyers and half sellers. Thus, each subject faces 1, 2 or 3 other subjects with the same role and 2, 3 or 4 other subjects with the opposite role, and states separately beliefs about bids and beliefs about offers. On the one hand, a proportion format with base 100 could confuse subjects since the actual number of other buyers (sellers) is 1, 2, 3 or 4 (depending on market size and one’s own role). On the other hand, a proportion format with base equal to the actual number of other buyers (sellers) has two limitations. First, in a markets with four participants the wording ‘out of 1’ would appear when a subject states beliefs about the choice of the only other subject with the same role. Second, different wording would appear across markets (since markets differ in size) and across questions (since the numbers of other buyers and other sellers differ). It is out of the scope of this paper to investigate the effects of differences in wording.
 16.
The quadratic scoring rule is an incentivecompatible mechanism, which provides subjects with an incentive to report their beliefs truthfully, provided that subjects are risk neutral and do not distort probabilities. However, rewarding subjects for the accuracy of their beliefs and using the quadratic scoring rule to determine the reward may have several limitations. Rutström and Wilcox (2009) and Palfrey and Wang (2009) report evidence on the effects that the elicitation of beliefs may have on choice behavior, including the possibility of more strategic behavior, lower risk aversion and overconfidence. Blanco et al. (2010) investigate whether subjects employ the belief elicitation task in order to hedge against adverse outcomes in the choice task. They find no evidence of hedging being a major problem in belief elicitation. Armantier and Treich (2013) report that increasing the payment in a proper scoring rule induces more biases towards reporting uniform probabilities.
 17.
It is worth pointing out that the rule is not necessarily incentivecompatible if the participant is rewarded for predicting the choice frequencies of a finite number of opponents. In such a setting, the rule is incentivecompatible if his beliefs corresponds to one of the outcomes that his opponents’ choices could generate, but it is not incentivecompatible for all possible beliefs that he could hold. Consider a subject facing two other buyers and reporting probabilistic beliefs \((p_1,p_2,p_3,p_4,p_5,p_6)\) about bids. The rule is incentivecompatible only if his beliefs match one of the 21 possible vectors \((f_1,f_2,f_3,f_4,f_5,f_6)\) of empirical fractions where \(f_j = \{0,\frac{1}{2},\frac{2}{2} \}\) for \(j=1,2,\ldots ,6\) and \(\sum _{j=1}^{6}f_j = 1\). CostaGomes and Weizsäcker (2008) makes an analogous argument.
 18.
Subjects are provided with a simplified explanation of the rule in the instructions. At the end of each round subjects receive feedback including a comparison between the probabilities assigned to each interval and the number of other bids and offers falling within each interval.
 19.
\(Y(r_1)=y_1\), \(Y(r_2)=y_1+y_2\), and so on.
 20.
Approximately 90 % of the beliefs are fitted with a unimodal beta distribution.
 21.
Several papers elicit traders’ nonprobabilistic beliefs about future prices in a double auction experiment (Smith et al. 1988; Sonnemans et al. 2004; Hommes et al. 2005; Haruvy et al. 2007). The definition of beliefs as market price forecasts is certainly appropriate in the investigation of the effect of past prices on traders’ beliefs about future price movements, which is the objective of the abovementioned papers. However, such a definition is less appropriate in the investigation of how beliefs affect strategic decisionmaking, which is the objective of this paper. The market price is determined jointly by the choice made by a subject holding beliefs and the choices made by other subjects and it is therefore the result of strategic interaction.
 22.
 23.
In the double auction, as discussed above, the ordering of choices determines the auction outcome and the relevant beliefs about the ordering are represented by a joint density of two order statistics (\(n\)th and (\(n+1\))th for a buyer, (\(n1\))th and \(n\)th for a seller). Since beliefs about the ordering of choices cannot be revealed by a bestpoint forecast of choice, the design implements the elicitation of a distributional forecast. An alternative, nonprobabilistic approach, not explored in the paper, is to take the order statistics of choice as object of forecast, and elicit a bestpoint forecast of the order statistics.
 24.
In the double auction, for example, there are \(N>2\) players with different roles and types (buyers and sellers with independent private valuations), the choice set is \([0,\infty )\), and an individual’s payoff depends on a function of other players’ ordered choices.
 25.
For example, consider a Cournot market of incomplete information with each seller’s constant marginal cost drawn from a commonly known distribution in \([c_L,c_H]\). Quantitysetting sellers act as strategic decisionmakers who choose a nonnegative level of production from a continuous and possibly unbounded choice set, bestresponding to their beliefs about their rivals’ production. Belief elicitation consists of dividing the choice set into \(J\) intervals and having each seller assign a vector \(y = (y_1,\ldots ,y_J)\) of subjective probabilities to the \(J\) intervals. Then the relation between observed production choices and subjective beliefs can be investigated.
 26.
Future work should evaluate alternative methods to choose the intervals in order to provide more accurate fitting and/or require less elicitation questions.
 27.
The sample distribution is reported for the first and the last round separately.
 28.
There are only 4 observations out of 990 of a bid or offer above $10: buyer 49 in round 12 bids $10.60, seller 21 in round 9 offers $11, seller 7 in round 2 offers $15, seller 29 in round 2 offers $18.50. Across all subjects, the probabilities assigned in round 1 to a bid above $10 are 0 % (65 % of subjects), 1–10 % (29 % of subjects), 11–20 % (3 % of subjects), 21–30 % (3 % of subjects). In round 15, the probabilities are 0 % (91 % of subjects), 1–10 % (9 % of subjects). Across all subjects, the probabilities assigned in round 1 to an offer above $10 are 0 % (64 % of subjects), 1–10 % (24 % of subjects), 11–20 % (5 % of subjects), 31–60 % (7 % of subjects). In round 15, the probabilities are 0 % (86 % of subjects), 1–10 % (8 % of subjects), 11–20 % (5 % of subjects), 21–30 % (1v of subjects).
 29.
This paper focuses on two benchmarks: the beliefs learning model using empirical beliefs and the BNE. Differently than in beliefs learning, in reinforcement learning (Erev and Roth 1998) players learn by observing what was successful in the past and play more often strategies that paid off relatively well in the past. The experienceweighted attraction (EWA) learning model (Camerer and Ho 1999) is a hybrid model that encompasses beliefs and reinforcement learning models as special cases. Nyarko and Schotter (2002) find that the belief learning model using elicited subjective beliefs outperforms both a reinforcement and EWA model. Alternatively to BNE, nonequilibrium models such as levelk and cognitive hierarchy (CH) could serve as benchmark. While the belief elicitation methodology described in the paper is readily applicable in the context of a comparison between elicited beliefs and levelk /CH beliefs, the feedback design should be modified, since levelk /CH models are most appropriate in oneshot games with no feedback regarding opponents’ choices or one’s own payoff, in order to minimize learning effects. While these extensions go beyond the scope of this paper, I consider them an interesting topic for further research.
 30.
Given private values and costs distributed uniformly over \([0,10]\), \(f_b(v) = f_s(c) = 0.1\).
 31.
Specific cases of empirical beliefs include Cournot beliefs, defined as opponents’ lastperiod actions, and fictitiousplay beliefs, defined as the average of opponents’ past actions. They are obtained by setting \(\gamma =0\) or \(\gamma =1\), respectively. See also Fudenberg and Levine (1998). Note that, since in the experiment each subject changes role in round 6 and 11, \(N_t\) is not constant.
 32.
The median estimated \(\gamma ^*\) is 1.02 for beliefs about bids and 0.99 for beliefs about offers. For round 1, I assume that \(\gamma \)weighted empirical beliefs prescribe \(\xi _{i,1,j}=1/6\) for \(j=1,\ldots ,6\). For previous work using \(\gamma ^*\)weighted empirical beliefs see Nyarko and Schotter (2002).
 33.
 34.
Defining as pairs of comparison round 5 vs. round 10 and round 5 vs. round 15, Wilcoxon signed rank test finds no significant differences in \(AAD(y,\nu )\) (round 5 vs. 10 \(p=0.1770\), round 5 vs. 15 \(p=4188\)). Wilcoxon signed rank test finds significant differences in \(AAD(y,\eta )\) (round 5 vs. 10 \(p=0.0000\), round 5 vs. 15 \(p=0000\)).
 35.
Oneside Wilcoxon signed rank test against no difference between \(AAD(y,\eta )\) and \(AAD(y,\nu )\) within the same round. In round 5 \(p< 0.0000\), round 10 \(p=0.0093\), round 15 \(p=0.1339\).
 36.
 37.
Computing the best responses prescribed by beliefs requires using beliefs \(\hat{y}\), \(\hat{\nu }\), \(\hat{\eta }\). \(\hat{\eta }\) is the continuous distribution obtained applying to the discrete distribution \(\eta \) the fitting methods described in Sect. 2 (already employed to define \(\hat{y}\)). First the fitted density and cdf of other subjects’ bids, \(\hat{\eta }_{b}\) and \(\hat{H}_{b}\), and the fitted density and cdf of other subjects’ offers, \(\hat{\eta }_{s}\) and \(\hat{H}_{S}\), are obtained. Then \(\hat{\eta }_{b}\), \(\hat{H}_{b}\), \(\hat{\eta }_{s}\) and \(\hat{H}_{s}\) are used to compute the fitted joint densities of the relevant order statistics.
 38.
It is worth pointing out that while there exists a unique BNE best response (as discussed in Appendix 4 and in Cason and Friedman (1997)), there may be multiple best responses to subjective beliefs and multiple best responses to empirical beliefs. In the case of a range of best responses, the midpoint of the range is used for comparison in Table 2. Best responses are expressed up to cents of a dollar, so as are bids and offers in the experimental design.
 39.
Moreover, the best responses prescribed by subjective, BNE or empirical beliefs may generate different expected payoffs. Table 12 reports summary statistics of the absolute difference in expected payoffs between best responses for round 5, 10 and 15. In round 15 the absolute difference in expected payoff between best response to subjective beliefs and best response to BNE beliefs ranges between $0 and $2.30 (mean $0.44 and median $0.12). Analogously, the absolute difference in expected payoff between best response to subjective beliefs and best response to empirical beliefs ranges between $0 and $2.38 (mean $0.41 and median $0.16).
 40.
Given subjective beliefs \(\hat{y}\), a participant expects to receive payoff \(\pi (b\hat{y})\) by choosing \(b\). If \(b \in b_{BR,\hat{y}}\), then \(b\) and \(b_{BR,\hat{y}}\) generate, given subjective beliefs \(\hat{y}\), equal expected payoffs \(\pi (b\hat{y}) = \pi (b_{BR,\hat{y}}\hat{y})\).
 41.
Given empirical beliefs \(\hat{\eta }\), a participant expects to receive payoff \(\pi (b\hat{\eta })\) by choosing \(b\). Given BNE beliefs \(\hat{\nu }\), a participant expects to receive payoff \(\pi (b\hat{\nu })\) by choosing \(b\). If \(b \in b_{BR,\hat{\eta }}\) then \(\pi (b\hat{\eta }) = \pi (b_{BR,\hat{\eta }}\hat{\eta })\), and if \(b \in b_{BR,\hat{\nu }}\) then \(\pi (b\hat{\nu }) = \pi (b_{BR,\hat{\nu }}\hat{\nu })\).
 42.
Choice is not consistent with the best response to any belief in 53 % of the observations.
 43.
In a setting in which valuations are drawn independently from a uniform distribution over $[0,10], the range $[3,7] exclude those valuations for which the BNE model predicts that buyers with a low value and sellers with a high cost will have a zero probability of trading. See Rustichini et al. (1994).
 44.
Table 13 reports the results across marketsize treatments and separately for first and last round. Within each marketsize treatment, there are no significant differences in the consistency of choice with subjective beliefs between first and last round (Wilcoxon signed rank test against no difference in consistency across pairs: \(p=0.999,p=0.999,p=0.8238\), for marketsize 4, 6, 8 respectively). Across marketsize treatments, the hypothesis that the rate of consistency of choice with subjective beliefs is the same for market size 4, 6 and 8 cannot be rejected either for round 1 (twosided Fisher exact test, \(p = 0.052\)) or for round 15 (\(p = 0.254\)).
 45.
The probability is defined as the ratio between the number of choices that generate an expected payoff equal to the one generated by the best response and the number of all possible alternatives from which a subject can randomly choose. Since in the experiment choices and payoffs are defined to the nearest cent, numbers are expressed in cents when computing the probability. The possible alternatives from which to randomly choose are restricted to the values between $0 and $10.
 46.
Choice is not consistent with the best response to any belief in 69 % of the observations.
 47.
How do the subsamples with extreme valuations compare with the one with midrange valuations? Among observations with a low value or a high cost, consistency of choice with the best response to subjective, empirical or BNE beliefs is 69.5, 69.5 and 71 %, respectively. Within this subsample, consistency artificially increases due to the extremely low sensitivity of expected payoff to choice. Among observations with a high value or a low cost, consistency of choice with the best response to subjective, empirical or BNE beliefs is 19.5, 16.2 and 18.5 %, respectively. For both subsamples with extreme valuations there is no evidence of a better performance of subjective beliefs, compared to empirical or BNE beliefs, in explaining observed choice.
 48.
Convergence in choices to the BNE requires choices to be consistent with the best response prescribed by BNE beliefs from a certain point in time onwards without deviation. Similarly, convergence in beliefs to the BNE requires subjective beliefs to prescribe a best response consistent with the best response prescribed by BNE beliefs from a certain point in time onwards without deviation.
 49.
Recall from Sect. 2 that subjects are rewarded based on the comparison of their subjective beliefs to the bidding behavior of other participants in the same auction market.
 50.
In the experiment subjects report separately their beliefs about the bids chosen by buyers and their beliefs about the offers chosen by sellers. Therefore, the accuracy of beliefs about bids and the accuracy of beliefs about offers can be analyzed separately or jointly. In the analysis that follows, I consider a joint measure of accuracy. Each measure in Table 5 is the sum of the measure for beliefs about bids and the measure for beliefs about offers. Table 14 is analogous to Table 5 but reports results separately by market size.
 51.
Analogous evidence of subjective beliefs’ ability to explain observed choices despite low predictive accuracy is reported by Nyarko and Schotter (2002), who elicit beliefs in a 2x2 game and compare subjective beliefs with empirical beliefs.
 52.
Linear score is defined as \(\$2  \sum _{j=1}^{6} n_{j} \left[ (1y_{j}) + \sum _{h\ne j} y_{h} \right] \) and \(AAD\) is defined as \( \frac{1}{6} \sum _{j=1}^6 y_{j}  n_{j}\), where \(y_{j}\) is the probability assigned to an opponent submitting a choice within interval \(j\), and \(n_{j}\) is the fraction of opponents submitting a choice within interval \(j\).
 53.
Twoside Wilcoxon signed rank test cannot reject the null hypothesis of no difference in accuracy between round 1 and 15 (quadratic rule \(p=0.8043\), linear rule \(p=0.7982\), \(AAD\) \(p=0.6147\)).
 54.
By Seidenfeld’s definition, a set of probabilistic predictions are calibrated if \(p\) percent of all predictions reported at probability \(p\) are true. In the experiment participants forecast the fraction of choices falling in each interval $[0,2], [2.01,4], [4.01,6], [6.01,8], [8.01,10] and [10.01,\(\infty \)). Beliefs are perfectly calibrated if for all the instances when the probabilities \((0.25, 0.5, 0.25, 0, 0, 0)\) are assigned, choices fall 25 % of the time in $[0,2], 50 % of the time in $[2.01,4], and 25 % of the time in $[4.01,6]; for all the instances when the probabilities \((0.4, 0.4, 0.2, 0, 0, 0)\) are assigned, choices fall 40 % of the time in $[0,2], 40 % of the time in $[2.01,4], and 20 % of the time in $[4.01,6]; and so on. Calibration adopts a frequentist approach to measure forecast accuracy, while proper scoring rules adopt a Bayesian approach.
 55.
The mostoften stated beliefs (0.2,0.2,0.2,0.2,0.2,0) are the best calibrated.
 56.
Denote with \(y_t = (y_1, \ldots , y_6)\) the vector of probabilities that subject \(i\) assigns in round \(t\) to choices falling in the intervals \((1,\ldots ,6)\). Subject \(i\) submits \(T\) vectors, one in each round. Of the \(T\) vectors, there are \(K\) distinct ones, identified as \(y_k = (y_{k1}, \ldots , y_{k6}) \) with \(k=1,..,K\). Each distinct \(y_k\) occurs in \(m_k\) instances. Denote with \(n_k = (n_{k1}, \ldots , n_{k6})\) the relative frequency of choices falling in the intervals in the \(m_k\) instances when \(y_k\) is submitted. Weighting the squared difference between \(y_k\) and \(n_k\) by \(m_k\), the Briercalibration score is \(\frac{1}{T} \sum _{k=1}^{K} m_k (y_k  n_k) (y_k  n_k)' \). It ranges in [0,2], 0 corresponding to perfect calibration.
 57.
In a set of oneshot games with no feedback, CostaGomes and Weizsäcker (2008) find evidence of poor calibration (calibration scores of 0.432 and 0.374 for Rows and Columns players, respectively). In repeated games, Camerer et al. (2002) find that beliefs are very well calibrated (calibration scores of 0.006 and 0.011 for Borrower and Lender players, respectively). Camerer et al. (2002) find that Borrower beliefs and Lender beliefs are similarly accurate at forecasting the behavior of Borrowers.
 58.
Participants play several rounds as buyer and several rounds as seller. In each round, independently of their role, participants state their beliefs about bids and their beliefs about offers. This allows us to make two pairwise comparisons for each individual, either keeping the role fixed (Does an individual playing as buyer/seller score better at forecasting bids or offers?) or keeping the object of forecast fixed (Does an individual forecasting bids/offers score better when playing as buyer or seller?). I use a oneside Wilcoxon signed rank test against no difference in calibration score across pairs of observations. A participant forecasting bids scores better when playing as seller than buyer (\(p<0.0000\)). A participant forecasting offers scores better when playing as buyer than seller (\(p=0.0005\)). A participant playing as buyer scores better at forecasting offers than bids (\(p<0.000\)). A participant playing as seller scores better at forecasting bids than offers (\(p<0.000\)).
 59.
Kirchkamp and Reiß (2011) argue that ‘one reason for erroneous best replies is connected to the handling of probabilities, particularly when transforming the expected bidding strategies used by others (together with the underlying distribution of valuations) into the probability distribution of winning bids’.
 60.
 61.
This procedure is equivalent to determining the interval over which the aggregated demand and supply schedules cross, and setting the market price at its midpoint, as the example in Fig. 2 shows.
 62.
If excess demand or excess supply arises, priority is given to sellers whose offers are smallest and to buyers whose bids are largest. A fair lottery then determines who trades among the remaining subjects on the long side of the market.
 63.
\(\psi _{(n+1)}\) is the \(n+1\)th order statistic among the 2\(n\) choices submitted by all agents. \(\zeta _{(n)}\) is the \(n\)th order statistic among the 2\(n\)1 choices submitted by the other agents. In terms of all 2\(n\) choices (including one’s own), a winning bid \(b\) must be at least equal to \(\psi _{(n+1)}\). In terms of the other 2\(n\)1 choices (exclusing one’s own), a winning bid \(b\) must be at least equal to \(\zeta _{(n)}\).
 64.
Analogously, if the subject assigns probability \(\alpha \) and \(1\alpha \) to the intervals \([y, y+2]\) and \((y+2, y+4]\) respectively, where \(\alpha >0.5\), then I let \(t = \frac{2 \sqrt{\frac{1\alpha }{2}} }{1  \sqrt{\frac{1\alpha }{2}}}\). Then the subjective probability density function takes the form of a triangle with a base with endpoints \(y\) and \(y+2+t\) and with a height \(h=\frac{2}{t+2}\).
 65.
Therefore, if \(\alpha <\beta \) then the first triangle has base with endpoints \(l_1+2t\) and \(l_1+4\) and height \(h=\frac{2(\alpha +\beta )}{t+2}\) with \(t = \frac{2 \sqrt{\frac{\alpha }{2(\alpha +\beta )}} }{1  \sqrt{\frac{\alpha }{2(\alpha +\beta )}} }\). If \(\alpha >\beta \), then the first triangle has base with endpoints \(l_1\) and \(l_1+2+t\) and height \(h=\frac{2(\alpha +\beta )}{t+2}\) with \(t = \frac{2 \sqrt{\frac{\beta }{2(\alpha +\beta )}} }{1  \sqrt{\frac{\beta }{2(\alpha +\beta )}} }\). The second triangle has base with endpoints \(l_3\) and \(r_3\) and height \(\frac{2 (1\alpha \beta )}{r_3l_3}\).
 66.
Rustichini et al. (1994) show that there exists a family of asymmetric smooth equilibria, which can be computed numerically. To simplify the analysis, I consider approximate symmetric bid and offer functions. The approximation should not affect the analysis since the family of asymmetric smooth equilibria is contained in a small neighborhood of the symmetric bid and offer functions. Also Cason and Friedman (1997) use approximate symmetric bid and offer functions in their experimental implementation of the Rustichini et al. (1994) double auction.
 67.
Recall that the highest possible private cost in the experiment is $9.99.
 68.
Table 16 reveals that 19 % of the observations (188 out of 990) consist of either a buyer bidding above value or a seller offering below cost. While a percentage of 19 % is surprisingly high, several qualifications needs to be made. Differences exist across subjects: some subjects incur in overrevelation more often than others. Among participants, 70 % of them overreveal their private information in 3 or fewer of the 15 total rounds. Most importantly, the expected monetary losses generated by overrevelation are small. I compute, based on the elicited subjective beliefs, the expected payoffs generated by the actual bidding choices above value or below cost and I compare them with the expected payoffs generated by bidding at value or at cost. The percentage of observations, for which the expected monetary loss generated by overrevelation is greater than or equal to 10 cents (50 cents) is equal to 10 % (5 %).
 69.
I compare \(VUR(v,b)\) and \(CUR(c,a)\), which are computed with respect to the observed bids and offers \(b\) and \(a\), with the underrevelation ratios computed with respect to the bids and offers prescribed by the BNE strategies \(b_{BNE}\) and \(a_{BNE}\), defined as \(VUR(v,b_{BNE})= (vb_{BNE})/v\) and \(CUR(c,a_{BNE})= (a_{BNE}c)/(9.99c)\). The medians of \(VUR(v,b_{BNE})\) and \(CUR(c,a_{BNE})\) are equal and approximately 10 %, indicating that experiment participants reveal more private information than they would according to the BNE.
 70.
Cason and Friedman (1997) report qualitatively comparable results, in spite of several differences in the experimental design. First, buyers choose a bid above what the BNE would prescribe and sellers choose an offer below what BNE would prescribe. Second, subjects reveal more of their private information than what they would according to BNE strategies: for inexperienced subjects the median value underrevelation ratio is 6.5 % and the median cost underrevelation ratio is 9.1 % (for experienced subjects the ratios are 2.5 and 2.4 %, respectively).
 71.
A comparison with Cason and Friedman (1997) is possible for the markets with eight subjects. They find a higher trading efficiency and a slightly higher fraction of prices within the CE price interval. Recall that in their experiment there are more trading rounds (30 instead of 15).
References
Armantier, O., & Treich, N. (2009). Subjective probabilities in games: An application to the overbidding puzzle. International Economic Review, 50(4), 1079–1102.
Armantier, O., & Treich, N. (2013). Eliciting beliefs: Proper scoring rules, incentives, stakes and hedging. European Economic Review, 62, 17–40.
Bellemare, C., Sebald, A., & Strobel, M. (2010). Measuring the willingness to pay to avoid guilt: Estimation using equilibrium and stated belief models. Journal of Applied Econometrics. https://ideas.repec.org/p/iza/izadps/dp4803.html.
Bellemare, C., Sebald, A., & Strobel, M. (2011). Measuring the willingness to pay to avoid guilt: Estimation using equilibrium and stated belief models. Journal of Applied Econometrics, 26, 437–453.
Bellemare, C., Bissonnette, L., & Kröger, S. (2012). Flexible approximation of subjective expectations using probability questions. Journal of Business and Economic Statistics, 30(1), 213–245.
Blanco, M., Engelmann, D., Normann, H., & Koch, A. (2010). Belief elicitation in experiments: Is there a hedging problem? Experimental Economics, 13(4), 412–438.
Camerer, C., Ho, T., Chong, J., Weigelt, K. (2002). Strategic teaching and equilibrium models of repeated trust and entry games. Mimeo: CalTech.
Camerer, C., & Hua Ho, T. (1999). Experienceweighted attraction learning in normal form games. Econometrica, 67(4), 827–874.
Cason, T., & Friedman, D. (1997). Price formation in single call markets. Econometrica, 65(2), 311–345.
Cheung, Y. W., & Friedman, D. (1997). Individual learning in normal form games: Some laboratory results. Games and Economic Behavior, 19(1), 46–76.
CostaGomes, M., & Weizsäcker, G. (2008). Stated beliefs and play in normalform games. Review of Economic Studies, 75(3), 729–762.
Danz, D. N., Fehr, D., & Kübler, D. (2012). Information and beliefs in a repeated normalform game. Experimental Economics, 15(4), 622–640.
Engelberg, J., Manski, C., & Williams, J. (2009). Comparing the point predictions and subjective probability distributions of professional forecasters. Journal of Business and Economic Statistics, 27(1), 30–41.
Erev, I., & Roth, A. (1998). Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria. The American Economic Review, 88(4), 848–881.
Fudenberg, D., & Levine, D. (1998). Learning in games. European Economic Review, 42(3), 631–639.
Haruvy, E., Lahav, Y., & Noussair, C. (2007). Traders’ expectations in asset markets: Experimental evidence. The American Economic Review, 97(5), 1901–1920.
Hoffrage, U., Lindsey, S., Hertwig, R., & Gigerenzer, G. (2000). Communicating statistical information. Science, 290(5500), 2261–2262.
Hommes, C., Sonnemans, J., Tuinstra, J., & Velden, H. V. D. (2005). Coordination of expectations in asset pricing experiments. Review of Financial Studies, 18(3), 955.
Hyndman, K., Özbay, E., Schotter, A., & Ehrblatt, W. Z. (2012). Convergence: An experimental study of teaching and learning in repeated games. Journal of the European Economic Association, 10(3), 573–604.
Kirchkamp, O., & Reiß, J. P. (2011). Outofequilibrium bids in firstprice auctions: Wrong expectations or wrong bids. The Economic Journal, 121(557), 1361–1397.
Manski, C. (2002). Identification of decision rules in experiments on simple games of proposal and response. European Economic Review, 46(4–5), 880–891.
Manski, C. (2004). Measuring expectations. Econometrica, 72(5), 1329–1376.
Manski, C., & Neri, C. (2013). Firstand secondorder subjective expectations in strategic decisionmaking: Experimental evidence. Games and Economic Behavior, 81(C), 232–254.
Nyarko, Y., & Schotter, A. (2002). An experimental study of belief learning using elicited beliefs. Econometrica, 70(3), 971–1005.
Palfrey, T., & Wang, S. (2009). On eliciting beliefs in strategic games. Journal of Economic Behavior & Organization, 71(2), 98–109.
ReyBiel, P. (2009). Equilibrium play and best response to (stated) beliefs in normal form games. Games and Economic Behavior, 65(2), 572–585.
Rustichini, A., Satterthwaite, M., & Williams, S. (1994). Convergence to efficiency in a simple market with incomplete information. Econometrica, 62(5), 1041–1063.
Rutström, E., & Wilcox, N. (2009). Stated beliefs versus inferred beliefs: A methodological inquiry and experimental test. Games and Economic Behavior, 67(2), 616–632.
Seidenfeld, T. (1985). Calibration, coherence, and scoring rules. Philosophy of Science, 52, 274–294.
Smith, V., Suchanek, G., & Williams, A. (1988). Bubbles, crashes, and endogenous expectations in experimental spot asset markets. Econometrica, 56(5), 1119–1151.
Sonnemans, J., Hommes, C., Tuinstra, J., & Velden, H. V. D. (2004). The instability of a heterogeneous cobweb economy: A strategy experiment on expectation formation. Journal of Economic Behavior & Organization, 54(4), 453–481.
Acknowledgments
I had the opportunity to present earlier versions of this work at the 2010 International Meeting of the Economic Science Association, and at Northwestern University, University of St.Gallen, University of Zurich, LUISS, Universidade Nova de Lisboa, CSEF, Bank of Italy. I am grateful to seminar participants and to Charles Manski, Marco Ottaviani, Brian Rogers, Charles Bellemare, and Jacob Goeree for their comments. A previous version of this work circulated as ‘Strategic Thinking and Subjective Expectations in a Double Auction Experiment’.
Author information
Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Appendices
Appendix 1: Figures and Tables
Appendix 2: The double auction model
This section provides an overview of the double auction model by Rustichini et al. (1994).
The market is populated with \(n\) buyers and \(n\) sellers. Each buyer has an endowment of $10 and each seller has one unit of an indivisible good. Each buyer can purchase at most a single unit of the good from any seller, and each seller could sell a single unit of the good to any buyer. Each buyer has a private value and each seller has a private cost for the good. Private values and costs are independently drawn from the uniform distribution over \([0,10]\). A subject’s private value or cost is her own private information, and the process by which private values and costs are drawn is common knowledge among subjects.
All buyers and sellers simultaneously submit their bids or offers. Sorting all bids and offers in increasing order as \(\psi _{(1)} \le \psi _{(2)} \le \cdots \le \psi _{(2n)} \), the market price is set at the midpoint between the middle two figures, at \(p = 0.5 \psi _{(n)} + 0.5 \psi _{(n+1)}\).^{Footnote 61} Buyers who submit a bid larger than or equal to \(p\) and sellers who submit an offer smaller than or equal to \(p\) trade. If trading at \(p\), a buyer with private value \(v\) earns a payoff equal to \(vp\), and a seller with private cost \(c\) earns a payoff equal to \(pc\). Participants who do not trade earn zero.^{Footnote 62}
Lets consider a buyer \(i\) with private value \(v\). According to the auction rules, the relationship between her bid \(b\) and the bids and offers of the other \(2n1\) participants determines what her payoff will be, depending on whether she trades and, if so, at what price. Sorting all other \(2n1\) bids and offers in increasing order as \(\zeta _{(1)} \le \zeta _{(2)} \le \cdots \le \zeta _{(2n1)}\), buyer \(i\) trades if \(b > \zeta _{(n)}\). If \(\zeta _{(n)} < b < \zeta _{(n+1)}\), the price is \(p=0.5\zeta _{(n)}+ 0.5 b\). If \( b > \zeta _{(n+1)}\), the price is \(p=0.5\zeta _{(n)}+ 0.5 \zeta _{(n+1)}\). Buyer \(i\) chooses bid \(b\ge 0\) without knowing the other 2\(n\)1 bids and offers but holding probabilistic beliefs about them.^{Footnote 63}
Assuming buyer \(i\) behaves as a riskneutral subjective expected utility maximizer, given her beliefs about the realizations of \(\zeta _{(n)}\) and \(\zeta _{(n+1)}\), represented by the joint density \(f_{(n),(n+1)}\), she chooses bid \(b\) in order to maximize her expected payoff:
where \(\zeta _{(n)}\) and \(\zeta _{(n+1)}\) are defined over the set \([0,10]\), the first term is the expected payoff if \( \zeta _{(n)} < b < \zeta _{(n+1)}\) and the second term is the expected payoff if \( b > \zeta _{(n+1)}\). The inner integral integrates over all possible values of \(\zeta _{(n)}\) and the outer one over all possible values of \(\zeta _{(n+1)}\).
Analogously, the beliefs of seller \(i\), who has private cost \(c\) and submits offer \(a\), can be represented by \(f_{(n1),(n)}\), the joint density of \(\zeta _{(n1)}\) and \(\zeta _{(n)}\), and her expected payoff can be written as
where \(\zeta _{(n1)}\) and \(\zeta _{(n)}\) are defined over the set \([0,10]\), the first term is the expected payoff if \( \zeta _{(n1)} < a < \zeta _{(n)}\) and the second term is the expected payoff if \( a < \zeta _{(n1)}\). The inner integral integrates over all possible values of \(\zeta _{(n1)}\) and the outer one over all possible values of \(\zeta _{(n)}\).
Summing up, a buyer’s beliefs are represented by joint density \(f_{(n),(n+1)}\) and a seller’s beliefs are represented by joint density \(f_{(n1),(n)}\). Denoting the density functions of other buyers’ bids and other sellers’ offers with \(g_{b}\) and \(g_{s}\) respectively, and their cumulative density functions with \(G_{b}\) and \(G_{s}\) respectively, \(f_{(n),(n+1)}\) and \(f_{(n1),(n)}\) can be written as function of the beliefs about other buyers’ bids (\(g_b\) and \(G_b\)) and the beliefs about other sellers’ offers (\(g_s\) and \(G_s\)).
To show it, consider a general formulation with \(m\) buyers and \(n\) sellers. Denote \(k=m\). Denote with \(f_{(k),(k+1)}\) the joint density of the \(m\)th and (\(m+1\))th order statistics of the \(m+n1\) bids and offers. Then:
Denote \(k=m1\). Denote with \(f_{(k),(k+1)}\) the joint density of the (\(m1\))th and \(m\)th order statistics of the \(m\)+\(n\)1 bids and offers. Then:
The formulas for the marginal densities of order statistics are as follows. Denote \(k=m\).
For the buyer:
For the seller:
Appendix 3: Parametric fitting of beliefs
Depending on (a) the number of intervals where positive probability is placed, and (b) whether the intervals, where positive probability is placed, are adjacent to each other, I fit:

(1)
a triangular distribution when positive probability is placed (i) over a single interval or (ii) over two adjacent intervals,

(2)
the union of two triangular distributions when positive probability is placed (i) over two nonadjacent intervals or (ii) over three intervals of which two but not all three are adjacent (iii) over four intervals, consisting of two disjoint pairs of adjacent intervals,

(3)
the union of three triangular distributions when positive probability is placed over three interval none of which is adjacent to any other,

(4)
a unimodal beta distribution when positive probability is placed over three or more intervals all adjacent to each other,

(5)
the union of a beta distribution and a triangular distribution when positive probability is placed over more than three intervals, of which at least three, but not all, are adjacent to each other.
Here below I describe how the fitting is performed in each case. Figure 3 shows an example for each case, the most common being the case in which beliefs are fitted with a unimodal beta distribution (approximately 90 % of observations), followed by the case in which beliefs are fitted with a triangular distribution (approximately 7 % of observations).
Triangular distribution
If positive probability is assigned to only one interval, \([l,r]\), then I assume that the support of the subjective distribution is \([l,r]\) and I fit a triangular distribution over it. The fitted isosceles triangle has base \(rl\) and height \(\frac{2}{rl}\). If positive probability is assigned to two adjacent intervals and if equal probability is assigned to each interval, then I assume that the support of the subjective distribution is the union of the two intervals and the fitted isosceles triangle has base \(4\) and height \(\frac{1}{2}\).
If positive probability is assigned to two adjacent intervals and a higher probability is assigned to one interval than to the other, then I assume that the subjective distribution has the shape of an isosceles triangle and that its support contains entirely the interval that was assigned a higher probability and partly the other interval. If the subject assigns probability \(\alpha \) and \(1\alpha \) to the intervals \([y, y+2]\) and \((y+2, y+4]\), respectively, where \(\alpha <0.5\), then, the fitted isosceles triangle has base with endpoints \(y+2t\) and \(y+4\) and height \(h=\frac{2}{t+2}\), with \(t = \frac{2 \sqrt{\frac{\alpha }{2}} }{1  \sqrt{\frac{\alpha }{2}} }\).^{Footnote 64}
Union of two triangular distributions
If positive probability is assigned to two nonadjacent intervals, then I assume that the support of the subjective distribution is the union of the two intervals and I fit a triangular distribution over each interval. For example, probability \(\alpha \) is assigned to the interval \([l_1,r_1]\) and probability \(1\alpha \) to the interval \([l_2,r_2]\), then I assume that the support of the distribution is the union of \([l_1,r_1]\) and \([l_2,r_2]\). The isosceles triangle fitted over \([l_1,r_1]\) has base \(r_1l_1\) and height \(\frac{2 \alpha }{r_1l_1}\). The isosceles triangle fitted over \([l_2,r_3]\) has base \(r_2l_2\) and height \(\frac{2 (1\alpha )}{r_2l_2}\).
If positive probability is assigned to three intervals of which two but not all three are adjacent, then I assume that the support of the subjective distribution is the union of the intervals and I fit two triangular distributions, one over the two adjacent intervals and another other the nonadjacent interval. For example, suppose that the intervals \([l_1,r_1]\), \([l_2,r_2]\) and \([l_3,r_3]\) are assigned probability \(\alpha \), \(\beta \) and \(1  \alpha  \beta \), respectively, and that \([l_1,r_1]\) and \([l_2,r_2]\) are adjacent to each other (with \(l_2>l_1\)), while \([l_3,r_3]\) is not adjacent to any of the other intervals. Then one triangle is fitted over the union of \([l_1,r_1]\) are \([l_2,r_2]\) and one triangle is fitted over \([l_3,r_3]\), following the procedures already described for fitting one triangular distribution.^{Footnote 65}
If positive probability is assigned to four intervals, consisting of two disjoint pairs of adjacent intervals, then I also fit two triangular distributions. Each triangular distribution has support over a pair of adjacent intervals and the fitting is done following the procedure already described for fitting one triangular distribution.
Union of three triangular distributions
If positive probability is assigned to three intervals none of which is adjacent to any other, then I assume that the support of the subjective distribution is the union of the intervals and I fit a triangular distribution over each intervals, following the procedure already described for fitting one triangular distribution.
Unimodal beta distribution
If positive probability is assigned to three or more intervals all adjacent to each other, then I fit a generalized unimodal Beta distribution over the intervals. The cumulative distribution function for a unimodal Beta distribution evaluated at \(x\) is denoted \(Beta(x,\alpha ,\beta ,l,r)\), where \(\alpha \) and \(\beta \) are shape parameters and \(l\) and \(r\) are location parameters determining the support for the distribution over the range \([l,r]\). If a subject does not assign positive probability to the right tail interval \([10.01,\infty )\), then the lower bound \(l\) of the support for the fitted Beta distribution will coincide with the left endpoint of the leftmost interval with positive probability and the upper bound \(r\) of the support will coincide with the right endpoint of the rightmost interval with positive probability and, therefore, the parameters \(l\) and \(r\) will be fixed. Thus, fitting the data with a Beta distribution requires solving the problem \(\min _{\begin{array}{c} \alpha ,\beta \end{array}} \sum _{j=1}^{6} [Beta(r_{j},\alpha ,\beta ,l,r)  G(r_{j}) ]^2\), where \(G(r_{j})\) is the sum of the subjective probabilities assigned up to the interval with right endpoint \(r_j\), inclusive.
If instead a subject assigns positive probability to the upper unbounded interval [$10.01,\(\infty \)), I let the location parameter \(r\) be a free parameter in the minimization of the least squares problem. I restrict \(r\) to lie within the most extreme value recorded in the data, which is 18.50. Thus, the problem becomes \(\min _{\begin{array}{c} \alpha ,\beta ,r<18.50 \end{array}} \sum _{j=1}^{6} [Beta(r_{j},\alpha ,\beta ,l,r)  G(r_{j}) ]^2\).
Union of a Beta distribution and a triangular distributions
If positive probability is assigned to more than three intervals, of which at least three but not all are adjacent to each other, then I fit a unimodal beta distribution over the three or more adjacent intervals and a triangular distribution over the remaining one or two intervals. I follow the procedures already described for fitting a triangular distribution and a unimodal Beta distribution.
Goodness of fit is assessed by the average absolute deviation between the fitted and the elicited beliefs. In most cases the average absolute deviation is below 0.01, and in all cases below 0.06.
Appendix 4: Descriptive Results
Descriptive results about bidding behavior
In this section I illustrate how the observed bidding choices compare with the bidding behavior prescribed by the riskneutral BNE model, as presented by Rustichini et al. (1994).^{Footnote 66} Figure 4 illustrates the approximate riskneutral bidding functions predicted by the model.
Since the double auction is a strategic environment with private information, strategic misrepresentation of private information is a key feature within the BNE model. As a measure of how much subjects reveal of the private information they hold, I use the value underrevelation ratio and the cost underrevelation ratio, as defined by Cason and Friedman (1997). The value underrevelation ratio \(VUR(v,b)\) is the fraction of the buyer’s private value, \(v\), discounted in the chosen bid, \(b\), and the cost underrevelation ratio \(CUR(c,a)\) is the cost markup relative to the highest possible cost. Thus, \(VUR(v,b)= (vb)/v\) and \(CUR(c,a)= (ac)/(9.99c)\).^{Footnote 67} A positive ratio corresponds to underrevelation and a negative ratio corresponds to overrevelation.^{Footnote 68}
The upper panel of Table 16 reports the median value and cost underrevelation ratios. Results are reported for (i) the entire sample, and separately for (ii) the subsample in which underrevelation occurs and (iii) the subsample in which overrevelation occurs. Within subsample (ii) the median \(VUR(v,b)\) and \(CUR(c,a)\) are 7 and 4 %, respectively.^{Footnote 69}
The lower panel of Table 16 reports the magnitude of the percent deviation of the BNE best response from the observed choice, defined as \(D(b_{BNE},b)=(b_{BNE}b)/b_{BNE}\) for buyers and \(D(a_{BNE},a)=(a_{BNE}a)/a_{BNE}\) for sellers. Within subsample (ii), the median deviation is approximately 0 for buyers and 2 % for sellers. ^{Footnote 70}
Descriptive results about market performance
Table 17 reports a description of market performance across the experimental auctions. The number of trades taking place in each round ranges between zero and three, with one or two trades per round being the most common outcome. Over all observations, including those when no trade occurs, mean profits are $0.88. In a competitive equilibrium (CE) buyers and sellers submit respectively their private values and private costs without engaging in any strategic misrepresentation of their private information. Across all experimental auctions, trading efficiency, defined as the percentage of the gains from exchange realized by traders in comparison with the CE, is 78 % and prices are within the CE price interval in 50 % of the rounds.^{Footnote 71}
Rights and permissions
About this article
Cite this article
Neri, C. Eliciting beliefs in continuouschoice games: a double auction experiment. Exp Econ 18, 569–608 (2015). https://doi.org/10.1007/s1068301494201
Received:
Revised:
Accepted:
Published:
Issue Date:
Keywords
 Probabilistic beliefs
 Belief elicitation
 Private information
 Experiments