Experience and rationality under risk: re-examining the impact of sampling experience

A recent strand of the literature on decision-making under uncertainty has pointed to an intriguing behavioral gap between decisions made from description and decisions made from experience. This study reinvestigates this description-experience gap to understand the impact that sampling experience has on decisions under risk. Our study adopts a complete sampling paradigm to address the lack of control over experienced probabilities by requiring complete sampling without replacement. We also address the roles of utilities and ambiguity, which are central in most current decision models in economics. Thus, our experiment identifies the deviations from expected utility due to over- (or under-) weighting of probabilities. Our results confirm the existence of the behavioral gap, but they provide no evidence for the underweighting of small probabilities within the complete sampling treatment. We find that sampling experience attenuates rather than reverses the inverse S-shaped probability weighting under risk. Electronic supplementary material The online version of this article (10.1007/s10683-019-09641-y) contains supplementary material, which is available to authorized users.


Online Appendices of Experience and Rationality under Risk: A Re-examination of the Description-Experience Gap
Ilke Aydogan 1 , Yu Gao 2

Online Appendix 1: Experimental Instructions
Instructions : Part 1 In this part of the experiment, you will face a series of choice questions involving choices between two prospects. Examples of choices are presented below. Risk is generated by throwing two ten-faced dice and adding the results together. One dice has the values 00, 10, …, 90 and the other has the values 0, 1, ..., 9.
Thus the sum yields a random number between 0 up to 99, and each of these numbers is equally likely. For instance, in Figure 1 the random number generated by the two dice is 30+8=38.  In this situation, you will be asked to choose between two prospects. Prospect A pays €24 if the dice give a number between 0 and 32 and €17 if the dice give a number between 33 and 99. Prospect B pays €56 if the dice give a number between 0 and 32 and €9 if the dice give a number between 33 and 99. You can choose your preferred prospect by clicking on the button next to it.
After you made your choice, a 'Confirm' button will appear. Please click on it to proceed. Before you press the "Confirm" button, you may change your choice between Prospect A and B. But once you press the "Confirm" button, you can no longer go back to change your choice.
We will now test your understanding of the instructions.
Assume you have been selected as one of the two participants who can play a question for real and that the question below was randomly selected.

Figure 3
Please answer the following questions.

Question 1
If you chose prospect A, and the number generated by the dice is 61, then how much will you receive from this prospect? €24 €56 €9 €17

Question 2
If you chose prospect B, and the number generated by the dice is 61, then how much will you receive from this prospect? €24 €56 €9 €17 After answering the above comprehensive questions correctly, subjects will be directed to part 2.
In part 2 of the experiment, subjects will be randomly assigned to either the DFD treatment or the DFE treatment.
The following is the instructions for the DFD treatment.

Instructions: Part 2
In this part of the experiment, you will face 28 choice questions, involving choices between two prospects.
Risk is generated by a random draw, without looking, from an urn containing balls with monetary values written on them. Figure 1 shows the urns that we are going to use.

Figure 1
In each question, you will face two urns that contain balls with different values. You are asked to select the urn from which you prefer to draw a single ball randomly. The value of the ball drawn from the selected urn will determine your payment. Figure 2 presents a choice situation that you will face during the experiment. The left and right illustrations represent the urns containing balls. The total number of balls in each urn is shown. In this case each urn contains 20 balls. All other relevant information about the content of the urns is presented below the illustrations. In this case, the left urn contains 19 balls each with value of €96 and 1 ball with value of €24 whereas the right urn contains 20 balls each with value of €92.

Figure 2
You will choose the urn that you prefer to draw a ball from by selecting "Left" or "Right". After you made your choice, a "Confirm" button will appear. Please click on it to proceed to the next question. Before you press the "Confirm" button, you can change your choice as many times as you like. But once you press the "Confirm" button, you can no longer go back to change your choice.
At the end of the experiment, if you are selected to play a choice question in this part for real, the urn that you selected in that question will be prepared by the experimenter in front of you. The urn will be then covered, and you will draw one ball from the urn without looking inside. The single ball that you draw from the urn will determine your payment.
The following is the instructions for the DFE treatment.
Risk is generated by a random draw, without looking, from an urn containing balls with monetary values written on them. Figure 1 shows the urns that we are going to use.

Figure 1
In each question, you will face two urns that contain balls with different values. You are asked to select the urn from which you prefer to draw a single ball randomly. The value of the ball drawn from the selected urn will determine your payment.  Before you choose the urn that you prefer to draw a ball from, you will learn about the values of balls in both urns. For this, you are asked to sample balls one by one from each urn by clicking the corresponding button "sample left" or "sample right", shown in Figure 2. The value of each ball sampled will be presented on the screen for 2 seconds immediately after each click. For example, if you click on the "sample right" button, you will see the screen in Figure 3 which tells you the monetary value of the ball draw is €27.

Figure 3
The sampling will be without replacement. It means that a ball drawn from the urn will not be put back into the urn. The number of balls remaining in urns will be updated on the screen after each draw. You can sample in whichever order you like. In case you want to take notes of the values that you observe, paper and pen are provided on your desk.
When both urns are empty, and you have observed the value of every ball in both urns, a "Proceed to the choice stage" button will be shown. By clicking on it, the balls will be returned into the urn where they belong. Then you will be directed to the choice stage. The resulting screen is shown in Figure 4. After you made your choice, a "Confirm" button will appear. Please click on it to proceed to the next question. Before you press the "Confirm" button, you can change your choice as many times as you like.
But once you press the "Confirm" button, you can no longer go back to change your choice.
At the end of the experiment, if you are selected to play a choice question in this part for real, the urn that you selected in that question will be prepared by the experimenter in front of you. Then the single ball that you will draw from the urn will determine your payment.
By clicking "Next", you will be directed to a sample question which will familiarize you with the experimental questions.   The analysis was based on unknown objective probabilities rather than observed relative frequencies. Choice patterns were analyzed according to their compatibility with inverse S-shaped probability weighting.

Individual heterogeneity was controlled in statistical analysis
We implemented Bayesian hierarchical estimation procedure as follows. The Goldstein and Einhorn (1987) probability weighting function is ( ) = '( ) '( ) *(+,() ) . The probability of choosing the risky prospect was calculated using Luce (1959)  The individual level noise parameter Q were assumed to come from a lognormal distribution.
Similarly, to facilitate the hierarchical modeling, we used the logarithmic transformation of Q , i.e., Q = ( Q e ), where the prior of Q e assumed to follow ( l , l ). The group level mean, l , was assumed to be uniformly distributed ranging from -2.3 to 2.3, which results in a uniform distribution ranging from 0.1 to 10 in the exponential scale. The group level standard deviation l was uniformly distributed ranging from 0 to 1.33. The upper bound of 1.33 was determined as the standard deviation of the prior distribution of the group level mean, (−2.3, 2.3), following Nilsson et al. (2011, pg. 88).
The MCMC algorithm was implemented in WinBUGS run through R software. Three chains, each with 60000 iterations were run, after a burn-in of 10000 iterations. To reduce the autocorrelation, only every 10th sample was recorded. Convergence was checked by Gelman-Rubin statistics, and by visual inspection of trace plots.  Prelec's (1998) Compound Invariance Family Prelec (1998)'s compound invariance family is given by ( ) = ,'(,opq) ) . The parameter determines the curvature and captures the sensitivity towards changes in probabilities. Here, < 1 indicates inverse S-shape and likelihood insensitivity, and > 1 indicates S-shape and likelihood oversensitivity. The parameter determines the elevation, and it is an index of pessimism. Higher values of indicate less elevation and more pessimism. Table A6.1 and Figure A6.1 show the results of our Bayesian hierarchical estimations with this family.  Estimations with Prelec's (1998)

compound invariance family
The estimation procedure was the same. Prelec's compound invariance family is given by ( ) = ,'(,opq) ) . The ranges of the prior distributions were from 0.1 to 2 for both Q and Q . 2. Check if the direction of rounding is biased in the two groups of DFE and DFD.
We compute the bias caused by rounding in two steps: 1. Calculate the numbers in lotteries without rounding. 2. Take the difference between the actual numbers before rounding (obtained in step 1) and after rounding (the numbers we used in the experiment).
For example, if 92.2 was rounded to 92, then the difference would be 0.2.

Compare with each other (pvalues)
Median Reported are the differences caused by rounding. We conducted Wilcoxon unpaired tests.
Example 3 presents the notes taken by Subject 3409. Although we detect some notes taken by this subject for every choice question, we observe that some of his or her notes mention only about the sampled outcomes but not about their frequencies. For example, the notes concerning the choice question with 95% probability (indicated inside the square) show only the sampled outcomes but not the observed frequencies of those.

Online Appendix 9: Recency effects
We examine the role of recency in our data as follows. We define recency in situations where the majority of the rare outcome observations (i.e., 1 out of 1 for 0.05, 2 out of 2 for 0.10, and 3 out of 4 for 0.20) were observed at the last half of the sequence. We estimate the impact of recency by running random effects logistic regressions, where the dependent variable is over-or under-weighting of probabilities and the independent variable is a dummy variable for recency. The good-and bad-event probabilities are pooled in the estimations summing up 258 observations (6 choices involving rare outcomes for 43 subjects). The odds ratio is estimated as 1.133 indicating a slightly higher odds of overweighing when the rare outcome is observed recently. However, this ratio did not differ from one (p-value=0.673). Therefore, it did suggest a significant recency effect.