Introduction

Applying the concepts of ecosystem services and natural capital in practice means making decisions that consider not only the financial costs and benefits of projects but also the effects of those decisions on the natural environment. For decision makers, to assess the environment together with financial costs and benefits requires comparability between the metrics used to measure change. However, it is common for environmental changes to be measured in a specific unit which is natural to the environmental change itself. For example, measuring changes to climate in terms of CO2e or flooding hazards in terms of risk to populations or households. To allow for comparability, economists use economic values as a common unit of account. For environmental goods, these economic values need to be estimated using non-market valuation techniques—one of the most used method of non-market valuation is the contingent valuation (CV) method, a stated-preference method which surveys people to elicit their values.

The validity of the CV method, however, has been called into question by the repeated observations of violations of procedural invariances; that is to say, the preferences expressed by respondents are observed to differ systematically when changes are made to the manner in which values are elicited (Ariely et al. 2003; Akter et al. 2008; Bateman et al. 2009; Hanley et al. 2009; Ready et al. 2010; Poe 2016). One explanation which has been proposed (Ready et al. 1995; Welsh and Poe 1998; Ready et al. 2001; Flachaire and Hollard 2007; Brouwer 2011; Akter and Bennett 2013) is that respondents are uncertain about their preferences and that their uncertainty precipitates systematically different responses to different question formats.Footnote 1 In other words, it is the process of requiring individuals to express values in CV surveys as if they had well-defined certain preferences that leads to violations of procedural invariance. Testing this hypothesis is the key contribution of this article. Of course, lessons from this study are relevant to other stated-preference methods that are susceptible to the anomalies under investigation in this study, for example, discrete choice experiments.

In this paper, we describe the outcomes of an empirical experiment designed to examine whether uncertainty might provide an explanation for three commonly observed violations of procedural invariance: (i) systematic differences in the values implied by responses to dichotomous choice (DC) CV questions when compared with open-ended (OE) CV questions (DC-OE Disparity), (ii) systematic shifts in implied WTP according to the precision of the figure presented to respondents in DC-CV questions (bid precision effect) and (iii) systematic shifts in implied WTP resulting from differences in the magnitude of an initial bid amount presented to respondents in a CV survey (bid magnitude effect or starting point bias). In each case, we present a plausible explanation as to why each anomaly might be explained through a decision heuristic brought about by respondents trying to answer the question truthfully when their underlying preferences are stable but uncertain. Our experiment involves a split-sample study in which we first present separate treatment groups with the types of CV questions that have been observed to evoke the different anomalies. We follow up with questions that allow those respondents to then reveal their underlying uncertain preferences. The simple hypothesis of our study is that while the initial CV questions may reveal patterns of anomalous responses, the uncertain preferences reported in the follow-up questions should not systematically differ across treatment groups.

If we can show that respondents’ uncertain preferences are invariant to external cues in the initial CV elicitation tasks, then the prognosis for the CV method is rather encouraging; by allowing for the possibility of uncertainty, CV methods can elicit preferences that conform to many of the expectations of standard economic theory.

To assess this proposed explanation for CV, anomalies require an elicitation technique which allows for the expression of uncertainty. One widely applied approach is the multiple-bounded discrete choice (MBDC) method (Welsh and Bishop 1993; Welsh and Poe 1998; Alberini et al. 2003; Vossler et al. 2004; Wang et al. 2013; Mahieu et al. 2014) The MBDC method presents respondents with an ordered list of bids. For each bid, respondents report on a certainty scale their likelihood of being willing to pay that amount. Accordingly, the MBDC method typically presents respondents with a multiple-bounded choice across bid amounts, consistent with the payment card method,Footnote 2 combined with a polychotomous choice from a scale of certainty ranging from definitely yes to definitely no. For each respondent, MBDC elicitation provides data recording a range of values for which that respondent is certain they would pay, we label this the certainty-yes range, and a range of values over which they are certain they would not pay, we label this the certainty-no range. Between those two, there may exist a range of values over which they are uncertain—we label this the uncertainty range.

The article is organised as follows. In the following section, we review the three violations of procedural invariance that form the focus of this study and theorise plausible explanations of those anomalies resulting from uncertain preferences. We then outline our semi-parametric estimator and present the experimental design while briefly describing the case study in which it has been applied. The results of the empirical application are presented next followed by a discussion of the implications of our findings and finally, we conclude with some closing remarks.

Violations of procedural invariance and uncertainty

It has been hypothesised that the underlying uncertainty in individuals’ preferences may explain violations of procedural invariance in CV studies (Ready et al. 2001; Flachaire and Hollard 2007). Here, we focus specifically on three such anomalies: DC-OE disparity, bid-level precision effects and bid-level magnitude effects. While numerous explanations for these anomalies have been presented in the literature, here, we focus on the hypothesis that each anomaly results from the fact that CV questions assume certainty in preferences; the anomalies arise as a result of respondents adopting decision heuristics that allow them to answer those questions when their underlying preferences are actually uncertain.

It is well established in the literature that DC methods of elicitation tend to return higher estimates of WTP than OE methods (Boyle et al. 1996; Brown et al. 1996; Ready et al. 1996). A number of authors have previously argued that uncertainty is an explanation for that observation (Ready et al. 1995, 2001; Welsh and Poe 1998; Flachaire and Hollard 2007). At the core of those arguments is the conjecture that in the presence of uncertainty, respondents interpret OE and DC questions very differently. In particular, when faced with an OE question, respondents are believed to report a value that they are reasonably certain they would pay. In contrast, when presented with a DC question offering a bid amount of which they are uncertain, respondents are believed to react as if the question is asking them whether there is some possibility they would pay that amount. In the words of Flachaire and Hollard (2007), “anomalies come from the fact that, when uncertain, respondents tend to answer yes. Indeed, if the bid belongs to his range of acceptable values, a respondent answers yes...” (p. 192). Ready et al. (2001) report empirical findings that support that assertion. In their data, they observe that respondents who self-report that they are unsure tend to say “yes” when answering DC questions. In contrast, when answering an OE question, respondents will tend to state that they are not prepared to pay that amount. The difference in WTP estimates derived from the two elicitation methods may, therefore, be explained simply by allowing for the possibility of uncertain preferences.

The bid precision anomaly has received relatively little attention in the CV literature but a number of studies from marketing and psychology document the fact that the precision with which figures are presented to respondents matters (Janiszewski and Uy 2008; Loschelder et al. 2014). Thomas et al. (2010) use laboratory experiments to investigate the effect of the roundness or precision of prices on buyer behaviour and show that buyers tend to be more willing to accept posted prices for goods when those prices are presented as a precise figure as opposed to an equivalent-magnitude rounded figure. Importantly, they also observe that the strength of this price-precision effect increases with the level of uncertainty the individual holds for the value of the good. The authors conjecture that respondents use the precision as a signal of price negotiability; they suggest that respondents might be more willing to say “no” to a rounded-figure price, especially when uncertain about whether their value matches that price, in the belief that there may be room to negotiate that price downwards. We contend that a similar heuristic might be at work in DC-CV surveys that attempt to mimic posted-price market settings. The heuristic that leads individuals with uncertain preferences to answer “no” to rounded-figure prices may lead respondents to CV surveys to answer “no” to a rounded bid presented in a DC-CV question.

The bid magnitude anomaly has been observed across a range of CV elicitation formats where it is commonly referred to as starting point bias. A widely documented result is that the bid value offered in an initial DC question systematically influences the response to subsequent valuation questions. A number of interpretations of starting point bias have been proposed, such as the initial value signalling the cost or alternatively acting as an anchor (McFadden 1994; Herriges and Shogren 1996; Flachaire and Hollard 2007; Bateman et al. 2009). A frequently posited explanation is that the initial bid value creates the possibility, at least momentarily, that the value being estimated is near to the initial value. A plausible alternative explanation might be found in uncertain preferences. To illustrate, imagine two individuals with identical but uncertain preferences, one individual is initially offered a low bid and one is offered a high bid. The low bid is comfortably within the certainty-yes range and so, the individual would answer yes they would pay. Conversely, the high bid is comfortably within the certainty-no range and so, that individual would answer no. Both individuals are then asked a second valuation question for the same bid which is within their uncertainty ranges. The individual initially offered the low bid is coming from a state of certainty about paying to a state of uncertainty; whereas the individual initially offered the high bid is coming from a state of certainty about not paying to uncertainty. For the individual coming up from the low bid, a natural reaction might be to reason that, “I was previously certain I would pay now I am not certain so to signal that change in state I’ll answer no”. The reverse is true for the other individual, a way to signal that there is now a possibility they might pay would be to answer yes. In other words, by adopting a simplifying heuristic to deal with their change in certainty, the individuals may want to express their different state of certainty by reversing their answers to the initial question. Again the conjecture is that the anomaly is not the consequence of shifting preferences; but that respondents interpret repeated questions differently based on the magnitude of the previous bid.

As we have shown, the DC-OE anomaly, different responses to precise and rounded bids, and different responses to bids of different magnitude might plausibly be explained through simplifying heuristics adopted by respondents of CV questions with uncertain preferences.Footnote 3 The key prediction of that explanation is that the elicitation procedure does not change respondents’ underlying uncertain preferences but instead might lead them to express those preferences differently when confronted with different forms of CV question. The experiment that we describe subsequently tests for such patterns by eliciting uncertainty ranges and observing if value-irrelevant details of the elicitation procedure lead to variation in the uncertainty ranges.

Modelling uncertain contingent valuation data using a semi-parametric estimator

For the analysis in this paper, we use a novel econometric method for analysing responses to MBDC questions. Our estimator adapts multi-state duration models developed in the medical statistics literature (Frydman 1995; Commenges 2002; Frydman and Szarek 2009) to analyse the progression of respondents’ willingness to pay (WTP) along the money scale. In particular, the estimator we describe is a semi-parametric three-state duration-dependent Markov model which allows us to explore how the certainty range and uncertainty range reported by respondents are affected by various factors (in our case, the type of CV question faced in the initial task) and whether the size of those two ranges affect each other. Our econometric estimator adds to the toolkit of methods previously developed to study uncertain preferences in CV data (Wang 1997; Alberini et al. 2003; Evans et al. 2003; Kobayashi et al. 2012; Mahieu et al. 2017). Our model is similar to Kobayashi et al. (2012) in that both model the thresholds, or the transition between levels of certainty. In the Kobayashi et al. (2012) model, the thresholds are analysed using a Bayesian approach which requires truncated normally distributed error terms. Our estimator can be viewed as an alternative with the parametric assumptions removed. As far as we are aware, this is the first time that this form of multiple-state duration modelling has been used in economic analysis.

As illustrated in Fig. 1, as respondents consider successively larger money amounts, they transition between three states: the certainty of saying yes, uncertainty and the certainty of saying no. This of course implies that there are two associated thresholds: the threshold at which respondents switch from a state of “certainly would pay” to a state of “uncertainty”; and the threshold between a state of “uncertainty” and the state where they “certainly would not pay”. For the analysis in this paper, these two thresholds are defined using the MBDC card as being, respectively, either the WTP amount at which respondents switch from “definitely would pay” to “probably would pay” or that amount at which the respondent switches from “probably would not pay” to “definitely would not pay”.

Fig. 1
figure 1

Three state duration-dependent Markov process showing the transition in respondents’ certainty as money amounts increase

Full size image

To construct our econometric model, we assume that each respondent, i, knows the highest amount they certainly would pay, an amount we label ti and the lowest amount they certainly would not pay, an amount we label xi. The gap between these two values defines their uncertainty range, the width of which we define as wi such that xi = ti + wi.

Accordingly, at the heart of our econometric model is a calculation of the probability of observing a respondent reporting intervals of the width ti and wi. We write that probability as

$$ \Pr \left[{t}_i,{w}_i\right]=\Pr \left[{t}_i\right]\Pr \left[{w}_i\ |\ {t}_i\right] $$
(1)

Observe that we allow for the possibility that the width of the uncertainty range wi may be dependent on the width of the certainty-yes range ti. In the MBDC exercise, respondents reveal information on their preferences over a finely spaced grid defined by the M bid points; 0 = b0 ≤ b1 ≤ b2 ≤  …  ≤ bM ≤ bM + 1 = ∞. Accordingly, our data are discrete in nature identifying only the interval between bid points in which ti and xi fall. Now, imagine that individual i indicates that they are certain they would pay each of the first nit bid amounts. Subsequently, they report that they are in a state of uncertainty over the next niw bid amounts. Accordingly, for all bid amounts, bm, where m > nit + niw, they are certain they would not pay. For the purposes of developing our estimator, we summarise that discrete data using the following dummy variables:

  • \( {\delta}_{ij}^t \)(j = 1, 2, … , M + 1) is a set of dummy variables identifying the certainty-yes range, where \( {\delta}_{ij}^t=1 \) if respondent i stated that they certainly would pay bj (such that \( {\delta}_{ij}^t=1 \) for all j = 1, 2, … , nit intervals) and \( {\delta}_{ij}^t=0 \) otherwise.

  • \( {d}_{ij}^t \)(j = 1, 2, … , M + 1) is a dummy variable indicating the bid interval within which ti must fall. It is identified as the bid interval after the highest bid amount that respondent i indicated they certainly would pay.

The notation is a little different for the state of uncertainty. In particular, we are now concerned with the number of bid intervals over which a respondent reports a state of uncertainty, while, for the time being, we ignore the fact that individuals may enter this state at different bids. For the purpose of clarity, we use k to index the uncertainty range, where k = 1, 2, … , K, and K is the greatest number of bid intervals in the uncertainty range observed in the data. Accordingly,

  • \( {\delta}_{ik}^w\ \left(k=1,2,\dots, K\right) \) is a set of dummy variables identifying the uncertainty range, where \( {\delta}_{ik}^w=1 \) if respondent i stated that they were uncertain (such that \( {\delta}_{ik}^w=1 \) for all k = 1, 2, … , niw intervals) and \( {\delta}_{ik}^w=0 \) otherwise.

  • \( {d}_{ik}^w\ \left(k=1,2,\dots, K\right) \) is a dummy variable indicating the bid interval within which wi must fall. It is identified as the first bid interval before the bid amount that respondent i indicated they certainly would not pay.

Our model adopts the maximally flexible parameterization of Pr[ti] in which a set of parameters pj(j = 1, 2,  … , M + 1) are estimated that capture the probability of respondent i having a certainty-no range that ends in interval j. Accordingly,

$$ \Pr \left[{t}_i\right]=\prod \limits_j{p_j}^{d_{ij}^t} $$
(2)

Following Frydman (1995), we parameterise Pr[wi], that is, the probability that respondent i has an uncertainty range of width wi, using the hazard function. In particular, we specify the hazard function using the logistic form;

$$ {h}_k^w\left({t}_i\right)=\frac{\lambda_k\ {e}^{\beta {t}_i}}{1+{\lambda}_k\ {e}^{\beta {t}_i}}\ \left(k=1,2,\dots, K\right) $$
(3)

where \( {h}_{ik}={h}_k^w\left({t}_i\right) \) represents the probability that respondent i transitions from their uncertainty range to their certainty-no range after k intervals of uncertainty. Observe that the hazard is expressed with maximal flexibility through the estimation of a set of parameters λk (k = 1, 2,  … , K) that define the baseline hazard. At the same time, we allow for the width of certainty-yes range, ti, to influence the hazard through the parameter β. For example, with a positive β, the hazard is increasing with ti, in other words, longer certainty-yes ranges are associated with shorter ranges of uncertainty. Conversely, with a negative β, the hazard is decreasing with ti, in other words, longer certainty-yes ranges are associated with longer ranges of uncertainty (further details on the modelling procedure are provided in Online Resource 1).

Experimental design

Survey respondents each faced a single valuation exercise made up of three tasks followed by socioeconomic questions (Online Resource 2). In task 1, respondents were randomly allocated by an unseen process into one of eight treatment groups, seven groups received a single-bounded DC question at a specific bid and the other group received an OE question. The DC bids were chosen according to two criteria: that they represented reasonable values suggested by prior focus group testing and that they generated data that allowed subsequent testing of our hypotheses. Accordingly, five DC bids of £5, £30, £60, £100 and £150 were included to provide a range in magnitude. In addition, we include two precise bids of £28.70 and £31.30 that could be contrasted with the similar magnitude bid of £30. The use of the open-ended method, a method rarely used by economists in recent times (Johnston et al. 2017) and much maligned for its lack of incentive compatibility properties (Carson and Groves 2007; Carson et al. 2014), provides us with an important comparison, a group of respondents who see no external cues regarding the WTP value during the initial valuation exercise.

Task 2 and task 3 allowed respondents to express uncertainty about their WTP and were completed by all respondents regardless of their treatment group. Following the procedure of Li and Mattsson (1995) and Ready et al. (2001), task 2 presented respondents with a follow-up question that required them to state the level of certainty they attached to their DC or OE answer from task 1. Five responses were available:

  • I definitely would pay the amount of money.

  • I probably would pay the amount of money.

  • I am not sure if I would pay this amount of money.

  • I probably would not pay the amount of money.

  • I definitely would not pay the amount of money.

Task 3 used a novel version of the MBDC method to establish the values over which respondents are certain and uncertain. The MBDC card deliberately rejects the uneven (e.g. exponentially increasing) spacing of value amounts typical of most payment cards (Rowe et al. 1996; Welsh and Poe 1998; Kerr 2000) so as to avoid any implicit suggestion that higher amounts are less plausible. Even spaced, single-unit increments obviously result in long payment cards; however, to avoid respondent fatigue, cards were given to the respondent to fill in so that they could quickly skip all payment levels regarding which they had complete certainty and just focus on defining the uncertainty thresholds. This allowed respondents to work rapidly through the bids amounts. The bids on the MBDC card ranged from £1 to £500, increasing in £1 increments, this range was held constant for all respondents to mitigate the influence of range effects on WTP. The bids were presented over two pages which were shown to respondents in advance (Online Resource 3). At the start of task 3, the interviewer marked on the MBDC card the response that the respondent gave to tasks 1 and 2, for example if they were in the £5 DC treatment group and they said yes in task 1 and that they probably would pay that amount in task 2 the interviewer ticked the probably would pay box next to £5. The respondent was then asked to consider the next bid and indicate their level of certainty about paying that amount. At this point, the MBDC card was handed over to the respondent and the respondent worked through the other amounts themselves.

Using the split-sample experiment outlined, we test the hypothesis that it is the process of requiring individuals to express values in CV surveys as if they had well-defined certain preferences that leads to violations of procedural invariance. To test this, we allow respondents to express uncertain preferences and test these responses for invariance to external cues in the elicitation procedure.

For the three anomalies, we focus in on specific data from the survey. For the DC-OE disparity, we contrast responses to task 1 from those offered a DC question to those offered an OE question. We then recode those responses to the same level of certainty (definitely would pay), using certainty responses from task 2. For the DC treatment groups, respondents who were not definitely certain in task 2 were recoded to “No”. For the OE treatment group, we could not recode in the same way because those respondents had not received a referendum-type question and so instead used the response from task 3 to estimate a WTP value they were definitely certain of paying. For the bid precision anomaly, we use the £28.70, £30 and £31.30 groups, initially testing for any differences in task 1 responses between the precise and rounded groups. Subsequently, we use task 3 to test for any variance in the uncertainty responses, in particular, the number of responses with positive WTP values, the size of the reported uncertainty ranges and the lower and upper bounds of those ranges. This process is repeated for the bid magnitude anomaly using the £5, £30, £60, £100 and £150 treatment groups. If uncertainty is a good explanation, we would expect to observe consistent uncertainty ranges across the treatment groups.

Implementation

The degree of familiarity (or unfamiliarity) with the environmental good being valued could clearly influence the issue of response uncertainty. For this reason, we chose to study a good that is reasonably familiar to survey respondents. Our specific case study concerns potential improvements in coastal protection (extending the size of the beach through the installation of more groynes) in the town of Southwold in Suffolk, UK. One to one in-person interviews were conducted over the summer of 2004 by four interviewers at three locations close to areas that would receive the additional coastal protection if the project were to go ahead. The interviews were restricted to residents of the UK who included both local residents and those visiting from elsewhere in the UK. The average respondent visited the beaches of Southwold approximately five times a year and so had a good level of knowledge about the beaches. To control for any differences in knowledge within the respondent sample, the coastal management proposal was described by the interviewer and illustrated with maps and visual representations of the site before and after. Survey respondents were informed that the existing defences would be maintained by government funding but that additional improvements would require them to contribute through an increase in general taxation.

Results

The study collected a sample of 952 respondents across eight treatments. Of those, 36 were classified as unusable for the subsequent analysis; exclusions being due to incomplete MBDC tasks (e.g. stating only a single “not sure” figure and no values for other polychotomous choice options). In addition to this, 4 respondents ticked that they were certain they would pay all the way up to £500 (the upper limit of the payment card). This data, although possibly very important for total WTP estimates in standard CV analyses, fails to provide any information about the location or width of the range of values to which the respondent is uncertain and so is ignored for the purposes of this article. As such, the total usable sample was 916 respondents, with the OE group containing 272 respondents and the DC groups each containing between 85 and 95 respondents. Online Resource 4 provides details of the socioeconomic composition of each treatment group. No significant differences are observable across socioeconomic characteristics in the eight treatments, suggesting that the randomisation to treatment groups was successful.

Violation of procedural invariance: (1) Dichotomous Choice-Open Ended disparity

Our experiment definitively replicates the familiar disparity in preferences between DC and OE elicitation. Figure 2 summarises the preferences expressed by each of the five (rounded) DC treatment groups and the OE treatment group in task 1. The markers indicate the proportion of respondents willing to pay the bid presented to their DC treatment group. Preferences for the OE treatment group are summarised through the WTP survivor function (calculated using the Kaplan-Meier estimator). For clarity, the function plots out the proportion of respondents who stated a WTP greater than or equal to each amount. At each bid, the preferences elicited using DC questions imply substantially higher WTP than those elicited with OE.

Fig. 2
figure 2

Proportion of “Yes” responses to dichotomous choice questions (task 1) and empirical survivor function of open-ended treatment group (task 1) against bid amount/willingness to pay

Full size image

Our central question is whether this DC-OE disparity can be explained as resulting from uncertain preferences. Ready et al. (2001) hypothesise that in a DC setting, a respondent may state that they are willing to pay a bid amount lying in their uncertainty range, but, in response to an OE question, submit a WTP value from the bottom of that range. We test that hypothesis by comparing responses to the DC and OE questions once they have both been standardised to the same level of certainty; following Ready et al. (2001), we standardise on amounts that respondents “definitely would pay” from responses to task 2.Footnote 4 The results of that analysis are reported in Table 1.

Table 1 Unstandardised and certainty-standardised acceptance proportions under open-ended and dichotomous choice elicitation methods
Full size table

In line with the hypothesis, once responses have been standardised to the same level of certainty, we see considerable convergence between DC and OE responses. Contrary to the findings of Ready et al. (2001), however, our data suggest that DC respondents continue to indicate higher levels of WTP than respondents to OE. Indeed, a statistical comparison shows those differences to be significant at the £150 level of WTP (p value 0.011) and marginally significant at the £30 level of WTP (p value 0.051).

Ready et al. (2001) also report that nine of the 11 respondents (82%) who classed their level of certainty as “not sure” responded “yes” to the DC question. This is taken as supporting evidence of uncertain respondents answering affirmatively to DC elicitation questions. Unfortunately, our data provides less compelling evidence. Only 12% of the 25 respondents that classed their level of certainty as “Not Sure” opted to answer “yes”.

Violations of procedural invariance: (2) bid precision effect

Our second stated-preference anomaly concerns systematic differences in responses in DC elicitation resulting from differences in the precision of the bid. To study that anomaly, we presented different treatment groups with bids of £28.70, £30 and £31.30. Confirming previous evidence from the marketing literature (e.g. Thomas et al. (2010)), our data shows that substantially fewer respondents answered “yes” to the rounded bid of £30 (34%) than answered “yes” to the precise bids of £28.70 and £31.30 (50%) in task 1 (p value 0.011). A precise bid, it appears, encourages higher levels of bid acceptance than a rounded bid.

Again, our focus is on discovering whether uncertain preferences might explain this bid precision effect. To assess that hypothesis, we begin by considering whether bid precision affects the number of respondents indicating positive WTP. MBDC data from task 3 shows that those offered a precise bid were more likely to report a positive WTP, 64% for £28.70 and 53% for £31.30, than those offered a rounded bid, 51% for £30. While the difference in proportions for the £28.70 and £30 groups is statistically significant at the 10% level of confidence (p value 0.098), overall, there is only weak evidence to suggest that bid precision systematically impacts on the likelihood of respondents stating a positive WTP.

Next, we explore in detail the uncertainty ranges reported in the MBDC exercise by respondents with a positive WTP. Table 2 compares the mean amount the respondent is certain they would pay and the mean uncertainty ranges. As reported in the final column of Table 2, we see no evidence of statistically significant differences in either the level of WTP or the range of uncertainty in WTP between those offered precise or rounded DC bids.

Table 2 Summary of uncertainty ranges and percentage of responses with lower or upper bounds of their uncertainty range close to the dichotomous choice bids offered in task 1 for dichotomous choice treatment groups £28.70, £30 and £31.30
Full size table

The means of the data provide evidence that supports the hypothesis that uncertain preferences are unaffected by bid-level precision. A rather different picture emerges, however, as we begin to disaggregate the MBDC data. Observe the bottom half of Table 2 which explores the degree to which the lower and upper bounds of respondents’ uncertainty ranges are shaped by the DC bid they encounter. The table lists the numbers of respondents in each treatment group for whom the minimum or maximum of their uncertainty range is close to that bid. We define “close” as being in the range £28 to £30 for the £28.70 group, £29 to £31 for the £30 group and £30 to £32 for the £31.30 group. Notice that a substantially larger proportion of respondents report an uncertainty range that begins close to the bid when that bid is precise (56% for £28.70 and 33% for £31.30) than when that bid is rounded (21% for the £30 group) (p value of 0.003). Though most of the work is being done by the comparison of the £28.70 to £30 groups (p value 0.000).Footnote 5 This result is robust to different definitions of “closeness” with definitions ranging from the nearest £1 to the nearest £5 producing qualitatively identical results.

Table 2 also reveals that the precision of the bid only appears to focus the lower bound of uncertainty ranges and not the upper bound. A statistical comparison of the proportions of respondents with an upper bound to their uncertainty range close to their bid reveals no significant differences between precise-figure and rounded-figure treatment groups (p value 0.835).

Violations of procedural invariance: (3) bid-level magnitude effect

Our investigation of the bid magnitude effect focuses on the £5, £30, £60, £100 and £150 treatment groups. We begin our analysis by studying the proportion of respondents in each DC treatment group that reported a positive WTP amount in task 3, the MBDC question. The clear anomaly in those data is the £5 group, where some 65% of respondents reported some positive level of WTP in the MBDC exercise. Statistical comparison through a series of pairwise tests reveals significantly higher proportions of positive responses for that group compared with all other bid levels; at the 1% level of significance for the £60 and £100 groups (p value 0.000 and p value 0.005) and at the 10% level of significance for £30 and £150 groups (p value 0.058 and p value 0.074). For all other pairwise comparisons, no significant differences are observed. Barring the £5 group, it appears that bid magnitude does not significantly impact on the proportion of respondents reporting a positive WTP.

Table 3 repeats our analysis of the focusing effect; that is to say, the degree to which respondents fix either the lower or upper bounds of their reported uncertainty ranges to the DC bid. Again, we observe no discernible impact of bid magnitude on the upper bound of the reported uncertainty range (p values 0.310 to 0.784). Moreover, amongst the £30, £60, £100 and £150, we see no significant impact on the lower bound of their uncertainty range. The only notable exception is again the £5 group, where some 51% of respondents report a lower bound to their uncertainty range close to the DC bid compared with 10 to 22% across other groups. Again, those differences prove significant at the 1% level for all pairwise comparisons with the £5 group (p values 0.000 to 0.009). Of course, £5 could just happen to be a more common lower bound across the survey respondents in general. The final column of the table clearly shows this is not the case as the £5 treatment group has a much high percent of respondents with lower bounds of their uncertainty range close to £5 (51%) than all other groups (3 to 21%). Overall, the data provides some evidence to suggest that the £5 bid is sufficiently “precise” as to invoke similar responses to the precise-figure bids.

Table 3 Percentage of responses with lower or upper bounds of their uncertainty range close to the dichotomous choice bids offered in task 1
Full size table

Looking once again at the data in Table 3, it is of course rather surprising that amongst the rounded bid groups, we observe almost the same proportion of respondents stating that their uncertainty range starts close to the bid they are presented with independent of whether that bid is £30, £60, £100 or £150. If uncertainty ranges are unaffected by the magnitude of bids, it would perhaps be more likely to see those proportion falling.

To understand this result better, Fig. 3 shows the average uncertainty range reported in the MBDC exercise with the OE group represented by the lower-most horizontal bar with the DC treatment groups above. If the uncertainty ranges were invariant to the magnitude of the bid presented in the first task, we would expect to see the bars in Fig. 3 stacked vertically. It is clear that this is not the case, Fig. 3 shows an upward trend in the mean lower bound of the uncertainty range from £9.80 for those offered £5 to £45.40 for those offered £150, the only exception to this upward trend is the mean lower bound for the £100 group. There is a similar upward trend in the mean width of the uncertainty range with the exception again being the £100 group which has the largest mean uncertainty range width. An F-test of equality of means between the multiple treatment groups reveals significant difference at the 99% confidence level showing that at least one of the treatment groups has a different mean lower bound and different width of their uncertainty range to another treatment group (p value 0.000 and 0.007).

Fig. 3
figure 3

Uncertainty ranges for mean willingness to pay for respondents allocated to different initial dichotomous choice bid levels or to the open-ended elicitation method

Full size image

With regard to the effect of bid magnitude on stated uncertainty ranges, we are left with a final question; what causes the width of the average uncertainty range to increase as bid magnitude increases. One possibility is that respondents with higher WTP simply have a larger uncertainty range. In that case, presenting respondents with larger magnitude bids will have an indirect effect on uncertainty ranges; larger magnitude bids result in higher overall levels of WTP which in turn result in increases in reported uncertainty ranges. Alternatively, the bid magnitude may have a direct effect on uncertainty ranges; that is to say, the magnitude of the bid itself induces greater uncertainty in respondents’ preferences.

To understand which of those two hypotheses is best supported by out data, we employ our semi-parametric estimator. We define a set of dummy variables q0 to q4 with q0 defining the £5 treatment and q1 to q4 defining the other DC treatments (such that, q1 = £30 DC treatment and q4 = £150 DC treatment) and use those to parameterise the two “WTP state durations”; width of the certainty-yes range (t) and width of the uncertainty range (w). Parameters α= [α0α1 … α4] capture the impact of DC treatment on the width of the certainty-yes range; β captures the impact of level of WTP on width of the uncertainty range while β0β1 …  β4 capture the independent effect of DC treatment on the width of the uncertainty range. For the purposes of identification, we set β0 = 0 and α0 = 0 such that the £5 treatment forms our comparator group.

Table 4 reports parameters of the estimated model. The first two columns report the parameters associated with the probability of transitioning from a state of “certainly would pay” to one of “uncertainty”. Observe that the probability of transition for each of the DC treatment groups is significantly different from that of £5 treatment group at greater than the 99.9% confidence level. Moreover, the treatment group parameters are all negative indicating that individuals offered an initial bid of £30 or more had significantly lower transition hazards compared with the £5 group. In other words, respondents in those treatment groups are more likely to report a longer certainty-yes range. Observe also that the hazard tends to decrease for higher bid amounts; that is to say, the larger the magnitude of bid presented in the initial task, the larger the amounts that respondents continue to report they are certain they would pay. Reflecting the pattern seen in Fig. 3, the only exception is the £100 group, which appears to exert a weaker pull on the lower bound of respondents’ uncertainty ranges than would be expected from the pattern of the other rounded bids.

Table 4 Semi-parametric estimator output
Full size table

The parameter estimates reported in the final two columns of Table 4 are those associated with the width of respondents’ uncertainty ranges. Since the model is parameterised in terms of the hazard function, the highly significant and negative β reveals that the higher up the WTP scale a respondent enters uncertainty, the smaller their transition probability is for exiting uncertainty. In other words, respondents who state higher certain WTP amounts also exhibit wider uncertainty ranges. Now, consider the parameters estimated on the treatment dummy variables. The results show evidence that, having controlled for the WTP-level effect, only the £100 group has a significant larger uncertainty range than might be expected. For each of the other groups, we cannot reject the hypothesis that the DC bid only impacts on the uncertainty range indirectly by altering the overall magnitude of WTP.

Discussion

This article set out to explore the hypothesis that common violations of procedural invariance observed in CV studies may arise because respondents with uncertain preferences are required to answer as if those preferences were precisely defined even when valuing complex and unfamiliar environmental non-market goods. To answer that, we explored three common elicitation anomalies, DC-OE disparity, bid precision effect and bid magnitude effect (starting point bias) through the analysis of CV survey designed to specifically allow for the expression of uncertainty.

Our data provide little support for the Ready et al. (2001) hypothesis that DC-OE disparity can be resolved through an uncertainty explanation. We observed some evidence of convergence when answers under the two formats were compared at the same level of respondent certainty but significant differences remained. In addition, our data contradicts one key finding of Ready et al. (2001) as we found no increased propensity for respondents to answer “yes” when presented with a DC bid that falls in their uncertainty range. Indeed, our data suggested the opposite tendency with a large majority of respondents in those circumstances answering “no”. The contradictions between the empirical results of this paper and those of Ready et al. (2001) along with the failure amongst economists to agree on a definitive method to measure uncertainty (Vossler et al. 2003; Akter et al. 2008) suggest that CV methods that attempt to recode or calibrate WTP values according to respondent uncertainty should be treated cautiously before being applied to important policy questions.

Our data also indicate that accounting for preference uncertainty does not resolve the bid precision anomaly or the bid magnitude anomaly. In both cases, we observe uncertainty ranges that are malleable to the nature of the bid presented in the initial DC valuation question. In particular, increases in the precision of the bid elicit uncertainty ranges whose lower bounds are more closely focused on that bid amount. The focusing of the lower bound of the uncertainty range suggests that the bid precision presented to respondents in a DC question not only alters their readiness to answer “yes” to that question but also fundamentally alters the nature of the underlying uncertain preferences they report. In addition to the precision of the bid amount, other psychological pricing effects such as “odd pricing”—where a price of 99p seems a better deal for customers—may also be part of the explanation (Koschate-Fischer and Wüllner 2017), One theory is that people pay particular attention to the left digits of a price and save on mental application by downgrading the importance of the other digits (Thomas and Morwitz 2005; Shampanier et al. 2007).

Similarly, increasing the magnitude of the bid caused the lower bound of the uncertainty range to shift upwards. Our data also indicate that differences in the width of uncertainty ranges are mostly captured by the indirect effect of bid magnitude on the overall level of WTP. The only exception is the £100 treatment group who report a significantly wider uncertainty range than might be expected from the indirect effect alone. We postulate that this may also result from a bid precision effect with more rounded bids eliciting wider uncertainty ranges as it acts as a weaker anchor for fixing the lower bound of the uncertainty range.

Overall, preferences expressed in response to a single-bounded dichotomous choice bid level are substantially and significantly distorted simply through making very minor alterations in the precision of a bid. This results contrast with contemporary advice on best practice for CV design (Johnston et al. 2017). Interestingly, our data also provides some insights as to the preferences reported by respondents when no cue is given. Responses to OE elicitation return WTP estimates that are conservative compared with those implied by DC elicitation. Indeed, our follow-up MBDC question reveals that respondents draw their OE values consistently from the low end of their uncertainty range. Moreover, observing Fig. 3 and contrasting the average uncertainty range of the OE group to those in the DC treatments suggest that OE elicitation does not result in ranges that are excessively wide and, in the opinion of these authors, suggest WTP values in an entirely plausible range.

The results of this study have to be handled with care, for a number of reasons. The survey data although a large sample overall, reduces to sample groups of just below 100 for each DC treatment. This sample is reduced further when considering respondents who stated a willingness to pay off greater than £0. In addition, the survey data does not allow for analysis of protest voting amongst respondents, an issue which has been shown to affect the estimation of WTP values (Jorgensen et al. 1999). This could be a potential confounder for the OE elicitation results as that method is known to induce respondents to strategically misrepresent their preferences. However, if one presumes that a possible manifestation of such strategic behaviour would be to indicate zero WTP, we find no evidence to indicate that happens with greater frequency in the OE treatment compared with the DC groups (47% of respondents in the OE treatment group stated a zero WTP in the MBDC exercise compared with 48% across all DC groups).

Finally, raising the issue of uncertainty may itself stimulate additional consideration of values amongst respondents (Carson et al. 2007; Bateman et al. 2008; Poe and Vossler 2009) or allow for strategic behaviour (Carson and Groves 2007). However, as the approach to uncertainty elicitation is constant across all respondents, any effects of such introspection should not differ across elicitation format.

Conclusions

Directly asking respondents in CV surveys their WTP is one of the few quantitative methods available to assess full economic value (including both use and non-use values) of non-market goods. It therefore remains vitally important to better understand the reasons for consistently observed violations of procedural invariance in such surveys. Unfortunately, our study shows that allowing for uncertainty in preferences may not provide the simple panacea to the problems of CV anomalies.

Our experiment provides some noteworthy new insights into the ways in which the preferences expressed in response to CV questions may be shaped by the nature of those questions. Most significantly, we find that in a single-bounded DC question, significantly more respondents answer affirmatively when presented with a precise bid than when presented with a rounded bid of the same approximate magnitude. Second, we find that the uncertainty ranges reported in MBDC elicitation respond in systematic ways to cues given by bids in initial DC tasks. Uncertainty ranges shifted up the WTP scale by increasing the magnitude of that bid and the lower bound (but not the upper bound) of uncertainty ranges are increasingly focused on the bid the greater the precision of the bid. While much has been made of the incentive compatibility of the single-bounded DC approach (Carson and Groves 2007; Carson et al. 2014), a clear conclusion of our research is that that elicitation method is not robust to other factors that may influence expressions of preferences.

Despite a long tradition of claims for the theoretical superiority of DC elicitation methods (Arrow et al. 1993; Carson and Groves 2007), our results suggest an empirical reality; the cues provided by bids act to systematically alter responses to single-bounded DC questions and shape the expressions of uncertain preferences reported in a follow-up MBDC exercise. Given those findings, we reach the conclusion that there may be some merit in revisiting the much-maligned open-ended form of value elicitation.