“I Was Trying to Do the Maths”: Exploring the Impact of Risk Communication in Discrete Choice Experiments
Risk is increasingly used as an attribute in discrete choice experiments (DCEs). However, risk and probabilities are complex concepts that can be open to misinterpretation, potentially undermining the robustness of DCEs as a valuation method. This study aimed to understand how respondents made benefit–risk trade-offs in a DCE and if these were affected by the communication of the risk attributes.
Female members of the public were recruited via local advertisements to participate in think-aloud interviews when completing a DCE eliciting their preferences for a hypothetical breast screening programme described by three attributes: probability of detecting a cancer; risk of unnecessary follow-up; and cost of screening. Women were randomised to receive risk information as either (1) percentages or (2) percentages and icon arrays. Interviews were digitally recorded then transcribed to generate qualitative data for thematic analysis.
Nineteen women completed the interviews (icon arrays n = 9; percentages n = 10). Analysis revealed four key themes where women made references to (1) the nature of the task; (2) their feelings; (3) their experiences, for instance making analogies to similar risks; and (4) economic phenomena such as opportunity costs and discounting.
Most women completed the DCE in line with economic theory; however, violations were identified. Women appeared to visualise risk whether they received icon arrays or percentages only. Providing clear instructions and graphics to aid interpretation of risk and qualitative piloting to verify understanding is recommended. Further investigation is required to determine if the process of verbalising thoughts changes the behaviour of respondents.
The authors are grateful for feedback received at the Society for Medical Decision Making’s 36th and 37th annual meetings. We are also grateful to experts Professor Gareth Evans and Professor Tony Howell for their clinical input on framing the choice question for a hypothetical breast screening programme in the UK. We are grateful to Dr Michelle Harvie for her assistance in specifying the relevant background questions and Ms Paula Stavrinos for her feedback on the introductory materials and video used in the survey. We are also grateful to Professor Stephen Campbell of the Centre for Primary Care at The University of Manchester for providing his thoughts and comments on the interview schedule and providing feedback as the themes developed.
DR and KP conceptualised the research question, helped develop the interview schedule, read transcripts, provided feedback on the analysis of the data and contributed to the writing of the manuscript. CV arranged and conducted all interviews, completed the analysis of the qualitative data, and prepared the first draft of the manuscript.
Ethical approval for this study was granted by The University of Manchester’s Research Ethics Committee. All participants provided consent.
Preparation of this manuscript was made possible by a grant awarded by The Swedish Foundation for Humanities and Social Sciences (Riksbankens Jubileumsfond) for a project entitled ‘Mind the Risk’. Caroline Vass was in receipt of a National Institute for Health Research (NIHR) School for Primary Care Research (SPCR) Ph.D. Studentship between October 2011 and 2014. The views expressed in this article are those of the authors and not of the funding bodies.
Conflict of interest
Caroline Vass, Dan Rigby and Katherine Payne declare that they have no conflicts of interest.
- 3.McFadden D. Conditional logit analysis of qualitative choice behaviour. In: Zarembka P, editor. Front economy. New York: Academic Press Inc; 1974. p. 105–42.Google Scholar
- 5.Smyth RL, Watzin MC, Manning RE. Investigating public preferences for managing lake champlain using a choice experiment (England: Rubenstein School of Environment and Natural Resources, Aiken Center, University of Vermont, Burlington, VT 05045, USA). J Environ Manag [Internet]. 2009;90(1):615–23.CrossRefGoogle Scholar
- 13.FDA. Patient preference information voluntary submission, review in premarket approval applications, humanitarian device exemption applications and de novo requests, and inclusion in decision summaries and device labeling. U.S. Dep. Heal. Hum. Serv. Food Drug Adm. Cent. Devices Radiol. Heal. 2016; FDA-2015-D.Google Scholar
- 19.Watson V, Ryan M, Watson E. Valuing experience factors in the provision of chlamydia screening: an application to women attending the family planning clinic (United States: Health Economics Research Unit, University of Aberdeen, Foresterhill, Aberdeen, UK). Value Health [Internet]. 2009;12(4):621–3.CrossRefGoogle Scholar
- 27.Cheraghi-Sohi S, Hole AR, Mead N, McDonald R, Whalley D, Bower P, et al. What patients want from primary care consultations: a discrete choice experiment to identify patients’ priorities (United States: National Primary Care Research and Development Centre (NPCRDC), University of Manchester, Manchester, United Kingdom). Ann Fam Med [Internet]. 2008;6(2):107–15.CrossRefGoogle Scholar
- 29.Ericsson KA, Simon HA. Protocol analysis: verbal reports as data (revised edition). Cambridge: MIT Press; 1993.Google Scholar
- 35.Website center for bioethics and social sciences in medicine. University of Michigan. Available from: www.iconarray.com. Accessed 15 Aug 2018.
- 37.Lincoln YS, Guba EG. Naturalistic inquiry. Evaluation Program Plann. Thousand Oaks: Sage Publications; 1985.Google Scholar
- 40.QSR International Pty Ltd. NVivo qualitative data analysis software. Version 10; 2012.Google Scholar
- 42.Hole AR. Small-sample properties of tests for heteroscedasticity in the conditional logit model. Econ Bull. 2006;3:1–14.Google Scholar
- 43.Czajkowski M, Barczak A, Budziński W, Giergiczny M, Hanley N. Within- and between-sample tests of preference stability and willingness to pay for forest management. Univ St Andrews Discuss Pap Environ Econ. 2014;6:1–24.Google Scholar
- 46.Hess R, Siegrist M. Risk communication with pictographs: the role of numeracy and graph processing. Judgm Decis Mak. 2011;6(3):263–74.Google Scholar
- 53.Varian HR. Microeconomic analysis. 3rd ed. New York: W. W. Norton & Company; 1992.Google Scholar
- 59.Durning SJ, Artino AR, Beckman TJ, Graner J, Van Der Vleuten C, Holmboe E, et al. Does the think-aloud protocol reflect thinking? exploring functional neuroimaging differences with thinking (answering multiple choice questions) versus thinking aloud. Med Teach. 2013;35(9):720–6.CrossRefPubMedGoogle Scholar