Skip to main content
SpringerLink
Log in

Making Use of Technology to Improve Stated Preference Studies

  • Practical Application
  • Published:
The Patient - Patient-Centered Outcomes Research Aims and scope Submit manuscript

Abstract

The interest in quantifying stated preferences for health and healthcare continues to grow, as does the technology available to support and improve health preference studies. Technological advancements in the last two decades have implications and opportunities for preference researchers designing, administering, analysing, interpreting and applying the results of stated preference surveys. In this paper, we summarise selected technologies and how these can benefit a preference study. We discuss empirical evaluations of the technology in preference research, with examples from health where possible. The technologies reviewed include serious games, virtual reality, eye tracking, innovative formats and decision aids with values clarification components. We conclude with a critical reflection on the benefits and limitations of implementing (often costly) technology alongside stated preference studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Soekhai V, de Bekker-Grob EW, Ellis AR, Vass CM. Discrete choice experiments in health economics: past, present and future. Pharmacoeconomics. 2019;37:201–26.

    Article  PubMed  Google Scholar 

  2. Mahieu P-A, Andersson H, Beaumais O, Crastes dit Sourd R, Hess S, Wolff F-C. Stated preferences: a unique database composed of 1657 recent published articles in journals related to agriculture, environment, or health. Rev Agric Food Environ Stud. 2017;98:201–20.

    Article  Google Scholar 

  3. Ryan M, Gerard K. Using discrete choice experiments to value health care programmes: current practice and future research reflections. Appl Health Econ Health Policy. 2003;2:55–64.

    PubMed  Google Scholar 

  4. Loomis J. What’s to know about hypothetical bias in stated preference valuation studies? J Econ Surv. 2011;25:363–70.

    Article  Google Scholar 

  5. Lindhjem H, Navrud S. Using internet in stated preference surveys: a review and comparison of survey modes. Int Rev Environ Resour Econ. 2011;5:309–51.

    Article  Google Scholar 

  6. Drost E. Validity and reliability in social science research. Educ Res Perspect. 2011;38:105–23.

    Google Scholar 

  7. Sculpher MJ, Pang FS, Manca A, Drummond MF, Golder S, Urdahl H, et al. Generalisability in economic evaluation studies in healthcare: a review and case studies. Health Technol Assess (Rockv). 2004;8:iii–117.

    CAS  Google Scholar 

  8. Vass C, Boeri M, Karim S, Marshall D, Craig B, Ho KA, et al. Accounting for preference heterogeneity in discrete-choice experiments: an ISPOR Special Interest Group report. Value Health. 2022;25:685–94.

    Article  PubMed  Google Scholar 

  9. Train K. Discrete choice methods with simulation. 2nd ed. Cambridge: Cambridge University Press; 2009. https://doi.org/10.1017/CBO9780511805271

    Book  Google Scholar 

  10. Hole AR. DCREATE: stata module to create efficient designs for discrete choice experiments. Stat Softw Compon. 2015;S458059:1–22.

    Google Scholar 

  11. Gu Y, Hole AR, Knox S. Fitting the generalized multinomial logit model in Stata. Stata J. 2013;13:382–97.

    Article  Google Scholar 

  12. van Cranenburgh S, Collins AT. New software tools for creating stated choice experimental designs efficient for regret minimisation and utility maximisation decision rules. J Choice Model. 2019;31:104–23.

    Article  Google Scholar 

  13. Karim S, Craig BM, Vass C, Groothuis-Oudshoorn CGM. Current practices for accounting for preference heterogeneity in health-related discrete choice experiments: a systematic review. Pharmacoeconomics. 2022;40:943–56.

    Article  PubMed  Google Scholar 

  14. Evans JR, Mathur A. The value of online surveys: a look back and a look ahead. Internet Res. 2018;28:854–87.

    Article  Google Scholar 

  15. Lefever S, Dal M, Matthíasdóttir Á. Online data collection in academic research: advantages and limitations. Br J Educ Technol. 2007;38:574–82.

    Article  Google Scholar 

  16. Forbes H, Oprescu FI, Downer T, Phillips NM, McTier L, Lord B, et al. Use of videos to support teaching and learning of clinical skills in nursing education: a review. Nurse Educ Today. 2016;42:53–6.

    Article  PubMed  Google Scholar 

  17. Charvin M, Launoy G, Berchi C. The effect of information on prostate cancer screening decision process: a discrete choice experiment. BMC Health Serv Res. 2020;20:467.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Lim SL, Yang JC, Ehrisman J, Havrilesky LJ, Reed SD. Are videos or text better for describing attributes in stated-preference surveys? Patient. 2020;13:401–8.

    Article  PubMed  Google Scholar 

  19. Smith IP, Whichello CL, de Bekker-Grob EW, van Mölken MPMHR, Veldwijk J, de Wit GA. The impact of video-based educational materials with voiceovers on preferences for glucose monitoring technology in patients with diabetes: a randomised study. Patient. 2023;16:223–37.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Clark R. Learning from serious games? Arguments, evidence, and research suggestions. J Gaming Virtual Worlds. 2007;47:56–9.

    Google Scholar 

  21. Vass CM, Davison NJ, Vander Stichele G, Payne K. A picture is worth a thousand words: the role of survey training materials in stated-preference studies. Patient. 2020;13:163–73.

    Article  PubMed  Google Scholar 

  22. Vass CM, Boeri M, Poulos C, Turner AJ. Matching and weighting in stated preferences for health care. J Choice Model. 2022;44:100367.

    Article  Google Scholar 

  23. Flogie A, Aberšek B, Aberšek MK, Lanyi CS, Pesek I. Development and evaluation of intelligent serious games for children with learning difficulties: observational study. JMIR Serious Games. 2020;8:e13190.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Graafland M, Schraagen JM, Schijven MP. Systematic review of serious games for medical education and surgical skills training. Br J Surg. 2012;99:1322–30.

    Article  CAS  PubMed  Google Scholar 

  25. Flynn TN, Bilger M, Malhotra C, Finkelstein EA. Are efficient designs used in discrete choice experiments too difficult for some respondents? A case study eliciting preferences for end-of-life care. Pharmacoeconomics. 2016;34:273–84.

    Article  PubMed  Google Scholar 

  26. Louviere JJ, Islam T, Wasi N, Street D, Burgess L. Designing discrete choice experiments: do optimal designs come at a price? J Consum Res. 2008;35:360–75.

    Article  Google Scholar 

  27. DeShazo JR, Fermo G. Designing choice sets for stated preference methods: the effects of complexity on choice consistency. J Environ Econ Manage. 2002;44:123–43.

    Article  Google Scholar 

  28. Jonker MF, Donkers B, de Bekker-Grob E, Stolk EA. Attribute level overlap (and color coding) can reduce task complexity, improve choice consistency, and decrease the dropout rate in discrete choice experiments. Health Econ. 2019;28:350–63.

    Article  PubMed  Google Scholar 

  29. Rossetti T, Hurtubia R. An assessment of the ecological validity of immersive videos in stated preference surveys. J Choice Model. 2020;34:100198.

    Article  Google Scholar 

  30. Zhao Y, van den Berg PEW, Ossokina IV, Arentze TA. Comparing self-navigation and video mode in a choice experiment to measure public space preferences. Comput Environ Urban Syst. 2022;95:101828.

    Article  Google Scholar 

  31. Farooq B, Cherchi E, Sobhani A. Virtual immersive reality for stated preference travel behavior experiments: a case study of autonomous vehicles on urban roads. Transp Res Rec. 2018;2672:35–45.

    Article  Google Scholar 

  32. Nuñez Velasco JP, Farah H, van Arem B, Hagenzieker MP. Studying pedestrians’ crossing behavior when interacting with automated vehicles using virtual reality. Transp Res Part F Traffic Psychol Behav. 2019;66:1–14.

    Article  Google Scholar 

  33. Mokas I, Lizin S, Brijs T, Witters N, Malina R. Can immersive virtual reality increase respondents’ certainty in discrete choice experiments? A comparison with traditional presentation formats. J Environ Econ Manage. 2021;109: 102509.

    Article  Google Scholar 

  34. Arellana J, Garzón L, Estrada J, Cantillo V. On the use of virtual immersive reality for discrete choice experiments to modelling pedestrian behaviour. J Choice Model. 2020;37:100251.

    Article  Google Scholar 

  35. Rigby D, Vass CM, Payne K. Opening the ‘black box’: an overview of methods to investigate the decision-making process in choice-based surveys. Patient. 2020;13:31–41.

    Article  PubMed  Google Scholar 

  36. Vass CM, Boeri M. Mobilising the next generation of stated-preference studies: the association of access device with choice behaviour and data quality. Patient. 2021;14:55–63.

    Article  PubMed  Google Scholar 

  37. Liebe U, Glenk K, Oehlmann M, Meyerhoff J. Does the use of mobile devices (tablets and smartphones) affect survey quality and choice behaviour in web surveys? J Choice Model. 2015;14:17–31.

    Article  Google Scholar 

  38. Arias López MDP, Ong BA, Borrat Frigola X, Fernández AL, Hicklent RS, Obeles AJT, et al. Digital literacy as a new determinant of health: a scoping review. PLOS Digit Heal. 2023;2:e0000279.

    Article  Google Scholar 

  39. Shields GE, Brown L, Wells A, Capobianco L, Vass C. Utilising patient and public involvement in stated preference research in health: learning from the existing literature and a case study. Patient. 2021;14:399–412.

    Article  PubMed  Google Scholar 

  40. Marta-Pedroso C, Freitas H, Domingos T. Testing for the survey mode effect on contingent valuation data quality: a case study of web based versus in-person interviews. Ecol Econ. 2007;62:388–98.

    Article  Google Scholar 

  41. Nielsen JS. Use of the internet for willingness-to-pay surveys: a comparison of face-to-face and web-based interviews. Resour Energy Econ. 2011;33:119–29.

    Article  Google Scholar 

  42. Grewenig E, Lergetporer P, Simon L, Werner K, Woessmann L. Can internet surveys represent the entire population? A practitioners’ analysis. Eur J Polit Econ. 2023;78:102382.

    Article  Google Scholar 

  43. Miller CA, Guidry JPD, Dahman B, Thomson MD. A tale of two diverse qualtrics samples: information for online survey researchers. Cancer Epidemiol Biomarkers Prev. 2020;29:731–5.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Shariq S, Cardoso Pinto AM, Budhathoki SS, Miller M, Cro S. Barriers and facilitators to the recruitment of disabled people to clinical trials: a scoping review. Trials. 2024;24:171.

    Article  Google Scholar 

  45. Marshall D, Mansfield C, von Butler L, MacDonald K. Preventing, detecting, and analyzing data from suspected fraudulent respondents in online surveys, with examples from health preference studies. ISPOR. 2023. https://www.ispor.org/conferences-education/event/2023/02/14/default-calendar/how-to-handle-fraudulent-responses-in-health-preference-studies. Accessed 11 Apr 2024.

  46. Gonzalez JM, Grover K, Leblanc TW, Reeve BB. Did a bot eat your homework? An assessment of the potential impact of bad actors in online administration of preference surveys. PLoS ONE. 2023;18: e0287766.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Wang J, Calderon G, Hager ER, Edwards LV, Berry AA, Liu Y, et al. Identifying and preventing fraudulent responses in online public health surveys: lessons learned during the COVID-19 pandemic. PLOS Glob Public Health. 2023;3: e0001452.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Veldwijk J, Marceta SM, Swait JD, Lipman SA, de Bekker-Grob EW. Taking the shortcut: simplifying heuristics in discrete choice experiments. Patient. 2023;16:301–15.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Ryan M, Watson V, Entwistle V. Rationalising the “irrational”: a think aloud study of discrete choice experiment responses. Health Econ. 2009;18:321–36.

    Article  PubMed  Google Scholar 

  50. Vass C, Rigby D, Payne K. “I was trying to do the maths”: exploring the impact of risk communication in discrete choice experiments. Patient. 2019;12:113–23.

    Article  PubMed  Google Scholar 

  51. Cooke L, Cuddihy E. Using eye tracking to address limitations in think-aloud protocol. IEEE Int Prof Commun Conf. 2005;653–8.

  52. Ryan M, Krucien N, Hermens F. The eyes have it: Using eye tracking to inform information processing strategies in multi-attributes choices. Health Econ (UK). 2018;27:709–21.

    Article  Google Scholar 

  53. Krucien N, Ryan M, Hermens F. Visual attention in multi-attributes choices: what can eye-tracking tell us? J Econ Behav Organ. 2017;135:251–67.

    Article  Google Scholar 

  54. Genie MG, Ryan M, Krucien N. Keeping an eye on cost: what can eye tracking tell us about attention to cost information in discrete choice experiments? Health Econ (UK). 2023;32:1101–19.

    Article  Google Scholar 

  55. Vass C, Rigby D, Tate K, Stewart A, Payne K. An exploratory application of eye-tracking methods in a discrete choice experiment. Med Decis Mak. 2018;38:658–72.

    Article  Google Scholar 

  56. Khushaba RN, Greenacre L, Kodagoda S, Louviere J, Burke S, Dissanayake G. Choice modeling and the brain: a study on the electroencephalogram (EEG) of preferences. Expert Syst Appl. 2012;39:12378–88.

    Article  Google Scholar 

  57. Montague R, Harvey A. Using fMRI to study valuation and choice. Adv Brain Neuroimaging Top Health Dis Methods Appl. 2014. https://doi.org/10.5772/58257.

    Article  Google Scholar 

  58. Vass C, Rigby D, Payne K. Watching ME watching you: understanding the effect of experimental setting on stated preferences. Value Health. 2020;23:S476–7.

    Article  Google Scholar 

  59. Ali A, Kalatian A, Choudhury CF. Comparing and contrasting choice model and machine learning techniques in the context of vehicle ownership decisions. Transp Res Part A Policy Pract. 2023;173:103727.

    Article  Google Scholar 

  60. Hensher DA, Ton TT. A comparison of the predictive potential of artificial neural networks and nested logit models for commuter mode choice. Transp Res Part E Logist Transp Rev. 2000;36:155–72.

    Article  Google Scholar 

  61. van Cranenburgh S, Wang S, Vij A, Pereira F, Walker J. Choice modelling in the age of machine learning: discussion paper. J Choice Model. 2022;42:100340.

    Article  Google Scholar 

  62. Hillel T, Bierlaire M, Elshafie MZEB, Jin Y. A systematic review of machine learning classification methodologies for modelling passenger mode choice. J Choice Model. 2021;38:100221. https://doi.org/10.17863/CAM.52743.

    Article  Google Scholar 

  63. Snaman JM, Helton G, Holder RL, Wittenberg E, Revette A, Tulsky JA, et al. MyPref: pilot study of a novel communication and decision-making tool for adolescents and young adults with advanced cancer. Support Care Cancer. 2021;29:2983–92.

    Article  PubMed  Google Scholar 

  64. Cole A, Khasawneh A, Adapa K, Mazur L, Richardson DR. Development of an electronic healthcare tool to elicit patient preferences in older adults diagnosed with hematologic malignancies. Lect Notes Comput Sci. 2022;210–28.

  65. Witteman HO, Ndjaboue R, Vaisson G, Dansokho SC, Arnold B, Bridges JFP, et al. Clarifying values: an updated and expanded systematic review and meta-analysis. Med Decis Making. 2021;41:801–20.

    Article  PubMed  PubMed Central  Google Scholar 

  66. Munro S, Stacey D, Lewis KB, Bansback N. Choosing treatment and screening options congruent with values: do decision aids help? Sub-analysis of a systematic review. Patient Educ Couns. 2016;99:491–500.

    Article  PubMed  Google Scholar 

  67. Bansback N, Marra C, Cibere J. Improving decision-making about medications in individuals with suspected knee osteoarthritis using a web application. J Rheumatol. 2015;42:1327.

    Google Scholar 

  68. Hauber B, Coulter J. Using the threshold technique to elicit patient preferences: an introduction to the method and an overview of existing empirical applications. Appl Health Econ Health Policy. 2020;18:31–46.

    Article  PubMed  Google Scholar 

  69. Heidenreich S, Trapali M, Krucien N, Tervonen T, Phillips-Beyer A. Two methods, one story? Comparing results of a choice experiment and multidimensional thresholding from a clinician preference study in aneurysmal subarachnoid hemorrhage. Value Health. 2024;27:61–9.

    Article  PubMed  Google Scholar 

  70. Schneider P, Brazier J, Devlin N, van Hout B. The EQ-5D-5L OPUF survey: quantifying health priorities on the individual person level using compositional preference elicitation techniques. 2022;7:A32.1–A32.2. https://doi.org/10.1136/bmjgh-2022-ISPH.88.

    Article  Google Scholar 

  71. Schneider PP, van Hout B, Heisen M, Brazier J, Devlin N. The Online Elicitation of Personal Utility Functions (OPUF) tool: a new method for valuing health states. Wellcome Open Res. 2022;7:14.

    Article  PubMed  PubMed Central  Google Scholar 

  72. Wright SJ, Vass CM, Ulph F, Payne K. Understanding the impact of different modes of information provision on preferences for a newborn bloodspot screening program in the United Kingdom. MDM Policy Pract. 2024;9:23814683241232936.

    Article  PubMed  PubMed Central  Google Scholar 

  73. Boye K, Ross M, Mody R, Konig M, Gelhorn H. Patients’ preferences for once-daily oral versus once-weekly injectable diabetes medications: the REVISE study. Diabetes Obes Metab. 2021;23:508–19.

    Article  CAS  PubMed  Google Scholar 

  74. Gelhorn H, Ross MM, Kansal AR, Fung ET, Seiden MV, Krucien N, et al. Patient Preferences for Multi-Cancer Early Detection (MCED) screening tests. Patient. 2023;16:43–56.

    Article  PubMed  Google Scholar 

  75. Oliveri S, Lanzoni L, Veldwijk J, de Wit GA, Petrocchi S, Janssens R, et al. Balancing benefits and risks in lung cancer therapies: patient preferences for lung cancer treatment alternatives. Front Psychol. 2023;14:1062830.

    Article  PubMed  PubMed Central  Google Scholar 

  76. Seo J, Tervonen T, Ueda K, Zhang D, Danno D, Tockhorn-Heidenreich A. Discrete choice experiment to understand Japanese patients’ and physicians’ preferences for preventive treatments for migraine. Neurol Ther. 2023;12:651–68.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors thank the audience of the ISPOR webinar on this topic for their feedback and questions. They also thank Ludwig von Butler for his helpful contributions during the webinar and for the helpful feedback on early ideas and drafts.

Author information

Authors and Affiliations

  1. RTI Health Solutions, Manchester, UK

    Caroline Vass

  2. Open Health, Belfast, UK

    Marco Boeri

  3. University of Manchester, Manchester, UK

    Gemma Shields

  4. Evidera, Bethesda, MD, USA

    Jaein Seo

  5. Queen’s University of Belfast, Belfast, UK

    Marco Boeri

Authors
  1. Caroline Vass
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marco Boeri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gemma Shields
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jaein Seo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Caroline Vass.

Ethics declarations

Funding

This research received no specific funding.

Conflicts of interest/competing interests

Caroline Vass, Marco Boeri, Gemma Shields and Jaein Seo have no conflicts of interest that are directly relevant to the content of this article.

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Availability of data and material

Not applicable.

Code availability

Not applicable.

Authors’ contributions

All authors contributed to the conception and design of the paper. The first draft of the manuscript was written by all authors and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vass, C., Boeri, M., Shields, G. et al. Making Use of Technology to Improve Stated Preference Studies. Patient (2024). https://doi.org/10.1007/s40271-024-00693-8

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s40271-024-00693-8

Search

Navigation