The Patient: Patient-Centered Outcomes Research

, Volume 3, Issue 4, pp 275–283 | Cite as

Using Best-Worst Scaling Choice Experiments to Measure Public Perceptions and Preferences for Healthcare Reform in Australia

  • Jordan J. LouviereEmail author
  • Terry N. Flynn
Original Research Article


Background: One of the greatest difficulties in evaluating healthcare system reform in any country is that governments often do not clearly articulate what it is they are attempting to do. In Australia, a recent inquiry set out 15 principles to guide the reform process, but it remains unclear how the Australian public values the principles, how such values vary across the country, and, more fundamentally, if Australians understand the principles.

Objectives: To evaluate the Australian healthcare reform principles from the perspective of the Australian public, to test if such preferences are valued consistently across geographic and socioeconomic strata, and to test for the degree of understanding of the principles among the public.

Methods: We employed best-worst scaling (BWS), a stated-preference method grounded in random utility theory, to elicit public preference for 15 healthcare reform principles. The BWS tasks were incorporated into an online survey that also gathered geographic and socioeconomic information and included questions relating to the understanding of the reform principles. Respondents were a geographically diverse set of Australians who were randomized to receive one of two versions of the survey, each containing a block of 15 choice tasks. Tasks in block one contained a subset of the choice tasks containing subsets of seven principles based on a balanced incomplete block design, while tasks in block two contained tasks with eight principles defined by the complement of the former.

In each BWS task, respondents were simply asked to identify the most and least important principle. Analysis of preference was based on assigning the most valued principles a ‘1’ and the least valued principles ‘−1’, and with each item appearing eight times in each block, preferences were analyzed over a cardinal utility scale bounded by −8 and +8. Analysis was based on simple summary statistics and stratified by geographic and socioeconomic measures.

Results: A sample of 204 respondents participated in the survey (a participation rate of 85%). Quality and safety was the most important principle and a culture of reflective improvement and innovation was the least important. Public voice and community engagement was the second least important principle and was also understood by barely half the respondents.

Conclusions: This research demonstrates how random-utility-based methods can be used to provide estimates of the importance of reform principles that have known statistical properties. The BWS task used forced respondents to discriminate between the principles on offer, unlike rating scales. Researchers and practitioners in healthcare should consider using BWS tasks in preference to rating scales.


Community Engagement Healthcare Reform Balance Incomplete Block Design Supplemental Digital Content Priority Score 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The authors thank participants in the study and Edward Wei for programming the survey.

No sources of funding were used to conduct this study or prepare this manuscript. The authors have no conflicts of interest that are directly relevant to the content of this article.

Supplementary material

40271_2012_3040275_MOESM1_ESM.pdf (231 kb)
Supplementary material, approximately 236 KB.


  1. 1.
    Lee JA, Soutar G, Louviere JJ. Measuring values using best-worst scaling: the LOV example. Psychol Mark 2007; 24(12): 1043–58CrossRefGoogle Scholar
  2. 2.
    Ben-Akiva M, Morikawa T, Shiroishi F. Analysis of the reliability of preference ranking data. J Bus Res 1991; 23: 253–68CrossRefGoogle Scholar
  3. 3.
    Finn A, Louviere JJ. Determining the appropriate response to evidence of public concern: the case of food safety. J Public Pol Mark 1992; 11(1): 12–25Google Scholar
  4. 4.
    Marley AAJ, Louviere JJ. Some probabilistic models of best, worst, and best-worst choices. J Math Psychol 2005; 49:464–80CrossRefGoogle Scholar
  5. 5.
    Flynn TN. Valuing citizen and patient preferences in health: recent developments in three types of best-worst scaling. Exp Rev Pharmacoeconomics Outcomes Res 2010; 10(3): 259–67CrossRefGoogle Scholar
  6. 6.
    Flynn TN, Louviere JJ, Peters TJ, et al. Best-worst scaling: what it can do for health care research and how to do it. J Health Econ 2007; 26(1): 171–89PubMedCrossRefGoogle Scholar
  7. 7.
    Marley AAJ, Flynn TN, Louviere JJ. Probabilistic models of set-dependent and attribute-level best-worst choice. J Math Psychol 2008; 52: 281–96CrossRefGoogle Scholar
  8. 8.
    National Health and Hospitals Reform Commission. A healthier future for all Australians: final report June 2009. Barton (ACT): Commonwealth of Australia, 2009 [online]. Available from URL: nhhrc/publishing.nsf/Content/nhhrc-report [Accessed 2010 Aug 25]Google Scholar
  9. 9.
    Thurstone LL. A law of comparative judgment. Psychol Rev 1927; 34: 273–86CrossRefGoogle Scholar
  10. 10.
    Flynn TN, Louviere JJ, Peters TJ, et al. Estimating preferences for a dermatology consultation using Best-Worst Scaling: comparison of various methods of analysis. BMC Med Res Methodol 2008; 8: 76PubMedCrossRefGoogle Scholar
  11. 11.
    Louviere JJ, Street DJ, Burgess L, et al. Modelling the choices of single individuals by combining efficient choice experiment designs with extra preference information. J Choice Model 2008; 1(1): 128–63Google Scholar
  12. 12.
    pureprofile [online]. Available from URL: http://www.pure [Accessed 2010 Aug 30]Google Scholar
  13. 13.
    Louviere JJ, Woodworth G. Design and analysis of simulated consumer choice or allocation experiments: an approach based on aggregate data. J Mark Res 1983; 20: 350–67CrossRefGoogle Scholar
  14. 14.
    Baumgartner H, Steenkamp J-BEM. Response styles in marketing research: a cross-national investigation. J Mark Res 2001; 38(2): 143–56CrossRefGoogle Scholar
  15. 15.
    Steenkamp J-BEM, Baumgartner H. Assessing measurement invariance in cross-national consumer research. J Consum Res 1998; 25(1): 78–90CrossRefGoogle Scholar
  16. 16.
    Flynn TN, Louviere JJ, Peters TJ, et al. Using discrete choice experiments to understand preferences for quality of life: variance scale heterogeneity matters. Soc Sci Med 2010; 70(12): 1957–65PubMedCrossRefGoogle Scholar
  17. 17.
    Fiebig DG, Keane MP, Louviere J, et al. The generalized multinomial logit model: accounting for scale and coefficient heterogeneity. Marketing Sci. Epub 2009 Jul 23 [online]. Available from URL: content/abstract/mksc.1090.0508v1 [Accessed 2010 Apr 21]Google Scholar
  18. 18.
    Lancsar E, Louviere JJ. Conducting discrete choice experiments to inform healthcare decision making: a user’s guide. Pharmacoeconomics 2008; 26(8): 661–77PubMedCrossRefGoogle Scholar
  19. 19.
    Leatherman S, Sutherland K. The quest for quality in the NHS: a chartbook on quality of care in the UK. Oxford: Radcliffe Publishing Ltd, 2005Google Scholar
  20. 20.
    Mooney G. What else do we want from our health services? Soc Sci Med 1994; 39(2): 151–4PubMedCrossRefGoogle Scholar
  21. 21.
    Dolan P, Cookson R, Ferguson B. Effect of discussion and deliberation on the public’s views of priority setting in health care: focus groups study. BMJ 1999; 318: 916–9PubMedCrossRefGoogle Scholar
  22. 22.
    Mason H, Jones-Lee M, Donaldson C. Modelling the monetary value of a QALY: a new approach based on UK data. Health Econ 2009; 18: 933–50PubMedCrossRefGoogle Scholar
  23. 23.
    Akkazieva B, Gulacsi L, Brandtmuller A, et al. Patients’ preferences for healthcare system reforms in Hungary: a conjoint analysis. Appl Health Econ Health Policy 2006; 5(3): 189–98PubMedCrossRefGoogle Scholar

Copyright information

© Adis Data Information BV 2010

Authors and Affiliations

  1. 1.Centre for the Study of Choice (CenSoC)University of Technology SydneySydneyAustralia

Personalised recommendations