Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Using Best-Worst Scaling Choice Experiments to Measure Public Perceptions and Preferences for Healthcare Reform in Australia


Background: One of the greatest difficulties in evaluating healthcare system reform in any country is that governments often do not clearly articulate what it is they are attempting to do. In Australia, a recent inquiry set out 15 principles to guide the reform process, but it remains unclear how the Australian public values the principles, how such values vary across the country, and, more fundamentally, if Australians understand the principles.

Objectives: To evaluate the Australian healthcare reform principles from the perspective of the Australian public, to test if such preferences are valued consistently across geographic and socioeconomic strata, and to test for the degree of understanding of the principles among the public.

Methods: We employed best-worst scaling (BWS), a stated-preference method grounded in random utility theory, to elicit public preference for 15 healthcare reform principles. The BWS tasks were incorporated into an online survey that also gathered geographic and socioeconomic information and included questions relating to the understanding of the reform principles. Respondents were a geographically diverse set of Australians who were randomized to receive one of two versions of the survey, each containing a block of 15 choice tasks. Tasks in block one contained a subset of the choice tasks containing subsets of seven principles based on a balanced incomplete block design, while tasks in block two contained tasks with eight principles defined by the complement of the former.

In each BWS task, respondents were simply asked to identify the most and least important principle. Analysis of preference was based on assigning the most valued principles a ‘1’ and the least valued principles ‘−1’, and with each item appearing eight times in each block, preferences were analyzed over a cardinal utility scale bounded by −8 and +8. Analysis was based on simple summary statistics and stratified by geographic and socioeconomic measures.

Results: A sample of 204 respondents participated in the survey (a participation rate of 85%). Quality and safety was the most important principle and a culture of reflective improvement and innovation was the least important. Public voice and community engagement was the second least important principle and was also understood by barely half the respondents.

Conclusions: This research demonstrates how random-utility-based methods can be used to provide estimates of the importance of reform principles that have known statistical properties. The BWS task used forced respondents to discriminate between the principles on offer, unlike rating scales. Researchers and practitioners in healthcare should consider using BWS tasks in preference to rating scales.

This is a preview of subscription content, log in to check access.

Table I
Table II
Table III
Table IV
Fig. 1
Fig. 2
Table V


  1. 1.

    Lee JA, Soutar G, Louviere JJ. Measuring values using best-worst scaling: the LOV example. Psychol Mark 2007; 24(12): 1043–58

  2. 2.

    Ben-Akiva M, Morikawa T, Shiroishi F. Analysis of the reliability of preference ranking data. J Bus Res 1991; 23: 253–68

  3. 3.

    Finn A, Louviere JJ. Determining the appropriate response to evidence of public concern: the case of food safety. J Public Pol Mark 1992; 11(1): 12–25

  4. 4.

    Marley AAJ, Louviere JJ. Some probabilistic models of best, worst, and best-worst choices. J Math Psychol 2005; 49:464–80

  5. 5.

    Flynn TN. Valuing citizen and patient preferences in health: recent developments in three types of best-worst scaling. Exp Rev Pharmacoeconomics Outcomes Res 2010; 10(3): 259–67

  6. 6.

    Flynn TN, Louviere JJ, Peters TJ, et al. Best-worst scaling: what it can do for health care research and how to do it. J Health Econ 2007; 26(1): 171–89

  7. 7.

    Marley AAJ, Flynn TN, Louviere JJ. Probabilistic models of set-dependent and attribute-level best-worst choice. J Math Psychol 2008; 52: 281–96

  8. 8.

    National Health and Hospitals Reform Commission. A healthier future for all Australians: final report June 2009. Barton (ACT): Commonwealth of Australia, 2009 [online]. Available from URL: nhhrc/publishing.nsf/Content/nhhrc-report [Accessed 2010 Aug 25]

  9. 9.

    Thurstone LL. A law of comparative judgment. Psychol Rev 1927; 34: 273–86

  10. 10.

    Flynn TN, Louviere JJ, Peters TJ, et al. Estimating preferences for a dermatology consultation using Best-Worst Scaling: comparison of various methods of analysis. BMC Med Res Methodol 2008; 8: 76

  11. 11.

    Louviere JJ, Street DJ, Burgess L, et al. Modelling the choices of single individuals by combining efficient choice experiment designs with extra preference information. J Choice Model 2008; 1(1): 128–63

  12. 12.

    pureprofile [online]. Available from URL: http://www.pure [Accessed 2010 Aug 30]

  13. 13.

    Louviere JJ, Woodworth G. Design and analysis of simulated consumer choice or allocation experiments: an approach based on aggregate data. J Mark Res 1983; 20: 350–67

  14. 14.

    Baumgartner H, Steenkamp J-BEM. Response styles in marketing research: a cross-national investigation. J Mark Res 2001; 38(2): 143–56

  15. 15.

    Steenkamp J-BEM, Baumgartner H. Assessing measurement invariance in cross-national consumer research. J Consum Res 1998; 25(1): 78–90

  16. 16.

    Flynn TN, Louviere JJ, Peters TJ, et al. Using discrete choice experiments to understand preferences for quality of life: variance scale heterogeneity matters. Soc Sci Med 2010; 70(12): 1957–65

  17. 17.

    Fiebig DG, Keane MP, Louviere J, et al. The generalized multinomial logit model: accounting for scale and coefficient heterogeneity. Marketing Sci. Epub 2009 Jul 23 [online]. Available from URL: content/abstract/mksc.1090.0508v1 [Accessed 2010 Apr 21]

  18. 18.

    Lancsar E, Louviere JJ. Conducting discrete choice experiments to inform healthcare decision making: a user’s guide. Pharmacoeconomics 2008; 26(8): 661–77

  19. 19.

    Leatherman S, Sutherland K. The quest for quality in the NHS: a chartbook on quality of care in the UK. Oxford: Radcliffe Publishing Ltd, 2005

  20. 20.

    Mooney G. What else do we want from our health services? Soc Sci Med 1994; 39(2): 151–4

  21. 21.

    Dolan P, Cookson R, Ferguson B. Effect of discussion and deliberation on the public’s views of priority setting in health care: focus groups study. BMJ 1999; 318: 916–9

  22. 22.

    Mason H, Jones-Lee M, Donaldson C. Modelling the monetary value of a QALY: a new approach based on UK data. Health Econ 2009; 18: 933–50

  23. 23.

    Akkazieva B, Gulacsi L, Brandtmuller A, et al. Patients’ preferences for healthcare system reforms in Hungary: a conjoint analysis. Appl Health Econ Health Policy 2006; 5(3): 189–98

Download references


The authors thank participants in the study and Edward Wei for programming the survey.

No sources of funding were used to conduct this study or prepare this manuscript. The authors have no conflicts of interest that are directly relevant to the content of this article.

Author information

Correspondence to Professor Jordan J. Louviere.

Electronic supplementary material

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Louviere, J.J., Flynn, T.N. Using Best-Worst Scaling Choice Experiments to Measure Public Perceptions and Preferences for Healthcare Reform in Australia. Patient-Patient-Centered-Outcome-Res 3, 275–283 (2010).

Download citation


  • Community Engagement
  • Healthcare Reform
  • Balance Incomplete Block Design
  • Supplemental Digital Content
  • Priority Score