Skip to main content

Development of a Discrete-Choice Experiment (DCE) to Elicit Adolescent and Parent Preferences for Hypodontia Treatment



Our objective was to develop and test a discrete-choice experiment (DCE) survey to elicit adolescent and parent preferences for dental care for hypodontia (a developmental condition where one or more teeth fail to develop).


This was a mixed-methods study. Participants were adolescents (aged 12–16 years) with hypodontia and their parents and the dentists providing hypodontia care. Stage one entailed attribute development, as follows. (1) Attribute identification: systematic review of hypodontia literature; interviews with adolescents with hypodontia (n = 8) and parents (n = 8); observation of hypodontia clinical consultations (n = 5); environmental scan of hypodontia patient information resources (n = 30); and systematic analysis of social media posts (n = 176). (2) Attribute selection: stakeholder consultation to develop items for a questionnaire; rating and ranking questionnaire for adolescents with hypodontia and parents (n = 18); further stakeholder consultation. Stage two involved the development of the DCE survey, and stage three included the pre-testing using cognitive interviews with adolescents (n = 12) and parents (n = 8) to assess face and content validity.


The attribute long list included 27 attributes focusing on service delivery and treatment outcome, from which seven ‘important’ attributes were selected for pre-testing. Cognitive interviewing suggested adolescents found the DCE choice tasks challenging to understand; the survey was modified to enhance its acceptability. One attribute was excluded as it showed poor validity with adolescents. Pre-testing suggested DCE choice tasks encouraged thinking and discussion about preferences for treatment.


Including the target respondent group in all stages of DCE development ensured the final DCE survey was valid and acceptable. DCE methods appear to be a useful tool for exploring joint decision making alongside conventional preference elicitation.

FormalPara Key Points for Decision Makers
Including the target population at all stages of the discrete-choice experiment (DCE) development optimised the identification and selection of valid and relevant attributes.
The mixed-methods approach to pre-testing provided rich data to guide modifications to the DCE to improve its validity and suitability for use with adolescents.
The proposed use of the DCE survey to observe joint adolescent–parent preference elicitation and decision-making processes is supported by evidence gained from this study.


Hypodontia is the developmental absence of one or more teeth, excluding third molars. It affects 3.5–6.5% of the population in the permanent dentition and is a life-long condition [1]. Diagnosis usually occurs in adolescence, necessitating treatment planning between young patients, their parents and the dental team. Often, more than one treatment option is available, each with differing outcomes and a range of long-term impacts [2]. Understanding adolescent preferences is fundamental to supporting effective decision making and provision of evidence-based care [3]. To date, no studies have explored adolescent and parent preferences for dental treatment for hypodontia or their joint decision-making processes.

Discrete-choice experiments (DCEs) enable preferences to be estimated for complex goods/services using constructed hypothetical scenarios; this enables manipulation and control over the choice situation [4]. This paper reports the studies involved in the development of a DCE survey to elicit adolescent, parent and joint adolescent–parent preferences for hypodontia care. DCE methods have been widely used in healthcare [5,6,7], but their application in dentistry is more limited [8]. Preference elicitation using DCEs with adolescents is an emerging area; however, to date, only six studies in healthcare have included respondents aged < 18 years [9,10,11,12,13,14]. Previous DCE studies have measured joint preferences, but these have been advocate preferences, for example, a parent selecting preferences for their child [15,16,17], health professionals choosing for their patients [18] or a comparison of preferences between independent respondent groups [19, 20]. Only one healthcare study used DCEs to measure preferences of related respondents [21], and this was a pilot study with only nine parent–child dyads. Outside healthcare, one study reported using DCE methods to compare the individual and joint preferences of 45 couples for beaches; however, no qualitative information was obtained to understand the negotiation process [22].

This research aims to develop a DCE survey to elicit preferences for hypodontia care from adolescents and adolescent–parent dyads. The DCE survey was tested to establish its validity for use with adolescents and the feasibility of using the DCE as a tool to explore joint decision-making processes.


A number of multi-attribute stated-preference-elicitation methods exist, but little robust evidence is available to determine which is most appropriate. DCE was chosen because the underlying task—in this case choosing between alternative treatments—more accurately reflects the real-world decision context than other methods, such as best–worst scaling, where the best and worst attributes are selected. Furthermore, this study provided a valuable opportunity to examine the suitability of DCEs for eliciting joint and adolescent preferences and to explore whether DCEs have scope to encourage adolescent and parent engagement in decision making.

DCE development was undertaken in sequential steps (Fig. 1) following best practice guidance [23,24,25]. Particular attention was given to the intended use of the survey for adolescent and joint adolescent–parent preference elicitation. Qualitative methods were incorporated at all stages of the DCE development to ensure adolescent and parent perspectives were represented [26,27,28,29]. The methods and approach for analysis for each stage are summarised in the following sections.

Fig. 1

Steps in the development of the discrete-choice experiment (DCE) survey

Compliance with Ethical Standards

Ethical approval was granted by the North West National Health Service (NHS) Research Ethics Committee (15/NW/0804), and permission was granted by each participating site. Additional ethical approval was obtained from the University of Leeds Dental Research Ethics Committee for the attribute selection survey involving members of the public recruited via Facebook (DREC 040917/SB/235 on 9.11.17). The study was adopted by the National Institute for Health Research (NIHR) Clinical Research Network and conducted in accordance with the ethical principles outlined in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.

Stage One: Attribute Development

Attribute development included two distinct steps:

  1. 1.

    Attribute identification.

  2. 2.

    Attribute selection and framing.

Attribute Identification

The attributes of hypodontia treatment were identified using mixed methods involving important stakeholder groups, including dentists and service providers in primary and secondary care and adolescents with hypodontia and their parents.

Dentist perspective was captured through a systematic review of outcomes used in primary empirical studies to assess the effectiveness and impact of hypodontia care [30]. In total, 56 papers were identified that provided evidence about the outcomes judged to be important by researchers and service providers. Alongside this, the information provided to adolescents and parents by dentists during the decision-making process was analysed using two methods. First, written (n = 18) and online (n = 12) patient information for hypodontia and its treatment was analysed [31]. Second, five inter-disciplinary clinical consultations between adolescents, their parents and the dental team were observed by one researcher (SB), and the content of the discussion was analysed. For the clinical observations, participants were purposively sampled from two hospital clinics to provide diversity in the sample based on adolescent demographics (age, sex, severity of hypodontia, stage of treatment) and the dental team (type of dental specialist, grade and experience). Consultations were audio-recorded without interference from the researcher and transcribed for analysis using the framework method [32, 33].

To capture adolescent and parent perspectives, semi-structured interviews with young people aged 12–16 years with hypodontia (n = 8) and their parents (n = 8) were undertaken. Adolescents were identified from treatment clinics in two hospitals using a purposive sampling matrix based on age, sex, severity of hypodontia and stage of treatment. A topic guide was developed and iteratively developed for use in the interviews. Interviews were undertaken with the adolescent first, then the parent was invited to join to add their perspective. Interviews were audio-recorded, transcribed and analysed using the framework method. To provide additional information from the perspective of people affected by hypodontia, a systematic search and evaluation of posts relating to hypodontia on social media was performed [34].

Information gained from the attribute-identification methods was synthesised by one researcher (SB). A preliminary framework for coding and categorising attributes was devised and revised iteratively by the research team. Attribute themes developed under two broad categories: attributes relating to service delivery (process) and attributes relating to treatment outcome (Table 1). Different approaches for describing attributes were recorded.

Table 1 List of all attributes of hypodontia care identified from the mixed methods in stage one with an indication of the source

Attribute Selection and Framing

Attribute selection followed an iterative process to encourage an evidence-based and patient-centred set of attributes; few previous studies have explicitly reported this process [6,7,8,9,10,11,12,13,14,15,16,17,18,19,20]. Mixed methods were used, including stakeholder consultation and a questionnaire to rate and rank attributes.

Stakeholder consultation involved adolescent–parent representatives, dental professionals involved in hypodontia care and members of the research team with expertise in stated-preference methods and decision making. Three pairs of adolescent–parent representatives were identified by one member of the research team (SB) during routine clinical activity and invited to participate in consultations as ‘experts’ rather than as research participants. The purpose of the consultation process was explained, and training, support and the opportunity to withdraw was provided. At their request, the adolescent–parent representatives were seen separately to the other groups to ensure the adolescents felt at ease and to encourage open discussion. The dental professionals’ group included a convenience sample of five specialists selected from the clinical teams working with one researcher (SB). Clinicians were consulted in groups or individually to fit with clinical commitments.

The first consultation process aimed to refine the raw data from attribute-identification methods into a long list of potential attributes. The attributes were presented to each group and discussed to explore their meaning, relevance and perceived application to hypodontia care. Notes were made throughout the consultations and used to revise and modify the attribute framework. Adolescent–parent consultations explored the perceived relationship within and across attribute concepts and assessed the face validity of attributes. Attributes that were duplicates or irrelevant to adolescents and parents were excluded to develop a long list of attributes.

The long list of attributes was used to design an online questionnaire for adolescents with hypodontia and parents to rate and rank the attributes; this included 16 attributes related to service delivery and 11 related to outcome (Table 1 in the Electronic Supplementary Material [ESM]). Respondents were asked to rate how important each attribute was on a scale of 1 (not important) to 5 (very important) and select the six most important attributes for service delivery and outcome. The questionnaire is available on request. The questionnaire was converted into an online survey tool and distributed to participants via a unique link. Participants were recruited from a Facebook support group for people with hypodontia, limited to people living in the UK.

Attribute scores from the questionnaire were used to guide further stakeholder consultation to select ‘important’ attributes for testing. Aggregate scores were discussed, and the most useful and relevant attribute for representing a concept was identified. Adolescents and their parents confirmed the acceptability and face validity of the attributes, informed the suitability of dimensions for the attribute levels and provided information about appropriate language and framing.

Stage Two: DCE Survey Development

DCE design and survey development followed best practice guidance [24, 25], including consideration of the 10-point checklist for conjoint experiment design provided by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) [23].

Experimental Design

A fractional factorial experimental design was selected because of the impractical number of options created by a full factorial design (35 × 22 = 972 choices). A multinomial logit model with main effects design was specified in Ngene v1.2 (ChoiceMetrics) with d-optimising efficiency, excluding dominant choice sets. The design was checked for level balance across choice tasks and plausibility. As the ideal number of choice tasks remains controversial [35], eight choice tasks were presented in each survey version in the pre-testing to assess how respondents coped. Blocking was used to increase the number of tasks that could be included; pre-testing included 32 choice tasks in total (four survey versions each with eight pair choices).

Choice Task Construction

Task construction aimed to reduce cognitive burden and optimise acceptability for the target respondent group. Adolescents’ ability to trade-off attributes is seldom reported, and evidence for the optimum number of tasks is weak. To minimise the complexity of the choice tasks, paired choice tasks were used.

An opt-out was desirable to reflect that dental treatment for hypodontia is elective; however, there was concern that an opt-out may lead to high levels of non-response where the trade-off was judged to be difficult [36]. A status quo treatment [23] was not possible because of variations in individual care plans. To manage these challenges, a ‘no treatment’ option was included for pre-testing, but this was offered after a preference was given for one of the treatment options.

Generic labels were used for the options (treatment A and treatment B) because they did not reflect exact treatment options and there was a possibility that labels might encourage detrimental heuristics [37]. Two qualifying questions were included with the choice task: strength of preference (How much do you like treatment X?) and perceieved difficulty of choice (How easy was it for you to pick?) [23]. These were measured using a 5-point scale anchored with ‘a lot’ to ‘not at all’ and ‘very easy’ to ‘very difficult’, respectively.

Survey Design

Survey components included an introduction to the survey and explanation of its purpose, questions to obtain demographic data, an explanation of the attribute and DCE task followed by the eight DCE choice tasks, and background questions identifying factors that may influence preference ratings. Background questions included questions about knowledge (based on the domains of the Illness Perception Questionnaire [38]), beliefs about dental treatment (developed from the Beliefs about Medicine Questionnaire [39] and the Beliefs about Surgery Questionnaire [40]), decision making (based on the SURE tool [41]), dental anxiety (5-point Likert scale) and the impact of hypodontia (using questions from a hypodontia-specific tool [42]).

A commercial research company experienced in collecting and managing stated-preference data created an online survey delivered through a secure platform. Screening questions and methods for obtaining consent were tested to ensure acceptability for use in a future remote online survey. Screening questions asked respondents to confirm they had hypodontia, were aged 12–16 years, lived in the UK and had parental consent to answer the survey. Adolescent understanding of the questions was checked throughout the pre-testing, and parents were explicitly asked whether they found the level of consent acceptable.

Stage Three: Testing the Discrete-Choice Experiment (DCE) Survey

Given the scant evidence about DCE design for adolescents and joint preference elicitation, rigorous pre-testing was undertaken using combined qualitative and quantitative methods. Pre-testing aimed to explore the acceptability and feasibility of proposed methods in a real-world sample.


Pre-testing was undertaken using two approaches:

  • Cross-sectional survey employing cognitive interview methods [43] to test the validity and acceptability of the DCE survey. Contemporaneous analysis of feedback facilitated iterative modification and re-testing.

  • Analysis of preference data to provide coefficient estimates for each attribute level. Estimates were then compared with qualitative feedback to estimate the validity and importance of attributes.

Sampling and Recruitment

A sample of 12 adolescents and eight parents were invited to participate, providing a total of 160 observations. Eligible participants were invited from two hospital orthodontic departments in Yorkshire. This included (1) adolescents aged 12–16 years with varying levels of severity of hypodontia and at any stage of treatment and (2) parents or guardians of adolescents with hypodontia. The only exclusions were people with hypodontia related to craniofacial syndromes with significant comorbidities that would impact on healthcare provision, and any participants unable to provide consent. Purposive sampling was used based on age, sex, severity of hypodontia and stage of treatment.

Data Collection and Analysis

Participants (14 individual participants and three adolescent–parent dyads) completed the DCE survey using a laptop in a private room. The researcher (SB) prompted participants to verbalise thoughts as they worked through the survey and at the end asked questions focusing on the relevance and importance of the attributes for making decisions about hypodontia treatment, the perceived complexity of the DCE task, time taken to complete the survey and clarity of instructions. All verbal responses were noted.

Choices in the DCE choice tasks were collected automatically by the online survey tool, and the numerical preference data were collated for analysis using a conditional logit model (Box 1 in the ESM). The small sample size meant estimates were interpreted with caution, and no subgroup analyses were undertaken.


Stage One: Attribute Development

Attribute identification methods resulted in the identification of five attribute concepts relating to service delivery and six attribute concepts relating to treatment outcome and impact (Table 1). Each concept included multiple attributes, and different descriptions were used within each attribute, for example, the attribute ‘bite’ was described as ability to bite, ability to chew, confidence to bite, ability to eat hard food, avoidance of certain foods. The source of each attribute is provided in Table 1; this demonstrates the value in using mixed methods to ensure comprehensive attribute identification. Many of the attributes identified in the literature review were clinical markers that allow objective measurement of dental health and treatment effect by dental professionals, but these were difficult for adolescents and parents to understand and interpret. Clinical consultations and patient information resources rarely included comprehensive information relating to patient experience and outcome measures.

Initial elimination of inappropriate attributes resulted in a long list of 27 attributes relating to treatment process (n = 16) and treatment outcome (n = 11) (Table 1 in the ESM). These were included in a rating and raking questionnaire, which was completed in October–November 2017 by 11 parents (two fathers, eight mothers and a guardian) and eight adolescents. The adolescent respondents showed a spread of age ranges and an equal sex balance but a disproportionately high number of people with severe hypodontia. The questionnaire showed differences in adolescent and parent scores (Fig. 1 in the ESM); however, general trends were identified, and these were used to discuss with stakeholders which attributes were most important to include. The selection process identified seven attributes representing different attribute concepts that were agreed to be important to both adolescents and parents. Stakeholder input was sought regarding appropriate ranges for the attribute levels based on best available evidence identified in the literature review and clinical experience. Adolescent and parent advice regarding language and framing of attribute levels to convey their intended meaning accurately to respondents was followed.

Stage Two: Survey Development

The attributes and attribute levels included in the DCE choice tasks for pre-testing are described in Table 2 in the ESM. The approach and rationale for survey development is described in Sect. 2.3. An example of the full survey is available on request.

Stage Three: Pre-Testing

Pre-testing explored the acceptability of the DCE survey method, the validity of the attributes, and the ability of the method to encourage respondents to consider what is important to them.

Acceptability of DCE Survey Method

The DCE survey method required modification to improve acceptability, as both adolescents and parents experienced difficulties understanding the purpose and method of the choice tasks. Adolescents in particular asked for ways to help process the task, such as highlighting the ‘better’ attribute in each scenario or a function to allow the preferred attribute in each scenario to be selected individually rather than having to make a choice between the whole profile. The ability to perform the DCE choice task appeared to be related to respondents’ ability to understand the concept of trading off attributes rather than the age of the respondent. Different approaches were trialled until respondents demonstrated understanding of the choice tasks and the concept of trading-off.

Validity of Attributes

The validity of the DCE method was assessed by comparing verbal feedback and preference estimates derived from the choices made in the DCE tasks. Statistical analysis of the preference data found most attribute coefficient signs behaved as expected based on the verbal feedback; that is, attribute levels that were perceived to be beneficial showed a positive sign, indicating a gain in utility, whereas those perceived as detrimental had a negative sign, indicating a loss in utility (Table 2 in the ESM). Compromised appearance caused the greatest reduction in utility, followed by severe problems during treatment and no improvement in bite, which reflected respondent feedback that these were highly unfavourable.

Importantly, the verbal feedback and preference estimates for ‘waiting time’ and ‘discomfort’ were not in agreement. The positive attribute coefficient associated with increased waiting time suggested adolescent and parent respondents gained utility from waiting, but this was contradicted by the verbal feedback, which suggested increased waiting time is not at all desirable. Similarly, for the parent group, there was a gain in utility for severe discomfort, but parents reported a preference for avoiding discomfort. It is likely that these results arose erroneously because of the small sample size; as a result, the attributes are to undergo further testing.

‘Cost of dental treatment in the future’ was shown to be problematic for adolescents in both the verbal feedback and the preference estimates. Adolescents reported little understanding or interest in cost, and there was little awareness that certain dental treatments will carry future costs. The attribute coefficients for adolescents suggested slightly more utility loss from a future cost of £50 per year than £250 a year, which suggests an irrational response to the cost attribute. On the other hand, parents’ response to cost was as expected, with the highest cost demonstrating the greatest reduction in utility.

The disparity between the qualitative feedback and preference estimates emphasise the importance of including detailed pre-testing to examine attribute validity. In this study, reliance on preference estimates only would have misinformed attribute selection.

Validity of DCE Tasks

Cognitive interviewing and the additional probing questions during pre-testing provided evidence that the DCE tasks encouraged respondents to consider what is important to them. Completion of the DCE choice tasks by adolescent–parent dyads promoted greater discussion around preferences, treatment choices and willingness to compromise. The attributes provided prompts and a shared language to help adolescents articulate their concerns and priorities more effectively. Adolescents reported that outcome was most important and that they would ‘put up’ with less favourable process attribute levels. Parents placed value on appearance after treatment and avoiding problems during treatment. Two parents of adolescents near the end of treatment or post-treatment commented that talking through the tasks helped them realise their child’s values had changed throughout the course of dental treatment.

Future Study

Modifications to the DCE Survey

The findings from the pre-testing, combined with further stakeholder input, have been used to revise the DCE survey to optimise its validity for use with the target respondents (Table 3 in the ESM). Contradictory results for ‘waiting time’ and ‘discomfort’ mean both will be included in the final DCE for further investigation. ‘Future cost of dental treatment’ was excluded based on the evidence from both methods of pre-testing, which indicated cost is not valid for the approach with adolescent respondents.

Estimates of attribute coefficients from analysis of the preliminary DCE preference data were used to improve the efficiency of the experimental design. To address the dominance of appearance, the design produced overlaps in the appearance attribute between profiles in 18 of the 28 choice tasks. The final survey contains five parts: introduction (including screening and consent); demographic questions; DCE choice task; background questions; and closure (including sources of support). The presentation of choice tasks was simplified as much as possible, and full descriptions of each attribute were made easily accessible through information buttons throughout the tasks (Fig. 2 in the ESM).

Planned Data Collection

Evidence from this study has been used to inform the future principal DCE survey methods (“Box 1”). This will involve both a face-to-face survey using cognitive interview methods to observe individual and joint decision making between adolescent–parent dyads alongside a traditional cross-sectional online survey method to elicit preferences from adolescents and parents (Table 2).

Table 2 Description of the pilot and final survey


This systematic research approach provided evidence to inform the design of a DCE that is acceptable to adolescents with hypodontia and their parents for preference elicitation. The methods and data that were key to informing the design of the DCE tasks and survey are reported, and we are confident that—by including the target population during research development [44,45,46] and using patient-centred research methods during attribute identification, selection and testing—we have ensured the attributes are relevant to the target audience. The cognitive interview method effectively tested respondents’ understanding of attributes, the number of tasks that could be managed and different approaches for explaining the attribute levels and task concept. This approach provided more information than would have been gained by a larger-scale online pilot, which would only have provided preference data, and addressed issues arising from the lack of guiding evidence for DCE use with adolescents. For example, cognitive interviewing emphasised that ‘waiting time’ and ‘discomfort’ had not behaved as expected and should be included for further testing and confirmed that ‘cost of dental treatment in the future’ was a problematic concept for adolescents and should not be included in the final DCE. It may be argued that cost was valid for parents and its inclusion would have been beneficial to assess its importance for parents; however, the copayment system for dental care in the UK presents challenges for valid interpretation of willingness-to-pay estimates and—although another DCE study has managed this by using DCE designs with and without a cost attribute [47]—this was infeasible within this study.

We acknowledged that the data presented were elicited from a small sample and, although this is appropriate for the development phase of a DCE, further data collection and analysis with a larger sample is required to describe preferences that are representative of adolescents and parents making decisions about hypodontia treatment. Furthermore, the validity of DCE methods for use with adolescent respondents requires careful evaluation; a number of complementary methods to investigate reliability and validity [48, 49] have been included in the future DCE survey (Table 4 in the ESM).

This study provides useful insight into the future work needed to address service and patient needs. Attribute identification demonstrated discrepancies in which aspects of care adolescents, parents and clinicians considered important. For example, outcomes identified in the literature review included clinical markers of treatment effect but few patient-centred outcomes, such as impact of treatment on everyday activities. Future research into treatment efficacy and delivery needs to ensure outcome measures are used that will contribute to an evidence base that is relevant to patients. Pre-testing highlighted that patients and families have inadequate understanding of certain treatment attributes and there is a clear need for more decision support for patients and families to help them identify their own preferences.

The authors’ proposed use of the DCE developed by this process as a tool for exploring decision making between family members is novel. With appropriate use of qualitative methods, this approach should provide insight into how people with hypodontia and their families make treatment trade-offs; this may support changes to the consultation process and clinical care delivery. Our findings suggest that age is not a reliable indicator of whether or not adolescents have the capacity to be involved in complex decision making about treatment. The ability of adolescents of different ages to conceptualise short- and long-term consequences and trade-off of their treatment varied. This highlights one of the challenges for dental staff when attempting to involve people with hypodontia and parents in clinical decision making.


Mixed methods were essential to identify and select attributes that are important to adolescents with hypodontia and their parents. Use of cognitive interviewing during pre-testing of the DCE survey tool enabled deeper exploration of adolescent and parent experience and understanding. Pre-testing highlighted that ‘future cost’ is not a valid attribute for adolescents and that DCE choice tasks can be complex for both adolescents and parents. The protocol for the principal DCE survey draws on the findings from the pre-testing stage: alongside a traditional cross-sectional online survey method to elicit adolescent and parent preferences, a face-to-face survey using cognitive interview methods is planned to observe joint decision making in adolescent–parent dyads and to further explore the validity of DCE methods.


  1. 1.

    Polder BJ, Van’t Hof MA, Van der Linden F, Kuijpers-Jagtman AM. A meta-analysis of the prevalence of dental agenesis of permanent teeth. Community Dent Oral Epidiol. 2004;32:217–26.

    Article  Google Scholar 

  2. 2.

    Gill DS, Barker CS. Counselling patients with hypodontia. Dent Update. 2008;35:344–6 (348–50, 352).

    Article  PubMed  Google Scholar 

  3. 3.

    Coulter A, Collins A. Making shared decision-making a reality. No decision about me, without me. London: The King’s Fund; 2011. ISBN 9781857176247.

    Google Scholar 

  4. 4.

    Mangham LJ, Hanson K, McPake B. How to do (or not to do) … Designing a discrete choice experiment for application in a low-income country. Health Policy Plan. 2009;24(2):151–8.

    Article  PubMed  Google Scholar 

  5. 5.

    Lanscar E, Louviere J. Conducting discrete choice experiments to inform healthcare decision making. Pharmacoeconomics. 2008;26(8):661–77.

    Article  Google Scholar 

  6. 6.

    de Bekker-Grob EW, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2012;21:145–72.

    Article  PubMed  Google Scholar 

  7. 7.

    Clark MD, Determann D, Petrou S, Moro D, de Bekker-Grob EW. Discrete choice experiments in health economics: a review of the literature. Pharmacoeconomics. 2014;32:883–902.

    Article  PubMed  Google Scholar 

  8. 8.

    Barber SK, Bekker HL, Khambay BS, Pavitt S, Meads D. Eliciting preferences in dentistry: a systematic review. JDR Clin Transl Res. 2018.

    Article  Google Scholar 

  9. 9.

    Brown DS, Poulos C, Johnson FR, Chamiec-Case L, Messonnier ML. Adolescent girls’ preferences for HPV vaccines: a discrete choice experiment. Adv Health Econ Health Serv Res. 2014;24:93–121.

    Article  PubMed  Google Scholar 

  10. 10.

    Bekker-Grob EW, Hofman R, Donkers B, et al. Girls’ preferences for HPV vaccination: a discrete choice experiment. Vaccine. 2010;28(41):6692–7.

    Article  PubMed  Google Scholar 

  11. 11.

    Hofman R, de Bekker-Grob EW, Richardus JH, de Koning HJ, van Ballegooijen M, Korfage IJ. Have preferences of girls changed almost 3 years after the much debated start of the HPV vaccination program in the Netherlands? A discrete choice experiment. Harper DM, ed. PLoS One. 2014;9(8):e104772.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. 12.

    Marshall HS, Chen G, Clarke M, Ratcliffe J. Adolescent, parent and societal preferences and willingness to pay for meningococcal B vaccine: a discrete choice experiment. Vaccine. 2016;34:671–7.

    Article  CAS  PubMed  Google Scholar 

  13. 13.

    Sung L, Alibhai SM, Marie-Chantal E, Teuffel O, Cheng S, Fisman D, Reiger DA. Discrete choice experiment produced estimates of acceptable risks of therapeutic options in cancer patients with febrile neutropenia. J Clin Epidemiol. 2012;65:627–34.

    Article  PubMed  Google Scholar 

  14. 14.

    Wang B, Chen G, Ratcliffe J, Afzali HHA, Giles L, Marshall H. Adolescent values for immunisation programs in Australia: a discrete choice experiment. PLoS One. 2017;12(7):e0181073.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. 15.

    Nafees B, Setyawan J, Lloyd A, Ali S, Hearn S, Sasane R, Sonuga-Barke E, Hodgkins P. Parent preferences regarding stimulant therapies for ADHD: a comparison across six European countries. Eur Child Adolesc Psychiatry. 2009;23(2):1189–200.

    Google Scholar 

  16. 16.

    Lloyd A, Hodgkins P, Dewilde S, Sesane R, Falconer S, Sonuga Barke E. Methylphenidate delivery mechanisms for the treatment of children with attention deficit hyperactivity disorder: Heterogeneity in parent preferences. Int J Technol Assess Health Care. 2011;27(3):215–23.

    Article  PubMed  Google Scholar 

  17. 17.

    Waschbusch DA, Cunningham CE, Pelham WE Jr, Rimas HL, Greiner AR, Gnagy EM, et al. A discrete choice conjoint experiment to evaluate parent preferences for treatment of young, medication naive children with ADHD. J Clin Child Adolesc Psychol. 2011;40(4):546–61.

    Article  PubMed  PubMed Central  Google Scholar 

  18. 18.

    Shah RD, Rasinski KA, Alexander GC. The influence of surrogate decision makers on clinical decision making for critically ill adults. J Intensive Care Med. 2015;30(5):278–85.

    Article  PubMed  Google Scholar 

  19. 19.

    Harrison M, Milbers K, Hudson M, et al. Do patients and health care providers have discordant preferences about which aspects of treatments matter most? Evidence from a systematic review of discrete choice experiments. BMJ Open. 2017;7:e014719.

    Article  PubMed  PubMed Central  Google Scholar 

  20. 20.

    Malhotra C, Farooqui MA, Kanesvaran R, Bilger M, Finkelstein E. Comparison of preferences for end-of-life care among patients with advanced cancer and their caregivers: a discrete choice experiment. Palliat Med. 2015;29(9):842–50.

    Article  PubMed  Google Scholar 

  21. 21.

    Bray N, Yeo ST, Noyes J, Harris N, Edwards RT. Prioritising wheelchair services for children: a pilot discrete choice experiment to understand how child wheelchair users and their parents prioritise different attributes of wheelchair services. Pilot Feasibility Stud. 2016;2:32.

    Article  PubMed  PubMed Central  Google Scholar 

  22. 22.

    Beharry-Borg N, Hensher DA, Scarpa R. An analytical framework for joint vs separate decisions by couples in choice experiments: the case of coastal water quality in Tobago. Environ Resour Econ. 2009;45:95–117.

    Article  Google Scholar 

  23. 23.

    Bridges JFP, Hauber AB, Marshall D, Lloyd A, Prosser LA, Reiger DA, Johnson FR, Mauskopf J. Conjoint analysis applications in health—a checklist: a report of the ISPOR good research practices for conjoint analysis task force. Value Health. 2011;14:403–13.

    Article  PubMed  Google Scholar 

  24. 24.

    Johnson, et al. Constructing experimental designs for discrete-choice experiments: report of the ISPOR conjoint analysis experimental design good research practices task force. Value Health. 2013;16:3–13.

    Article  Google Scholar 

  25. 25.

    Hauber, et al. Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR conjoint analysis good research practices task force. Value Health. 2016;19:300–15.

    Article  PubMed  Google Scholar 

  26. 26.

    Coast J, Al-Janabi H, Sutton EJ, Horrocks SA, Vosper AJ, Swancutt DR, Flynn TN. Using qualitative methods for attribute development for discrete choice experiments: issues and recommendations. Health Econ. 2012.

    Article  PubMed  Google Scholar 

  27. 27.

    Janssen EM, Segal JB, Bridges JF. Framework for instrument development of a choice experiment: an application to type 2 diabetes. Patient. 2016;9(5):465–79.

    Article  PubMed  Google Scholar 

  28. 28.

    Janssen EM, Benz HL, Tsai JH, Bridges JF. Identifying and prioritizing concerns associated with prosthetic benefit-risk assessment: a mixed-methods approach. Expert Rev Med Devices. 2018;15(5):385–98.

    Article  CAS  PubMed  Google Scholar 

  29. 29.

    Vass C, Rigby D, Payne K. The role of qualitative research methods in discrete choice experiments. Med Decis Mak. 2017;37(3):298–313.

    Article  Google Scholar 

  30. 30.

    Barber SK, Bekker HL, Meads D, Pavitt S, Khambay BS. Identification and appraisal of the outcome measures used to evaluate hypodontia care: a systematic review. AJODO. 2018;153(2):184–94.

    Google Scholar 

  31. 31.

    Barber SK, Pavitt S, Meads D, Khambay BS, Bekker HL. Assessment of information resources for people with hypodontia. BDJ Open. 2018;4:18001.

    Article  PubMed  PubMed Central  Google Scholar 

  32. 32.

    Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13:117.

    Article  PubMed  PubMed Central  Google Scholar 

  33. 33.

    Ritchie J, Lewis J, Nicholls CM, Ormston R. Qualitative research practice. A guide for social science students and researchers. 2nd ed. London: Sage; 2014.

    Google Scholar 

  34. 34.

    Barber SK, Lam Y, Pavitt S, Hodge T. Is social media the way to engage people to share their experiences of dental care? JADA. 2018;149(6):451–9.

    PubMed  Google Scholar 

  35. 35.

    Vanniyasingam T, Cunningham CE, Foster G, Thabane L. Simulation study to determine the impact of different design features on design efficiency in discrete choice experiments. BMJ Open. 2016;6:e011985.

    Article  PubMed  PubMed Central  Google Scholar 

  36. 36.

    Veldwijk J, Lambooij MS, de Bekker-Grob EW, Smit HA, de Wit GA. The effect of including an opt-out option in discrete choice experiments. Cameron DW, ed. PLoS One. 2014;9(11):e111805.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. 37.

    De Bekker-Grob EW, Hol L, Donkers B, van Dam L, Habbema JDF, van Leerdam ME, Kuipers EJ, Essink-Bot ML, Steyerberg EW. Labeled versus unlabelled discrete choice experiments in health economics: an application to colorectal cancer screening. Value Health. 2010;13(2):315–23.

    Article  PubMed  Google Scholar 

  38. 38.

    Weinman J, Petrie KJ, Moss-Morris R, Horne R. The illness perception questionnaire: a new method for assessing the cognitive representation of illness. Psychol Health. 1996;11:431.

    Article  Google Scholar 

  39. 39.

    Horne R, Weinman J, Hankins M. The beliefs about medicines questionnaire: the development and evaluation of a new method for assessing the cognitive representation of medication. Psychol Health. 1999;14:1–24.

    Article  Google Scholar 

  40. 40.

    Francis J, Wileman S, Bekker H, Barton G, Ramsay C. Beliefs about surgery: development and validation of an instrument to assess cognitive representations about surgery, in the context of a chronic illness. Psychol Health. 2009;24(10):1125–37.

    Article  PubMed  Google Scholar 

  41. 41.

    Légaré F, Kearing S, Clay K, et al. Are you SURE?: Assessing patient decisional conflict with a 4-item screening test. Can Fam Physician. 2010;56(8):e308–14.

    PubMed  PubMed Central  Google Scholar 

  42. 42.

    Akram AJ, Jerreat AS, Woodford J, Sandy JR, Ireland AJ. Development of a condition-specific measure to assess quality of life in patients with hypodontia. Orthod Craniofac Res. 2011;14(3):160–7.

    Article  CAS  PubMed  Google Scholar 

  43. 43.

    Green A. Verbal protocol analysis. Psychologist. 1995;8(3):126–9.

    Google Scholar 

  44. 44.

    Gooberman-Hill R, Burston A, Clark E, et al. Involving patients in research: considering good practice. Musculoskelet Care. 2013;11(4):187–90.

    Article  Google Scholar 

  45. 45.

    Helter TM, Boehler CEH. Developing attributes for discrete choice experiments in health: a systematic literature review and case study of alcohol misuse interventions. J Subst Use. 2016;21(6):662–8.

    Article  PubMed  PubMed Central  Google Scholar 

  46. 46.

    Burton CD, Entwistle VA, Elliott AM, et al. The value of different aspects of person-centred care: a series of discrete choice experiments in people with long-term conditions. BMJ Open. 2017;7:e015689.

    Article  PubMed  PubMed Central  Google Scholar 

  47. 47.

    Essers BA, van Helvoort-Postulart D, Prins MH, Neumann M, Dirksen CD. Does the inclusion of a cost attribute result in different preferences for the surgical treatment of primary basal cell carcinoma? A comparison of two discrete choice experiments. Pharmacoeconomics. 2010;28(6):507–20.

    Article  PubMed  Google Scholar 

  48. 48.

    Krucien N, Gafni A, Pelletier-Fleury N. Empirical testing of the external validity of a discrete choice experiment to determine preferred treatment option: the case of sleep apnea. Health Econ. 2015;24(8):951–65.

    Article  PubMed  Google Scholar 

  49. 49.

    Janssen EM, Marshall DA, Hauber AB, Bridges JFP. Improving the quality of discrete choice experiments in health: how can we assess validity and reliability. Expert Rev Pharmacoecon Outcomes Res. 2017;17(6):531–42.

    Article  PubMed  Google Scholar 

Download references

Author information




SB was responsible for participant recruitment, data collection, primary analysis and manuscript preparation. DM provided statistical support. All authors contributed to the study design and protocol development, data analysis and revision of the manuscript.

Corresponding author

Correspondence to Sophy Barber.

Ethics declarations

Conflicts of interest

Sophy Barber, Hilary Bekker, Joachim Marti, Sue Pavitt, Balvinder Khambay and David Meads have no financial or other conflicts of interest.


Sophy Barber is funded through the NIHR Doctoral Research Fellowship (DRF-2015-08-052). The research is supported by the NIHR infrastructure at Leeds. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

Data availability statement

The software code and model are provided in the ESM. The full dataset is available from the authors on request.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 229 kb)

Box 1 Protocol for Main Discrete-Choice Experiment Survey

Box 1 Protocol for Main Discrete-Choice Experiment Survey


  • To measure adolescent, parent and joint adolescent–parent preferences for hypodontia care.

  • To explore the negotiation process for reaching joint preference.

  • To evaluate the validity of DCE for eliciting adolescent and joint preferences.


Cross sectional: UK-wide online survey to elicit preferences for hypodontia care from adolescents with hypodontia and parents of young people with hypodontia.

Face-to-face: Observation of individual and joint DCE survey completion by adolescent–parent dyads to explore the negotiation process for joint decision making


Adolescents with hypodontia and their parents.

Sample Size

Cross-sectional: Sample size calculation is based on the pilot data following current suggested best practice. This indicates a minimum of 100 respondents per group or 50 per subgroup are required for estimation of preference data and to estimate the effect of subgroup factors (age, sex, severity of hypodontia, stage of treatment and anxiety). A recruitment matrix will be used to ensure adequate numbers per subgroup.

Face-to-face: Based on guidance for exploratory qualitative methods, a preliminary sample size of 15–30 adolescent–parent dyads is proposed. Purposive sampling will be used based on age, sex, severity of hypodontia and stage of treatment. Data analysis will be undertaken contemporaneously to assess data saturation, and further recruitment will be undertaken if necessary.


Cross-sectional: Online survey to be completed at home by either (1) the adolescent alone, (2) the parent alone or (3) the adolescent and parent together.

Face-to-face: First adolescents and parents will complete individual surveys separately then they will repeat the DCE choice tasks together under observation using a naturalistic approach. The repeat DCE choice tasks will be the same version, but tasks will be ordered randomly to allow comparison of individual and joint preference data. The session will take place in a private room at each site and will be audio-recorded. At the end of the survey, the researcher will conduct a short interview to explore respondent experience of the DCE


Cross-sectional: Participant identification by the direct clinical care team at a number of participant identification centres across the UK. All adolescents meeting the inclusion criteria will be invited to participate. A £5 shopping voucher will be given to all respondents to acknowledge their time contribution. Permission will be requested for contact via email to complete a re-test of the DCE choice tasks of a randomly selected sample of 10% of respondents

Face-to-face: Recruitment from three hospital sites in Yorkshire serving a diverse population. Potential participants will be identified by the direct clinical care team and all those eligible will be invited to participate. Participants will be reimbursed for travel, and the adolescent will receive a £10 shopping voucher to acknowledge their time


Statistical analysis of preference data will be undertaken using Stata v15 (StataCorp LLC, TX, USA)

  • MNL model with main effects to estimate attribute level coefficients and relative importance of attribute levels with covariant analysis to determine the effect of specific demographic characteristics (age, sex, severity of hypodontia, stage of treatment, anxiety).

  • Betas derived from the MNL will be used to calculate marginal rates of substitution (willingness to wait, willingness to accept risk).

  • External validity will be tested via the predictive validity of the DCE: Individual-level preferences for attribute levels will be estimated from the MNL model. A decision rule will be used to determine the level at which a treatment is assumed to be preferred. The congruence between the predicted and actual treatment choice will be tested.

Qualitative analysis of the observation and interview data

  • Audio-recordings will be transcribed and analysed using a phenomenological approach.

  • A thematic framework will be developed to allow within- and cross-case comparison.

  • Triangulation of data from the observation and interview will enhance reliability.

DCE, discrete-choice experiment; MNL, multinomial logit.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (, which permits any noncommercial use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Barber, S., Bekker, H., Marti, J. et al. Development of a Discrete-Choice Experiment (DCE) to Elicit Adolescent and Parent Preferences for Hypodontia Treatment. Patient 12, 137–148 (2019).

Download citation