FormalPara Key Points for Decision Makers

Including the target population at all stages of the discrete-choice experiment (DCE) development optimised the identification and selection of valid and relevant attributes.

The mixed-methods approach to pre-testing provided rich data to guide modifications to the DCE to improve its validity and suitability for use with adolescents.

The proposed use of the DCE survey to observe joint adolescent–parent preference elicitation and decision-making processes is supported by evidence gained from this study.

1 Introduction

Hypodontia is the developmental absence of one or more teeth, excluding third molars. It affects 3.5–6.5% of the population in the permanent dentition and is a life-long condition [1]. Diagnosis usually occurs in adolescence, necessitating treatment planning between young patients, their parents and the dental team. Often, more than one treatment option is available, each with differing outcomes and a range of long-term impacts [2]. Understanding adolescent preferences is fundamental to supporting effective decision making and provision of evidence-based care [3]. To date, no studies have explored adolescent and parent preferences for dental treatment for hypodontia or their joint decision-making processes.

Discrete-choice experiments (DCEs) enable preferences to be estimated for complex goods/services using constructed hypothetical scenarios; this enables manipulation and control over the choice situation [4]. This paper reports the studies involved in the development of a DCE survey to elicit adolescent, parent and joint adolescent–parent preferences for hypodontia care. DCE methods have been widely used in healthcare [5,6,7], but their application in dentistry is more limited [8]. Preference elicitation using DCEs with adolescents is an emerging area; however, to date, only six studies in healthcare have included respondents aged < 18 years [9,10,11,12,13,14]. Previous DCE studies have measured joint preferences, but these have been advocate preferences, for example, a parent selecting preferences for their child [15,16,17], health professionals choosing for their patients [18] or a comparison of preferences between independent respondent groups [19, 20]. Only one healthcare study used DCEs to measure preferences of related respondents [21], and this was a pilot study with only nine parent–child dyads. Outside healthcare, one study reported using DCE methods to compare the individual and joint preferences of 45 couples for beaches; however, no qualitative information was obtained to understand the negotiation process [22].

This research aims to develop a DCE survey to elicit preferences for hypodontia care from adolescents and adolescent–parent dyads. The DCE survey was tested to establish its validity for use with adolescents and the feasibility of using the DCE as a tool to explore joint decision-making processes.

2 Methods

A number of multi-attribute stated-preference-elicitation methods exist, but little robust evidence is available to determine which is most appropriate. DCE was chosen because the underlying task—in this case choosing between alternative treatments—more accurately reflects the real-world decision context than other methods, such as best–worst scaling, where the best and worst attributes are selected. Furthermore, this study provided a valuable opportunity to examine the suitability of DCEs for eliciting joint and adolescent preferences and to explore whether DCEs have scope to encourage adolescent and parent engagement in decision making.

DCE development was undertaken in sequential steps (Fig. 1) following best practice guidance [23,24,25]. Particular attention was given to the intended use of the survey for adolescent and joint adolescent–parent preference elicitation. Qualitative methods were incorporated at all stages of the DCE development to ensure adolescent and parent perspectives were represented [26,27,28,29]. The methods and approach for analysis for each stage are summarised in the following sections.

Fig. 1
figure 1

Steps in the development of the discrete-choice experiment (DCE) survey

Full size image

2.1 Compliance with Ethical Standards

Ethical approval was granted by the North West National Health Service (NHS) Research Ethics Committee (15/NW/0804), and permission was granted by each participating site. Additional ethical approval was obtained from the University of Leeds Dental Research Ethics Committee for the attribute selection survey involving members of the public recruited via Facebook (DREC 040917/SB/235 on 9.11.17). The study was adopted by the National Institute for Health Research (NIHR) Clinical Research Network and conducted in accordance with the ethical principles outlined in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.

2.2 Stage One: Attribute Development

Attribute development included two distinct steps:

  1. 1.

    Attribute identification.

  2. 2.

    Attribute selection and framing.

2.2.1 Attribute Identification

The attributes of hypodontia treatment were identified using mixed methods involving important stakeholder groups, including dentists and service providers in primary and secondary care and adolescents with hypodontia and their parents.

Dentist perspective was captured through a systematic review of outcomes used in primary empirical studies to assess the effectiveness and impact of hypodontia care [30]. In total, 56 papers were identified that provided evidence about the outcomes judged to be important by researchers and service providers. Alongside this, the information provided to adolescents and parents by dentists during the decision-making process was analysed using two methods. First, written (n = 18) and online (n = 12) patient information for hypodontia and its treatment was analysed [31]. Second, five inter-disciplinary clinical consultations between adolescents, their parents and the dental team were observed by one researcher (SB), and the content of the discussion was analysed. For the clinical observations, participants were purposively sampled from two hospital clinics to provide diversity in the sample based on adolescent demographics (age, sex, severity of hypodontia, stage of treatment) and the dental team (type of dental specialist, grade and experience). Consultations were audio-recorded without interference from the researcher and transcribed for analysis using the framework method [32, 33].

To capture adolescent and parent perspectives, semi-structured interviews with young people aged 12–16 years with hypodontia (n = 8) and their parents (n = 8) were undertaken. Adolescents were identified from treatment clinics in two hospitals using a purposive sampling matrix based on age, sex, severity of hypodontia and stage of treatment. A topic guide was developed and iteratively developed for use in the interviews. Interviews were undertaken with the adolescent first, then the parent was invited to join to add their perspective. Interviews were audio-recorded, transcribed and analysed using the framework method. To provide additional information from the perspective of people affected by hypodontia, a systematic search and evaluation of posts relating to hypodontia on social media was performed [34].

Information gained from the attribute-identification methods was synthesised by one researcher (SB). A preliminary framework for coding and categorising attributes was devised and revised iteratively by the research team. Attribute themes developed under two broad categories: attributes relating to service delivery (process) and attributes relating to treatment outcome (Table 1). Different approaches for describing attributes were recorded.

Table 1 List of all attributes of hypodontia care identified from the mixed methods in stage one with an indication of the source
Full size table

2.2.2 Attribute Selection and Framing

Attribute selection followed an iterative process to encourage an evidence-based and patient-centred set of attributes; few previous studies have explicitly reported this process [6,7,8,9,10,11,12,13,14,15,16,17,18,19,20]. Mixed methods were used, including stakeholder consultation and a questionnaire to rate and rank attributes.

Stakeholder consultation involved adolescent–parent representatives, dental professionals involved in hypodontia care and members of the research team with expertise in stated-preference methods and decision making. Three pairs of adolescent–parent representatives were identified by one member of the research team (SB) during routine clinical activity and invited to participate in consultations as ‘experts’ rather than as research participants. The purpose of the consultation process was explained, and training, support and the opportunity to withdraw was provided. At their request, the adolescent–parent representatives were seen separately to the other groups to ensure the adolescents felt at ease and to encourage open discussion. The dental professionals’ group included a convenience sample of five specialists selected from the clinical teams working with one researcher (SB). Clinicians were consulted in groups or individually to fit with clinical commitments.

The first consultation process aimed to refine the raw data from attribute-identification methods into a long list of potential attributes. The attributes were presented to each group and discussed to explore their meaning, relevance and perceived application to hypodontia care. Notes were made throughout the consultations and used to revise and modify the attribute framework. Adolescent–parent consultations explored the perceived relationship within and across attribute concepts and assessed the face validity of attributes. Attributes that were duplicates or irrelevant to adolescents and parents were excluded to develop a long list of attributes.

The long list of attributes was used to design an online questionnaire for adolescents with hypodontia and parents to rate and rank the attributes; this included 16 attributes related to service delivery and 11 related to outcome (Table 1 in the Electronic Supplementary Material [ESM]). Respondents were asked to rate how important each attribute was on a scale of 1 (not important) to 5 (very important) and select the six most important attributes for service delivery and outcome. The questionnaire is available on request. The questionnaire was converted into an online survey tool and distributed to participants via a unique link. Participants were recruited from a Facebook support group for people with hypodontia, limited to people living in the UK.

Attribute scores from the questionnaire were used to guide further stakeholder consultation to select ‘important’ attributes for testing. Aggregate scores were discussed, and the most useful and relevant attribute for representing a concept was identified. Adolescents and their parents confirmed the acceptability and face validity of the attributes, informed the suitability of dimensions for the attribute levels and provided information about appropriate language and framing.

2.3 Stage Two: DCE Survey Development

DCE design and survey development followed best practice guidance [24, 25], including consideration of the 10-point checklist for conjoint experiment design provided by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) [23].

2.3.1 Experimental Design

A fractional factorial experimental design was selected because of the impractical number of options created by a full factorial design (35 × 22 = 972 choices). A multinomial logit model with main effects design was specified in Ngene v1.2 (ChoiceMetrics) with d-optimising efficiency, excluding dominant choice sets. The design was checked for level balance across choice tasks and plausibility. As the ideal number of choice tasks remains controversial [35], eight choice tasks were presented in each survey version in the pre-testing to assess how respondents coped. Blocking was used to increase the number of tasks that could be included; pre-testing included 32 choice tasks in total (four survey versions each with eight pair choices).

2.3.2 Choice Task Construction

Task construction aimed to reduce cognitive burden and optimise acceptability for the target respondent group. Adolescents’ ability to trade-off attributes is seldom reported, and evidence for the optimum number of tasks is weak. To minimise the complexity of the choice tasks, paired choice tasks were used.

An opt-out was desirable to reflect that dental treatment for hypodontia is elective; however, there was concern that an opt-out may lead to high levels of non-response where the trade-off was judged to be difficult [36]. A status quo treatment [23] was not possible because of variations in individual care plans. To manage these challenges, a ‘no treatment’ option was included for pre-testing, but this was offered after a preference was given for one of the treatment options.

Generic labels were used for the options (treatment A and treatment B) because they did not reflect exact treatment options and there was a possibility that labels might encourage detrimental heuristics [37]. Two qualifying questions were included with the choice task: strength of preference (How much do you like treatment X?) and perceieved difficulty of choice (How easy was it for you to pick?) [23]. These were measured using a 5-point scale anchored with ‘a lot’ to ‘not at all’ and ‘very easy’ to ‘very difficult’, respectively.

2.3.3 Survey Design

Survey components included an introduction to the survey and explanation of its purpose, questions to obtain demographic data, an explanation of the attribute and DCE task followed by the eight DCE choice tasks, and background questions identifying factors that may influence preference ratings. Background questions included questions about knowledge (based on the domains of the Illness Perception Questionnaire [38]), beliefs about dental treatment (developed from the Beliefs about Medicine Questionnaire [39] and the Beliefs about Surgery Questionnaire [40]), decision making (based on the SURE tool [41]), dental anxiety (5-point Likert scale) and the impact of hypodontia (using questions from a hypodontia-specific tool [42]).

A commercial research company experienced in collecting and managing stated-preference data created an online survey delivered through a secure platform. Screening questions and methods for obtaining consent were tested to ensure acceptability for use in a future remote online survey. Screening questions asked respondents to confirm they had hypodontia, were aged 12–16 years, lived in the UK and had parental consent to answer the survey. Adolescent understanding of the questions was checked throughout the pre-testing, and parents were explicitly asked whether they found the level of consent acceptable.

2.4 Stage Three: Testing the Discrete-Choice Experiment (DCE) Survey

Given the scant evidence about DCE design for adolescents and joint preference elicitation, rigorous pre-testing was undertaken using combined qualitative and quantitative methods. Pre-testing aimed to explore the acceptability and feasibility of proposed methods in a real-world sample.

2.4.1 Methods

Pre-testing was undertaken using two approaches:

  • Cross-sectional survey employing cognitive interview methods [43] to test the validity and acceptability of the DCE survey. Contemporaneous analysis of feedback facilitated iterative modification and re-testing.

  • Analysis of preference data to provide coefficient estimates for each attribute level. Estimates were then compared with qualitative feedback to estimate the validity and importance of attributes.

2.4.2 Sampling and Recruitment

A sample of 12 adolescents and eight parents were invited to participate, providing a total of 160 observations. Eligible participants were invited from two hospital orthodontic departments in Yorkshire. This included (1) adolescents aged 12–16 years with varying levels of severity of hypodontia and at any stage of treatment and (2) parents or guardians of adolescents with hypodontia. The only exclusions were people with hypodontia related to craniofacial syndromes with significant comorbidities that would impact on healthcare provision, and any participants unable to provide consent. Purposive sampling was used based on age, sex, severity of hypodontia and stage of treatment.

2.4.3 Data Collection and Analysis

Participants (14 individual participants and three adolescent–parent dyads) completed the DCE survey using a laptop in a private room. The researcher (SB) prompted participants to verbalise thoughts as they worked through the survey and at the end asked questions focusing on the relevance and importance of the attributes for making decisions about hypodontia treatment, the perceived complexity of the DCE task, time taken to complete the survey and clarity of instructions. All verbal responses were noted.

Choices in the DCE choice tasks were collected automatically by the online survey tool, and the numerical preference data were collated for analysis using a conditional logit model (Box 1 in the ESM). The small sample size meant estimates were interpreted with caution, and no subgroup analyses were undertaken.

3 Results

3.1 Stage One: Attribute Development

Attribute identification methods resulted in the identification of five attribute concepts relating to service delivery and six attribute concepts relating to treatment outcome and impact (Table 1). Each concept included multiple attributes, and different descriptions were used within each attribute, for example, the attribute ‘bite’ was described as ability to bite, ability to chew, confidence to bite, ability to eat hard food, avoidance of certain foods. The source of each attribute is provided in Table 1; this demonstrates the value in using mixed methods to ensure comprehensive attribute identification. Many of the attributes identified in the literature review were clinical markers that allow objective measurement of dental health and treatment effect by dental professionals, but these were difficult for adolescents and parents to understand and interpret. Clinical consultations and patient information resources rarely included comprehensive information relating to patient experience and outcome measures.

Initial elimination of inappropriate attributes resulted in a long list of 27 attributes relating to treatment process (n = 16) and treatment outcome (n = 11) (Table 1 in the ESM). These were included in a rating and raking questionnaire, which was completed in October–November 2017 by 11 parents (two fathers, eight mothers and a guardian) and eight adolescents. The adolescent respondents showed a spread of age ranges and an equal sex balance but a disproportionately high number of people with severe hypodontia. The questionnaire showed differences in adolescent and parent scores (Fig. 1 in the ESM); however, general trends were identified, and these were used to discuss with stakeholders which attributes were most important to include. The selection process identified seven attributes representing different attribute concepts that were agreed to be important to both adolescents and parents. Stakeholder input was sought regarding appropriate ranges for the attribute levels based on best available evidence identified in the literature review and clinical experience. Adolescent and parent advice regarding language and framing of attribute levels to convey their intended meaning accurately to respondents was followed.

3.2 Stage Two: Survey Development

The attributes and attribute levels included in the DCE choice tasks for pre-testing are described in Table 2 in the ESM. The approach and rationale for survey development is described in Sect. 2.3. An example of the full survey is available on request.

3.3 Stage Three: Pre-Testing

Pre-testing explored the acceptability of the DCE survey method, the validity of the attributes, and the ability of the method to encourage respondents to consider what is important to them.

3.3.1 Acceptability of DCE Survey Method

The DCE survey method required modification to improve acceptability, as both adolescents and parents experienced difficulties understanding the purpose and method of the choice tasks. Adolescents in particular asked for ways to help process the task, such as highlighting the ‘better’ attribute in each scenario or a function to allow the preferred attribute in each scenario to be selected individually rather than having to make a choice between the whole profile. The ability to perform the DCE choice task appeared to be related to respondents’ ability to understand the concept of trading off attributes rather than the age of the respondent. Different approaches were trialled until respondents demonstrated understanding of the choice tasks and the concept of trading-off.

3.3.2 Validity of Attributes

The validity of the DCE method was assessed by comparing verbal feedback and preference estimates derived from the choices made in the DCE tasks. Statistical analysis of the preference data found most attribute coefficient signs behaved as expected based on the verbal feedback; that is, attribute levels that were perceived to be beneficial showed a positive sign, indicating a gain in utility, whereas those perceived as detrimental had a negative sign, indicating a loss in utility (Table 2 in the ESM). Compromised appearance caused the greatest reduction in utility, followed by severe problems during treatment and no improvement in bite, which reflected respondent feedback that these were highly unfavourable.

Importantly, the verbal feedback and preference estimates for ‘waiting time’ and ‘discomfort’ were not in agreement. The positive attribute coefficient associated with increased waiting time suggested adolescent and parent respondents gained utility from waiting, but this was contradicted by the verbal feedback, which suggested increased waiting time is not at all desirable. Similarly, for the parent group, there was a gain in utility for severe discomfort, but parents reported a preference for avoiding discomfort. It is likely that these results arose erroneously because of the small sample size; as a result, the attributes are to undergo further testing.

‘Cost of dental treatment in the future’ was shown to be problematic for adolescents in both the verbal feedback and the preference estimates. Adolescents reported little understanding or interest in cost, and there was little awareness that certain dental treatments will carry future costs. The attribute coefficients for adolescents suggested slightly more utility loss from a future cost of £50 per year than £250 a year, which suggests an irrational response to the cost attribute. On the other hand, parents’ response to cost was as expected, with the highest cost demonstrating the greatest reduction in utility.

The disparity between the qualitative feedback and preference estimates emphasise the importance of including detailed pre-testing to examine attribute validity. In this study, reliance on preference estimates only would have misinformed attribute selection.

3.3.3 Validity of DCE Tasks

Cognitive interviewing and the additional probing questions during pre-testing provided evidence that the DCE tasks encouraged respondents to consider what is important to them. Completion of the DCE choice tasks by adolescent–parent dyads promoted greater discussion around preferences, treatment choices and willingness to compromise. The attributes provided prompts and a shared language to help adolescents articulate their concerns and priorities more effectively. Adolescents reported that outcome was most important and that they would ‘put up’ with less favourable process attribute levels. Parents placed value on appearance after treatment and avoiding problems during treatment. Two parents of adolescents near the end of treatment or post-treatment commented that talking through the tasks helped them realise their child’s values had changed throughout the course of dental treatment.

3.4 Future Study

3.4.1 Modifications to the DCE Survey

The findings from the pre-testing, combined with further stakeholder input, have been used to revise the DCE survey to optimise its validity for use with the target respondents (Table 3 in the ESM). Contradictory results for ‘waiting time’ and ‘discomfort’ mean both will be included in the final DCE for further investigation. ‘Future cost of dental treatment’ was excluded based on the evidence from both methods of pre-testing, which indicated cost is not valid for the approach with adolescent respondents.

Estimates of attribute coefficients from analysis of the preliminary DCE preference data were used to improve the efficiency of the experimental design. To address the dominance of appearance, the design produced overlaps in the appearance attribute between profiles in 18 of the 28 choice tasks. The final survey contains five parts: introduction (including screening and consent); demographic questions; DCE choice task; background questions; and closure (including sources of support). The presentation of choice tasks was simplified as much as possible, and full descriptions of each attribute were made easily accessible through information buttons throughout the tasks (Fig. 2 in the ESM).

3.4.2 Planned Data Collection

Evidence from this study has been used to inform the future principal DCE survey methods (“Box 1”). This will involve both a face-to-face survey using cognitive interview methods to observe individual and joint decision making between adolescent–parent dyads alongside a traditional cross-sectional online survey method to elicit preferences from adolescents and parents (Table 2).

Table 2 Description of the pilot and final survey
Full size table

4 Discussion

This systematic research approach provided evidence to inform the design of a DCE that is acceptable to adolescents with hypodontia and their parents for preference elicitation. The methods and data that were key to informing the design of the DCE tasks and survey are reported, and we are confident that—by including the target population during research development [44,45,46] and using patient-centred research methods during attribute identification, selection and testing—we have ensured the attributes are relevant to the target audience. The cognitive interview method effectively tested respondents’ understanding of attributes, the number of tasks that could be managed and different approaches for explaining the attribute levels and task concept. This approach provided more information than would have been gained by a larger-scale online pilot, which would only have provided preference data, and addressed issues arising from the lack of guiding evidence for DCE use with adolescents. For example, cognitive interviewing emphasised that ‘waiting time’ and ‘discomfort’ had not behaved as expected and should be included for further testing and confirmed that ‘cost of dental treatment in the future’ was a problematic concept for adolescents and should not be included in the final DCE. It may be argued that cost was valid for parents and its inclusion would have been beneficial to assess its importance for parents; however, the copayment system for dental care in the UK presents challenges for valid interpretation of willingness-to-pay estimates and—although another DCE study has managed this by using DCE designs with and without a cost attribute [47]—this was infeasible within this study.

We acknowledged that the data presented were elicited from a small sample and, although this is appropriate for the development phase of a DCE, further data collection and analysis with a larger sample is required to describe preferences that are representative of adolescents and parents making decisions about hypodontia treatment. Furthermore, the validity of DCE methods for use with adolescent respondents requires careful evaluation; a number of complementary methods to investigate reliability and validity [48, 49] have been included in the future DCE survey (Table 4 in the ESM).

This study provides useful insight into the future work needed to address service and patient needs. Attribute identification demonstrated discrepancies in which aspects of care adolescents, parents and clinicians considered important. For example, outcomes identified in the literature review included clinical markers of treatment effect but few patient-centred outcomes, such as impact of treatment on everyday activities. Future research into treatment efficacy and delivery needs to ensure outcome measures are used that will contribute to an evidence base that is relevant to patients. Pre-testing highlighted that patients and families have inadequate understanding of certain treatment attributes and there is a clear need for more decision support for patients and families to help them identify their own preferences.

The authors’ proposed use of the DCE developed by this process as a tool for exploring decision making between family members is novel. With appropriate use of qualitative methods, this approach should provide insight into how people with hypodontia and their families make treatment trade-offs; this may support changes to the consultation process and clinical care delivery. Our findings suggest that age is not a reliable indicator of whether or not adolescents have the capacity to be involved in complex decision making about treatment. The ability of adolescents of different ages to conceptualise short- and long-term consequences and trade-off of their treatment varied. This highlights one of the challenges for dental staff when attempting to involve people with hypodontia and parents in clinical decision making.

5 Conclusions

Mixed methods were essential to identify and select attributes that are important to adolescents with hypodontia and their parents. Use of cognitive interviewing during pre-testing of the DCE survey tool enabled deeper exploration of adolescent and parent experience and understanding. Pre-testing highlighted that ‘future cost’ is not a valid attribute for adolescents and that DCE choice tasks can be complex for both adolescents and parents. The protocol for the principal DCE survey draws on the findings from the pre-testing stage: alongside a traditional cross-sectional online survey method to elicit adolescent and parent preferences, a face-to-face survey using cognitive interview methods is planned to observe joint decision making in adolescent–parent dyads and to further explore the validity of DCE methods.