The COS-WSL project was registered on the Core Outcome Measures in Effectiveness Trials initiative (COMET) website (http://www.comet-initiative.org/studies/details/1399) which includes ongoing and completed COS protocols and studies in various medical fields, all of which are freely available to the public. The present study protocol was written in accordance with the Core Outcomes Set-STAndardised Protocol Items (COS-STAP) [38] and the Core Outcome Measures in Effectiveness Trials (COMET) Handbook [39].
The COS-OWSL will be developed through a four-phase process: a scoping review for existing outcomes, qualitative interviews, a two-round Delphi survey, and a consensus meeting. Patients will be involved in the qualitative interviews, Delphi survey, and consensus meeting to provide their preferences and choice of outcomes regarding orthodontically induced WSLs.
Phase 1: Scoping review
To determine what to measure and to identify the existing knowledge, we will conduct a scoping review of all outcomes used in published clinical trials regarding the prevention and treatment of orthodontically induced WSLs and develop an inclusive list of the outcomes. The methods and findings of this scoping review will be reported in accordance with the PRISMA extension for Scoping Reviews (PRISMA-ScR) [40].
In this review, we will search three databases: MEDLINE (via PubMed), Embase, and the Cochrane Central Register of Controlled Trials (CENTRAL). The dates of coverage will be restricted to the most recent 10 years to obtain an up-to-date pool of relevant outcomes [39]. We will use both free-text and subject headings in our search strategy, based on the following main concepts: “tooth demineralization” OR “white spot lesion,” “orthodontics,” and “randomized controlled trial” OR “controlled clinical trial.” In addition to electronic searches, we will hand search references cited in those identified eligible studies and relevant systematic reviews. A search of grey literature will not be performed in this scoping review, as it is considered unlikely to provide new relevant outcomes or significantly alter the final list of the outcomes [41].
The inclusion criteria will be as follows: (1) study type, randomized controlled trials (RCTs) and controlled clinical trials (CCTs); (2) target population, orthodontic patients without (for prevention) and with orthodontically induced WSL-affected teeth (for treatment) with any dentition and in any health condition from both genders and all age groups; (3) intervention, an intervention for preventing or treating WSLs should be implemented and compared with another intervention or a placebo; and (4) outcome, a minimum of one subjective or objective outcome measure should be clearly reported. No restrictions on settings or follow-up periods will be adopted. The exclusion criteria will be in vitro or in situ studies, case reports, case series, observational studies, studies reported in languages other than English, and publications without available full texts.
Two authors will screen the identified records independently and in duplicate, according to the abovementioned eligibility criteria. Titles and abstracts will be screened first, and if they lack sufficient information to make a judgment, the corresponding full texts will be obtained and examined. From each included study, two authors will extract verbatim the study characteristics and outcomes used [39]. A pilot study will be conducted to calibrate the authors and develop a template for data extraction using 10% of all included studies (randomly selected). Thereafter, two authors will extract data from the rest of included studies independently and in duplicate. Any discrepancy during screening and data extraction will be settled through discussion with the other authors.
After data extraction, outcomes with similar definitions but different terminology will be merged into a single standard outcome. The outcomes of WSL prevention and treatment will be listed separately. Then, two authors will categorize these outcomes into outcome domains. Any discrepancy in the outcome classification will be discussed through group consensus among all authors. Finally, a preliminary list of outcome domains will be developed accordingly to provide information for the following phases.
Phase 2: Qualitative interviews
After the scoping review for all the existing outcomes for orthodontically induced WSLs, qualitative interviews will be performed to identify and fill the evidence gaps and to determine outcomes relevant to patients and dental professionals. The purpose of the qualitative interviews is to understand the perspectives of patients and dental professionals on orthodontically induced WSLs, and to ensure that no important outcomes are overlooked [39, 42].
Purposeful sampling will be used to recruit information-rich participants for the qualitative interviews to include as diverse ranges of opinions from patients and dental professionals as possible [43]. The recruitment of patients and dental professionals will be conducted in China. We will recruit patients with different ages, genders, occupations, types of WSL-affected teeth (primary or permanent, anterior or posterior), severity of WSLs, and experience of any intervention for WSLs (prevention or treatment). Patients under 18 years old will be allowed to express their opinions, and opinions from their parents or carers who are familiar with their condition will also be collected to form combined opinions [44]. We will also recruit dental professionals with different ages, genders, dental specialties, and level of experience in the prevention or treatment of WSLs.
Semi-structured interview questions will be developed to help ask all participants similar questions and will be piloted with a small group of patients and dental professionals. A dentist and researcher will interview the patients and dental professionals through face-to-face interviews or phone calls one by one. The researcher will be familiarized with the questions and receive training in interview skills prior to interviews. Data collection from interviews will be ended until data saturation is reached, which is defined as no generation of new themes and outcomes. When no new themes or outcomes emerge, we will continue to interview two more participants to confirm the saturation of data [45, 46].
The content of the interviews will be transcribed and analyzed. To protect respondents’ privacy, the interview will be anonymized, and all the raw data will be stored in one author’s computer and kept confidential. A content analysis will be conducted to identify outcomes or outcome domains reported by patients and dental professionals. Two authors will perform the coding procedure for the content analysis independently and in duplicate and then categorize the responses into the predefined outcome domains, with any disagreement resolved through discussion with the other authors. Only new outcomes or outcome domains that are suggested by two or more participants will be included in the outcome list [47].
Two authors will combine the outcomes from the qualitative interviews with the outcomes identified from the scoping review and develop a list of outcomes relevant to researchers, patients, and dental professionals for the following consensus process.
Phase 3: Delphi survey
The online Delphi survey will consist of two-round anonymous questionnaires to specify which outcomes the key stakeholders consider most important and to reach a preliminary consensus among them.
Purposeful sampling strategies will be adopted to recruit a diverse, international participant pool for the Delphi survey, with involvement from three stakeholder groups: patients, dental professionals, and researchers [48]. We will send potential participants a personalized email to explain the objectives of the survey and invite them to participate. To minimize attrition, the importance of their completion of the whole 2-round Delphi process will be emphasized, and individuals who agree to complete both rounds of the survey will be included. Before the survey, we will send emails to ask participants to opt in. Then, we will provide a link to the Delphi survey and the contact information of investigators to the participants who agree to participate by email. Participants who do not complete the questionnaire in round 1 will not be invited for round 2. In both rounds of the survey, email reminders will be sent every 2 weeks to participants who have not finished the survey to maximize completion rate. Each round of the survey will last for a month; if the response rate of round 2 is low, we will keep the survey open for another 2 weeks and send additional reminders to minimize potential attrition bias [39].
Both medical terms and plain language descriptions will be provided in the questionnaire to avoid ambiguity [48]. The outcomes will be voted for prevention and treatment separately and presented by outcome domains. Outcome domains and outcomes within each domain will be presented in a random order. Prior to the official release, the full text of the questionnaire will be piloted with several representatives from each stakeholder group to refine the language and increase readability for all stakeholders.
In round 1 of the Delphi survey, participants will be provided with the list of outcome domains with candidate outcomes to decide which outcomes are most important. The importance of each candidate outcome will be rated on a 1 to 9 scale recommended by the GRADE group, in which 1 to 3 represents that an outcome is of limited importance, 4 to 6 important but not critical, and 7 to 9 critical [49]. At the end of the round 1 survey, the participants will also be asked to list outcomes that they consider relevant but not included in the questionnaire. For the existing outcomes, the participants will also be allowed to make suggestions to improve the clarity of the wording. We will revise the relevant wording of the terms and definitions accordingly for the subsequent round.
The responses from round 1 will be summarized by calculating the percentage of participants scoring each score from 1 to 9 by stakeholder groups. Suggestions for new outcomes in the first round Delphi survey will be reviewed by two researchers independently and in duplicate. For responses that are unclear or ambiguous, we will ask the respondents for clarification. Only new outcomes suggested by two or more participants will be included. After removing duplicate responses and responses that show up only once, all remaining responses will be grouped into domains. If any new outcomes cannot be grouped into the existing domains, new outcome domains will be developed accordingly.
A greater number of outcomes can significantly decrease the response rates in a Delphi survey [50]. Therefore, to reduce attrition and the burden on respondents, only outcomes that meet the following criteria will be included in the second round of the survey: more than 50% of the respondents score between 7 and 9 and less than 15% score between 1 and 3 [51]. Additional outcomes suggested by at least 2 participants in round 1 will also be rated in round 2. On the top of the round 2 questionnaire, the summary results of round 1 will be fed back to each participant along with their own scoring in round 1. At the end of the round 2 questionnaire, the participants will be asked to rank their top 10 choices in order of importance [52].
The participants will be divided into two groups—patients and experts (dental professionals and researchers)—to summarize the rating scores and to evaluate consensus. The consensus criteria will be consistent with a previous study on COS development [51]. Consensus is divided into consensus in and consensus out. Consensus in, which means the outcome should be included in the COS, will be defined as at least 70% of participants in each group scoring between 7 and 9 and less than 15% scoring between 1 and 3 for an individual outcome. Consensus out, which means the outcome should be excluded from the COS, will be defined as at least 70% of participants in each group scoring between 1 and 3 and less than 15% scoring between 7 and 9. All other score distributions will be considered as a lack of consensus for the inclusion in the COS. Outcomes that remain undetermined after the 2-round Delphi survey will be discussed in a consensus meeting for the final COS-OWSL. In addition, we will analyze how individual scores change through the Delphi process to show if participants are moving towards consensus or not.
Phase 4: Consensus meeting
To achieve a final consensus on COS-OWSL, an online international consensus meeting will be held after the Delphi survey. A group of key stakeholders will be invited to the meeting to discuss the remaining outcomes that have not reached a consensus in the Delphi survey, the definition and terminology of included outcomes, and the number of outcomes to be included in the final COS-OWSL. Outcomes that achieve consensus and are included in the COS-OWSL (consensus in) after the two-round survey will also be discussed and voted to confirm their importance.
The key stakeholders attending the consensus meeting will include representatives of patients, dental professionals, and researchers. Attendees will be selected broadly from those who have extensive knowledge or experience of orthodontically induced WSLs. From each group of stakeholders, we will also invite all participants who have completed both rounds of the Delphi survey to attend the consensus meeting.
A researcher who does not engage in discussion or voting will host the online consensus meeting. The results of the Delphi survey will be presented to all attendees at the consensus meeting. The outcomes that have not reached consensus in the Delphi survey will be discussed in detail. The attendees will be asked to offer their perspectives on the importance of each outcome. Then, they will vote for each outcome as “in” or “out” for the inclusion in the final COS-OWSL. Voting will be conducted anonymously using an online application. A researcher not involved in the meeting will collect and analyze all data. Feedback will be provided to the attendees in descriptive statistics after each round of voting. Consensus will be reached on an outcome when the majority (>70%) vote as “in” or “out”, and no further voting of that outcome will be required. Outcomes that do not achieve a consensus will be further discussed to explore and resolve divergence. The meeting will alternate between voting and discussion for three rounds. If no consensus is achieved in an outcome for three rounds of discussion and voting, the outcome will not be further voted and included in the final COS-OWSL. Following the above process, we will develop the final COS-OWSL and present the outcomes by appropriate outcome domains.
Dissemination
The study results will be disseminated through peer-reviewed academic journals and international conferences. To promote the uptake of the COS-OWSL, we will engage with relevant groups/organizations such as the Cochrane Oral Health Group and International Association for Dental Research. We will also provide all the participants with a summary of the results.