Prioritizing core components of successful transitions from child to adult mental health care: a national Delphi survey with youth, caregivers, and health professionals

Abstract

Youth accessing mental health care often experience a disruption in care as they attempt to transition between child and adolescent mental health services (CAMHS) and adult mental health services (AMHS). Few studies have evaluated interventions seeking to improve the experience and outcomes of CAMHS–AMHS transitions, in part due to lack of consensus on what constitutes best practices in intervention success. As such, the aim of this study was to engage patients, caregivers, and clinicians to prioritize core components of successful CAMHS–AMHS transitions which can be used in the design or evaluation of transition interventions. As such, a Delphi study was conducted to determine core components of successful CAMHS–AMHS transitions. Guided by the principles of patient-oriented research, three balanced expert panels consisting of youth, caregivers, and clinicians ranked and provided feedback on the importance and feasibility of core components of CAMHS-AMHS transitions. Components endorsed as feasible or important with ≥ 70% agreement from any panel moved to the next round. As a result, a list of 26 core components of CAMHS–AMHS transitions has been refined which can be used in the design, implementation, or evaluation of interventions intended to improve transition experiences and outcomes for youth in mental health care. Youth and families were engaged in an expert advisory role throughout the research process, contributing their important perspectives to the design and implementation of this study, as well as interpretation of the findings.

Introduction

Mental illness is a prevalent issue, particularly among youth. Approximately 1 in 5 children and adolescents have at least one mental health problem [1, 2]; with up to 70% persisting into adulthood [3]. Adolescents and young adults are also increasingly accessing care for their mental health in recent years [4]. For those using mental health services, the transition of care from child and adolescent mental health services (CAMHS) to adult mental health services (AMHS) is acknowledged as a challenging health systems hurdle [5,6,7,8]. A common point of termination in mental health services occurs at age 18, during which youth receiving CAMHS must discontinue care and transition out [9, 10]. A successful transition ensures Continuity of Care (CoC) through a planned health care process that addresses personalized therapeutic and developmental needs of the patient [11]. If CoC is not achieved at the point of transition, youth who are engaged in CAMHS and require ongoing mental health care after age 18 are at risk of worsening mental health and functioning [12,13,14]. Currently, as many as 6 of 10 youth disengage from service use [12, 15]. Negative outcomes of disengaging from mental health care services include more severe mental health problems, inadequate medication monitoring, re-engaging later in crisis, increased involvement with the justice system, among others [12, 16,17,18,19]. Despite this, there is a lack of research evaluating interventions to improve CoC for youth transitioning from CAMHS to AMHS and a lack of agreement on what interventions designed to support CoC during transitions should focus on.

A scoping review of interventions to support CAMHS–AMHS transitions revealed that few interventions had been evaluated [20]. This review concluded that research may be impeded by the lack of psychometrically sound instruments to appropriately evaluate transition programs and interventions. In our own scoping review of components of CAMHS to AMHS interventions, we identified and extracted an extensive list of 26 core components from 57 articles and 33 documents [21]. Results demonstrated a lack of consensus among experts as to which components are important and feasible to implement in the evaluation of programs to improve CoC from child to adult mental health services [21]. Recent work to identify core process and outcome indicators for child to adult care transitions in other health care specialties, such as hematology and rheumatology, have primarily engaged health care professionals to the exclusion of youth and families [22,23,24,25]. Within mental health care, however, research has demonstrated that various stakeholders (i.e. youth, caregivers, and clinicians) have different experiences and needs when it comes to supporting transitions in mental health care [17, 26]. Furthermore, youth and caregivers have demanded to not only be included in this research but also to be engaged in its co-design [27, 28]. As such, studies ought to engage patients (youth and caregivers) as experts alongside clinicians, policy-makers and researchers when developing tools for evaluating transitions from CAMHS to AMHS. Delphi studies are an ideal methodology for integrating diverse stakeholders in a decision-making process.

A Delphi study is a multi-staged survey designed to achieve consensus among experts on an important issue [29, 30] and has been used extensively in health care [31] and more recently mental health care [32]. Information is gathered using a series of online survey rounds with feedback from the previous round integrated into the subsequent rounds using clear and transparent criteria. The Delphi method prevents domination of the consensus process by one or a few experts by including individuals from various locations and areas of expertise to respond anonymously [33]. This facilitates voices from all stakeholders to be heard equally. This method is particularly effective in engaging stakeholders, including patients and clinicians, in research and ensuring that the priorities of all participants (including youth and their caregivers) are known [30, 34, 35].

With this in mind, the objective of this Delphi study was to engage patients, caregivers, and clinicians as experts in prioritizing and refining the core components of effective CAMHS–AMHS transitions that were identified in our scoping review [21].

Methods

Study design

A modified e-Delphi method was utilized. This method involved creating initial survey items based on an extensive review of the available literature, as well as input from expert advisors with lived experience [36,37,38,39,40]. Survey items were then distributed to three expert panels for ranking based on importance and feasibility. This study was reviewed by the University of Toronto Health Sciences and the Centre for Addiction and Mental Health Research Ethics Boards.

Expert advisory committee

This study utilized the priniciple of Patient-Oriented Research (POR), guided by the Strategy for Patient-Oriented Research Patient Engagement Framework (SPOR-PEF) in developing the methodology needed to enable patients to engage as experts [41]. As such, a group of patients and knowledge users (KU) formed a study Expert Advisory Committee (EAC). The EAC was composed of three youth, two caregivers, and one sibling, all with lived experience [42,43,44], as well as four knowledge users [45] (one health care administrator, one decision-maker, and two community leaders) from relevant child and youth mental health agencies. The EAC met regularly (every 2–4 months) with the study team over the course of the project to advise on recruitment strategies, development of the supplemental materials (i.e. informational video [46]) interpretation of the findings, knowledge translation goals and strategies to reach youth, families and other stakeholders, and dissemination of the project findings.

Delphi expert panel members

Participants who met criteria for one of the three expert panel groups were recruited. These panels include (a) Panel 1, comprised of service users, specifically youth (aged 19–25) who have received mental health care from a CAMHS and have transitioned to an AMHS; (b) Panel 2, comprised of family members or caregivers of youth aged 19–25 who had received mental health care from a CAMHS and have transitioned to AMHS at the time of recruitment; and (c) Panel 3, comprised of clinicians from any professional designation (i.e. registered nurse, physician, etc.), who have had at minimum one year of experience in research or clinical treatment of youth and/or early adult mental health care (ages 16–25) and mental health administrators who have responsibility for the delivery and/or oversight of CAMHS.

Recruitment of expert panel members

Purposive sampling recruitment took place over a period of 6 weeks. For Panel 1 (youth) and 2 (caregivers) our project partners and EAC members sent out the project information via email to their contacts and listserv’s, which included youth, caregivers, and/or clinician/administrator. For Panel 3, we utilized three strategies to identify potential panel members: (1) relevant conference attendee lists were searched, (2) professional associations membership lists were reviewed, and (3) authors of manuscripts that were reviewed in the scoping review were search. All identified experts were then invited via email to participate in the survey. If interested, individuals contacted the research team via email at which point they were sent an invitation email by the principal investigator (KC) to introduce the study and confirm their eligibility. Stakeholders and experts were given the opportunity to recommend additional individuals they felt should be included as participants. In recognition of their time and commitment, youth and caregiver participants received a $40 honorarium gift card for participating in Round 1 of the Delphi survey. In Round 2 all participants (including clinicians and administrators) across all panels received the honorarium for participation to optimize response rate and at the suggestion of the study EAC.

Survey development

A scoping review of the literature was previously conducted by the research team, which informed the items that would be included in the Delphi study [21]. In total, 26 core components, grouped into six core elements, of CAMHS to AMHS transitions were derived from the academic and gray literature in this scoping review (see column “Initial description of core components” in Table 1). This list was reviewed in-person by the EAC for modifications and edits (e.g. wording, order, omissions) before the first Delphi survey round was administered to participants. Following their review, the EAC determined, by consensus, that a core component regarding access to peer support should be included, leading to the addition of component 4.11. The final survey consisted of 27 core components that were rated on a 9-point Likert scale on two domains: importance and feasibility. Importance referred to how significant or meaningful the core component is to the measurement of transition quality between CAMHS and AMHS. Feasibility referred to how realistic it would be to implement the core component in practice. These criteria have previously been used in Delphi studies examining transitions to adult healthcare services [22, 25], and their definitions are aligned with prior Delphi studies [47, 48]. All three panels of participants were provided space for additional comments and/or feedback regarding each core component, if desired. At the end of the survey, participants could also provide overall comments, including suggestions of any missing components that should be added.

Table 1 Initial components of successful CAMHS-AMHS transitions generated by scoping review by Cleverley et al. (2020) and final components following Delphi rounds

Demographic questions specific to each expert panel were included as part of Round 1 (see Table 2). Youth, caregiver, and clinician/administrator experts were asked panel-specific demographic information, based on review from the EAC, to ensure that collected data was appropriate and provided meaningful information about each panel. For example, youth were asked to provide information such as their age, gender, and living situation, whereas clinicians were asked to provide primary clinical population served, years of professional experience, etc.

Table 2 Characteristics of expert panels

Determining consensus

Having ≥ 70% of any of the three panels highly endorse a core component (i.e. rate it at 8 or 9 on the 9-point Likert scale) on either the importance or feasibility domain ensured that the component was carried over onto the next round [49].

Delphi rounds

Each round of the survey was developed and administered using a secure web-based software platform designed to support data capture for research studies, REDCap (Research Electronic Data Capture) [50]. Participants could complete the survey over several sittings, saving their answers and returning at a later time, if needed. Participants were invited to participate via email and provided a web link to the survey. Each round was open for 4 weeks. Participants who complete at least one core component question were considered to be partials and included in the study, even if they did not complete the entire survey. To optimize response rates, reminder emails with the survey links were sent every 7 days to participants who either did not open the survey or only partially completed the survey. Partial and complete participants from Round 1 were invited to participate in Round 2. The de-identified quantitative (panel-specific range of the ratings and mean rating for each core component) and qualitative (comments across all panels, edits to the core components, and the suggestions to add new core components) results from Round 1 were included in Round 2 [32]. In Round 2 participants were asked to review the information from Round 1 and either maintain or modify their original ratings. All data was collected between April and July 2018.

Statistical analysis

All analyses were undertaken using SPSS 25 [51].The demographic characteristics of each panel were summarized using descriptive statistics, including frequency counts and proportions, and means and standard deviations.

For each of the 27 core components, the percentage of participants within each panel who highly endorsed each core component (i.e., rated 8 or 9) was calculated separately for importance and feasibility. Any core component highly endorsed by ≥ %70 of at least one panel on importance or feasibility was retained for the second round. This procedure was followed for both Round 1 and Round 2.

Qualitative analysis

Qualitative comments from Round 1 were reviewed and summarized by the research team and reviewed with the EAC. Comments were selected if they were relevant and re-occurring among panel experts. Qualitative comments were extracted and analyzed using Braun and Clarke’s [52] process of thematic analysis to identify key themes of experts’ ratings. This involves familiarization with the data through extensive review, and subsequent generation of themes, which in turn need to be reviewed and defined. One graduate prepared research assistant reviewed all comments and grouped each one into overarching themes. A qualitative review after Round 2 further sub-categorized comments based on their context. The PI (KC) reviewed the preliminary analysis to ensure that the themes accurately captured the depth of the comments.

Results

Participant characteristics

Of the 87 experts sent the Round 1 survey, 58 participated in the survey, resulting in a participation rate of 66.7%. The three panels included 21 clinicians and administrators, 17 caregivers, and 20 youth. All participants who provided a response in Round 1 (58) were sent the Round 2 survey, of which 51(87.9%) responded to at least one item.

In the caregiver panel, nearly two-thirds were biological mothers (64.7%), and most resided in the province of Ontario, Canada (88.2%). Just over half had female children (58.5%) and their child’s first access to mental health care was, on average, at age 14 (sd = 4.4). The most commonly reported diagnosis was an anxiety disorder (64.7%), and the majority of their children were still receiving mental health care (58.8%). See Table 2 for a full description of the panels.

The youth experts were mostly female (80.0%) and from Ontario (80.0%). Just over half were currently enrolled in school (60.0%) and living with parents/in a family home (55.0%). Almost all reported a diagnosis of an anxiety disorder (95.0%), with fewer reporting a mood (65%) and/or personality (30.0%) disorder. On average, the youth first accessed mental health care at age 15 (sd = 2.9), and many were still using mental health services (55%). The mean age of the youth experts was 21.5 (sd = 1.8). See Table 2.

In the clinician and administrator panel of experts, the largest professional group was psychiatrists (38.1%), followed by nurses (23.8%) and service managers (14.3%). Most of the panel members were practicing at an Ontario academic hospital (81.0%), and nearly all primarily with a clinical population of children and youth under 18 (95.2%). On average, the panel members had worked in their profession for 20.5 years (sd = 10.2). See Table 2.

Round 1

Results of the Round 1 survey are displayed in Supplementary File 1, indicating the number and percentage of experts ranking each core component as 8 or 9 (i.e. highly endorsed) for importance or feasibility for each panel.

Importance Among clinicians/administrators, the level of agreement for importance was the greatest for core component 1.6, which was highly endorsed by 100% (see Supplementary File 1). Eight other core components were highly endorsed by ≥ 90% of the panel (1.4, 1.5, 3.2, 3.3, 4.2, 4.3 4.8–4.10), while six core components did not reach ≥ 70% agreement (2.2, 3.1, 4.5, 4.11, 5.2 and 6.1). Among caregivers, the level of agreement was the greatest for core components 4.6 and 4.7, which were highly endorsed by 92.9%. These were the only two core components highly endorsed by ≥ 90% of the panel; only two core components did not reach ≥ 70% agreement (2.2 and 4.3). Among youth, level of agreement for importance was the greatest at just over 94% highly endorsed for core components 1.6 and 4.2; these were the only two core components highly endorsed by ≥ 90% of the panel. Seven core components did not reach ≥ 70% agreement (1.1, 1.3, 2.1, 4.1, 4.4, 4.6, and 5.2). Across the panels, 14 core components were highly endorsed by ≥ 70% in each group, while none the core components were endorsed by < 70% in all three panels.

Feasibility Within all panels, levels of agreement for feasibility were much lower. Among the clinicians/administrators, the greatest level of agreement at ≥ 70% highly endorsed was found for core components 4.3, 4.8, and 4.9, meaning that the remaining 25 core components did not reach 70% agreement (see Supplementary File 1). Among the caregivers, level of agreement was greatest at 78.6% highly endorsed for core component 4.9, with five other core components being highly endorsed by ≥ 70% (3.2, 4.1, 4.8, 4.10, and 6.1). Among the youth, the greatest level of agreement was 77.8% highly endorsed for core component 4.9, with only one other core component achieving ≥ 70% agreement (4.2). Across the panels, one core component was endorsed by ≥ 70% (4.9) in each group.

Changes from Round 1 to Round 2

Only 1 core component (5.2) was removed and no new core components were added, resulting in a total of 26 core components to be used for Round 2.

Round 2

Of the experts who participated in Round 1, 19 clinicians/administrators (90.4%), 14 caregivers (82.4%), and 18 youth (90%) participated in Round 2 (i.e. responded to at least one item). The results of the Round 2 survey are displayed in Supplementary File 2, indicating the number and percentage of experts ranking each core component as 8 or 9 (i.e. highly endorsed) for importance or feasibility for each panel.

Importance Among the clinicians/administrators, the level of agreement for importance was the greatest for core components 1.6, 4.2, and 4.8 (100.0% highly endorsed), with eight other core components highly endorsed by ≥ 90% of the panel (1.4, 1.7, 2.1, 3.2, 3.3, 4.3, 4.6, and 4.9). Only three core components did not reach ≥ 70% agreement (down from 7 in Round 1; 2.2, 3.1, and 4.11). Among the caregivers, the level of agreement was greatest at 100% for core components 1.4, 4.9, and 4.10, with 10 other core components highly endorsed by ≥ 90% (1.1, 1.6, 3.2, 3.3, 4.2, 4.5–4.7, 5.1 and 5.3). Three core components did not reach ≥ 70% agreement (up from 2 in Round 1; 1.7, 2.1, and 4.11). Among the youth, the level of agreement for importance was the greatest for core component 4.2 (94.4%) and this was the only core component that was highly endorsed by ≥ 90%; four core components did not reach ≥ 70% agreement (down from 7 in Round 1; 2.1, 2.2, 3.1, and 4.1). Across the panels, 20 core components were highly endorsed by ≥ 70% by each group (up from 14 in Round 1), but, mirroring Round 1, none of the core components were endorsed by < 70% in all three panels. There were differences in the core components that were ranked most important and feasible by the panels (see Table 3).

Table 3 Top 3 Components Based on Percentage Highly Endorsed per Panel for Feasibility and Importance in Round 2

Feasibility Similar to Round 1, levels of agreement for feasibility were much lower than for importance. Among the clinicians/administrators, three core components were highly endorsed by > 70% (4.8, 4.9, and 5.3, see Supplementary File 2). For the caregivers, only one core component was endorsed by ≥ 70% (4.9). Among the youth, four core components were endorsed by ≥ 70% (1.5, 1.6, 4.6, and 4.8). None of the core components were endorsed at ≥ 70% in all three panels.

Round 2 consensus and final list of core components

Any core component highly endorsed at ≥ 70% on importance or feasibility by at least one panel was retained. This resulted in all 26 core components being retained.

Final list of transition core components

The final list of transition core components, with all changes made following EAC input and both Delphi Rounds, is available in Table 1 (see column “Final description of core components”).

Qualitative themes

Three themes were identified from the open-ended qualitative comments provided by experts when asked to elaborate on their reasoning and differences in the rating of the core components in both Rounds. The themes identified were: (1) need for CAMHS and AMHS collaboration; (2) suggestions on how to operationalize core component in practice; and (3) barriers to implementation in practice. A sample of supporting participant quotes for each of these themes are provided in Supplementary File 3. Clinicians and service administrators repeatedly emphasized the need for enhanced communication and active engagement/relationship development between AMHS and CAMHS to increase collaboration between these services. Many participants from all three panels made suggestions on how to implement the core components, such as creating integrated care pathways/plans, ensuring appropriate education and training of involved care providers, and providing readily available online centralized transition resources. A number of barriers to component implementation and feasibility were mentioned in comments across all three expert panels, such as (1) AMHS wait times, (2) challenges posed by differences in CAMHS and AMHS service funding structures, and (3) resources required for training on transitions best practices and for administrative support.

Discussion

This Delphi study provided the opportunity to refine a list of core components of CAMHS to AMHS transitions with the expertise of youth, caregiver, and clinician/administrators. The initial list of 26 components generated from the scoping review [21] all moved forward to Delphi Round 1 after review by the EAC, with one addition, resulting in 27 components being evaluated. No further components were added after the Delphi rounds (though one was removed), indicating both that the initial scoping review was robust and that the available literature reflects the reported needs and experience of youth, caregivers, and healthcare providers fairly accurately. This method of using an exhaustive literature review to generate the initial survey, rather than having the panels create the list of core components, may allow for Delphi studies to be conducted with reduced burden on the participating experts who are providing their time and energy to assess the survey items. The component that was removed, “Transfer young adult when his/her condition is stable”, was taken out of the list between Rounds 1 and 2. This was because of relatively low endorsement of importance and feasibility as well as comments from all three panels questioning its inclusion due to the complexity inherent to some of the youths’ illnesses and limitations caused by age cut-offs for services. As one caregiver panel member wrote, “stability is often a day to day situation, not an ongoing state in some cases”.

This Delphi study used a multi-panel approach with the goal of balancing the input of health care professionals, youth, and caregivers. Keeping all core components with a high level of endorsement within at least one of the groups, ensured that items deemed highly feasible/important to one group, but not others, would be retained. A number of previous Delphi studies on pediatric to adult health care transitions have used expert panels dominated by, or exclusively made up of, health care professionals with either no or a minority of experts from consumer and/or caregiver groups [22,23,24,25, 53, 54]. Among single panel Delphis, most included only a few patient/caregiver members, and as such there is a risk of their rankings carrying less weight than those of health care professionals when attempting to reach a consensus. The balanced panels in this study allow youth and caregivers to be engaged as partners with equal levels of expertise in determining criteria for successful transitions.

Despite the diverse sources of expertise among our panels, there was an overall high level of agreement for importance within and between the expert panels. However, the components with the very highest endorsement for importance within each panel (i.e. each panel’s top three) differed between the youth, caregiver, and clinician/administrator panels. This suggests that youth, caregivers, and clinician/administrators, while agreeing on the importance of the core components overall, may prioritize needs differently when engaged in the transition process. This difference highlights the importance of further engaging patients and families in the co-design of transition interventions based on these core components. Of note, the youth panel had an overall lower level of consensus in their endorsement of core components than either of the other two panels. This may indicate that individual differences between youth accessing care warrant a tailored approach when managing the transition process. This is reflected in existing qualitative research which has found that youth have a high need for flexibility when it comes to the planning and timing of CAMHS to AMHS transitions [26, 28].

Compared to importance, very few core components received a high level of endorsement for feasibility from any of the panels. Experts from all three panels identified that a lack of resources, as well as the siloed administration of child and adult mental health systems, pose significant barriers to implementing the components. Panelists further described in their comments a lack of connection between the two mental health systems, leading to lengthy wait lists for adult services or difficulty finding appropriate adult services, ultimately resulting in long periods of time with no access to mental health care. In many countries, CAMHS and AMHS are completely separate organizations, programs, and/or services [9, 10, 55]. As discussed in the initial literature review [21], separate funding structures and service priorities between CAMHS and AMHS often limits the feasibility of collaboration between child and adult clinicians and programs throughout the transition process.

By the end of Round 2, six components were endorsed for importance by all three panels, and for feasibility by at least one panel. These core components (1.5, 1.6, 4.6, 4.8, 4.9 and 5.3) were focused on role clarity, partnering with youth, and communication of up-to-date information to youth and team members. Given that these components received some endorsement for feasibility (compared to the other core components, which did not receive sufficient endorsement for feasibility by any panels), there may be fewer barriers to their implementation in the current mental health care system. These components were also agreed upon by all panels to be important, indicating that they could represent ideal targets for initial intervention development and evaluation.

Youth have an essential role in contributing to the use of innovative research strategies (such as the Delphi methodology). In this project specifically, youth experts increased the validity of core components and had an active role in the interpretation of the results. In youth mental health research, patient engagement has been increasingly emphasized by funders and in academic literature [41, 44, 56], and recognized as an important pillar of research practice to ensure research success and relevance [42, 57]. Further, through our initial scoping review [21], we identified the need for concensus studies which engaged youth and families as partners in prioritizing components of successful transitions. As a result, this study utilized an integrated knowledge translation process [58] to actively engage youth as equal contributing project partners at all stages of the study. Youth partners led the application of engagement principles across the study, such as ongoing monitoring for youth-friendly language. These contributions ensured that the final list of core components used language that intentionally balanced specificity and flexibility, demonstrating the importance of partnering with youth to best reflect the lived expertise of all stakeholders in study results.

Future work is required to support the operationalization and implementation of these core components into clinical practice. An initial project would be to translate them into a transition checklist specifically created for the context of mental health services, as has been done in chronic or complex health inteventions and settings [59]. This checklist can then guide the design and development of new transitions interventions, the evaluation of existing transition interventions, and the identification of gaps in current interventions and services. This Delphi study has demonstrated the value and feasibility of youth and caregiver engagement. As such, this checklist ought to be co-created with youth and caregivers. Engaging youth and caregivers in the co-design and evaluation of mental health interventions can increase their relevance and impact for youth receiving the services [60], and youth voices should ideally be included at all levels of mental health service provision [61]. Future projects utilizing these components to design, implement, and evaluate transition interventions should continue to engage youth and caregivers to ensure that interventions are appropriate and meaningful for their intended recipients.

Limitations

While this was a national Canadian study, the panel members were overwhelmingly from Ontario (the most densely populated province in the country), despite multiple attempts to increase recruitment from other provinces and territories. While Canada has a universal public health care system, health care services are administered by provincial and territorial governments. As such variation between provinces in mental health service delivery [62] means the application of these core components in different regions may be limited.

Additonally, the demographic make up of the the youth panel members lacked diversity in terms of gender, race, and diagnosis. For example, while the youth panel reported diversity in cultural groups most identified with, participants overwhelmingly identified as female. Future work should strive for diverse panel members, particularly for sex, gender, race, diagnosis, and primary language, as these youth may have different experiences with transitions in care. Additionally, the use of national list servs for recruitment, while a common method for recruitment in mental health delphi studies [63], may have resulted in less diverse panel membership. Given we have no detail of the demographic make up of list serv members, we have no way of knowing whether the panel members we recruited represented the diversity of the members. This method of recruitment also precludes our ability to determine the response rate of those who received the initial information email to participate versus those who consented to participated. However, the response rate of those who contacted the research team via the informational email versus those who consented was determined and reported in the results.

Another limitation may result from our caregiver panel size. Panel sample sizes of approximately 20 members have been shown to produce stable findings in Delphi studies [32].The caregiver panel was smaller than this at 17 participants and this group experienced a greater level of attrition than the other panels between the two rounds. Further, as part of maintaining participants’ confidentiality, their last names were not requested on the survey, leading to the possibility that there were caregiver-youth dyads participating in the survey. Inclusion of dyads in the survey responses may have limited the diversity of the panels and could have the potential to bias the results based on specific shared experiences within these dyads.

Finally, this study resulted in a very low level of concensus for feasibility of the core components. Previous work using the Delphi methodology has found that panelists report that assessment of ‘feasibility’ can be challenging [64], which, along with the low levels of endorsement found in the current study, indicates that this may be a concept that is not sufficiently concrete or specific to ask experts to rate in future Delphi studies.

Conclusion

This Delphi study applied the principles of patient-oriented research to refine a list of core components for successful transitions between child and adult mental health services. Authentic engagement of youth and families as equal experts alongside healthcare professionals has resulted in a list of core components which reflect the identified needs and lived experience of service users as well as providers. These 26 core components should be used in the design and evaluation of programs or interventions to improve transitions from CAMHS to AMHS. Ultimately, the development and evaluation of transition interventions are needed to promote safe and seamless transitions and improve outcomes for youth requiring ongoing mental health care.

Availability of data and material

The data are available from the authors upon reasonable request—subject to ethical permissions and participant consent.

References

  1. 1.

    Offord D, Boyle M, Szatmari P et al (1987) Ontario Child Health Study II: Six-month prevalence of disorder and rates of service utilization. Arch Gen Psychiatry 44:832–6

  2. 2.

    Ontario Ministry of Child and Youth Service (2006) A shared responsibility: Ontario’s policy framework for child and youth mental health. http://www.children.gov.on.ca/htdocs/English/documents/specialneeds/mentalhealth/framework.pdf

  3. 3.

    Kessler RC, Chiu WT, Demler O et al (2005) Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry 62:617–627. https://doi.org/10.1001/archpsyc.62.6.617

    Article  PubMed  PubMed Central  Google Scholar 

  4. 4.

    Wiens K, Bhattarai A, Pedram P et al (2020) A growing need for youth mental health services in Canada: examining trends in youth mental health from 2011 to 2018. Epidemiol Psychiatr Sci. https://doi.org/10.1017/S2045796020000281

    Article  PubMed  PubMed Central  Google Scholar 

  5. 5.

    Davidson S, Cappelli M (2011) We’ve got growing up to do: Transitioning youth from child and adolescent mental health services to adult mental health services. Ontario Centre of Excellence for Child and Youth Mental Health, Ottawa

  6. 6.

    Singh SP, Evans N, Sireling L, Stuart H (2005) Mind the gap: the interface between child and adult mental health services. Psychiatr bull 29:292–294. https://doi.org/10.1192/pb.29.8.292

    Article  Google Scholar 

  7. 7.

    Wilson M (2016) Health Accord must address growing mental health crisis in Canada. The Globe and Mail

  8. 8.

    Canadian Intergovernmental Conference Secretariat (2016) Provincial and territorial health ministers focus on strengthening health care for all canadians. https://scics.ca/en/product-produit/news-release-provincial-and-territorial-health-ministers-focus-on-strengthening-health-care-for-all-canadians/

  9. 9.

    Mental Health Commission of Canada (2015) Taking the next step forward: building a responsive mental health and addictions system for emerging adults. https://www.mentalhealthcommission.ca/English/issues/child-and-youth/taking-next-step-forward

  10. 10.

    Signorini G, Singh SP, Marsanic VB et al (2018) The interface between child/adolescent and adult mental health services: results from a European 28-country survey. Eur Child Adolesc Psychiatry 27:501–511. https://doi.org/10.1007/s00787-018-1112-5

    Article  PubMed  Google Scholar 

  11. 11.

    Singh SP, Paul M, Ford T et al (2010) Process, outcome and experience of transition from child to adult mental healthcare: multiperspective study. Br J Psychiatry 197:305–312. https://doi.org/10.1192/bjp.bp.109.075135

    Article  PubMed  Google Scholar 

  12. 12.

    Islam Z, Ford T, Kramer T et al (2016) Mind how you cross the gap! Outcomes for young people who failed to make the transition from child to adult services: the TRACK study. BJPsych Bull 40:142–148. https://doi.org/10.1192/pb.bp.115.050690

    Article  PubMed  PubMed Central  Google Scholar 

  13. 13.

    Hofstra MB, Van der Ende J, Verhulst FC (2000) Continuity and change of psychopathology from childhood into adulthood: a 14-year follow-up study. J Am Acad Child Adolesc Psychiatry 39:850–858. https://doi.org/10.1097/00004583-200007000-00013

    CAS  Article  PubMed  Google Scholar 

  14. 14.

    Paul M, Ford T, Kramer T et al (2013) Transfers and transitions between child and adult mental health services. Br J Psychiatry Suppl 54:s36-40. https://doi.org/10.1192/bjp.bp.112.119198

    Article  PubMed  Google Scholar 

  15. 15.

    Blum RW, Garell D, Hodgman CH et al (1993) Transition from child-centered to adult health-care systems for adolescents with chronic conditions. A position paper of the Society for Adolescent Medicine. J Adolesc Health 14:570–576. https://doi.org/10.1016/1054-139x(93)90143-d

    CAS  Article  PubMed  Google Scholar 

  16. 16.

    Reid R, Haggerty J, McKendry R (2002) Defusing the confusion: concepts and measures of continuity of healthcare. Canadian Health Services Research Foundation, Ottawa, Ontario

  17. 17.

    Hovish K, Weaver T, Islam Z et al (2012) Transition experiences of mental health service users, parents, and professionals in the United Kingdom: a qualitative study. Psychiatr Rehabil J 35:251–257. https://doi.org/10.2975/35.3.2012.251.257

    Article  PubMed  Google Scholar 

  18. 18.

    Burnham Riosa P, Preyde M, Porto ML (2015) Transitioning to adult mental health services: perceptions of adolescents with emotional and behavioral problems. J Adolesc Res 30:446–476. https://doi.org/10.1177/0743558415569730

    Article  Google Scholar 

  19. 19.

    Pottick KJ, Bilder S, Vander Stoep A et al (2008) US patterns of mental health service utilization for transition-age youth and young adults. J Behav Health Serv Res 35:373–389. https://doi.org/10.1007/s11414-007-9080-4

    Article  PubMed  Google Scholar 

  20. 20.

    Di Rezze B, Nguyen T, Mulvale G et al (2016) A scoping review of evaluated interventions addressing developmental transitions for youth with mental health disorders. Child Care Health Dev 42:176–187. https://doi.org/10.1111/cch.12306

    Article  PubMed  Google Scholar 

  21. 21.

    Cleverley K, Rowland E, Bennett K et al (2020) Identifying core components and indicators of successful transitions from child to adult mental health services: a scoping review. Eur Child Adolesc Psychiatry 29:107–121. https://doi.org/10.1007/s00787-018-1213-1

    Article  PubMed  Google Scholar 

  22. 22.

    Sobota AE, Shah N, Mack JW (2017) Development of quality indicators for transition from pediatric to adult care in sickle cell disease: a modified Delphi survey of adult providers: Soasdbota easdt aasdl. Pediatr Blood Cancer 64:e26374. https://doi.org/10.1002/pbc.26374

    Article  Google Scholar 

  23. 23.

    Suris J-C, Akre C (2015) Key elements for, and indicators of, a successful transition: an international Delphi study. J Adolesc Health 56:612–618. https://doi.org/10.1016/j.jadohealth.2015.02.007

    Article  PubMed  Google Scholar 

  24. 24.

    Akre C, Suris J-C, Belot A et al (2018) Building a transitional care checklist in rheumatology: a Delphi-like survey. Jt Bone Spine 85:435–440. https://doi.org/10.1016/j.jbspin.2017.09.003

    Article  Google Scholar 

  25. 25.

    Sun H, Breakey VR, Straatman L et al (2019) Outcomes, indicators, and processes in transitional care in adolescents with haemophilia: a Delphi survey of Canadian haemophilia care providers. Haemophilia 25:296–305. https://doi.org/10.1111/hae.13699

    Article  PubMed  Google Scholar 

  26. 26.

    Broad KL, Sandhu VK, Sunderji N, Charach A (2017) Youth experiences of transition from child mental health services to adult mental health services: a qualitative thematic synthesis. BMC Psychiatry 17:380. https://doi.org/10.1186/s12888-017-1538-1

    Article  PubMed  PubMed Central  Google Scholar 

  27. 27.

    CMHO Youth Action Committee (2018) From crisis to quality: bridging gaps in child and youth mental health services. Toronto, Ontario. http://www.thenewmentality.ca/crisistoquality/

  28. 28.

    Cleverley K, Lenters L, McCann E (2020) “Objectively terrifying”: a qualitative study of youth’s experiences of transitions out of child and adolescent mental health services at age 18. BMC Psychiatry 20:147. https://doi.org/10.1186/s12888-020-02516-0

    Article  PubMed  PubMed Central  Google Scholar 

  29. 29.

    McKenna HP (1994) The Delphi technique: a worthwhile research approach for nursing? J Adv Nurs 19:1221–1225. https://doi.org/10.1111/j.1365-2648.1994.tb01207.x

    CAS  Article  PubMed  Google Scholar 

  30. 30.

    Powell C (2003) The Delphi technique: myths and realities. J Adv Nurs 41:376–382. https://doi.org/10.1046/j.1365-2648.2003.02537.x

    Article  PubMed  Google Scholar 

  31. 31.

    Fletcher-Johnston M, Marshall SK, Straatman L (2011) Healthcare transitions for adolescents with chronic life-threatening conditions using a Delphi method to identify research priorities for clinicians and academics in Canada. Child Care Health Dev 37:875–882. https://doi.org/10.1111/j.1365-2214.2011.01318.x

    CAS  Article  PubMed  Google Scholar 

  32. 32.

    Jorm AF (2015) Using the Delphi expert consensus method in mental health research. Aust N Z J Psychiatry 49:887–897. https://doi.org/10.1177/0004867415600891

    Article  PubMed  Google Scholar 

  33. 33.

    Boulkedid R, Abdoul H, Loustau M et al (2011) Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS ONE 6:e20476. https://doi.org/10.1371/journal.pone.0020476

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  34. 34.

    Owens C, Ley A, Aitken P (2008) Do different stakeholder groups share mental health research priorities? A four-arm Delphi study. Health Expect 11:418–431. https://doi.org/10.1111/j.1369-7625.2008.00492.x

    Article  PubMed  PubMed Central  Google Scholar 

  35. 35.

    Guise J-M, O’Haire C, McPheeters M et al (2013) A practice-based tool for engaging stakeholders in future research: a synthesis of current practices. J Clin Epidemiol 66:666–674. https://doi.org/10.1016/j.jclinepi.2012.12.010

    Article  PubMed  Google Scholar 

  36. 36.

    Bovopoulos N, Jorm AF, Bond KS et al (2016) Providing mental health first aid in the workplace: a Delphi consensus study. BMC Psychol 4:41. https://doi.org/10.1186/s40359-016-0148-x

    Article  PubMed  PubMed Central  Google Scholar 

  37. 37.

    Shoemaker ES, Kendall CE, Mathew C et al (2020) Establishing need and population priorities to improve the health of homeless and vulnerably housed women, youth, and men: a Delphi consensus study. PLoS ONE 15:e0231758. https://doi.org/10.1371/journal.pone.0231758

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  38. 38.

    Hill NTM, Shand F, Torok M et al (2019) Development of best practice guidelines for suicide-related crisis response and aftercare in the emergency department or other acute settings: a Delphi expert consensus study. BMC Psychiatry 19:6. https://doi.org/10.1186/s12888-018-1995-1

    Article  PubMed  PubMed Central  Google Scholar 

  39. 39.

    Robinson J, Hill NTM, Thorn P et al (2018) The #chatsafe project. Developing guidelines to help young people communicate safely about suicide on social media: a Delphi study. PLoS One 13:e0206584. https://doi.org/10.1371/journal.pone.0206584

  40. 40.

    Kingston AH, Morgan AJ, Jorm AF et al (2011) Helping someone with problem drug use: a delphi consensus study of consumers, carers, and clinicians. BMC Psychiatry 11:3. https://doi.org/10.1186/1471-244X-11-3

    Article  PubMed  PubMed Central  Google Scholar 

  41. 41.

    Canadian Institute of Health Research (2014) Strategy for patient-oriented research putting patients first. https://cihr-irsc.gc.ca/e/documents/spor_framework-en.pdf

  42. 42.

    Ennis L, Wykes T (2013) Impact of patient involvement in mental health research: longitudinal study. Br J Psychiatry 203:381–386. https://doi.org/10.1192/bjp.bp.112.119818

    Article  PubMed  Google Scholar 

  43. 43.

    Powers JL, Tiffany JS (2006) Engaging youth in participatory research and evaluation. J Public Health Manag Pract 12:S79–S87. https://doi.org/10.1097/00124784-200611001-00015

    Article  Google Scholar 

  44. 44.

    Banner D, Bains M, Carroll S et al (2019) Patient and public engagement in integrated knowledge translation research: are we there yet? Res Involv Engagem 5:8. https://doi.org/10.1186/s40900-019-0139-1

    Article  PubMed  PubMed Central  Google Scholar 

  45. 45.

    Canadian Institute of Health Research (2016) Knowledge User Engagement. In: Knowledge translation. https://cihr-irsc.gc.ca/e/49505.html. Accessed 21 Jul 2020

  46. 46.

    Cleverley Lab (2018) CIHR Delphi Study Information Video. https://www.youtube.com/watch?v=h-84DChcDI0&feature=emb_title

  47. 47.

    Boulkedid R, Sibony O, Goffinet F et al (2013) Quality indicators for continuous monitoring to improve maternal and infant health in maternity departments: a modified Delphi survey of an international multidisciplinary panel. PLoS ONE 8:e60663. https://doi.org/10.1371/journal.pone.0060663

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  48. 48.

    Hermann RC, Palmer H, Leff S et al (2004) Achieving consensus across diverse stakeholders on quality measures for mental healthcare. Med Care 42:1246–1253. https://doi.org/10.1097/00005650-200412000-00012

    Article  PubMed  Google Scholar 

  49. 49.

    Kleynen M, Braun SM, Bleijlevens MH et al (2014) Using a Delphi technique to seek consensus regarding definitions, descriptions and classification of terms related to implicit and explicit forms of motor learning. PLoS ONE 9:e100227. https://doi.org/10.1371/journal.pone.0100227

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  50. 50.

    Harris PA, Taylor R, Thielke R et al (2009) Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42:377–381. https://doi.org/10.1016/j.jbi.2008.08.010

    Article  Google Scholar 

  51. 51.

    Corp IBM (2017) IBM SPSS Statistics for Windows. Armonk, New York

    Google Scholar 

  52. 52.

    Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3:77–101. https://doi.org/10.1191/1478088706qp063oa

    Article  Google Scholar 

  53. 53.

    Le Roux E, Mellerio H, Jacquin P et al (2019) Practical generic guidelines for paediatric-to-adult transition for adolescents with chronic disease. Eur J Pub Health 29:442–448. https://doi.org/10.1093/eurpub/cky258

    Article  Google Scholar 

  54. 54.

    van den Brink G, van Gaalen MAC, de Ridder L et al (2019) Health care transition outcomes in inflammatory bowel disease: a multinational Delphi study. J Crohn’s Colitis 13:1163–1172. https://doi.org/10.1093/ecco-jcc/jjz044

    Article  Google Scholar 

  55. 55.

    Singh SP (2009) Transition of care from child to adult mental health services: the great divide. Curr Opin Psychiatry 22:386–390. https://doi.org/10.1097/YCO.0b013e32832c9221

    Article  PubMed  Google Scholar 

  56. 56.

    Ontario Centre of Excellence in Child and Youth Mental Health (2016) Evidence in-sight: best practices in engaging families in child and youth mental health. Ottawa, Ontario. https://www.cymh.ca/en/

  57. 57.

    Bell E (2015) Young persons in research: a call for the engagement of youth in mental health research. Am J Bioeth 15:28–30. https://doi.org/10.1080/15265161.2015.1088977

    Article  PubMed  Google Scholar 

  58. 58.

    Canadian Institute of Health Research (2012) Guide to knowledge translation planning at CIHR: integrated and end-of-grant approaches. Ottawa, Ontario. https://cihr-irsc.gc.ca/e/45321.html

  59. 59.

    McManus M, White P, Pirtle R et al (2015) Incorporating the six core elements of health care transition into a medicaid managed care plan: lessons learned from a pilot project. J Pediatr Nurs 30:700–713. https://doi.org/10.1016/j.pedn.2015.05.029

    Article  PubMed  Google Scholar 

  60. 60.

    Jenkins EK, Bungay V, Patterson A et al (2018) Assessing the impacts and outcomes of youth driven mental health promotion: a mixed-methods assessment of the Social Networking Action for Resilience study. J Adolesc 67:1–11. https://doi.org/10.1016/j.adolescence.2018.05.009

    Article  PubMed  Google Scholar 

  61. 61.

    Hawke LD, Mehra K, Settipani C et al (2019) What makes mental health and substance use services youth friendly? A scoping review of literature. BMC Health Serv Res. https://doi.org/10.1186/s12913-019-4066-5

    Article  PubMed  PubMed Central  Google Scholar 

  62. 62.

    Butler A, Adair C, Jones W et al (2017) Towards quality mental health services in Canada: a comparison of performance indicators across 5 Provinces. Centre for Applied Research in Mental Health & Addiction, Vancouver BC. https://www.sfu.ca/carmha/publications/prov_indic_2017.html

  63. 63.

    Hart LM, Wade T (2020) Identifying research priorities in eating disorders: a Delphi study building consensus across clinicians, researchers, consumers, and carers in Australia. Int J Eat Disord 53:31–40. https://doi.org/10.1002/eat.23172

    Article  PubMed  Google Scholar 

  64. 64.

    Morgan AJ, Jorm AF (2009) Self-help strategies that are helpful for sub-threshold depression: A Delphi consensus study. J Affect Disord 115:196–200. https://doi.org/10.1016/j.jad.2008.08.004

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Kristin Cleverley held the CAMH Chair in Mental Health Nursing Research at the University of Toronto during the completion of this study. We thank the Margaret and Wallace McCain Centre for Child, Youth and Family Mental Health Youth Engagement Facilitators for their contributions to this study.

Funding

This research was funded by a grant from the Canadian Institute of Health Research (FRN 151744). The research team is independent of the funding agencies and the funding agencies were not involved in the design, interpretation or writing of the manuscript.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Kristin Cleverley.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Ethics approval

Ethical approval was obtained from the Centre for Addiction and Mental Health (119/2016) and the University of Toronto (32982).

Consent to participate

All participants in this study provided electronic informed consent via REDCap in accordance with procedures approved by the ethics committees. Parental consent for youth participants was not required from any of the ethics committees because all participants enrolled in the study were over the age of 19.

Consent for publication

Include appropriate statements.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Cleverley, K., McCann, E., O’Brien, D. et al. Prioritizing core components of successful transitions from child to adult mental health care: a national Delphi survey with youth, caregivers, and health professionals. Eur Child Adolesc Psychiatry (2021). https://doi.org/10.1007/s00787-021-01806-6

Download citation

Keywords

  • Transition
  • Continuity of care
  • Mental health
  • Youth
  • Delphi study
  • Consensus
  • Patient-oriented research