FormalPara Clinician’s capsule

What is known about the topic?

While certain aspects of ED care are associated with satisfaction, no evaluations specific to children/adolescents with mental health concerns have been performed.

What did this study ask?

How satisfied are families/children with ED mental health care delivery, and which aspects are associated with satisfaction scores.

What did this study find?

We need to improve ED mental health care delivery, focusing on enhancing access to care by mental health providers.

Why does this study matter to clinicians?

To improve patient and family satisfaction, ED and mental health administrators need to adapt models of ED mental health care delivery that include streamlined access to mental health care practitioners and facilitate access to community-based mental health supports for ongoing care.

Background

Over the past decade, visits to emergency departments (EDs) by children and adolescents for mental health care have increased [1, 2], a trend exacerbated by the COVID-19 pandemic [3]. Although the process of assessing suicidal ideation is well-studied, little attention has been paid to understanding and improving patient-reported experiences.

Satisfaction is an important measure of patient experience and is a good indicator of service quality, future service usage and continuity of care adherence [4,5,6]. Knowledge of satisfaction can provide insight regarding expectations for care [7] and how services can be improved. Although certain aspects of pediatric ED care are known to be associated with satisfaction (e.g., interpersonal interactions, communication, provider skills, wait times) [8], no evaluations specific to ED mental health care delivery have been performed. Thus, we sought to assess child/adolescent and parent/caregiver satisfaction with ED mental health care, and to determine which aspects of ED care receive the highest and lowest satisfaction scores. Based on what is known about satisfaction with ED care, we hypothesized that positive associations would be identified between general satisfaction and perceived wait time [8,9,10,11] and ED provider courtesy and compassion [8, 10, 12].

Methods

Study design and setting

This cross-sectional study was embedded in a prospective implementation study conducted in two tertiary care pediatric EDs (Stollery Children’s Hospital and the Alberta Children’s Hospital) in Alberta, Canada [13]. Study outcome measures and processes were determined in collaboration with patient and parent partners. Data were collected between February 1, 2020, and January 31, 2021, and reflect care (Fig. 1) prior to implementation of a new clinical care pathway. Using a complete case ascertainment sampling strategy, all patients meeting the eligibility criteria were approached either in-person or via telephone after the ED visit. Research Ethics Board approval was obtained, and the caregivers of eligible participants provided consent; assent was obtained when appropriate. Adolescents ≥ 14 years of age who presented without a legal guardian participated as mature minors. Results are reported in accordance with the STROBE guidelines [14].

Fig. 1
figure 1

Standard of care for mental health patients in the participating emergency departments. ED, Emergency Department; RN, Registered Nurse. *Mental health team member can include a mental health nurse, counsellor, or psychiatrist. δ Discharge resources can include nothing (e.g., follow-up with family physician or existing mental health provider), provision of pamphlets with options to family to coordinate follow-up, provision of mental health care coordination phone number, referral to a specific program, and/or follow-up by a hospital-based mental health, outreach home care team

Full size image

Population

Eligible participants were < 18 years old and presented with any of the following Canadian Emergency Department Information System [15] presenting complaints documented at triage: anxiety, bizarre behaviour, concern for patient’s welfare, depression/suicidal, disruptive behaviour, homicidal behaviour, insomnia, self-harm, situational crisis, or violent behaviour. Those with acute medical and/or physical safety concerns were ineligible including children brought by protective/police services or ambulance, chief complaints relating to schizophrenia/psychosis, behavioural syndromes requiring medical clearance, or significant self-harm. Children who had previously participated were also ineligible.

Outcome measures

The primary outcome was total satisfaction with ED mental health care delivery quantified by the Service Satisfaction Scale (SSS-10) [16, 17]; Online Resource 1. The scale is comprised of 10 items (child/adolescent version) or 12 items (parent/caregiver version) that utilize a 5-point Likert-scale plus three open-ended questions that elicit opinions on what worked well during care, and what should be changed [16]. SSS-10 items are divided into two subscales: manner and skills of the staff (5 items) and perceived outcomes (5 items for child/adolescent version; 7 items for parent/caregiver version). Total scale scores range from 10 to 50 (child/adolescent version) or 12 to 60 (parent/caregiver version), with higher scores indicating greater satisfaction [16]. Secondary outcomes were SSS-10 item associations with the SSS-10 general satisfaction question, patient and ED visit characteristic associations with the total satisfaction score (i.e., sum of SSS-10 elements), and experiences with ED mental health care.

Data collection

Data were collected as soon as possible following the ED visit, as parent partners advised that to minimize stress, research recruitment would ideally be performed following the ED visit. Care experiences, satisfaction, and demographic data were collected via a questionnaire completed by telephone or online. Telephone-based questionnaires followed a standardized process led by a trained research assistant. Online data collection occurred within the study’s REDCap database. ED visit characteristic data were collected via medical record review. Data collected included International Classification of Diseases, Version 10, Canada (ICD-10-CA) discharge diagnoses code assigned based on chief complaint and physician notes: F codes for mental and behavioural disorders and R and X codes for intentional self-harm that did not require medical care. Length of stay (LOS) was defined as the time the child/adolescent was in the ED from triage to discharge; triage time was defined as the time of ED triage.

Data analysis

The total SSS-10 score, representing an individual’s satisfaction with care, was calculated by summing the individual item scores. Individual SSS-10 item scores were used to identify aspects of ED mental health care respondents were most and least satisfied with. We evaluated associations of general satisfaction with ED mental health care using Pearson’s correlation coefficient, associations between total satisfaction score with patient and ED visit characteristics using multivariable regression analyses, and compared satisfaction scores between the two participating EDs using student’s t test (Online Resource 2).

An inductive thematic analysis was conducted using the open-ended SSS-10 question responses to identify positive and negative care experience themes. Thematic coding was performed by one author to identify and categorize excerpts to find emerging themes and patterns. To ensure consistency and accuracy, two independent reviewers reviewed the codes and assigned themes to 50 randomly selected participants. Themes are reported using frequency (frequency of each theme divided by the total number of responses) and intensity (proportion of codes describing a particular theme divided by the total number of codes) effect sizes [18]. Ratios of positive to negative feedback were calculated to permit interpretation of the relationship between theme frequency and intensity and to determine which themes were most strongly associated with dissatisfaction and satisfaction.

Analyses were conducted using R software (Version 1.14.4, Vienna, Austria). Statistical tests were two-tailed and P values of < 0.05 were considered statistically significant.

Results

Study participants

Of 970 potentially eligible children and adolescents, 73.0% consented and 66.6% of those that consented provided data for analysis; Fig. 2. Sixty-five percent of participants received care at the Alberta Children’s Hospital. The median time to data collection was 14 days (IQR 8–22). Participants were predominantly female (56.3%) and Caucasian (71.2%), with a median age of 13 years (IQR 11–15 years); Table 1. Anxiety and stress-related disorders (39.5%), suicidal ideation (26.0%), and mood disorders (25.0%) were the most common discharge diagnoses.

Fig. 2
figure 2

Overall study screening and enrollment

Full size image
Table 1 Participant and ED visit characteristics
Full size table

Satisfaction with ED mental health care

Parents/caregivers and adolescents were most satisfied with confidentiality and respect for their child’s rights; Table 2, Online Resource 3. Parents/caregivers were least satisfied with how ED services helped reduce their child’s symptoms/problems (mean 3.0, SD 1.2) and how ED services helped their child get well and stay well (mean 3.1, SD 1.2). Adolescents were least satisfied with how ED services helped reduce their symptoms/problems (mean 3.0, SD 1.0). Aspects of care satisfaction differed between sites; Table 2. Greater satisfaction with care was reported for parents/caregivers whose child received care at Alberta Children’s Hospital (mean 42.4, SD 9.8, p = 0.003).

Table 2 Parent/caregiver satisfaction with ED mental health care (means with standard deviations [SD])
Full size table

Correlations with satisfaction

The amount of help a child/adolescent received had the strongest, positive association with general satisfaction (r = 0.85, 95% CI 0.83–0.87), with the weakest association for how long the child/adolescent had to wait (r = 0.31, 95% CI 0.23–0.38); Online Resource 4. Receipt of an evaluation by a mental health team member (OR = 15.39, 95% CI 5.40–43.90, p = 0.004) and psychiatry consultation (OR = 8.97, 95% CI 3.15–25.58, p = 0.05) were positively associated with the total satisfaction score; Online Resource 5. Self-identification as Asian (OR = 90.83, 95% CI 31.85–259.00, p = 0.01) and ‘other’ ethnicities (OR = 51.26, 95% CI 17.98–146.18, p = 0.04) was positively associated with total satisfaction score compared to Caucasian participants; self-identification as being of mixed ethnicity (OR = 0.04, 95% CI 0.01–0.10, p = 0.04) was negatively associated with total satisfaction score.

Themes for ED mental health care

Qualitative feedback was provided by 57.0% of participants, with respondents representing older children than non-respondents; Online Resource 6. Twelve themes were created, and the greatest amount of positive feedback (frequency and intensity) pertained to ED provider attitude and interpersonal skills; Table 3. Expectations regarding standards of care received the most negative feedback followed by wait times and access to mental health and addictions specialists. The latter theme also had the greatest negative to positive ratio in terms of frequency (66:1) and intensity (81:1), followed by the wait time and COVID-19 themes; Table 3. Online Resource 7 provides information on themes and their definitions.

Table 3 Satisfaction theme frequency and intensity effect sizes, n (%)
Full size table

Discussion

Interpretation

In this study, which explored satisfaction with care delivered to children presenting for ED care with acute mental health concerns, study participants were most satisfied with confidentiality and respect in the ED and least satisfied with how ED services helped reduce the symptoms and/or problems that led them to seek ED care. Satisfaction was associated with perceived amount of help received in the ED, evaluation by a mental health team member, and psychiatrist consultation. Comments provided by participants demonstrated satisfaction with ED provider attitudes and interpersonal skills and dissatisfaction with access to mental health and addictions care, wait time, and the impact of COVID-19. Our findings indicate that there is a need to improve ED mental health care delivery, with a focus on enabling timely access to ED mental health providers. Access to outpatient/community-based mental health care is also needed to complement care received in the ED and to provide continuity of care for youth with mental health concerns.

Comparison to previous studies

Previous pediatric studies examining satisfaction with general ED care, utilizing satisfaction measures other than the SSS-10, report that positive ED provider attitude, interpersonal skills, and high-quality provider interactions are important to patients [8, 10, 19], and are positively associated with increased parent/caregiver satisfaction with ED care [10, 12]. However, in our study, how ED services help reduce symptoms and how they help the child/adolescent get and stay well, received the lowest satisfaction scores. This may reflect the inability of EDs to provide high-quality mental health care due to inadequate protocols, lack of standardized tools, limited ED provider training, and a shortage of mental health specialists [20,21,22].

Our findings that access to mental health and addictions care had the greatest ratio of negative to positive comments aligns with the adult ED literature. While a Canadian ED study of adults with mental health concerns reported similar reasons for dissatisfaction [23], an Australian adult study described receipt of a mental health team member and psychiatry consultation as being positively associated with satisfaction [24]. Since access to consultation by a mental health team member or psychiatrist is not a standard component of pediatric ED mental health care in Canada due to the lack of resources in most institutions, with the ED providers limiting access to those in greatest need (Fig. 1), patient expectations are often not met. Moreover, as comprehensive mental health care is not universally accessible under Canada's healthcare plan, and mental health services for children are under-resourced and under-funded [25], the limited availability of outpatient and community-based mental health care [26, 27] continues to place greater pressure on the ED to deliver mental health care.

Strengths and limitations

A novel finding in our study was the positive association between being Asian or of another non-White ethnicity with satisfaction, while mixed ethnicity was negatively associated. Prior non-mental health specific studies suggest that ethnicity does play a role in patient satisfaction and perceptions of ED care [28, 29]. In Canada, First Nations patients often report negative ED experiences, and they may be exposed to discrimination and racism in the ED [30]. Mental health specific research from the United States has highlighted important racial and ethnic disparities in pediatric ED presentations for mental health concerns [31]. To the best of our knowledge, only one Canadian ED mental health study has focused on race and ethnicity; it identified that First Nations adolescents are more likely to present to the ED for mental health concerns. However, this study did not examine whether race or ethnicity was associated with ED services provided or satisfaction [32].

As our consent rate was only 73.0%, the results may be subject to non-response and volunteer biases, and thus possibly underrepresenting the perspective of those with negative ED interactions. Limiting survey completion to English speaking participants resulted in the omission of perspectives from non-English speaking participants. We were not permitted to document reasons for, or characteristics of patients electing not to participate and thus cannot compare participants to non-participants or provide reasons for non-participation. Further, we lacked representation of children in foster care or group homes where consent from legal guardians could not be obtained. Although we attempted to capture the perspectives of adolescents, as only 40 mature minors participated, we cannot confirm if adolescent perspectives differ from those of parents/caregivers. Our eligibility criteria prevented the inclusion of children with certain mental health presentations which limits the generalizability of our findings to populations such as those with psychosis or self-harm requiring medical care [13]. Due to the multiple steps involved in obtaining consent to contact, consent and ultimately survey completion, many surveys were completed outside of our target survey completion window. In addition, the SSS-10 is not specifically designed for ED use and the items do not account for the dynamic processes and multiple care providers that are core components of ED care.

Clinical implications

This study demonstrates that while parents/caregivers and children and adolescents with mental health concerns are satisfied with ED providers and that satisfaction with ED care is associated with receipt of a mental health team member or psychiatrist consultation, they are less satisfied with how mental health services helped address their child's concerns and/or symptoms. Our findings should be used to inform ED mental health care delivery models which need to focus on enhancing the provision of timely access to pediatric mental health specialists. Funding and resources are needed to improve connections to outpatient and community-based mental health supports to enable the early identification, management, and prevention of mental health concerns.

Research implications

Future research initiatives targeting the implementation of novel models of care, the monitoring and setting quality benchmarks, and an evaluation of the impact of implementing standardized mental health tools in the ED, are needed. Prospective studies should compare satisfaction between parent–child dyads. Moreover, an evaluation of approaches to connect all youth, without an existing mental health care provider relationship, to a post-ED visit mental health care visit would likely have an impact on satisfaction.

Conclusion

While parents/caregivers and children and adolescents were satisfied with ED providers, satisfaction with ED care was associated with receiving a mental health team member or psychiatrist consultation. In addition, they were less satisfied with how mental health services in helped reduce their child’s mental health concerns and/or symptoms. This may reflect the challenge of providing adequate mental health care due to limited resources in the ED and the community settings. This knowledge should inform ED mental health care delivery models with a focus on providing improved and timely access to ED pediatric mental health specialists and connections to outpatient resources to ultimately improve outcomes for children with mental health concerns.