FormalPara Key Points for Decision Makers

Understanding the aspects that patients and professionals view as important when assessing quality in mental healthcare is essential in ensuring quality evaluations that are fit for purpose.

This study identifies aspects that mental health service users and professionals deem important for assessing mental healthcare quality and outlines similarities and differences between service user and professionals’ views.

A comprehensive six-theme framework for understanding quality in mental healthcare from the viewpoint of the service user and frontline mental health professional in the UK is presented, featuring a high degree of concordance between service users and professionals.

1 Introduction

Providing access to good quality mental healthcare is a global priority [1]. Evaluating the quality of mental healthcare is important for service improvement and for optimising the patient (service user [SU]), experience. However, the criteria on which to evaluate “quality” in mental healthcare is not universally recognised nor readily apparent. There are many different aspects that could be used as indicators of quality, including those typically prioritised by policymakers, such as quantifying speed of access and readmissions [2, 3], yet what is considered meaningful may differ across stakeholders, including SUs and mental healthcare professionals (MHPs). There are gaps in evidence supporting quality assessment in mental healthcare globally, particularly in identifying and developing outcomes that are meaningful to SUs and that incorporate their views and treatment decisions [4].

In the UK National Health Service (NHS), recent mental healthcare policy has focused on improving access to psychological therapies [5], and adopting standardised quantitative measures of service quality in areas such as access, effectiveness, safety, experience and efficiency [2, 3]. However, there is a risk that if policymakers adopt a top-down, target-driven approach, policy-defined quality indicators and targets may not align with the perspectives of SUs and MHPs on how quality should be measured. There is value, therefore, in exploring how SUs and MHPs appraise current quality metrics and what, if any, other aspects they think should be incorporated in an accurate evaluation of service quality. The added value in exploring the perspectives of SUs and MHPs to determine quality indicators is three-fold. First, identifying quality indicators in this way increases their content validity, by ensuring they are relevant, understandable, and inclusive of what matters most to SUs and people working with them (i.e. MHPs). Second, it helps to reinforce SU empowerment, by ensuring that they have a say in topics that affect them. Third, it enables any convergence and divergence between SU and MHP’s views to be identified, which is of use to informing further service provision and evaluation. Previous research with SUs on their experiences of mental healthcare have identified elements of care that may matter to people receiving services, including the quality of the relationship between the SU and provider [6,7,8,9]; having a safe healthcare environment [7, 9]; and autonomy over treatment decisions [10]. Fewer studies with MHPs have also indicated areas that may matter from a professional’s perspective in appraising service quality, including the therapeutic alliance between SUs and MHPs [11, 12]; sufficient time allocated in therapy [13]; and smooth transitions between hospital and community care [14]. It is not clear from this work alone, however, whether SUs and MHPs think that these, or other, aspects of care should form the basis of factors incorporated in a quality assessment, what relative importance they place on each aspect, and whether there is concordance between SU and MHP’s views.

To the authors’ knowledge, no high-quality work eliciting preferences for different attributes of quality in mental healthcare in the UK has been published with a recognised decision-making methodology, such as a discrete choice experiment (DCE) [15]. As described elsewhere [16,17,18,19], a necessary first stage for a DCE is detailed qualitative research to inform the (in this case, quality) attributes that are important for inclusion. Furthermore, understanding what aspects SUs and MHPs think should be incorporated in a quality evaluation is likely to provide insight into potential areas of inconsistency and to be of use to decision makers overseeing quality evaluations in the future.

The study objectives were to: (1) identify attributes that SUs and MHPs deem important for accurately assessing quality in mental healthcare to help inform attribute selection in a subsequent DCE and (2) explore any similarities and differences between SU and MHP’s perspectives on the attributes identified. The findings informed attribute selection for a DCE eliciting preferences from SU, MHPs and the general population for aspects of mental healthcare quality [20].

2 Methods

This qualitative study involved face-to-face semi-structured focus groups with mental healthcare SUs and primarily frontline MHPs affiliated with a UK NHS Foundation Trust. This project was designed to understand the experiences and preferences of participants based on health economic theory (i.e. recognising multiple attributes contributing to service quality that involve trade-offs) [21]. Focus groups were selected as an appropriate qualitative method to support subsequent attribute selection [21], and because they facilitated exploring the group consensus on what aspects of healthcare quality were considered important. While the researchers view themselves as most aligned with a post-positivist paradigm, the framework qualitative method was followed, which involves an interplay of epistemological, methodological and pragmatic considerations [22].

2.1 Recruitment and Participants

The focus groups were advertised by the NHS Trust using e-mails and posters, with interested individuals contacting the research team. Participants were convenience sampled and the inclusion criteria was intentionally broad. Service users included adults (aged 18+ years) who had used mental healthcare. Mental health professionals included those who had worked in the delivery of mental health services at any grade. All participants were fluent in English. Service users received a £15 shopping voucher for taking part.

Six focus groups were conducted, three with SUs (n = 14; eight women, six men; two groups of five, and one group of four) and three with MHPs (n = 8; six men, two women; two groups of three, and one group of two). A range of mental health problems was represented. Mental health professionals included mental health nurses, recovery workers, occupational therapists and a quality improvement manager. Three to six focus groups are normally sufficient to reach adequate data saturation, with up to 90% of themes uncovered [23], and was considered sufficient for the scope of this project (i.e. to help inform attribute generation for a subsequent DCE) [21].

2.2 Procedure

Focus groups took place between April and September 2018 at the host research institution and were audio recorded. The groups were facilitated by the first author, an experienced male mixed-methods Research Fellow, with experience interviewing vulnerable people [24, 25], and were attended by a mental health nurse, as a source of support. The facilitator had no prior relationship with the participants. A topic guide (Electronic Supplementary Material [ESM]) was developed based on a review of NHS policy documents and available data used to measure quality in mental healthcare in England [2, 3, 5].

In each focus group, following written informed consent, the facilitator introduced himself, the research nurse, and explained their reasons for doing the research and the session aims. The researcher provided an example quality aspect from the topic guide (e.g. ease of access), which varied across groups and acted as an initial prompt. The discussion was participant led, with prompts as necessary to ensure all topics were covered. During discussions, a visual map of themes was drawn by the facilitator (on a whiteboard or flip chart) to aid participants and facilitate discussions about how different aspects may fit together (see Fig. 1). The focus groups lasted between 57.63 and 71.65 min (mean = 63.26 min, standard deviation = 5.89 min).

Fig. 1
figure 1

Example of a whiteboard discussion “map” used to facilitate the focus group sessions

Full size image

2.3 Analysis

Data were transcribed verbatim and checked for errors. The data were analysed using a framework analysis [26], following six stages (adapted from [27]):

  1. 1.

    Familiarisation: The researchers independently read and re-read one SU and one MHP transcript, while listening to the recordings.

  2. 2.

    Coding: The study authors independently coded the same two transcripts in hard copy. Thematic codes were informed deductively by a priori themes from the topic guide (ESM) and inductively by emerging themes from the data.

  3. 3.

    Developing a framework: The researchers met to compare their coding and consensus was reached on a working analytic framework, where subthemes were grouped into higher order themes. The first author then repeated steps (1–2), applying the draft framework to the next two transcripts, while noting any new or inconsistent subthemes. The second author repeated step (1). The two researchers met again to discuss and refine the draft framework. This process was repeated for the final two transcripts, and a final framework was agreed.

  4. 4.

    Indexing: The final framework was applied to fresh copies of all transcripts on Nvivo version 12 (QSR International Pty Ltd., 2018) by the first author. Any observed relationships between themes were coded. The second author checked two randomly selected transcripts for agreement.

  5. 5.

    Charting: Microsoft Excel was used to summarise the data, using one row per subtheme and one column per participant, with a separate sheet for each theme. Each cell in the matrix was populated using verbatim quotes.

  6. 6.

    Interpretation and mapping: The researchers met to discuss the interpretation of the data. A description was written about each subtheme, which included any observed similarities or differences between SUs and MHPs and/or disconfirming cases. A conceptual map of themes was produced. Results are discussed below within themes, with reference to subthemes in bold, and supported by quotations.

3 Results

Twenty-one subthemes were coded, hierarchically ordered within six themes. The themes and their relationships are mapped in Fig. 2 and their coverage in the data is in the ESM. Adequate data saturation was observed with no new themes indexed in the final two focus groups (ESM). A summary of the subthemes produced during the framework analysis is included in the ESM.

Fig. 2
figure 2

Thematic framework. Relationships between themes are shown with arrows. Thicker arrows represent that the relationship was coded more often in the data. MHP mental health professional, SU service user

Full size image

3.1 Accessing Mental Healthcare

Speed of access to services was acknowledged by both SUs and MHPs as important, but there was a trade-off with other aspects of quality, such as providing the right type of care, or managing and minimising risk through appropriate triaging.

“I think in order to kind of, erm, make sure that people, people are offered services quicker and better access, sometimes something is offered which is just a bit of a poor alternative” (Mental Health Professional 1 [MHP1], in Focus Group 4 [FG4])

Ease of access, including geographical location, physical access and other barriers, was also noted as important, but trade-offs were observed as above; some SUs stated that they would be prepared to travel further if they received better quality care in other respects, such as being treated as a person (i.e. being treated with dignity and respect and not being reduced to a disease, diagnosis or number). Both SUs and MHPs acknowledged the impact on SUs if access was disrupted.

“Part of me was thinking about access and physical because (…) if I was in hospital my family are all this side of Sheffield. But, I don’t think it’s important enough against that whole thing of us as people” (SU3, FG1)

Frequency of access was discussed by SUs as important to quality, with examples shared of a reduced frequency of contact with their workers, attributed to a reduced capacity and resources in the service.

“I used to have a person come to see me every week, that’s been changed to every 2 weeks and then they come for a really short amount of time (…) there’s no time to do anything” (SU4, FG1)

3.2 Assessing the Benefits of Care

Both SUs and MHPs agreed that person-specific aims and outcomes of mental healthcare should be prioritised, and its benefit should be judged primarily by asking the SU. However, some MHPs stressed the need for outcomes to be professionally supported, based around realistic and appropriate goals.

“The actual service user being able to tell you they feel better. If they can tell you, that you did something (…) [it] should definitely be determined by the service user” (SU1, FG3)

Both SUs and MHPs spoke to the limits of aggregate targets, which could lead services to prioritise those targets over other elements of care. This includes the ‘top-down’ implementation of care targets that service the needs of the organisation, not necessarily the SU. A target of “discharge”, for example, may not reflect whether that discharge is appropriate or well managed and may increase pressure to inappropriately discharge people. Service users disliked aggregate targets, while MHPs identified some value, but only as secondary to person-specific aims.

“Here’s target one, target two, target three, am I as a worker meeting that target and we’re getting lost in the care plan actually, taking precedence at the moment” (SU1, FG3)

“I worry about those kind of quantitative, erm, tick box kind of ways of measuring recovery (…) that kind of one size fits all approach” (MHP1, FG4)

A recovery focus model was endorsed by both SUs and MHPs, helping SUs to live with their condition in a fruitful, independent and functional manner. Service users did not like “end goals” (i.e. reaching the goal(s) of a course of treatment), which implied failure if someone required support again. For MHPs, providing a recovery focused service was associated with underlying capacity and resources.

“What the recovery model means to me is I’ll get to a place where I can live with it and it’s kind of a cope-able with thing (…) I’m not gonna get better from being a schizophrenic, I’ve just got to learn to live with being a schizophrenic” (SU4, FG1)

3.3 Co-ordinated Approach

Mental healthcare should feature good co-ordination in services, including during transitions and discharge. Concerns were raised by SUs and MHPs about SUs not meeting different services’ criteria and “falling through the gaps”. For SUs, good transitions were well planned, justified and involved good communication. For MHPs, co-ordinated working with other teams, including community services, was positive. Proactive care was also endorsed by SUs and MHPs, including prioritising early intervention and not waiting until SUs had reached “crisis” stage.

“People don’t meet the thresholds and other teams are kind of battening down the hatches and saying we can’t take it anymore, people just fall through the gaps” (MHP1, FG2)

The benefits of community and wider services beyond the health service, including peer support, were championed by both groups. Concerns were raised from MHPs about funding cuts to community and wider services, with health services increasingly having to try and plug the gap.

“They weren’t supplemental, they weren’t, you know, the cherry on the cake, they were kind of core intrinsic elements of, of the wellbeing of a community” (MHP2, FG2)

Continuity of care was seen as important, including seeing the same person, and being treated to the same standard by different professionals. Service users and MHPs agreed that fewer changes in personnel throughout treatment was optimal, facilitated a good therapeutic relationship and helped to minimise risk. Service users expressed a fear of discharge as a discontinuity of care and emphasised that support should still be available if needed.

“Talking therapies where you’re expected to give somebody, to talk to somebody perhaps about some of the most darkest parts of your life perhaps, you have to build up that trust to be able to trust that person to tell them that. So if you come in the week after and it’s somebody different, all that effort’s lost on both your part and on the part of the service” (SU2, FG3)

3.4 Delivering Mental Healthcare

For multiple MHPs and at least one SU, efficient triage and risk planning were central to optimise care and minimise risk. Triage was linked to speed of access, with those deemed to be at greater risk being prioritised sooner. Risk was also discussed in terms of discharge and relapse and that continuity of care may facilitate a better risk perception for MHPs.

“You need an immediately accessible triage (…) taking that risk and giving it to somebody who can assess that risk (…) that needs to be immediate” (MHP1, FG5)

The capacity and resources of the mental health service were perceived as directly related to quality. Service users recognised the financial limitations of the service. Some SUs felt forced to pay for private therapy, creating a financial barrier to care. Mental health professionals acknowledged a lack of funding and felt under pressure to provide a quality service with limited resources.

“The financial issues aren’t gonna get any better, with mental health services. What they do seem to be doing is struggling on in isolation” (SU4, FG6)

“We haven’t got the resources to provide, what we did before. How are we going to you know provide a quality service given we’re taking away essentially” (MHP3, FG2)

The healthcare environment was recognised as having an impact on perceived quality, including perceived safety and confidentiality, and the quality of the aesthetics. Service users described the need for a “safe” environment, where their confidentiality was protected. Mental health professionals commented on the aesthetics of the environment, noting that poor aesthetics were embarrassing, sent a message to SUs and affected staff morale.

“You don’t go in and there’s wires sparking, you know, and it feels dangerous kind of thing. But (…) I mean safe in terms of confidential” (SU2, FG3)

Good communication throughout the mental health service was important. Service users stressed the need for communication with them to be honest, unpatronising and consistent. Open and honest communication from management was seen as important by MHPs, particularly to staff who may already feel vulnerable. Mental health professionals saw communication as two-way, where management should be receptive to what frontline workers are telling them.

“Occasionally I’ve been rung up and told, er, we’re cancelling the appointment (…) I would say about 50% of the time that happened. The other 50% of the time, they wouldn’t tell me (…) I would turn up and then learn that, er, that person was off sick” (SU3, FG3)

Service users and MHPs both championed equity in care, acknowledging that the current system was inequitable. Service users expressed a suspicion that MHPs were choosing who they wanted to work with and were avoiding more resource-intensive SUs. Mental health professionals described how the quality of care differed across time and place and how gatekeepers deciding whether to accept a new SU would have to consider the impact on quality for existing SUs.

“It’s called the Goldilocks zone. What the psychologist is doing is they’re putting their finger into the porridge and going you’re too hot we don’t want to treat you (…) you’re too cold, you’re too salty, so we don’t want to treat you (…) They’re getting away with a pick and mix like Woolworths” (SU1, FG3)

3.5 Individualised Care

Offering and receiving the right type of care, perceived to have the most benefit for SUs, was seen as critical to quality. Service users believed strongly that they knew what care was best for them and that it was not always available. Mental health professionals acknowledged capacity and resource limitations meant they were not always able to provide optimal care. Some MHPs expressed that care should be evidence based and decided upon through consultation, not just what the SU believes is best, diverging from the SU view.

“The therapy that was working for me was EMDR therapy. I’ve had CBT thrown at me over the last 8 months (…) that is totally the wrong thing and no one said is CBT the right thing for you [participant’s name]? And the answer is very clear: No” (SU2, FG1)

“They’ve come in with this, ‘I feel I’ve got this’ (…) And we say, ‘based on the information that we’ve got, we’ve made an assessment that it’s this, we’d like to offer you this instead’. Even though we think they’re offering a high quality service based on their accurate assessment, based on their perceptions they might think they’re getting an awful, poor quality service” (MHP1, FG5)

A person-centred care approach was advocated by SUs and MHPs. Service users emphasised the need to avoid a one-size-fits-all approach and put the SU first. Mental health professionals tended to agree that the SU should have a central role, but some diverged by sharing concerns that the expertise of the clinician should be recognised and preferred a joint approach.

“It’s important that you move away from one size fits all. Obviously it’s, it’s, it’s best fit, but the individual has, has a lot more insight into the unique nature of their own mental health” (SU3, FG6)

“I think there’s something about we’re doing this together, you know, you have a role in this too (…) It’s a shared journey, task whatever you call it” (MHP3, FG2)

In a high-quality service, SUs expected to be treated as a person, not reduced to a disease, diagnosis or number. This included being treated with dignity and respect and not being prejudged based on their records.

“You don’t always get treated as a person (SU3: No you don’t). You get treated as sort of like kind of like a nuisance or (SU3: Yeah, a number). Yeah or like a number or (SU3: A disease)” (SU4, FG1)

3.6 Role of the Person Providing Care

While they acknowledged that MHPs need to be knowledgeable and appropriately qualified, SUs placed more weight on the personal qualities of MHPs including compassion, empathy and caring; characteristics seen as fundamental to a good therapeutic relationship. Mental health professionals showed some divergence from this view, emphasising that it was important that the professionals knew what they were doing and it was not “just about being nice”.

“I think it’s more about whether the person is good with other people rather than whether the person has a qualification. Like, the best kind, some of the best people that I’ve ever had work with me have been like kind of like a support worker or something” (SU4, FG1)

The therapeutic relationship between SU and MHP was acknowledged as important by both parties. A good relationship is achieved by avoiding prejudgements, removing power differentials, actively listening, maintaining honesty and developing reciprocal trust. Service users stressed the interpersonal qualities of the MHP and being treated as a person as two additional important facilitators. Mental health professionals noted several barriers, including limited resources.

“The talking therapy itself is probably only half of the story. That the relationship you have with the therapist is probably, is half as much important again” (SU2, FG3)

“Well you’ve got to have some kind of relationship, you know, before you do anything, you know, you’ve gotta develop that haven’t you? And if that’s not there, nothing else falls into place really” (MHP3, FG2)

Finally, the well-being of MHPs needs to be considered as it impacts the quality of healthcare directly and indirectly. Both MHPs and SUs independently raised the issue of frontline MHPs’ well-being, with SUs expressing sympathy and acknowledging the pressure their workers were under.

“I feel sorry for the staff. Because like they’re, they’ve been cut, and they’re having to do like they need to be an octopus (…) the pressure they must be under” (SU5, FG6)

4 Discussion

The objectives of this research were to: (1) identify attributes that SUs and MHPs deem important for accurately assessing quality in mental healthcare to help inform attribute selection in a subsequent DCE and (2) explore any similarities and differences between SU and MHP’s perspectives on the attributes identified. With regard to the first objective, this research has identified a six-theme framework of attributes (reflecting 21 subthemes, highlighted in bold in the Results section) that SUs and frontline MHPs deemed important for assessing quality in mental healthcare. While most of these features, such as continuity of care [28], co-ordination in services [29] and qualities of the MHP [30], have been investigated in other studies on the experience of mental healthcare, this has often occurred in relative isolation. The current work benefits from asking SUs and MHPs to think comprehensively of all aspects they deem important for assessing quality and the relationships between them.

Whilst acknowledging that coding coverage is an imperfect indication of attribute importance [31], some of the most talked about themes included person-centred care, capacity and resources, receiving the right type of mental healthcare, the relationship between SU and MHP, and qualities of the MHP. Many of these themes focus on putting the person central to mental healthcare and building the right therapeutic relationship. All have been identified as important to mental healthcare in previous work, corroborating the current findings [30, 32,33,34,35,36]. For example, a focus group study on the desirable qualities of mental health nurses identified themes of respect towards SUs as persons and empathy, compassion and effective communication [30]. Another qualitative study featuring focus groups and interviews with MHPs reported on the impact of (a lack of) capacity and resources for trauma-informed mental health services [32]. Discussion on the aforementioned attributes was greater than on some other aspects that have been the focus of recent UK policy, such as access [5]. They also represent areas where a deficit in tools to measure quality has been identified and thus indicates a direction for future work [4]. In particular, a recent position piece highlighted a lack of development and implementation in performance measures designed to reflect patients’ views and treatment choices, noting that “few endorsed mental health quality measures assess patient-centered care” [4, p. 32].

Complementary work builds upon the framework produced here to identify whether the relative preferences for aspects of quality differs between SUs, MHPs and the general public to ascertain the relative importance of the attributes identified in this study using a DCE. The methodology and results of this DCE are reported elsewhere [20], and this includes further details on how the framework identified in this qualitative study was used, alongside other sources, to help inform the final choice of attributes and levels for the DCE design.

Regarding the second objective, there was a large degree of concordance between SUs and frontline MHPs on the elements that were important for assessing quality. In particular, SUs and MHPs agreed on the value of person-specific aims, person-centred care, the recovery focus model and the importance of continuity in care. Equally, SUs and MHPs both identified problems with top-down aggregate targets, limited capacity and resources, and inequity in care. However, some areas of divergence emerged. For example, SUs believed strongly that they knew best what care and goals were right for them. Some MHPs expressed caution and re-emphasised the role of the expert and evidence base in this process, which they viewed at risk of being lost. Further, SUs stressed more strongly the importance of the personal qualities of MHPs, while MHPs commented that expertise and knowledge were just as important as “being nice”.

The focus within some of the themes differed for SUs and MHPs. For example, co-ordination in services for MHPs was not just about SUs’ transitions, but the importance of working with other teams, including community services. Moreover, communication for SUs primarily concerned the frontline service, while MHPs stressed the need for appropriate two-way communication with management.

In addition to identifying areas of agreement and disagreement between SUs and MHPs on measuring quality in mental healthcare, the findings from this study extend existing knowledge in at least three ways. First, as different aspects of quality were considered together, this provided a unique insight into trade-offs SUs and MHPs identified between aspects of quality. For example, SUs expressed that they were prepared to travel further to receive care where they were treated with more dignity and respect. Mental health professionals identified that focusing on faster access as a quality marker may result in a poorer alternative of mental healthcare, and more weight should be placed on delivering the right type of care. Second, several nuances within subthemes emerged that have not been as widely discussed in previous studies. This includes fear of discharge, which has historically between discussed in the context of inpatient care [37], but here extends to wider mental healthcare provision and is linked to the limitations of top-down targets. The subtheme of communication also emphasises the importance of (two-way) communication between MHPs and management in a high-quality communication process, not simply the quality of communication SUs receive from the service. Third, some subthemes were identified that have received scant attention in prior qualitative literature on quality in mental healthcare. Specifically, to the authors’ knowledge, SU and MHP’s perspectives on how MHP well-being is intertwined with the quality of service delivery have seldom been highlighted. Further, insights into equity of care, including perceived inequities across time and place and different “levels” of MHP, are described in this study, which have received little qualitative exposure in the past.

4.1 Strengths and Limitations

Strengths of this work include an inclusive approach of incorporating SU’s perspectives alongside those of MHPs, supporting the derivation of a bottom-up framework of quality indicators. Limitations of this work include the homogeneous sample, which was recruited from a single UK NHS Trust. While saturation was adequate and the number of focus groups was predetermined based on published guidance [23], it is possible that other attributes would be identified in other geographical locations or by other MHPs, such as psychologists. Second, limited descriptive data were collected to describe the study sample. Third, while occurring implicitly throughout the research, no explicit reflective exercise (e.g. reflective journal) was conducted by the researchers on how their backgrounds may have shaped data collection and interpretation.

5 Conclusions

A comprehensive six-theme framework (reflecting 21 subthemes) for understanding quality in mental healthcare, incorporating the views of SUs and frontline MHPs, has been developed. Higher coverage in the data was observed for themes such as person-centred care and capacity and resources than themes such as speed of access. Novel trade-offs were identified between aspects of quality, such as being prepared to travel further to be treated more like a person (i.e. with dignity and respect). While many themes corroborate prior research findings, nuances and subthemes were identified in the data that have received less prior attention, such as fear of discharge from mental health services as linked to the limitations of top-down targets and perspectives on equity of care. Service users and MHPs displayed a high degree of concordance in their views on the aspects important for assessing quality, with minor areas of divergence, such as SUs believing strongly what was right for them when it came to care and MHPs re-emphasising the role of the expert in this process. This framework can be used to help inform the selection of a meaningful set of quality indicators in mental health for research and practice.