Introduction

Globally, people with a mental health condition experience greater morbidity and mortality due to chronic disease compared to the general population [1, 2]. Tobacco smoking, poor nutrition, harmful alcohol use and physical inactivity are modifiable health risk behaviours associated with chronic disease development [3,4,5,6] and are more prevalent among people with a mental health condition compared to the general population [1, 7,8,9]. Although people with a mental health condition are interested in modifying these behaviours, they may need additional support to do so [10,11,12,13]. Mental health services are well placed to provide such support due to the routine nature and frequency of contact, and established rapport between client and clinician [14,15,16,17,18,19].

The delivery of preventive care (routine assessment and management of risk behaviours) by health care providers is an evidence-based intervention to improve health behaviours [20,21,22]. Clinical practice guidelines recommend the systematic provision of preventive care during routine health service appointments [23, 24]; including within mental health services [25, 26]. An endorsed approach to providing preventive care is the ‘5As’ model: ‘ask’ about behaviours, ‘assess’ interest in change, provide ‘advice’ to change, provide behaviour change ‘assistance’, and ‘arrange’ referral [24, 27]. To overcome barriers such as time constraints, the abbreviated ‘AAR’ model (assess, advise, and refer) is recommended at a minimum [28, 29] and has demonstrated effectiveness [30], including among clients attending mental health services [31]. Despite the evident potential for mental health services to provide preventive care, systematic review evidence demonstrates that preventive care is infrequently provided in mental health services internationally [10, 20, 32], and there is a need to develop and implement strategies to support services in providing care.

Systematic review evidence shows there have been few trials assessing the effectiveness of implementation strategies to support mental health clinicians to provide preventive care [9]. The review identified 20 studies that assessed the impact of any strategy to increase at least one component of the 5As for any of the four key risk behaviours. Fourteen were found to be effective in increasing at least one element of preventive care provision. Effective strategies included clinician training, educational materials, health information systems, local consensus processes, authority and accountability, and reminders.

A previous study by the research team was the only study identified in the review and the only study the authors could locate, that aimed to increase care for all four behaviours in routine mental health appointments [32]. The multiple baseline trial assessed the effectiveness of six implementation strategies in increasing AAR throughout 12 community mental health services. Client-reported receipt of preventive care before and after the implementation support package increased for only one of 16 outcomes. An evaluation of the same implementation strategies in the same health district but within generalist community health services resulted in increased assessment and advice outcomes [33, 34]. The difference in effectiveness between the two studies may have been due to the implementation strategies being generic, and not adequately addressing the clinical, professional, and organisational factors that are specific to the community mental health settings. This indicates a need for future research to develop and test implementation strategies to increase preventive care that do appropriately consider the context of community mental health service delivery.

Participatory approaches provide opportunities to include key stakeholders in design, maximising the potential that implementation strategies are suitable and meet the needs of end-users [35, 36]. Involving mental health clinicians in the development of implementation strategies to support practice change has been suggested to enhance the likelihood of strategies being acceptable and aligning with the context [37]. Several implementation trials in mental health settings which have involved mental health clinicians in strategy design have demonstrated improved clinical practices (e.g. increased conversations about smoking [38] and metabolic syndrome screening [39]), and improved client outcomes (e.g. increased primary care access [40] and reduced cigarette consumption [38]).

This study aimed to assess the effectiveness of an implementation support package on community mental health clinicians’ routine delivery of preventive care (assessment, advice and referral) for four health risk behaviours (tobacco smoking, harmful alcohol consumption, physical inactivity and inadequate fruit and vegetable intake) compared to no implementation support. The participatory approach and fidelity of the implementation strategies were also described.

Methods

Study design and setting

A non-randomised controlled trial was undertaken with two community mental health services that provide care to adult clients with a mental health condition (one control site and one target site) in one health district in New South Wales (NSW), Australia. In Australia, community mental health services are typically government-funded services and provide early diagnosis, acute clinical care, and rehabilitation to clients with a range of serious mental health diagnoses (e.g. schizophrenia, severe depression or anxiety, and comorbid substance use). These services are the most accessed specialty mental health service in Australia [41]. The health district in which the study was conducted has a policy requiring clinician assessment, advice and referral to ongoing behaviour change support services for clients with chronic disease risk behaviours, including smoking, inadequate fruit and vegetable consumption, physical inactivity, and harmful alcohol consumption [42]. Based on consultation with the district’s Mental Health Executive Director, community mental health services within the district were considered eligible based on geographical feasibility for the research team delivering the support strategies, and services with equivalent staff sizing. Of the four services meeting these criteria, two were excluded due to involvement in similar research, the remaining service in closest geographical proximity to the research team was allocated as the target site, and the other as the control site.

Following a two-phase participatory design process (details provided below), a four month support package consisting of multiple implementation strategies aiming to support clinicians in the delivery of preventive care was undertaken with the target site (November 2019 to February 2020). Outcome measurement included telephone surveys conducted with cross-sectional samples of clients at baseline over six months (26 April 2019 to 17 October 2019) and follow-up over five months (12 May 2020 to 14 October 2020).

The first COVID-19 wave in Australia began to emerge during the delivery of the implementation strategies (January 2020), with state-wide lockdowns occurring at the end of the implementation period (March 2020) and continuing until the third month of the follow-up data collection period.

The four primary outcomes for the study were client-reported receipt of assessment of risk status for all four health behaviours; advice for all risks for which they were assessed (for those with at least one risk); referral offer for at least one risk for which they were assessed (for those with at least one risk); and complete care (assessment of all behaviours, advice for all risks, referral for at least one risk). Secondary outcomes were client receipt of assessment, advice and referral for each of the four behaviours (tobacco smoking, alcohol consumption, physical activity and fruit and vegetable intake (12 outcomes)).

The trial was approved by the Hunter New England Human Research Ethics Committee (Approval No. 18/11/21/4.06) and the University of Newcastle Human Research Ethics Committees (Approval No. H-2019–0108). Reporting is in accordance with the Standards for Reporting Implementation Studies (StaRI) Statement. The trial was registered with the Australian and New Zealand Clinical Trials Registry (ACTRN12619001379101).

Participants and recruitment

Eligibility

At baseline, adult clients of both sites were eligible to be recruited for data collection if they: had attended at least one individual in-person appointment within the previous 4 months and had a phone number listed in the health district’s electronic medical record. At follow-up, eligibility criteria were adjusted to also include telehealth appointments within the previous four months, due to service changes in appointment types as a result of COVID-19 restrictions. Clients were not excluded from follow-up data collection if they had been selected for, or participated in, the survey at baseline. At both time-points, additional eligibility criteria were assessed on contact by trained interviewers: English speaking, mentally and physically capable of responding to survey items, and not currently living in aged care facilities or gaol.

Recruitment

Potentially eligible clients were identified via electronic medical records. We aimed to sample approximately n = 600 clients (n = 300 per site) at baseline and at follow-up to have 80% power to detect a 14% increase in assessment and a 19% increase in advice and referral outcomes [32, 43]. For both the target and control sites, approximately 30 clients were randomly selected each week across both sites during the baseline and follow-up periods. Clients were mailed an information sheet informing them of the survey and data collection procedures. They were provided with a toll-free number they could call if they did not wish to be contacted for participation. Clients were contacted approximately two weeks later via telephone by trained interviewers employed by the health service and asked whether they would like to participate. If clients identified as being of Aboriginal and/or Torres Strait Islander origin, they were offered the opportunity to have their interview conducted by an Aboriginal interviewer. Participants could withdraw from the study during the survey, and participation in data collection and receipt of care were independent.

Target condition

The Community Mental Health Service allocated to the target site received a four month implementation support package aiming to increase their provision of preventive care to clients using the AAR model of care. The service employed 47 clinical staff at the start of the trial (including full and part-time staff equivalent to approximately 21 full-time staff members), and 46 at the end of the trial, with 39 staff being employed for the full 4 months. The implementation support strategies included within the trial were designed with target site staff, during the six months prior to implementation. The model of care and implementation support package (including the participatory approach) are outlined below.

Model of preventive care

The intervention was clinician provision of the preventive care model ‘Assess, Advise, Refer’ (AAR), in line with the health district’s preventive care policy. The policy directs clinicians to offer clients an assessment of four health risk behaviours (tobacco smoking, inadequate fruit and vegetable consumption, physical inactivity, and harmful alcohol consumption). Clients identified as being at risk as defined by Australian national guidelines and recommendations [44,45,46,47] for any of the health risk behaviours are: provided brief advice on the benefit of to changing their behaviours to meet the Australian national guidelines; and, offered a referral to specialised behaviour change support services (e.g. evidence-based, state-wide telephone support services New South Wales (NSW) Quitline for smoking [48], the NSW Get Healthy Information and Coaching Service for inadequate fruit and vegetable intake, inadequate physical activity and harmful alcohol consumption [49], or a local general practitioner (GP), or allied health provider).

Implementation support package

The initial selection of implementation strategies was informed by implementation research and review evidence [9, 50,51,52,53]. Five implementation strategies were identified by researchers as potentially suitable to trial in the community mental health service and were proposed to managers and clinicians: (1) champion/local opinion leaders; (2) clinician educational meetings/training and educational materials; (3) enabling resources and prompts for clinicians (staff activation); (4) client activation materials; and (5) audit and feedback (see Table 1). The components and contents of the strategies were modified according to input and feedback provided by clinical managers and clinicians over a two-phase participatory design process that occurred over seven months. This allowed the implementation strategies proposed initially to be refined to suit the context of the community mental health care service.

Table 1 Implementation support package: strategy proposed in the participatory design process, description of the final strategy for inclusion, and delivery to service

Participatory design process: phase one

An initial implementation working group (including the service manager, clinical coordinator, psychiatrist, dietitian, senior mental health nurse, and the research team) met six times over a five month period prior to the implementation support package commencement. The working group discussed whether and how the five proposed implementation strategies might offer potential solutions to current challenges in delivering preventive care. Such challenges reported in the literature, and relevant to the service, include a lack of resources, time, skills, and perception of client motivation. Four of the five proposed implementation strategies were perceived as applicable (excluding champions/local opinion leaders) to address these barriers (Table 1). There were always at least three service staff represented at each of the six meetings. Usual support was provided to clinicians regarding preventive care delivery from the service manager during phase one.

Participatory design process: phase two

All service clinicians were provided the opportunity to participate. At commencement of phase two, clinicians were informed by service management about the upcoming implementation support package and the outcomes of the implementation working group meetings in phase one. Over two months, all clinicians were invited to three meetings that occurred during their regular morning in-service meeting session. The purpose of these sessions was to introduce the four implementation strategies as planned at the end of phase one and to provide clinicians opportunities to offer input regarding their content and delivery.

The care delivery model was augmented, at the request of clinicians, to include resources to support clinicians to ‘Assist’ client behaviour change (in line with the 5As model; though not assessed as an outcome of the study). For example, the resource booklets provided education on motivational interviewing tips with examples of phrases that clinicians might use, as well as brief, practical tips clinicians might offer clients to assist behaviour change.

Final implementation support package

The research team finalised the implementation strategies based on input from phases one and two and provided the implementation working group with a final set of strategies for approval. Project personnel (program manager and PhD student) supported delivery of the implementation support package and provided ongoing assistance as needed or requested through face-to-face visits with managers and clinicians, and phone calls and/or e-mails to managers. See Table 1 for a description of strategies proposed during the participatory design process, and a full description of the final strategies included. The final implementation support package was based on four of the five strategy categories initially proposed in phase one and included 11 component sub-strategies (Table 1):

Clinician educational meetings/training and educational materials: training sessions (45-minute face-to-face training sessions covering six topics); and four educational resource booklets for clinicians (one for each of the four behaviours).

Enabling resources and prompts for clinicians (staff activation): paper-based assessment tool (for clinicians to assess client risk); referral forms (paper-based referral forms for NSW Get Healthy Service and Quitline); client handout resources (paper-based resources that could be given to clients by clinicians); and a clinician poster (poster for the staff meeting room).

Client activation materials: conversation cards (coloured conversation starter cards); a client poster (poster placed in the client waiting room); and a client self-assessment form (paper-based form for clients to self-assess risk).

Audit and feedback: Recording preventive care discussion and a preventive care assessment tool.

Control condition

The control site continued to provide their usual preventive care practices and could receive the usual support provided by the health district to do so.

Data collection and measures

Outcome data relating to preventive care delivery to clients was obtained via client computer-assisted telephone interviews (CATIs) administered by a trained and experienced team of telephone interviewers employed by the health district and utilised an established CATI facility and calling protocols (see Fig. 1 for participant recruitment flow). The interviewers undertaking the client surveys were not informed of which site the client had attended, although for the follow-up interviews, this would be known by the end of the survey due to the inclusion of items only asked of participants who had attended the target service. The statistician undertaking the analysis was not blinded to site allocation.

Fig. 1
figure 1

Flow diagram of client recruitment

*Clients were coded out of scope if they could not recall an appointment with the community mental health service, were non-English speaking, were too sick, or were in prison, aged care, or hospital

Client-reported data

Demographic and clinical characteristics

At baseline and follow-up, participants reported their gender, age, highest education level attained, employment status, marital status, and the mental health condition/s for which they had received care from the service for within the previous four months. Client postcode and the number and mode (face to face, or telephone) of service appointments within the previous four months were obtained from the electronic medical record system.

Health risk behaviours

At baseline and follow-up, participants reported their levels of tobacco smoking, alcohol consumption (AUDIT-C) [54], physical activity (IPAQ) [55], and fruit and vegetable intake. Survey items were based on recommended assessment tools [56]; and previous surveys undertaken with community health and mental health clients [8, 34, 43]. In line with Australian health behaviour guidelines, clients were defined as being at-risk if they reported smoking any tobacco products in the previous four months [57], consuming on average less than five serves of vegetables and/or less than two serves of fruit per day [47], consuming more than ten standard drinks on average per week or more than four standard drinks on any one occasion in the previous four months [46] or engaging in less than 150 minutes of moderate activity, or less than 75 minutes of vigorous activity, or an equivalent combination of both, per week, OR, less than two days of strength/resistance physical activity per week [44].

Receipt of preventive care

Assess: Clients were asked to report whether a clinician had assessed each of their risk behaviours, at any of their appointments during the last four months (yes/no/don’t know).

Advice: Clients who were classified as at risk and had reported receiving assessment were then asked for each relevant behaviour, whether their clinician advised them (at any of their appointments during the last 4 months) about the national behaviour recommendations and to modify that risk behaviour(s) (yes/no/don’t know).

Refer: For each behaviour where they were at-risk and had reported receiving assessment, clients were asked whether their community mental health clinician had (at any of their appointments during the last 4 months) offered them a referral to a relevant specialist or service (for example NSW Get Healthy Service, Quitline, GP, dietician or exercise physiologist, dependent upon behaviour).

Delivery of the implementation support strategies

Project personnel including a post-doctoral researcher and PhD student were responsible for recording the delivery of support strategies listed in Table 1. Project records consisted of a standardised spreadsheet that was updated on a consistent ‘as it occurred’ basis to log strategy delivery, which was stored on a secure drive accessible to the researchers.

Statistical analysis

Descriptive statistics were used to summarise delivery of the implementation support strategies. Analyses were undertaken using SAS 9.4. Descriptive statistics were used to describe client demographic and clinical characteristics, health risk behaviour prevalence, and receipt of preventive care elements. Demographic variable categories were condensed as per Table 2. Dichotomous variables were calculated to define health risk behaviours (see Supplementary Table 2).

Table 2 Sample characteristics by group and time

Four dichotomous (yes/no) primary outcome variables were created: (1) client receipt of assessment for all four behaviours (regardless of risk status); (2) of those who had received assessment and were classified as at-risk for a behaviour, receipt of advice for relevant risk behaviours; (3) of those who had received assessment and were classified as at-risk for a behaviour, receipt of at least one referral offer for a relevant risk behaviour; and (4) a composite variable, ‘complete care’, reflecting receipt of all three preceding primary outcome variables. Twelve secondary outcomes for preventive care receipt were: assessed for each behaviour (4 outcome variables); advice for each at-risk behaviour they were assessed for (4 outcome variables); and received a referral offer for each at-risk behaviour they were assessed for (4 outcome variables) (Supplementary Table 1).

Logistic regression models utilising a group-by-time interaction term were developed for each of the outcome variables (16 models: 4 primary outcomes, 12 secondary outcomes; Table 2) to examine change in care delivery from baseline to follow-up in the target compared to the control group. The interaction term assessed whether the change over time is different between the groups. Change in care delivery was determined to be significantly different between groups if the adjusted p-value for the group by-time interaction term was < 0.05 in the regression model, and ORs were also calculated to compare the odds of care delivery between target and control groups at follow-up. Logistic regression models were adjusted for three continuous variables: age, telehealth appointment counts, and in-person appointment count during the previous four months to participation. These variables were selected as they could act as confounders to the receipt of preventive care, as identified in the existing literature [10, 32, 34, 43]. Appointment counts were particularly relevant due to changes in service care delivery from baseline to follow-up as a result of COVID-19 restrictions. Sensitivity analyses were conducted using logistic regression models (16 outcomes) that excluded repeat participants to account for not having independent observations in the sample. Unadjusted logistic regression models were run within groups between time points (baseline and follow-up) for each outcome to assess within-group changes in care delivery and were determined to be significantly different within groups if the p-value was < 0.05.

Results

Sample characteristics

Of the 3116 clients selected to participate, 2072 were contactable and hence able to be assessed for eligibility (66.5%). Of the 1796 eligible potential participants (contactable and eligibility assessed minus those who did not meet the inclusion criteria), 889 (49.5%) consented to participate, with 860 included in the analysis (baseline n = 429, follow-up n = 431) (Fig. 1). Sample characteristics are presented in Table 2. The most prevalent mental health condition was depression at baseline (control 64.3%; target 60.9%) and at follow-up (control 53.7%; target 40.1%). The most prevalent risk behaviour was inadequate fruit and vegetable intake at baseline (control 96.9%; target 97.5.1%) and at follow-up (control 98.2%; target 96.4.1%), and the least prevalent risk behaviour was alcohol overconsumption at baseline (control 47.6%; target 43.7%) and at follow-up (control 47.0%; target 33.3%).

Effectiveness on preventive care levels

Between-group effects

There were no significant differential changes over time between groups for any of the primary or secondary outcomes (Table 3). The sensitivity analyses excluded n = 80 participants (control n = 25; target n = 55) who completed both surveys and demonstrated similar results to the models using the entire sample, with no new significant differential changes.

Table 3 Adjusted logistic regression models for primary and secondary outcomes

Within-group effects

At each site, there was one significant within-group effect from baseline to follow-up. Participants were less likely to receive nutrition assessment at follow-up compared to baseline in the target site (OR = 0.61; [0.42–0.87]; P = 0.01) and less likely to receive nutrition advice in the control site (OR = 0.54; [0.29–1.00]; P = 0.05) (Supplementary Table 3).

Implementation support package delivery and fidelity

The delivery of the implementation support strategies is summarised in Table 1. Of the four strategies planned for delivery, one was delivered as intended (clinician training and educational resources), two were delivered but not as originally intended (enabling resources and prompts for clinicians, and client activation materials), and one was not delivered (audit and feedback). Regarding the strategy delivered as intended (clinician training and educational resources), nineteen to thirty staff attended each training session which occurred in the first six weeks of the implementation period, and delivery of the educational resources took a staged approach and occurred in the following weeks (7, 8 and 10). The two strategies delivered to some extent (clinician enabling resources and client activation materials), although provided to the service, had poor uptake due to COVID-19 protocols and late-stage implementation (see footnotes Table 1). For example, the client activation conversation cards were removed from the waiting room and staff were advised to not provide shared hardcopy materials to clients (including the clipboard and pen for the client self-assessment form). Furthermore, the enabling resources were delivered at the end of the implementation period, which limited their opportunity for use. For example, the poster in the staff meeting room to prompt preventive care would have had limited visibility as staff review meetings were reduced when COVID-19 restrictions took effect. The audit and feedback strategy, although identified as appropriate by staff, was unable to be implemented due to service changes and COVID-19 limitations. Firstly, service changes included the health district-wide reformatting of clinical case review forms so that individual services were no longer able to amend the forms for their own purposes, including the changes that were to be made as part of the support package to enable the collection of preventive care provision data during review meetings. Secondly, there was a significant reduction of review meetings to avoid face-to-face contact and align with COVID-19 protocols.

Discussion

To the author’s knowledge, this is the first study to examine the effectiveness of an implementation support package designed using a participatory approach that aimed to increase preventive care delivery for multiple health risk behaviours by community mental health services. The four month implementation support package was ineffective in increasing the four primary outcome measures of (1) assess for all behaviours, (2) advice for all relevant risk behaviours, (3) refer for any relevant risk behaviour, and (4) complete care, nor the 12 secondary outcome measures. The implementation support package was not delivered as planned; with three of four final strategies delivered, of which two were not implemented as intended due to contextual variables including the impact of the COVID-19 pandemic on service delivery. Hence, findings should be interpreted within this context.

It is likely that the limited fidelity in delivery of support strategies impacted overall effectiveness. The two components within the clinician training and education strategy were delivered as intended. During the implementation period, the training sessions occurred within the first six weeks and the educational resources were delivered consistently. However, other strategies were partially delivered, delivered late, or not delivered at all due to a number of issues, including those associated with the emerging COVID-19 pandemic such as state-wide health service changes and difficulty engaging with services. Existing research literature would suggest it is likely that the limited success of the implementation support package in the current study is at least partly due to these strategies not being delivered as intended [58,59,60].

Although it is not possible to determine the impact of the emerging COVID-19 pandemic on fidelity of support strategy delivery, or on staff capacity to provide preventive care, it is likely to have had an impact. The first COVID-19 pandemic wave triggered federal and state governments to implement guidelines limiting face-to-face contact between staff and specifically requesting that health facilities limit in-person contact with clients. These changes came into effect during the end of the implementation period and remained during follow-up data collection. The changes to clinical service provision leading up to and during the COVID-19 lockdown placed unprecedented strain on the health system due to factors such as wearing masks, general concerns about occupational health risks and the widespread implementation of new telehealth systems and protocols. Staff absenteeism increased due to COVID-19 protocols including non-attendance at work with respiratory symptoms, abiding by stay-at-home orders due to close contact (usually children or other family members), or testing positive to COVID-19, and a small number of clinicians were redeployed to man COVID-19 call centres which placed a greater burden on remaining clinicians. For example, service data collected in 2020 between the end of March and the start of July showed 26 staff took sick leave related to COVID-19 testing and isolation protocols resulting in a total of 136 days ineligible to work (including admin and clinical staff). Each staff member was additionally absent for four hours for vaccinations.

Furthermore, services also moved to telehealth methods of service delivery wherever possible, a significant change to practice. Clinical records indicate a substantial reduction in in-person appointments and appointments overall was experienced by clients in aim of limiting cross-infection (Table 2), a finding reflective of the wider NSW healthcare system [61]. Appointments needed to include COVID-19 screening questions and advice regarding vaccines and testing thus reducing the time available for other care provision. This reduction may have meant there were less opportunities for care which may have affected preventive care delivery. COVID-19 increased the side effects of many common psychiatric medications and as a result, clinicians needed to focus on modifying dosage particularly for antipsychotics due to the risk of worsening mental illness. Clinicians may have been more likely to prioritise the provision of acute care due to the deteriorating mental health of clients [62, 63]. The observed reductions in some preventive care elements in both the target (nutrition assessment) and control sites (nutrition advice) may have been contributed to by such factors (Supplementary Table 3). Studies indicate mental health staff face competing clinical priorities as a barrier to providing preventive care generally [64, 65] and such a barrier may have been heightened during COVID-19.

There have been calls to publish more explicit accounts regarding the involvement of end-users in health service research [66,67,68]. This study describes the process and outcomes of a two-phase participatory approach to design a support package consisting of multiple implementation strategies for community mental health service clinicians, which may aid knowledge translation and evaluation of participatory methods [69, 70]. A participatory approach involving mental health clinicians and managers was utilised to design implementation strategies to address clinical, professional, cultural, and organisational factors that distinguish community mental health service delivery from other community health services [65, 71, 72].

The current study engaged in a two-phase participatory design process which involved a smaller and more senior working group to identify challenges and potential solutions (e.g. clinician training and educational resources), followed by involvement with a larger group of all clinicians to provide feedback on outcomes from the working group (e.g. details and content of training and educational resources). It is possible this process did not adequately capture the needs and wants of clinicians ‘on the ground’ and tailor strategies accordingly. The participatory design process occurred over a total of seven months, including six meetings over five months with the implementation working group of senior staff (phase one), and three meetings over two months with on-the-ground clinicians (phase two), followed immediately by the delivery of the four month implementation support package. Such timing may not be sufficient, particularly with limited time and opportunity to develop the implementation strategies between the design phases. This meant several strategies were delivered later than intended during the implementation period and ultimately were more severely impacted by COVID-19. Furthermore, the barriers to preventive care provision were likely to have changed between the participatory design process which was undertaken prior to the pandemic emergence and full implementation due to the rapidly changing context of COVID-19. An understanding of the context of preventive care delivery in community mental health services gained from the participatory process used may not have been sufficient to select effective implementation strategies, and a more systematic and rigorous method of participatory research design, such as co-design workshops, may be beneficial [68].

In phase two, clinicians requested more focus of the implementation strategies on how to provide support to clients in line with the ‘assist’ component of the 5As (discussion of the benefits and barriers to change, providing counselling to change behaviours (such as motivational interviewing), and/or providing additional supports including pharmacotherapy, educational materials or self-help materials) [27, 73], which could suggest that clinicians would like to improve their skills in preventive care delivery to provide greater behaviour change support to their clients, instead of, or in addition to, referring on to other services. As such, the education and training package included a large amount of content on motivational interviewing, conversation starters, hints and tips for behaviour change and practical resources to offer clients. Other literature supports this, with studies demonstrating that staff working in community-based mental health settings want to assist mental health clients in making behaviour changes [74].

As previously discussed, not all implementation strategies were delivered as intended, a finding that may have contributed to the absence of measured impact. For example, there is evidence for the potential of client activation materials [75] however this strategy had limited fidelity in the current study largely due to changes in service protocols and restrictions resulting from COVID-19. Future research should implement and measure the suitability and effectiveness of client activation materials in community mental health settings. Fidelity could potentially be improved in future research with implementation support strategies that are robust and are designed within the context on virtual staff training and care delivery. For example, by employing an implementation support officer or clinical champions in a dedicated role to reduce the burden on mental health staff to deliver certain strategies, and to reinforce the importance of preventive care delivery and provide assistance and support to clinicians. Future research should consider involving clients in content development [76]. Overall, achieving improved fidelity requires sufficient resourcing for delivery of implementation strategies, as well as a high level of planning in which commitment from all stakeholders in the clinical setting should be negotiated.

Limitations

With regard to limitations, the design of the current study included two sites (one target and one control) which meant the implementation support package was delivered in one community mental health service in a single health district, thereby potentially limiting its generalisability to other districts and jurisdictions. The lack of randomisation for the two sites may have compromised the ability to estimate probabilities of differences due to potential confounding factors. However, randomisation of sites in community trials has been suggested as unacceptable for pragmatic trials within health services [77]. The lower than anticipation completion rate led the study to be underpowered, which may have contributed to the lack of significant results. The main outcome measures were all client self-report receipt of preventive care. There is an absence of research directly comparing different methods of data collection to measure preventive care delivery in mental health services (e.g. client-report, staff report and medical record audit); however, the literature indicates client report of preventive care received has strengths relative to other approaches [20, 78].

Conclusion

The implementation trial was not effective at increasing preventive care receipt by people accessing a community mental health service; potentially contributed to by limited fidelity and impacts of COVID-19. Findings suggest there is a need to continue to understand what is required to increase preventive care delivery in community-based mental health settings. Further investigation is required to determine optimal participatory design methods to develop effective implementation support strategies that lead to increased preventive care delivery in community mental health settings, including those that support delivery of care within the ongoing context of COVID-19.