Introduction

In a clinical audit, prospective data of the treatment process and of patient outcomes is collected in a systematic way. Based on these data, feedback is given to provide hospitals insight into their own performances. Also, in a clinical audit, hospitals are benchmarked on pre-established quality indicators to define best practices that can serve as examples for underperforming hospitals. The aim of this continuous process is that hospitals implement changes in the treatment process based on the feedback, with the ultimate goal to improve the quality of care [1].

In the last decade, audits for different diseases have been initiated [2,3,4,5,6,7,8]. The participation in an audit can be a major logistic challenge to hospitals. Active participation may involve a high registration load because data recorded in the electronic health record (EHR) often has to be copied into a separate clinical audit database. This implies incorporation of the data entry process into routine hospital working procedures. To be able to deliver meaningful feedback to hospitals and for an audit to be successful in improving quality of care, it is important that as many hospitals as possible participate and that the data registry is as complete as possible. In order to ensure the successful implementation of new nationwide multidisciplinary clinical audits, reasons for both willingness (‘facilitators’) and refusal (‘barriers’) to participate in an audit should be clear. Knowledge about these facilitators and barriers may also offer opportunities to enhance participation in ongoing audits. In literature, potential facilitators and barriers for conducting a clinical audit are described [9, 10]. To our knowledge, evidence is lacking about which facilitators and barriers actually influence hospital participation in new and ongoing audits.

The aim of this study was to identify which factors (facilitators and barriers) experienced by hospital staff are associated with hospital participation in a nationwide multidisciplinary clinical audit on hip fracture care that was recently started in the Netherlands, the Dutch Hip Fracture Audit (DHFA).

Methods

Dutch Hip Fracture Audit

The DHFA is a continuous nationwide multidisciplinary hip fracture audit, which was started in April 2016 at the Dutch Institute for Clinical Auditing [8]. In the Netherlands, there are approximately 18,500 hip fracture patients each year in 81 hospitals [11]. All these hospitals were invited to participate in the DHFA. Table 1 outlines what participation in the DHFA actually involves for a hospital.

Table 1 What does participation in the DHFA mean at hospital level?
Full size table

Identification of factors influencing participation in the DHFA

To identify factors that may influence participation by medical professionals in the DHFA, the first author conducted semi-structured interviews with 8 trauma surgeons, 3 orthopaedic surgeons and 1 geriatrician/internist between 1 June 2017 and 8 November 2017. All the interviewees were members of at least one of the medical associations involved in the multidisciplinary DHFA.

The structure of the interviews was developed specifically for the DHFA according to the Theoretical Domains Framework [12, 13] (see Appendix 1). This framework was chosen, as determinants of behaviour related to both willingness and refusal to participate in the recently started clinical audit had to be identified [14]. The interviews were recorded and transcribed verbatim. The transcriptions were analysed according to the directed content approach, using the software package ATLAS.ti (ATLAS.ti Scientific Software Development GmBH, Berlin, Germany) [15]. First, possible factors that influence participation were highlighted in the transcripts, and secondly, the highlighted phrases were coded by two independent researchers. Since agreement of this double coding was sufficient in the first two transcripts (Cohen’s kappa of 0.89—see Appendix 2), the remaining 10 interviews were coded by one researcher (SV). In total, 76 factors that potentially influence participation in the DHFA were identified.

Development and distribution of the questionnaire

The questionnaire aimed to explore which factors, identified in the semi-structured interviews, are actually independently associated with participation of hip fracture surgeons in the audit. As the surgeons are mainly responsible for the setup of the DHFA within hospitals, the questionnaire was not sent to geriatricians and internists. In the Netherlands, hip fracture surgery is performed by both orthopaedic surgeons and trauma surgeons. At the moment, the trauma surgeons treat the majority of hip fracture patients in the Netherlands, while orthopaedic surgeons operate relatively more patients who need a total hip replacement after a hip fracture.

The first part of the online questionnaire included questions covering the respondent characteristics: type of surgeon (orthopaedic or trauma surgeon), years of experience, annual number of hip fracture operations, type of hospital (academic or general hospital), familiarity with the DHFA (yes or no) and hospital level of participation in the DHFA. The level of participation was divided into four categories: not participating in the DHFA, and having no intention to do so; not participating in the DHFA, but intending to do so; partially participating in the DHFA with < 80% of the patients are registered; and actively participating with ≥ 80% of the patients are registered. If the respondent indicated in the first part not to be familiar with the audit, the second part of the questionnaire was not shown.

The second part of the questionnaire included the factors identified in the interviews that may influence participation in the DHFA. To limit the length of the questionnaire, the 76 factors were grouped into 21 overarching factors (Table 2). Henceforward, the word ‘factor’ refers to these 21 overarching factors. For each factor, the respondents could indicate their level of agreement on a 4-point Likert scale: totally disagree, partly disagree, partly agree and totally agree. All questions had to be filled out to complete the questionnaire.

Table 2 Characteristics of the 109 respondents divided by participation degree
Full size table

First, the questionnaire was distributed among the orthopaedic surgeons and trauma surgeons attending the annual national trauma congress in November 2017. After the congress, a link to the questionnaire was sent to all orthopaedic and trauma surgeons who are members of the Dutch Trauma Society, excluding those who had already completed the questionnaire. The targeted number of respondents was arbitrarily set at 100.

Analysis

The data from the questionnaire was analysed using IBM SPSS Statistics® version 22. The respondent characteristics and the level of agreement to each of the 21 factors were reported using descriptive statistics. The categories for level of agreement were dichotomised for each factor into ‘disagree’ (combining the response categories ‘partly disagree’ and ‘totally disagree’) and ‘agree’ (combining ‘partly agree’ and ‘totally agree’), because for many factors, some categories were chosen by less than five respondents. For similar reasons, the four categories of participation were combined into two different dichotomous variables: the participation variable with ‘not participating’ versus ‘participating’ and the active participation variable with ‘partially participating with < 80% of patients registered’ versus ‘actively participating with ≥ 80% of patients registered’.

For both participation and active participation in the DHFA, the associations with respondent characteristics and the 21 overarching factors obtained from the interviews were first studied in a univariable logistic regression analysis. Next, respondent characteristics and factors found in the univariable analysis to be associated with participation or active participation (p < 0.10) were entered in a forward stepwise multivariable logistic regression analysis, with the factors/respondent characteristics as independent variables, and participation/active participation in the DHFA as dependent variables. In the multivariable model, predictors for participation or active participation were considered statistically significant if the p value was less than 0.05. Factors significantly increasing the participation in the DHFA were classified as facilitators, and factors significantly decreasing the participation in the DHFA as barriers.

Results

At the national trauma congressin November 2017, 50 questionnaires were completed. Since the targeted number of respondents was at least 100, we sent out 257 questionnaires by mail after the congress.

In total (i.e. at the congress and by mail) 109 surgeons completed the questionnaire, when the questionnaires received at the congress and by mail were combined. Seven surgeons (four orthopaedic surgeons and three trauma surgeons) who completed the questionnaire did not know about the existence of the DHFA. The characteristics of the respondents across the different categories of participation are shown in Table 2. Our questionnaire was completed by 87 trauma surgeons (80%) and 22 orthopaedic surgeons (20%). Almost 14% of the respondents worked in an academic hospital and 86% in a general hospital.

Factors most agreed on to influence participation in the DHFA

The three factors obtained from the interviews that were most agreed on in the questionnaire by both participants and non-participants concerned organisational aspects: ‘staffing capacity should be made available for the data collection in the DHFA (95.1%)’, ‘data import from the EHR into the DHFA should be automated (95.1%)’ and ‘participation in the DHFA should be supported financially by the hospital board (94.1%)’ (Table 3).

Table 3 Respondents’ agreement (partly/totally agree (versus partly/totally disagree)) with each factor for participation in the DHFA. The participating group is further stratified by participation degree
Full size table

Factors associated with actual participation in the DHFA

In the univariable analyses, none of the 21 overarching factors as identified in the interviews was significantly associated with actual participation in the DHFA (p > 0.10). Of the respondent characteristics, type of hospital (academic versus general hospital), type of surgeon (orthopaedic versus trauma) and annual number of hip operations had a univariable association with participation in the DHFA (p < 0.10) and were entered in the multivariable analysis (Table 4). Of those, working in an academic hospital (odds ratio [OR] 0.15, 95% confidence interval [CI] 0.04–0.52, p < 0.01) and being an orthopaedic surgeon (OR 0.30, 95% CI 0.10–0.90, p = 0.03) were independently associated with lower participation in the DHFA.

Table 4 Multivariable logistic regression analysis of participation in the DHFA, including factors with univariable p < 0.10
Full size table

Factors associated with active participation in the DHFA

Within the subgroup of surgeons who participated in the DHFA, three factors (‘the DHFA data provides insight into the actual quality of hip fracture care for relevant external parties’, ‘the DHFA working group makes a proper assessment what data can be made available to relevant external parties’ and ‘the added value of the DHFA is not clear’) were associated (p < 0.10) with active participation (i.e. registration of more than 80% of the hip fracture patients) in the univariable regression analyses and therefore entered in the multivariable analysis (Table 5). Of these, only the facilitator ‘the DHFA data provides insight into the actual quality of hip fracture care for external parties’ was significantly associated with active participation in the DHFA (OR 3.19, 95% CI 1.14–8.95, p = 0.03) in the multivariable regression analysis.

Table 5 Multivariable logistic regression analysis of participation degree (< 80% of the patients registered versus > 80% of the patients registered), including factors with univariable p < 0.10
Full size table

Discussion

This study identified factors associated with hospital staff participation in a recently started nationwide clinical audit, the Dutch Hip Fracture Audit (DHFA). The most often agreed on factors influencing participation were related to the organisational context of the audit, i.e. how the data collection of the DHFA should be organised. However, only physician characteristics (type of hospital and type of surgeon), and no factors were independently associated with participation in the DHFA. For active participation (≥ 80% of patients registered), the opinion of surgeons that the DHFA data provides relevant external parties with insight into the actual quality of hip fracture care was significantly associated with the level of participation.

The factors identified through the semi-structured interviews are in line with the possible facilitators and barriers of a clinical audit previously described in literature [9, 10]. The additional value of our study is that it showed that none of these factors is actually associated with hospital participation, except that making data available to relevant external parties can serve as a facilitator for active participation.

The organisational context of an audit

The primary goal of a clinical audit is to improve the quality of care. Similar to our results, most respondents in a survey among US surgeons also felt that an audit (the National Surgical Quality Improvement Program, NSQIP) would contribute to a better quality of care [16]. Interestingly, in the present study among both participants and non-participants, respondents agreed more on factors relating to the organisational context of the data collection than on the quality enhancement factors of an audit. At the start of the DHFA, no arrangements were made for the data collection and it was up to the hospital staff themselves to find a way to collect all the data for the audit. This might explain why the respondents in this study agreed more on the organisational factors of the audit rather than factors influencing its usefulness as a tool to improve the quality of patient care.

In literature, a previously reported facilitator for a successful audit is easy data collection, through the use of modern medical record systems [9]. A study of Cornish et al. showed that lack of support in data collection can also be a reason not to participate in an audit: in a survey among colorectal surgeons, the main reason not to participate in a 6-year running audit was the lack of technology support in the data submission [17]. Once an integrated audit has been established, minimal time and effort of a senior practitioner is needed [18].

As the factors most agreed on in the questionnaire were not specific to hip fracture management, our findings are applicable to upcoming clinical audits in general, regardless of their topic. Therefore, it is advisable that data collection is automated as much as possible and that time and resources are made available before the start of any audit. Hospitals in which the organisation of ongoing audits is well arranged can serve as examples for other hospitals where clinical auditing has yet to be implemented. Hospitals lacking the financial means to hire adequate staff for the DHFA data collection solved this by training of a research nurse in their employ, how to collect the data. They are now responsible for the data collection, but under the direct supervision of a surgeon in case of uncertainties.

Factors that influence participation in an audit

Orthopaedic surgeons and trauma surgeons working in an academic hospital might be less likely to participate in the DHFA than orthopaedic surgeons and trauma surgeons working in a general hospital, respectively. This finding may be explained by the fact that in the Netherlands, orthopaedic and academic surgeons operate considerably lower numbers of hip fracture patients. For this reason, they might consider the effort of implementing an audit into the routine working procedures for relatively few patients per year not be worthwhile. This consideration is also true from a statistical point of view. Low-volume hospitals/wards are less interesting for benchmarking, as analyses for hospitals/wards with low patient volumes are underpowered to detect variation in outcomes of care [19]. One lesson to be learned from our findings, which we believe applies to every clinical audit, is that physicians treating low numbers of audit patients are less likely to participate in an audit. The questionnaire was only sent to surgeons, but it is most likely applicable for all physicians who treat a low number of audit patients.

Factors that influence the participation degree

A facilitator for active (≥ 80% of patients registered) participation in an audit might be to make data available to relevant external parties (such as the National Health Care Institute and the healthcare insurance companies) and give them insight into the actual quality of hip fracture care by benchmarking hospitals. A possible explanation for this finding is that making hospital-specific data available soon after the start of an audit works as a trigger for hospitals to start registering, as they do not want to lag behind other hospitals. In the survey of Cornish et al., the national benchmarking facility also appeared to be the main reason to submit data to the bowel cancer audit [17]. Also, two other survey studies found that the general opinion of surgeons was that audit results should be publicly reported at hospital level [16, 20].

The optimal timing to make data available for relevant external parties is hard to define. From the second year of the DHFA on, the results of process indicators have been made available to the National Health Care Institute and the healthcare insurance companies. From the third year on, hospitals can choose to also make hospital-specific outcome indicators available, but from the fourth year onwards, this will be mandatory. The National Hip Fracture Database (NHFD) in the UK also made outcome indicators publicly transparent at hospital level in the third year after its start in 2007 [21].

In our opinion, it is important for all kinds of upcoming clinical audits to realise that data being available to external parties can be a determining factor for hospitals to participate actively in an audit. Therefore, it is advisable to agree at an early stage on how and when data will be disclosed to other relevant external parties.

Lessons learned from other audits

Interventions in other audits that proved to be helpful in their implementation included financial rewards and obligatory participation. The need of financial support by the hospital board was also the third most agreed-on factor in our study. The NHFD has also shown that a financial reward, the Best Practice Tariff, can serve as a facilitator for hospitals to participate in a nationwide audit [22]. The Best Practice Tariff consists of six standards that have to be registered in the NHFD. If these standards are met, a hospital receives a financial reward [23].

Governmentally imposed participation has proven to facilitate participation. After the participation in the Dutch Surgical Colorectal Audit had become a quality indicator for the Health and Youth Care Inspectorate in the Netherlands, the participation rate rose to almost 100% [1, 2].

Limitations

The grouping of the 76 factors identified in the semi-structured interviews into 21 overarching factors for the questionnaire has induced loss of information detail; it may have caused loss of the value of individual factors. The reason to group the factors was that, although we identified so many factors in the interviews, the questionnaire still should be clear and feasible and preferably be filled out within 10 min. We felt that sending out a questionnaire with all 76 factors would not render us the responses we needed. Another reason to group factors was from a statistical point of view: 76 factors and 5 respondent characteristics would induce the risk of multiple testing. Even with the 21 overarching factors this cannot be explained.

Another limitation is that the questionnaire was only sent to members of the Dutch Trauma Society, and therefore included only trauma and orthopaedic surgeons. Consequently, physicians from other medical disciplines such as geriatricians and internists were not invited to fill out the questionnaire. As a result, facilitators and barriers that are specific to non-surgeons were not included. This was a deliberate choice, since the setup of the DHFA data registry in a hospital is mainly the responsibility of the orthopaedic and trauma surgeons.

Regarding the representativeness of the respondents to the survey, the ratio between trauma surgeons and orthopaedic surgeons in our study (80/20) was comparable with that in the DHFA data for 2017 (71/29). However, the surgeons working in an academic hospital were overrepresented (14%) in the survey compared with the DHFA data for 2017 in which only 1% of the hip fracture patients were treated in an academic hospital. In addition, as responses were anonymous, possible clustering of respondents working at the same hospital may have occurred ,leaving unknown whether the respondents geographically represented the whole country. Both issues imply a possible selection bias.

Yet another limitation is that two of the three orthopaedic surgeons who were interviewed were not familiar with the DHFA. We may therefore have missed possible factors that are especially important for orthopaedic surgeons.

Conclusion

We learned from this study that the data collection process of a nationwide clinical audit should either be performed by additional staff or be automated in order to avoid an increase in registration load for physicians. Physicians treating low numbers of audited patients are less likely to participate in an audit. Active participation in an audit may be promoted by offering relevant external parties, such as insurers, healthcare authorities or policymakers, insight into the actual quality of care. The use of these results will accommodate the successful implementation of new audits and the provided tools may offer opportunities to optimise participation in running audits.