Background

Danish cancer patients have lower survival and more advanced disease stages at treatment initiation than many other European countries [1, 2]. The ability among general practitioners (GPs) to interpret and respond to symptoms plays a pivotal role in cancer detection as 85% of all cancer patients initiate their diagnostic pathway in general practice [3, 4]. Increasing evidence suggests that prompt referral from general practice matters [5–7], and a UK study found that a high propensity to use urgent referrals was associated with better survival of cancer patients [8].

Cancer detection in general practice is, however, a challenging task. The symptoms of cancer are diverse, they develop over time, and they tend to mimic symptoms of trivial and common conditions [9]. Some symptoms are unspecific, e.g. fatigue and weight loss. Others are considered to be alarm symptoms [10] although they have a low positive predictive value of cancer of only 2-10%, depending on age, gender and cancer type [11–13]. Danish cancer patients have increased visits in general practice already six months before the diagnosis [14], and one in four waits for more than 20 days in general practice until referral [4, 15]. Consequently, there may be room for improvements.

To support the GPs’ decision strategies for referral, a specific Continuing Medical Education meeting (CME-M) was developed and implemented as part of the Danish National Cancer Plan III in 2012 [16]. The developed CME-M consisted of central elements for cancer detection in primary care [17]. So far, the initial evaluation has shown that the CME-M has affected the knowledge among GPs on cancer diagnosis and their attitude towards own role in cancer detection. The CME-M also lowered the GPs’ assessed risk of cancer in urgently referred patients [18]. Yet, we need to explore whether the change in GPs’ knowledge and attitude was translated into a behavioural change.

The aim of this study was to investigate the effect of the CME-M on the use of urgent cancer referrals by investigating urgent referral rates and the timing of urgent referrals in terms of length of primary care interval and number of patient contacts with general practice before referral.

Methods

The setting of the study was the Central Denmark Region with approx. 1.3 million inhabitants and 8,000 new cancer patients annually [19]. The Danish healthcare system is tax-funded with free access for citizens to medical advice and treatment in general practice and hospitals. More than 98% of the Danish citizens are listed with a specific general practice, which they must consult for medical advice. GPs serve as gatekeepers to specialised care, such as diagnostic investigations and hospital care [20]. In order to ensure consistency in the GPs’ selection of patients for urgent cancer referrals, national referral guidelines were made for each cancer fast-track pathway [21, 22]. As an example, the referral guideline for colorectal cancer state that cancer should be considered if a patient above 40 years of age presents at least one of the following symptoms: visible rectal bleeding, discharge of mucus or changes in bowel habits or stools for a four-week period, iron-deficiency anaemia and unspecific symptoms as pain or weight loss [21]. All Danish citizens have a civil personal registration (CPR) number, which enables linkage of information at the individual level between national registries and allows identification of the practice at which each citizen is listed [23].

Design

We conducted a before-after study with a stepwise enrolment of GPs in the CME-Ms [17]. All practices in the Central Denmark Region were allocated to one of eight clusters by exploiting the already existing municipal units. The Regional Cancer Quality Unit assigned the random order in which all GPs (practices) in the cluster were offered the CME-M at 3-week intervals within the period 1 September 2012 to 1 May 2013 (Fig. 1). GPs were registered if they participated in the CME-M [17].

Fig. 1
figure 1

Time periods used for classifying patient populations. The cluster-specific time period used for identifying urgently referred patients in the primary care referral database (light boxes = time before CME-M; dark boxes = time after CME-M). The method at the top was applied for evaluating CME-M effect on referral rates and patient contacts with general practice. The method at the bottom was used for evaluating CME-M effect on primary care intervals

Full size image

Intervention: continuing medical education meeting

The content of the CME-M was developed through a comprehensive process, which has been described previously in detail [17]. The CME-M provided the GPs with new knowledge on symptoms of cancer and presented risk assessment tools (RATs) for lung, colorectal, ovarian and prostate cancer [13, 24]. Use of investigations and reasons for missed opportunity [25, 26] were discussed in terms of risk of false reassurance [27, 28]. Mechanisms affecting referral thresholds were debated based on patient cases [29].

Outcomes

Use of urgent referrals was operationalised as: 1) urgent referral rate per GP (i.e. number of urgently referred patients during six months per GP in the practice) and 2) urgent referral rate per 1000 patients in the practice (i.e. number of urgently referred patients during six months per 1000 listed patients aged ≥ 40 years in the practice). Timing of urgent referrals was operationalised as: 1) prior contacts (i.e. number of daytime contacts with general practice from urgently referred patients before referral) and 2) primary care interval (i.e. number of days from first presentation in general practice until urgent referral to secondary care) [30].

Practice population

The practices were identified in the Danish provider number registry. The number of included practices depended on the outcome to be measured. For urgent referral rates, practices were included if registered from six months before to six months after their CME-M date. For prior contacts, practices were included if registered from 18 months before to six months after their CME-M date. For primary care interval, practices were included if registered in the period from 1 September 2012 to 1 May 2013 provided that they had completed one-page forms (Additional file 1).

Patient populations

The urgently referred patients were identified by applying an algorithm, which was developed for the purpose, to the primary care referral database [31]. The algorithm had a sensitivity of 93.6%, a specificity of 97.3%, a positive predictive value of 83.6% and a negative predictive value of 99.0% for identifying urgently referred patients aged ≥ 40 years without prior cancer among all referred patients aged ≥ 40 years [31].

For urgent referral rates, patients were included if: 1) identified by the algorithm, 2) referred from an included practice from six months before until six months after the CME-M date of the practice, 3) age ≥ 40 years and 4) no prior cancer.

For prior contacts, patients were included on the same criteria as above. Furthermore, they were also required to have been listed with the same practice for one year prior to the referral date.

For primary care interval, patients were included if: 1) identified by the algorithm, 2) referred from an included practice from 1 September 2012 to 1 May 2013, 3) age ≥ 40 years, 4) no prior cancer and 5) receipt of one-page form stating the date of first symptom presentation and the date of referral.

Patients already diagnosed with cancer were excluded based on data in the Danish Cancer Registry [32] registered in accordance with the International Classification of Diseases, 10th revision: C0-C9, except for non-melanoma skin cancer (C44) [33].

Variables

Patient contacts with general practice were obtained from the Danish National Health Service Register [34]. The registration is based on fee-for-service remuneration of the providing GP, and the records in this registry are generally considered highly complete [35]. We used two variables: 1) face-to-face (F2F) contacts and 2) total contacts, which included F2F, phone and e-mail contacts.

Date of first presentation of symptom in general practice and date of referral were collected by requesting GPs to complete a one-page form including the patient’s CPR number each time a patient was referred for suspected cancer during an 8-month period around the CME-Ms (1 September 2012 to 1 May 2013) (Fig. 1) [17]. Furthermore, in the hospital patient administrative system, patients registered as investigated as part of an urgent referral were identified every other week. If the one-page form was missing, the relevant GP was requested to complete a form and to specify the patient’s route to investigation. This was done to increase the completeness of data for urgently referred patients.

The development, pilot-testing, coding and data transfer of the forms have been reported elsewhere [17].

Age and gender distribution of the patient population for each practice was obtained from the patient list system on 1 January 2013.

The Danish Deprivation Index (DADI), which is used to adjust for differences in socioeconomic factors, was calculated based on data from the Integrated Database for Labour Market Research [36]. The DADI has a value between 10 and 100; the higher the number, the more deprived population. The variables used were: (i) proportion of adults aged 20–59 years with no employment, (ii) proportion of adults aged 25–59 years with no professional education, (iii) proportion of adults aged 25–59 years with low income, (iv) proportion of adults aged 18–59 years receiving public welfare benefits (transfer payments or social security benefits), (v) proportion of children from parents with no education and no professional skills, (vi) proportion of immigrants, (vii) proportion of adults aged 30+ years living alone and (viii) proportion of adults aged 70+ years with low income (= the lowest national quartile).

A modified Charlson Comorbidity Index score [37] was obtained for each referred patient by using data from the Danish National Patient Register with the referral date as the index date [38]. We divided the comorbidity scores into “none” (no recorded disease), “moderate” (score of 1 or 2) and “high” (score of 3 or more) [15].

The suspected cancer type stated at the one-page form was divided into two groups of cancer detection difficulty based on the available evidence [39]. Cancers considered easier to detect included suspected cancer in kidney, bladder, breast, head and neck, female genitalia, nevus (melanoma), penis, testis and gastrointestinal system, whereas cancers considered harder to detect included suspected cancer in pancreas, liver and gall bladder, brain, lymph node and bone marrow, lung, prostate, connective tissue including fat, muscle and bones, and suspicion based on unspecific serious symptoms.

Statistical analyses

Practices were divided into two groups: practices with at least one CME-M-participating GP (“CME-M practices”) and practices without any CME-M-participating GPs (“reference practices”).

The time point for a CME-M session for each cluster was defined as the point that separated the time of observation in “before CME-M” and “after CME-M” (Fig. 1). The data collected before and after the CME-M provided paired responses for each outcome. The practice and patient populations were described by their characteristics.

Urgent referral rates were indirectly age-sex standardised as we used the full patient population in the Central Denmark Region as standard population divided into ten-year age groups (40–49, 50–59, etc.).

The CME-M effect on urgent referral rates was analysed using mixed-effects negative binomial regression [40] with a random effect associated with the intercept for each practice to make comparisons before and after CME-M within and between groups. The model allowed dealing with repeated observations on practice level. The time of observation (before and after CME-M) was treated as a fixed-effect variable. In the analysis of urgent referral rates per GP, the number of GPs per practice was treated as an exposure variable. The effects within groups were reported as incidence rate ratios (IRRs). Comparison of effects between groups was reported as a ratio of the IRRs. The analyses were adjusted for cluster, type of practice (single-handed/partnership) and DADI index. A single-handed practice was defined as a practice with only one GP. A stratified analysis was performed on practice type.

Prior contacts were measured in different time intervals: 0–1 month, 0–3 months and 4–6 months preceding referral. The number of contacts during the 7–12 months preceding referral was calculated to estimate an average habitual contact rate per month for each patient.

The CME-M effect on prior contacts was analysed using mixed-effects negative binomial regression with a random effect associated with the intercept for each practice and time of observation as a fixed-effect variable to make comparisons before and after CME-M within and between groups while adjusting for patient clustering in the practices. The results were reported as IRRs or a ratio of the IRRs. Due to convergence problems for F2F within the last month preceding the referral, the analysis was based on ordinary negative binomial regression applying cluster robust variance at practice level.

For the 10% of patients with most frequent contact, the number of F2F contacts and total contacts were dichotomised based on the 90th percentile (number of contacts within 0–1 month was 4 for F2F and 5 for total contacts). A mixed-effects logistic regression [41] allowed for random effects at practice level, and the observation time (before and after) was treated as a fixed-effect variable. The effects were reported as odds ratios (ORs) for having most contacts within groups and ratio of the ORs between groups.

The analyses were adjusted for cluster, practice type, patient gender, age, habitual monthly contact rate and comorbidity. Furthermore, a stratified analysis on practice type was performed.

The primary care intervals were calculated as medians, 75th and 90th percentiles. We dichotomised the intervals based on the 75th and 90th percentiles from before the CME-M. Comparisons within and between groups were estimated with mixed-effects logistic regression with a random effect associated with the intercept for each practice adjusting for patient clustering in the practices. The results were reported as ORs or a ratio of the ORs. The analyses were adjusted for cluster, practice type, patient gender, age, co-morbidity and cancer detection difficulty. Furthermore, stratified analyses were performed on cancer detection difficulty and type of practice.

The statistical software Stata 13.0 (StataCorp LP, TX, USA) was used for the analyses [42].

Results

Study population

A total of 148 of the 396 general practices (37.4%) participated in the CME-M (Table 1). Compared to the reference practices, the CME-M practices were more often partnership practices, the GPs were slightly younger and more often female, the practice population size per GP was lower, the practice population consisted of fewer male and slightly less deprived patients (Tables 1 and 2).

Table 1 Characteristics of practices, GPs and patients used for CME-M impact on urgent referral rates
Full size table
Table 2 Characteristics of practices, GPs and patients used for CME-M impact on patients’ prior contacts
Full size table

The patients used for calculation of the primary care interval were included by 139 CME-M practices and 199 reference practices (Table 3).

Table 3 Characteristics of practices, GPs and patients used for CME-M impact on patients’ primary care interval
Full size table

Impact of CME-M on urgent referral rates

The CME-M was associated with a two-fold relative rise in urgent referral rates per GP when comparing CME-M practices with reference practices. Within the CME-M group, this increase was statistically significant (IRR: 1.05 (95% CI: 1.01;1.08) (Table 4). Each CME-M practice referred one patient more per 1000 patients in their underlying practice population during the 6-month period after the CME-M compared to before; this increase was three times higher than the increase in reference practices (Table 4). When we stratified on practice type, the partnership practices who participated in the CME-M increased their urgent referral rate per GP (IRR: 1.06 (95% CI: 1.02;1.10) and their urgent referral rate per 1000 patients (IRR: 1.06 (95% CI: 1.01;1.12) from before to after the CME-M.

Table 4 The CME-M impact on urgent referral rates
Full size table

Compared with the reference practices, none of these findings were statistically significant (Table 4). Note that the stratified analyses on practice type showed a higher use of urgent referrals among CME-M-participating single-handed practices compared to non-participating single-handed practices (Table 4).

Impact of CME-M on prior contacts

There was a statistically significant reduction in the number of total contacts within the last month before referral in the CME-M group compared to the reference group (IRR-ratio: 0.97 (95% CI: 0.94;0.99)) (Table 5). Note the tendency that the proportion of patients contacting the most in terms of total contacts decreased from before to after in the CME-M group (OR-ratio: 0.90 (95% CI: 0.86;1.02)) (Table 5).

Table 5 The CME-M impact on patients’ prior contacts with general practice
Full size table

Impact of CME-M on primary care intervals

The proportion of patients who waited 14 days or more in the CME-M practices statistically significantly increased from before to after the CME-M compared to the reference practices (OR-ratio: 1.47 (95% CI: 1.06;2.03)) (Table 6). This was also seen as a tendency for other estimates although these figures were not statistically significant. When we stratified on cancer detection difficulty, no statistically significant differences were found in the primary care intervals (Table 6).

Table 6 The CME-M impact on patients’ primary care interval (PCI)
Full size table

Sensitivity analyses

When we excluded patients urgently referred within one or two months after the CME-M date, the CME-M effect on urgent referral rates and prior contacts did not differ considerably from the main results (Additional file 2).

Discussion

Main findings

Forty percent of general practices participated in the cancer diagnostic CME-M. Increased use of urgent referrals was found in the CME-M group from before to after the CME-M. However, this increase was only small in absolute numbers and was not statistically significantly different from the figures for the reference group. Compared with the reference group, the CME-M seemed to reduce the number of total contacts with general practice in the CME-M group during the last month before the urgent referral. Furthermore, the proportion of patients with more than five contacts in total during the last month decreased in the CME-M group; together this indicates a lower referral threshold.

The CME-M was associated with an increase in the reported primary care intervals.

Strength and limitations

The before-after design allowed us to perform a robust evaluation with corrections for baseline measures of a natural experiment. We controlled for calendar time by comparing CME-M group and reference group and by including the stepwise enrolment of GPs in the CME-M into the modelling of data, which diminished the influence of increasing cancer-related knowledge over time.

Another strength was the well-defined and well-described study population and the valid identification of the individuals listed with the studied practices. Additionally, we were able to link selected information on all included individuals to relevant register data at the individual level.

The main limitation of this study was the natural selection to participate in the CME-M. Participating practices and GPs differed from non-participating practices and GPs, particularly single-handed practices differed on the use of urgent referrals (Table 4). This indicates that CME-M participating practices cannot be directly compared to non-participating practices. The CME-M practices had a higher use of urgent referrals among 1000 patients at baseline; this indicates that their potential for improvement was lower than for the entire study base, which may have underestimated the effect of the CME-M. However, the true direction of such bias is difficult to establish.

The use of the referral algorithm allowed us to base the identification of urgently referred patients on register data [31]. We have no reason to believe that patients from CME-M practices had different chances of being identified by the algorithm than patients from reference practices. On the contrary, the patient population used for the evaluation of the CME-M effect on the primary care interval required that the referring GP completed the one-page form about the urgently referred patient. The CME-M practices were more likely to complete one-page forms (Table 2), whereas the reference practices lost compliance over time. We do not know whether this was a differentiated non-response (e.g. in cases with long primary care intervals), but it could be the case as also indicated by the paradoxical result. However, the proportion of patients in the “harder to detect” group of referrals actually increased in the reference group after the CME-M [39].

To investigate the reasons for the observed increase in the primary care interval combined with an indication of lowered referral threshold, we performed sensitivity analyses. These did not point towards a delayed effect of the CME-M or a “cleaning up process” of patients who had recently been seen in general practice because of unspecific symptoms. The most likely explanation of our finding is a change in the awareness of unspecific symptoms among GPs in CME-M practices, which could have led the GPs to report an earlier date of first symptom presentation. Thus, the GPs may have gained a more realistic idea about the usual time intervals in general practice for patients suspected of cancer.

Comparison with other studies and clinical implications

To our knowledge, this study is among the first to evaluate a CME-M on GPs’ use and timing of urgent cancer referrals. In accordance with our findings, an English before-after study found that the use of RATs for lung and colorectal cancer in general practice was associated with an increased number of urgent referrals during a 6-month period [43]. The reason for this could be that the GPs’ awareness of potential cancer symptoms and attitude towards urgent cancer referral changed through the CME-M. Similar results have also been found in qualitative studies about the use of RATs [44, 45] and in our previous analysis of the impact of the CME-M on GP knowledge, attitude and intentions [18].

Our intervention was developed on the basis of theoretical frameworks and reviews of empirical results to ensure optimal likelihood of positive effect [44–47]. The CME-M consisted of a variety of elements [18], and we could not validly identify the most effective parts of the CME-M; this calls for further research in changing the GPs’ behaviour [46].

In our study, the non-participating practices were more often single-handed, the GPs were more often males, and their practice population was larger, more deprived and with more male patients than in participating practices. Similar findings were also reported in a comprehensive quality development project on chronic diseases conducted in the Central Denmark Region in 2010 [47]. These findings may result from differences in the GPs’ possibility to participate (e.g. time constraints) [48]. However, it may also be explained by the theories of varying readiness for change of behaviour [49]; some GPs belong to a group known to be susceptible for new knowledge (innovators, early adopters and early majority), whereas other GPs are known to be sceptical (late majority and laggards). One way to change the pattern of non-participation could perhaps be to develop specific CME initiatives to better fit the GPs’ preferences for education and individual learning style [50].

Conclusion

We found a statistically significant reduction in the number of total contacts with general practice within the month preceding an urgent referral and an increase in the reported primary care intervals from before to after in the CME-M group compared to the reference group. A CME-M may contribute to changing the threshold for referring the patients and the GPs’ understanding of the primary care interval.