Three thousand five hundred ninety seven reports were identified from the electronic search (2753 after removal of duplicates) (Fig. 1) plus 28 from the grey literature search. After screening and eligibility assessment, 51 studies were included, describing 42 interventions. Seventeen studies were from the USA, twelve from the UK, eight from Australia, five from Canada, four from Norway, two from both Japan and New Zealand and one from Chile (Table 1).
The length of follow up ranged from one to 32 years. The number of participants included in the studies ranged from 7 to 2988. Sample sizes were generally small and 7 studies had less than 20 participants. In 8 studies the outcome was the self-reported location where the trainee or doctor was practicing after the intervention. Some studies used national databases or practice address as the outcome measure.
There were no randomised control trials (RCTs). 38 used a cross-sectional design, of which 30 did not include a comparison group, and 8 had a between-group comparison. Thirteen studies were a longitudinal design, of which six lacked a comparison group. Of the seven longitudinal comparison studies, one was a before and after comparison and six compared two parallel groups.
The representativeness of the included participants was generally good, however the absence of a comparison group resulted in a high risk of bias in many studies (Table 2). Assessment of the outcome and follow-up was generally low risk of bias. Most studies were described in an adequate or detailed manner and had potential or good generalisability. 21/51 of the included studies had a conflict of interest; primarily the study authors who undertook and evaluated the intervention were part of the same organisation that delivered the intervention.
Interventions could be broadly categorised into 13 groups: retainer schemes, re-entry schemes, support for professional development or research, specialised recruiters or case managers, well-being or peer support initiatives, recruiting rural students, rural or primary care focused undergraduate placements, rural or underserved postgraduate training, marketing, delayed partnerships, international recruitment, financial incentives and mixed interventions. Results are presented from strongest to weakest evidence.
The strongest evidence was for financial incentives, eleven studies evaluated interventions which provided financial incentives in return for an obligation of service [21–31]. Six studies had a comparison group [21, 22, 24, 28, 29, 31]. Two Japanese studies examined a strategy which obligated students to a nine year service agreement in their home region in exchange for fully funded undergraduate training (medical school) [21, 22]. After the nine years, students were 4.2 times more likely (no statistical significance reported) to work in rural areas compared to non-obligated students . A comparative study of financial incentives compared with no financial incentives in West Virginia found similar retention rates after the obligation period (32 % 14/44 vs 38 % 41/108), with no statistical significance reported . A study of five separate financial incentives found that retention rates were statistically significantly higher for obligated than non-obligated doctors (Hazard ratio 0.70 95 % CI 0.51 to 0.96) . Three studies, only one of which had a comparison, assessed the National Health Service Corps (NHSC) scheme in the USA which used financial incentives, loan repayment or scholarship throughout their medical education [23–25]. The only one of these studies with a comparison group showed that NHSC participants had a lower retention rate compared to non-NHSC participants (29 % versus 52 %, p value < 0.001) . One study found doctors were 3.2 times less likely to leave an underserved area if they were fulfilling a service obligation in repayment for a funding during medical school or during postgraduate training .
The remaining studies did not have a comparison group and were therefore difficult to draw conclusions from. A postgraduate voluntary bonding scheme in New Zealand which recruited trainees to hard-to-staff communities for five years, with payments starting after the third year and no penalty for withdrawal, found that 89 % of graduates had opted out of the scheme three years after entering .
Recruiting rural students
Evidence to support recruiting rural students was also found. Six studies, only one of which had a comparison group, evaluated recruiting rural students to medical school, with the expectation that some would return to their home town for practice [32–37]. The comparative study found that 68%were still practicing family medicine in the same rural area up to 16 years after graduating compared to 46 % in the comparison group (p = 0.03) . While the remaining five studies [33–37] found that a large proportion of individuals recruited from rural areas subsequently work in rural areas (one study reported up to 90 %) , the lack of a comparison group makes it difficult to determine what would have happened if recruitment from rural areas had not taken place.
Four studies (three without comparison groups) evaluated international recruitment schemes [38–41]. Three initiatives waived certain visa or work requirements to enable IMGs to work in USA or Australia if they agreed to work in rural or underserved areas for an obligated period of up to ten years [39–41]. The comparative study from USA found doctors recruited entering practice without J-1 Visa Waivers in rural communities had a significantly higher retention rate than their visa waiver colleagues (p < 0.001) . These schemes recruited IMGs with varied retention rates. All studies reported success in recruiting international doctors (range 7 to 145), but the three lacking of a comparison group were difficult to draw conclusions [38–41]. Three studies found that a significant number of IMGs did not stay in rural practice (73 % 19/26) , did not complete the three year obligation period (30 %, 22/72)  or did not work beyond the initial years contract (19 %, 2/7) .
Rural or primary care focused undergraduate placements (i.e. undergraduate placements refers to placements during medical school)
Three studies from the USA looked at rural undergraduate placements in primary care settings [42–44]. One comparative study found that 23 % (156/677)  of individuals with rural experience during their undergraduate were practicing in rural areas compared to 12 % (32/260)  of students without (statistical significance not reported). One study found a higher percentage of graduates with rural exposure in medical school subsequently worked in rural areas than those without (n = 1393, 26 % vs 7 %, p < 0.001) . The final study reported a high proportion of students practicing primary care in rural areas after rural placements, but without a comparison group it is difficult to draw conclusions .
Rural or underserved postgraduate training
Three studies evaluated postgraduate training in rural/underserved areas [45–47]. One comparative study from the USA found that doctors who were trained in a community health centre serving underserved communities were statistically significantly more likely (odds ratio 2.7, 95 % CI 1.6 to 4.7) to work in underserved areas compared to doctors who had not . Two studies from Australia did not have a comparison group but one reported that a small percentage (14 %) of individuals reported that they were influenced against rural practice after their placements .
Well- being or peer support initiatives
Three studies provided social and emotional support to rurally isolated doctors [48–50]. One Australian study using a before and after comparison found a moderate reduction of 5 % (98/187 compared with 102/221, statistical significance not reported) in those planning to leave rural practice after a support initiative was introduced . Two studies from northern Norway reported on a tutorial group which primarily provided support for postgraduate
doctors serving an internship in a rural area [49, 50]. The authors found good recruitment (twice as many as expected)  and retention (65 % five year retention) , but the results were confounded by place of graduation and growing up in that area making it impossible to disaggregate the effects of the tutorial group.
Two studies evaluated marketing strategies for recruiting residents to a primary care training program [51, 52]. A promotional video marketing in the USA was associated with lower recruitment with only 29 % (35/120) of those receiving the video applied compared to 54 % (69/128) of those who did not (p < 0.0001) . In a non-comparative study, 48 % of trainees recruited in the North of Scotland study stated that a blog which posted views and experiences of current primary care trainees positively influenced their choice of location for primary care training .
Weak evidence was found for mixed incentives. Five small studies [53–57] evaluated mixed incentives, combining continued medical education, financial and undergraduate placement incentives to recruit and retain doctors, with mixed results. None of the studies included a comparison group and therefore it is difficult to draw conclusions about either individual components or the intervention as a whole.
Two of these studies evaluated one scheme in Alberta, which used financial incentives, CME and rotations aimed at undergraduates, postgraduates and currently practicing doctors [56, 57] 35 % indicated the scheme had a critical or moderate effect on their decision to move or stay in Alberta but after the scheme was initiated the number of rural primary care doctors in Alberta actually reduced .
Support for professional development and academic opportunities
Five studies, without a comparison group, focused on interventions which aimed to provide primary care doctors with an increase in academic skills, particularly in teaching and research [58–62]. Three studies reported the London Initiative Zone Educational Incentive scheme (LIZEI) aimed to improve recruitment, retention and refreshment of London GPs via various schemes which focused on increasing the academic aspect of training through academic/research associate schemes [58–60]. The scheme reported high levels of retention (75 % from the London Academic Training Scheme (LATS) cohort continued to practice in London) , but the lack of a comparison group makes this difficult to interpret, two further studies echoed this result, and found high levels of retention in London [59, 60]. The two other studies were from Australia [61, 62]. One found that 80 % of attendees (341/426) of continuing medical education (CME) workshops reported that they were less likely to remain in rural practice without CME . The other study lack sufficient detail to allow interpretation of the results .
Four studies, without a comparison group, assessed retainer schemes in the UK, including the Women’s Doctors Retainer Scheme [63, 64], the GP retainer scheme  and the Doctors’ retainer scheme . Retainer schemes allow primary care doctors to work reduced hours (a maximum of four sessions a week, a minimum of one) with an educational component, for up to five years. Participants from a retainer scheme in Scotland reported that the scheme prevented them leaving medicine (32 % of former members (33/104) and 46 % of current members (69/152) . All studies showed high retention of primary care doctors from retainer schemes (86 % (91/105) , 91 % (33/36)  and 71 % (10/14)  but the lack of a comparator group made it difficult to draw conclusions about the effect of the scheme.
One small study  with a comparison group evaluated a re-entry scheme, developed to help doctors to return to general practice as a partner (i.e. partner is a term used in the UK for a GP who makes a financial investment into a practice, and can therefore benefits from any profits (or losses) it makes, they must also oversee how the practice is run). The scheme rebuilt their confidence using needs based tutorials. Six months after the course 2 out of 14 attendees had returned to working as partners. 11 out of 14 attendees had taken ‘positive steps’ (this was not explained further) to return to general practice or had increased their time commitment to medicine. Compared with 1 in the control group (comparison group denominator not reported), who had ‘made plans’ to return to general practice . The numbers were too small to draw conclusions.
Weak evidence was found for the value of delayed partnerships. Two studies without comparison groups looked at delaying partnership after GP training by adding up to two years of post-vocational training [68, 69]. This included sessions at a mentor practice gaining general experience, varied locum experience and protected time for further training education . Another scheme added one year of extra training which included exposure to the financial, managerial aspects of partnership, as well as clinical time . The lack of a comparison group made it impossible to draw conclusion about the effectiveness of delayed partnerships.
Specialised recruiter or case managers
The weakest evidence was found for specialised recruiters or case managers. Two cross sectional studies (non-comparative) used specialised recruiters or case managers to recruit doctors to rural areas [70, 71]. They provided a holistic approach to recruitment, identifying any particular needs of the doctor, helping to support them through the transition and encouraging community development activities. While both studies reported successful recruitment (17 doctors in 18 months  and 8 primary care providers in two years ), the impact of a case manager is unclear without a comparison group. A bachelor’s degree in a health related field plus health related work experience (two years minimum) was required for the specialised recruiter post in the USA, but was not compulsory .