Key messages

  • The National Health Service Corps (NHHS) Loan Repayment Program has an important role in placing health professionals in medically underserved areas.

  • NHSC clinicians’ experiences at their service delivery sites were more strongly associated with clinicians’ future intentions, compared to their individual and community-level characteristics.

  • Providing NHSC clinicians with mentorship, peer support, and technical assistance during their service could increase the retention of these professionals in underserved areas following service completion.

Introduction

Communities across the United States (U.S.) are experiencing health professional shortages, making the retention of existing professionals a key priority. In a 2022 report from the National Association of Community Health Centers, 68% of health centers reported losing 5% to 25% of their workforce in the previous six months, and 15% reported losing 25% to 50% of their workforce [1]. The loss of these health care professionals in underserved areas exacerbates existing shortages. Rapid changes in the health care system, intensified by the COVID-19 pandemic, have contributed to increased burnout of health professionals and high turnover rates [2]. Workforce shortages among behavioral health care providers also reduce the country’s ability to respond to the opioid crisis [3]. The U.S. Health Resources and Services Administration (HRSA) Bureau of Health Workforce (BHW) develops programs that help improve the supply and distribution of the health workforce. In 2018 and 2019, Congress appropriated additional funding to the National Health Service Corps (NHSC) for the express purpose of expanding and improving access to quality opioid and other substance use disorder treatment in rural and underserved areas nationwide. HRSA utilized this funding to expand the number of clinicians delivering medications for opioid use disorders and other substance use disorders through the implementation of two new NHSC loan repayment programs (LRP): the NHSC Substance Use Disorder (SUD) LRP and the NHSC Rural Communities LRP. The NHSC LRP offers primary care medical, dental, and behavioral health clinicians who completed their professional training the opportunity to reduce qualifying educational debt accrued during their training in exchange for providing health care at NHSC-approved sites such as facilities in underserved areas [4]. As part of an independent evaluation of HRSA’s investments in the NHSC expansion, this study sought to identify whether there were patterns in clinicians’ experiences at sites using workplace environmental measures of resources, supports, technical assistance, and challenges and whether these patterns were associated with clinicians’ intention to remain working in underserved areas following the completion of their service. In addition, we aimed to understand if clinicians’ demographic characteristics and characteristics of the communities where they served were associated with patterns of experiences at sites.

Prior work has demonstrated the benefits of the NHSC program, including expanding behavioral health care services at NHSC sites, particularly in rural areas [5]. Community health centers with NHSC behavioral health clinicians can provide primary and behavioral health care services at lower costs compared to health centers with non-NHSC clinicians, especially in rural areas and NHSC clinicians increase the overall number of mental health care services a facility [6, 7]. NHSC clinicians may also enhance the recruitment of other essential health care providers, thereby increasing the overall staffing capacity at health centers [7].

However, prior studies on retention have had limited analysis of organizational, and community-level factors that might affect retention [8, 9]. For example, researchers have used physician demographic and socio-economic background characteristics [10, 11] and facility characteristics to predict retention [12]. Surveys of NHSC clinicians have examined associations between clinicians’ work experiences and satisfaction to ascertain if these factors were important to understanding and reducing turnover [13,14,15].

This analysis updates prior work related to NHSC clinician retention and provides the first analysis of intentions among clinicians in the new LRPs: Substance Use Disorder and Rural Community, to understand whether there are patterns of clinicians’ experiences at sites and whether such patterns are associated with their intention to remain working in underserved areas. Adding to previous research, this study includes measures of work environments, such as the availability of mentorship, work/life balance, and caseloads, recognizing the important role that these factors have on clinician burnout and retention [16,17,18,19]. National policy frameworks, such as the National Academy of Medicine’s “National Plan for Health Workforce Well-Being,” recognize that workplace demands and resources contribute to burnout or well-being [20, 21]. Thus, this analysis includes both individual socio-demographic factors noted in earlier studies, such as race/ethnicity, experience living in rural or other underserved areas, the desire to work with underserved populations, and attributes of their work environment and community characteristics. New analyses that examine how the workplace environment is correlated with workplace retention are also important as the health care workforce has changed rapidly due to the convergence of the COVID-19 pandemic, the opioid epidemic, and significant issues with provider burnout.

Data and methods

Data sources

Administrative data

We obtained application data for new NHSC LRP clinicians from FY 2019, FY 2020, and FY 2021 NHSC LRP clinicians from HRSA’s internal BHW Management Information System Solution. The clinician application data provided four measures of clinicians’ background demographic characteristics such as gender, underrepresented minority, disadvantaged background, and rural background status. Please see Supplemental Table S1 for the background characteristics of the clinicians in the Standard, SUD, and Rural Community LRPs. The application data also contained federal codes for the counties where clinicians served which were used to link to publicly available data (described below).

Web-based surveys

NORC, the independent evaluator, designed, tested, and fielded three web-based surveys of NHSC clinicians. The three cross-sectional surveys were fielded to new FY 2019-FY 2021 NHSC LRP participants in the fall of each year from 2020 to 2022. (Each respondent participated in only one survey; there were no duplicate responses). The survey was pilot-tested with six NHSC clinicians from different regions of the country before the first round of fielding. NORC survey experts and external subject matter experts also reviewed the survey. The pilot subjects and experts provided feedback on the survey grammar, question length, and question format. All responses were completely confidential within NORC. Clinicians did not receive any financial incentive for participation and their participation was voluntary. The average NHSC clinician survey response rate was 61.7%. Response rates were calculated using the American Association for Public Opinion Research Response Rate 2 [22]. Approval for the survey was obtained from NORC’s Institutional Review Board. The surveys and their administration were approved by the U.S. Government’s Office of Management and Budget; the questionnaires, as well as the outreach materials, are available online [23]. Additional details about the survey development, fielding, questions used in this analysis, links to the surveys, and response rates are provided in Supplemental Material S7.

The dependent variable was the intention to remain working in an underserved area, a dichotomous measure constructed from an affirmative response to one of two survey questions regarding whether the clinician planned to provide patient care following service completion: (1) at an NHSC site; or (2) at a non-NHSC site in an underserved area. The survey measures selected for analysis were questions about clinicians’ experiences at sites concerning resources, support, technical assistance, and challenges to providing care (Supplement S7 provides the specific survey questions). Our final analysis included 9341 respondents who answered this question and responded to the questions about site experiences.

Public data

To examine the characteristics of counties where clinicians in the study served, we used the Social Vulnerability Index (SVI) from the Centers for Disease Control and Prevention and the Agency for Toxic Substances and Disease Registry and the University of Wisconsin Population Health Institute’s County Health Rankings [24, 25]. These data were used because aspects of the community could affect a clinician’s decision to remain working in an underserved area. While clinicians entering the NHSC are aware that they will work in underserved areas, there may be variations in community resources, such as affordable housing, and recreational, educational, and economic opportunities for their families. The County Health Ranking data provided a within-state ranked value based on each county’s score for six composite indexes that reflect conditions that influence health and well-being: (1) length of life; (2) quality of life; (3) health behaviors; (4) social and economic factors; (5) environmental factors; and (6) access to care.

Analyses

We employed a latent class analysis (LCA) to identify typologies of clinicians’ experiences at sites, and the association of these typologies with the intention to remain working in an underserved area and with individual-level background characteristics. To select measures for the LCA, we first conducted bivariate regression analyses to test for statistically significant differences (at P < 0.05) between those who do and do not intend to remain working in underserved areas for each of the measures examined for inclusion in the LCA. The measures tested for inclusion in the LCA were the survey measures on workplace experiences (described above), individual-level background demographic measures, and community characteristics as measured by the SVI and the county health ranking data.

Second, using the measures from the bivariate analyses that were most strongly associated with the intention to stay, we implemented a series of latent class models to understand whether there are typologies of clinicians’ experiences such as whether there were patterns for results to questions about site experiences). The LCA method has the advantage of using statistical modeling to classify persons based on response data. The accuracy of the classification can be assessed by statistical fit indices and with other theoretical and empirical understanding of how measures may cluster together. We used Bayesian Information Criterion (BIC) to compare model fit, and entropy to understand how well the classes were distinguished.

To identify the number of classes, our initial LCA models used measures for which there was at least an 8% difference between clinicians who do and do not intend to serve. This threshold was used to find the fewest number of covariates that drive class sorting, avoid data redundancy and highly correlated variables, and overfitting the model which could result in finding rare, spurious classes [26, 27]. To refine the model fit and improve entropy, we incorporated additional measures for which there was a 7% or greater difference between the two groups of clinicians (on the intention to serve). The results of the LCA assign each respondent to a latent class; the assignment of individuals to classes is probabilistic. All the variables selected for inclusion for fitting the LCA had Spearman's rank correlation coefficients (rho) of less than 0.4.

We also used the same measures selected for the final LCA model to conduct additional LCA, stratified by discipline, since experiences at sites may vary by discipline.

Following the LCA, which assigned individuals to classes, our third step was to characterize each class using bivariate logistic regression, with class membership as the outcome variable, to examine how experiences and background characteristics varied across the classes, and to test the association of each class with the intention to stay. Regression models included survey weights that adjusted for nonresponse across the three loan repayment programs. While the overall average response rate was 61.7%, clinicians in the standard LRP had a lower average response rate (59.9%) compared to the SUD and Rural Community clinicians (67.5% and 65.9%, respectively). These weights allowed survey results to reflect the distribution of the overall loan repayment program population, by program participation. Results were also adjusted for the false discovery rates (FDR) [28]. All analyses were conducted using Stata SE 17 for Windows (Stata Corp, College Station, Texas).

Results

Overall, three-quarters (78.7%, N = 7192) of NHSC clinicians who were included in the LCA reported they intended to continue to work in an underserved area. Table 1 shows the results for the variables selected for inclusion in the LCA (those with at least a 7% difference between groups). It shows the proportion of affirmative responses to the measures that most differentiated between those who intended to stay and those who did not, the test for significant differences between groups, and the size of the difference. The table is sorted by the size of the difference (see Supplemental Table S2 for a similar table for the results of the remaining measures that were not selected for inclusion in the model).

Table 1 Measures with largest differences between NHSC clinicians who do and do not intend to remain working in a medically underserved area following service completion
Full size table

Table 2 shows the results of the latent class model and reports the goodness-of-fit indices for the four models used in our analyses. The largest change in Bayesian Information Criterion (BIC) occurred between the one-class and two-class models. The BIC further decreased as more classes were added to the model, however, the change was smaller with each additional class. Further, the BIC will decrease as the number of classes increases, but can reflect patterns unique to the data and are of limited importance. Entropy also decreased with additional classes, suggesting less differentiation. Because our conceptual basis was understanding whether there was a general dichotomy between positive and negative experiences at sites, we selected the two-class model. However, because statistical tests indicated more than two patterns in the data were possible, and to verify our hypothesis that classes could be distinguished based on measures of site experiences, we examined the characteristics of the four-class Model b. This model had a lower BIC but also had lower entropy (see Supplemental Table S3 for the four-class model characteristics and tests for significant differences between the class with the highest proportion who intend to serve and the other three classes).

Table 2 Latent class model fit statistics
Full size table

The distribution of classes is shown in Table 3. A significantly larger proportion of class 1 intends to remain working in an underserved area (following completion of their service obligation), although this class represents a smaller overall proportion of the survey population.

Table 3 Class distribution and intention to serve
Full size table

Figures 1, 2, 3, 4 show a series of charts that demonstrate class variation in site experiences. Please see Supplemental Table S4 for the values shown in the graphs and FDR-adjusted p-values for the test for significant differences between groups. Clinicians who intended to remain working in an underserved area reported more of each type of support and resources available at their site, received more technical assistance, and had fewer needs for technical assistance and fewer challenges. We also note there was substantially smaller variation between classes in the demographic measures of underrepresented minority and disadvantaged background status, female, and being from a rural area. We also conducted multivariable logistic regression analyses (with each of the background characteristics in the model) to examine the association of these background characteristics with the intention to remain, and found that there was no association between underrepresented minority and disadvantaged background status with the intention to remain. We observed that females and persons from rural areas were more likely to intend to remain working in underserved areas. Supplemental Table S5 and Supplemental Table S6 show results of the differences between classes for the top 10 reasons, stratified by discipline.

Fig. 1
figure 1

Percent of clinicians who needed or received or technical assistance, by class membership (*indicates significant differences between Class 1 and Class 2 on all items in the chart; IT information technology; FY fiscal year)

Full size image
Fig. 2
figure 2

Percent of NHSC clinicians with resources, supports, and challenges at NHSC sites, by class membership (*indicates significant differences between Class 1 and Class 2 on all items in the chart; ^Other benefits may include paid parental leave or life insurance; SUD substance use disorder; FY fiscal year)

Full size image
Fig. 3
figure 3

Percent of NHSC clinicians who reported reasons to apply to the LRP and to the NHSC site, by class membership (*indicates significant differences between Class 1 and Class 2 on categories shown in the chart; LRP loan repayment program; FY fiscal year)

Full size image
Fig. 4
figure 4

Background characteristics of NHSC clinicians and influences on the decision to continue to work in underserved area, by class membership (*indicates significant differences between Class 1 and Class 2 on categories shown in the char; FY fiscal year)

Full size image

Discussion

Findings point to the importance of receiving technical assistance at sites, as classes varied significantly in receipt of technical assistance for clinical issues (such as safety and quality) and health system financing, as well as mentorship and support for professional development opportunities. These factors were significantly more common among those who intend to remain working in underserved areas. NHSC sites could consider expanding opportunities for NHSC clinicians’ professional development through mentorship programs, encouraging participation in teaching, or supporting residency and training programs.

Challenges at sites, including inefficient management practices, high caseloads, and insufficient team-based care, were more common among clinicians who do not intend to work in underserved areas; improving these issues could help improve retention. For example, decreasing administrative burden and improving clinician efficiency by providing additional administrative and support staff for team-based care or implementing policies that address care providers’ workloads and workflows could improve work-life balance and mitigate workforce-related stressors [29,30,31,32,33]. Staff shortages at all levels, from clinical providers to administrators to support staff, can affect provider burnout, but addressing these issues also requires policy and systems-level changes [34].

A competitive salary was more commonly reported (a 5% difference) among clinicians who intend to remain working in underserved areas, although the difference between groups on this factor was smaller than measures of workplace experiences. Data from the Multi-State Clinician Retention Collaborative indicated 81% (n = 966) of NHSC clinicians (including clinicians in the NHSC Scholarship Program) reported they were satisfied with their work and practice, but fewer clinicians were satisfied with their compensation (51%, n = 608) and workload (36%, n = 429) [13]. From our study, we cannot draw conclusions about the role of salary in influencing clinicians’ decisions to remain working in underserved areas.

This study supports the health workforce knowledge base that demonstrates the important role of the workplace environment on retention [20, 21]. In addition, our study shows that while the financial incentive of the loan repayment influences NHSC clinicians to apply to the program, their service delivery experiences are strongly associated with their intention to remain working in underserved areas, an important finding for improving health workforce retention in underserved areas. Our findings also partially support earlier findings which demonstrated that NHSC clinicians are strongly mission-focused and their desire to work with underserved populations is a significant predictor of future intentions to continue working in these areas. We also find that, unlike earlier analyses, demographic characteristics (underrepresented minority and disadvantaged background status) were not associated with the intention to stay. This finding suggests clinicians from diverse backgrounds and working in many different communities may predict working in underserved areas only if work conditions are positive and supportive.

Limitations

The results cannot be construed as causal because clinicians’ intentions could affect how they perceive their service experiences. The main outcome was dichotomized based on affirmative responses to the questions on intention to stay—respondents who answered “No” and “Don’t know” were coded into one group. Latent class membership cannot be empirically tested and misclassification error is inherently part of the LCA. There may still be unmeasured site and individual-level factors that affect a clinician’s intention to stay in an underserved area. Furthermore, while our study uses measures of clinicians’ experiences at sites to classify respondents, in addition to their direct responses to a question on what factors influence their intention to stay, self-reported data may be subject to certain biases. For example, saliency or actor-observer bias could influence clinicians’ perceptions of challenges or resources at sites, and other factors could similarly influence their intention to stay decision [35]. Further study that captures measures that influence clinicians’ decision-making over time and their association with the resulting outcome (to leave or stay) may be useful for program operations and investments. Findings may be most generalizable to clinicians who participate in loan repayment programs and/or work in underserved settings. Finally, future work should more closely examine the role of community resources and characteristics, such as housing, educational and economic opportunities for family members, in the retention of health care workers. While county-level measures of community health and social needs were not strongly associated with clinicians’ intention to remain, our analyses were limited in scope. We explored these concepts using county-level data, but a more specific investigation of the availability, quality, and accessibility of these resources and their influence on clinicians’ decisions could inform policy choices to support federal, state, and private investments in these rural communities.

Conclusions

The findings from this study emphasize the importance of supportive work environments for the retention of NHSC providers in underserved areas. While the majority of NHSC LRP clinicians are committed to serving underserved populations, this values-based retention factor can be strengthened by enhancing their work environment through resources for mentorship and peer support, technical assistance, professional development opportunities, and addressing challenges stemming from high caseloads and inefficiencies in care delivery through technical assistance to site leadership for organizational development. NHSC providers are located in underserved areas where the number of providers is by definition, low, as is the likelihood of other support staff—this workforce constraint can contribute to increased stress and burnout [36, 37]. Staffing issues are critical to address to provide the team-based, integrated care services that many NHSC sites typically champion. Comprehensive staffing models would enhance support for clinicians but also enable more efficient and higher-quality health care for the patients and clients they serve. However, the National Advisory Council on the NHSC noted that these improvements require policy changes at the health system level, in addition to strengthening the health workforce pipeline [38]. Future studies could identify how the factors that influence retention vary for clinicians with different types of disciplines and for administrative staff who are integral to health care organizations in underserved areas. Examining such analyses in concert could suggest how to create successful models of retention, which could be examined for replicability by other organizations serving vulnerable populations.