Background

Many countries across the world have serious health workforce challenges [1]. In 2020, a global shortage of 15.4 million health workers with huge skill mix imbalances and maldistributions was reported [2]. Sub-Saharan Africa (SSA) is disproportionally affected, with relatively few health workers and a high burden of disease [3, 4]. The density of physicians in SSA countries (< 0.3/1000 population in 2018) was very low as compared to the high-income countries (2.0–5.0) [5, 6].

Like many African countries, Ethiopia invested to increase the number of healthcare workers. It has been implementing the national human resources for health strategic plan which set a goal of increasing the stock of physicians by five-fold from 5411 in 2016 to 28,121 in 2025 [7]. As a result, medical schools have been expanded from 5 in 2005 to 43 in 2022, including 10 private colleges [8]. The annual number of graduates has increased tenfold and reached 1500 – 1600. Despite the positive strides, addressing the health workforce challenges in the country is far from finished. For instance, the density of physicians was still low, at 0.1 per 1000 people [9].

Increasing the number of training institutions can only improve population health when the quality of training is ensured. Workforce quality is an important part of the solutions to the global human resource crisis. Enrolment of so many students with limited medical school adaptations has fueled the quality issues in the country [10,11,12]. Despite the commendable efforts in expanding medical education, Ethiopia has lagged behind the WHO’s recommendations in reforming and implementing medical curricula, expanding student clinical sites and simulation-based training, and strengthening accreditation [13]. There have been shortages of learning resources and experienced faculty [14, 15]. With the rapid expansion, the training quality concerns have further deepened which might have affected student learning. Practice analysis of Junior physicians also reported substantial clinical skill gaps [16].

The effect of the training quality gaps on the competence of medical students is not well studied. Therefore, we conducted a study to assess the clinical competence of graduating medical students and associated factors.

Methods and materials

Study design and period

This pretest assessment is part of a quasi-experimental study aiming to assess the impact of project interventions in improving the quality of medical education in Ethiopia. Considering the complexity of the medical education environment for a random control study, we used a quasi-experimental design which is well suited to examine the cause-and-effect relationships among variables [17]. A posttest part of this quasi-experimental study will be repeated in 2025. Changes in the clinical competence and the associated factors from the pretest level will be used as evaluation measures. This pretest study was carried out in July and August 2022.

Study setting and study population

There were 43 medical schools in Ethiopia, including 33 public and 10 private schools. Most medical schools admit high school graduates using a direct entry scheme while some accept graduates in the health and other science fields using a graduate entry scheme. The duration of medical education is 6 academic years, including 1 year of internship at the end. Starting from the third academic year, medical education is provided at hospitals and other clinical settings. The study population for this study was the 1556 undergraduate medical students who completed or were nearly completing their internship program and expected to graduate in 2022.

Sample size and selection criteria

Ten schools that had graduating classes around the same period were selected in consultation with the Ethiopian Medical Association and the Ministry of Health. Of the 10 selected schools, four employed graduate entry schemes, and two were private schools. About 875 medical students were expected to graduate from the 10 schools in 2022. To determine the sample size, we considered a 95% confidence level, 80% statistical power, 1:1 optimal allocation (sample ratio of intervention to comparison), a moderate effect size of 0.5, and a design effect of two. We calculated the minimum sample size of 218 graduating students. After including a non-response rate of 10%, the final sample size was found to be 240.

Sampling procedures

To include 240 students from the 10 medical schools, we considered recruiting 24 students per school. Because our study aimed at observing students’ competence using a 12-station objective structured clinical examination (OSCE), including 24 students ensured better competency observations and the attainment of adequate data points at each school. To develop the sampling frame, we requested the lists of graduating medical students from the deans’ offices. Using a lottery method, we randomly selected 24 students. We provided the lists of students to the assessors (data collectors) who invited students to participate in the study. If the selected students were not willing to participate for any reason, the data collectors thanked them and did not replace them.

Measurements and instruments

The key variables of interest were clinical competence, clinical learning environment (CLE), simulation training, and practice exposure. We assessed clinical competence using a 12-station OSCE, a reliable method to assess clinical skills [18, 19]. Using global and national standards, OSCE case scenarios, assessment rubrics, and assessor instructions were developed [20,21,22]. The stations focused on the core competencies required for the provision of safe medical care which included 10 manned stations: taking a focused history, conducting pericardium examination, providing patient education and counseling for diabetes mellitus, conducting Leopold’s maneuver, conducting manual vacuum aspiration (MVA) for incomplete abortion, managing childbirth, performing wound suturing, taking emergency response for polytrauma, obtaining consent for hernia repair, and performing lumber puncture (LP). The remaining two stations were unmanned and focused on interpreting chest X-rays and complete blood cell counts for tuberculosis (TB) patients; and prescribing medication for a malaria case. To ensure content validity, we reviewed the stations with both subject matter experts and medical educators. The assessment rubrics had 4 to 7 items and followed a five-point global rating scale (GRS), where 1 meant “poor performance”, 2 “unsatisfactory performance”, 3 “satisfactory performance but not good”, 4 “good performance”, and 5 “excellent performance”. To assess the clinical learning environment, we used a validated clinical learning evaluation questionnaire (CLEQ) [23], a tool for measuring a learning climate in the clinical settings for undergraduate medical students with 18 items organized in four domains: clinical cases, motivation of students, supervision by preceptors, and clinical encounter organization. Similarly, we developed another 10-item structured tool to assess the quality of simulation training using guidelines and literature [24, 25]. Students self-assessed their experiences on each item of the two questionnaires on 5 points Likert scale, where 1 meant strongly disagree, 2 disagree, 3 neutral, 4 agree, and 5 strongly agree. In addition, we developed a structured tool to determine the practice exposure of students to 12 task procedures in the past 12 months. The list of procedures was adopted from the national scope of practice and training curricula. There were also variables about the background characteristics of medical students.

Data collection

The OSCE was administered by 18 physicians and medical educators. A two-day training was given to assessors on data collection procedures, tools, ethical principles, data quality assurance, and the CommCare software application. Assessors informed study participants about the purpose, procedure, and ethical principles of the study and obtained consent. Study participants completed the CLEQ, simulation training quality, and practice exposure questionnaires. The study participants were encouraged to provide accurate information and/or the best plausible response to each item. For the OSCE stations, the data collectors made sure that all essential logistics (standardized patients, models, medical equipment, medical supplies, assessor instructions, and assessment rubrics) were available. The data collectors asked the study participants to undertake the required tasks at each OSCE station within 12 minutes. They conducted direct observations of students’ performance and rated them exclusively using the GRS. The data collectors were also closely supported by 6 supervisors to check errors and omissions. The OSCE assessment rubrics had a total of 64 items and an average Cronbach’s alpha value of 0.79 with a range of 0.62–0. 89 (Table 2).

Data management and analysis

We exported the data from CommCare v. 2.53 to SPSS v. 27 for data cleaning and statistical analyses. Summary statistics were computed for all key variables to check outliers and missing data. Cronbach alpha coefficients were computed to assess the consistencies of items listed in each competence. Means, medians, standard deviations, proportions, tables, and graphs were computed. Mean scores for the 12 competencies and the overall composite mean score were computed. Mean satisfaction scores on CLE and simulation training were also calculated. To conduct desired statistical tests using continuous quantitative variables, we decided to merge many items of CLE and simulation into a single one by transforming the Likert scale data into composite mean scores [26]. The five-point Likert scale measures of CLE and simulation training were also grouped into two categories (strongly agree and agree as “agree”, and strongly disagree, disagree and neutral as” disagree”), and proportions were conducted to give a meaningful interpretation. Proportions were calculated for practice exposure. Since the curriculum did not specify thresholds for the number of clinical procedures expected to be performed, the median for each procedure was used as a cutoff point to decide between high and low exposures. The median values were used as measures of central tendency since the data had outlier values. An independent sample t-test was used to make comparisons between male and female students, private and public schools, direct and graduate entries, students with high and low clinical exposures, and students with high and low CGPA. We also checked the necessary assumptions for regression analysis and ensured the model’s adequacy [27]. Bivariate and multiple linear regression models were then fitted. The outcome variable was the competence score. The independent variables were age, sex, school type, cumulative grade point average (CGPA), school entry scheme, composite satisfaction scores for the four CLE domains, and simulation training. We considered all independent variables with P < 0.025 at the bivariate level for inclusion in the multivariable regression analysis. A 95% statistical confidence interval and p-value < 0.05 were used for making statistical significance decisions. Students are expected to master essential skills for safe and beginner-level healthcare delivery at the point of graduation. A 75% cut-off score which is recommended in mastery learning was used in comparing the competence scores [28].

Data quality assurance

We adopted a standardized CLEQ data collection tool to assess the clinical learning environment. In the case of no standardized tools, the questionnaires for OSCE, simulation training, and practice exposure were reviewed and validated by medical education experts. We recruited senior medical education experts who have experience in conducting OSCE as data collectors. A two-day training was provided to data collectors to standardize data collection. Study investigators along with supervisors ensured that quality data were collected. We used an electronic data collection application to prevent data entry errors and supervised the data collection process.

Ethics

Ethical approval for the study was obtained from the Ethiopian Public Health Association and Johns Hopkins Bloomberg School of Public Health Institutional Review Board with IRB number 21116. Permission to conduct the study was also obtained from the Ministry of Health (MOH) and the deans of medical schools. Study participants provided informed oral consent, and measures were taken to protect autonomy and data confidentiality.

Results

Background characteristics

A total of 218 graduating medical students took part in this study with a response rate of 90.8%. Their mean age was 27.1 years. The majority of study participants were males (70.2%), from public schools (86.2%), and had a cumulative grade point average (CGPA) of more than 3.00 at the beginning of the internship (74.5%). Graduates from private medical schools were younger (mean age 25.7 vs. 27.4 years), had higher proportions of female students (66.7% vs. 23.9%), and had higher CGPA (3.33 vs. 3.19) than those from public medical schools (Table 1).

Table 1 Background characteristics of study participants (N = 218)
Full size table

Competence scores of graduating medical students

The overall mean competence score of graduating medical students was 72%. The highest scores were observed for obtaining consent for hernia repair (81%), and interpreting chest X-rays and CBC for TB patients (78%). On the other hand, competence scores were relatively low in conducting MVA for incomplete abortion (62%), performing LP (64%), and managing childbirth (66%) (Fig. 1).

Fig. 1
figure 1

Mean clinical competece scores of graduating medical studetns in percentage

Full size image

There was no statistically significant difference in overall student competence scores between public (71.6) and private (71.7) schools. However, students from public schools had significantly better scores in taking a focused history (p = 0.001), conducting precordium examination (p = 0.002), and obtaining consent for hernia patients (p = < 0.001). On the other hand, students from private medical had significantly better scores in patient education and counseling (p = 0.03), prescribing medication for a malaria case (p < =0.001), and wound suturing (p = 0.02) (Table 2).

Table 2 Mean competence scores of study participants in percentage by school type
Full size table
Table 3 Mean competence scores of study participants in percentage by medical school entry schemes
Full size table

Clinical learning environment

Medical students had an overall mean CLE satisfaction score of 75.2%. The motivation of students during clinical practicum had the highest score (83.7%). In addition, the majority of the students knew their learning limitations (91.7%), enjoyed learning at clinical practice sites (88.5%), and thought that the supervisors were good role models (89.9%). However, supervision of students during practicum had a low score (71.4%). Significant of them also reported that the way the supervisors dealt with medical students was satisfactory (40.8%), the number of students in clinical sessions was appropriate (56.0%), and the assessment of clinical learning was aligned with objectives (53.7%) (Fig. 2).

Table 4 Mean competence scores of study participants in percentage by sex
Full size table
Fig. 2
figure 2

Percent of medical graduates who were satisfied with CLE items and satisfaction scores by CLE domain (N = 218)

Full size image

Simulation training quality

Overall, 51% of participants were satisfied with the quality of simulation training. Specifically, 77% of respondents said the number of students at the skills development lab (SDL) was appropriate, and 61% acknowledged that supportive trainers were available. About two-thirds of the respondents expressed dissatisfaction with the availability of models and equipment at the lab, and the feedback provided at each practice session, and did not enjoy learning at the skills lab (Fig. 3).

Table 5 Mean competence scores of study participants in percentage by cumulative GPA
Full size table
Fig. 3
figure 3

Percent of graduating medical students who were satisfied with the simulation training quality (N = 218)

Full size image

Clinical practice exposure

Of the 12 procedures assessed, the majority of students performed the following tasks more than five times: nutrition assessment (95.9%), urinary catheterization (94.5%), intravenous (IV) cannulation (93.1%), giving oxygen (92.5%), and nasogastric (NG) tube insertion (91.7%). On the contrary, significant proportions of students never performed venipuncture (34.4%), lumbar puncture (LP) (30.7%), manual vacuum aspiration (MVA) (30.3%), and assisted normal delivery (9.6%) (Fig. 4).

Fig. 4
figure 4

Percent of graduating medical students who conducted 5 or more procedures and never conducted

Full size image

Factors affecting competence scores of graduating medical students

Female medical students had 2.4% higher competence scores compared to their male counterparts (p = 0.03). On average. Medical students with a CGPA of < 3.00 had 7.1% lower competency scores compared to those with a CGPA of > 3.50 (p = 0.001). Similarly, students with a CGPA of 3.00–3.49 had on average 3.7% lower competency scores than those with a CGPA of > 3.50 (p = 0.001). For a unit increase in the satisfaction score of students’ motivations in the CLE, the mean competence score increased by 12.7% (p = 0.020) (Table 6).

Table 6 Bivariable and multivariable linear regression results to assess factors affecting the competence of graduating medical students
Full size table
Table 7 Mean competence difference of study participants by level of practice exposure (number of conducted procedures)
Full size table

Discussion

After a successful primary healthcare expansion, Ethiopia has strengthened secondary and tertiary care aiming to increase its responsiveness to the population’s health needs. This progress has stimulated the rapid expansion of medical training in the country. With no congruent attention given to maintaining the training quality, the expansion has affected the medical schools in meeting the minimum pre-service education standards [28, 29]. Understanding the real effects of rapid training expansion and the challenges it poses is a critical step for improvement; particularly in contexts like Ethiopia where there is scanty evidence. To that end, we conducted this research to answer two main questions: what level of clinical competence did the undergraduate medical students master at the point of graduation? And which factors were associated with competence development?

The results of this study showed that the graduating medical students had suboptimal competence scores as a whole and in many competence areas as compared to the 75% cut-off score, signifying students’ capability gaps required for essential healthcare delivery. The pervasive shortages of experienced faculty and learning infrastructure, and challenging practical learning in Ethiopia’s medical schools coupled with the underdeveloped medical education regulation practices might be the underlying factors [7, 15, 30]. Comparable competence scores were also reported by studies conducted in Ethiopia and elsewhere [31,32,33,34]. Challenges of medical education due to shortages of critical training inputs and processes were similarly reported in Tanzania [35]. The competence gaps among the study participants made it clear that the medical graduates were not fully prepared for the responsibilities of general practitioners listed under the national scope of practice guidelines [20]. This means that the new graduates’ performance, confidence, professional identity, career progression, and quality of life can be affected [36, 37]. This all can have huge implications for the standards of patient care.

Effective preservice education for medical students requires high-quality clinical preceptorship and simulation training [38]. Repeated practice opportunities in clinical sites can boost the competencies learned and experiences acquired [39,40,41]. To that end, ensuring an optimal number and variety of cases in clinical settings is vital [42]. However; as stipulated in this study, performing hands-on clinical procedures by the medical students proved relatively more difficult. And a significant proportion of the students also had fewer practice exposures. On top of that, our study depicted that the medical students had challenging simulation and clinical learning environments. Studies conducted in other countries also discovered that the psychomotor abilities of final-year medical students were not fully developed [43,44,45]. The large number of enrolled students in Ethiopia’s medical schools might negatively affect the practical training in both simulation and clinical settings. Introducing medical education program accreditation and regulation has the potential to motivate schools to pursue quality [46]. Other researchers also corroborated our reports of the unnecessary effects of the rapid training scale-up and overwhelming students in Ethiopia [10,11,12, 29, 30]. However, many of the study participants had favorable perceptions regarding the number of students during practice. This might trigger questions about how well the schools used clinical rotations and scheduling to offset practice site overcrowding. And did the schools have adequate clinical sites used for student practice? [47]As per the findings of our study, the medical students’ motivation in clinical learning was associated with competency development. Unfortunately, the existing CLE gaps including the suboptimal availability of case varieties, motivation and supervision of students, and organization of clinical encounters affected the quality of student practice which might diminish the competence development [48,49,50].

Similar to what we reported, good academic performances were also associated with competence attainment in other studies [51]. This entails that medical schools should ensure that well-prepared students are enrolled and effectively taught, evaluated, and supported students across all stages of the curriculum. Despite many programmatic reports suggesting gender disparity in Ethiopia disfavoring females [52], this study depicted that female medical students had higher competence scores than males. They also had better scores in managing TB and malaria cases, conducting manual vacuum aspiration and lumbar puncture. Female nonphysician anesthesia students in Ethiopia similarly outperformed their male counterparts [53]. However, the male midwifery students had a better performance than the females [54].

Strengths and weaknesses

Covering all the required clinical competency domains and considering all types of medical schools found in the country enabled us to generate high-quality evidence. We conducted a direct observation of student performance using OSCE tools which have acceptable reliabilities and high objectivity. The multiple quality indicators were evaluated in the causal chain of educational inputs, processes, and outcomes, providing a better picture of the training. To address logistical challenges, we widened the data collection period to include all schools as the academic calendars of medical schools were variable. The shortage of OSCE logistics was solved in collaboration with the medical schools. Since we did not get standardized assessment rubrics for our purpose, experts assisted in developing and piloting rubrics based on the curricula and standards.

Conclusions

Medical students had suboptimal clinical competence. Lower competence scores were found in clinical procedures. A better CLE, higher cumulative GPA and academic performance, and more practice exposure were associated with high competence scores. We recommend that medical schools need to expand student clinical sites to primary healthcare units and private health facilities. Effective scheduling and clinical rotations are required to boost practice opportunities. Expanding and/or developing preceptors should be conducted. It is also imperative to address the simulation training gaps. Strengthening licensing examinations is also a way forward to ensure the graduates are fit for practice. Research studies are needed to understand the effects of the current medical education status on patient outcomes. Additional investigation is also required to assess the medical students’ ethics, leadership, communication, and collaboration skills.