Background

There has been a growing focus on outcomes-based medical education over the past two decades, driven by the desire to ensure that physicians have the right knowledge, skills, and attitudes to serve their patients and communities [1, 2]. To support this growing trend, several groups around the world have identified the competencies that the doctor are required to have. The “global minimum essential requirements (GMER)” were sponsored by the China Medical Board of New York (CMB) in 1999 and implemented in a number of Chinese medical schools [3]. The USA’s Accreditation Council for Graduate Medical Education (ACGME) Outcome Project (1999) introduced six domains of clinical competencies and the Royal College of Physicians and Surgeons of Canada (RCPSC) developed CanMEDS as a framework for physician competencies organized around seven roles [4]. Other frameworks have been developed by The General Medical Council of UK [5], the Indiana University School of Medicine (IUSM) [6], and researchers in Taiwan [7]. Although there are some differences among them, these general medical frameworks have a considerable amount in common.

Application of these general frameworks is underway in several countries. For example, the GMC developed a version of Good Medical Practice for each specialty in 2002 and the ACGME has drafted milestones for many specialties [5.8]. However, across countries there is considerable diversity in both the delivery of healthcare and the nature of medical education, which might result in variability in the competencies as well [8]. Given these differences, it is preferable to generate local competencies rather adapting or adopting those that already exist in other countries [9]. The purpose of this study is to identify the competencies necessary for good medical practice in China.

Doctors are the backbone of the Chinese health care system because they play a crucial role in patient management, disease prevention, and health promotion. As a result, the quality of medical care can be improved by making changes in medical education system. A first step in this direction is the development of competencies that might help to drive reforms in medical education.

Several methods for developing competencies are available including theoretical analysis, behavioral event interview, the Delphi method [10, 11], the expert-novice approach [12, 13], and work analysis [14, 15]. For example, the GMER competencies were created by the leadership of the International Institute on Medical Education [3]. The CanMEDS project developed the competencies by consulting experts and healthcare organizations, conducting a systematic review of the literature, and undertaking a Delphi process [4]. The ACGME Outcome Project convened 33 experts who decided the competencies of doctors [16], These are qualitative methods.

The Occupational Information Network (O*NET) questionnaire (O*NET) was developed under the auspices of the US Department of Labor as a tool for work analysis [17]. The U.S. Department of Labor sponsors an online, freely available O*NET database which reports the results of various job analyses and makes available the questionnaires it uses. O*NET methods have been applied successfully to a wide range of jobs, including those in healthcare and teaching. Importantly, research has shown that the O*NET questionnaires have good applicability to jobs in healthcare and they have been useful in building physician competencies models in the US [17–22].

Among the cluster of O*NET questionnaires, the General Work Activity (GWA) and Work Style (WS) questionnaire were suitable tools to assess competencies [23]. This study reports the results of administering the O*NET-GWA and O*NET-WS questionnaires to doctors from seven provinces in China [18]. The American Psychological Association defined general work activity as “an aggregation of similar job activities/behaviors that underline the accomplishment of major work functions” [24]. The O*NET-GWA questionnaire included four domains: information input, mental processes, work output and interacting with others [25]. It can be considered as the element that above the “iceberg” of competencies [26, 27]. Meanwhile, Work Style was defined as “work- and job-related personal characteristics” [28]. The O*NET -WS questionnaire included two domains: decision-making work styles and interpersonal work styles [29]. It can be considered as the element that beneath the “iceberg” of competencies model [26, 27]. This is a quantitative method [23] and most previous surveys about competencies applied qualitative methods [10–15]. These data were submitted for analysis to produce competencies model of doctors that could be used in the reform of Chinese medical education.

Methods

Questionnaires

To collect the opinions of doctors on the competencies that doctors should have after the 3 years of residency training or with the equal experience, we used the O*NET -GWA (Additional File 1: GWA quetionnaire) and the O*NET-WS questionnaires (Additional File 2: WS questionnaire). The O*NET-GWA questionnaire consisted of 41 items such as “How important is getting information to the performance of your current job?” The O*NET-WS questionnaire consisted of 16 items such as “How important is persistence to the performance of your current job?” Responses of the O*NET-GWA and O*NET-WS questionnaire were all captured on 5-point Likert scales where 1 is “Not important” and 5 is “Extremely important”.

The questionnaire was translated to Chinese by a panel of health professionals including experienced doctors, nurses, and a clinical psychologist working at a teaching hospital. It was pretested on 20 doctors, who were asked to comment on the acceptability and clarity of the items and the scale as a whole. The final translated items used for data collection were generated through consensus on the wording, clarity and cultural equivalence of items.

Participants

There were 23,170 hospitals with 2,138,836 doctors in China by the end of 2010 according to the Statistics Year Book 2011 [30]. We recruited doctors with the help of the North China Center for Medical Education Development (NCC) [31], which is a collaboration of 18 medical institutions representing most of China (i.e., well beyond northern China). The NCC collaborative institutions administered the questionnaires in their local provinces.

We performed stratified sampling to ensure those surveyed were similar to the population of doctors throughout China. Data were collected from 7 provinces which covered 7 different geographic regions of China [32]: Shandong Province (East China), Guangxi Province (South China), Shanxi Province (Central China), Hubei Province (North China), Liaoning Province (northwest China), Sichuan Province (Southwest China), and Xinjiang Province (Northeast China).

Sampling was stratified at the level of institutions to match the percentage of all certified physicians employed in the medical institutions in China according to China Health Statistics Yearbook 2011 [30]. In each province, we surveyed 1 medical college affiliated hospital with 60 doctors; 1 provincial hospital with 60 doctors; 2 municipal hospitals with 35 doctors each; 1 district hospital with 12 doctors; 2 rural hospitals with 19 doctors each; 2 community health service centers (CHSC) with 10 doctors each; 2 rural clinics with 10 doctors each. In total, 280 doctors in 11 medical establishments were surveyed in each province and 1960 doctors in 77 hospitals were surveyed throughout China.

Data collection

Seven trained surveyors collected data from May to September 2012. The surveyors received training together by the same person to ensure they understood the questionnaire and how the data needed to be coded. They administered the questionnaire in person to each participant. The surveyor spent about 5 min explaining the purpose of the study and then the participants were given 20 min to complete the questionnaire independently. The responses of the participants were anonymous and the results remained confidential. In addition to the questionnaire, the demographic and occupational characteristics of the participants were gathered.

Of the 1960 questionnaires, 63 were removed from analysis because the doctors indicated that they did not wish to participate, 30 were removed because they lacked demographic information, 70 were removed because more than 10 questions (20 % of the questions) were unanswered [33–36], and 21 were removed because the participant marked the same answer to all questions. This left 1776 questionnaires (90.61 %) for analysis. If less than 20 % questions were unanswered, the missed data would be replaced with means [33–36].

Data analysis

There were two stages of analysis. In the first stage, exploratory factor analysis (EFA) was conducted to establish the factor structure of the scale. In the second stage, confirmatory factor analysis (CFA) was conducted to verify the factor structure and to ascertain the competencies of Chinese doctors.

The sample size requirements for these analyses were a 1:10 to 1:15 ratio of questions to participants [25]. Total questions on the O*NET-GWA and O*NET-WS surveys were 57 so between 570 and 855 participants were adequate for the analysis, and the number of participants far exceeded these values. To perform exploratory factor analysis and confirmatory factor analysis, the participants were randomly divided into two groups of approximately 50 % each for analysis.

In exploratory factor analysis, after deducting the overlap between each of the 57 items and its related domain, factor loadings of more than 0.60 were considered satisfactory [25]. Items with factor loading of less than 0.6, or with cross factor loadings greater than 0.35 were removed from further analysis. The Kaiser-Meyer-Olkin- Kriterium (KMO) statistic and Bartlett’s spherical check were calculated to check for sample suitability for the factor analysis There are two ways which determined the number of factors for consideration. Firstly, it is Kaiser's eigenvalue-greater-than-one rule (K1 or Kaiser criterion) [34]. We compute the eigenvalues for the correlation matrix and determine the number of the eigenvalues greater than 1. This number is the number of factors included in the model. Secondly, it is Cattell's scree plot [35]. We compute the eigenvalues for the correlation matrix, and then plot the values. By examining the graph, we determine the last substantial drop in the magnitude of eigenvalues. The number of plotted points before the last drop is the number of factors included in the model. The factors were recalculated after items were removed from initial exploratory factor analysis. Cronbach’s α coefficient was calculated as an estimate of the reliability of the questionnaire. An alpha of 0.7 to 0.9 was considered good [36].

In confirmatory factor analysis, the selection of the best fitting model was based on several fit indices. For acceptable model fit, chi-square should be low, the Comparative Fit Index (CFI) should be higher than 0.90, and Root Mean Square Error of Approximation (RMSEA) should be lower than 0.08 [37]. Maximum likelihood was used to estimate parameters in these analyses.

The data were analyzed using IBM SPSS® version 17.0 and AMOS® version 20.0 (SPSS Inc., Chicago, IL, USA) for Windows®. A P-value of < 0.05 was considered to be statistically significant.

Ethical approval

The Bioethics Advisory Commission of Harbin Medical University approved the protocol. The completed questionnaires did not contain any identifying information about the individual subjects. Each participant gave written consent. Participation in the study was totally voluntary, participants were paid, and they had the option of declining to answer specific questions or to leave the entire questionnaire blank. All data were kept confidential and data protection was observed at all stages of the study.

Results

Characteristics of the doctors and the questionnaires

The average age of the doctors was 39.45 years (SD = 8.20), with 919 (51.7 %) male doctors and 857 (48.3 %) female doctors, and the ratio was similar to the male to female ratio of the doctors in China (1.34:1) according to China Health Statistics Yearbook 2011. The sample includes 401 trainees, 572 attending physicians, 463 associate professors and 340 chairs of the department. Of the participants, 519 were general internists, 425 were surgeons, 408 were gynaecologists, 340 were pediatricians and 84 were from others clinical departments. There were 1460 urban doctors and 316 rural doctors. The demographic characteristics of the participants are shown in Table 1. Frequencies and means for each item were shown in Table 2.

Table 1 Demographic characteristics
Full size table
Table 2 Frequencies and means for each item
Full size table

Exploratory factor analysis

The data were randomly divided into two groups of approximately 50 % using SPSS. One group contained 917 participants and it was used for exploratory factor analysis based on principal components analysis (PCA). The underlying dimensions were assumed to be correlated with each other and promax rotation was applied to relax the orthogonal constraint to allow for correlated factors [25]. The Kaiser-Meyer-Olkin-Kriterium (KMO) statistic was 0.97 and Bartlett’s spherical check was χ2 = 38417.52(df = 1596)and P < 0.001. Together these indicated that the study data were suitable for factor analysis.

For the factor analysis of the O*NET-WS questionnaire, items WS5 (Cooperation), WS6 (Concern for Others), and WS15 (Innovation) had loadings >0.35 on more than one construct and they were removed from analysis. WS7 (Social Orientation) had a factor loading of less than 0.6, and it was also removed. Twelve items were left and the analysis yielded two factors. Given the content of the items, the factors were named (a) professional behavior; and (b) professional values (Table 3).

Table 3 Exploratory factor analysis results of the O*NET-WS scales
Full size table

For the factor analysis of the O*NET-GWA questionnaire, items WA5 (Estimating the Quantifiable Characteristics of Products, Events, or Information), WA31 (Resolving Conflicts and Negotiating with Others), and WA39 (Performing Administrative Activities) had loadings greater than 0.35 on more than one factor and they were removed from further analysis. WA1 (Getting Information), WA11 (Thinking Creatively), WA12 (Updating and Using Relevant Knowledge), WA14 (Scheduling Work and Activities), WA16 (Performing General Physical Activities), WA19 (Working with Computers), WA24 (Documenting/Recording Information),WA30 (Selling or Influencing Others), WA32 (Performing for or Working Directly with the Public), WA38 (Providing Consultation and Advice to Others) and WA40 (Staffing Organizational Units) had factor loadings less than 0.6, and they were also removed. 27 items were left for analysis and 4 factors were identified. Given the content of the items, the factors were named (a) technical procedural skills; (b) diagnosis and management; (c) teamwork and administration; and (d) communication (Table 4).

Table 4 Exploratory factor analysis results of the O*NET-GWA scales
Full size table

The initial percentage of variance accounted for by Professional behavior, Professional values, Diagnosis and management, Technical procedure, Teamwork and administration, and communication were 32.88 %, 23.52 %, 18.95 %, 17.96 %, 12.02 %, and 11.62 % respectively.

After deleting the items, we performed the exploratory factor analysis again. The Mean (SD) for final factor scores and percentage of variance accounted for by each factor are shown in Table 5. The included items in each factor were the same as the first exploratory factor analysis after deleting the items. The percentage of variance accounted for by professional behavior and professional values of the O*NET-WS questionnaire were 37.06 % and 22.86 % respectively. The percentage of variance accounted for by technical procedural skills, diagnosis and management, teamwork and administration and communication of the O*NET-GWA questionnaire were 21.06 %, 19.90 %, 14.36 % and 11.36 % respectively. The Mean (SD) for professional behavior, professional values, technical procedural skills, diagnosis and management, teamwork and administration and communication were 30.78(4.54), 14.62(2.38), 21.73(7.46), 35.31(6.19), 21.39(3.84) and 21.39(3.84) respectively.

Table 5 The Mean (SD) for final factor scores and percentage of variance accounted for by each factor
Full size table

The final Cronbach’s alpha of the two questionnaires were 0.90 and 0.94, satisfying the requirement of being greater than 0.7. The minimum corrected-item-total correlation was 0.45, which was greater than what is recommended and the entire model explained 64.25 % of the variance.

Confirmatory factor analysis

The other half of the data was used for confirmatory factor analysis (CFA). The construct reliability was 0.9. The average variance extracted (AVE) for each construct exceeded 0.6. The six dimensions were in line with common criteria [38] and had good convergent validity (Fig. 1). Applying the criteria of Boomsma 2000 [39] and Byrne 2010 [40] produced a conclusion of good model fit (Table 6).

Fig. 1
figure 1

Results of second-order confirmatory factor analyses. Arrows showed causal paths. All paths were significant standardized path coefficients. Ovals signify latent variables (i.e. the constructs of doctors’ competencies of GWA and WS items)

Full size image
Table 6 Model fit index
Full size table

Discussion

This study found that an outcomes-based framework for competencies in China would include diagnosis and management, communication, teamwork and administration, professional behavior, professional values, and technical procedural skills. We used the O*NET-GWA and O*NET-WS questionnaires and conducted quantitative analyses. In contrast, work done in other countries has relied mainly on qualitative methods, including theoretical analysis, behavioral event interviews, and the Delphi technique [10, 11]. It is reassuring that our work produced similar results.

The O*NET questionnaires were powerful tools for work analysis. The O*NET -Standard Occupational Classification system (O*NET-SOC) which included 1094 occupational titles have almost covered the entirety of occupations of US [41]. When applied the O*NET-GWA and O*NET-WS questionnaires to the occupation of doctor, the explained for the generic occupations needs to be kept close to the underlying elements of doctors. Thus the factors were named in light of medical content of the practice of physicians. The six-factor solution of the O*NET-GWA and O*NET-WS questionnaires was consistent with previous studies (Table 7) [25, 29] and outcome-based medical education reforms in China and worldwide. Compared with the competencies framework developed by the ACGME [42], our doctor’s competencies model was similar in diagnosis and management and technical procedural skills (the ACGME’s patient care and medical knowledge), communication, teamwork, and administration (the ACGME’s interpersonal and communication skills), and professional behavior and professional values (the ACGME’s professionalism). Likewise, compared with the CanMEDS framework [4], our doctor’s competencies model was similar in diagnosis and management and technical procedural skills (medical expert & scholar in CanMEDS), communication (communicator and health advocate in CanMEDS), teamwork and administration (collaborator and manager in CanMEDS), and professional behavior and professional values (professional in CanMEDS).

Table 7 Comparisons of competencies frameworks
Full size table

Although these findings are consistent with frameworks developed elsewhere, our study does have limitations. We could only collect data from seven provinces. Although they cover different regions of the country, a broader sample might generate different results. Likewise, in our sample we had more urban than rural doctors. This might influence the applicability of our framework to the rural health system.

Since 1910, Abraham Flexner’s report has helped shape the face of medical education both in America and around the world [43]. In fact, China adopted a Flexnerian model as its medical institutions developed. However, the political, social, and economic environment has changed significantly since Flexner’s day [44]. This has generated the need for reform which has largely been filled by competencies-oriented medical education models [45] such as those developed in Canada [46] and the United States [47]. Our study provides a model, relevant to China, which will support educational reform efforts.

The findings of this study will have practical implications for health professions education. In the past, the majority of medical graduates in China have entered the health care system directly, without further training. Starting in 1993, China established postgraduate training programs for doctors [48]. However, clinical skills and medical knowledge are the primary focus of these training programs. Competencies such as communications, teamwork and administration, professional behavior, and professional values were not addressed. The findings of the study will help align the competencies developed during training with the needs of the healthcare system.

Finally, this study lays the groundwork for future research directions. Quantitative methods were used to establish this preliminary model of competencies. Further work based on qualitative methods such as behavioral event interview, focus group interviews, and the Delphi technique might suggest amendments to the model. In addition, collecting and analyzing the opinions of nurses, patients, and administrators will build and enrich the model of doctor’s competencies.

Conclusions

Our results provide support for the reliability and validity of the Chinese version of the O*NET-GWS and O*NET-WS for doctors. The Chinese doctors’ competencies framework includes six elements: (a) technical procedural skills; (b) diagnosis and management; (c) teamwork and administration; (d) communication; (e) professional behavior; and (f) professional values. These findings are relevant to China, consistent with its current situation, and similar to those developed in other countries.