European Radiology

, Volume 20, Issue 4, pp 967–977 | Cite as

Competency-based (CanMEDS) residency training programme in radiology: systematic design procedure, curriculum and success factors

  • Erik JippesEmail author
  • Jo M. L. van Engelen
  • Paul L. P. Brand
  • Matthijs Oudkerk
Open Access
Radiological Education



Based on the CanMEDS framework and the European Training Charter for Clinical Radiology a new radiology curriculum was designed in the Netherlands. Both the development process and the resulting new curriculum are presented in this paper.


The new curriculum was developed according to four systematic design principles: discursiveness, hierarchical decomposition, systematic variation and satisficing (satisficing is different from satisfying; in this context, satisficing means searching for an acceptable solution instead of searching for an optimal solution).


The new curriculum is organ based with integration of radiological diagnostic techniques, comprises a uniform national common trunk followed by a 2-year subspecialisation, is competency outcome based with appropriate assessment tools and techniques, and is based on regional collaboration among radiology departments.


The application of the systematic design principles proved successful in producing a new curriculum approved by all authorities. The principles led to a structured, yet flexible, development process in which creative solutions could be generated and adopters (programme directors, supervisors and residents) were highly involved. Further research is needed to empirically test the components of the new curriculum.


Radiology education Medical education Curriculum development Systematic design procedures Diffusion of innovation The Netherlands 


Based on the Canadian Medical Education Directions for Specialists (CanMEDS) [1] framework of key competencies for medical specialists the Royal Dutch Medical Association (RDMA) decided in 2004 that postgraduate training and education for all medical specialists in the Netherlands should be reformed. In the same year, the national Board of Health Care Professions and Education (BHCPE) in the Netherlands was instituted, in which decisions on structure, innovation, quality (monitoring) and finances of all medical specialty training programmes were integrated [2]. The Netherlands was divided up into eight regions for medical education to coordinate postgraduate training in each region [3].

In 2006, the Dutch Radiology Society (DRS) assigned a national project team (Revision of Radiology Curriculum, “Herziening Opleiding RAdiologie” or HORA in Dutch) with the task of developing a new radiology training curriculum, taking into account guidelines both from the RDMA and from the European Association of Radiology (EAR), a collaboration of national scientific radiological societies in Europe, which published the European Training Charter for Clinical Radiology (ETCC), an outline for an organ-based curriculum comprising a common trunk and a subspecialty programme in 2005 [4]. The multitude of both the complex developments and the stakeholders in a variety of fields involved turned the accomplishment of the HORA project into a great challenge. We, therefore, decided to apply a systematic development approach based on evidence-based design principles. Both the development process and the resulting new curriculum [5] are presented in this paper.

Materials and methods

The Concilium Radiologicum (National Board for Education and Training in Radiology) of the DRS put together the HORA project team with experienced university and general hospital radiologists, residents and educationalists. The project team realised that the construction of a new curriculum comprised two major areas. The first major area comprises, the medical knowledge of radiology, radiological procedures and radiology organisation (the content of the new curriculum) which was based on the ETCC [4] and put together by the experienced radiologists and radiology residents in the project team. The ETCC formed a valuable starting point for multiple reasons. First of all, many authoritative and experienced radiologist from almost all national scientific radiological societies in Europe worked on this charter for years. Using these valuable insights saved time and energy. Second, the charter was officially approved by the EAR which committed the DRS. Third, using an officially approved European charter helped in gaining acceptance from the programme directors in the Netherlands for the curriculum changes. The ETCC was translated by an experienced medical translator in Dutch and corrected by the radiologists in the project team. The radiologists in the project team took into account that the increased technological possibilities of magnetic resonance imaging (MRI) and computed tomography (CT), and the implementation of picture archiving communication systems (PACS) have increased the diagnostic capabilities and efficiency of radiology departments. This increases not only the workload of the radiology department, but also the specialised knowledge required by the radiologist to effectively use these diagnostic capabilities to answer clinicians’ questions. It was assumed that future radiologists will increasingly engage in multidisciplinary collaboration with clinicians, and that interventional radiology will be a rapidly evolving field, shifting towards more minimally invasive procedures [6, 7]. The residents in the project team advocated the interests of their fellow residents and were responsible for communicating the developments in the new curriculum to them. The residents in the project team were first contact for questions of their fellow residents.

The second major area comprises, the educational outline and the toolbox of the curriculum which was provided by the Dutch version of the CanMEDS competency-based framework and the corresponding (mandatory) assessment instruments. The educationalists in the project team advised the radiologists how to translate the general framework to the context of radiology. Although different medical specialists show similarities in the competencies they are supposed to possess, important differences can occur between different specialties, however. For example, the competency “communicator” will be interpreted differently for a psychiatrist and a radiologist. For the psychiatrist, verbal communication is at the heart of the diagnostic and therapeutic repertoire, whilst digital or written communication is more important in the radiological service. Translation of the general framework to every medical specialty is therefore necessary to make the framework practical and meaningful. In this process, the educationalists gave advice on the building blocks of the curriculum, assessment strategy and instruments, quality control and implementation. Furthermore, the educationalists were responsible for coordinating the different (sub)tasks in the design of the new curriculum. Other activities of the educationalists included the organization of workshops and meetings with programme directors and the communication with the RDMA. Further educational support was available from the Dutch Advisory Board for Postgraduate Curriculum Development (DABCPD) [8] and the educational working group of the RDMA. Competency-based education is inspired by social constructivism. This modern learning theory assumes that knowledge is individually and socially constructed. Individuals give their own meaning and interpretation to the things that happen in their environment, and this meaning and interpretation is also shaped by interaction with other people. In the context of curriculum development, this means that learning should be an engaging, constructive and active process that takes place in or is easily transferred to realistic practical situations [9].

The project team organised the development process of the new curriculum around the principles of systematic design. This design approach has its fundamentals in design methodology and has repeatedly proved successful in new product development (material products as well as intangible services). Systematic design intends to stimulate creativity while, at the same time, the controllability of the design process is increased significantly. Modern systematic design embraces a large and diverse set of methods and approaches but can be characterised along four underlying principles [10]. The first principle, discursiveness, implies semi-structured guidance for the design process through a step-by-step yet iterative scheme of activities. Hierarchical decomposition refers to decomposing the design tasks into smaller, easier to handle but interdependent subtasks, setting up a so-called morphology for the design task. This step also includes the formulation of several design specifications for the whole design problem. These specifications are the requirements of the design. Systematic variation deals with the way of searching for solutions for subtasks and combining these following the morphological scheme into solutions for the complete design task. To prevent designers from endlessly searching for ever better solutions the principle of satisficing was introduced by Simon in 1969. According to this principle searching for an acceptable solution is by far superior to searching for an optimal solution [11]. Finally the solutions for the subtasks are combined into an overall solution and tested against the formulated design specifications.


First, we present the features of the new curriculum and we explain how the (medical) knowledge of radiology, radiological procedures and radiology organisation, and the educational framework/toolbox of competency-based education are applied. Second, we present how we applied the principles of systematic design to the development process of the new curriculum. We provide a timeline of the development process and discuss unforeseen developments and delays the project team had to deal with during the process.

Features of the new curriculum

See Table 1 for the unique features of the new curriculum.
Table 1

Features of the new curriculum

Organ based

Uniform national 3-year common trunk, followed by a 2-year subspecialisation into one of eight directions

Competency outcome based with appropriate assessment tools and techniques

Based on regional collaboration among radiology departments, with exchange of residents between departments

Organ based

The new curriculum is organ systems based (Table 2), with integration of radiological techniques within these organ systems (Table 3). This makes it possible to anticipate the more specialised knowledge about specific diseases needed for answering clinicians’ questions.
Table 2

Organ systems modules and subspecialisations in the new curriculum

Organ systems module in the common trunk

Organ systems modules in the subspecialisation




Neuro & head and neck

Head and neck






Abdominal (gastrointestinal, urogenital & gynaecology)








Table 3

Radiological techniques in the new curriculum

Conventional radiology


Computed tomography

Magnetic resonance imaging



Nuclear diagnostics

Common trunk and subspecialisation

The common trunk encompasses a uniform national training programme on the basics of all organ systems (Table 2). The subspecialisation programme in the last 2 years of training focuses on one of eight subspecialisations (Table 2). There are a number of reasons to implement the common trunk and subspecialisation structure. First, the ETCC adopted this model. Second, this structure anticipates the rapidly growing knowledge base. It seems impossible to learn and keep up to date will all relevant knowledge in radiology. The total knowledge base is therefore divided into eight subspecialisations. This allows individual radiologists to become experts in and keep up to date with a specific subspecialisation, whilst the aggregated individual competencies in the team of radiologists comprise the total knowledge base needed. Third, the structure gives residents the motivational opportunity to distinguish themselves from their colleagues. Fourth, the new structure formalises and recognises expertise of different radiology departments. Because of differences in patient populations and existing investments, radiology departments differ in the procedures they perform and the knowledge bases they possess. Departments can enhance their expertise by teaching the associated subspecialisation. Finally, the structure improves the match of supply and demand of labour. The inflow of radiology residents into subspecialisations can be adjusted to the number of radiologists with particular profiles required in the field.

The resident can specialise in one subspecialisation or continue general radiology training in the last 2 years. Fifty per cent of the time in the last 2 years is devoted to the subspecialisation. The other 50% is devoted to general radiology. The subspecialisation is open to residents who perform above average. In addition to general and specialised radiology training, the new curriculum encourages residents to pursue scientific training. This consists of 1 day each week devoted to science, equalling 20% of total training time. This training option is open for residents who have conducted PhD research or are about to finish one.

Competency outcome based

Based on the seven CanMEDS core competencies, a specific competency profile for radiology was put together [12]. For example, the general competency “communication” was subdivided into “communication with clinicians” and “communication with patients and family”. Within “communication with clinicians”, specific competencies such as “the resident reports relevant, accurate and explicit findings of radiological diagnostics timely” were formulated.

The organ systems comprise the themes, or basic building blocks, of the curriculum. For each organ system, competency levels (Table 4) were assigned for year one, year three and year five [13]. For year five, two competency levels were assigned for each subspecialisation; one competency level for residents who follow the specific subspecialisation and one competency level for residents who follow general training.
Table 4

Competency levels in the new curriculum [13]

Level 1: knowledge possession

Level 2: performing with high supervision by radiologist

Level 3: performing with moderate supervision by radiologist

Level 4: performing without supervision by radiologist

Level 5: supervising and educating during the performance

For each of the organ system modules, three entrustable professional activities (EPAs) were composed. These are information-rich activities which assist residents and radiologists in focusing educational attention [14]. CanMEDS competencies were attributed to each of the organ system modules and for each of the EPAs. For example, within the organ systems module “Cardiac”, the EPA “patient with signs of angina pectoris as interpreted from the CT/MRI” was formulated. The CanMEDS competency “collaboration” and “professional” were attributed to this EPA, because fast and efficient consultation with the cardiologist is required.

In the new curriculum, multiple assessment instruments are being used (Table 5) to assess progress of the residents on the competency levels for the organ systems modules and the EPAs. Multiple assessment instruments are necessary to draw reliable and valid conclusions [15]. For each assessment instrument, competencies were assigned and frequency requisites formulated (Table 5).
Table 5

Assessment blueprint (instruments, accompanying competencies and frequency of use)

Assessment instrument


Frequency of use

Mini-clinical evaluation exercise (mini-CEX)

Medical expert, communicator, collaborator, scholar, manager, health advocate and professional

Ten observations each year (obligatory)

Multi-source feedback

Medical expert, communicator, collaborator, manager and professional

Once each year (not obligatory)

Objective structured assessment of technical skills (OSATS)

Medical expert, communicator and professional

Ten observations each year (not obligatory)

Critical appraised topic (CAT)

Medical expert, communicator and scholar

Twice each year (obligatory)

Progression tests

Medical expert, scholar and health advocate

Twice each year (obligatory)

Portfolio and periodical meetings with programme director

Medical expert, communicator, collaborator, scholar, manager, health advocate and professional

1st year, five meetings; 2nd and 3rd year, three meetings

4th and 5th year, two meetings

Assessment can be formative (guiding future learning, promoting reflection) or summative (making an overall judgement about competence, qualification for higher levels of responsibility) [15]. The assumptions of the assessment strategy of the new curriculum include:
  • Assessment is mainly formative.

  • Summative assessment takes place at the end of each year of training. Multiple observations in varying settings from different assessment instruments are necessary to make a summative judgement about overall competence. Single observations from one assessment instrument are insufficient for sound judgement of competence.

  • The assessment instruments assess all competencies in every year of training.

  • Residents need to master the competencies on the required competency levels.

  • Premature termination of the training is possible when the assessment results indicate (persisting) disfunctioning on at least two competencies.

  • The team of supervising radiologists are responsible for the assessment.

  • Use of assessment instruments and providing feedback requires training for both the supervisors and the residents.

The mini-CEX is a method of assessing competencies in real-life clinical practice. It consists of a short observation by a qualified medical specialist of a resident demonstrating clinical skills, using a predefined scoring format, in congruence with the competency profile of the radiologist, followed by a structured feedback conversation [16]. Assessment in the feedback conversation is formative. When feedback is brought safely, timely, specific and well structured, this assessment can be a valuable mechanism of supervision and learning [17].

Multi-source feedback is a method of gaining feedback from multiple people with different functions in the radiology department [18]. This method is intended to be used as input for annual progress assessment interviews between resident and programme director.

The objective structured assessment of technical skills (OSATS) is a method of assessing technical skills and is particularly suitable in the operation theatre [19], or, for radiology, in the intervention room.

The critically appraised topic (CAT) consists of a systematic literature search on a specific structured clinical question [20].

A national knowledge progression test is being applied annually to demonstrate progress throughout the training period and allow group comparisons.

In their portfolios, residents collect evidence about their personal development. Results of assessments are recorded, and evidence of acquired knowledge and skills is collected, including repeated reflection on acquired competencies, competency levels, and a personal strengths–weaknesses analysis [8]. The portfolio is reviewed in the annual progress assessment interviews.

All assessment instruments were adapted for use in radiology and after adaptation not assessed for validity and reliability. However, the psychometric properties of every single instrument was tested in other settings. Epstein provides an excellent overview of the properties of many assessment instruments [15]. For example, research has shown between seven and eleven different observations with the mini-CEX are necessary to achieve a generalizable global estimate of competence [15, 21].

Regional collaboration

Historically, the entire radiology residency training was provided at a single medical centre. Although some radiology departments and hospitals made agreements on multi-centre training programmes and regional collaboration in education, this was not formalised in the national curriculum or legislation. The new curriculum recognises that training can benefit from alternating training periods in both university and general hospitals, and from an active collaboration between centres in regional educational activities. The national curriculum and educational framework can be used in different ways in every region of medical education, but the exchange of residents within a region is mandatory. Furthermore, every region is required to make agreements on which subspecialisations in the last 2 years of the curriculum they aspire to offer.

The development process of the new curriculum


Figure 1 illustrates the design and implementation process. The design process was largely specified beforehand and divided into several design steps. The process was highly iterative. Because of developments in later phases prototypes were refined and improved.
Fig. 1

Timeline of the design and implementation process of the new curriculum

Hierarchical decomposition

From the start of the project, the HORA project team realised that the support of the radiology programme directors was a prerequisite for the new curriculum to be accepted. For this reason, the HORA project team organised three plenary meetings (Fig. 1) involving all programme directors. For the first plenary meeting, several design specifications of the new curriculum formulated by the HORA project team (see Table 6) were discussed with the programme directors, and agreed upon by all but one.
Table 6

Design specifications and accompanying discussion points for the new curriculum

Design specification

Discussion point

1. A new curriculum needs to be designed

No discussion

2. The new curriculum needs to encompass training periods in university and general hospitals

Some programme directors were concerned about reduction of autonomy due the training in both university and non-university hospitals

3. The exchange of residents between university and general hospitals needs to be organised within the eight regions of education in the Netherlands

Some programme directors were concerned about reduction of autonomy

4. The new curriculum needs to be competency based and modularly designed

Some programme directors were not convinced of the advantages of competency-based education and were worried about the additional time needed for using the new instruments

5. The new curriculum needs to consists of a 3-year common trunk and 2-year subspecialisation

Some programme directors were afraid that insufficient radiologists with a general profile would be trained

6. The new curriculum needs to be organ based

Some programme directors were concerned about the amount of time and energy necessary for organising the rooms, equipment, teams and ICT infrastructure differently

For the second plenary meeting, all programme directors were asked to discuss the specifications of the new curriculum within their region of medical education, and to come up with further specifications and discussion points for every specification. The second meeting made clear that all eight regions of medical education supported the specifications, and several useful discussion points were put forward (Table 6). For the third plenary meeting the HORA project team broke down the design task into smaller, but interdependent subtasks according to the guidelines provided by DABPCD and the working group RDMA (see Table 7 for the morphological scheme of the design task).
Table 7

Morphological chart: subtasks and solutions in the new curriculum

Design task


Possible solutions

Time period of decisions

Primary responsibility


Design of new curriculum radiology

1. Profession analysis/radiology specific profile

The general CanMEDS frameworka, the European Training Charter [4] and the Good Practice Guide For European Radiologists [12]


Project team HORA

For each competency several radiology-specific subcompetencies were formulated


2. Themes/building blocks

The organ based systems provided by the European Training Charter [4]


Project team HORA

See Table 1 (left side)


3. Assessment strategy and instruments

Assessment strategy can be formative (oriented towards formulating learning points) or summative (making a judgement about competence). Many assessment methods are possible [20]. For example:


Project team HORA

See Table 3


- Multiple choice questions/progression test


- Essay


- Short answer question


- Oral exam


- Objective structured clinical examination


- In-training evaluation report (e.g. mini-CEX)


- Multi-source feedback


- Critical appraised topic


- Portfolios and logbooks


4. Teaching philosophy

Competency-based educationa


Project team HORA

Competency-based education


5. Teaching activities

Many teaching activities are possible. For example:


Project team HORA

- Radiological workstation


- Patient reporting

- Consultations


- Radiological workstation

- Multidisciplinary conferences


- Cursory conferences

- Cursory conferences


- Journal clubs

- Structured series of conferences


- Critical appraised topic

- Critical appraised topic/“holy hour”


- Teaching round

- Intervention room/operating room


- Outpatient clinic


- Patient round


- Intervention room/operating room


- Consultations


- Mortality/morbidity review


- Multidisciplinary conferences


- Structured series of conferences


6. Teaching materials

Many teaching materials are possible and need to be made. For example:

Short and medium

Project team HORA for the short term and the regions of medical education for the medium term

The curriculum was written with chapters corresponding to the subtasks in this table. Teaching materials for the assessment instruments were developed (see Table 3). Other teaching materials were or have to be developed by the regions of medical education


- Curriculum/training programme


- Assessment instruments


- Course materials


- Guidelines/protocols


- Literature


7. Training/curriculum structure

Curriculum structure


Project team HORA

3-year common trunk and 2-year subspecialisation. In the subspecialisation phase 50% of the time is devoted to general radiology and 50% to the chosen subspecialisation


Possible solutions:


- General training


- Subspecialised training


- Combination of general training and subspecialised training


Collaboration with other radiology departments for the exchange of residents

- Collaboration with radiology departments within a region of medical education or between regions of education


Project team HORA

Collaboration and exchange of residents takes place among the radiology departments within the regions of education. Exceptions can be made for individual residents


- Duration of exchange periods; between 6 months and 4 years


Project team HORA

Minimum of 1 year


- Possibility of unconnected exchange periods e.g. two periods of 6 months


Project team HORA

The exchange can be unconnected


- Year of training in which the exchange has to occur; year one to year five


Region of medical education



- Labour law and appointments; separate appointments for every hospital or one appointment with the hospital in which the main part of training is followed


Region of medical education

Variable, predominantly by one appointment


- Financing of residents, by the hospital in which the resident follows the training at the time or by the hospital in which the residents follows the main part of training


Region of medical education

Variable, predominantly by the hospitals in which the resident follows the main part of training


8. Quality monitoring

Requirements for hospitals to offer the common trunk and to offer subspecialisations


National subspecialisations sections for radiology specific requirements. RDMA for more general requirements

Formulated for every subspecialisation


Requirements for individual supervisors



Formulated for all medical specialists


Quality control mechanisms to monitor curriculum implementation. Possible mechanisms include:


Project team HORA/Concilum Radiologicum

HORA 2 project team was installed to monitor the implementation of the new curriculum. The project team consists of one representative of every region of education and one educationalist. Further mechanisms which are used include:


- New project team to coordinate and monitor the curriculum implementation

- Official quality audits/site visits


- Periodical reporting by programme directors

- Periodical reporting by the representative of every region of education


- Official and non-official quality audits/site visits by independent observers

- Digital questionnaire to all radiologists and residents


- Questionnaire to programme directors, radiologists and residents

aMandatory by the RDMA, possible solutions were limited

Systematic variation

For each subtask, several solutions were formulated (Table 7). For some subtasks the solutions were limited by legislation. For example, competency-based education was chosen by the RDMA as the teaching philosophy for all Dutch medical postgraduate training.


After the solutions were formulated, the HORA project team made a distinction between solutions that had to be solved in the short (within months), medium (within 2 years) and long term (within 5 years), and between decisions that ought to be made by the project team HORA or by other parties (see Table 7). The decisions that had to be made in the short term were prepared by the HORA project team and discussed in the third plenary meeting with all programme directors. Agreement could be reached on all short-term decisions (see Table 7). Many decisions were delegated to the regions of medical education to allow for optimal alignment to the local context, and to avoid information overload of the HORA project team. For each subspecialisation, a dedicated section of DRS members formulated quality criteria for the radiology department offering subspecialisations, along with an approval procedure for radiology departments, and a registration system for residents and radiologists who successfully complete a subspecialisation training programme.

Finally the total design of the curriculum (Table 7) needs to be tested against the design specifications (Table 6). All design specifications were met.

Unforeseen developments and approvals

After the three plenary sessions, the curriculum was completed pending approval by several authorities (Fig. 1). After approval by the Concilium Radiologicum, the curriculum was evaluated and approved by the DABPCD. The DRS approved the new curriculum in a general members meeting. Before formally approving the new curriculum, the RDMA decided to introduce their own educational advisory group (working group RDMA) to give educational approval. Although this working group approved the curriculum this caused a 1-year delay. Subsequently, the curriculum had to be formally approved by the RDMA. Because of a high workload (other medical specialties were also completing their new curricula), the RDMA approval procedure caused another 1-year delay. The final step was approval by the Dutch Minister of Healthcare. To avoid further delay, the working group RDMA advised the HORA project to start implementation in September 2008, whilst formal approval of the RDMA and the Dutch Minister of Healthcare was still pending. For this reason, the development and implementation process overlapped for about 15 months. The RDMA approved the curriculum in June 2009.


We have described the development procedure and main components of the new radiology curriculum in the Netherlands. Only when the new curriculum has been in use for a number of years will we be able to reflect on the components of the curriculum and its impact on the performance of young radiologists and the organisation of radiology departments. Here, therefore, we will discuss the systematic development process.

The highly dynamic and non-linear development process was influenced by many stakeholders, developments and unforeseen factors. Although this is the rule rather than the exception in such innovation and change projects [22], it is possible to discuss the usefulness of the systematic design procedures, and to identify success factors in this project of complex change.


The design process was largely specified beforehand in a logical step-by-step iterative approach. The specification beforehand proved valuable because it was clear to everyone what steps needed to be taken and which results needed to be achieved for particular meetings. Another strength of the discursiveness principle is that unforeseen developments can be incorporated. This seemed useful, especially during the curriculum approval phase. Despite the 2-year delay caused by an additional approval step and the high workload of the regulatory authorities, these steps significantly improved the quality of the curriculum.

Hierarchical decomposition

The HORA project team broke down the overall design problem into several subtasks. For the overall design six design specifications were formulated, and these were discussed with the programme directors. These steps proved extremely useful. First, it made the complex design task easier. Second, involving the programme directors from the start of the process in deciding on the specifications of the new curriculum made them responsible for the ultimate results and prevented them from re-discussing the specifications and the necessity of curriculum change in a later phase.

Systematic variation

The HORA project team, the regions of medical education, and the programme directors came up with several solutions for the subtasks, which allowed for an efficient division of tasks among individual members and subgroups, avoided information overload and involved the relevant stakeholders.


Making a distinction between short-, medium- and long-term decisions helped to focus the debate on decisions that needed to be taken at a specific moment in time. By outsourcing and decentralising some decisions, people were able to adapt these to local circumstances, and information overload of the HORA project team was avoided. It has been shown that innovations are adopted better and more quickly when adopters can tailor the innovations to their own needs [23].

Overall, the use of systematic design principles to structure the development process proved highly valuable. They led to a structured, yet flexible, development process in which creative solutions could be generated and adopters (programme directors, supervisors and residents) were highly involved. This is a prerequisite for the adoption and successful implementation of the new curriculum.

Strengths, limitations and suggestions for further research

The description of the design (content) of the new curriculum, as well as the development process, adds to the existing knowledge base and could be useful for scientific communities of radiology and programme directors who are about to start or are currently busy with designing a new curriculum. Several papers have appeared on curriculum design and evaluation of the CanMEDS framework within medical education [24, 25, 26, 27, 28] and radiology [29], but the integral description of the design and of the development process according to evidence-based systematic design principles has not been documented before.

There are some limitations. First, the paper lacks empirical data on the implementation and ultimately the effectiveness of the design of the curriculum and the application of design principles. It would be interesting to empirically test the implementation and the effectiveness of the subtasks of the design (Table 7), for example the organ systems-based modules (Does it improve the quality of radiological service? What is the influence on the quantity of requests for radiological research? Is there any change in the subjective perception of the radiologists of their jobs? What is the influence on effective manpower management?). It would also be interesting to empirically test the importance of every principle in such complex innovation processes. Second, the paper is limited to the Netherlands. Because of differences in patient populations and the organisation of radiology departments the curriculum probably needs some adjustments before it can be implemented in other countries.


The HORA project team incorporated many complex developments and succeeded in producing an exciting new radiology curriculum in the Netherlands. This curriculum is necessary for the future of radiology in the Netherlands, providing not only high quality training for radiology residents, but also advancing the science of radiology in the next few decades. The use of the systematic design principles proved useful in the development process of the new curriculum. The next challenge is to implement the curriculum in everyday practice.



We would to like thank the participants of the HORA project (J.C.N.M. Aarts MD, H.J. Baarslag MD PhD, E. Coerkamp MD, O.M. van Delden MD PhD, M.W. de Haan MD PhD, M. Heitbrink MD, F.B.M. Joosten MD PhD, S.P. Kerssemakers MD, P.N.M. Lohle MD PhD, A.K. Meininger MSc, R.B. Rouw MD, A. Tielbeek MD PhD) and the programme directors for their support and advice during the development process of the new curriculum.

Open Access

This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.


  1. 1.
    CanMEDS 2000 project (2000) CanMEDS 2000: extract from the CanMEDS 2000 project societal needs working group report. Med Teach 22:549–554CrossRefGoogle Scholar
  2. 2.
    Board of Health Care Professions and Education in the Netherlands (BHCPE) (2009) Accessed 5 June 2009
  3. 3.
    Blijham GH, Mens MAP (2005) OOR-zaak en Gevolg, Nederlandse Federatie van Universitair Medische Centra visie-document (Cause and effect, Dutch Federation of University Medical Centers vision-statement), Badoux drukkerij BVGoogle Scholar
  4. 4.
    European Association of Radiology (EAR) (2005) Detailed curriculum for the initial structured common programme, detailed curriculum for subspeciality training. EAR, ViennaGoogle Scholar
  5. 5.
    HORA project team (2009) Herziening Opleiding Radiologie (HORA) (Revision of radiology curriclum). Accessed 5 June 2009
  6. 6.
    Dalla Palma L (2006) Tomorrow’s radiologist: what future? Radiol Med 111:621–633CrossRefPubMedGoogle Scholar
  7. 7.
    Matin B, Bates DW, Sussman A, Ross P, Hanson R, Khorasani R (2006) Inpatient radiology utilization: trends over the past decade. Am J Roentgenol 186:7–11CrossRefGoogle Scholar
  8. 8.
    Scheele F, Teunissen P, Van Luijk S et al (2008) Introducing competency-based postgraduate medical education in the Netherlands. Med Teach 30:248–253CrossRefPubMedGoogle Scholar
  9. 9.
    Dochy F, Heyen L, Mosselaer H (2002) Assessment in onderwijs (Assessment in education). Lemma, UtrechtGoogle Scholar
  10. 10.
    Leenders R, van Engelen J, Kratzer J (2007) Systematic design methods and the creative performance of new product teams: do they contradict or complement each other? J Prod Inn Man 24:166–179CrossRefGoogle Scholar
  11. 11.
    Simon HA (1969) The sciences of the artificial. MIT Press, MAGoogle Scholar
  12. 12.
    European Society of Radiology (ESR) (2007) Good practice guide for European radiologists. ESR, ViennaGoogle Scholar
  13. 13.
    Miller GE (1990) The assessment of clinical skills, competence and performance. Acad Med 65:S63–S67CrossRefPubMedGoogle Scholar
  14. 14.
    Ten Cate O (2005) Entrustability of professional activities and competency-based training. Med Educ 39:1176–1177CrossRefPubMedGoogle Scholar
  15. 15.
    Epstein R (2007) Assessment in medical education. N Engl J Med 356:387–396CrossRefPubMedGoogle Scholar
  16. 16.
    Norcini J, Blank L, Duffy FD, Fortna G (2003) The mini-CEX: a method for assessing clinical skills. Ann Intern Med 138:476–481PubMedGoogle Scholar
  17. 17.
    Pendleton D, Schofield D, Tate P, Havelock P (2003) The consultation: an approach to learning and teaching. the new consultation: developing doctor–patient communication. Oxford University Press, OxfordGoogle Scholar
  18. 18.
    Violato C, Lockyer JM, Fidler H (2008) Changes in performance: a 5-year longitudinal study of participants in a multi-source feedback programme. Med Educ 42:1007–1013CrossRefPubMedGoogle Scholar
  19. 19.
    Wilkinson JR, Crossley JG, Wragg A, Mills P, Cowan G, Wade W (2008) Implementing workplace-based assessment across the medical specialties in the United Kingdom. Med Educ 42:364–373CrossRefPubMedGoogle Scholar
  20. 20.
    Bandiera G, Sherbino J, Frank J (2006) The CanMEDS assessment tools handbook: an introductory guide to assessment methods for the CanMEDS competencies. Royal College of Physicians and Surgeons of Canada, OttawaGoogle Scholar
  21. 21.
    Alves de Lima A, Barrero C, Baratta S et al (2007) Validity, reliability, feasibility and satisfaction of the mini-clinical evaluation exercise (Mini-CEX) for cardiology residency training. Med Teach 29:785–790CrossRefPubMedGoogle Scholar
  22. 22.
    Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O (2004) Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 82:581–629CrossRefPubMedGoogle Scholar
  23. 23.
    Rogers (2003) Diffusion of innovations. Free Press, New YorkGoogle Scholar
  24. 24.
    Chou S, Cole G, McLaughlin K, Lochyer J (2008) CanMEDS evaluation in Canadian postgraduate training programmes: tools used and programme director satisfaction. Med Educ 42:879–886CrossRefPubMedGoogle Scholar
  25. 25.
    Mickelson JJ, Macneily AE (2008) Translational education: tools for implementing the CanMEDS competencies in Canadian urology residency training. Can Urol Ass J 2:395–404Google Scholar
  26. 26.
    Frank J, Danoff D (2007) The CanMEDS initiative: implementing an outcomes-based framework of physician competencies. Med Teach 29:642–647CrossRefPubMedGoogle Scholar
  27. 27.
    Freiman A, Natsheh A, Barankin B, Shear N (2006) Dermatology postgraduate training in Canada: CanMEDS competencies. Dermatol Online J 12:6PubMedGoogle Scholar
  28. 28.
    Ringsted C, Hansen T, Davis D, Scherpbier A (2006) Are some of the challenging aspects of the CanMEDS roles valid outside Canada? Med Educ 40:807–815CrossRefPubMedGoogle Scholar
  29. 29.
    Baerlocher M, Asch M (2006) The interventional radiologist as “clinician”: what does it mean? Canmeds for the interventional radiologist. Can Assoc Radiol J 57:25–29PubMedGoogle Scholar

Copyright information

© The Author(s) 2009

Authors and Affiliations

  • Erik Jippes
    • 1
    Email author
  • Jo M. L. van Engelen
    • 2
  • Paul L. P. Brand
    • 3
    • 4
  • Matthijs Oudkerk
    • 5
  1. 1.Postgraduate School of Medicine, Wenckebach InstituteUniversity Medical Centre Groningen, University of GroningenGroningenthe Netherlands
  2. 2.Product Development and Strategy, Faculty Economics and BusinessUniversity of GroningenGroningenthe Netherlands
  3. 3.UMCG Postgraduate School of MedicineUniversity Medical Centre GroningenGroningenthe Netherlands
  4. 4.Amalia Children’s Clinic, Isala Klinieken ZwolleZwollethe Netherlands
  5. 5.Department of RadiologyUniversity Medical Centre Groningen, University of GroningenGroningenthe Netherlands

Personalised recommendations