It is relatively easy to demonstrate that any individual single assessment in whatever format has severe limitations. In an easy to quantify area such as reliability, it is quite clear from many studies that substantial testing time and a lot of sampling is needed to achieve reliable scores, irrespective of the method being used. Most of our assessments in actual practice are unreliable. In a much more difficult area—how assessment influences learning—it is quite clear that much of our assessment is rather reductionist. Sometimes very poor learning strategies are reinforced (i.e., cognitive and behavioral rote learning), often opposing the intent of the curriculum. There is also very little information value in common assessment practices. We often limit the information to grades or to other quantitative information with little richness and therefore limited learning value. Our predominant system of assessment is to complete every modular element of a curriculum with an assessment. We hardly look at how a learner develops over time. Mastery of every element implicitly assumes mastery of the whole. In a similarly difficult area of validity, it is quite clear that any method of assessment is just a snap shot of what we wish or could assess with an individual learner. In all, trying to optimize everything in a single method of assessment at a single moment of time is doomed to fail. This notion is the basis for optimizing the assessment program as a whole.

In a programmatic approach to assessment, each individual assessment is seen as a single data point that provides only limited information on a learner. Given this limited information, we should optimize the assessment differently. Individual data points are not very suitable for high stake decisions. But they are suitable for giving feedback to the learner. In programmatic assessment, pass/fail decisions are removed from the individual assessment and the focus is on feedback. The feedback may be quantitative (in scores) or qualitative (in words) in nature all depending on the method of assessment. The more complex the skill we are assessing the more meaningful qualitative information will be. So individual data points are optimized for their learning value and richness of information, not for decision-making on learner progress. The decision-making can only be done with confidence when sufficient data points have been gathered. For decisions on leaner progress, other optimization strategies are in order. The stake of the decision is related to the number of data points needed. Very high stake decisions, for example promotion to the next year, should be based on many data points. Lower stake decisions—i.e., an intermediate judgment on learner progress—may have fewer data points. High stake decisions are made robust through all kinds of safe guards in the decision-making process. Usually decision-making is done by a committee. Amount of deliberation is depending on the clarity of the information. Earlier feedback cycles will limit the surprise element of decisions being taken. A justification to the decision-making will make it more acceptable. The size of committee and independence of committee members will contribute to the credibility of the judgment being made. All these elements of due process make the decision-making robust. Finally, better learning is promoted by mentoring. From the reflection and feedback literature, we know that learners need to be supported in using feedback to guide their learning. Therefore, in a programmatic assessment approach, mentors are used to guide and coach the learner longitudinally. Mentors also play a role in defining remediation activities that may be needed as a result of the incoming learning and assessment information.

So overall, programmatic assessment is an integrative, more holistic view on assessment in which not everything is optimized within one method or within one moment of testing time. Assessment is optimized differently for different purposes and collectively the assessment is robust for stimulating good learning and valid decision-making. Programmatic assessment is neutral in the choice of method of assessment. The choice for using a certain method at a certain point in time fully depends on the educational justification for using that method in that moment of time and its relationship to the total fit of the method in the program.

Some schools have adopted programmatic assessment successfully. A whole number of schools are working towards it. Quite clear is the fact that programmatic assessment is hard to implement. It requires a completely different orientation towards assessment, demanding a lot from teachers and from organizations. It requires a different culture around assessment and like any cultural change, they are hard to attain. Our current culture is a predominantly summative one with assessment as the constituent hurdles. In programmatic assessment, a learning oriented view is the predominant culture. The summative orientation to assessment fits to a classic “mastery oriented” or behaviorist view on education, whereas the programmatic assessment view fits to a more constructivist view on education. Competency-based education both at the undergraduate and postgraduate level of training strongly resonates with programmatic assessment. Time and further ongoing research will tell how useful this approach of assessment will be.

Further reading:

Schuwirth, L. W. T. and C. P. M. Van der Vleuten (2011). “Programmatic assessment: From assessment of learning to assessment for learning.” Medical Teacher, 33(6), 478–485.

Van der Vleuten, C. P., et al. (2012). “A model for programmatic assessment fit for purpose.” Medical Teacher, 34(3), 205–214.

Van Der Vleuten, C., et al. (2015). “Twelve Tips for programmatic assessment.” Medical Teacher, 37(7), 641–646.