To our knowledge, this is the first study to examine the impact of dual-processing instruction on the learning process of medical trainees. Despite three different experiments in three groups of trainees, verbal instructions to apply dual-processing did not result in improved overall diagnostic accuracy. While no signal of benefit was present among advanced learners, we cannot rule out a significant interaction between content and dual-processing instructions or between instruction and bias among intermediate and novice trainees.
Several aspects of this intervention that may have led to our findings need be addressed: application to a complex skill, content dependence, dependence on expertise level, and the strength of the instruction.
Application to a complex skill
Prior successes of reasoning instruction interventions were all within the realm of visual diagnosis (Ark et al. 2006, 2007; Hatala et al. 1999; Regehr et al. 1994), where the cognitive load is constrained to a visual space. Cardiac physical exam, similar to many clinical problems, involves the integration of multimodal perception. This added integration likely results in an inherently higher intrinsic and extrinsic cognitive load on the clinician (van Merrienboer and Sweller 2010).
The intrinsic cognitive load in cardiac diagnosis may be too great for clinicians to also allot working memory to deliberately guiding their reasoning practices. Clinicians may deal with the cognitive load by automating the more mundane components of the task. Further, use of procedural scripts may result in less opportunity to modulate the cognitive processes as directed in the reasoning instructions.
Instructions to promote dual-processing had a varied effect on diagnostic accuracy based on the content of the station. Diagnosis of the rarest lesion in each set was harmed by instructions for dual-processing (aortic coarctation in the experienced learners and atrial septal defect in the intermediates). In contrast, there was a suggestion of benefit in diagnosing common lesions among intermediates and overcoming bias among novices.
The nature of the content studied should be carefully considered in future research into dual-processing instructions. Consistent with this idea, Chamberland et al. (2011) have recently reported that case familiarity appears to have an influence on the benefit observed from having students engage in self-explanations relevant to their diagnostic performance. Sampling of cases needs to include a mix of rare and common, and likely a larger sample of total cases, in order to better understand the interaction of individual diagnoses and reasoning strategy and to reduce the amount of within subject variance that added noise to these experiments. The previously published ECG studies routinely used 14–20 test cases to overcome content dependence.
Importantly, the simulator has a finite set of abnormalities. While the findings themselves are authentic, the known limited repertoire of the simulator generates a problem space that is much narrower than clinicians face in routine practice. As a result, we may underestimate the need for dual-processing in clinical practice by using a research oriented model that allows non-analytic pattern recognition around a few key features with little variability in presentation.
Prior success of dual-processing instruction was found among absolute novices (Ark et al. 2006, 2007; Eva et al. 2007), but not among intermediate level trainees (Sibbald and de Bruin 2011). No overall effect of dual-processing instruction was found in our study among formal medical trainees across a range of expertise levels. However, as in the studies reported with absolute novices (Eva et al. 2007), dual processing instructions may have helped out relatively novice medical students overcome the biasing influence of an inaccurate diagnostic suggestion.
Diagnoses were specifically chosen to challenge each level of trainee so that the effect of dual-processing instructions on learning could be assessed. Our assumption was that the directive strength of verbal prompting would be greatest when faced with new material. However, the high probabilities assigned to the correct diagnosis across all levels suggested that most were familiar with these diagnoses.
Of note, even the novice medical trainees included in this study would have spent up to 100 h in the first several years of the medical school conducting cardiac physical exams. As a result, their exposure to the process is substantial. Given this experience, many may have already been taught to use a dual-processing approach or intuitively balanced their reasoning strategies thus negating any intervention effect. This is suggested by the similarity in the number of features identified by both reasoning groups, suggesting a similar cognitive strategy used by both. Alternatively, interfering with the reasoning process at a more advanced stage of clinical development may be detrimental as suggested by the expertise literature, which shows that manipulating the thought process of experienced clinicians may lead to an expertise reversal effect or paradoxical worsening of performance (Kalyuga 2007). Given the rapidity with which medical knowledge and skill is acquired in formal medical education and the speed with which previously seen cases have been found to begin influencing judgments on novel cases, finding absolute novices in a medical training setting may be difficult. Further study might target more novice first year medical students.
Directive strength of the intervention
Because cardiac physical examination is used on a daily basis, all of our participants may have an entrenched routine. As a result, it may be difficult to modify with a single verbal instruction. We incorporated both a learning and practice phase to encourage the use of the reasoning instruction; a successful approach in previous studies (Ark et al. 2006, 2007; Eva et al. 2007). However, we did not measure reasoning strategy directly, and cannot be sure whether this approach was effective in our context. Additionally, the test phase was delayed in these experiments for the groups most expected to reveal an influence of reasoning instruction (i.e., the more novice trainees). This was done for the sake of making data collection feasible. However, we may have missed an immediate benefit of dual-processing instruction among novices and intermediates. Nevertheless, it suggests, at a minimum, that if there was an effect of instruction it was short lived. Further work is needed to explore how to strengthen cognitive interventions to more effectively modulate behavior of more experienced clinicians.
In addition to the methodological issues outlined above, several other limitations should be mentioned. First and foremost, the number of trainees was small. Estimated power to detect a 20 % difference in probability with dual-processing was 0.88, 0.60 and 0.87 among experienced, intermediate and novice trainees (Lenth 2007). The study was underpowered to detect smaller effects, which may be educationally important. In addition, only four lesions could be tested because of the time constraints involved, which differs from the twenty ECGs used in the previous positive studies (Ark et al. 2006, 2007; Eva et al. 2007). Second, several trainees struggled in assigning diagnostic probability ratings; some provided probabilities that summed to >100 %. While this is a common approach to assessing decision making, it is not without important limitations. Trainees may disproportionately weight small or large percentages and may have different concepts of what percentage constitutes sufficient diagnostic certainty for ascribing a diagnosis to a patient or providing a therapy. Third, exaggerated findings on the simulator may have decreased the need and efficacy of dual-processing instruction. Finally, the group process implemented in the instructional and practice phases of this research to overcome the feasibility issues inherent in providing individualized training (as was done in the ECG studies) may have lessened the strength of the intervention. If some individuals in the group mentioned similarity to past examples and others deliberately focused on the feature presentations, then even those in the undirected group would have experienced dual processing practice.
The application of dual-processing strategies to reduce cognitive error requires further investigation. Its blind application to clinical medicine needs to be tempered by these results. As applied to cardiac auscultation on a cardiopulmonary simulator, instructions to encourage dual-processing were not successful in reducing cognitive error.
Future study should focus on the utility of promoting dual-processing among novices while also considering content dependence, intervention strength, the complexity of the skill required for diagnosis, and the role that group practice might play in overcoming the biases of individuals.