Background

The ability for appropriate self-assessment and reflection appear crucial in medical professionals but this core competency is limited across physicians [1]. It is therefore not surprising that self-reflection and reflective practice is an issue of high scientific relevance in medical education [2]. As a theoretical basis, different models of reflection exist, which illustrate reflective practice within two potential dimensions, i.e. an iterative dimension, whereby reflection is triggered by experience and a vertical dimension, which includes different levels of reflection on experience [2]. Against this theoretical background, many potential measures have been developed to assess the ability to self-reflect or self-assess. Williams et al. categorized them in a systematic review into three types, rubrics, self-reported scales and observed behavior [3]. These instruments need to be applied individually, depending on different influential factors, while no instrument appears superior to others [3]. Recently, further approaches to support and/or foster self-assessment and reflective practice have been reported, including virtual reality [4], briefing and debriefing sessions [5], or video-based approaches [6]. Those are only several examples from the emerging field of self-assessment and self-reflection in medical education research.

Recently, a novel tool has been introduced to support self-reflection and to facilitate learning progress. PRISM is a visual tool based on a metaphor [7]. The applications for PRISM are broad; while initially developed to visualize suffering, a growing variety of applications have been reported [8]. The structure of the PRISM task as a visual metaphor allows it to generate information about personally salient appraisals [8]. Thus, this method has also a potential in medical education. To the authors’ knowledge this is the first study to examine the use of PRISM in medical education.

The aims of this pilot observational study were to demonstrate the potential of PRISM as a tool to I) facilitate and II) quantify perceptions of learning progress in a small group of clinical dental students and to test the reliability of the measure. Based on these two aims, the study tested two hypotheses: I) repeated quantitiative measurements of PRISM are stable within one week without education events in a mixed cohort of undergraduate dental students. II) students’ PRISM responses will reflect expected improvements in competencies (including knowledge, skills and perceived training need) during a particular course.

Methods

Study design

This observational study has been reviewed and approved by the ethics committee of the Medical Faculty of University of Leipzig, Germany (No: 117/20-ek). All participants were informed verbally and in writing about the study and provided their written informed consent. The study consisted of two parts: (I) repeated application of PRISM in a cohort of 30 undergraduate dental students over one week without any education events and (II) the evaluation of PRISM findings during a clinical simulation course in conservative dentistry and periodontology.

Participants

Volunteers were recruited among undergraduate dental students in their clinical years of study. For part (I), 10 students each from 3rd, 4th and 5th year of study were included. The 3rd year students were recruited prior to their simulation course and 4th as well as 5th year students were recruited after their clinical course in conservative dentistry and periodontology. For part (II), 29 students in 3rd year were included and followed-up during eight weeks of their clinical simulation course in conservative dentistry. Students were included irrespective of their age or gender. There were no further inclusion or exclusion criteria.

PRISM interviews

PRISM was initially developed in the field of psychology/psychosomatic medicine to assess and visualize patient suffering [7]. For the PRISM task, a white metal board (297 × 210 mm), represents a defined context (in this study, ‘your dental studies’). The board includes a fixed yellow circle (7 cm in diameter) at the bottom right hand corner representing the identified Subject (in this study, ‘myself as a X-year dental student’). Different coloured Object discs (5 cm diameter) can then be placed in relation to the Subject. The Object discs represent different study-related issues, e.g. “your theoretical skills in the field of conservative dentistry”. For quantitiative assessment, the distance between Subject and Object was measured in millimeters (Fig. 1). Beside of the total distance between Subject and Object, the relationship between Objects was also assessed according to the distance between them and whether different Objects were placed in a close or branched cluster (Fig. 2). This clustering was assessed to evaluate the perceived relationship between the different sub-topics of the same topic (disciplines of conservative dentistry). This allowed assessment of the ways in which students distinguish between the sub-topics and whether their perceptions change during the course. For example, placing sub-topics into a close cluster was taken to indicate a low distinction between the topics, what might be caused by limited experiences and knowledge in this respect. Placing topics in a branched cluster was taken to indicate a high distribution between topics, potentially related to a gain in experiences in the different sub-topics.

Fig. 1
figure 1

Measurments of the PRISM task. All distances were measured in millimetre using a ruler. A: Distance between the outer edge of Object and Subject disc were measured as the main quantitative result. B: Distance between different Objects can be measured and were in the current study only used to classify the cluster of Objects (see Fig. 2). C: If any Object disc touches the edge of the Subject disc, its distance was set as “0 mm” (as was the distance for Objects placed in the center of the Subject)

Full size image
Fig. 2
figure 2

Illustration of Object clustering. A Objects are close to each other (“close cluster”), B Objects are branched with a remarkable distance between each other (“branched cluster”)

Full size image

PRISM interviews

One dentist (GS), trained in the use of PRISM, performed all the interviews. This dentist was not involved in the general appraisal of the clinical courses of the students. The PRISM task was used in addition to and independently of students’ regular appraisals. Interview settings were standardized as far as was possible. Interviews started with an explanation of the task and one example question (“Imagine this blue disk is burger and this green disc is broccoli. Where would you put these disks to reflect how much do you like them at the moment?”). A short explanation was repeated at each interview appointment. Interviews consisted of 18 questions, regarding students; skills, knowledge and their perception of remaining training needs in different fields in dentistry (Table S1). The task aimed to evaluate students’ subjectively perceived competencies, i.e. their recent self-perceived knowledge, practical skills and related remaining training needs. In part (I), all participants underwent three interviews within one week (Monday – T1, Wednesday – T2, Friday – T3), including the same questions (each 10–12 min). For part (II), students were interviewed at the beginning, middle (after four weeks) and end of their simulation course in conservative dentistry. Each interview lasted 10–12 min.

Statistical analysis

The statistical analysis was performed with SPSS for Windows, version 24.0 (SPSS Inc., U.S.A.). Values are presented as mean values with standard deviation or percentage, respectively. For part (I), values between the three time points were compared to each other for the total cohort and also in the three sub-groups (3rd, 4th and 5th year). For part (II), values for the total cohort were compared between beginning, middle and end of the simulation course. Normal distribution was tested with Shapiro–Wilk test. Based on normality of distribution, the general linear model or Friedman test was applied, while sign test was used for ordinal data. More than two independent, non-normal distributed samples were compared using Kruskal–Wallis test. For the analysis of categorical data, a chi-square test was applied. The significance level was set at p < 0.05.

Results

Participants

For Part (I), 30 students were included, which had a mean age of 23.9 ± 3.3 years and a balanced gender distribution (50% male and female, respectively). In this group, 10 students each were in 3rd, 4th and 5th study year. The sample for Part (II) consisted of 3rd year students (n = 29) undergoing a course in conservative dentistry. They had a mean age of 22.9 ± 3.0 years and had a majority of female participants (72% vs. 28%).

Evaluation of PRISM after 3 interviews

When PRISM was repeated at three times within one week (Monday, Wednesday, Friday), the variation within the whole group (N = 30) between values did not reach statistical significance (p > 0.05, Table 1). Among the three subgroups, 4th and 5th year dental students showed stable values across the week (p > 0.05, Table 2). Within the 3rd year students, significant differences were found between the three measurement points (p < 0.05) despite the students having had no educational events during the week, while only the clusters remained stable (Table 2).

Table 1 PRISM results for the total cohort (n = 30) including students from 3rd, 4th and 5th year (each n = 10) at three time points within one week (Monday, Wednesday, Friday). Values are given as mean and standard deviation and represent values in millimetre. Significance level was p < 0.05
Full size table
Table 2 P-values for the comparison of PRISM values between T1-T3 within students from 3rd, 4th and 5th year (each n = 10). Significance level was p < 0.05
Full size table

Comparing the three subgroups, i.e. 3rd, 4th and 5th year students (each n = 10), 3rd year students had significantly higher values in theoretical knowledge, practical skills and training need, including the whole field of conservative dentistry as well as all sub-topics (p < 0.01, Table 3). This indicates that 3rd year students perceived their theoretical knowledge and practical skills as less well developed and their training needs as greater. Also, 3rd year students showed a lower amount of branched cluster of objects than the other two groups (p < 0.05, Table 3).

Table 3 Comparison of PRISM data between students from 3rd, 4th and 5th year (each n = 10) at the third PRISM interview within one week (T3). Values are given as mean and standard deviation and represent values in millimetre. Significance level was p < 0.05
Full size table

Evaluation of PRISM over 8 weeks of dental education

During the 3rd year stusents’ simulation course, the PRISM task showed significant reductions in the distances between the Subject and Objects reflecting conservative dentistry or its sub-topics (p ≤ 0.01, Table 4). This indicated that over the duration of the simulation course, students perceived improvements in their skills and knowledge as well as reduction of their training needs. There was also a substantial change in clusters during the course – there were more branched clusters at the end of the course than at the beginning across the sub-topics regarding their practical skills (p < 0.01) and training need (p = 0.01; Table 4). This indicates that students at the end of the course distinguished between the sub-topics, potentially as a result of increased experiences and knowledge in those topics.

Table 4 PRISM data for the total 3rd year cohort (n = 29) at three time points during the course of conservative dentistry (beginning, middle (after 4 weeks), end of the course (after 8 weeks)). Values are given as mean and standard deviation and represent values in millimetre. Significance level was p < 0.05
Full size table

Discussion

Main results

In a mixed cohort of undergraduate dental students in their clinical years, PRISM values remained stable over one week in the absence of any education events. This supports the reproducibility of the PRISM task i.e. its reliability. Over the duration of a clinical simulation course, PRISM indicated that students’ perceived competencies improved, indicating that PRISM is sensitive to visualize self-perceived learning progress.

To the authors’ knowledge, this is the first quantitative assessment of PRISM in medical or dental education, or indeed any higher education. There is therefore no directly comparable published work. As noted in the introduction, PRISM is a visual metaphor initially applied to measure suffering [9] but its use has extended more recently to a growing range of different applications [8]. In its original application, the PRISM task has been demonstrated to have good reliability, yielding consistent results when repeated at intervals of a few hours to three days [9,10,11]. Further studies confirmed the reliability of PRISM to assess the disease burden of patients, e.g. for patients with psoriasis [12], and even for measuring pain-related suffering across different countries [13]. The present results of part (I) of the current study are consistent with this, indicating the PRISM yields reliable data in the setting of dental education.

In this context, it was conspicious that 3rd year students showed less stable results in part (I) than the 4th and 5th year students. This might be explained by the structure of dental studies in Germany. In the first two years of dental studies, students focus on basic subjects in medicine and dental technology. In their 3rd year, students encounter conservative dentistry and its sub-topics for the first time. Students might have difficulty understanding their competencies in a subject they have never heard about, or may even mix up the sub-topics across interviews. This would result in the limited reproducibility in this particular year group. On the other hand, one could argue that students should perceive themselves as incompetent in a subject in which the students had no experience or teaching. On this basis, PRISM values would be expected to be particularly stable in this year group. Another possibility is that, as expected, the PRISM task encourages self-reflection by the students and they attempt to guage to what extent their learning to date will be relevant to topics they have not yet encountered. Further work is needed to resolve this.

Quantitative analysis for PRISM has been repeatedly performed in its original application form; different clinical studies have measured the suffering or disease burden of patients, e.g. with ulcerative colitis, chronic inflammatory vulvar disease, liver cirrhosis or organ transplantation [14,15,16,17]. These studies choosed different ways of quantification, including measurement in centimentre, millimeter or distinction of values [14,15,16,17]. Based on the individuality of the PRISM task, a remarkable range is often observed, especially if values are measured in millimeter, however, this also brings a high sensitivity [15].

A reduction of the distance between Subject and Objects was observerd during the clinical simulation course, reflecting improvements in students’ perceived competencies. This supports PRISM as a sensitive tool to visualize and measure learning progress in undergraduate dental students. As expected, the largest difference was between 3rd year and the two other year groups (Table 3). This can be explained by the fact that 3rd year students undergo the simulation course, while 4th and 5th year students work with real patients. It is recognised that pre-clinical simulation in dental education is not entirely satisfactory, because simulated and real patients differ significantly [18]. Considering the simulation course alone, 3rd year students perceive a substantial improvement in their competencies during the course (Table 4).

The clustering of Objects i.e. students’ perceived competencies yielded further information about the students’ perceptions. At the beginning of the 3rd year simulation course, the students placed all the sub-topics in a tight cluster, consistent with a lack of discrimination, expected because of their lack of prior experience of the sub-topics. As the course progressed, the sub-topics showed a branched structure, consistent with the students developing a more differentiated and nuanced appraisal of the sub-topics due to their teaching and their growing practical experience. As expected, the branched structure of sub-topics was also evident in the responses of the 4th and 5th year students. The clustering of PRISM Objects might therefore provide a visual summary of the student’s conceptualization of a domain of knowledge. How, using PRISM, a student clusters the sub-topics of one overall topic or domain e.g. the whole field of conservative dentistry, may provide insights into how the students perceives structural relationships and hierarchies among topics or sets of technical skills, and into the student’s capacity to distinguish among different components of knowledge or technical performance. This is also expected to substantially facilitate discussion between student and teacher not only because PRISM provides a visual summary as the basis for discussion but also because of the properties that PRISM shares with other metaphors [8]. For these reasons, further work on clustering of Objects in the PRISM task in medical or dental education settings might be fruitful.

This pilot study has shown that PRISM can be applied reliably to assess quantitatively students’ perceptions of their learning needs and is sensitive to change. Further work is needed to assess how PRISM compares with existing numerical scales and other appraisal tools used in dental education. Numerical and similar scales are convenient for students and teachers because they are simple to administer and evaluate [19]. PRISM, although brief to administer, takes longer to administer and evaluate that a umerical scale. However, PRISM has the advantage that it can be readily incorporated into the appraisal discussion between student and teacher, as has been shown in others of its applications [8].

Strengths and limitations: The main limitation of the present work is that it is a pilot study, involving a small number of selected students and a single teacher. Comparisons between the year groups were limited because of their small numbers. The students were volunteers and might thus have been particularly motivated to work with PRISM. The teacher was well trained and experienced in the use of PRISM and some training will be necessary for other teachers applying PRISM. Also, without knowing whether the PRISM interviews were likely to work, it would have been wrong to integrate them into the students’ course, but this meant that the PRISM interviews were separate from and additional to the course. However, the results indicate the feasibility of applying PRISM to dental education. Nevertheless, further work is necessary to confirm the generalizability of the findings. As noted above, there is no gold standard for student self-appraisal, but further work should at least compare PRISM with other self-appraisal tools.

The current study has several key strengths. It evaluated a novel tool for self-assessment of competencies and learning needs in undergraduate dental students. Although the PRISM measure has been applied in a variety of settings, to the authors’ knowledge this is the first study in which PRISM has been applied in higher education. It is also the first study in which interpretation of PRISM has been extended to include clustering of the objects under investigation. The results of the study have been promising – even with the small numbers of students recruited, the PRISM task has been shown to be reliable and to yield results consistent with expectations.

Conclusion

This pilot study has demonstrated that PRISM has potential as a reliable tool to quantify perceived competencies in undergraduate dental students in their clinical years of study. Furthermore, PRISM generates a visual summary of each student’s perceived learning progress. Even allowing for the limitations of the study, its results suggest that use of PRISM would be a promising approach to extend the instruments for appraisal in dental education. If the generalizability of PRISM is confirmed among dental students, it should also be applicable to medical students as well as to those following postgraduate studies.