Introduction

Over the last two decades, general practitioners (GPs) have been recognized to play a key role in early melanoma diagnosis [1, 2]. Early detection is essential to reduce morbidity and mortality of patients with melanoma [3]. However, melanoma diagnostic abilities of GPs in Europe have been demonstrated to be suboptimal [4,5,6].

In order to help GPs deal with suspicious skin lesions, melanoma diagnostic training programs have been developed. Initially, these programs trained GPs to recognize melanoma clinically (with the naked eye) [7]. Later, programs teaching diagnosis using dermoscopy became more widespread [8, 9]. However, due to the long training period required to become competent, dermoscopy is underused for the diagnosis of melanoma in general practice [4, 10, 11]. Therefore, it is essential, at the very least, to train GPs in naked-eye recognition of melanoma.

Most training programs teaching naked-eye melanoma recognition have been shown to improve GPs’ short-term diagnostic accuracy and management in a training setting (on photographs of skin lesions) [12,13,14,15,16,17]. Nonetheless, it is also important to ensure that the competences acquired by GPs are maintained over time [18,19,20]. Only four studies were found in which long-term (\(\ge\) 6 months) accuracy of GPs in melanoma diagnosis using clinical examination alone was evaluated [21,22,23,24]. The results of these studies were heterogeneous. One study showed a drop of GPs’ competences at 6 months post-training [21] and a second study showed a decrease over an 8-month period [22]. However, a major French study conducted in real-life demonstrated a positive long-term impact of their training with a decrease in the incidence of very thick melanomas (Breslow thickness ≥ 3 mm) over a 3-year period after a screening campaign performed by participating GPs [23]. Finally, a recent case–control study assessing 1662 referrals to dermatologists reported better quality of referrals by GPs trained in clinical melanoma diagnosis than by untrained GPs over a 10-month period [24]. Unfortunately, these two latter studies poorly evaluated the keys to the long-term sustainability of the acquired competences.

The aim of our prospective interventional study was to determine the short- and long-term competences of GPs after a 1-h training session, designed by our study team, on naked-eye melanoma diagnosis and management. We specifically sought to determine whether the competences acquired by the GPs were maintained over time and reviewed the reasons for their potential long-term decline.

Material and Methods

Study Design

This prospective interventional study took place over an 18-month period from November 2015 to May 2017 and was approved by the Ethical Committee of the UCLouvain, Belgium. A 1-h training session in melanoma diagnosis was integrated into the annual mandatory medical education program of eight groups of Continuing Medical Education from the five French-speaking districts of Belgium (Hainaut, Brabant-Wallon, Province du Luxembourg, Province de Liège, and Bruxelles).

Training Program

A 1-h training session, designed by our study team primarily targeting GPs, was common to all participants. The session included the teaching of three widely used naked-eye melanoma diagnostic tools (the ABCD rule [25], “pattern recognition” [26], and the “ugly duckling” sign [27]) and instructions in basic knowledge on melanomas and pigmented skin lesions (PSL) such as risk factors of melanoma, preferred locations of melanoma according to gender, normal/abnormal evolution of PSL according to the patient’s age, and characteristics of the most common benign PSL (seborrheic keratosis, typical nevus, blue nevus, congenital nevus, halo nevus, actinic lentigo, and subungual hematoma). Management of PSL was also taught during the training sessions. To illustrate the diagnostic tools, a large number of clinical photographs of melanomas and benign lesions were shown.

Data Collection

All the study data was collected using REDCap® (Research Electronic Data Capture, Vanderbilt University, USA) tools hosted at the Cliniques universitaires Saint-Luc in Brussels (Belgium) [28, 29].

Prior to training, the GPs completed a three-section questionnaire. This questionnaire was the result of a consent between two GPs (C.D. and S.L.) and a dermatologist (I.T.). Section A of the questionnaire collected socio-demographic and medical practice data, Section B investigated the GPs’ usual melanoma diagnostic practices, and Section C evaluated their basic melanoma knowledge and their diagnostic accuracy and management of PSL.

Section A—Characteristics of the Study Population

Participating GPs were asked about their gender, age, workplace (urban, suburban, or rural area), work practice (solo, pair, medical group, or local network practice), and the number of patients seen per year.

Section B—Melanoma Diagnostic Practices

To explore the way in which GPs diagnose melanoma and manage their patients in daily medical practice, ten multiple-choice questions were formulated.

Section C—Basic Melanoma Knowledge and PSL Diagnostic Accuracy and Management

To assess the efficacy of our melanoma training session, GPs completed this section of the questionnaire before training, immediately after training, and 1 year later. The details of Section A and B are described in a previous publication [6].

For this third section, participants were randomly subdivided into two groups, with 45 GPs in group A and 44 GPs in group B. Two sets (A and B) of similar questions were created covering each essential training topic. Each group of GPs received one of the two sets to fill in prior to and immediately after training. The questionnaires completed prior to training were collected by the organizing team before the start of the training session. The sets were switched between groups at 1-year post-training. To avoid recall bias, the questions on the same topic were similar but not identical (there was no perfect parallelism between the diagnoses and/or management proposed within these similar questions). This 1-year questionnaire was sent by return mail to the GPs.

The section C questionnaire was divided into two parts and all the questions and answers were peer-reviewed and validated by six dermatologists. The first part consisted of five multiple-choice questions that explored basic melanoma knowledge (see Appendix 1). GPs were asked about the preferred locations of melanoma according to gender, criteria for urgent melanoma management, characteristics of high-risk melanoma patients, and patient follow-up. In the second part, GPs were asked to assess 10 clinical photographs of PSL and their clinical history (see Appendix 2). Every clinical history included the patient’s age, gender, localization, and evolution (new onset or old lesion, change in shape or size) of the lesion. For each lesion, GPs were asked to select the correct diagnosis or the appropriate patient management from among five proposals. It is important to note that the clinical photographs used in the questionnaires were similar but not identical to those presented at the training session.

Statistical Analysis

The statistical analysis was performed using the SAS software (SAS© 9.4, SAS Institute Inc., USA). Descriptive statistics were used to summarize the results considering absolute numbers, percentages, and 95% confidence intervals for the three periods of assessment. In addition, the GPs’ average number of correct answers to the 10 clinical cases was summarized using mean ± standard deviation and the GP’s responses to clinical cases corresponding to melanomas were also grouped according to a threshold of 50% correct answers. McNemar’s test for binary matched-pairs data was used to test for variations between section C of the questionnaires prior to and immediately after training with regard to the number of GPs who correctly identified or gave the appropriate management for the clinical cases. All tests were considered to be significant for P value < 0.05. Statistical inference was not made for the comparison with the 1-year questionnaire because of the low number of GPs who responded to this questionnaire and the subsequent correction to be made on the type I error in case of multiple comparisons.

Results

As part of their mandatory continuing medical education, 89 GPs participated in the training and completed the entire questionnaire prior to training. These 89 GPs also filled in Section C (basic melanoma knowledge and PSL diagnostic accuracy and management) immediately after training. Despite several reminders, only 27 GPs (30%) completed Section C of the questionnaire and returned it by mail 1 year after training.

Section A—Characteristics of the Study Population

The GPs’ characteristics are summarized in Table 1. The overall average age of the 89 participating GPs was 45.6 years (median = 45 years) with a M/F sex ratio of 0.98. One year after training, 40% of the responding GPs were under 35 years of age and 70% of them were female.

Table 1 Characteristics of the study population
Full size table

Section B—Melanoma Diagnostic Practices

The results of this section according to the GPs’ socio-demographic background and practice type were examined and discussed in a previously published scientific paper [6].

Section C—Basic Melanoma Knowledge and PSL Diagnostic Accuracy and Management

The data collected before the training are available in the aforementioned study [6]. Short- and long-term results are the subjects of the present study.

Short-Term Competences

Eighty-nine GPs (45 in group A and 44 in group B) responded to Section C of the questionnaire prior to and immediately after training. Table 2 shows the GPs’ correct responses to the basic melanoma knowledge questions, and Table 3 shows the short-term evolution of the GPs’ PSL diagnostic accuracy and management. Comparison of data from the basic melanoma knowledge questionnaire shows a significant improvement in the immediate post-training outcomes of group B for lentigo maligna management much greater than for group A, and the reverse (a greater improvement in the immediate post-training outcomes for group A than for group B) for the question on characteristics of high-risk melanoma patients. It should also be noted that pre-training results differed between the two groups for melanoma patient follow-up and characteristics of high-risk melanoma patients.

Table 2 Short-term evolution of the GPs’ melanoma basic knowledge
Full size table
Table 3 Short-term evolution of the general practitioners’ appropriate diagnostic accuracy and management
Full size table

In addition, Table 4 illustrates the number of GPs who appropriately managed \(\ge\) 50% of the melanoma cases, i.e., at least one melanoma among the 2 melanomas of the 10 clinical cases in question set A and at least two of the three melanomas (including the lentigo maligna of the ear) presented among the 10 clinical cases in set B. As the number of melanomas differed in the two sets (A and B) of the questionnaire, we chose to use \(\ge\) 50% rather than absolute scores to present the results. This number doubled from 19 (42.2; CI =[27.8; 56.7]) GPs to 39 (86.7%; CI =[76.7; 96.6]) GPs in group A (P < 0.001) and increased from 21 (47.7%; CI =[33.0; 62.5]) GPs to 36 (81.8%; CI =[70.4; 93.2]) GPs in group B (P < 0.001). However, only a few benign lesions were better recognized as harmless and not requiring referral to the dermatologist (Table 3).

Table 4 The evolution of the general practitioners’ appropriately managed melanomas over time
Full size table

Long-Term Competences

Of the 89 GPs, only 27 (30%) completed the 1-year questionnaire. In total, 18 GPs of group A filled in question set B and question set A was answered by nine GPs of group B. Table 5 shows the long-term evolution of these 27 GPs’ competences in melanoma and PSL diagnosis and management.

Table 5 Short-term and long-term evolution of 27 general practitioners
Full size table

One year after training, the GPs’ correct responses to the basic melanoma knowledge questionnaire were maintained only for one question and increased for another question in group A. Unfortunately, this was not the case for the last three questions. In group B, the 1-year results were similar or even worse than the pre-training results except for one question. In the second part of the questionnaire, the number of correctly managed clinical cases significantly decreased in the long-term. In group A, the GPs’ mean overall score immediately after training of 5.1 ± 1.76 out of 10 correctly managed clinical cases dropped to 3.8 ± 1.52 at 1-year post-training. For the nine GPs in group B, their immediate post-training score of 5.9 ± 1.83 out of 10 correctly managed cases fell drastically to 2.6 ± 2.24 at 1-year post-training. This score was even lower than their pre-training score of 3.7 ± 2.00. Except for the blue nevus, benign lesions were already relatively unrecognized immediately after training and were still difficult for GPs to diagnose in the long-term. Furthermore, of the 18 GPs in group A, 17 (94.4%; CI =[83.9; 100.0]) appropriately managed \(\ge\) 50% melanomas immediately after training and only 9 (50.0%; CI =[26.9; 73.1] GPs did so in the long-term (Table 4). Unfortunately, the same trend was observed in group B with 8 out of 9 GPs (88.9%; CI =[68.4; 100.0]) who properly managed \(\ge\) 50% melanoma cases in the short-term, but only 5 (55.6%; CI =[23.1; 88.0]) who did so in the long-term.

Discussion

The aim of our prospective interventional study was to determine the short- and long-term competences of GPs after a 1-h training session on naked-eye melanoma diagnosis and management. We specifically sought to determine whether the competences acquired by the GPs were maintained over time and reviewed the reasons for their potential long-term decline.

Short-Term Competences

Results of the short-term evaluations revealed a significant increase in the immediate post-training assessment on most topics in the melanoma knowledge questionnaire. The greater improvements in the immediate post-training results of group B for lentigo maligna management, and group A for characteristics of high-risk melanoma patients, are most likely due to the non-identical but similar questions on the same training topic presented to both groups and mistakenly considered of equal difficulty by the authors. The difference in the pre-test results between the two groups for the questions on melanoma patient follow-up and characteristics of high-risk melanoma patients is probably also due to more difficult questions in set A than in set B.

The significant increase in the immediate post-training results of the PSL diagnostic accuracy and management are consistent with the main goal of the training session which was to teach melanoma diagnosis and management. However, most benign lesions were not diagnosed as such, and were referred to the dermatologist for further assessment or excision. Therefore, it is likely that the training did not focus sufficiently on the diagnosis of benign lesions and other differential diagnoses of melanoma, such as pigmented basal cell carcinoma. These PSL were still very poorly recognized after training which could result in over-diagnosis of melanoma. While over-diagnosis could lead to psychological stress for the patient, unnecessary referrals to dermatologist, and increased treatment costs [30], it would be in patients’ best interests for GPs not to underdiagnose melanoma, which is an aggressive cancer except when diagnosed at an early stage.

Long-Term Competences

Only 27 GPs (30%) responded to the 1-year questionnaire. Our main finding was that the competences they had acquired immediately after the training were not maintained over time.

Our results are in line with other studies and suggest that the long-term loss of acquired competences after training programs for GPs is common. Figure 1 shows the evolution of the GPs’ mean overall scores in diagnosing and managing skin lesions after nine different training programs in melanoma diagnosis. Overall, these graphs illustrate that GPs’ competences tend to improve for up to 6 months after training. However, beyond this limit, a gradual loss of competences occurs, despite the availability of refresher training material after six of the training sessions. The refresher training material varied between the different programs, and included unlimited e-learning access [31], self-assessment paper-based training sessions [32, 33], a second face-to-face training session at three months post-training [17], access to teledermatology feedback services [34], the provision of PDF-files of the course [35], and a triage algorithm for dermoscopic assessment of pigmented skin lesions [36]. Our study was one of the three training programs that did not provide refresher material [37,38,39].

Fig. 1
figure 1

a Short- and long-term evolution of general practitioners’ (GPs’) mean overall score in diagnosing and managing skin lesions reported in nine studies on training in melanoma diagnosis. b Distribution of the corresponding learning-curve slopes (\(\Delta \mathbf{S}\mathbf{c}\mathbf{o}\mathbf{r}\mathbf{e}/\Delta \mathbf{T}\mathbf{i}\mathbf{m}\mathbf{e})\) over time. The graphs make no distinction between sessions that trained GPs in naked-eye recognition of skin lesions and/or diagnosis using dermoscopy. However, we only included studies that specified the short- and long-term scores of the GPs’ competences in the period after the educational program. The learning-curve slope evaluates the improvement or deterioration in the GPs’ competences over a period of time (expressed in months). Figure 1b shows that the competences tend to improve when measured up to 6 months after training, while competences measured at 6 months and beyond tend to decrease. In addition, the availability of refresher training material had no impact on the sustainability of the GPs’ competences over the long-term. To note that Badertscher et al. [33] and Augustsson et al. [34] used the median score instead of the mean test score, which could be a limitation in the comparison of these studies since the median and the mean are not always equal

Full size image

There are several possible reasons for this long-term decline in the GPs’ competences, such as lack of daily opportunities to use the skills and knowledge obtained during training, lack of time in clinical practice to evaluate skin lesions, lack of confidence in their diagnostic abilities, the absence of prompt dermatologist’s feedback on the lesions assessed, and a gradual decline in the GPs’ motivation for self-training. Similarly, the low response rate (30%) observed in our study 1 year after training may also reflect a lack of time and loss of motivation for training in melanoma diagnosis. The low response rate of GPs may also possibly reveal a lack of confidence in their ability to assess skin lesions and fear of being judged for not having maintained the acquired skills.

Robinson et al. [36] and Marra et al. [24] proposed that refresher training might help physicians retain accuracy in melanoma diagnosis. However, they did not specify the type and form of these refresher activities. However, as demonstrated in Fig. 1, the long-term competences decreased in all studies, both in the training programs with additional refresher material and in those without it. We speculate that this decrease may be related to a progressive decline in the motivation and interest of GPs in skills they do not regularly use. In this respect, Marra et al. [24] encouraged the organization of “face-to-face” refresher training sessions at regular intervals in order to maintain appropriate management of skin lesions. This idea could be extended to interactive e-learning sessions, which have increasingly widespread since the COVID-19 pandemic in 2020. Finally, the ideal frequency of such refresher training sessions should be established. Figure 1 shows that the threshold between improvement and deterioration in competences was 6 months after training. This suggests that refresher sessions should take place at a regular frequency of at least once every 6 months.

In view of all these findings, we would recommend organizing some short interactive e-learning refresher sessions every three to 6 months after a training session. These sessions could consist of “real-life doctor-patient scenarios and problem solving” and should also include some valuable take-home messages that are easy to apply in the doctor’s office. As GPs have little time to devote to training, these sessions should not exceed 15 to 20 min. We are convinced that these refresher sessions would motivate GPs to maintain an interest in the field and encourage them to apply the competences they have acquired more regularly in their clinical practice.

Strengths and Limitations

Due to its geographic distribution in the five French-speaking districts of Belgium, the distribution of ages and gender, and the proportion of urban and rural GPs, the study population was representative of Belgium’s population of French-speaking GPs. As this study was conducted in the context of the GPs’ mandatory continuing medical education program, it is reasonable to assume that their inclusion was not significantly determined by their initial interest in melanoma diagnosis. Although the 10 clinical cases and five multiple-choice questions used in the two evaluation sets (A and B) were similar but not identical, we were able to avoid recall bias at 1-year post-training by switching the two sets of questionnaires between groups.

This study also has some limitations. Firstly, the 1-year questionnaire was sent to the GPs by return mail, which resulted in a significant drop-out of the participating GPs. Since only two of the four annual continuing medical education courses are mandatory, conducting the 12-month evaluation at one of these meetings, 1 year after the training, would have statistically reached 50% of the participants. In this context, we chose to send the questionnaire by mail in the hope of reaching more GPs. Unfortunately, this was not the case. As only 27 GPs completed the questionnaire, we could not test any significant inference on the outcomes of the three evaluation periods. Only descriptive statistics were obtained.

Secondly, the two groups of GPs who evaluated question sets A and B were constituted on the basis of their membership in the participating continuing medical education groups. Therefore, we cannot exclude socio-demographic differences between groups that could have influenced the pre- and post-training results. In addition, the process of switching the two evaluation sets (A and B) between groups made us realize that the questions in set A were more difficult than those in set B. This may also have influenced the results of the two groups.

Finally, this study assessed the GPs’ diagnostic and management skills solely in a training setting. When measured on photographs of skin lesions, the competences do not always reflect the GPs’ performance (changes in practice pattern) in their daily clinical practice. To further determine whether the measured competences reflect the GPs’ performance in daily practice, it would have been necessary to assess the number of melanomas the participants had diagnosed or referred to dermatology during the 12-month period between the two assessments.

Conclusion

This study measured the accuracy of GPs in the diagnose and management of PSL in a training setting after a 1-h training session on naked-eye melanoma diagnosis. Although the GPs’ competences improved in the short-term, they significantly decreased 1 year later. This is in line with other studies which showed that the skills acquired in training improved for up to 6 months, before rapidly regressing thereafter. To maintain and improve the GPs’ competences over a longer period, an essential aid may be the organization of timely refresher training sessions at regular intervals. However, further studies are needed to assess whether these refresher sessions are able to prevent the loss of the diagnostic and management skills that GPs acquire during trainings.