Background

With the development of information science and technology (IST), digital health has begun to play an important role in clinical practice [1]. The World Health Organization (WHO) has proposed digital health as an umbrella term that broadly defines the use of digital technology in health care [1]. The spread of coronavirus disease (COVID-19) has highlighted the importance of telemedicine, indicating its significance in digital health [2]. As IST is expected to continue to develop in the future, digital health will play an even greater role [3], and this will require health professionals to adapt to it [4,5,6,7].

It has been reported that physician competencies regarding eHealth and telemedicine include the appropriate use of IST based on the understanding of its benefits and risks, access and equity in digital technology, practice of patient care using IST, and ethical considerations [8, 9]. However, in a recent study on the development of digital health competencies, first- to seventh-year European medical students (N = 451) self-assessed competencies such as remote patient monitoring systems and the application of AI in radiology as low or very low [10]. Although many current medical students are digital natives, they lack the knowledge and skills required to utilize IST in the context of medicine [10,11,12].

Consequently, recent studies have highlighted the importance for medical students to acquire the competencies necessary to use IST related to digital health [11,12,13]. Competencies necessary to use IST in medicine and learning objectives related to the confidentiality of medical information have been included in pregraduate education in the United States, the Netherlands, Canada, the United Kingdom, and Singapore [14,15,16,17,18]. Additionally, it was recently reported that medical students’ digital health literacy includes skills for using electronic health records and digital ethical attitudes as well as knowledge of the advantages and disadvantages of telemedicine and medicine using advanced technologies [13]. In Japan, learning objectives related to the use of IST are described as competencies in the Model Core Curriculum for Medical Education (MCC)—an official guide for undergraduate education [19]. In the 2017 revision, only the operation of electronic health records and protection of patients’ electronic personal data were listed as subcompetencies [20]. In the 2022 revision, the use of IST was listed as one of the 10 core competencies, along with a variety of additional subcompetencies, such as the understanding of ethics and the basic principles of IST applied to medical research and medical care.

Currently, medical education in Japan focuses on the operation of electronic medical records and the protection of personal information [21, 22], while the other aspects of IST literacy education remain ambiguous. In addition, information is lacking on how students who have studied under the current curriculum self-assess their competencies to use IST, and what factors enhance their self-assessed competencies.

Therefore, we conducted a questionnaire survey among sixth-year medical students in Japan who were studying how to operate electronic medical records and protect patients’ medical information in the current Japanese curriculum. Further, we identified the factors related to self-assessment regarding the acquisition of competencies necessary to use IST. We addressed the following research questions:

Research Question 1: How do Japanese medical students under the current curriculum self-assess the acquisition of the competencies required to use IST?

Research Question 2: What factors are related to improvement in the self-assessment of the competencies that are graded as low?

Through this assessment, we expect to provide essential data for future medical curriculum management.

Methods

Study design

We conducted a questionnaire-based study.

Study setting and participants

We conducted the survey from November 1, 2022 to February 28, 2023 among final year (sixth year) medical students at 82 medical schools in Japan. After graduating from high school, Japanese medical students enter medical school following completion of courses in English, mathematics, natural sciences, and social studies, but not in informatics. In medical school, students generally complete a six-year curriculum, with most completing it by November of their final year, after which, there is a period focused on preparation for the national medical examination. Given this background, we determined that this was the appropriate time to survey the acquisition of these competencies in Japan’s current medical curricula.

We used G*Power 3.1.9.6 [23] to determine the sample size, assuming a moderate effect size (f = 0.15) based on Cohen [24]. Our sample included 143 students (5% significance level), 80% power, and 16 independent variables.

Measures

We consulted the learning objectives described as competencies in the MCC 2022 Revision to prepare the items for assessing medical students’ competencies necessary for using IST.

The MCC follows the concept of outcome-based education and identifies practical competencies that students should acquire and master by the time they graduate. The learning objectives are divided into four tiers: the first tier outlines the overview and objectives of the competencies; the second tier describes the components of each competency; the third tier, specific competencies are expressed by nouns (e.g. Knowledge of advanced IST); and the fourth tier describes the actions that learners can take and the expected outcomes. The first tier of information technology includes one item indicating IST, the second tier includes four, the third includes six, and the fourth includes 13. This curriculum is updated every three to five years after a review by medical education experts. Participants’ responses regarding the 13 items examining the learning objectives in the fourth tier (indicated by specific behaviors) were rated on a 4-point Likert scale (1 = not able, 2 = not very able, 3 = somewhat able, and 4 = able). Further, we assessed the presence or absence of information technology classes at medical schools using the Tier 3 learning objective categories to broadly capture the implementation of the classes at each university (0 = none, 1 = yes).

We used Microsoft Forms to create a questionnaire comprising three sections. The first section contained questions on sociodemographic data, such as medical students’ age, gender, type of institution (medical university or medical school of a general university), and whether they have had any work experience with IST. The second section asked about the students’ learning experiences before and after entering medical school, including whether they had learned about IST inside or outside school, when they first used IST inside or outside school, and whether they had experiences in problem-solving classes. The final section inquired about medical students’ competencies necessary to use IST. Table 1 outlines the relevance between the four-tier subcompetencies in the MCC and the question items in the present study.

Table 1 Correlation between MCC Four-Tier subcompetencies and Study question items
Full size table

Data collection

After obtaining permission from the 82 medical schools in Japan, we recruited participants for the survey. We mailed information about the study, along with a survey form to the schools. The faculty of each school distributed this information to sixth-year medical students. The students received a detailed research description that explained the purpose and significance of the study, outlined the survey items, and emphasized that participation was anonymous and voluntary.

Data analysis

For the items on competencies necessary to use IST based on the fourth tier learning objectives, we examined the means and standard deviations, medians and interquartile ranges. Further, we used Cronbach’s alpha to assess internal consistency—an alpha coefficient of 0.7 or more indicating a sufficient level of consistency [25].

To examine the factors that improve the self-assessment of the fourth tier learning objectives, we conducted a logistic regression analysis in which self-assessment was divided into two values: “able/somewhat able” and “not able/not very able.” To examine the factors that improve the low self-assessment of competencies, the learning objectives of the fourth tier that had a higher percentage of “not able” and “not very able” were used as the dependent variables in the logistic regression analysis. We chose factors that were associated with self-assessment in the univariate analysis as the independent variables (with a criterion of p < 0.25) [26].

In this study, we considered p < 0.05 as statistically significant, and performed all statistical analyses using EZR. We used the graphical user interface of R (The R Foundation for Statistical Computing, Vienna, Austria; Saitama Medical Center, Jichi Medical University, Saitama, Japan) to perform the analyses [27]. A medical statistician (MM) validated all the statistical analyses.

Ethical consideration

The study description stated that participation was voluntary and would not affect the students’ grades. Medical students who agreed to participate accessed the questionnaire.

The study was approved by the Ethics Review Committee of Jichi Medical University School of Medicine (reference number: 22–097).

Results

Demographic characteristics and pre- and post-medical school learning experiences

The 824 participants who agreed to participate in the study provided valid responses with no missing values for any questionnaire items. Of the participants, 63.7%, 35.2%, and 1.1% were male, female, and others, respectively. The median age was 25 years (IQR: 24–26). Of the participants, 50.9% were affiliated with a medical university. A total of 2.63% of participants had work experience in IST prior to medical school. Furthermore, 55.7% had learned IST in the classroom, and 23.7% had learned it outside the classroom before entering medical school. Participants who had started using IST in the classroom while they were in elementary school were the highest percentage (39.2%). In contrast, 44.5% of the respondents started using IST for self-study after entering medical school. In addition, 52.7% of the students attended problem-solving classes before entering medical school. Table 2 presents the percentage of students who attended classes on IST after entering medical school.

Table 2 Attendance rate for Information Science and Technology Classes among Medical School entrants
Full size table

Self-assessment of competencies required to use IST

To determine the reliability of the overall competencies required to use IST, we checked the correlation matrix among the items and calculated Cronbach’s alpha. None of the items had a correlation coefficient lower than 0.4 (Appendix 1). The Cronbach’s alpha coefficient for the overall competencies was 0.853, and no item (Appendix 2) had a Cronbach’s alpha coefficient lower than 0.9.

Table 3 presents the results of the self-assessment of competencies mentioned under the fourth tier learning objectives. The mean scores for IT-01-02-01 and IT-01-02-02 were the highest at 3.50 and 3.37 (SD 0.82 and 0.69). In contrast, more than 30% of the students answered “not able or not very able” on three items (IT-01-01-02, IT-01-01-03, and IT-02-02-01).

Table 3 Self-Assessment of Information Science and Technology Competencies for Fourth-Tier Learning objectives
Full size table

Relevant factors affecting the learning objectives related to competencies self-assessed as low

We conducted a logistic regression analysis using the self-assessments of IT-01-01-02, IT-01-01-03, and IT-02-02-01 as dependent variables (Table 4). We did not consider gender with a variance inflation factor value greater than 2 as an independent variable because of multicollinearity (Appendix 3).

Table 4 Logistic Regression Analysis for Information Science and Technology Competencies Self-Assessments
Full size table

In the model that examined IT-01-01-02, the experience of taking classes IT-03-01 and IT-03-02, as well as problem-solving classes, was statistically significant. In the model that examined IT-01-01-03, the factors experience of taking classes IT-02-02 and IT-03-02, and experience with problem-solving classes were statistically significant. Similarly, in the model examining IT-01-01-03, the factors experience of taking classes IT-02-01, IT-02-02, IT-03-01, and first use of information science and technology outside of school were statistically significant.

To evaluate the goodness-of-fit of the logistic regression models, we used Nagelkerke’s pseudo R-squared (Nagelkerke R²). The logistic regression analysis showed that the Nagelkerke R² values were 0.2194 for IT-01-01-02, 0.1923 for IT-01-01-03, and 0.2195 for IT-02-02-01, indicating that the analyses for IT-01-01-02 and IT-02-02-01 had similar explanatory power, each explaining approximately 21.94% and 21.95% of the variance respectively. The analysis for IT-01-01-03 had a slightly lower explanatory power, explaining about 19.23% of the variance.

Discussion

We conducted this cross-sectional investigation to examine the extent to which Japanese medical students perceive that they have acquired competencies to use IST upon completion of the curriculum, and the factors that may promote or hinder such acquisition.

We found that (1) medical students’ self-assessed certain competencies as high and others as low. Those with particularly low self-assessment included “Understand an overview of the regulations, laws, and guidelines related to information science and technology in medicine” (IT-01-01-02), “Discuss ethical issues, such as social disparities caused by the digital divide, that may arise in the use of information science and technology in medicine” (IT-01-01-03), and “Understand information science and technology related to medical care (medical information systems, wearable devices, applications, artificial intelligence, telemedicine technology, and the Internet of Things [IoT]) and discuss their potential applications” (IT-02-02-01). (2) The factor associated with improvement in the three self-assessed lower learning objectives (IT-01-01-02, IT-01-01-03, and IT-02-02-01) was the experience of taking a specific IST class at the medical school. Additionally, we found that for IT-01-01-02 and IT-01-01-03, participation in problem-solving classes before medical school was associated with improved self-assessment. For IT-02-02-01, the experience of learning IST outside formal classes before entering medical school correlated with higher self-assessment.

Factors correlated to IST-related subcompetencies pertaining to regulatory and ethical issues

IT-01-01-02 and IT-01-01-03, which define subcompetencies addressing regulatory and ethical issues related to IST in medicine, were associated with experiences with problem-based learning prior to medical school and experience with IT-03-02 classes (learning skills using IST in medicine classes in medical school). As IST is constantly evolving, the relevant regulations are changing [28], and in Japan, the Act on the Protection of Personal Information in Relation to Data Privacy [29] is revised in principle every three years. Medical students must adopt a learning strategy that responds to rapid changes, and they must learn to apply the knowledge and skills they have acquired. Therefore, they must be metacognizant of their own knowledge and skills in IST and explore learning issues on their own. This learning process is consistent with the problem-solving learning framework [30]. The reason for the low self-evaluation of these two academic objectives may be related to the fact that the participants had little experience with problem-solving-based learning before entering medical school. Although Japan’s courses of study have been repeatedly revised since 1990 based on the standard of being able to spontaneously identify problems, think independently, make judgments, and express oneself, the 2010s did not see a departure from rote learning [31]. Therefore, participants in this study may have lacked experience with problem-based learning before entering medical school; in fact, 47% of the students had no experience with problem-based learning. Furthermore, a survey conducted by the Ministry of Education, Culture, Sports, Science and Technology indicated that even for 53% of students with experience, many were not good at problem-solving learning when they enrolled in junior high school [32]. In other words, they were unable to set their own tasks and learn about the rapidly changing regulations and ethical issues related to IST; consequently, their self-assessment was low.

Factors correlated to subcompetencies pertaining to IST use in medical practice

The results suggest that the low self-assessment of IT-02-02-01 is related to a lack of learning opportunities associated with IST in medicine within the curriculum. We found that students had the least experience with the IT-02-02 (advanced knowledge of IST class), but their experience with this class was associated with an improvement in the self-evaluation of IT-02-02-01. The correlation between the self-assessment of IT-02-02-01 and the “experience of learning IST outside of regular classes before entering medical school” suggests that the habit of self-learning content that cannot be learned through the MCC is essential for acquiring knowledge regarding IST in medical care. Among the participants, 24% had learned IST outside the formal curriculum before entering medical school, which may have resulted in the low self-assessment of IT-02-02-01.

Thus, learning experience within the curriculum is one of the most important factors in acquiring knowledge on IST in medicine, and therefore, future studies should re-examine whether IT-02-02-01 can be improved by a curriculum based on the MCC 2022 Revision. However, reports from other countries state that computer literacy indicators for medical students focus on software use and Internet safety [21, 22]. These studies assume that there is a lack of learning experience related to IST in medicine within the curriculum. Additionally, as IST is constantly evolving, maintaining an up-to-date curriculum that comprehensively covers the subject can be challenging. Establishing fundamental competencies for active learning for medical students and enabling them to continuously learn about IST on their own may be an essential improvement measure for acquiring these competencies.

Limitation and future research plan

To the authors’ knowledge, this study is the first to describe medical students’ self-assessment of competencies related to the use of IST and the factors that affect this assessment. The results of this study are expected to be used for future curriculum management of competencies related to IST.

However, this study has several limitations. First, it was conducted only among Japanese medical students. This may affect the results’ generalizability because medical schools in Japan follow a six-year pattern and medical students extensively use IST in their daily lives. Second, this study was conducted over a single year; thus, it is possible that events that occurred in FY2022 may have affected the results. Finally, the survey on the acquisition of competencies to use IST was based on self-assessment by medical students. It is possible that the students’ responses contained cognitive biases, which questions the credibility of the self-assessment.

In the future, we expect that countries other than Japan will conduct surveys on medical students’ acquisition of competencies required to use IST, and the factors that affect their self-assessment. By comparing the results from other countries, we can determine the educational systems and cultural backgrounds that positively influence the acquisition of such competencies. Next, it is desirable to survey the actual status of acquisition of competencies required to use IST over several years. In this study, we found a correlation between the self-assessment of IST and the participants’ learning experience. Therefore, it is important to avoid bias in the responses related to factors that affect participants’ self-assessment. Finally, the competencies required to use IST should be examined objectively. Because self-assessed competencies are based on subjective responses from medical students, a third party should observe and assess medical students’ competencies required to use IST. No effective indicator can objectively measure the competencies required to use IST [6]; thus, we believe that further development is required to clarify the results.

Conclusion

We conducted a survey to identify the factors that improve Japanese medical students’ self-assessment of their acquisition of competencies required to use IST. The results showed that, among the learning objectives, medical students self-assessed their lack of mastery of regulatory and ethical issues related to IST, as well as their understanding of IST in medical care as low. The factors that enhanced medical students’ self-assessment were their experience of taking classes on IST, their experience of problem-based learning and learning IST outside formal classes, and the way they learn about IST. However, the rapidly evolving IST and the corresponding changes in regulation and curriculum have mandated that medical students adapt to the dynamic nature of IST. Therefore, we believe that the first step toward using IST in medicine is the recognition of the importance for medical students to learn about IST, adopt active learning methods as learners, and have experience in applying the skills they have acquired. We expect that the findings of this study will enhance medical education researchers’ awareness of the actual situation regarding students’ acquisition of competencies necessary to use IST and will facilitate the development of more effective interventions.