What is already known

  • Nursing care for critically ill patients includes essential care tasks that support the promotion of patients' health or the maintenance of their clinical condition.

  • These various educational tools, for further development and general use, should be well introduced and their impact on multiple applications should be evaluated.

What this paper adds

  • In the meta-analysis, improvement in knowledge (SMD = 0.91), skills (SMD = 0.52), and self-confidence (SMD = 0.96) effect size was noticed in the technology-based educational tools.

  • This study explained that technology-based training solutions such as virtual reality, simulation-based e-learning, social networks, etc., have significant potential to improve outcomes.

Introduction

Nurses, as one of the most important providers of health services, play an essential role in the persistence of health care, and promotion at different levels of health services. To maintain patient safety, nursing managers should provide appropriate training protocols to improve the knowledge of nurses. Lee and Chang [1] stated that nursing involves four different levels of professional competence, and critical care units nurses must have the highest professional level of nursing. The services and care provided in this unit require higher vigilance and quality than other units [2]. Patients in critical care units often experience multiple organ dysfunction, hemodynamic instability, complex medication regimens, as well as vulnerability to stress for both themselves and their families. Nursing care for critically ill patients includes essential care tasks that support the improvement of patients' health or the maintenance of their clinical condition [3, 4].

Most patients in this unit are usually under mechanical ventilation and are unconscious or in a coma. Thus, providing care to patients requires nurses who are equipped with up-to-date knowledge, alertness, and expertise [5]. The conditions of patients in critical care units are variable and very complex, and nurses in these wards need to be able to assess and provide care for critically ill patients, evaluating symptoms, and intervene with initial assessments as well as treatment to avoid unexpected risks [6]. In critical care, the nurse constantly encounters patients whose health status changes rapidly. These require quick decisions in a short time despite massive stress [7]. Research shows that there is a clear correlation between nurse skills and patient outcomes. The complexity of the role of nurses in these units requires a structured and continuous training program from elementary to graduate courses [8]. Nurses or nursing students learn through ongoing training to quickly identify problems or abnormalities in patients by evaluating data, influential factors, and potential health risk factors [9].

Until now, various nursing educations in university or in-service courses have been presented in a traditional way and as a lecture. However, this approach has its own problems. For example, not only does it require more human, financial, and equipment resources, but also people have to leave their work environment to participate in the course. Thus, traditional teaching methods should be changed to improve learning experiences and facilitate lifelong learning. Teaching strategies that include hands-on experience through doing and communicating as well as talking with others promote more meaningful learning. This approach will develop creativity and innovation for both students and teachers [10].

The unique capability of information technology has provided the possibility that soon, educational systems, as well as other areas, will be ultimately affected [11]. Currently, many universities around the world use information technology to develop and improve medical education [12]. In 1986, computers were predicted to become an inevitable part of the medical education system [13]. In 2000, nursing education strategies expanded from simple online reading courses to learning through various mobile devices as well as interactive learning with peers and educators [14]. Now, with the implementation of social distancing protocols to contain the spread of the COVID-19 disease, the use of technology in education and learning has peaked [15]. While the traditional lecture style focuses on face-to-face lectures, making the learner a passive participant, technology-based learning has emerged as a new learner-centered teaching method which facilitates learner participation and feedback [16]. Studies have shown that the electronic learning method has been successful in teaching various fields of nursing concepts and skills, such as drug calculations, arterial blood gas interpretation, electrocardiogram interpretation, control of vital signs, triage, report writing, the correct method of hand washing as well as many other cases [17,18,19]. It has also changed the level of knowledge, behavior, and performance of nurses and nursing students. Learning management systems worked so well that nursing courses began migrating online. Currently, many educators use simulators to teach students how to diagnose heart problems [20]. The Cardiopulmonary Resuscitation (CPR) Training System of the American Heart Association is also one of the most widely used useful educational tools. Other programs such as drug management, anatomy, physiology are also used [21]. Nursing simulation tools, augmented reality and virtual reality have also recently become popular and are used to stimulate motivation and improve learning [14].

Aim of the study

For further development and general use, various educational tools should be well introduced and their impact on various applications should be evaluated. Accordingly, the present study aims to review, synthesize, and analyze the effects of technology-based educational tools for training critical care nurses and nursing students. Meta-analysis was also used to examine this impact on nurses’ or nursing students' skills, knowledge, self-confidence, and attention. It helps summarize different scientific documents and summarize them in an objective way with minimum personal opinions.

Methods and materials

This systematic review and meta-analysis were conducted based on the Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) tool proposed by Matthew J Page et al. [22]. PRISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. PRISMA primarily focuses on the reporting of reviews evaluating the effects of interventions, though it can also be used as a basis for reporting systematic reviews with objectives other than evaluating interventions [23]. The filled PRISMA checklist is given as supplementary material (Appendix Table 1). Also, this review was conducted in line with the Cochrane Handbook for Systematic Reviews and Interventions but no protocol was registered [24]. We applied the quantitative and qualitative analysis process to summarize the screened papers and generate new notable insights.

Data sources and search strategy

Medline (through PubMed), EMBASE, Scopus, and Web of Sciences (WOS) were selected as core search databases. The mentioned databases were selected because of their coverage of qualitative and health research. We identified papers with a time limit, where articles published from 2010 to 18 Feb 2022, were examined. The search strategy used in this study involved a combination of keywords and Medical Subject Headings (Mesh) terms and Emtree related to “nursing”, “Education”, “technology”, “Computer education”, and “Intensive care”. The Emtree thesaurus is a hierarchically structured, controlled vocabulary for biomedicine and the related life sciences. It includes a whole range of terms for drugs, diseases, medical devices and essential life science concepts. Emtree is employed to index all of the Embase content. Hence, Emtree is the collection of standardized keywords in Embase. The use of standard keywords for each concept leads to the formulation of a complete search strategy. The complete list of keywords applied in the search strategy is provided in Table 1. Reference manager software (EndNote X8, Thomson Reuters) was utilized to collect references and exclude duplicates.

Table 1 Search strategy for all databases
Full size table

Selection criteria

Inclusion criteria

The inclusion criteria set, which were admitted in this systematic review (SR) and meta-analysis, are outlined in Fig. 1. Types of studies in this review included Randomized Clinical Trial (RCT) and Non-Randomized Clinical Trial (NRCT). Accordingly, the PICO model was selected for this purpose. This means a reliable and comprehensive question should comprise four parts that recognize the patient problem or Population (P), Intervention (I), Comparison (C), and Outcome(s) (O).

Fig. 1
figure 1

Inclusion criteria based on PICO in this SR and meta-analysis

Full size image

Exclusion criteria

Articles were excluded if they were:

  • (1) observational studies or non-experimental studies,

  • (2) conference papers,

  • (3) studies in which technology-based educational tools were excluded,

  • (4) studies in which the target group were not nursing students or nurses were excluded,

  • (5) non-English papers.

Study selection

In this stage, the assessment of studies was done by more than one reviewer. Two reviewers independently screened the titles and abstracts of the identified studies. Any disagreement between the reviewers was resolved through discussion with a third researcher. The full text of the citations was retrieved and examined if it was supposed potentially relevant by two reviewers too.

Data extraction

Here, the main classification of reviewed articles was determined. Three authors synthesized and analyzed the main specifications of included papers, after which they extracted the main key items of the papers. The authors assessed the extracted information and validated the main elements. If there was any discrepancy in the extracted data, a consensus was reached with thorough discussion after repeating the same extraction process. General characteristics such as authors, publication year, country, journal, education domain, population plus sample size (with a mean age of participants), study design, technology-based intervention, setting, sessions details (number, duration, and frequency), measurement time point, follow-up duration, learning outcomes, evaluation results, main message, and reported limitations were extracted from individual studies. Before extracting the data from the full texts of the articles, an interrater reliability check between the evaluators was performed. At this stage, 50% of the included articles and 20% of the excluded articles were randomly selected by two authors, and interrater reliability checks were performed. There was no disagreement between the authors. The following data were extracted from the selected studies and entered into a structured form in Excel.

Study risk of bias assessment

To assess the risk of bias in individual included papers, the Cochrane Collaboration’s risk of bias tool was applied. Studies with a high or unclear risk of bias for the blinding of assessors or incomplete outcome data categories were considered as high risk of bias [25]. This tool addressed external validity, internal validity, and interpretability. Assessments were conducted by three independent authors. Two reviewers established consensus scores and resolved disagreements.

Evidence synthesis and analyses

We imputed the mean changes (before, after) and the pooled standard deviation of the study outcome values in the meta-analysis. All analyses related to meta-analysis were performed using a model with random effects. Estimated values for pooled effect sizes in all learning outcomes were shown with Standardized Mean Differences (SMD); this method was chosen due to the different parameter scales in the selected studies. Heterogeneity was assessed with I2, τ2 tests, while publication bias was evaluated by Egger’s, Begg’s test. All analyses were conducted in Stata v16 and EndNote X9 was applied for resource management. Hence, Mean Gain (MG) and SD Pooled from pre and post-intervention were inputted into Stata v16.

Results

A total of 3410 relevant articles were resulted from the search from 2010 until Feb 2022. After the removal of duplicates, 2318 articles remained. The process of searching the four main databases and identifying studies based on the PRISMA diagram is displayed in Fig. 2. Title and abstract screening led to the omission of 1991 articles. In the first examination, 323 papers seemed relevant, and their full text was investigated. After examining the full text of the identified papers and applying the inclusion plus exclusion criteria, ten studies were included in this systematic review and nine of them were synthesized in meta-analysis.

Fig. 2
figure 2

The PRISMA diagram for the search of records and study selection

Full size image

Risk of bias (RoB) in the included studies

All articles have the lowest bias value regarding detection and attrition aspects. Four studies were assessed to have an unclear RoB for any crucial concerns about bias not covered in the other domains in the tool. Nine papers were considered to have a low RoB in reporting ways, and one of them had a high risk. Totally, based on results, nine papers were assessed as good quality studies, though not without risk of bias. Qualitative assessment for all the individual papers is shown in Fig. 3. Remarkably, selection bias and performance bias refer to “biased allocation to interventions due to inadequate generation of a randomized” and “bias due to knowledge of the allocated interventions by participants and personnel during the study” were assessed for some studies with high risk. For performance bias, two aspects have been considered in the studies: blinding participants and blinding personnel or researchers. In studies with high bias risk, neither participants nor personnel were blinded, whereas, in studies with low risk, participants were not blinded, but staff and assessors were blinded.

Fig. 3
figure 3

Cochrane Collaboration’s tool for assessing the risk of bias in papers

Full size image

General characteristics of the included studies

Table 2 presents the general characteristics of the included studies. The oldest and newest articles were published in 2016 and 2022, respectively. For most investigations, 60% (6/10) of papers were conducted in South Korea and Iran; the distribution of papers based on countries is presented in Fig. 4. In the screened studies, 737 nurses and nursing students participated; 367 participants were in the experimental groups (EG), and 366 were in the control groups (CG). Eight studies reported the mean age of participants; the range of mean age was 21.18 years old to 34 years old. The sample size ranged from 44 to 122 participants (IQR1:47, median: 68, IQR3: 100). In seven studies, the participants were ward nurses, and in three studies, nursing students were trained. However, the number of intervention sessions was not homogeneous and, in some experimental ones, was not mentioned clearly.

Table 2 General characteristics of the included studies (n = 10)
Full size table
Fig. 4
figure 4

The distribution of articles based on countries

Full size image

Note that the study design was mostly experimental and of the Randomized Controlled Trial (RCT) type. Four studies (40%) were designed in the form of RCT, one study was in the form of a clustered randomized controlled trial, and finally, five studies were set in the form of quasi-experimental design (before and after with two experimental and control groups).

In seven studies, included participants were samples from Intensive Care Units (ICU) (n = 7, 70%); in one study, the participants' setting was Neonatal Intensive Care Unit (NICU) (n = 1, 10%), and again in two investigations, participants were included from Cardiac Intensive Care Unit (CICU) (n = 2, 20%).

In the included studies and experimental groups, different technology and innovative educational contexts were used; video-oriented learning and virtual reality-based environments were the most popular technologies applied to train the nurses. In three studies (n = 3, 30%), the educational environment was interactive videos, and in two studies (n = 2, 20%), virtual reality was employed with its appendices. It has also been used in two studies of mobile or Windows-based applications. The control group received routine training in six studies such as booklet, article, and guidelines. Note that there was a follow-up period in only four studies.

Meta-analysis

Since one of the articles did not provide a pre-intervention evaluation, it was removed from the meta-analysis, and its results were presented only in our systematic review. The results of the meta-analysis are reported shown in Table 3. According to the results for skill parameter Standardized Mean Difference (SMD) was 0.52 (CI 95%, -0.15–1.20), suggesting that the standardized mean skill was 0.52 higher in the experimental education group than in the comparison group (Fig. 5). However, this difference was not significant (p = 0.13). Pooled effect size was higher for knowledge (SDM = 0.91, CI 95%, -0.32- 2.15 (p = 0.15)) (Fig. 6) and self-confidence (SDM = 0.96, CI 95%, -0.12- 2.06 (p = 0.08)) in the experimental group compared to the comparison group. In addition, in the analysis of subgroups, the pooled effect size of staff compared to students (0.50 vs. 1.49) and for self-confidence was estimated as (2.11 vs. 0.40). Eventually, less value was estimated for attention effect in the comparing groups.

Table 3 Pooled standard mean difference, heterogeneity and publication bias according nursing learning outcomes
Full size table
Fig. 5
figure 5

Pooled standard mean difference according to skill between experimental and control groups

Full size image
Fig. 6
figure 6

Pooled standard mean difference according to knowledge between experimental and control groups

Full size image

Risk of bias and heterogeneity

Additional analyses showed that no publication bias was observed in any of the analyses (p < 0.05). Regarding the true heterogeneity (τ2) among studies, in most variables it was very low (less than one).

Limitation of studies

All mined papers reported some limitations and faced important challenges. The limitations mentioned in the studies are provided in Table 4.

Table 4 Reported limitations of included studies
Full size table

Discussion

Principal findings

The main intention of our systematic review and meta-analysis was to assess as well as screen the critical results related to the effectiveness of applying technology-based educational tools for the nurses and nursing students in ICUs. Accordingly, this study was conducted to examine the prior studies on the effects of technology-based learning for nursing to provide the primary data for evidence-based nursing research by assessing the objective use of the characteristics as well as the effects of learning tools through a meta-analysis. To our knowledge, this investigation chiefly focuses on determining experimental interventions using educational tools to train special functions/skills, knowledge, self-confidence, attention, etc.

Overall, the ten studies included were evaluated as good quality studies but had some risk of bias. The results concerning the risk of bias, especially performance bias (i.e., the blinding of participants and personnel), can partly be explained by the chosen evaluation policy. In studies with high bias risk, neither participants nor personnel was blinded, whereas, in studies with low risk, participants were not blinded, but staff and assessors were blinded. Also, we assessed the quality of methodological quality of the included studies.

Ten studies included in this systematic review were conducted after 2016. Further, 60% of them were performed in Korea and Iran. This shows that technology-based learning has been around since 2010, when innovative technologies began to be widely distributed. In addition, it is an inevitable result that many studies have been conducted in Korea, where technology-based learning infrastructures have already been established [36]. This study provides evidence that technology-based learning has beneficial effects on various learning outcomes, including knowledge acquisition, trust, and satisfaction with learning compared to traditional learning methods [13, 37].

Three studies had focused on nursing students including senior nursing students and undergraduate final year. Most studies had been done on nurses since nurses in ICUs need basic training, and their job sensitivity is very high. The findings of this study can provide guidance for nursing instructors, indicating that the use of technology-based educational tools is an effective solution to transfer students from the learning environment to clinical practice. In this systematic review, skills, knowledge, performance trust, learning attitude, and learning satisfaction were essential and practical criteria for technology-based learning intervention studies.

The present meta-analysis has shown that in most cases, the effect of the intervention on the skills, knowledge, and self-confidence of the participants was powerful and significant. However, the magnitude and direction of the effect of technology-based learning on learning outcomes seem highly situational [3, 36]. As a result, the impact of modern learning tools is likely to be influenced by many, possibly confounding, factors that differ across different learning methods, topics, and outcomes. In some studies, the tests were executed between the post-intervention scores of the intervention and control groups, rather than between the mean differences from baseline evaluation.

In this study, the results of technology-based educational tools consisted of six studies that had measured skills, followed by five studies evaluating knowledge; three studies had assessed self-confidence in performance and learning satisfaction. Eventually, two investigations had checked the learning attention. Also, in this study, case–control interventional studies were included, while single-group before-after studies were excluded from the review to minimize the heterogeneity of the studies. Remarkably, five studies were performed in a quasi-experimental design. Indeed, it seems that considering the characteristics of nursing research performed on nurses and nursing students, there are certain limitations for the full implementation of such cases since quasi-experimental studies are equivalent to randomized trials.

Scientifically speaking, researchers should try to reduce the bias of quasi-experimental research to determine the impact of technology-based learning tools accurately. Most previous studies did not provide information on the course and timing of the intervention. Thus, more efforts should be made to correct this issue in future research [27, 32]. In addition, blinding research participants was impractical since the use of technology-based tools by nurses or students could not be hidden, so blinding nurses and students are tough. The meta-analysis results revealed that there was an overall positive effect size for the target variables. Improved skills, knowledge, performance confidence (confidence), as well as attitude in studies were reported, and differences were significant. Nevertheless, there was no positive effect on nurses’/students' attention; this may be due to the few studies that had evaluated the attention variable.

The results of our study led to similar results to the findings of recent meta-analyses related to learning based on new technologies in nursing education. Based on the systematic literature review by Voutilainen A. et al. [38], the applied e-learning method was more effective than the conventional teaching methods; the new techniques had the potential to improve the learning outcome significantly. Another meta-analysis revealed that smartphone-based mobile learning could effectively improve nursing students' attitudes and that the use of these smartphones had also a significant positive effect on improving knowledge and skills [36]. Another study proved that simulation-based learning had moderate to substantial effects on enhancing knowledge acquisition, self-confidence, and learning satisfaction among undergraduate nursing students [39].

Nevertheless, the difference between our study and recent meta-analyses was that we looked at educating nurses and students who were gaining knowledge and skills in the ICU, and their timely intervention was critical [3, 40]. Technical, assessment, relational, and teamwork competencies are all required for optimal performance. ICU nurses monitor patients, administer medications, assist patients with basic needs, chart care, and respond to emergencies. Unlike some other nurses, their patients are often intubated and ventilated [41]. They must know the ins and outs of more equipment than nurses who practice in a lower-stakes environment. Also, they are highly trained and skilled safety–critical professionals working as part of a multidisciplinary team [9, 31].

Based on the results of this systematic study, the included studies had significant limitations and challenges that cannot be ignored. The most critical challenges included limited sample size, limited generalization of the findings, and a short period of learning (limited timeframe) to evaluate the effectiveness of educational tools.

Strengths and limitations

This review has combined the results of risk of bias assessment (Cochrane tool) and meta-analysis. There have been strengths and weaknesses in this study. The strengths of the study are as follows: (1) applying an extensive search strategy to identify a large number of studies (3410 investigations), (2) conducting searches to retrieve studies in four important databases, including WOS, Scopus, Medline (through PubMed), and Embase, (3) reviewing and evaluating studies to extract data by five authors independently, (4) using comprehensive tools to evaluate the quality of included studies and to assess the risk of bias.

We have also encountered some limitations in this study. The difficulty of comparing studies is due to the heterogeneity of the results, so we interpreted outcomes with caution, and no generalization of the effects on nursing education seems appropriate. Also, book chapters, letters, non-English articles, and conference proceedings were excluded.

Conclusion

This systematic review and meta-analysis highlighted improving nurses' and nursing students' knowledge, skills, self-confidence, and motivation to use educational tools based on innovative technologies. This study explained that technology-based training solutions such as virtual reality, simulation-based e-learning, social networks, etc., have significant potential to improve outcomes such as the specific knowledge and skills of nurses or nursing students in ICUs. Also, these tools will lead to the satisfaction of the target group and enhance patients' quality of life due to proper training of nurses. The effects of the interventions are strongly influenced by the time of the intervention. However, it can be suggested that if the learning method based on the new technologies tested is more effective than conventional teaching methods, they are likely to improve the learning outcome significantly.