Background

The COVID-19 pandemic is rapidly transforming the landscape of the healthcare system and virtual healthcare solutions are playing a key role in this change [1]. It has also presented unique challenges in the healthcare quality improvement (QI) sphere and highlighted the need for a dynamic approach that enables QI structures and policies to adapt to the pandemic environment [2]. QI principles offer useful strategies for implementing and sustaining meaningful change [3] and staff trained in QI principles can play a critical role in responding to these emerging challenges by accelerating the pace of learning [4]. QI is increasingly being recognised as an important skill for healthcare professionals [5] and is an important component of medical education and training [6]. The immediate focus of the COVID-19 response of the healthcare sector has been on ensuring continuity of care for patients and communities. However, the pandemic has also had a profound impact on medical education delivery and how healthcare professionals will be educated in future [7]. As researchers and practitioners rush to explore ways to support healthcare professionals during the pandemic, it is also important to rethink the way QI training and education is delivered to healthcare professionals.

Rather than viewing COVID-19 as a disruption to healthcare QI education and training, it can be considered as an opportunity to improve distance learning techniques and benefit from digital hyper-connectivity to enhance education delivery that can extend into the post-pandemic environment [8]. Application of technology-enhanced learning is often cited as a pedagogical advancement for a curricular transformation of medical education [9]. However, the pandemic has made it an inescapable necessity for the healthcare system to adapt to virtual ways of working.

Online learning platforms have the potential to bring healthcare professionals together to share knowledge and collaborate in QI teaching, learning and education [10]. Beyond the pandemic situation, well designed, self-directed e-learning programmes which are responsive to the dynamic healthcare sector, may lead to better knowledge retention as compared to traditional didactic lectures [11]. There is much to be understood about the usefulness of distance learning modalities in effectively delivering QI training. The purpose of this rapid narrative review is to collate studies to identify what works and what does not in delivering QI training and education using distance learning modalities. A rapid evidence assessment summarises research findings in a systematic manner, within time and resource constraints and is suited to the current situation [12].

The review aims to answer the following questions:

  • What distance learning modalities are being used to train healthcare staff and students in QI methods?

  • What is the efficacy of distance learning QI programmes?

  • What were the advantages and limitations in delivering QI programmes using distance learning modalities?

By answering these questions, we aim to synthesise guidelines and recommendations for those who are dealing with the challenge of adapting and delivering QI training to healthcare professionals through distance learning modes.

Methods

Three databases (PubMed, Web of Science and Scopus) were searched to identify studies published between 2015 and 2020. The inclusion criteria were QI training programmes or courses targeting healthcare professionals and students with at least one component of the programme being delivered online. Only studies with primary data were included. Conference proceedings, editorials, protocols, and book sections were excluded. Studies that did not explicitly teach QI principles in the programme were excluded. Papers with no full text available or no English translation available were also excluded. Reference lists of included papers and grey literature search of Google Scholar was also conducted to identify further papers. The search and screening processes are documented in Fig. 1 (Prisma Diagram) and search strategy is presented in Additional file 1: Search strategy. The critical appraisal of the studies was conducted using the Quality Improvement Minimum Quality Criteria Set (QI-MQCS)—Version 1.0 tool which possesses acceptable psychometric properties for critical appraisal to support systematic reviews containing diverse quality improvement intervention (QII) evaluations [13].

Fig. 1
figure 1

PRISMA diagram. Study screening and selection process

Full size image

Results

The summary of the 19 studies included in this evidence synthesis is presented in Table 1.

Table 1 Summary of studies
Full size table

Study characteristics

Although, the overarching aim of the included studies was to improve QI skills of healthcare professionals and students, the studies differed in design, evaluation, and analytical methods used. Most studies had a mixed methods design and 17 of the 19 studies were based in the United States. Design of the interventions was also variable; most studies used a blended learning method combining online and in-person modes while only six studies [17, 22, 26, 27, 30, 32] were entirely delivered online. Some blended learning modules conducted classroom-based sessions followed by support through online modules and QI project completion, while others used a flipped curriculum approach where participants completed online modules prior to the in-person sessions such as seminars, workshops, lectures and QI project completion.

Quality assessment

All included studies were deemed to be of good quality even though some studies did not report on all areas evaluated by the QI-MQCS tool. All studies discussed the rationale behind the intervention, organisational motivation, description of the intervention and implementation approach. Some studies included limited information about describing sustainability or the potential for sustainability of the interventions and explicitly naming the study design. None of the studies were excluded based on quality assessment and a detailed quality assessment is attached in Additional file 2: Quality assessement of included studies.

Distance learning modes

The online delivery modes used by studies included online modules [14,15,16, 20,21,22,23,24, 28,29,30,31], access to web-based curricula [17], virtual learning environments [18], webinars [19, 30, 32], calls [19, 32], web-based QI portals [22], smartphone apps [27], emails [26, 27], access to package of tools [30], virtual whiteboard [21] and video lessons [25]. Instead of developing their own distance learning content, most studies relied on the completion of the Institute of Healthcare Improvement’s (IHI) online modules [15, 16, 20, 23, 31, 32], many of which are free to use. The rationale behind using IHI’s modules is that it provides a standardised methodology which does not require prior faculty proficiency or entail an increase in educational time commitment [20]. The IHI methodology is designed to help organisations in identifying and closing gaps via a standard improvement methodology [32]. Another advantage is that an institutional subscription to the IHI programme provides access to comprehensive QI training and allows tracking the progress of participants [24]. Some studies adapted IHI modules [29, 32] to their local context while other used self-developed content [17,18,19, 22, 27, 28, 30]. The major online modalities used are summarised in Table 2.

Table 2 Description of online modes
Full size table

Only a few studies discussed the tools/software used to deliver the online QI training components. One study used Adobe Connect and Blackboard for delivering a web-based QI curriculum [17] while for a digital storytelling pedagogy, researchers recommended participants to use freely available software such as Windows Moviemaker or Apple iMovie [18]. Another QI collaborative used iLab which is a secure, online workspace [19] while a study that developed a web-based QI portal used WordPress CMS platform, social media account integration and a network management site called Hootsuite [22]. A microlearning app called Qstream was used in another training programme [27].

Efficacy of QI training

Studies used various evaluation methods; some focused on programme level factors such as course completion rates [16, 27], learner reaction to training [14, 17, 25, 32], engagement level of participants [14, 26, 28], participant perceptions of the online module content and quality [20], preferred training delivery mode [16, 22, 27] and document and report submission by participating teams [32]. Studies evaluated the impact of training on participant comfort [31] and confidence [20, 28] in using QI. Many studies also assessed improvement in participants’ QI knowledge, skills, attitudes and behaviours as a result of the training [14,15,16,17, 21, 23,24,25, 27, 31]. Relatively few studies measured improvement in patient safety skills and knowledge of participants post-programme [23, 24]. How participants implemented QI skills and knowledge also constituted a part of outcome evaluation in various studies. This included development of action and improvement plans [18], number of QI projects developed [20] and completed [28] and results attained from these projects [19]. Improved results for the patients were also used as a proxy for outcome evaluation [14, 15, 17,18,19, 27, 30, 32].

Most of the included studies reported achieving desired outcomes such as improved QI knowledge and skills [14, 21, 24, 25, 27, 28], positive reaction from participants towards the training [17, 19, 20, 32], implementation of QI knowledge by participants [18, 29] and confidence to use the learned skills in future [22, 28]. One study reported no improvement in the measure being tracked [32]. In the studies with control groups, the intervention participants demonstrated improved QI knowledge [16], improved comfort with QI methods [31] and greater participation than the control groups [26]. One study did not demonstrate any significant difference in post-intervention safety indicators between intervention and comparison group [30].

This shows that majority of the interventions were successful in demonstrating the desired results. However, there was scant information around the role played by mode of delivery in the attainment of these outcomes. One study using a control group concluded that use of online content in conjunction with in-person sessions as being more effective in improving QI knowledge than only relying on online content [23]. The flipped curriculum [16] and utilisation of web-based platforms to deliver advanced QI training [17] proved to be effective methods for teaching QI. Since there is a shortage of comparable prior studies on web-based tools for QI education, it is challenging to compare results across similar interventions and more longitudinal studies may be required to analyse outcome trends over time [22].

Benefits of online QI education

An online QI programme can virtually connect users and provide them with an environment that balances training and practice [22]. Online delivery of QI training programmes requires fewer resources [19], reduces the burden on sites and instructors [17] and the organisations do not need to maintain QI faculty [17]. It is useful in delivering a centralised QI curriculum to distributed learners [17] hence increasing the reach of the programme [19, 27]. Additionally, online learning seemed to balance the educational time constraints and clinical responsibilities in educating healthcare professionals [16]. Moreover, the IHI teaching modules used in this study are widely recognised and accessible to all programmes and learners [16]. Interactive, distance learning modules which occur in real time with multi-way communication, feedback and tailored education proved to be effective [17]. Online modules which are interactive were preferred by participants over static computer-based modules [16].

Participants in online QI programmes have more control over their learning time [27], allowing them to complete much of the curriculum at a time convenient to them [31]. Programmes that are interactive and real-time in nature lead to better and personalised engagement from participants [17]. In the case of a flipped classroom where participants complete online modules prior to in-person sessions, the online component enables maximisation of in-class time [21]. However, many participants expressed that in-class sessions and in-class application were more effective than online content in enhancing their QI knowledge [16]. Similarly, viewing didactic material on videos beforehand enables participants to use classroom time to clarify concepts [25] and reduces the overall time required for the curriculum [20]. Alternatively, quiz-based online courses provide real-time feedback, engagement and healthy competition, but are more suited to reinforce concepts taught in classrooms and supplement other QI activities rather than as standalone activities [27].

Another advantage of online programmes using innovative methods such as digital story telling is that it engages the participants in learning a new skill and creates a level playing field in terms of the anxiety associated with a new experience [18]. Virtual discussion boards also have the advantage of providing a safe space where participants can freely express their opinions and ideas which they might not feel comfortable doing face-to-face [18]. Similarly, using game mechanics and team-based competition in a safe virtual environment is an effective participant engagement strategy [26].

Participants were generally positive about features of online programmes such as open discussion forums, closed groups, private messaging, and feedback submission forms [22]. Using tools such as leader boards motivates participants to engage and provides a sense of status [26]. Group size between two to eight participants worked best [21]. Virtual formats also allow for easier modifications in the curriculum length, content, and level which are important considerations in training design [17].

Open communication, stakeholder buy-in, and continuous feedback were necessary in developing a shared vision and QI site ownership [22]. Educational content developed by faculty with practical and teaching QI experience strengthens the programme [27]. The early involvement of key stakeholders and SMART (specific, measurable, attainable, relevant, and time-bound) goals proved to be critical to the success [29]. QI coaches also play an important role in distance learning programmes as well and a study concluded that having tailored coaching support for each team was a useful aspect of the programme [14]. In the case of students, providing an opportunity for experiential learning through QI project completion alongside a faculty member was also important [31].

Limitations of online methods

Evidence suggests that mobile and asynchronous educational technologies have the potential to overcome barriers related to teaching QI methods [26]. However, studies have also identified some limitations to such approaches. Participants often valued the learning application sessions conducted in person, more than the online components [16]. In the same way, although learners enjoyed asynchronous learning and online delivery, they preferred assessment questions that focused on application of concepts rather than information acquisition [27]. Online modes offer limited networking opportunities [19].

Customising didactic materials to suit programme and participant needs is a time-intensive task [28]. Developing an online site can be resource-intensive and lead to functional problems [19]. Since data and reporting systems are external and independent from the QI education sites, it is difficult to integrate these as site resources [22]. Online programmes also require facilitators who have QI knowledge [32] as well as technical support in case participants face any technical challenges [20] such as phone line chatter as reported in a study [32]. Additionally, the adoption and use of a new technology requires significant run-time [22]. Although participants are familiar with the use of devices such as mobile phones and computers, their use may be limited because of text character restrictions and email fatigue [27]. Although the IHI Open School modules are widely used and effective, one study recommended augmenting the content to suit local needs [23] and online content such as videos should be at an appropriate level and pace suitable to the participants [25].

With blended learning programmes, a major challenge was group session scheduling so that participants could attend without disruption of clinical responsibilities [28]. Some participating teams raised concerns specific to collecting and reporting the measurement data and perceived the measures to be complex and not well-matched to the teams’ goals [19]. Apart from the challenges associated with online delivery, teams also experience other challenges such as demand of other work duties and inability to meet as a team during implementation [14].

Discussion

The training modalities discussed in the included studies can be broadly categorised into e-learning programmes and blended learning programmes. A previous systematic review comparing online with face-to-face education for healthcare professionals concluded that online programmes had comparable knowledge gains and benefits to onsite or face-to-face training [33]. However, there is a gap in literature evaluating the success of QI training delivery online. The purpose of this evidence review is to focus on efficacy of QI training and education interventions being delivered online to synthesise recommendations for adapting QI training content into e-learning materials.

The review highlights that distance learning approaches to QI help in overcoming barriers to traditional QI training such as shortage of trained faculty and deficiencies in organisational structure to support QI education [24]. Additionally, many participants preferred blended approaches to traditional approaches [32]. This review highlights important lessons for future programmes including balancing virtual and non-virtual methods, improving the technology and providing resources and support specific to learners [19]. Like other QI programmes, distance learning QI education also requires substantial commitment from the organisation, collaboration among participants, faculty, and leaders for success [14].

Another recommendation for those considering online delivery of QI programmes is to build relationships with institutional QI units to identify resources and link the programme to an institutional network of QI education [28]. Instead of developing home-grown solutions, future collaboratives can purchase or customise existing applications for their technical infrastructure [19]. Programmes aiming to deliver QI curriculum online also have the option of collaborating with other programmes that already have faculty with QI expertise or use publicly available online QI courses [28].

Coaches play an important role in the success of the initiatives and should connect with teams early on to provide supplemental support through coaching calls [19]. In terms of content, programme developers should also explore utilising psychological learning effects of spaced learning and testing [26] and identify more conceptually and methodologically appropriate performance measures [19]. The four levels of the Kirkpatrick Model, which measure participant reaction to the training, their learning, behavioural change and achievement of results are often used in the evaluation of training programmes [34]. An important consideration for future programme evaluation is that reactions to the training do not necessarily correspond to actual knowledge increase therefore QI curricula should be evaluated on multiple criteria rather than just on participant reaction [17]. Institutional leadership and environment play an important role in the effectiveness of QI programmes and should be considered in programme design [29].

There is a need for curriculum and training designs to evolve to the needs of the new generation of healthcare professionals with an increasing emphasis on technological tools to overcome the generational difference between educators and learners [35]. The review also highlighted that online training programmes offer better psychological safety for learners as compared to traditional programmes in medical education. This is in congruence with literature on psychological safety in medical education which postulates that medical students are better able to concentrate on their learning in a safe learning environment where there is neither a need to constantly self-monitor themselves nor a fear of being judged or scorned [36]. Some important questions to be considered in delivering QI programmes through distance learning modes are summarised in Table 3.

Table 3 Important questions for virtual training adaptation
Full size table

Limitations

This rapid evidence review aimed to synthesise research and recommendations quickly and comprehensively to adapt traditional QI programmes for online delivery in response to the COVID-19 pandemic which has disrupted healthcare systems and QI training programmes. The review employed a systematic search strategy with robust and transparent screening processes. Owing to the time and resource constraints, only one reviewer was involved in the screening process. However, two additional reviewers critically reviewed the study synthesis to ensure quality. It is recognised that a systematic review is neither appropriate nor possible in every situation [37]. This rapid narrative reviews serves a specific and time-sensitive purpose and will be useful for policy-makers and QI programme designers seeking to make quick decisions about adapting training to meet needs of healthcare staff during the COVID-19 pandemic.

Conclusion

The findings of this review have important implications for those looking to adapt traditional QI programmes to a virtual environment. Virtual training environment holds great potential in delivering standardised training remotely, which will be of utmost importance in the post-COVID environment. However, caution should be practised, and a realistic evaluation of capabilities and needs should be conducted before making adaptation decisions. Factors such as programme design, mode of delivery, technical skills, implementation support and contextual factors are important considerations. Virtual QI and training of healthcare professionals and students is a viable, efficient, and effective alternative to traditional QI education that will play a vital role in building their competence and confidence to improve the healthcare system in post-COVID environment.