Abstract
It is widely acknowledged that graduates need to develop skills and competences beyond the theoretical knowledge nurtured within higher education curricula. In the last twenty years there has been an increased interest in supporting learners with work-based learning (WBL) (e.g. apprenticeships, practice-based lab sessions, project-based learning). The experience of COVID-19-related lockdowns has ignited a push to support and provide these opportunities for skills development online. In this systematic literature review, the authors explored (1) which innovative technologies and online WBL typologies are implemented in online WBL in higher education; and (2) to what extent there is evidence that online WBL is effective. From an initial pool of 269 studies identified from two datasets, the authors selected thirteen studies which implemented and evaluated online WBL. In total, 1,015 respondents were included in these studies (M = 84.58, SD = 118.28, range: 7–390). The authors’ findings indicate that most studies used descriptive, qualitative approaches to explore the lived experiences of participants, mostly from Australia. Substantial differences were found in the designs of online WBL practices and technologies, although there was limited robust evidence of effectiveness due to a lack of evidence-based evaluation approaches. The authors encourage WBL researchers to be more precise in their design parameters of online WBL, and to consider (quasi)experimental designs to measure the impact of their approaches.
Résumé
Apprentissage par le travail en ligne : une revue systématique de la littérature sur ce thème – Il est largement admis que les diplômés ont besoin d’acquérir des savoir-faire et compétences dépassant le cadre des connaissances théoriques acquises par le biais des programmes d’enseignement supérieur. Ces vingt dernières années le soutien aux apprenants qui suivent des apprentissages par le travail (APT) (ex. : apprentissages, séances de travaux pratiques en laboratoire, apprentissage basé sur des projets) a connu un intérêt croissant. L’expérience des confinements liés à la COVID-19 a poussé à soutenir et à offrir des possibilités d’acquérir des compétences en ligne. Pour cette revue systématique de la littérature, les auteurs ont examiné (1) les technologies innovantes et typologies d’apprentissage par le travail en ligne mises en œuvre dans l’enseignement supérieur et (2) la mesure dans laquelle on peut prouver que l’APT en ligne est efficace. Partant d’un ensemble de 269 études déterminées à partir de deux ensembles de données, ils ont sélectionné 13 études ayant mis en œuvre et évalué des APT en ligne. Au total, ces études portaient sur 1 015 répondants (valeur moyenne = 84,58, écart-type = 118,28, fourchette : 7–390). Les conclusions des auteurs indiquent que la plupart des études emploient des approches descriptives et qualitatives pour explorer les expériences vécues par les participants, majoritairement d’Australie. Ils ont trouvé des différences considérables en ce qui concerne la conception des pratiques et technologies d’APT en ligne, bien que peu de preuves solides soient venues étayer leur efficacité du fait de l’absence de méthodes d’évaluation fondées sur des preuves. Les auteurs encouragent les chercheurs dans le domaine de l’apprentissage par le travail à être plus précis dans leurs paramètres de conception de l’APT en ligne et d’envisager des modèles (quasi) expérimentaux pour mesurer l’impact de leurs approches.
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Avoid common mistakes on your manuscript.
Introduction
It is widely acknowledged that graduates need to develop skills and competences beyond the theoretical knowledge nurtured within higher education curricula (Jackson and Collings 2018; Perusso and Wagenaar 2021; van de Werfhorst 2014). University–industry collaboration in the context of knowledge transfer is commonly known as work-based learning (WBL). According to the European Education and Training Monitor 2021 (EC 2022), WBL is recognised as a means to help young people and adults transition more smoothly from education to the labour market.
Over the past 20 years, a wide body of WBL literature has recognised that it is important to provide students with opportunities for academic and workplace learning, Definitions of WBL differ primarily according to the location in which they take place – in industry or in academic environments (Lester and Costley 2010; Major 2016; Smith 2012; Wood et al. 2020). In this study we follow the definition by David Major, whereby WBL is defined as
fully accredited, negotiated, modules or programmes of planned learning through work delivered by higher education providers (Major 2016, p. 27).Footnote 1
When situated in industry, WBL usually takes the form of internships and apprenticeships, while in an academic environment it traditionally includes on-campus projects, simulations of working and/or different forms of entrepreneurship courses in collaboration with industry (Gerken et al. 2012; Jackson et al. 2017; Lester and Costley 2010; Perusso and Wagenaar 2021; Smith 2012). As argued by Judie Kay et al. (2019), these activities could also take place in the form of micro-placements,Footnote 2 online projects or placements, hackathons, competitions and events, incubators/start-ups, and consulting.
With the emergence of technology, and in particular since the experience of COVID-19-related lockdowns (Seetal et al. 2021), several different forms of blended and online WBL have been developed (Schuster and Glavas 2017; Wood et al. 2020). For example, Kathy Jordan et al. (2016) explored the lived experiences of pre-service teachers who followed a practice-based online WBL approach using Google sites while simultaneously obtaining work experience at a local school. Kate Robinson et al. (2020) implemented a work-based simulation with a virtual patient capable of expressing various verbal and non-verbal responses to encourage clinical reasoning and communication skills.
In a recent meta-analysis, Yvonne Wood et al. (2020) categorise WBL according to three main approaches: (1) conventional WBL taking place in industry; (2) simulated WBL in an academic environment; and (3) remote WBL focused on students completing authentic tasks for an organisation through a remote connection. A different categorisation of technological approaches is defined by Lisa Schuster and Charmaine Glavas (2017), who distinguish between the function of technology (supporting vs delivering WBL experience) and the degree of technological involvement (low vs high). In their review, most of the 22 identified studies found that technology was being used in a supportive capacity (ibid.). However, with the increased sophistication of educational technology and WBL approaches, particularly since the experience of the COVID-19 pandemic, there is an urgent need to update our state-of-the-art understanding of online WBL.
In this systematic literature review (SLR), we aim to explore how educators and institutions have implemented online WBL in the last five years. While the term “online” has many different conceptualisations, in this article we focus primarily on studies where participants learned and worked online, remotely, at a distance and/or in a virtual format. We acknowledge that the literature features various conceptualisations and definitions of WBL and related terms (e.g. work-integrated learning, workplace learning, work-based experience), and that these terms are often used interchangeably. The common denominator of these concepts is the aim to give students structured and recognised opportunities to benefit from university–industry cooperation. Using the conceptual framework of Schuster and Glavas (2017) and Education 4.0 (Fisk 2017; Hussin 2018), we will explore which educational technologies and online WBL typologies have been implemented, and whether online WBL is effective and impactful.
Literature review of online WBL
To the best of our knowledge, there is no specific review on online WBL, but there are two reviews on e/online work-integrated learning (WIL): Schuster and Glavas (2017) and Wood et al. (2020). In the more recent meta-analysis of WIL and remote, virtual and simulated WIL, Wood et al. (2020) reviewed work published in the International Journal of Work-Integrated Learning as well as three conference proceedings on WIL from Australia and New Zealand in the period 1999–2020. In total, they identified 56 articles, of which nine discussed examples of remote WBL and 21 of simulated WBL. Wood et al. (ibid.) found medium to high authenticity in these studies evaluating and/or implementing remote and simulated WBL. While this review provides important insights into WBL practices in Australasian contexts, it offers limited reflection on the design principles of educational technology.
In an earlier SLR on eWIL, Schuster and Glavas (2017) explored six databases in the period January 2000 to January 2016 and identified 22 articles. They clustered these into four typologies:
-
technology-supported learning (where technology is used to support information and administrative processes);
-
technology-facilitated learning (where technology prepares, supports and assesses students before, during and after WBL);
-
technology-blended learning (where a combination of online and offline activities allows agents to work collaboratively); and
-
technology-based learning (where immersive technology is employed and all interactions are technology-mediated).
The latter two typologies were more focused on actual deliverance of WBL experience.
Fifteen out of the 22 studies identified by Schuster and Glavas (ibid.) mainly reflected technology being used backstage or in support of online WBL. Three studies were identified as depicting technology-blended learning (Fong and Sims 2010; Holmes 2011; Peach et al. 2013), while another three (four reported in the paper) were identified as demonstrating technology-based learning (Godat 2007; Mundkur and Ellickson 2012; Stoker 2015). In other words, up to 2016, most of the studies identified by Schuster and Glavas (2017) primarily reflected the use of technology in a supportive capacity. The few studies which did show technology being used more intensively were mostly small-scale, lacking strong experimental designs to test the impact of innovations. One limitation of both Wood et al. (2020) and Schuster and Glavas (2017) is the restricted search string, which focused only on WIL and not on the broader concept of WBL.
Education 4.0
As we expected most blended/online WBL studies since 2016 to reflect intensive use of technology to deliver online WBL, we anticipated that the typology of Schuster and Glavas (2017) might not be fine-grained enough to distinguish between the different innovations used in online WBL. Our expectation was that, if more studies since 2016 were to be identified as evaluating and/or implementing technology-based learning, it would be both conceptually and empirically important to be able to distinguish how educators designed these technology-based approaches. We therefore decided to use the concept of Education 4.0 as another approach to categorise innovative educational practice within online WBL. Definitions of Education 4.0 vary, but usually focus on innovation, novelty, use of technology, and connections with employment and industry (Hussin 2018; Rienties et al. 2021). The number 4.0 makes a connection with the view that there have been four industrial revolutions to date, with the current “fourth industrial revolution” (Industry 4.0) being increasingly automated, making use of modern smart technologies and the internet of things.Footnote 3 With reference to Peter Fisk (2017), Anealka Aziz Hussin (2018) lists nine characteristics associated with Education 4.0.
Nine characteristics of Education 4.0
-
(1)
Learning can take place anytime, anywhere.
-
(2)
Learning will be personalised to individual students.
-
(3)
Students have a choice in determining how they want to learn.
-
(4)
Students will be exposed to more project-based learning.
-
(5)
Students will be exposed to more hands-on learning through field experience (e.g. internships, mentoring projects, collaborative projects).
-
(6)
Students will be exposed to data interpretation in which they are required to apply their theoretical knowledge to numbers and use their reasoning skills to make inferences based on logic and trends from given sets of data.
-
(7)
Students will be assessed differently and the conventional platforms to assess students may become irrelevant or insufficient.
-
(8)
Students’ opinion will be considered in designing and updating the curriculum.
-
(9)
Students will become more independent in their own learning. Adapted from Hussin (2018)
Research questions (RQ)
We conducted this SLR as part of the Relevant Assessment and Pedagogies for Inclusive Digital Education (RAPIDE) project.Footnote 4 Building on the two recent review studies on online WBL (Schuster and Glavas 2017; Wood et al. 2020), our own updated SLR, presented here, aimed to explore how educators and institutions actually implemented online WBL from 2016 until 2021, using the conceptual frameworks of Schuster and Glavas (2017) and Education 4.0. Furthermore, we sought to investigate whether or not these approaches were successful in achieving their aims. Before starting out, we defined the following research questions:
RQ1: Which educational technologies and online WBL typologies are implemented in online WBL in relation to Education 4.0?
RQ2: To what extent is there evidence that online WBL is effective and impactful?
Methods
We followed recommendations from SLR approaches that use a multi-phase process (Divjak et al. 2022; Moher et al. 2010; Rienties et al. 2021).
The search
Phase 0
We searched two research databases, Science Direct and Web of Science, twice: on 14 and 21 May 2021. We chose these two sources because of their ranking as academic research databases, and because they feature good coverage of studies relevant for an international review.Footnote 5 We restricted our search to papers published in English during the five-year period 2016–2021, including articles available in pre-print until 21 May 2021, thereby increasing the chance that a particular study covered online WBL. We used the following search string: “Online” OR “Remote” OR “Distance” OR “virtual” AND “work-based learning” OR “work-based learning” OR “WBL” OR “work-integrated learning” AND “higher education”. Since there is no universal clarity on the concept of online learning, we specifically included different variants of online learning (remote, distance, virtual).
Furthermore, while Schuster and Glavas (2017) and Wood et al. (2020) only included “work-integrated learning” in their search, we extended our own search by including WBL in its various forms. These search terms identified 269 unique publications across the two databases. We excluded publications identified using the search criteria if any of the following exclusion criteria applied:
-
(1)
the focus was on primary and/or secondary education;
-
(2)
the focus was on face-to-face WBL without any online/blended component;
-
(3)
the focus was on professional learning outside higher education (e.g. online WBL in an industry for its employees); and/or
-
(4)
the work did not report on actual implementation of online/blended WBL (i.e. the item identified turned out to be a theoretical/conceptual/review paper).
The process of coding online WBL studies
The first author of this article (BR) manually screened the titles, abstracts and keywords to check whether a WBL study qualified for inclusion or not. Subsequently, we excluded 228 studies, primarily because these were not focused on blended/online WBL. Often these studies mentioned either the possibility of moving some WBL elements online, or used an online survey to gather information about a “physical” WBL programme rather than focusing on online WBL, so these were excluded.
Phase 1
The remaining 41 studies were read in depth by six members of the RAPIDE project team and categorised based upon four inclusion criteria in Phase 1:
-
(1)
Is this study about WBL at least referring to blended or fully online WBL?
-
(2)
How does it reflect the use of technology or pedagogy in an innovative way?
-
(3)
Is the innovation evaluated, and if so, how?
-
(4)
Should we proceed to review this article in Phase 2?
By including experts from a range of disciplines (e.g. computing, education and medicine) from three European countries (Croatia, the Netherlands and the UK), we created an inclusive multi-disciplinary team of coders to analyse the literature and the innovative nature of the technology/pedagogy used in an online WBL context. The coders reviewed on average 6.83 studies (range: 6–11); and led to our selection of 16 studies for inclusion in Phase 2.
Phase 2
In Phase 2, six members of the RAPIDE project team participated in a follow-up one-hour online training and discussion of the online coding scheme using 27 variables. Coders were randomly allocated a new set of studies to code in comparison to their initial coding in Phase 1, thereby ensuring that at least two coders checked and independently coded each online WBL featured in the 16 selected items of literature. For RQ1, we applied an open coding approach to capture which (innovative) technologies were used in online WBL. Subsequently, we adopted the online WBL framework proposed by Schuster and Glavas (2017), which, as mentioned above, distinguishes between the function of technology (supporting vs delivering WBL experience) and the degree of technological involvement (low vs high). In line with an earlier study (Rienties et al. 2021), we also coded the respective design used in each study by the nine key Education 4.0 characteristics (Fisk 2017; Hussin 2018). For RQ2, the coders indicated how the implementation of effectiveness and impact was measured.
All 16 studies were double-coded. Afterwards, the first coders from Phase 1 checked the codes from the second coders in Phase 2, discussed any differences, and agreed on the final coding. If a study did not indicate an implementation of online WBL with some form of data collection, we removed it from further analysis. We thereby ended up with a total of 13 studies. Figure 1 details the three phases described.
Results
Educational technologies, online WBL typologies and Education 4.0
In terms of RQ1, we selected a total of 13 studies for inclusion in this SLR on online WBL. Seven studies focused on undergraduate students (e.g. Bayerlein 2020), followed by a mix of undergraduate and postgraduate students (Glavas and Schuster 2020; Mikroyannidis et al. 2020), postgraduate students (Schech et al. 2017; Stewart et al. 2016), and unknown (Jeske and Axtell 2016a, 2016b). Ten studies were based in Australia and two in the UK, while three studies included data from multiple countries (Jeske and Axtell 2016a, 2016b; Schech et al. 2017). Given the large geographical distances in Australia and the various government initiatives in that country to encourage WBL and research interests in WIL, it is perhaps not surprising that most studies were found in this context.
As indicated in Table 1, several of the studies referred to the development and implementation of online WBL within existing educational courses. For example, in a Bachelor of Education programme in Australia, Jordan et al. (2016) explored the lived experiences of 210 pre-service teachers who followed a practice-based online WBL approach using Google sites while simultaneously obtaining work experience at a local school. Using a distributed open collaborative course design, an interactive and collaborative approach between teachers and local schools was developed in order to create an online, open and distributed online WBL model using a range of open technologies. In an Australian online WBL course on advanced mental health skills, Victoria Stewart et al. (2016) developed an intervention mapping framework to guide the development of a work-based curriculum for ten postgraduate learners. According to Glavas and Schuster (2020), online WBL practices are often hosted by universities on their technological platforms, such as their virtual learning environment (VLE).
At the same time, several studies reflected new technologies or approaches to deliver online WBL beyond the functionality of their institutional VLE. For example, in a cross-cultural simulation, 37 Swedish and Australian International Relations students worked on an authentic complex study (Timor-Leste) for six weeks (Schech et al. 2017). The course consisted of four components: an online learning website, a live-streamed symposium, a six-hour simulation, and a debriefing.
Other studies specifically used the concept of simulated internships or experimental lab studies. For example, in a “compulsory non-placement work-integrated learning (WIL) activity” for seven Australian undergraduate accounting students, a simulated internship was developed whereby students assumed the role of an intern and were required to resolve simulated real-world challenges through a range of activities in a VLE (Bayerlein 2020). Alexander Mikroyannidis et al. (2020) provided learners with several network simulation tools in a virtual lab environment originally developed in the UK (PT Anywhere) where learners could experiment with a variety of virtual network devices and topologies by extending Packet Tracer (PT). Similarly, in a simulated learning environment (SLE), Robinson et al. (2020) used a virtual patient capable of expressing 77 verbal and non-verbal responses with 82 Australian undergraduate Speech Pathology students to encourage clinical reasoning and communication skills.
In a co-design approach using semi-structured interviews with 16 students who experienced online WBL at one Australian business school, Glavas and Schuster (2020) explored students’ lived experiences in order to distil effective design of online WBL. They identified four design principles:
-
(1)
provide an authentic online WBL experience (learning activities, assessment and technology);
-
(2)
carefully select and integrate technological platforms employed to support or deliver online WBL;
-
(3)
develop effective administrative processes to support online WBL; and
-
(4)
promote co-presence and relationship building.
Online WBL typology of Schuster and Glavas (2017)
Following the typology of Schuster and Glavas (2017) as indicated in Table 1, seven studies used technology primarily to deliver online WBL experience, while in four others technology played a mainly supporting role. Furthermore, nine studies used technology in an intensive manner, while four used it at a lower intensity. For example, Kelly Miller et al. (2020) illustrate how 34 Environmental Science students were able to show evidence of excellence in a particular graduate skill (e.g. teamwork) in order to earn a predefined digital award (a non-credit-bearing “Deakin Hallmark”) as determined by a collaboration between industry and Deakin University in Melbourne, Australia. Their WBL experience could be obtained offline, but their recording of digital credentials was facilitated online.
We identified three studies (Jeske and Axtell 2016a, 2016b; Sheridan et al. 2019) which were primarily focused on technology-facilitated approaches, whereby the WBL experience was supported by a high usage of technology. For example, Lynnaire Sheridan et al. (2019) illustrated how they designed their WBL course for business students with Work-Integrated Learning with Content and Assessment Online (WILCAO). The course consisted of eleven learning packets with discrete multimodal experiences for students.
Finally, we identified six studies (Bayerlein 2020; Jordan et al. 2016; Mikroyannidis et al. 2020; Patiar et al. 2017; Robinson et al. 2020; Schech et al. 2017) as taking a technology-based approach, whereby the WBL experience was delivered with a high usage of technology. For example in Mikroyannidis et al. (2020), PT Anywhere facilitated learning for anyone wishing to experiment with network simulations in their own time at their own pace, in both informal and formal learning contexts. A unique feature of PT Anywhere is the access to real facilities (rather than simulated/virtual labs), which allows learners to gain an authentic experience of working with state-of-the-art research and network facilities. More examples of technology-based approaches are provided in the next section.
Overall, this result highlights a substantial shift in focus, as most studies identified by Schuster and Glavas (2017) in the period 2000–2016 were mainly supportive and low on technology, while in the last five years more studies used technology more intensively to deliver online WBL. It is important to note that we were unable to classify the two studies by Debora Jeske and Carolyn Axtell (2016a, 2016b) in detail due to a lack of description of the respective course design.
Online WBL and Education 4.0
In terms of innovative educational designs using the principles of Education 4.0, nine out of 13 studies aimed to allow learners to learn anytime and anywhere, whilst 80 per cent promoted more independent learning and different ways of assessing students. Six out of 13 studies took into account students’ opinions in designing courses, while half of the studies aimed to provide more hands-on learning activities. This latter finding might perhaps be surprising, but may also reflect the fact that the other studies did not explicitly mention hands-on learning activities, since these were assumed to be already in place. However, few studies included in their design specific elements for personalisation and learning techniques. As indicated in Fig. 2, on average the technology-facilitated and technology-based studies were more focused on the wide spectrum of Education 4.0 characteristics than the technology-supported studies. The technology-based online WBL in particular featured a strong focus on project-based learning, hands-on learning and data interpretation.
Evidence that online WBL is effective and impactful
In terms of RQ2, all studies included in the final sample selected for this SLR except Robinson et al. (2020) could be classified as action research or case-study research. A total of 1,015 respondents were included in the thirteen studies (M = 84.58, SD = 118.28, range: 7–390), and most studies used qualitative approaches. Some studies used surveys (Patiar et al. 2017), some used interviews (Bayerlein 2020; Glavas and Schuster 2020; Jeske and Axtell 2016a, 2016b), and some used a combination of approaches (Jordan et al. 2016; Mikroyannidis et al. 2020; Robinson et al. 2020). For example, Leopold Bayerlein (2020), using semi-structured interviews with seven students in a simulated internship, noted substantial differences in terms of affective, cognitive and skill-based learning outcomes. These experiences seemed to be substantially influenced by whether or not peer learning was used in the simulated internships, and by the students’ understanding of practice.
In interviews with 16 students who previously participated in online WBL, Glavas and Schuster (2020) found that the authenticity of the experience (i.e. design, environment, assignment) substantially influenced the lived experience, as well as the students’ ability to create co-presence with other participants. Jordan et al. (2016) used both survey reports (18% response rate) and follow-up focus group discussions with 68 participants to measure the success of the design. 84% of respondents indicated that they were using teaching strategies, while 68% said that they learned from teachers and from being part of professional team and developing the professional behaviour required.
Similarly, Mikroyannidis et al. (2020) found that most of the 111 respondents using an online asynchronous session in PT Anywhere over a 24-hour period involving two network simulation exercises found it easy to use. Additionally, the majority of respondents (63%) agreed that they would continue using PT Anywhere beyond the scope of this evaluation session.
In a large-scale quantitative study of virtual field trips (VFTs), Anoop Patiar et al. (2017) used factor analysis and follow-up regression modelling to explore the lived experience of 182 Australian Bachelor in Hotel Management students. In this particular VFT, students were able to explore virtual hotel environments and interactive materials as preparation for going on placements afterwards. The researchers found that system quality, enjoyment and content quality were significant predictors of the students’ overall satisfaction. Interestingly, international students seemed to be more satisfied with the VFT experience than domestic Australian students (ibid.).
At the same time, based upon interviews with 19 e-interns, academics and professionals, Debora Jeske and Carolyn Axtell noted that not all participants had similarly positive experiences, and some academics and career professionals felt that online WBL did not give e-interns the opportunity to “experience corporate culture, etiquette, and learn about professional expectations” (Jeske and Axtell 2016a, p. 4). Furthermore, many participants indicated that they had underestimated the time and planning needed to organise an online WBL.
In one of the few comparison studies, Sheridan et al. (2019) compared the grades obtained during an online course capstone with those in the previous cohort and did not find substantial differences in learning outcomes (although no significance testing was conducted). Only two studies effectively measured how participants were engaging in an online WBL and whether or not this was in line with the intention behind its design. Using learning analytics approaches to actual user behaviour, Mikroyannidis et al. (2020) were able to track the session duration, exercise completion, number of learning activities, and most-used commands in PT Anywhere, and found a strong correlation in activities. In their virtual patient study, Robinson et al. (2020) implemented not only a (retrospective) pre-post test of (self-reported) communication skills, but also engaged a clinical educator to measure actual communication interaction patterns during two subsequent interactions with the virtual patient during participants’ half-day training. The findings indicated that the ratings given by the clinical educator were significantly higher during the second rating. Students’ self-reported communication skills were also higher after experiencing online WBL (ibid.).
Overall, while all the above studies seem to suggest some support or (retrospective) evidence for the view that online WBL leads to positive learning experiences, there was limited hard evidence for this. Apart from Robinson et al. (2020), none of the studies implemented a pre-post design, included a comparison group, or designed an appropriate experimental study to compare whether or not a proposed intervention worked.
Discussion and conclusion
An emerging body of literature indicates that innovative WBL practices are being used in blended and online environments (Glavas and Schuster 2020; Schuster and Glavas 2017; Wood et al. 2020). While since the early 2000s an increasing body of literature has focused on blended forms of WBL (Fong and Sims 2010; Gerken et al. 2012), interest in online and remote forms of WBL has risen recently with the impact of the COVID-19 pandemic (Divjak et al. 2022; Seetal et al. 2021), and it is expected that online WBL will continue to grow. The SLR we have presented here explored 13 different studies on online WBL published in the period 2016–2021. It provides a detailed understanding of the varied design approaches to online WBL practices and their effectiveness in developing graduate skills.
Our first main finding (RQ1) is the notably increased adoption of technology and online delivery of WBL relative to the SLR findings of Schuster and Glavas (2017). While technology supported WBL at relatively low intensity in most of the studies identified by Schuster and Glavas (ibid.), most of our identified studies published between 2016 and 2021 focused on delivering online WBL in a high-intensity manner (in the typology of Schuster and Glavas 2017, they were “technology-based”). However, as evidenced by the design descriptions and our categorisation of Education 4.0, there are substantial differences across these studies. Studies that reported technology-based or technology-facilitated WBL, which correspond to the upper levels of technology use, support all nine characteristics of Education 4.0. Specifically, the following characteristics of Education 4.0 are supported in full: learning anytime anywhere; application of new ways of assessing knowledge and skills; and students becoming more independent learners.
Based on the 13 studies we analysed, we conclude that five characteristics can be ensured by any level of technology implementation. These are:
-
personalised learning;
-
having a choice how to learn;
-
hands-on learning;
-
data interpretation; and
-
the use of students’ opinion to update the curriculum.
Project-based learning, however, requires the highest level of technology implementation. A more nuanced understanding of the design patterns of online WBL is therefore required, which would go beyond the current conceptualisations of online WBL.
Another main finding from this SLR is the lack of robust evidence on the effectiveness and impact of online WBL (RQ2). While several studies (e.g. Bayerlein 2020; Jordan et al. 2016; Miller et al. 2020) indicated a potential positive impact of online WBL in terms of learners’ satisfaction and engagement with the learning activities and opportunities for hands-on work experience, other than Sheridan et al. (2019) no study compared or contrasted the findings with an alternative design or control group. Furthermore, except for Robinson et al. (2020), none of the studies included a simple pre-post test design to explore what behavioural or cognitive gains students might have developed over time (Rogaten et al. 2019).
There is therefore a research gap in evaluating the effectiveness and impact of online WBL. Since “traditional” WBL is already taking place in academic and industry settings, empirical research could be conducted in many different directions. First of all, it is important to include all relevant stakeholders (students, teachers and industry partners) in empirical research because the effectiveness and impact of online WBL depends on all of them. Secondly, different parameters can be researched, all of which have an impact on online WBL effectiveness, such as students’ motivation for participating in online WBL, stakeholder expectations, satisfaction with the various forms of blended and online WBL, development of generic skills, or intended learning outcomes. Finally, future empirical studies should include a wider set of teaching methods, study areas, countries, and types of higher education institution beyond the current literature in order to provide a more comprehensive and inclusive overview on the effectiveness and impact of online WBL.
Limitations and practical implications
One of the key limitations of this research is the inconsistency in the use of “WBL” and related terminologies (e.g. work-integrated learning, workplace learning, work-based experience). As a result, it is not always clear whether articles focus on online WBL. The high number of articles excluded after Phase 0 reflects this uncertainty, but even within those accepted, the terminology was not always applied consistently. As reported above, we found very limited evidence for the effectiveness and impact of online WBL. Still, the sample of studies included does carry a number of practical implications.
Including practitioners as active participants can promote effective online WBL design and increase its quality. Furthermore, (co)designing authentic tasks together with all stakeholders and being mindful of the affordances and limitations of institutional and external technical solutions can help to nurture trust and confidence in online WBL. The limitations of this SLR research are related to the studies analysed above. In addition, we narrowed our focus to studies in English because it was the only common language of all the RAPIDE team members.
Future research should consider how factors such as the COVID-19 pandemic have influenced WBL and accelerated the pivot to purely online environments. We also recommend more consistent use of terminology and more theoretically grounded research on online WBL, with a clear research design and collection of evidence that supports the evaluation of the strengths and weaknesses of online WBL.
Data availability
All the data for this study can be found at the Open University Data Online https://ordo.open.ac.uk/, or directly here: https://figshare.com/s/03139944f24fb5dbe8b5.
Notes
Furthermore, as indicated by Major, WBL “allows for informal learning where that learning is carefully identified and evidenced and assessed for credit for prior experiential learning as part of a planned programme of work-based learning, and where that prior learning is deemed to be relevant to, and congruent with, the planned outcomes of study. This definition implies that students are assessed using explicit assessment criteria against agreed learning outcomes and requires evidence of learning … that equals any other subject or field of study within the higher education curriculum” (Major 2016, pp. 27–28).
A micro-placement is a short internship typically lasting only two to five weeks, often involving work experience on a specific project.
The Internet of Things (IoT) refers to a network of smart devices (e.g. smartphones, interactive voice-controlled virtual assistants, smart meters etc.) which are able to collect and share data.
Coordinated by the Faculty of Organization and Informatics, University of Zagreb, the Erasmus+ RAPIDE project was launched in March 2021. Prompted by the experience of COVID-19 lockdowns, its aim was to “co-create, implement and share innovative pedagogies and aligned assessment for relevant and inclusive digital education”. The project wrapped up on 28 February 2023. For more information, visit https://rapide-project.eu/ [accessed 6 September 2023].
As requested by the reviewers, we repeated the search to identify relevant so-called “grey” literature on 23–24 November 2022, including four databases: World Bank eLibrary (55), OECD iLibrary (371), UNESCO Digital Library (142), and Google Scholar (256). We followed the same procedure as described above, subsequently analysing 12 full-text papers. However, none of them met all of our pre-defined criteria.
References
Bayerlein, L. (2020). The impact of prior work-experience on student learning outcomes in simulated internships. Journal of University Teaching and Learning Practice, 17(4), Art. no. 4. https://doi.org/10.53761/1.17.4.4
Divjak, B., Rienties, B., Iniesto, F., Vondra, P., & Žižak, M. (2022). Flipped classrooms in higher education during the COVID-19 pandemic: Findings and future research recommendations. International Journal of Educational Technology in Higher Education, 19(1), 9. https://doi.org/10.1186/s41239-021-00316-4
EC (European Commission) (2022). Education and training monitor 2021 [online resource]. Luxembourg: Publications Office of the European Union. Retrieved 18 August 2023 from https://op.europa.eu/webpub/eac/education-and-training-monitor-2021/en/chapters/leaflet.html
Fisk, P. (2017). Education 4.0 … the future of learning will be dramatically different, in school and throughout life. Peterfisk.com, 24 January [online article; author’s own website]. Retrieved 18 August 2023 from https://www.peterfisk.com/2017/01/future-education-young-everyone-taught-together/
Fong, M., & Sims, R. (2010). e-WIL in student education. Interdisciplinary Journal of E-Learning and Learning Objects, 6(1), 45–60. https://doi.org/10.28945/1135
Gerken, M., Rienties, B., Giesbers, B., & Könings, K. D. (2012). Enhancing the academic internship learning experience for business education – A critical review and future directions. In P. Van den Bossche, W. H. Gijselaers, & R. G. Milter (Eds), Learning at the crossroads of theory and practice. Advances in Business Education and Training, vol. 4, (pp. 7–22). Dordrecht: Springer. https://doi.org/10.1007/978-94-007-2846-2_2
Glavas, C., & Schuster, L. (2020). Design principles for electronic work integrated learning (eWIL). The Internet and Higher Education, 47, Art. no. 100760. https://doi.org/10.1016/j.iheduc.2020.100760
Godat, M. (2007). Virtual golden foods corporation: Generic skills in a virtual crisis environment (a pilot study). Journal of Educational Multimedia and Hypermedia, 16(1), 67–84.
Goodchild, T., Alias, N. A., & Luaran, J. E. (2017). The evolution of pedagogy for non-traditional students at a UK higher education institution. In N. A. Alias & J. Eddy Luaran (Eds), Student-driven learning strategies for the 21st century classroom (pp. 179–197). Hershey, PA: IGI Global. https://doi.org/10.4018/978-1-5225-1689-7.ch013
Holmes, A. (2011). Reciprocal and complementary knowledge conversion in a work integrated learning collaboration. Studies in Learning, Evaluation, Innovation & Development, 8(2), 40–52.
Hussin, A. A. (2018). Education 4.0 made simple: Ideas for teaching. International Journal of Education and Literacy Studies, 6(3), 92–98. https://doi.org/10.7575/aiac.ijels.v.6n.3p.92
Jackson, D., & Collings, D. (2018). The influence of Work-Integrated Learning and paid work during studies on graduate employment and underemployment. Higher Education, 76(3), 403–425. https://doi.org/10.1007/s10734-017-0216-z
Jackson, D., Rowbottom, D., Ferns, S., & McLaren, D. (2017). Employer understanding of Work-Integrated Learning and the challenges of engaging in work placement opportunities. Studies in Continuing Education, 39(1), 35–51. https://doi.org/10.1080/0158037X.2016.1228624
Jeske, D., & Axtell, C. M. (2016a). Going global in small steps: E-internships in SMEs. Organizational Dynamics, 45(1), 55–63. https://doi.org/10.1016/j.orgdyn.2015.12.007
Jeske, D., & Axtell, C. M. (2016b). How to run successful e-internships: A case for organizational learning. Development and Learning in Organizations: An International Journal, 30(2), 18–21. https://doi.org/10.1108/DLO-09-2015-0073
Jordan, K., Elsden-Clifton, J., & Khan, B. H. (2016). Designing meaningful learning environments in a Work-Integrated-Learning course using a DOCC design. In B. H. Khan (Ed.), Revolutionizing modern education through meaningful e-learning implementation (pp. 1–17). Hershey, PA: IGI Global. https://doi.org/10.4018/978-1-5225-0466-5.ch001
Kay, J., Ferns, S., Russell, L., Smith, J., & Winchester-Seeto, T. (2019). The emerging future: Innovative models of work-integrated learning. International Journal of Work-Integrated Learning, 20(4), 401–413.
Lester, S., & Costley, C. (2010). Work-based learning at higher education level: Value, practice and critique. Studies in Higher Education, 35(5), 561–575. https://doi.org/10.1080/03075070903216635
Major, D. (2016). Models of work-based learning, examples and reflections. Journal of Work-Applied Management, 8(1), 17–28. https://doi.org/10.1108/JWAM-03-2016-0003
Mikroyannidis, A., Gómez-Goiri, A., Smith, A., & Domingue, J. (2020). PT Anywhere: A mobile environment for practical learning of network engineering. Interactive Learning Environments, 28(4), 482–496. https://doi.org/10.1080/10494820.2018.1541911
Miller, K. K., de St Jorre, T. J., West, J. M., & Johnson, E. D. (2020). The potential of digital credentials to engage students with capabilities of importance to scholars and citizens. Active Learning in Higher Education, 21(1), 11–22. https://doi.org/10.1177/1469787417742021
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2010). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine, 8(5), 336–341. https://doi.org/10.7326/0003-4819-151-4-200908180-00135
Mundkur, A., & Ellickson, C. (2012). Bringing the real world in: Reflection on building a virtual learning environment. Journal of Geography in Higher Education, 36(3), 369–384. https://doi.org/10.1080/03098265.2012.692073
Patiar, A., Ma, E., Kensbock, S., & Cox, R. (2017). Students’ perceptions of quality and satisfaction with virtual field trips of hotels. Journal of Hospitality and Tourism Management, 31, 134–141. https://doi.org/10.1016/j.jhtm.2016.11.003
Peach, D., Gomez, R., & Ruinard, E. (2013). Reconstructing spaces and places through blended work integrated learning. In P. Wyse, J. Billot, S. Frielick, J. Hallas, E. Whitehead & N. Buissink-Smith (Eds), Research and development in higher education: The place of learning and teaching, vol. 36. Refereed papers from the 36th HERDSA Annual International Conference (pp. 336–345). Milperra, NSW: Higher Education Research and Development Society of Australasia, Inc (HERDSA). Retrieved 18 August 2023 from https://eprints.qut.edu.au/63416/
Perusso, A., & Wagenaar, R. (2021). The state of work-based learning development in EU higher education: learnings from the WEXHE project. Studies in Higher Education, 47(7), 1423–1439. https://doi.org/10.1080/03075079.2021.1904233
Rienties, B., Balaban, I., Ferguson, R., Herodotou, C., Iniesto, F., Muccini, H., Sargent, J., & Virkus, S. (2021). Education 4.0 and computer science: A European perspective. In Proceedings of the 32nd Central European Conference on Information and Intelligent Systems (CECIIS) 2021, held 13–15 October in Varaždin, Croatia (pp. 139–146). Varaždin: Faculty of Organization and Informatics, University of Zagreb. Retrieved 18 August 2023 from http://archive.ceciis.foi.hr/app/public/conferences/2021/Proceedings/ELA/ELA4.pdf
Robinson, K. E., Allen, P. J., Quail, M., & Beilby, J. (2020). Virtual patient clinical placements improve student communication competence. Interactive Learning Environments, 28(6), 795–805. https://doi.org/10.1080/10494820.2018.1552869
Rogaten, J., Rienties, B., Sharpe, R., Cross, S., Whitelock, D., Lygo-Baker, S., & Littlejohn, A. (2019). Reviewing affective, behavioural, and cognitive learning gains in higher education. Assessment & Evaluation in Higher Education, 44(3), 321–337. https://doi.org/10.1080/02602938.2018.1504277
Schech, S., Kelton, M., Carati, C., & Kingsmill, V. (2017). Simulating the global workplace for graduate employability. Higher Education Research & Development, 36(7), 1476–1489. https://doi.org/10.1080/07294360.2017.1325856
Schuster, L., & Glavas, C. (2017). Exploring the dimensions of electronic work integrated learning (eWIL). Educational Research Review, 21, 55–66. https://doi.org/10.1016/j.edurev.2017.04.001
Seetal, I., Gunness, S., & Teeroovengadum, V. (2021). Educational disruptions during the COVID-19 crisis in Small Island Developing States: Preparedness and efficacy of academics for online teaching. International Review of Education, 67(1), 185–217. https://doi.org/10.1007/s11159-021-09902-0
Sheridan, L., Gibbons, B., & Price, O. (2019). Achieving WIL placement and theoretical learning concurrently: An online strategy for Higher Education Institutions. Journal of University Teaching and Learning Practice, 16(3), Art. no. 8. https://doi.org/10.53761/1.16.3.8
Smith, C. (2012). Evaluating the quality of work-integrated learning curricula: A comprehensive framework. Higher Education Research & Development, 31(2), 247–262. https://doi.org/10.1080/07294360.2011.558072
Stewart, V., Campbell, M., & Wheeler, A. J. (2016). Developing a postgraduate work-based curriculum using an Intervention Mapping Approach. Studies in Continuing Education, 38(3), 318–333. Stewart10.1080/0158037x.2015.1120192
Stoker, R. (2015). An investigation into blogging as an opportunity for work-integrated learning for journalism students. Higher Education, Skills and Work-Based Learning, 5(2), 168–180. https://doi.org/10.1108/HESWBL-01-2014-0002
van de Werfhorst, H. G. (2014). Changing societies and four tasks of schooling: Challenges for strongly differentiated educational systems. International Review of Education, 60(1), 123–144. https://doi.org/10.1007/s11159-014-9410-8
Wood, Y. I., Zegwaard, K. E., & Fox-Turnbull, W. (2020). Conventional, remote, virtual and simulated work-integrated learning: A meta-analysis of existing practice. International Journal of Work-Integrated Learning, 21(4), 331–354.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Disclosure statement
We would like to thank the two reviewers for their detailed and constructive feedback. This study was conducted within the project “Relevant assessment and pedagogies for inclusive digital education (RAPIDE)” and financed by the Erasmus+ Programme of the European Union within KA226 – Partnerships for Digital Education Readiness.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Rienties, B., Divjak, B., Iniesto, F. et al. Online work-based learning: A systematic literature review. Int Rev Educ 69, 551–570 (2023). https://doi.org/10.1007/s11159-023-10008-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11159-023-10008-y