1 Introduction

Educational technology implementations in higher education institutions are becoming increasingly popular as a way to improve learning and teaching. The Association for Educational Communications and Technology defines educational technology as “the study and ethical practice of facilitating learning and improving performance by creating, using and managing appropriate technological processes and resources” (Januszewski & Molenda, 2008, p. 1). Simply put, educational technology (EdTech) is the use of technology in different educational settings to enhance learning and improve educational outcomes.

Globally, higher education institutions (HEIs) are using technology-based learning tools such as learning management systems or virtual learning environments (Turnbull et al., 2022), virtual and augmented reality (Jantjies et al., 2018), chatbots (Neumann et al., 2021), videoconferencing (Al-Samarraie, 2019), social media (Chugh & Ruhi, 2019) and mobile learning (Kaliisa & Picard, 2017). EdTech tools like these help instructors create engaging learning experiences for their students, leading to several short and long-term academic and social outcomes (Bond & Bedenlier, 2019). Additionally, EdTech can be used to facilitate communication between students and instructors, as well as to provide individualised feedback to students (Bower, 2019).

However, it is important to note that the implementation of EdTech in HEIs is not without its challenges (Cabaleiro-Cerviño & Vera, 2020; Laufer et al., 2021). Hence, it is crucial for HEIs to carefully evaluate the effectiveness and impact of these technologies before adopting them. Implementation research involves understanding the factors that influence implementation and a ‘scientific inquiry into questions concerning implementation’ (p. 1), such as those related to diverse stakeholders, the environment, and the strategies that can facilitate implementation (Peters et al., 2014). Furthermore, implementation research explores whether educational efforts are achieving the expected goals and objectives by asking questions that focus on ‘What are we doing? Is it working? For whom? Where? When? How? And, Why?’ (Century & Cassata, 2016, p. 169). Often implementation outcomes focus on ‘acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, coverage, and sustainability’ (p. 2), which serve as indicators of the success or failure of the implementation efforts (Peters et al., 2013). Accordingly, our research questions were formulated with an emphasis on implementation outcomes.

Literature reviews over the past decade have explored the role of educational technology on stress and anxiety (Fernandez-Batanero et al., 2021), e-leadership (Arnold & Sangrà, 2018), acceptance (Granić & Marangunić, 2019), effectiveness (Delgado et al., 2015), and creativity (Henriksen et al., 2021), but none have specifically focused on ‘implementation of EdTech’ in ‘HEIs’ settings. To fill the gap, this study provides both a quantitative measure of attributes such as region, discipline, data collection method, technology, and methodology, as well as a further qualitative review of the body of literature about EdTech implementations in HEIs. Literature was collated using the PRISMA process, and qualitative data were thematically grouped using NVIVO. For the purposes of this study, we will not focus on any one specific technology but use EdTech as an overarching term that refers to the use of any EdTech.

The remainder of the paper is structured as follows. The following section outlines the research methodology adopted in this study. The results are presented in tabular and graphic format in the next section. This is followed by the qualitative analysis, which outlines the coding scheme developed from an iterative inductive analysis of the shortlisted articles. Then a brief discussion is presented, along with a framework to guide the future implementation of EdTech in HEIs. Finally, a summary is provided in the conclusion section, and the limitations are outlined.

2 Research methodology

Exploratory implementation research that focuses on exploring an idea, such as EdTech implementation in HEIs, can utilise historical literature reviews as its research method (Peters et al., 2013). Hence, we adopt a systematic-narrative hybrid literature review strategy that combines elements of both systematic and narrative literature reviews. Like systematic reviews, this hybrid approach employs a methodical and transparent search method, including identifying the inclusion and exclusion criteria for the selection of the literature, and then uses a qualitative narrative approach for the analysis focusing on the main findings and themes (Turnbull et al., 2023).

In line with implementation research and the identified gap, we structure our study around the following research questions to conduct an in-depth analysis of the literature:

  • RQ1. What are the common EdTechs implemented in HEIs?

  • RQ2. How do HEI stakeholders perceive the implementation of EdTech?

  • RQ3. What theoretical frameworks and models are relevant to EdTech implementations in HEIs and the metrics to gauge post-implementation success?

Based on the research questions and the scope of the review, the following inclusion and exclusion criteria (Table 1) were developed.

Table 1 Inclusion and exclusion criteria
Full size table

The identification, screening, and inclusion steps of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) flow diagram (see results in Fig. 1) were followed, particularly to assist in recording the flow of information through the phases of the systematic review (Page et al., 2021). Four trusted scholarly research platforms (Education Source/Education Research Complete/Academic Search Ultimate (EBSCO), ProQuest, Social Sciences Citation Index (Web of Science), and Gale Academic), which were freely accessible from the researchers’ institutional library, were selected for the relevant literature search. Furthermore, the databases incorporated in these research platforms include a wide range of journals relevant to educational technology. Keywords and phrases used to search the academic databases for the most relevant articles included: educational technology, ICT, higher education, implementation, learning, higher education institutions, university, college, success, failure, and education technology innovation. Boolean operators (And, Or, Not) were used to combine keywords in various combinations. Appropriate truncation and wildcarding were also utilised where possible, in addition to the lemmatisation and stemming capabilities available on some platforms.

The search of the scholarly databases was conducted in January 2023 by one author. The initial search of the selected research platforms revealed 234 articles. Of these, 81 were removed because they were duplicates, had no locatable PDF, contained no author information, or were in languages other than English. The remainder were then manually checked to ensure fitness to the topic and population. In doing so, the article’s title, keywords, abstract, and full text were considered. Then, to reduce the chances of bias and ensure quality, another author repeated the search to ensure the results were consistent. Both these researchers also assessed the relevance of the publications in light of the inclusion and exclusion criteria. A total of 46 papers were included in the final tally. Figure 1 displays further details of the selection and exclusion process.

Fig. 1
figure 1

PRISMA flow diagram depicting inclusion/exclusion processes for article selection. Note: This figure is adapted from the PRISMA 2020 flow diagram template for systematic reviews (Page et al., 2021, p. 5)

Full size image

3 Results

Papers shortlisted from the PRISMA process were analysed in two ways. First, quantitative data was tabulated and analysed based on a set of identified attributes from Pickering and Byrne (2013). Then further qualitative analysis was undertaken using NVIVO, identifying key themes across the sample paper set. The sections below provide more details on each set of results.

3.1 Quantitative overview

Table 2 displays the identified attributes of the shortlisted studies. The Discipline column identifies each paper's main discipline area of study, and the Technology column lists the type of technology highlighted in the study. “N” denotes studies that do not focus on a particular discipline or technology type. The Data Collection column lists the main type(s) of data collection strategies employed, while the Methodology column identifies the general research approach of each study.

Table 2 Study characteristics
Full size table

The main methodology employed in the 46 papers was perception analysis, followed by experiments and case studies. Perception analysis, in the context of this research, involves the analysis of responses to survey data, experiments refer to research designs that attempt to measure and compare the impacts of changes to learning conditions as a result of technology introduction, and the case study label applies to papers that examine particular self-contained phenomena within a case study framework. Figure 2 displays the distribution of methodologies by region of research.

Fig. 2
figure 2

Distribution of methodologies by region

Full size image

In terms of papers that had a particular technology focus, LMS studies (N = 5) were the most prevalent, followed by AL, AR_AI and Web (N = 3 each). Figure 3 shows the distribution of papers with a particular technology focus with respect to the main discipline areas examined by researchers.

Fig. 3
figure 3

Distribution of technologies by discipline focus. Note on other abbreviations: AL = Adaptive learning, AR_AI = Augmented Reality/Artificial Intelligence, LMS = Learning Management System, QR = Quick Response, SM = Social Media. The educational discipline classification ‘N’ indicates that the study did not have a particular discipline focus

Full size image

3.2 Qualitative analysis

The PDFs of each study were imported into NVIVO, a qualitative analysis software, for further examination and analysis. Significant issues and phenomena were progressively coded using an iterative, inductive process to identify recurring themes in the data. Figure 4 depicts the final coding scheme developed from an analysis of the contents of the 46 papers in this study. There are four main coding categories: educational technologies implemented, implementation perceptions, implementation models and frameworks, and metrics used to gauge implementation success. Each category comprises one or more codes that identify more specific phenomena or issues which may overlap with other codes identified in this study. The rest of this section is devoted to analysing and discussing the significance of findings inherent in each code.

Fig. 4
figure 4

Final coding scheme

Full size image

3.2.1 Educational technologies

This section outlines five technology types identified in our research: LMS and related technologies, interactive technologies, visualisation and mobile technologies, web-based tools, and social media (SM). The studies were classified under each category on the basis of the main technology explored in each paper. The visualisation and mobile technologies categories were combined in the discussion because several studies focused on visualisation techniques that utilised portable mobile devices.

LMS and related technologies

One of the most prevalent technologies included in the examined papers was LMS, with nine papers featuring substantial comments on the implementation and use of this technology. In one paper, the study focused on the use of the LMS, Sakai, with student engagement in online resources and found a positive correlation between online access to resources and academic achievement (Crampton et al., 2012). Another dimension of LMS use identified in our review was stress and anxiety experienced by students in LMS environments. Test anxiety in LMS-moderated instances was explored in the study of language learning and the impact of the effectiveness of the Edmodo LMS as an assessment tool (EkmekÇİ, 2016). The findings revealed that students reported reduced test anxiety when delivered by this platform. This has implications for academic achievement, which generally improves by alleviating the adverse effects of stress and anxiety in learning situations (Jamieson et al., 2022).

We also identified studies that explored user perceptions of LMS efficacy which is important to the development of an understanding of the issues and challenges users face which must be resolved to successfully integrate this technology in HEI course delivery environments (Adnan et al., 2022) (more general stakeholder perceptions of EdTech implementation issues are discussed in the section, Stakeholder Perceptions). One such study (Garone et al. (2019) utilised a person-centric approach to investigating faculty acceptance of a new LMS, revealing three user categories: early adopters, early majority, and late majority. Each user category exhibited distinct characteristics, which helped to identify strategies to overcome LMS integration hurdles. A related study with an LMS user-perception focus looked exclusively at the effectiveness of the proprietary LMS, Blackboard, from a faculty and student perspective as a vehicle to enhance student learning (Jaiswal, 20202020) and found a high degree of satisfaction with the technology.

Teaching methods and approaches in LMS environments were also critiqued by Tsai (2015), who warned that traditional teaching methods might lead to unsatisfactory results when applied to online delivery modalities. Somewhat related to LMS in terms of their online presence are massive open online courses (MOOCs). MOOCs are online distance courses designed to be accessible to anyone with an internet connection, regardless of their geographic location and are typically open to unlimited participants (Rugube et al., 2022). Larionova et al. (2018) looked more generally at student perceptions of MOOCs as effective learning environments. They found that blended delivery and wholly online approaches that use real-time tutors did not detract from learning effectiveness compared to traditional class-based approaches. However, a study by Mellati and Khademi (2018) on the use of MOOC in language learning revealed that ubiquitous internet access and learners' emotional disposition impact overall MOOC effectiveness.

The final LMS-related technology identified in our study was educational management information systems (EMIS) which focus on providing HEI managers and executives with data and information to support decision-making (UNESCO, 2008). These were explored by Bravo et al. (2022) as a means to regulate quality in HEIs via stakeholder perceptions of the efficacy of these systems. We further discuss the findings of this study in the context of EdTech evaluation metrics in Section 3.4.

Interactive technologies

Interactive educational technologies in our coding scheme refer to systems that promote user engagement with automated educational tools. Three papers deal with issues surrounding the use of interactive tools that facilitated the sharing of information: a study that employed an audience response system called T1-Nspire that permits students to share their understanding of mathematical concepts via a shareable graphing tool (Pape & Prosser, 2018), an online feedback tool to enhance student capability to receive and reflect on peer and instructor feedback to assigned learning tasks (Roman et al., 2020), and an exploration into the use of synchronous technology using Adobe connect to enhance student learning outcomes (Wang et al., 2013). The unifying theme in all three studies is that technologies promoting information exchange between learners benefit individual learning outcomes.

Some studies also focused on more sophisticated interactive tools beyond simple automation processes. Two studies explored adaptive learning (AL) technologies that respond to individual stimuli and alter responses based on unique user input (Xie et al., 2019). Gregg et al. (2021) provided an overview of a six-year project examining the impact of AL on student performance, while Toktarova (2022) developed a model for an adaptive system for mathematics skills development within an eLearning environment. Augmented reality (AR) technology which supplements user views of the world by superimposing computer-generated visuals on observed reality (Carmigniani & Furht, 2011), was explored in a study of the application of AR-enriched notes to the delivery of educational content (Cabero-Almenara & Roig-Vila, 2019). The study found that enhancing learning materials with realistic AR content had a positive impact on student motivation. Finally, courseware generated by Artificial Intelligence (AI) using Acrobatiq SmartStart courseware was the focus of a study exploring the effectiveness of AI-generated courseware on student learning and instructor preparation which found positive impacts on student engagement and learning outcomes from the use of such courseware along with the capacity to develop course materials tailored to niche curriculum (Schroeder et al., 2022).

Visualisation and mobile technologies

Visualisation, especially dynamic content such as animations and simulations, are valuable ways to enhance the delivery of science-based material (McElhaney et al., 2014). Murthy et al. (2015) explored the incorporation of such content into established curricula in the context of their study on professional development programs on effective integration of educational technology. The study found that participants strongly preferred incorporating visual content in future course delivery operations. A study on using Quick Response (QR) codes in conjunction with video learning materials to enhance student performance in engineering courses (Ahmed & Zaneldin, 2019) highlighted the importance of linking QR codes to professional educational videos available via easily accessible pathways. Mobile technologies were also mentioned as significant agents of change. A 2014 study provided some insight into mobile learning using technologies such as iPads in the context of teacher education (Burke & Foulger, 2014). The authors applied a case study approach to their analysis of four institutions that were early adopters of mobile technologies to identify the factors such as pedagogical practices that are appropriate for educational delivery using this modality. A later study in 2020 explored foreign language teachers' perceptions of the use of mobile devices for language learning. The researchers concluded that while interest in mobile learning was high, actual deployment in course delivery was relatively low (Cavus et al., 2020).

Web-based tools

Web-based tools in this study are programs deployed by browsers and deliver self-contained functionality without the need to download and install specialised software. Thambirajah et al. (2022) explored the use of web-based dictionaries such as Alphadictionary.com and Skybrary.aero to improve mastery of English technical terms by aircraft maintenance students. The study found that blended learning and group learning environments were preferred by students when using this technology to expand their technical vocabulary. Web-based tools can also be used to facilitate access to publications and policy documents. For example, the use of a specialised educational web tool to disseminate journal publications and policy documents for an academic anesthesiology department provided evidence of increased utilisation of important publications by departmental trainees (Gopwani et al., 2021). A study by (Webb et al., 2015) looked at the use of web-deployed thin clients to deliver summative assessments from a technical and logistical perspective. They concluded that available web-thin clients' security and management features were adequate to carry out summative assessments.

Social media

Social media (SM) applications are technologies intended to connect users in a community-oriented space but are increasingly adapted for educational use. As indicated by two papers in our study, whether SM is perceived as a value-adding technology to educational pursuits is not universal. A Palestinian study on the use of Facebook by students in an undergraduate educational technology course revealed strong acceptance of this delivery modality as an effective way to learn (Shraim, 2014). This is in contrast to a study of US-based agricultural students on the comparative effectiveness of Twitter in the context of established Content Management Systems (CMS) and other proprietary tools (Murphrey et al., 2012). The conclusion drawn by the authors indicated that SM is not regarded as an effective pedagogical tool by students. Integration of SM technology into course delivery systems may well depend on unique discipline requirements and environmental factors inherent in the HEI that is implementing it.

3.2.2 Stakeholder perceptions

There are many stakeholder interests to consider when evaluating the impact of new technology on educational delivery systems. The main stakeholder groups determining e-learning success include students, educators, institutions, content providers, accreditation bodies, and employers (Adnan et al., 2022). In the studies included in our review, educators and students featured most prominently as information sources on EdTech effectiveness, closely followed by management and executive personnel. In the following discussion, we have segmented and consolidated the viewpoints of studies analysing stakeholder perceptions of EdTech effectiveness in terms of general enablers, barriers, and challenges to effective implementation.

Enablers

We found several evidence-based conclusions on HEI environmental conditions and activities that promote the effective implementation of educational technologies in discipline delivery in the examined literature. Much of this evidence was derived from analysing stakeholder views on EdTech efficacy. Indeed, prior to the implementation of new technologies and associated teaching practices, Jaiswal (2020) recommends soliciting the views of students and course educators on their potential usefulness and value. A study by Zhu and Engels (2013) on teacher and student perceptions of instructional innovation using EdTech in HEIs highlighted organisational factors such as goal orientation innovation, leadership, and collegial relationships as important predictors of stakeholder acceptance of instructional innovations involving new technologies. Institutional governance also features as an issue in the study of the cost-benefit of educational technology for pharmaceutical science faculty. The study found that greater institutional clarity of the goals and objectives of investing in EdTech helps maximise the value of these investments to course delivery systems (Harrison et al., 2019).

The availability of appropriate technical support and mobile device accessibility to LMS platforms are also significant facilitators of student acceptance of technology in flipped-classroom environments (Youhasan et al., 2021). The availability of mobile technologies is instrumental in promoting self-directed learning, one of the main objectives of flipped learning approaches (Hwang et al., 2015). Another factor promoting the acceptance of EdTech is appropriate teaching practices. Pedagogies aligned with using EdTech in course delivery were highlighted as the main driver of implementation success in a study of influencing factors to adopt technology-enhanced learning in Pakistani medical schools (Iqbal et al., 2018).

Appreciating millennials' learning preferences can also help identify opportunities to promote EdTech integration and use in course delivery practices. For example, implementing an online web tool to access a repository of Anesthesiology literature and policies was made possible by recognising millennial learners' predisposition towards tailored, online access to information (Gopwani et al., 2021). Finally, students themselves can serve as a useful vehicle to promote engagement with EdTech. For example, Hilburn and Maguth (2012) found that when social studies trainee teachers observed their classmates successfully utilising technology in learning situations, it promoted sharing technology adoption strategies.

Barriers

A significant barrier to EdTech implementation is faculty and institution assessment of students' capacity to cope with the introduction of new technologies. The study on the relationship between academic performance and online access to learning resources for distance students (Crampton et al., 2012) found that faculty overestimation of the ability of students to adapt to new technologies was a significant impediment to developing students’ online search and retrieval skills. Student abilities to cope with new technology were also highlighted as a possible impediment in a study on technology implementation in community college math discipline (Pape & Prosser, 2018). Educators, too, play a role in determining the success or otherwise of technology implementation. Kuleshova et al. (2022) highlight the lack of experience in applying EdTech to teaching situations as a significant barrier to implementation success. Significant institutional barriers to successful EdTech implementation were highlighted in some studies. These included a lack of technology-oriented culture, support for faculty, and appropriate training (Iqbal et al., 2018); a lack of awareness among faculty of educational policies on the use of technology (Habib & Johannesen, 2014); institutional cultures lacking ethics, transparency and accountability that demotivate academics to embrace significant change (Sart, 2014); and institutional aversion to perceived risks of implementing new technologies (Kvon et al., 2018).

Other challenges

EdTech implementation from a forward-looking perspective requires HEI stakeholder input to develop plans and strategies to address important implementation challenges. These include technological, pedagogical, organisational, and global challenges, as Mirata et al. (2022) argued in their study of technology-based learning at Tanzania’s Open University. One of the recommendations of the Gregg et al. (2021) AL study is that institutional thinking needs to move away from a tight focus on individual technologies to broader approaches that focus on resolving complex issues. HEIs are complex learning environments with unique challenges that require holistic solutions underpinned by robust pedagogical practices fused into the delivery mechanisms of technology-empowered course delivery structures.

Technical challenges also have to be overcome before EdTech implementation can become viable. The study on enhancing EFL student readiness to teach with Edtech Apps in Indonesia (Lisa et al., 2021) highlighted student access to the internet and affordable software tools for students as significant challenges to overcome before implementing technology-rich learning environments. The acquisition of adequate EdTech resources to service demand is also a concern, as educators cannot effectively plan course delivery incorporating new technology if insufficient classrooms are equipped with the new technology (Marcelo & Yot-Domínguez, 2018). Logistical issues related to the physical transport of EdTech assets from a central location to distributed sites to conduct assessments were also mentioned as a challenge in adopting technology to complete assessments (Webb et al., 2015).

3.3 Theoretical frameworks and models

Theoretical frameworks and models play a crucial role in EdTech implementation as they assist in implementation efforts and provide a structured approach to understanding how technology can enhance teaching and learning. Furthermore, educators, instructional designers or other decision-makers can use a theoretical framework or model to ensure that their technology integration efforts are aligned with established principles and best practices, leading to more effective and sustainable implementation outcomes (Alqudah, 2014; Young, 2008).

Our review revealed several theoretical frameworks and models: some well established in the literature, and others customised to specific situations. Technology acceptance theories were embedded in the theoretical frameworks of several studies. The Unified Theory of Acceptance and Use of Technology (UTAUT) relates the behavioural intention to use and actual use of technology to four constructs: performance expectancy, effort expectancy, social influence, and facilitating conditions (Venkatesh et al., 2016), and was built on the work of Davis (1989) who developed the Technology Acceptance Model (TAM). UTAUT was explicitly embedded as a theoretical construct to determine university teaching staff’s technology acceptance profiles with respect to the acceptance or otherwise of a new institutional LMS (Garone et al., 2019), while Murphrey et al. (2012) developed a framework and survey instrument for their SM study based on the UTAUT model. In contrast, a study exploring social media use in the context of technology acceptance (Lemay et al., 2019) was founded on a research design underpinned by TAM rather than UTAUT.

Another theoretical framework embedded in some studies was the Technological Pedagogical and Content Knowledge (TPACK), which is used to explore connections and interactions between content knowledge, technical knowledge, and pedagogical knowledge (Archambault & Barnett, 2010). Marcelo and Yot-Domínguez (2018) explicitly explored the question of technological and pedagogical knowledge integration from a teaching perspective based on the TPACK framework. Another Indonesian study developed and tested a TPACK-in Practice Model built on TPACK constructs for enhancing the preparedness of pre-service English teachers enrolled in university to apply technology to their teaching practices.

Three other studies presented unique and novel frameworks for the exploration of technology implementation issues for specific knowledge domains: the creation of a student-led committee as a framework to explore and make recommendations on effective EdTech implementation in medical education (Shenson et al., 2015); Toktarova (2022)’s adaptive mathematics training model (previously mentioned in the “Interactive technologies” section) that holistically combines competencies, subject areas adaptive content, and training trajectories; and a realist evaluation framework based on a detailed process map to investigate the effectiveness of e-learning lecture capture technology to deliver course content to technology students (King et al., 2017). However, in contrast to more established frameworks such as UTAUT, TAM, and TPACK, the extent to which these three models can be generalised to technology implementation approaches in other knowledge domains is not evident.

3.4 Metrics

The success or otherwise of EdTech integration into course delivery systems is contingent on the availability of suitable metrics to quantify the results. Grade improvement was a significant vindicating metric in two papers. Jaiswal (2020)’s exploration of the impact of integrating Blackboard LMS into student learning environments involved a measure of student pass/fail rates pre- and post-implementation. Another study on technology inclusion in an anatomy course also provided details of pass/fail rates as evidence of successful technology implementation (Toral Murillo et al., 2022). However, these measures should be treated with caution, as other influences besides the technology may have contributed to the improved results.

The dollar bottom line is often an ignored metric when academics attribute value to EdTech implementation. However, it is a core consideration of HEI administrators who have limited financial resources to fund academic programs (Luschei, 2014). A Canadian study that explored the value-for-money of EdTech implementation in a science department through a cost-benefit analysis with value-based approaches academics use to gauge success found a small positive net benefit expressed in dollar value (Harrison et al., 2019).

Risk was another metric featured in the study by Kvon et al. (2018) on LMS implementation (discussed in the “Barriers” section). The authors measured implementation risk via a survey instrument that contained an inventory of statements that students had to evaluate as High, Medium, Low, or No Risk. Quality is also a metric that is important to include as a measure of EdTech implementation success. While Bravo et al. (2022) focussed specifically on education managers’ perspectives on quality management (QM) and institutional EMISs, the authors conclude that a better understanding of manager and employee profiles will enhance EMIS effectiveness and strengthen institutional QM through customised training courses that address the characteristics of each group. As quality in educational delivery can be a difficult construct to measure objectively, the acceptance of EMIS as a tool to collect and synthesise data can only improve efforts to promulgate a culture of continuous improvement across HEIs.

4 Discussion

Looking at these results in aggregate, it is clear that the work of Cabaleiro-Cerviño and Vera (2020) and Laufer et al. (2021) is confirmed, and that this field can be quite wide-reaching and diverse, resulting in challenges in implementation. Further, the need to understand the factors that influence implementation and a ‘scientific inquiry into questions concerning implementation’ (p. 1), such as those related to diverse stakeholders, the environment, and the strategies that can facilitate implementation as identified by (Peters et al., 2014) is reflected across the methodologies used in the studies. However, it is also clear that the wide set of methodologies means clearer guidance is needed.

Similarly, as asserted by Peters et al. (2013), the low measure of studies with clear research questions and methodologies suggests that, indeed, outcomes do often focus on ‘acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, coverage, and sustainability’ (p. 2), serving as indicators of the success or failure of the implementation efforts (Peters et al., 2013), with a focus primarily on whether educational efforts are achieving the expected goals and objectives by asking questions that focus on ‘What are we doing? Is it working? For whom? Where? When? How? And, Why?’, as outlined by (Century & Cassata, 2016, p. 169).

This means that answers to our research questions, whilst existing, are quite broad. For RQ1, “What are the common EdTechs implemented in HEIs?”, the answer would appear to be that there is a breadth of technology used, and that this changes over time, with a current focus on Augmented Reality, Artificial Intelligence and Adaptive Learning. However, a further dive into qualitative data indicates that this is a quickly changing measure, and it is perhaps difficult to understand what is common in EdTech.

Similarly, the answer for RQ2, “How do HEI stakeholders perceive the implementation of EdTech?”, is also fairly broad. Stakeholder feeling appears to be closely tied to the use of the technology rather than the underlying pedagogy, indicating that a holistic answer to this question is complicated.

It is perhaps only with RQ3, “What theoretical frameworks and models are relevant to EdTech implementations in HEIs and the metrics to gauge post-implementation success?” is the answer narrow enough to be truly useful. It is clear from the review that a few frameworks, such as UTAUT, TAM, and TPACK are quite widely regarded as suitable approaches to understanding EdTech. However, as identified, even in the use of these framings, it is often only measures that are taken, and not a holistic framing of the space that is useful for future practical implementation. This suggests that there is room for a more rigorous framework for EdTech implementation.

5 Implications for HEIs – A framework for EdTech implementation

As suggested in the previous section, our study's results revealed multiple considerations relevant to the complex process of deciding on an EdTech implementation strategy that will maximise utility to diverse institutional stakeholders. Therefore, we propose a model (Fig. 5) incorporating five dimensions that could be included in EdTech implementation decision-making: technology, stakeholder perceptions, academic discipline, success metrics, and theoretical foundations.

Fig. 5
figure 5

Framework for EdTech implementation in HEIs

Full size image

The technology dimension encompasses the characteristics and functions of EdTech platforms that are under consideration for implementation. These include LMSs, interactive technologies, SM, visualisation technologies, mobile platforms and web-based tools, as outlined in this paper. Stakeholder perceptions of EdTech implementation are also critical inputs into decision-making, impacting implementation success from an HEI community acceptance standpoint. For example, students and faculty may be able to pinpoint enablers (e.g., the availability of appropriate technical support for new technology), barriers (e.g., the inability of students and faculty to adapt to new technologies without appropriate training), and challenges (e.g., acquiring affordable software options that enable students to maximise their use of new infrastructure). In many cases, the academic discipline may play a role in EdTech implementation decisions. Figure 3 displays papers in this study that align educational technologies with specific educational disciplines. For example, Jaiswal (2020) explored gaming technologies in the context of language learning. Success metrics in this model are quantitative indicators of implementation success. In this study, we have identified improved student results (e.g., assessment scores and pass rates), value-for-money, EdTech risk assessment, and quality outcomes as metrics worthy of incorporation into EdTech evaluation processes. Last, theoretical frameworks such as TPACK, TAM, and UTAUT can provide guidance in determining appropriate strategies to incorporate new technologies into existing course delivery structures. All five dimensions in this model need to be holistically considered when developing protocols to determine the choice of new technologies and how they will be implemented. In doing this, a more practical, repeatable and measurable focus on EdTech implementation can be taken by researchers in the space.

6 Conclusion

Implementing educational technology is a complex, multi-faceted decision-making process that requires input from many perspectives. In addition to the technical and pedagogical value of a technology of interest, HEIs must consider stakeholder acceptance, implementation challenges, theoretical foundations underpinning technology use, and evaluation metrics when choosing a pathway to infuse new technologies into existing course delivery frameworks.

In this study, we examined empirical research over a ten-year period that investigated EdTech implementation phenomena in a diverse range of disciplines, technologies, stakeholders, and contexts. Through the synthesis of stakeholder views on Edtech implementation, researcher observations, and educational outcome performance measures, this paper identifies common educational technologies, challenges to EdTech implementation, implementation frameworks, and evaluation metrics that underpin the effective deployment of EdTech assets in complex HEI environments. To that end, we propose a model for EdTech implementation decision-making that incorporates five key components (technology, stakeholder perceptions, academic discipline, success metrics, and theoretical frameworks) for developing EdTech selection protocols.

In dissecting the complexity inherent in making good choices when attempting to improve HEI course delivery systems, the proposed model and the findings underpinning it contribute to a better understanding of the key dimensions governing the successful implementation of educational technology in contemporary contexts. From a practical viewpoint, this comprehensive literature review can provide a roadmap to HEI decision-makers in making informed, evidence-based decisions in choosing and implementing EdTech tools that support their institution’s pedagogical approach and enhance the overall teaching and learning experience, ultimately improving student outcomes.

However, this study is not without its limitations. The empirical literature selection was constrained to English manuscripts from four carefully selected research platforms. Manuscripts from other sources were not captured, and studies in languages other than English were omitted. We also excluded studies not based on primary sources, such as surveys, interviews and observations. Including carefully selected research based on secondary data could yield other insights. Future research into EdTech implementation issues could address these limitations and expand the scope to include other educational contexts, such as schools and corporate training departments.