Introduction

Adaptive learning is a data-driven approach to instruction that enables personalised learning experiences in online and blended learning environments and the promotion of access and quality education at a large scale in higher education (Becker et al., 2018). Adaptive learning has shown potential in improving student outcomes (Green, 2018; Holthaus et al., 2019), decreasing course drop-out rates (Oxman & Wong, 2014) and equity gaps (Vignare et al., 2018), supporting non-traditional and low-income students, and providing interactive content in remote areas (Brown et al., 2020). Adaptive learning often narrowly refers to technologies or technological solutions, such as adaptive courseware (Vignare et al., 2018), adaptive learning systems (Ennouamani & Mahani, 2017; Zimmermann et al., 2005), and adaptive intelligent tutoring systems (Bagheri, 2015) that include instructional design, assessment, and real-time data analyses of each student. Based on students’ data that exhibit their learning progress and needs, an adaptive system can constantly modify instruction and learning activities to tailor them to the needs of each student during a course. However, technologies along do not yield the greatest impact on learning outcomes (Brown et al., 2020). Recent studies have recognised that although adaptive learning is based on technology, it remains closely linked to practices (Brown et al., 2020; Shelle et al., 2018). Similar to technological developments, such as Massive Open Online Courses (MOOCs) and mobile learning, adaptive learning is, therefore, based on technology and other relevant aspects (e.g., instructional design) that influence teaching and learning.

Despite its potential, the implementation of adaptive learning remains limited today (Cavanagh et al., 2020). The prediction of the wide implementation of adaptive learning by experts has definitely failed (Brown et al., 2020). A major reason for the slow implementation is that adaptive learning continues to evolve (Imhof et al., 2020). As such, its implementation occurs in various contexts, including different courses, different courseware products, and unique institutional conditions (Vignare et al., 2018). Another reason is that majority of adaptive learning applications today are commercial, which results in high licensing fees for their implementation in institutions (Johanes & Lagerstrom, 2017). Finally, as reported by Vignare et al. (2018), institutions remain reluctant to share the results of their implementation with others.

Recognising the positive impacts of adaptive learning on learning with the endeavour to accelerate its implementation, recent research has explored best practices and developed recommendations and models to support its implementation in the context of higher education (Alameen & Dhupia, 2019; Johnson & Zone, 2018; Tyton Partners, 2016; Vignare et al., 2018). A few studies have focused on identifying the key determinants (e.g., barriers) and dimensions influencing implementation (Bailey et al., 2018; Johnson & Zone, 2018; Kleisch et al., 2017; Tyton Partners, 2016), whereas others described key processes and steps (Vignare et al., 2018). Although recent studies on adaptive learning intended to identify critical factors, implementation strategies and outcomes, the connection among these elements remains unclear. Defining the relationships between these elements can however help to elucidate the underlying processes that play a role in both successful and failed implementation (Smith et al., 2020). According to Smith et al. (2020), doing so is the only way to open a ‘black box’ and reveal how relevant factors are interrelated and specific implementation strategies operate to predict outcomes, which denote how to reach the broad implementation of adaptive learning.

Addressing the described research gap, this study aims to explore the existing implementation models specific to adaptive learning and proposes an empirically supported implementation framework for adaptive learning by identifying relationships between determinants, implementation strategies, and corresponding outcomes. We draw on the basic framework structure proposed by Mirata and Bergamin (2019) and conduct a secondary analysis of qualitative data from the international Delphi study conducted in two universities against the background of distinct structural and socio-economic contexts (Mirata et al., 2020). The Delphi study aimed at identifying the relevant implementation challenges of adaptive learning at two universities. In total, 51 experts participated in the Delphi study. The experts were asked to reflect on the challenges and framework conditions required for the scaled implementation of adaptive learning at their respective universities. Apart from the identified challenges, the qualitative data revealed numerous comments on possible solutions to such challenges and the relationships between them, which we aim to investigate and synthesise. Qualitative data were managed and analysed using NVivo 12 Pro (QSR International Pty Ltd., 2018). For coding, we used various inductive coding techniques suitable for identifying connections (e.g., causal relationships) between key factors e.g., determinants, strategies, and outcomes (Charmaz, 2006).

The study poses the following main research questions:

What are empirical relationships between the determinants (e.g., barriers, facilitators) responsible for the successful and unsuccessful implementation of adaptive learning, the proposed strategies that help to overcome the barriers and leverage the facilitators, and the expected outcomes (e.g., scaled implementation)?

We begin with a brief description of the existent models specific to adaptive learning. We then describe the methodology and present the results of the research. Finally, we discuss the implications of our findings for research and practice as well as the limitations of the study. Table 1 presents the definition of terms.

Table 1 Terms and definitions
Full size table

Related works and implementation research in higher education

Although several studies focus on investigating the challenges and barriers of adaptive learning and propose recommendations and lessons learnt based on them (Bailey et al., 2018; Hall Giesinger et al., 2016; Tyton Partners, 2016), only few studies suggest frameworks and models (hereafter called models) related specifically to the implementation of adaptive learning (Johnson & Zone, 2018; Mirata & Bergamin, 2019; Vignare et al., 2018). Indeed, as it was pointed out by Brown et al. (2020), traditional implementation models for technological developments, such as adaptive learning, are nearly absent today, which may be due to the small number of successfully implemented adaptive learning systems in practice (Imhof et al., 2020). In the following, we present related models on the implementation of adaptive learning, one process model that describes key implementation processes and steps (Vignare et al., 2018) and two input models that consider the identified determinants (e.g., barriers, facilitators) and dimensions as key factors responsible for its successful implementation (Johnson & Zone, 2018; Mirata & Bergamin, 2019).

Johnson and Zone (2018) proposed an input implementation model of adaptive learning that depicts the strategy of the Colorado Technical University for scaled implementation. Apart from faculty engagement as a central category of the model, they identified student involvement, management commitment, investment in defining clear processes, full-time teaching staff and the recognition of the benefits of adaptive learning as critical factors influencing implementation. To support implementation through faculty engagement at the university, Kleisch et al. (2017) additionally suggested a specific faculty training and development model, which incorporates faculty training activities into navigation, instructional methods, content delivery, and course development.

In contrast to Johnson and Zone (2018), Mirata and Bergamin (2019) have suggested an implementation model of adaptive learning using the five key dimensions related to successful implementation. These dimensions are linked to technological, pedagogical, and organisational issues as well as societal regulations, cultural and political conditions that influence implementation in higher institutions. Additionally, this model outlined the basic structure of the determinants (e.g., prerequisites, barriers and facilitators) associated with the implementation of adaptive learning. Using a multidimensional model, Mirata and Bergamin (2019) advocated the complex nature of adaptive learning and the need to pursue a multi-perspective approach for its implementation.

Vignare et al. (2018) proposed a six-phase process model based on the best implementation practices of eight higher educational institutions in the United States. These institutions participated in the grant programme of the Bill and Milinda Gates Foundation, which aimed at accelerating the adoption of adaptive learning by public universities. The six phases of the model are organised into three large-scale stages related to planning, building, and using courseware in the implementation process. The planning stage includes, for example, the phase of establishing support by institutional stakeholders in the form of time commitment and the provision of required resources as well as a phase for discovering and decision-making, in which available resources are evaluated, whereas the adaptive technology and courses for adaptive implementation are selected. The second stage includes the following phases: blueprint design of the pilot adaptive course and further development of the adaptive courses for a pilot implementation. In the third stage, the course instructors teach their courses, which entail certain adaptive parts. Data for the pilot implementation are collected for the evaluation of the impacts on the success and improvement of students. In the last scaling phase, institutions expand the use of adaptive courses to the target level based on lessons learnt from the pilot implementation. Additionally, each model phase describes the required activities or strategies to reach the objectives of corresponding phases.

In contrast to the implementation model of Johnson and Zone (2018), Vignare et al. (2018) described each implementation phase, its objectives and specific strategies to meet these objectives. Their procedure is in line with the requirements of implementation research, whose biggest challenge remains the proposal of such strategies that are specific to relevant determinants and expected outcomes (Powell et al., 2015; Smith et al., 2020). In a review of implementation strategies, Powell et al. (2015) found, for instance, that proposed strategies were poorly designed and were mismatched with determinants (i.e., barriers were identified at the organisational level; however, strategies were focused on structures and processes at other levels). However, empirical research demonstrated that such relationships do not necessary require a unique one-to-one correspondence (Waltz et al., 2019). To address one barrier, for instance, more than one strategy may be required, whereas one specific strategy may address various contextual barriers and challenges. Therefore, proposing an implementation framework for adaptive learning that reveals which specific strategies are intended to address which challenges (e.g., barriers, determinants) and influence which outcomes is particularly important, because such empirically identified connections would aid in the understanding of which implementation strategies work where, why and under which conditions to achieve a scaled implementation. Moreover, identifying such empirical connections would help shift implementation research from descriptive studies of barriers that have dominated implementation research for too long (Proctor et al., 2012) to more explanatory research. As the reviewed literature has shown, descriptive studies continue to dominate implementation research on adaptive learning today too, and some effort should be made to move it toward a more explanatory direction. The synthesis of the elaborated models is presented in Table 2. This study addresses the described gap by proposing the empirically supported implementation framework for adaptive learning and intends to advance the implementation practices of adaptive learning and the field of implementation science in education, which remains limited today (Century & Cassata, 2016; Soicher et al., 2020).

Table 2 Summary of the elaborated implementation models of adaptive learning
Full size table

The framework can be useful for researchers and project leaders conducting implementation projects in their institutions. They can use the framework to reflect on the institution’s context and readiness for implementing adaptive learning at the beginning of the project by selecting determinants, i.e., contextual factors, from the framework that are relevant for their unique implementation setting. The framework can also be used to monitor and improve an ongoing implementation process by selecting and operationalising specific strategies that address the previously identified factors. Finally, it can help researchers, course instructors, and university management to raise awareness of critical factors and conditions as they plan to implement adaptive learning in their institutions or explain its currently limited use.

Methodology

This study aimed to identify the relationships between determinants, strategies, and expected outcomes to develop an empirically based implementation model for adaptive learning.

Research design and data collection

To reach these goals, we used qualitative data from the Delphi study previously conducted in two universities in Switzerland and South Africa, which were characterised by different study models, teaching traditions, implementation phases of adaptive learning and socio-economic conditions (Mirata et al., 2020). The Delphi method was used to collect anonymous statements from experts on such a complex topic as adaptive learning through an online communication process without face-to-face interaction (Nworie, 2011). A typical Delphi process contains multiple rounds of data collection and data analysis with intermediate summary reports on survey results, which the researchers provide participants after each round. Based on the results provided in the summary reports, the participants can modify or reconsider their previous knowledge on the topic in the following rounds. The Delphi method is considered an efficient technique in implementation science field to explore implementation processes and to promote conceptualisation (Vax et al., 2021). The main goal of our four-stage Delphi study was to identify, categorise, and prioritise the challenges of adaptive learning, viz. influential factors that prevented both universities from its scaled implementation. The first qualitative round focused on technology-based learning in general and served as the basis for designing a second qualitative round on adaptive learning. In the second round, experts were asked to identify and describe the challenges of adaptive learning their universities faced. Open-ended questions were used to collect the experts’ statements. The questions were designed based on the reviewed literature on adaptive learning implementation barriers and the categories of challenges identified in the first round, for example: “What needs to change in the use of learning technologies in your institution so that adaptive learning can be (a) widely applied by faculty and (b) properly used by students?” (Technological category); “What organisational conditions are still needed in your institution to enable the broad implementation of adaptive learning scenarios?” (Organisational category). The second round resulted in a list of challenges or factors from which the experts were asked to select up to ten the most relevant ones in the next third round. In the last fourth round, the selected challenges were ranked in order of priority for scaled implementation. In both quantitative rounds, the participants were asked to justify their decisions and to suggest some possible solutions to the challenges. Mirata et al. (2020) provides a detailed description of the Delphi procedure and data analysis.

To apply the Delphi method in two universities with different characteristics was motivated by the need to gain more insights into the challenges of adaptive learning from other contexts, because the implementation challenges of adaptive learning discussed in the literature were dominated by the views of just a few countries (e.g., U.S., UK, and Australia). However, comparing cases with different experiences in adaptive learning and from different contexts could help researchers reconceive factors that could influence the enactment of educational innovations, such as adaptive learning, gain different perspectives on the implementation problem (Johnson et al., 2016), and advance implementation science in education to yield generalisable knowledge (Proctor et al., 2012; Soicher et al., 2020). The results of the Delphi study presented the challenges of adaptive learning that were relevant for its scaled implementation in the two universities.

Panel participants and ethics

The study panel was carefully designed on the basis of the predetermined criteria to ensure that the experts possess extensive knowledge and profound experience in using technological innovation in educational settings (Elo et al., 2014). In total, 51 experts participated in the Delphi study. The heterogeneous sample of both universities included lecturers, researchers, management, and other academic staff, such as technology advisers and analysts. The ‘gatekeeper’ recruitment strategy was used to recruit the participants, according to which field representatives suggested appropriate experts for the study (Brady, 2015). The Delphi study at both universities was conducted at nearly the same time. The questionnaires were disseminated by email containing a link to a web-based survey. Participation was voluntary without negative effects, if the request was declined. Moreover, the participants explicitly stated agreement for each questionnaire and could withdraw from the study at any time without explanation.

Data analysis

The supplementary secondary analysis (Heaton, 2008) was conducted by re-using qualitative data from the previous Delphi study with the objective of establishing the empirical relationships between the identified challenges, strategies and outcomes. Data included answers to open-ended questions, comments and the justification of the experts for their ratings in the quantitative rounds. Data were analysed using NVivo 12 Pro (QSR International Pty Ltd., 2018). Although the category system of the challenges identified by the previous Delphi study was taken as the framework for the secondary analysis (challenges as focused codes), we remained open to the emergence of new challenges and other aspects throughout the secondary data analysis. In fact, the focused codes were useful means for further work on the information materials they contained (Charmaz, 2006). For further coding, we used inductive coding techniques suitable for identifying links (e.g., causal relationships) between the identified variables (Charmaz, 2006). In particular, we used the axial coding method as a frame to discover relationships around the ‘axis’ of previously identified challenges at the conceptual level by closely examining their contexts, consequences, and participants’ reactions to them. Two knowledgeable researchers coded the data. During the first-cycle coding, the principal researcher coded the data. Then, the coded data were sent to the second researcher for further amendments. During the second-cycle coding, two researchers met weekly to work through the coded material together for eliminating inconsistencies. Two researchers discussed the emergent connections and corresponding quotes until a consensus was reached. To ensure the quality of the research, we followed the recommendations proposed by Elo et al. (2014), which focused on the validation and reliability strategies suitable for qualitative research. The accuracy of the findings (validation) was ensured through reflexive discussions among the researchers about their experiences and the possible bias that may influence interpretation (researcher triangulation). Moreover, reliability was assured by a detailed documentation of the data analysis procedure and the definition of theoretical concepts in NVivo. To establish the trustworthiness of the findings, we used the quotations of the experts and a presentation of the proposed implementation strategies, which were rendered as close as possible to the original quotations.

Results

This section presents the adaptive learning implementation framework (ALIF) by describing its core elements and empirical relationships between determinants, corresponding strategies, and outcomes (Table 3). The proposed implementation framework consists of three core elements, namely, (a) determinants that prevent or enable implementation, (b) specific strategies that address these determinants, and (c) expected proximal and distal outcomes, which are influenced by the proposed strategies. ALIF illustrates the manner in which the identified determinants, strategies, and outcomes are linked in a sequence, whose relationships were empirically supported through qualitative data. The core elements are categorised into dimensions related to technological, pedagogical, and organisational challenges, which have been identified in the previous Delphi study (Mirata et al., 2020). These three dimensions represent the inner context of the implementing institution at the organisational level. Additionally, the core elements are differentiated across two implementation stages. The first stage (assessing organisational readiness) is characterised by awareness and exploration of critical factors and resources required for successful implementation. The second stage (building up capabilities/piloting) is characterised by the discovery of gaps and establishment of capabilities relevant for its scaled implementation. Finally, the proposed framework points out the role of the outer context, which describes the unique university environment that influences the implementation of adaptive learning at the societal and global levels (societal and global dimensions).

Table 3 Adaptive learning implementation framework
Full size table

Explaining the framework’s elements and their relationships

The following sections describe the findings in terms of the core elements of the framework, although elements do not occur separately but are rather linked with one another in a causal chain (i.e., determinants, strategies and outcomes; Table 3). The narratives of the participants are used to support the presented connections.

Determinants

The first finding of the study was that all challenges associated with effective implementation were classified into three determinant groups according to their valence, namely, facilitators with a positive impact (+); barriers with a negative impact (−) and challenges with a neutral impact (0). However, a specific determinant could act as a prerequisite, moderator, and strategy or an outcome according to its empirical connection with other elements. For example, in one empirical connection, the motivation of an instructor to use an adaptive learning system in teaching practices was identified as a facilitator that positively influenced the acceptability of adaptive learning by students, i.e. their perception that adaptive learning was agreeable or satisfactory (Fig. 1). However, in other empirical connections, motivation acted as a targeted intermediate outcome (Fig. 2). In such connections, the motivational aspects of the use of students and instructors of an adaptive learning system were influenced by determinants related to the use of learning analytics methods in online courses, robustness of an adaptive system, and the recognition of the advantages of adaptive learning by students and lecturers (Fig. 2).

Fig. 1
figure 1

Motivation as a facilitator (determinant)

Full size image
Fig. 2
figure 2

Motivation as an (intermediate) outcome

Full size image

The following quotes support the link between motivation and acceptability of adaptive learning. For example, several experts of both university panels reported that ‘adaptive learning implementation required much time, engagement and motivation of the instructors’ and ‘motivation was crucial for acceptance and implementation’. Another expert commented on the impact of the motivation of instructors to use an adaptive learning system in courses on the acceptance of students: ‘Only if instructors are engaged in using an adaptive learning system in their courses, all potentials of this technology can be brought to students. The students can use as much of this technology as instructors use it in their courses’.

The following statements from experts illustrate the influence of the above-mentioned determinants on motivation as an intermediate outcome (Fig. 2). For example, the qualitative data demonstrated that motivation was influenced by the robustness of the system and learning analytics capabilities. One expert was convinced that ‘a well-functioning adaptive learning system and the visualisation of students’ success in it could increase the enthusiasm of the lecture and the motivation of the students’ to use an adaptive system in courses. Another expert stressed the importance of learning analytics but expressed concern on the manner in which it is addressed in the current adaptive learning system at the Swiss university: ‘It needs better and more raw data and also a system that collects these data and motivates the use [of an adaptive system]. Moodle logs’ data are insufficient, and the feedback system is laborious’.

The Swiss university panel identified the unrecognised advantages of adaptive learning as one of the main challenges, although adaptive learning had been piloted at the Swiss university for several years. Thus, recognising the advantages remained one of the key determinants that influenced not only the acceptability of adaptive learning by all stakeholders but also the motivation of students. As qualitative data illustrated: ‘Students need to recognise the benefits of this [teaching] model to be motivated to take full advantage of the adaptive learning model’.

Another finding was that some determinants acted as important prerequisites associated with successful implementation. This finding implies that if such fundamental factors are absent, then the implementation of adaptive learning is doomed to fail since the very beginning. Along the technological dimension, internet issues, such as access, quality, affordability and the accessibility of appropriate technology (e.g., computers, notebooks) were identified as ‘fundamental factors’ or prerequisites among the South African panel only. In their responses, the experts pointed out that ‘internet costs are too high’ and ‘all technology needed for adaptive learning should be provided, which will have a large burden on the budget’. Indeed, high costs linked to tremendous investments into required technical infrastructure were the most common outcome of technological barriers required for the implementation of adaptive learning. However, the socio-economic context of an implementing institution may act as a moderator of the relationships between technological barriers and financial outcomes for an institution (Table 3).

Internet quality, availability and the primary technological infrastructure of the Swiss university were in place even prior to the implementation of adaptive learning, which exerted a non-significant impact on cost in comparison with the South African university. Additionally, as indicated by qualitative data, technological determinants could influence not only the implementation itself but also broader sociocultural aspects, such as equity in education. This link was identified solely within the South African panel. Other experts commented on equity and inclusiveness as other distal outcomes influenced by the availability of the necessary technology: ‘Provision for access to technology would be crucial to prevent the increasing digital divide [between students] as a result of efforts to expand technology-enhanced learning’. ‘[One of the challenges is] availability of technology to all students (colour does not matter)’. These findings imply the important role of an outer context regarding some moderator determinants in an empirical relationship between determinants and outcomes.

Additionally, the qualitative data revealed two neutral challenges related to the need to redesign the curriculum and courses for adaptive learning and to shift to adaptive pedagogy that involved changes in traditional teaching methods and instructor roles. One participant of the South African panel noted that, ‘it is necessary that all lecturers must realise that their role as lecturers have changed. They are not information givers but rather the guide on the side. In other words, they have to become facilitators of their students’ learning’.

The participants of both panels commented on both neutral challenges as related to the affordances of adaptive learning and recognised that ‘the development of adaptive activities were costly’ and ‘the time-consuming course development required monetary incentives for the lecturer, training, and support’.

Outcomes

Outcomes refer to the effects of purposive actions undertaken to implement adaptive learning. Although the participants were requested to identify the challenges of adaptive learning relevant to scaled implementation only, the qualitative data revealed various other distal (e.g., equity in education) and intermediate (e.g., acceptability, cost, and increase of return of investment) outcomes.

The most frequently mentioned factor by the experts of both panels was the acceptability of adaptive learning by all stakeholders, that is, their perception that adaptive learning is agreeable, satisfactory, and advantageous in comparison with traditional online learning models, which we identified as the desired intermediate outcome. The participants of both panels stated acceptance as an important challenge in the implementation process: ‘Acceptance by lecturers and students must be created’; ‘Full acceptance by the students [is missing]’ and ‘Lecturers need to put in more effort to start integrating technology in their learning environments. Lecturers are in a comfort zone and do not want to give technology a chance’.

A strong connection, i.e. a high frequency of statements’ mentioning, was noted between acceptability and the determinant linked to the recognition of the advantages of adaptive learning. The following quotes supported the cited connection: ‘The acceptance must be improved through the recognition of the added value [of adaptive learning] by everyone: what is the benefit for me personally? Action steps in this regard must be clearly communicated’; ‘I am completely convinced that if an adaptive learning concept is didactically and technically well designed, students, lecturers, and society will recognise that this teaching and learning concept has advantages over other models and will quickly accept it. I also believe that the international visibility of the success of the teaching concept through publications and conferences increases the acceptance and thus the successful implementation. To make all this possible, I think that competent personnel and good working conditions are absolute prerequisites’. This finding indicates that the lack of understanding by the stakeholders about the forms of benefits that they can gain from adaptive learning influence their acceptability of adaptive learning. This finding was relevant for both panels and particularly for the Swiss university panel. Finally, pointing out to this connection, the participants suggested various strategies to influence the relevant determinant for enhancing acceptability. Apart from a didactically sound course design, skilled staff, and favourable working conditions, the most effective strategies suggested were the communication of the advantages and presentation of evidence-based results on the influence of adaptive learning on the success and drop-out rate of students and the satisfaction of stakeholders: ‘The advantages of adaptive learning and that [our university] finds this strategically important must be clearly communicated and, of course, the appropriate resources must be made available’.

Implementation strategies

Implementation strategies refer to various methods and techniques for supporting the implementation of adaptive learning. Qualitative data demonstrated that the suggested implementation strategies were empirically linked to specific determinants (e.g., barriers, facilitators) as well as to corresponding outcomes. In such data-driven connections, each strategy intends to address a corresponding determinant to reach an aspired outcome by overcoming a barrier or leveraging a facilitator. The most frequently proposed strategies were directly or indirectly linked to training, support, communication, and resources, which should be ensured by the university management and implementation team during the implementation process. The following quote was a representative statement of numerous participants that commented on the relevance of various forms of resources: ‘A comprehensive implementation of adaptive learning systems ties up huge resources (personnel, time, and money)’.

Addressing some technological barriers, some proposed strategies targeted, however, not only the scaled implementation but also other distal outcomes, such as equity in education, which indicated that implementing adaptive learning might present implications at the socio-economic and sociocultural levels. One participant of the South African panel commented on ‘the provision of relevant support opportunities [by the university] to students with potential who did not have sufficient opportunity to gain access [to technology]’ to diminish the increasing technological divide between students. Other experts from the same panel suggested that ‘the university management should drive national initiatives towards either providing exceptions on existing data pricing structures or obtain free data for educational use’ or even work with the ‘government’ and ‘private sector’ to address technological challenges to support the scaled implementation of adaptive learning and equity in education.

Finally, the experts reported other strategies to support the acceptability of adaptive learning. One expert noted that ‘people with enthusiasm’ must not only be involved in the project but also be ‘compensated’, which accordingly points to the role of incentives and, thus, financial support during the implementation process. Similarly, other experts stated that ‘the compensation model for lectures must be adjusted’ to the complex course development and ‘additional work must be remunerated for lecturers’ to ‘create’ more acceptability of adaptive learning.

Discussion

An implementation framework of adaptive learning, that is ALIF, provided insights into evidence-based relationships between determinants, implementation strategies and outcomes, which promoted a better understanding of the processes involved in the implementation process of adaptive learning at both universities. In this manner, insights can be gained into the reasons for the slow implementation of adaptive learning and into reflections on the strategies that can support it. This section discusses the results and compares ALIF with previously presented models. The discussion section is organised in terms of the key findings of the study.

Prerequisites and organisational readiness for adaptive learning

One finding of the study was that certain determinants acted as important prerequisites for the successful implementation of adaptive learning. For example, some prerequisites were associated with high costs (proximal outcome) that could result in substantial investments in terms of time, personnel, and funds by the university management. This finding suggests thus that the leaders of the implementation project and university management should evaluate the fundamental factors (prerequisites) at the initial phase of the project to estimate organisational readiness at their institutions to promote the success of the implementation endeavour. If the fundamentals are not in place, then the success of the adaptive project remains questionable, because fundamental challenges may result in large costs and efforts for the implementing institution (Brown et al., 2020; Buchanan et al., 2013). Additionally, Brown et al. (2020) warned implementing institutions against the underestimation of efforts for redesigning the curriculum and individual courses, which is required by adaptive learning. Indeed, the current findings revealed that the transformation of the curriculum and traditional courses is an important prerequisite for the scaled implementation of adaptive learning, which requires huge resources (e.g., time, incentives) and support services for students and instructors, which should be allocated by the university management to the project. However, the study found that several context variables, such as the favourable or unfavourable economic situation of the university, may act as a moderator in the causal connection between the described prerequisites and corresponding strategies (e.g., allocation of financial resources). In contrast to the Swiss university panel, which did not comment on internet issues as a challenge at all, the South African university panel identified internet and infrastructural issues as ‘fundamental’ challenges to the implementation project and recognised the large costs associated with these issues. This finding suggests that the institutional context plays a critical role in the implementation process of adaptive learning and questions the belief that adaptive learning can be a solution to the ‘iron triangle’ (i.e. cost, quality, and access) if a specific context is not considered (Gebhardt, 2018; Tyton Partners, 2016).

Another prerequisite for successful implementation identified by both expert panels was leadership commitment to adaptive learning. The literature broadly discussed the role of leadership commitment (Bailey et al., 2018; Johnson & Zone, 2018; Vignare et al., 2018). Leadership commitment directly affects the efficiency of implementation, as demonstrated by the current results. The reason is that the university management should agree to bear the costs associated with the necessary support in the form of all types of resources, such as for infrastructure improvement, staff training, and technical and didactical services for students and staff. According to Bailey et al. (2018), implementation risks occur when management leaders change or shift their priorities. To strengthen leadership commitment, the participants suggested two main strategies, which is similar to those identified by Vignare et al. (2018). The first is the alignment of the adaptive project with the broad university strategy, and the second is the communication of empirical results to all stakeholders on the influence of adaptive learning on student learning, which should aid in disclosing its benefits over traditional teaching models.

Overall, the current findings demonstrated that organisational readiness for the implementation of adaptive learning is a critical, multi-faceted construct that can be operationalised through (1) the technological readiness of the institution; (2) the individual readiness of students and staff to use adaptive learning systems in teaching and learning practices, and (3) leadership readiness to commit to adaptive learning by integrating it into the broader university strategy and by allocating the necessary resources for its successful implementation. Therefore, evaluating organisational readiness may be critical for the implementation of adaptive learning Although plenty of different instruments have been developed from educational, health services research and other fields to measure organisational readiness (Weiner et al., 2008), many of these instruments have reliability and validity issues with the measures. Therefore, identifying implementation determinants (e.g., barriers and facilitators) is used as an alternative, efficient method for assessing organisational readiness for implementation (Proctor et al., 2012; Vax et al., 2021). The proposed framework with the identified determinants and specific strategies may help institutions to assess organisational readiness by reflecting on the factors germane to the specific context of their institution. Finally, selecting and applying the corresponding strategies from the framework to the identified barriers can help institutions enhance the readiness when it has been found to be insufficient (Vax et al., 2021).

Enhancing acceptability of adaptive learning for scaled implementation

In the second implementation stage, in which institutions begin to pilot adaptive learning, other determinants become relevant for the scaled implementation. For example, the characteristics of the adaptive learning system, such as robustness, usability, and use of learning analytics methods, were considered for scaled implementation from the technological perspective. At the pedagogical level, the determinants were related, for example, to the motivational aspects of students and staff to use adaptive systems as well as their skills and competences.

The current study found that the most frequently mentioned challenge by the Swiss university panel was that neither students nor instructors recognised the advantages of adaptive learning over the traditional blended learning model established in their university. This notion holds despite the fact that adaptive learning has been used at the university in several courses for several years. The importance of recognising the benefits of adaptive learning by all parties involved in the project was also reflected in the implementation model of Johnson and Zone (2018). Thus, the usefulness of adaptive learning was conveyed through faculty training courses by demonstrating the student data available in the system, which instructors can use for immediate interventions in their courses. To enable the recognition of the benefits of adaptive learning, the participants of our study focused in contract on research and communication activities. They reported on the need to conduct ongoing research on the impact of adaptive learning on student learning and suggested the communication of research results and empirically proven benefits of adaptive learning to all stakeholders.

Moreover, the findings confirmed that the usability of the system was empirically connected to the acceptability of adaptive learning as an intermediate outcome of scaled implementation. However, we found that along with technological determinants, pedagogical determinants, such as faculty resistance, high staff workload, and the skills of instructors and students as well as their motivation to use adaptive learning were empirically connected to acceptability in this implementation stage. As adaptive learning refers not only to technology but also to practice (Brown et al., 2020; Shelle et al., 2018), the importance of motivational aspects along with the technological aspects to the implementation process of adaptive learning is unsurprising. In terms of motivational factors, the findings are consistent with motivational theories (Vallerand, 1997), which aim to explain the adoption behaviours of users. However, whether the motivation of the students and instructors to use adaptive learning is extrinsic, intrinsic or both, remains unclear from the qualitative data of the study.

The comprehensive review of the theoretical frameworks and models that inform implementation research in education was beyond the scope of this article, and several valuable syntheses have already been published (e.g., Century & Cassata, 2016; Denis & Lehoux, 2009; Tondeur et al., 2021). However, the two main findings of our research—the role of organisational readiness for adaptive learning implementation and its acceptance by the stakeholders—have spotlighted the multi-facet nature of adaptive learning and shown that researchers need to employ theories from different fields to be able to explain the processes involved in adaptive learning enactment and its successful scale up. On the one hand, educational and organisational change theories may play an important role in describing and explaining the processes of individual and organisational change in the process of implementation. One example is the Concerns-Based Adoption Model by Hall et al. (1973) that focuses on teachers’ individual change during the implementation of educational innovations. It may help researchers better describe, from the teachers’ perspective, how teachers perceive adaptive learning innovations and the extent to which they accept a new adaptive pedagogy in the practice. On the other hand, we should not underestimate the value of the theories that target the diffusion and use of educational innovations that are based on technology. The Technology Acceptance Model (TAM) by Davis (1989) states, for example, that the intention to use technology by individuals is affected by two main factors: perceived usefulness (e.g., the degree to which students and instructors believe that using adaptive learning systems will improve their performance) and perceived each of use (e.g., the perceived effort to use adaptive learning systems). By focusing more on the design of technological innovations, TAM may enable researchers to better understand how technological characteristics of adaptive learning systems, such as robustness, usability, and transparency of algorithms influence the acceptance of adaptive learning by stakeholders.

Implications for theory and practice

One of the biggest challenges of implementation research is the identification of the relationships between relevant determinants, strategies and outcomes to explain the processes involved in the success or failure of implementation (Smith et al., 2020). By identifying the empirical relationships amongst the three elements based on qualitative data from two universities, the proposed framework advances the implementation practices of adaptive learning and the field of implementation science in higher education, which remains very limited (Soicher et al., 2020). This study provides valuable information for implementation researchers and educational leaders who may opt for adaptive learning as a strategic asset for the future development of higher educational institutions. The results demonstrated that a higher educational institution should firstly assess its organisational readiness to formulate initiatives for adaptive learning by identifying the prerequisites and contextual variables along the framework dimensions germane to their individual context and reflect then on specific strategies and intended outcomes (e.g., acceptability, costs, scaled implementation). The current framework may be used by researchers and other stakeholders as a reflection tool, similar to the use of other implementation frameworks (e.g., Damschroder et al., 2009; Smith et al., 2020). However, the future research can focus on the further operationalisation of abstract concepts (e.g., determinants, strategies) with the purpose of measuring adaptive learning implementation more specifically and systematically. The development of the measurement instrument can be informed by previous research (Proctor et al., 2012; Century & Cassata, 2016) and drawn both on qualitative (e.g., interviews) and quantitative (e.g., surveys) methodologies. The operationalisation can begin, for example, with the search in the research literature for existing instruments with items pertaining to the concepts of the framework. Then, the research team should review the items on how they can be adjusted to the context of adaptive learning and, if necessary, create new ones.

Limitations

This study has its limitations. Although the findings of our study are grounded on real data (Charmaz, 2006) collected from responses to open-ended questions, comments, and justifications of experts at the two universities, we do not intend to generalise them to other higher educational institutions. On the one hand, many qualitative researchers are against the term generalisability (Lincoln & Guba, 2000); on their other hand, for those who advocate for the possibility of generalisation in qualitative research, certain conditions must be met (Creswell & Poth, 2018). To render generalisation possible in our case, future research should include additional qualitative data from other institutions that are undertaking other stages of implementation, employ different study models and have diverse socio-economic and cultural contexts. In addition, we remain aware of selection bias. The study referred to field representatives, who should best estimate the expertise of potential participants and propose them for the study (Brady, 2015). Thus, the results and conclusions remain context-dependent due to the theoretical foundations of qualitative research and selection bias. Finally, future studies should determine strategies for evaluating the proposed framework.

Conclusion

Based on the qualitative data of the Delphi study conducted at two institutions of higher education in Switzerland and South Africa, the study proposed an inductively developed implementation framework for adaptive learning. This framework included the critical determinants associated with the successful and unsuccessful implementation of adaptive learning, the specific strategies that address them, and various outcomes, including scaled implementation as the main indicator of successful implementation. The study identified the empirical relationships between determinants, strategies, and outcomes, which enhanced the understanding of the processes, involved in the implementation process of adaptive learning at both institutions and explained the reasons for the slow implementation. One of the reasons was the high costs associated with internet access, required technical infrastructure, improvement of the usability of an adaptive learning system, and the need to redesign the curriculum and courses in accordance with the adaptive pedagogy. We demonstrated that an adaptive learning initiative could also be at risk if leadership commitment is absent and if the staff, students, and university management lack a sufficient recognition of the advantages of adaptive learning. Thus, using the proposed framework could help implementing institutions to assess their organisational readiness for adaptive learning initiative by identifying the prerequisites relevant to their context at the initial phase of the project. It also could help institutes pay attention to critical determinants (e.g., usability, perceived usefulness, motivation), which are responsible for the wide acceptance of adaptive learning by stakeholders. The reason for this notion is that the concept of acceptability serves a central intermediate outcome in term of the manner of the scaled implementation of adaptive learning in higher educational settings. This study advances the implementation practices related to adaptive learning and the research on implementation in higher education. Finally, it increased the awareness of the institutions about the multidimensional and context-specific nature of the implementation of adaptive learning.