1 Introduction

A Learning Management System (LMS) is a web-based innovative educational system which has revolutionised the educational environment. Its ultimate aim is to improve a student’s ability to set educational goals, to organise the content and material being studied and to improve communication and collaboration between teachers and students (Cheng & Yuen, 2018). Although it confers clear benefits and has seen substantial market growth, LMS technology faces several challenges. The main challenge is the variations in the level of its use. Although it aims to improve learners’ ability to plan and set education goals, more than 90% of learners use LMS only as a data repository technology (i.e., for data storage) and for taking online tests. No more than 10% of learners use LMS for online collaboration or forum discussion (Awad et al., 2019). According to Capterra’s research in 2015, the average use of an LMS, unless it makes mandatory online testing requirements, falls to between 19 and 36% of the total number of learners enrolled on a course (Medved, 2017). Capterra reports that the retention rate of an LMS as a supplement to face-to-face learning is 21%, indicating that about four-fifths of the students in his sample do not use LMS at all. Even though governments, universities and service providers have made significant investments in new educational technologies to supplement the use of LMS, the full benefit of the LMS has not yet been realized (Barclay et al., 2018), nor have students yet been as satisfied as expected (Deng et al., 2019).

One of the main challenges facing LMS adoption is students’ failure to use this platform for all its intended purposes, i.e., for organisation and planning (Capterra, 2020). Students’ disengagement can negatively affect the chance that LMS will be widely used by learners and teachers. LMS success is judged by the level of use of and engagement with the LMS (Alraimi et al., 2015). The level of use can be measured by actual use or through the intention to continue using the application in the future (Hong et al., 2017). Thus, to improve learners’ engagement various academic attempts have been made, such as learning analytics (Yilmaz, 2020) and a personalised recommendation system (Yilmaz & Yilmaz, 2020). However, these significant research attempts focused mainly on the current use and the current benefits of the LMS, rather than seeking to explain the variation in the intention to continue using the LMS. The intention to continue use is a more robust measure of the engagement and implies the long-term effects on the learner’s future actions. Thus, to call an intervention meaningful (i.e., WVLO), its effect must be to enhance the intention to continue using the LMS (CIU). Accordingly, to set the research focus, the research question is “How to improve LMS intention to continue use by adding a new addition to it?”.

This research aims to integrate a new element in the LMS, which is, Visualising Weekly Learning Outcomes (VWLO). The main underpinning theory here is the Goal theory. According to this theory (Seijts et al., 2004), the clear visualisation of goals can stimulate people to have goals, and enable them to make plans. Thus, based on goal setting theory, this research proposed and found that developing a visualised weekly learning outcome in the LMS interface could improve the perceptions of the LMS as an educational tool that should be used in the future for improving learners’ cognitive absorption and self-regulation.

The visualised weekly learning outcomes, similar to personalised recommendation system (Yilmaz & Yilmaz, 2020), are seen as a dynamic dashboard in the main interface of the LMS on which to define and visualise the learning outcomes each week. Not only have the usefulness of course learning outcomes and learners’ engagement in general received little scholarly attention, but no research has examined the role of integrating learners’ perceptions of an LMS and the existence of the visualised weekly learning outcome (i.e., the dashboard of the learning outcome) at the LMS interface. Undoubtedly, learners can get used to integrating a fixed visualised learning outcome list at the interface. Thus, this research designed weekly visualised learning outcomes to connect with the topics covered each week and be updated weekly. The effect of integrating Weekly Visualised Learning outcomes (WVLO) on the learners’ perceptions of the LMS has never before been theorised.

2 Literature review

2.1 LMS success

Learning Management Success is defined in this research as the effective use of the LMS which may be translated into its continued use in the future. Various models and frameworks are developed in the literature to define the antecedents of use and antecedents of the intention to continue use. The Unified Theory of Acceptance and Use of Technology (UTAUT) were adopted by Sezer and Yilmaz (2019) to develop a Learning Management System Acceptance Scale which is based on performance expectancy, effort expectancy, facilitating conditions, and social influence. This paper is important because the developed scale was found to be valid and reliable using rigorous analytical methods. However, this paper deduced the construct from the literature aiming to examine the effect on the use of the LMS, rather than the intention to continue using the LMS in the future. Intention to continue use is different from the intention to use because intention to continue use reflects current satisfaction and the positive experience of using the system which induces a future use of the system (Abdullatif & Velázquez-Iturbide, 2020).

Several factors affect the intention to continue to use an LMS. The Technology Acceptance Model (TAM) paved the way to research in this area (Cheng & Yuen, 2018). The conventional models focus on the perceived Ease of Use (PEOU) and Perceived Usefulness (PU) as key explanatory factors of the variation in the CIU of an LMS. For instance, Joo et al. (2017) found in Korean high schools that students’ expectations, perceived enjoyment, and perceived usefulness (PU) were key antecedents to the CIU to use e-books. A similar study, but this time on m-learning, by Al-Emran et al. (2020), of 273 postgraduate students in the United Arab Emirates (UAE), found that perceived ease of use, attitude, perceived behavioural control, and subjective norms were significant predictors of the continued use of m-learning. Recently, the concept of TAM has been used in many papers, but no new perspectives have sufficed to explain the intention to continue using a particular educational technology. Although the ease of its use and the degree of its usefulness could be indicators of using or adopting the technology, they may not be the best for predicting the use of the learning management system (Tawafak et al., 2020). For example, Tawafak et al. (2020), studying 297 learners, used 16 hypotheses to establish that the most powerful predictors of the CIU of an LMS is academic performance, support assessment and student satisfaction with the course.

Antecedents are extended to include psychological factors. For instance, Hong et al. (2017), writing on 150 professional users of LMS in Taiwan, found that these students’ psychological traits and Internet cognitive failures (i.e., distractions caused by the Internet) were learning outcomes closely related to their intention to continue to use the LMS. Other psychological factors were found to be predictors of the CIU of the LMS, for example, social norms, system access and self-efficacy, as inferred by Revythi and Tselios (2019) in their study of 345 university learners. Not only psychological factors, but also technological factors may influence the CIU of the LMS. Yang et al. (2017), studying 294 respondents, revealed that the characteristics of the learning technology platform, such as system quality, course quality, and service quality play a significant role in determining the attitude to the LMS and thus the intention to continue to use it. Similarly in Maqableh et al. (2021), studying 476 learners, found that students’ satisfaction with an LMS and the value of the information released from it positively and significantly influenced the intention to continue. To sum up, PEOU and PU can be significant predictors of the use of a technology (i.e., the usability of the technology) in general; but LMS requires more than these for its regular use.

These papers focus on the role of the PEOU and PU on the LMS use, intention to use and intention to continue use. Indeed, none of the literature focused on the PEOU and PU of one of the addition to the application could improve the intention to use or to continue use the LMS. Indeed, this research will benefit from PEOU and PU as indicators of using the technology to operationalise the level of use of the addition (i.e. WVLO) and examine the effect of this use on the predicted using outcomes from this technology on the CIU of the LMS. This adoption of the PEOU and PU of adding technology (i.e. WVLO) is novel and proposed to play a significant role in explaining the variation in the CIU of the LMS.

2.2 Goal setting theory

This research uses goal setting theory to explain the role of the VWLO on the intention to continue using an LMS. According to goal-setting theory, the existence of a learning goal (i.e., the setting of a goal) and the characteristics of the goal (e.g., its clarity, contribution to ultimate goals, and relevance) affect the level of engagement as learning is conducted (Elliott & Dweck, 1988). According to Seidel et al. (2005), the clarity and coherence of goals lead to enhancing students’ competence in performing the learning tasks. Furthermore, Locke and Latham (2006) have suggested that if learning goals are to improve learners’ performance they require clarity and task specifications. Labarrete (2019) gave an empirical example of this theory in action, finding a relationship between exposing students to learning outcomes and the development of their self-perceptions and engagement levels. R. Labarrete (2019) conducted action research on a set of Philippine learners who used an LMS and found that it improved their educational outcomes to expose them to the required study skills before the course began. In support of this evidence, Foshee et al. (2015) further reported that in 75% of 2880 samples, the perceived competency for a college math subject was improved when students were provided via an online learning platform with the study skills required for a college maths course. However, none of the previous research used this theory to theorise the importance of having VOWL in the LMS. Thus, this research argues that the factors defined in the goal-setting theory could contribute to learners’ engagement and improve their ability to plan their efforts more effectively.

3 Theoretical framework

This theoretical framework of this research is presented in Fig. 1. The WVLO’s perceived characteristics (i.e. perceived ease of use and perceived usefulness) is proposed to affect the engagement level in terms of perceived cognitive absorption (PCA) and learning outcomes in terms of perceived learning self-regulation (PLSR). Increased PCA and PLSR due to the use of the WVLO is proposed to affect the intention to continue using LMS. A detailed explanation and theorisation of these propositions are given in the following sub-sections.

Fig. 1
figure 1

Design framework

Full size image

3.1 Antecedents of the continuation intention of LMS

This research proposes that the main predictors of using an LMS are PLSR and PCA.

The first antecedent proposed here is perceived self-regulation. In the context of this research, the main purpose of having LMS is to regulate and organise learners’ materials to support learners’ studies (Alkhasawnh & Alqahtani, 2019; You, 2016). Thus, the perceived benefits of having LMS are operationalised in the PLSR. PLSR is conceived as an active, constructive process that is made possible by technology, in which learners perceive they can set their learning goals and are enabled to monitor, regulate, and control their cognition and behaviour (Liaw & Huang, 2013). Therefore, unlike the term ‘learning self-regulation (LSR)’ which is confined to personal attributes, PLSR is operationalised as the perception of a technology as a self-regulated individual learning tool and as an enabler in regulating the content of e-learning (Liaw & Huang, 2013).

According to perception theory, perception is subjective and based on contextual factors, i.e. people’s perceptions and their interpretations are based on their experience, knowledge, background, and self-rating (Yammarino & Atwater, 1993). People’s perception of themselves may also have a significant effect on their intentions (Maselli & Altrocchi, 1969). In other words, if a system improves one’s perception of one’s capabilities and performance, it will strengthen one’s intention to keep using the system. There is academic evidence that PLSR and CIU are positively associated. The relationship between PLSR and CIU was studied in Concannon et al. (2018) and Bernardo et al. (2019). Bernardo et al. (2019) found that integrating self-regulation strategies in the educational process increases the intention to persist in the course. Similarly, Concannon et al. (2018), in their phenomenological study, found that courses that are designed to improve learners’ perceived self-regulation strengthen their persistence on the course and their intention to take similar courses in the future. Self-regulation is formed not only from learning outcomes but also from perception and experience, built up by the design of the course and the interfaces and characteristics of the LMS. No research has directly examined the effect of the PLSR from LMS on the CIU; however, by deduction from other literature, this research proposes that when learners perceive that the PLSR improves with the use of the LMS, their CIU will be improved.

  • H1: PSRL positively affects CIU

The second proposed antecedent of the CIU is the PCA. PCA is explained by the state of flow theory, first promulgated by Agarwal and Karahanna (2000). According to Getzels and Csikszentmihalyi (2017), the state of flow is reflected by the realisation of each performed action, the focusing of attention, neglect of self-mindfulness, a complete sense of autonomy, consistency requirements, and knowing the purpose of each activity required and performed. In such situations, PCA is useful and leads to positive outcomes; however, it must be subject to being directed to purposeful activities.

PCA is found to influence the continued use of social media in different contexts: for instance, a recent study by Hsu and Lin (2017) of 310 Facebook users shows that the CIU is affected by PCA. Using the same analogy in LMS, if students are fully engaged by the use of a learning technology, their engagement improves their attitude to the application of LMS, hence strengthening their intention to use it effectively. This is also shown by Moreno et al. (2016) in their examination of 251 students in Brazil. Venter and Swart (2018) in their study of an MS Office simulation of an interactive learning application further found that PCA directly affects the CIU. In the educational technology literature, Roca (2008) studied 172 university learners and found that cognitive absorption significantly improves the continued intention to use LMS. Another paper, by Léger et al. (2014), studied 36 students, and concluded that cognitive absorption has a significant effect on the learning outcomes. Therefore, scholars have found in e-learning systems a close association between the PCA and the intention to continue use, as in the study by Venter and Swart (2018) on a computer science undergraduate programme in South Africa on learning simulation modelling and as in the study of 730 undergraduate students in Malaysia who used an e-learning platform (Salimon et al., 2021). On the same lines, the present study proposes that improving the learner’s PCA will accordingly intensify the CIU.

  • H2: PCA positively affects the CIU

An LMS is an application that aims to improve learners’ perception of their personal ability to organise, structure and achieve learning objectives. Thus, using the deductive approach the concepts of usefulness and perceived benefits are operationalised into PLSR. It was shown in Basol and Balgalmis (2016) and Karlinsky-Shichor and Zviran (2015) that the effective use of a software application improves its perceived benefits, i.e. the PLSR. Since the PCA can improve the effective use of applications (Moreno et al., 2016). Similarly, Salimon et al. (2021) found that the PCA affects the perception of usefulness of the e-learning platform. Taking this approach to deduction, it can be extrapolated that the PCA from using LMS positively affects the PLSR from it.

  • H3: PCA positively affects PLSR

The interaction between PLSR and PCA is important in explaining the intention to use a technology (Lines et al., 2015). The PCA does not always affect the CIU because it can sometimes be perceived negatively. For instance, learners who had higher PCA in using Facebook were found to have a lower level of academic achievement (Rouis et al., 2011). At the same time, satisfaction from using the technology mediates the relationship between the PCA and CIU, as Jumaan et al. (2020) found in the research on the use of the internet by mobiles. Similarly, Lin (2009) found that perceived usefulness mediates the relationship between PCA and the CIU the technology of the virtual community. No previous research has examined the role of PLSR as a mediator in explaining the relationship between PCA and CIU in e-learning systems. Therefore, by deduction from these papers, PCA needs to be directed to the purposeful use of applications; otherwise, it could lead to poor results. Since the ultimate purpose of the application is to regulate one’s own learning activities, the present study proposes PLSR as a mediator between PCA and CIU.

  • H4: PCA positively affects the CIU mediated by PLSR

3.1.1 The effects of the WVLO

WVLO is proposed to affect the PCA and PLSR of LMS. Regarding the PCA, goal theory holds that the clarity of the requirements and objectives is a significant motivator of performance (Seijts et al., 2004). In other words, clear objectives can stimulate students’ interest and engagement via a structured roadmap of the learning outcomes. Such a roadmap would show the ways to acquire engagement as a skill or as a learning outcome (Kizilcec et al., 2017). Graphic representations of learning outcomes constitute a form of goal setting that can guide student’s week by week. The primary underlying justification here is that when students know what is required and feel that they are being guided, distortions are reduced; thus, their deep engagement can be improved, i.e., they are in a state of cognitive absorption. Seeing these representations of their learning outcomes every week positively affects PCA.

  • H5: Usability of visualized competency affects PCA positively.

  • H5a: PEOU of WVLO affects PCA positively.

  • H5b: PU of WVLO affects PCA positively.

  • H6: PWVLO usability affects CIU mediated by the positive effects of PCA

  • H6a: PEOU of WVLO affects CIU positively mediated by the positive effects of PCA

  • H6b: PU of WVLO affects CIU positively mediated by the positive effects of PCA

In the study by Liaw and Huang (2013) of the perceived satisfaction of 196 university students, perceived usefulness and interactive learning environments were shown to positively influence PLSR in an e-learning environment. In this research, the ease of use and usefulness of the WVLO are proposed to support learners’ core PLSR. The justification here is that the use of WVLO will provide structured instructions (Harden, 2002) based on the weekly required learning outcomes and topic-relevant cross-curricular skills of the course. In LMS, learners may not always be aware of the best strategies to adopt, or the right steps to take next to attain their set goals. Therefore, they need external guidance and information (Brooks et al., 2014). Once the students feel that they are in control of their learning and are equipped with the tools that enable them to engage and develop deeper learning, they will perceive that they have improved their planning and managing skills for learning and completing their learning tasks (Nguyen et al., 2018). Similar to the finding of Yilmaz and Yilmaz (2020) that Personalized Recommendation and Feedback Based on Learning Analytics, which is updatable and can guide the learners week by week, can improve learners self-regulation, this research applies the argument to WVLO. WVLO is proposed as a mechanism to improve students perceived self-regulation because it could improve the determination of goals, which then improves the perception of having control over their goal-seeking behaviour.

  • H7: WVLO usability affects PLSR positively.

  • H7a: PEOU of WVLO affects PLSR positively.

  • H7b: PU of WVLO affects PLSR positively.

  • H8: Perceived usability of WVLO affects CIU positively mediated by the positive effects of PLSR

  • H8a: PEOU of WVLO affects CIU positively mediated by the positive effects of PLSR

  • H8b: PU of WVLO affects CIU positively mediated by the positive effects of PLSR

According to self-regulation theory, the learners would feel that they were organized and well-prepared if they knew their learning goals. Since these are updated every week to reflect what the students should primarily be looking for, this presentation of competency could offer primary guidance to students. Such guidance informs the sense of control (i.e., the student can master the visualized skills), and flow (i.e., the students will follow the instructions in due order because they promise to culminate in acquiring the desired skills). Therefore, it is proposed here that the usability of the WVLO could improve users’ perception of the benefits of using LMS (i.e., PLSR) which in turn could improve the intention to keep using it in the future. Since the WVLO’s usability is proposed to affect the PLSR, and PLSR was also proposed to affect the CIU, the present study proposes that the usability of the WVLO affects the CIU mediated by PLSR.

  • H9: WVLO usability affects PLSR mediated by PCA

  • H9a: PEOU of WVLO affects PLSR positively mediated by the positive effects of PCA

  • H9b: PU of WVLO affects PLSR positively mediated by the positive effects of PCA

  • H10: WVLO usability affects CIU mediated by PCA & PLSR

  • H10a: PEOU of WVLO affects CIU positively mediated by the positive effects of PCA & PLSR

  • H10b: PU of WVLO affects CIU positively mediated by the positive effects of PCA & PLSR

4 Research methodology

This research adopted Case study approach (Yin, 2012) using mixed methods (Brown et al., 2017) translated into design science research (Gregor & Hevner, 2013). The case study is a computer science module of the Computer Science course in the first year of undergraduate programme at Westminster International University in Tashkent. There were several reasons for selecting this case. The first was that the teachers and students agreed to take part in this study. Second, the selected course was in the computer science department, unlike courses on technology literacy. Moreover, this course had one of the highest enrolments (400) for face-to-face teaching and LMS and 203 students registered to use the LMS. The course selected for study was in the Computer Science department, which pre-supposed IT literacy.

Adopting the design science research (Gregor & Hevner, 2013), the research will design a solution, then validating this solution and examining the effectiveness of it. Accordingly, this research had three phases: developing WVLO (five workshops with teachers), validating the design with teachers (four workshops) and ensuring students are using it (data interaction analysis), and examining the effect on students (Questionnaire).

4.1 First phase: developing the WVLO

This research adopted the participatory approach of setting up workshops for teachers to establish the defined and decomposed module learning outcomes into weekly instalments and associate them with the topics covered each week. Ten teachers were involved in the process (all of whom were teaching this module), which took five workshops and was followed up with Telegram. Telegram is the most often used social media platform in this context, and is used to follow up conversations, and capturing new ideas from teachers once they find them; they can be discussed virtually until face-to-face workshops discussions replace them. Each workshop is held once per week, giving five weeks in total for developing a solution. The researchers developing their solution in parallel with these workshops. The researchers took an active role in formulating the WVLO and validating the learning outcomes with the teachers. Finally, by the end of the fifth workshops, researchers designed the WVLO and integrated them in the LMS and ready for validation.

4.2 Second phase: validating the intervention

Validation conducted on two levels. The first level was with teachers before introducing the module while the second was in the module delivery time to assess the level of usage of the LMS. Thus, there were four workshops to validate the design and content with teachers. Based on action research approach, the feedback from each workshop is embedded in the design until all ten teachers were satisfied and endorsed the use of it in their module. To validate the student use of the WVLO during the semester time, the data interaction analysis was conducted.

4.3 Third phase: testing the impacts

Two weeks before the end of the semester, a survey was sent to the learners to examine the effects of the usability of the WVLO (as a proxy for the use made of them) on their perceptions of the LMS. The examination of the impact was avoided to be in the beginning of the year as students may feel anxious of the new intervention (i.e. WVLO) and avoided to be in the last week to avoid the exams and assignment stress. Thus, two weeks before the end of the semester was chosen for circulating the questionnaire.

The valid response was 151 out 203 registered students in the modules, showing a response rate of 74.4%. All of the participants were the same age (16) and in their first year at the college; 64% of the class were males and 36% were females. Almost the same ratio was found in the sample collected the (60.3% men and 39.7% women).

4.4 Questionnaire design

The questionnaire constructs are adopted from the literature and listed in Table 1. The CIU is a five-item construct borrowed from Abdullatif and Velázquez-Iturbide (2020) and Alraimi et al. (2015). PCA, perceived usefulness and perceived ease of use were concepts adopted from Roca et al (2006). PLSR was adapted from Basol and Balgalmis (2016) and Liaw and Huang (2013).

Table 1 Fornell Larker criterion test
Full size table

This research adopted four methods to ensure the validity of the constructs. They were the AVE test, the Fornell Larker test, HTMT, and Confirmatory Factor Analysis (i.e. the significance of the factor loading and significance of the model). All the reported AVEs were accepted because they were above 0.5. The Fornell Larker test, as in Table 1, presents the correlation matrix of the constructs. According to this test, the square root of AVE should be greater than the highest correlation in the matrix. In fact, the highest correlation was r = 0.766, p < 0.05, and the lowest square root of AVE was .785, which confirmed the existence of discriminant validity. This research also checked the Heterotrait–Monotrait ratio of correlations (HTMT), as recommended by Henseler et al. (2014). It was found that all the values were below 0.90, as shown in Table 2. This means that this study satisfied the criteria for discriminant validity. A confirmatory factor analysis test was conducted (see Table 3 for the standardised factor loadings). All the factor loadings were significant and above 0.6. The model showed a good fit, since \({\chi }^{2}/df\) =1.38 (below the threshold of 2) and the RMSEA of .050 (below the threshold of 1); both figures were accepted. All the constructs were reliable since the Cronbach alpha, Composite reliability, and Rho_A for all constructs were above 0.8 when the accepted threshold need be no more than 0.6.

Table 2 Heterotrait–Monotrait ratio of correlations (HTMT)
Full size table
Table 3 Research constructs’ factor loading and reliability tests
Full size table

4.5 Common method bias

Two approaches were taken to testing for the possibility of systematic error by checking for common method bias. They were the Variance Inflation Factors and Harman’s single factor test. In the first method, as advised by Kock (2015), all the independent and dependent variables were tested and found to be < 3.3, as shown in Table 4. For the second measure, Harman’s single factor test, dimension reduction, was used, and the extraction sum of squared loading had to be < 50%. In this research it was 49.0%, which also indicated no issue with common method bias.

Table 4 VIF table
Full size table

5 Solution development

The technical implementation of Visualised Competency was in three stages. The researcher used a revised version of Bloom’s taxonomy for guiding the process (Barari et al., 2020; Krathwohl, 2002). The first stage was to construct the WVLO framework builder. The outcomes of this stage were the learning outcomes and cross-curriculum learning outcomes that were agreed for the courses under review. This form allowed the researcher to lay out each parent competency along with its sub-rooted skills in preparation for being visualised according to their agreed time frame and matching learning content. The competency builder was used in the selector form to allow her to choose which competency to illustrate. The second stage developed the illustrated competency selector form. This form allowed the learning outcomes that were chosen for visual presentation on the course page to be managed. The third stage implemented the illustration of the selected competency on the course page.

The outcome of this stage displayed the expandable hyperlink tree of the selected competency framework, which allowed students to navigate through it and read the content of each sub-root of the illustrated competency tree. The image below shows an example of the learning outcomes in graphic form. The students can navigate through the branched text-structure, as shown on the left-hand side of the image. When a student clicks on any node (such as a title or subtitle of the competency), the description of the clicked node (on the right-hand side) is displayed. The figure below, Fig. 2, shows the view available to the administrator, and instructor; hence, the click counters are viewable.

Fig. 2
figure 2

Visualised competency

Full size image

6 Validation

The use of the visualised weekly learning outcomes was validated by means of interaction analysis. The interaction analysis involved the use of click counters and heat maps. The interaction analysis relied upon the heat map to gauge the learners’ interactions with the WVLO.

The WVLO features account for approximately 1% of all the clicks on the platform of the learning management system. This value was significant and constant because the percentage was stable and steady. Moreover, it was not expected that these features would be used every time the LMS was in play. The number of clicks became steady and relatively fixed at around 47 clicks per week for competency (Table 5). Then the clicks declined by around 10 per week for competency (R2 = 60.85%, P < 0.00). This was because the WVLO was a planning tool used most often at the beginning of a semester, a week, and an activity. The competency criterion is a planning tool determined by other factors, such as the difficulty and depth of the topic, together with the type of learning outcomes required.

Table 5 Click counter records of WVLO over the academic term
Full size table

The above findings may have been exaggerated by the frequent use made of these tools by a few students. To improve the evaluation of the intervention, the number of effective users was assessed, i.e., the effective users’ measure was adopted, in order to ensure that a significant number of learners used these technological interventions. ‘Effective users’ here denotes those who watched more than half of the learning videos on Moodle. The Moodle activity logs yielded the number of effective users. Estimates of the use of WVLO were based on the number of effective users; for example, as regards Table 6, 31 students watched more than two videos, though only eight watched all four videos. Conversely, WVLO earned around 6 clicks per week for those who watched all four videos and around 1.5 clicks for competency. Similarly, the heat map data validated the actual use of the tools, as reflected in the importance and value attached to them.

Table 6 Moodle log files analysis of effective users (left table) and effective students in relation to click counters (right table)
Full size table

7 Testing hypotheses

There were three main relationships to consider here: the PCA and PLSR of the LMS on the CIU, the WVLO usability on the PCA and PLSR of CIU, and the relationship between WVLO usability and the LMS CIU. The findings are shown in graphic form in Fig. 3 and detailed below.

Fig. 3
figure 3

Weekly visualised competency model

Full size image

As with the WVLO model, the LMS PCA and PLSR, in this model, show in Table 7 that both constructs had significant positive effects on the learners’ CIU for the LMS. PCA had a direct effect (.299, P < 0.01), confirming H2, an indirect effect through PLSR (.181, P < 0.05) confirming H4, and a total effect (.481, P < 0.01). PCA affected the PLSR with positive significance (.495, P < 0.01), confirming H3. PLSR had a significant positive effect on the CIU (.366, P < 0.01), confirming H1.

Table 7 WVLO model—effect of LMS PCA and PLSR on CIU
Full size table

WVLO usability affected the LMS features (i.e., PCA and PLSR). WVLO usability had a significant positive effect on the PCA (0.313, P < 0.01, 0.308, P < 0.01 for PEOU and PU). Similarly, WVLO usability had significant positive direct effects on the PLSR (0.218, P < 0.05) for PEOU but not for PU. The lower effect of 0.16, P > 0.05 is illustrated. The perceived usability showed a significant mediated relationship between PCA and PLSR, (0.155, P < 0.01) and (0.153, P < 0.01) for PEOU and PU, respectively. The total effects of PEOU and PU were significant (.373 and .312, P < 0.01).

WVLO usability had total, direct, and indirect effects. PEOU did not demonstrate a significant total effect on the CIU (.155, P > 0.1) but the PU had a significant positive total effect (.489, P < 0.01). Among the direct effects, PU, but not PEOU (− .076, P > 0.1), had a significant effect on the CIU (.282, P < 0.01). The total indirect effects were significantly positive for both PEOU and PU (.23, P < 0.01 and .206, P < 0.01) but the mediators were not all significant.

The proposed mediators were PCA, PLSR and PCA → PLSR. The first mediator, PCA, played a significant role for PEOU (.094, P < .05). It validated H6a but the role for the PU did not (.092, P > .05), causing H6b to be rejected. The second pair of mediators were PLSR and PU. The PLSR for PEOU showed 0.08, P = 0.05, but the PU showed 0.058, P > 0.05. The third mediator for the relationship between PEOU and CIU was PCA → PLSR did not have this significance (.057, P > 0.05). The same result is seen for its PU (0.056, P > 0.05).

8 Discussion

The literature records many attempts to improve learners’ engagement and cognitive presence on the LMS platform (Yilmaz, 2020). This research adds to the literature library a new tool for improving the student intention to continue using the LMS. The WVLO presented a structural breakdown of the module and relevant learning outcomes and/or weekly planned learning outcomes. This research theorised, examined, developed, and validated the role of integrating WVLO in the LMS to improve the perceptions and intentions regarding the LMS. This educational intervention aimed to show the weekly learning goal and its strategies to the student. This research model found that the Perceived Ease of Use (PEOU) and Perceived Usefulness (PU) of the WVLO played significant roles in explaining the variations in the LMS CIU, taking into account the improvement in the learners’ perceived self-regulation and PCA towards the LMS.

The first proposition concerned the relationship between WVLO and PCA. In adopting goal theory, the clarity of the goals improved the students’ engagement with the tasks, which improved the learners’ academic performance (Schippers et al., 2020). By analogy, this research proposed, and found, that WVLO improved the PCA. This indicated that exposing the detailed weekly basis of learning outcomes to students improved their perception of the clarity of goals and significantly improved students’ engagement. This research complemented the work of other researchers such as Labarrete (2019), who was able to improve learners’ commitment to their learning by exposing them to the required study skills before they began their study.

The second proposition referred to the relationship between WVLO and PLSR. The visualisation of the learning outcomes was proposed to improve the perceived self-regulation because it could improve goal determination, which in turn influenced help-seeking behaviour. Thus, if the learning outcomes required for each week were presented to students, this goal determination, i.e. the time frame and the mechanism to deliver it, was enough for learners to perceive that they could control and plan their study effectively. WVLO helped by improving their goal-seeking behaviour, which was a central dimension of learning self-regulation. This research confirmed this explanation by finding significant objective and quantitative evidence.

This research refined the result in Lee et al. (2016), that presenting learning outcomes improved the PLSR, by showing that improved determination of the goals could be the primary driver of the PLSR. In addition, instead of presenting the learning outcomes before the course, as Lee et al. (2016) did, the present research in PEOU and PU used scales to measure the level of the WVLO. The perceived usability of WVLO was shown to have a heavy impact on students’ PLSR, mediated by an improved PCA. As predicted by self-regulation theory, learners felt more organised and better prepared when they knew their goals clearly. Accordingly, the visualised competency improved the students’ PLSR, supported by an improved PCA. WVLO improved the students’ perception of control and flow, and hence it improved their PCA, which led them to use the LMS more effectively for organising their tasks and planning.

The third proposition was the relationship between WVLO and CIU. Based on the role of WVLO in improving the PCA and PLSR, this research found that the relationship was complex. No direct relationship with the WVLO on the CIU, without the effect of the PCA and PLSR, was clear. This research could not find evidence to support the proposed effect of the PEOU of WVLO on CIU. However, the PU of visualised competency showed a positive impact on students’ CIU, which confirmed H11b. This result indicated that the PU of the WVLO was an essential factor for driving the CIU of the LMS, whereas the PEOU of the WVLO was not a key factor.

The mediators here played an essential explanatory role in understanding the relationship between WVLO and CIU. Regarding the mediating effect of the PCA, H5a was confirmed. It proposed that the PEOU of the WVLO affected CIU mediated by PCA. In contrast, H5b proposed that the PU of the WVLO affected CIU mediated by PCA, but it was not found valid. This indicated that the PEOU could be useful only if the PCA improved while the PU of the WVLO did not need this mediator for improving the LMS CIU. It seemed that the PEOU could trigger the PCA of the LMS, which in turn improved the CIU. The mediating effect of the PLSR, both the PEOU and PU of the WVLO, was found on the LMS CIU. However, the mediating effect of PCA → PLSR was not significant for setting the relationship between the PU and PEOU and the LMS CIU. This confirmed the absolute importance of PLSR as a mediator, together with the mixed role played by the PCA in understanding the relationship between WVLO and LMC CIU.

The definition of success in this paper is divided into success of the intervention, success of the LMS and the outcome of the success. This research defined success of the intervention by the PEOU and PU of the application which is derived from the TRA. The justification for not selecting UTAUT or other acceptance models is that the intervention in itself is meaningless unless it is added to the LMS. Therefore, the measurement of the benefits, or performance expectations (Sezer & Yilmaz, 2019) is not tracked directly to the WVLO; but should be traced to the LMS with WVLO. Second, in defining success with the LMS, unlike other researchers who divided it into aspects like “benefits” in general (Ramírez-Correa et al., 2017), we operationalised the benefits into the self-regulation perception, since the main justification for having the LMS is to regulate and organise the material and communication for learners. Last, instead of having the intention to use as an indication of successful LMS (Mohammadi, 2015; Nurakun Kyzy et al., 2018), this research used the intention to continue use to reflect sustained use in the future instead of one single successful use (Abdullatif & Velázquez-Iturbide, 2020). While a single successful use could be influenced by the teacher or topic in this module, the intention to continue use will instead entail more reflection on the use of the LMS in the future.

9 Contribution

This research proposed a novel framework to develop and implement a new educational technology to improve the perceptions of the LMS so as to strengthen the intention to continue using it. This novel framework adopted goal setting theory to guide the researcher to develop weekly learning objectives and be on the front page of the module every week.

In the current literature, using visualised learning outcomes only once per semester made it difficult for learners to find and use them. Goal seeking theory dictates that a new intervention should be visible, relevant and available, and tasks should be broken down to fit smaller units than a semester and set in a shorter time frame. Thus, the new WVLO was designed to appear once per week so that learners could see the relationship between the topics covered week by week (goal-setting) and all the topics to be studied. By setting this mental connection, learners’ consequent perceptions of being self-regulated and the cognitive absorption from the LMS were proposed and found to be significantly increased.

10 Research implications

These research findings and results yielded several research implications. They relate to such stakeholders as developers, teachers and academics.

10.1 Developers

Traditionally, the reports and information were presented to the students on the learning dashboard and not on the main interface. This research argues that exposing students to the WVLO in their day-to-day routine learning activities improves their PCA and PLSR, affecting the CIU. Thus, the present research recommends that developers break down the dashboard elements and allocate these analytic tools to the interfaces related to day-to-day use. This would nudge learners and teachers to use the relevant analytic tools in taking decisions based on relevant information. Besides, reporting data and constantly exposing the learning goals to the students could have a positive psychological effect on their thinking and behaviour.

The weekly visualised planner integrated with the WVLO evaluation system improved the evidence-based system and provided a method of feedback which strengthened future learning outcomes. First, evidence was found that teachers benefited from the WVLO planner tool developed by the research. The current learning outcome planner tool focused mainly on presenting the learning outcomes for a whole semester, whereas this research planner tool provided the number of learning outcomes to master each week and presented them to the students. This enabled the teacher to see the number of clicks on each competency and the transferable skills that were gained. The research found that weekly learning outcomes made students more focused and more determined to follow their learning goals now that these were clearer. This engaged them, making them more capable of planning their activities.

The planner currently in use did not store learning outcomes and did not connect with the literature about compiling a comprehensive list of transferable skills. The researcher had to conduct her own systematic review to discover what the transferable skills for the teachers were. The teachers judged this to be a useful way of improving their planning ability. Thus, a system imported from the literature for teaching planners’ transferable skills was recommended. Third, to guard against the chance of miswriting or misunderstanding the use of specific transferable skills, it was recommended to ensure clarity by setting up evaluation technology that would recommend the best wording for the learning outcomes and the transferable skills. Fourth, planners would benefit from the click counter feeding into its interface to report on the number of clicks on each learning outcome and would implement a system to receive students' feedback on the comprehensibility of the learning outcomes and the transferable skills.

10.2 Teachers

The main research implications for teachers were the focus on awareness and the adoption of new practices. Regarding awareness, during the interviews, it was noticed that the teachers’ perception of the importance of having learning outcomes was becoming less keen. However, when they engaged in the development process, they found it to be useful for improving their module design. Thus, the first recommendation for teachers was to be aware of the importance of breaking down the learning outcomes into weekly instalments. Without such awareness, the usefulness of having the WVLO planners would be undermined. The technology had no value unless it was used.

Regarding the planning of the module, the module was designed to provide a certain number of learning outcomes per semester. The design was optimised for the topic and not for the learning outcomes. However, when teachers considered the planned learning outcomes each week, using the revised Bloom’s Taxonomy, it enabled them to design programmes based on the learning outcomes and not on the topics. Last but not least, teachers had to ensure that the learning outcomes were concise, clear and consciously determined, since the present research found that these requirements crucially ensured the level of engagement and improved learners’ ability to be self-regulated by means of LMS. This could be done by interviewing the students or using the social media analytic tools to revise the learning outcomes that were too debatable and/or unclear.

11 Research limitations

11.1 Generalisability of the findings

The nature of this case study meant that the generalisability of the results was limited to this case. However, it could be applicable in a different context with similar environmental conditions in terms of module design, teachers, students, and university context. The research boundaries here were limited to Moodle as the LMS, Tashkent as the country and culture, and the IT course module as the educational material. Moodle can be different on other platforms that may engage students differently. For instance, compared to Moodle, Blackboard is limited in its features and engagement tools. Moreover, some new LMS integrate advanced dashboard systems with plenty of information. This research focused on Moodle as an LMS with limited options on the dashboard but with high interactive features.

Another limitation for the generalisability of the findings that the students were selected from the computer science module which could indicate higher-than-average computer skills. That is, many students of literature or history do not use technology to the same extent and this may lead to a lower level of use of the VWOL or to unexpected findings.

11.2 Timing and frequency of data collection

Thus, the present research had one opportunity to run at the end of the semester. To begin with, there was a potential surprise element in starting a technology. At the absolute end, there was a time issue which risked diluting the Perceived Ease of Use (PEOU) through possible over-familiarity with the features. The third area of improvement for the survey lay in including demographic questions so as to assess the role of gender, age, background, and education, together with other country-specific factors. Despite the importance of having these factors in the study, the potential cost of having them could outweigh their benefits. Indeed, the class has similar students in terms of age, background, education and from more or less the same culture. The data had to come from one topic, one class, one teacher, and one intake. The variation in the demographic factors was thus limited; hence replicated studies should ensure a wider and more diverse group of participants to test inclusivity and, in turn, equality. The students might see this as a way of identifying them or asking what could be seen as personal questions. Moreover, the questionnaire was designed to be kept short and straightforward so as to merit a high response rate.

12 Measurement limitations

The measurement of the scales in the positivist questionnaire led to some areas for improvement. First, the PLSR was measured using a self-rated questionnaire, as commonly used in the literature. However, it could also be measured by the learners’ interactions on the computer (Çebi & Güyer, 2020). Both approaches were used in the literature, but the possibility of triangulating the data may have improved the value of the work. Nevertheless, the cost and effort of integrating the algorithms of the data interactions and seeking approval from the students to connect these data with their questionnaire presented a key challenge limiting the researcher’s ability to consider this option. Second, the PLSR was a perception and not necessarily real; i.e. this research focused on self-perception more than objective practices. Nevertheless, this research adopted rigorous validity and reliability tests to ensure that the scales that were used were helpful and not misleading.

Regarding the PCA, in the literature, some authors used 4-dimensional scales to measure the PCA, each of the dimensions having 4 items. This could be a valid point if the constructs were few in number. Nevertheless, if the present research had included 16 items for measuring the PCA the questionnaire would have been lengthy and the response rate low. The last point to include as a possible area for improvement was the ability to measure the use behaviour of the respondents regarding the interventions. Because there was no possible linkage between the data interaction information and the questionnaire, the only option available was to measure this behaviour through self-rated items. Questions about the behaviour could be raised or inflated but using second-order questions for measuring this behaviour would have given more valid responses. In other words, the questionnaire could have asked direct questions about use level, but this could have elicited misleading answers since the use level was relative and not easily measurable, whereas the assessing the perception of ease of use and usefulness using the Likert scale could give stronger and comparable figures.

Limitations regarding the statistical outcomes underlie the use of the Partial Least Square Method (PLS) as a structure equation modelling (SEM) technique. Rönkkö et al. (2016) give as examples the inability to define the model’s significance and its fitness level, the possibility of having inconsistent and biased estimations, and capitalisation on chance due to imposing certain parameters in the equation, not in the exploratory factor analysis. Thus, this research adopted Covariance based SEM to examine the model fitness and ensure the constructs were well suited, valid and reliable. Since Rönkkö et al. (2016) argue that the results of using Covariance based SEM may deviate, due to testing the results and not examining the validity and reliability of the constructs, this research proposes a replication of the findings by using the covariance based SEM to compare the findings with the PLS-SEM.

13 Future research

The learners’ performance and the relationship between visualising the weekly learning outcomes and learners’ academic performance should be studied before accepting the assumptions as unquestionable. Allocating learning outcomes per semester could improve performance because learners know much about the requirements of the semester, but they need to take into consideration previous exam papers, to improve their academic “scores”. This premise is believed to be correct, but, for the present research, having VWLO was as important as reinforcing the importance of the topics and making learners believe in them more and more. However, this could affect their ability to follow the teachers’ instructions and be guided by the past exam papers. The exam levels may cover only the “remember” aspects, while learning outcomes focus only on “application” practices. In other words, while students may focus on applying these lessons in their practices, this may not necessarily lead to their improving their academic performance. This relationship needs further study and investigation to ensure a positive association between having WVLOs and academic performance.

14 Conclusion

This research started by asking “How to improve LMS intention to continue use by adding a new element to it?” this research and found that WVLO, if perceived to be easy to use and useful, will improve learners’ cognitive absorption (i.e. their high level of engagement) from using the LMS and this engagement will lead to realisation of the main benefits from using the LMS (i.e. the perception of self-regulation). According to the technology acceptance framework, the perception of the benefits is a leading indicator of use and sustainable use, which is confirmed in this research. This research is novel in developing this framework; it could help in assessing the success of the new interventions (i.e. PEOU and PU), and the success of the LMS (i.e. the PCA and PLSR) which in turn could influence the intention to continue using the LMS in the future (i.e. the outcome of this success).