Keywords

1 Introduction

Online education requires learners to self-regulate their work, yet many of them are poor at self-regulated learning (SRL) [1]. SRL refers to the extent to which learners are active participants of their own learning progress by means of monitoring and controlling their motivation, metacognition, cognition, and behavior towards achieving their learning goals [2]. Research has shown that engaging in strategies related to SRL, such as goal settings and visualization of the student learning progress towards the achievement of the course learning goals, positively affects academic success in online learning settings [3, 4]. However, current SRL support aiming at assisting students in their progression towards the course goals has so far been limited [5]. The present demo study aims to fill this gap by presenting a learning analytics dashboard (LAD) aimed at assisting students in visualising their learning progress in relation to the achievement of the course learning objectives. Achievement of goals is one of the crucial determinant of students’ learning progress and has impact on academic outcomes [9].

Previous research stresses the importance of designing quality courses for students’ learning success and satisfaction [6]. One of the critical course learning design practices is articulating easily comprehensible learning objectives and their sequence to obtain the desired learning outcomes [7]. Learning objectives in a course refer to specific skills, knowledge and dispositions that students are expected to acquire or demonstrate at the end of the course [7, 8].

Today, higher education institutions are increasingly adopting digital technologies such as learning management systems (LMSs). Students using these systems produce activity logs which can be used to track and visualise their study efforts as a progression towards the achievement of course learning objectives.

Deployment of analytics technologies is becoming increasingly important in higher education. According to Susnjak et al. [10], analytics in this sector focuses on maximizing student retention rates by identifying at-risk individuals early on, and initiating interventions whilst improving the quality of the overall educational experience. One of such analytics innovation tools are LADs, which aim to provide learners with a “snapshot of how they are progressing in their courses" [10]. Overall, analytics tools such as LADs reveal learner insights that could prompt a reflection process that would not be realised in other way. In this study, we draw on the advances in visualisation technologies and learning analytics to present how a student-facing LAD, aiming to raise student awareness of how they progress toward the achievement of course learning objectives, could be designed and implemented in the setting of engineering higher education.

2 Case Study Settings

We use a second-cycle course on Information Visualization as a case study. The course gives six higher education credits, roughly mapping to 160 h of work, including attending lectures, reading, programming, presenting, testing, developing group projects, critiquing, discussing, and writing journals. For all the course learning activities, Canvas (the LMS) is used. Approximately 50 students register for the course every year. The course participants meet face-to-face in a visualization studio over the course of 16 two-hour meetings, half of which are lectures and the other half are student presentations and discussions. Of particular interest are the learning journals the students need to keep. The focus of the self-reflection learning journal are the individual intended learning outcomes (ILOs) of the students. The students submit three journal entries, one after each group project deliverable. Each journal entry is worth 10% of the final grade. In the journal entries, students must state their personal intended learning objectives. In each journal entry, they evaluate and reflect on what they have done, learned and planned for the next stage of work. They also reflect on the plans and actions that did not succeed towards advancing the project and the lessons learned from them. The students receive prompt feedback to their journal entries with a focus on the learning objectives and the reflections.

Fig. 1.
figure 1

A tentative Figma design of the learning analytics dashboard (LAD).

Full size image

3 Design Science Research Approach

When developing the student-facing LAD, we followed a Design Science Research methodology [11], which divides the design process into five major steps: 1. problem awareness; 2. suggestion; 3. development; 4. evaluation; and 5. conclusion. In this study, problem awareness (step 1) can be described as the recognition of segregation between ILOs and course activities. The suggestion (step 2) was the development of a student-facing LAD. We employed participatory design to define a LAD that met students’ needs, as suggested by previous studies [12,13,14]. Thirty-seven students in the targeted course participated in a F2F workshop in February 2023, where they were briefly introduced to the dashboard idea by the authors. The students were then asked to work in smaller groups to brainstorm what functionality the dashboard should deliver to meet their needs and preferences. The students submitted their suggestions through the tool Menti. The final part of the workshop involved the entirety of the audience further developing submitted suggestions, by providing useful insights into how they would like the dashboard to function and look like in terms of the interface design. The analysis of the student suggestions led to the development in Figma of the tentative design shown in Fig. 1.

This design was presented to a group of three students who provided their feedback. Major suggestions, such as removing the cluttered ILO distribution as presented under each assignment and adding the presentation for each assignment’s due date, were gathered and implemented for the final dashboard design. With the student feedback, we developed a functional prototype presented in the following section.

4 The ‘Learning Progress’ Dashboard

The dashboard, named ‘Learning Progress’, was developed using react.js, material UI and nivo, which is a library built on d3.js. Data from the LMS (Canvas) used in the case study course was included, with due dates and estimated time per assignment. The resulting implementation, using the tentative design and student feedback from the group session as blueprint, is shown in Fig. 2. The four major components of the dashboard include: 1. the assignment handler; 2. the learning outcome tracker; 3. the grade progression chart; and 4. the learning outcome description box.

The assignment handler (Fig. 2A) starts the workflow. Here, students are able to see the assignment. Through this component, students are able to view completed assignments at the bottom, deadlines for future assignments, and the teacher’s estimated completion time. The two lists of incomplete and completed assignments are scroll-able, allowing the student to monitor their assignments within one interface. Clicking on an incomplete assignment’s checkbox marks it as completed and moves the assignment into the completed list. Clicking on a completed assignment’s checkbox returns the assignment to incomplete status. Through this simple interaction, students are able to keep track of completed and incomplete assignments, how much time they should approximately set aside for each assignment, and when each assignment needs to be completed. Completed assignments feed the data stream, which is connected to and visualized in the learning outcome tracker and grade progression chart.

The learning outcome tracker (Fig. 2B) is the core of the visualization. It displays a student’s progression towards each ILO. This allows students to gauge their course progression in a previously unexplored manner, by directly connecting each assignment to relevant ILOs. Each assignment is credited with varying percentage values in respect to each learning outcome. By marking an assignment as complete, those values are directly displayed and added in the learning outcome tracker. As the values are percentage based, the maximum value a student can reach in each learning outcome is 100%.

Fig. 2.
figure 2

The four components. A: assignment handler; B: learning outcome tracker; C: grade progression chart; and D: learning outcome description box.

Full size image

The grade progression chart (Fig. 2C) displays an estimated current grade for students based on their completed assignments. Visualizing this progression allows students to readily assess the distance between grades and to experiment with combinations of assignments to reach a target grade. The learning outcome description box (Fig. 2D) provides students with immediate access to the ILOs. This reminds students of the learning outcomes and encourages them to actively review them as they complete assignments, thus giving assignments a new value prospective. Before, it was difficult to track the course ILOs through the details of the assignments.

5 Conclusions and Future Development

Based on initial usability evaluation results, future development of the ‘Learning Progress’ dashboard will explore new visualizations, which would further increase clarity and accessibility of student progression towards the course ILOs. Other development paths will enable personalized customizations of the learning dashboard. This could include simple features such as color, as well as more advanced ones such as adding personally developed goals to the assignment section. Other features will support students’ creativity when approaching assignments, such as allowing them to split assignments into arbitrary sub-tasks. Further development of the new features will be conducted through participatory design. The next step of this project is to apply the principles of Privacy by Design to address students’ possible privacy concerns in learning analytics, and to finally introduce the dashboard to all the students participating in the next course iteration [15]. This demonstration paper presented a student-centered learning analytics dashboard to assist students in their course goals’ achievement, in the setting of STEM higher education. Based on the rigorous course learning design, the designed dashboard shows the potential to support students in their regulation of learning, ultimately leading to improved learning outcomes.