Introduction

The use of videos as an educational resource is widely used at all educational levels (Moussiades et al., 2019; Scagnoli et al., 2019). Student interactions with educational videos can be a source of knowledge that can help to understand students’ behavior and to detect some conflictive issues (Buchner, 2018; Yassine et al., 2020). Thus, the analysis of these interactions may be a great asset to discover improvement opportunities to facilitate the transmission of knowledge (Altinpulluk et al., 2020; Yassine et al., 2020).

The educational videos include instructive content in platforms such as YouTube, EDU, or Khan Academy, but also in many MOOCs (Joo et al., 2018). These platforms generally provide learning analytic tools to analyze how students use videos (Aragoneses & Messer, 2020; Yoon et al., 2021).

Regarding the use of videos for teaching–learning specific courses like Physics, several approaches have been introduced with successful results in improving the students’ performance (Anggraini et al., 2020; Küchemann et al., 2020). On the other hand, the style of presentation of the videos for resolution of Physics problems has been shown to play also an important role (Morphew et al., 2020).

Video is an efficient and scalable medium to provide educational content and may have a strong influence on the acquisition of knowledge (Caracta et al., 2018; Tembrevilla & Milner-Bolotin, 2019; Poon et al., 2017). Thus, it is important to know how students interact with videos, in order to better understand their needs and therefore improve the learning processes (Yassine et al., 2020).

In our previous works (Perez-Navarro et al., 2021a, b), we analyzed perception of students about educational videos of different types and forms and the impact of these videos in the students’ performance in courses of introductory Physics. The perception of students was analyzed quantitatively and qualitatively, by using questionnaires and interviews as data source.

However, are students’ interactions with videos compatible with their perception? In the current work, we answer this question with the analysis of the same introductory Physics courses of previous work, but now considering the footprint left by students in the use of videos.

The purpose of this work is to understand how videos are consumed by students and see the relation between their perception and what they really did. In particular, the questions addressed in this work are the following: are videos mainly watched close to a milestone in the course (exam or delivery of an activity)?; are there different patterns of consumption between problem-solving and theory videosFootnote 1?; are there different patterns of consumption of videos according to the topics they deal with (Mechanics, Circuits, Electrostatic, and Magnetism)?; and, finally, are there different consumption behaviors according with the length of the videos?

Background

Previous works have dealt with the importance to estimate some complex perception variables of students or interaction patterns through collecting data from the students’ interaction with videos. Costey et al. look for patterns of video activity, looking for behavioral anomalies such as interaction peaks (Costley et al., 2020). By extending this observation, it is possible to identify patterns of student activity that may explain the peaks, including going to the beginning of new material, returning to lost content, or playing a short segment (Choi et al., 2019).

Several works already worked with the information obtained from the interaction between students and videos (Greiff et al., 2016; Shi et al., 2014), and some works have even dealt with Physics in higher education (Hasan et al., 2020) and Mechanics (Lin et al., 2017).

The students’ behavior patterns, the way they interact, and their purpose in using educational videos are currently of high interest for teachers and institutions (Hu et al., 2020; Silva et al., 2020; Walsh et al., 2019; Zhang et al., 2021; Dart, 2020; Yoon et al., 2021; Bakri et al., 2020). The analysis of those elements can lead to a better comprehension of how the students use educational videos in their learning process and to improve educational videos and the learning methodology associated (Fyfield et al., 2019; Luke, 2020; Yassine et al., 2020).

The interactions of the students with the video player buttons (play, pause, and search) can provide valuable information on the use of this resource (Merkt et al., 2021; Yoon et al., 2021). More generally, the number of views of each video and dates of visualization allow a more complex comparative study by context and by video content (different courses that may have different degrees of difficulty) (Walsh et al., 2019).

In previous works (Perez-Navarro et al., 2021a, b), we analyzed how students perceive “videos with hands,” that are videos in which the hands of the teachers are filmed while explaining a concept or solving a problem, in the context of introductory Physics courses both in online and in face-to-face environments. The main conclusions reached in both works, regarding the current work, are (1) students are very satisfied with videos and perceive them as a very useful resource; (2) they find equally useful both problem-solving and theory videos; and (3) they use videos to prepare themselves before addressing their activities and exams.

Hypotheses

The hypotheses in the current work are:

  1. H1.

    The length of a video affects the way students interact with it.

  2. H2.

    Students mainly watch videos a period before a deadline (an activity delivery or an exam).

  3. H3.

    Students watch problem-solving videos more than theory videos.

  4. H4.

    Students’ interaction with videos of theory is different than their interaction with problem-solving videos.

  5. H5.

    Students’ interaction with videos is different with videos of different topics.

In previous works (Perez-Navarro et al., 2021a, b), carried on merely through the students’ perceptions, hypotheses H1 and H2 were confirmed, while hypotheses H3, H4, and H5 were rejected. In this work, we will check if recorded data confirm students’ perception.

Methodology

The methodology followed is Design & Creation (Peffers et al., 2007). According to this methodology, an artifact is created to test the hypotheses. The videos created are the artifact.

Collected Data

The data collected and analyzed to contrast the hypotheses proposed are those indicated in Table 1.

Table 1 Data collected, their descriptions, and code, if applicable, analyzed in this work
Full size table

Sample’s Size and Typology Studied

The data used in this work were collected from undergraduate students along 5 semesters, between 2017 and 2020. They were from two first year courses of introductory Physics at UOCFootnote 2: (1) Physics I, included in the bachelor’s degree in Telecommunication Technologies Engineering; and (2) Physics Foundations of Computer Science, included in the bachelor’s degree named Computer Science Engineering.

Data over 1000 students were collected. Table 2 shows the number of students per course and per semester. Every academic year is divided in two semesters and indicated as follows: 201X_n, where X corresponds to the first year of the academic year (2017 means 2017–2018) and n corresponds to the semester (1 for first semester and 2 for the second semester). Physics I is only given the first semester of every academic year.

Table 2 Number of students involved in the study grouped by semester and course
Full size table

Aggregated information about the collected data can be seen in Table 3, which shows a summary of the characteristics of the videos used in this work. The information has been obtained from the recorded elements shown in Table 1: interactions, length, content, and topic of each video, aggregated by kind of video (theory or problem-solving). From the length, we show the mean, and the maximum and minimum length of each video. The videos are grouped by topic. “Theo” means “Theory” and “Prbl” means “Problem-solving videos.”

Table 3 Characteristics of the videos studied
Full size table

Methods Used for Collecting Data

To collect data from video usage, an analysis of the interactions of the students has been carried out. The first task to tackle was to find a way to record the visualization activity of the students. Instead of using a streaming server log file (Greiff et al., 2016) or generic browser user data (Shi et al., 2014), data was recorded through an extension for the UOC video tool, Present@ (Perez-Navarro et al., 2012a, b) called Analis@. This tool offers hypervideo functionality through H5P technology. H5P provides mechanisms to track the use of both video and hypervideo using xAPI technology (https://xapi.com/). Thus, any action a user takes on a video is stored in a Learning Record Store (LRS) database which is then exploited with data analytic tools to generate a tracking report available to teachers.

The Analis@ plugin was added to the videos provided throughout the courses between 2017 and 2020 to collect the interaction of students with the videos during the courses. The choice of Analis@ was motivated by the options availability of the private environment used at UOC and the privacy concerns to be considered.

Thus, in this work, we collected and typified data according to some metrics (see Table 1) through the interactions of the students with the videos, without influencing the students in any way. These interactions were collected anonymously to ensure that at no time we could reproduce the interactions made by a particular student.

The rest of the data needed for the purpose of this study (delivery dates, content, and content and topic of each video) is contextual to each course and was manually collected.

Methods Used for Analyzing the Collected Data

To analyze the collected data, we followed several steps.

First, we assessed variables’ dependencies checking if we could establish any kind of correlation between the different data. Therefore, we analyzed the influence of the number of reproductions, considered as a dependent variable, and the length regarding the total number of interactions. Since we checked first that the number of reproductions is an influential variable, we plotted the average of total number of interaction divided by the number of reproductions in those videos [(nPLY + nPAU + nSKS) / nVis] versus the length of every video.

To avoid the effect of the duration of the videos and the number of visualizations in the number of interactions, we normalized the variable number of the interactions of each type by number of visualizations and length of the videos in seconds. Thus, we have the number of interactions per visualization and per second:

$$nXXX\_Norm=\frac{\left(\frac{Number\; of\; interactions }{Number\; of\; visualizations}\right)}{seconds\; of\; the\; video\; length}$$
(1)

where XXX represent the interaction analyzed: play (PLY), pause (PAU), and search (SKS). The formulae for every interaction are:

$$nPLY\_Norm=\frac{\left(\frac{Number\; of\; play}{Number\; of\; visualizations}\right)}{seconds\; of\; the\; video\; length}$$
(2)
$$nPAU\_Norm=\frac{\left(\frac{Number\; of\; pauses}{Number\; of\; visualizations}\right)}{seconds\; of\; the\; video\; length}$$
(3)
$$nSKS\_Norm=\frac{\left(\frac{Number\; of\; searches}{Number\; of\; visualizations}\right)}{seconds\; of\; the\; video\; length}$$
(4)

These three formulae give the total number of interactions (play, pause, and search) normalized to the number of visualizations and to the duration of the video.

In this preliminary analysis, we got the necessary information to contrast the first hypothesis H1: “The length of a video affects the way students interact with it” proposed in this work. Thus, it has been studied the correlation of the total number of interactions regarding the number of reproductions and those of the average of number of interactions divided by the number of reproductions, performing a regression analysis in both cases. Subsequently, the kind of the distribution followed by the normalized data and the outliers found were studied. The addition of the Formulae (2), (3), and (4) results gives the number of total interactions, and therefore allow to check hypothesis H1. These formulae allow, also, to check whether the kind of interaction is different according to the length of the video.

The next step was to analyze the number of reproductions of videos performed by the students, per day, during the semesters studied. Therefore, we could check whether the consumption of videos is more intense near Continuous Assessment Tasks (CATs) and exams, which is an indicator of its potential usefulness to students and allows to contrast hypothesis H2: “Students mainly watch videos a period before a deadline (an activity delivery or an exam).”

Next, to check H3: “Students watch problem-solving videos more than theory videos,” we look for a significant difference in the frequency of using videos depending on its content (theory or problem-solving). Therefore, an inferential statistical study was carried on comparing the reproductions of each type of video. For doing so, a robust statistic methodology (Andersen, 2007; Yuen et al., 1990) has been used. The same methodology has been used to contrast H4. In both cases, the analyses were repeated by using regular Student’s t-test to perform a comparative.

Since the results regarding H4: “Students’ interaction with videos of theory is different than their interaction with problem-solving videos” were not conclusive, a principal component analysis (PCA) and a clustering analysis have been applied to get more insight about the analyzed data. In clustering, Euclidean distance has been used for the 3 types of interactions normalized (nPLY_Norm, nPAU_Norm, and nSKS_Norm) and the number of reproductions (nVis). The clustering analyses were performed adjusting the cutting height with two different perspectives: first, by considering the number of clusters expected, two (theory and problem-solving videos); second, to try to get a homogeneous distribution of videos into the clusters.

Finally, to check whether the topic of the video affects the way in which students interact with them (H5: “Students’ interaction with videos is different with videos of different topics”), we carried out a PCA and characterization study to be able to observe how the videos were classified by topic (Circuits, Mechanics, Electrostatics, and Magnetism). Thus, the variables used in the analysis by clustering were the topic regarding the interactions previously normalized (nPLY_Norm, nPAU_Norm, and nSKS_Norm).

Results

In this section, the results of applying the aforementioned methodology are shown.

Preliminary Results for Raw Data Analysis

In this section, we treat the data to look for dependencies and to study their distribution.

Influence of the Videos’ Number of Reproductions on the Number of Interactions

The total number of interactions registered in a video could be related to the number of reproductions. To show it graphically, we have plotted the total number of the interactions performed per video in front of the number of reproductions of that video (Fig. 1). In this figure, these values are plotted in a bar diagram and a regression is performed using the maximum of the value of each bar to check the correlation.

Fig. 1
figure 1

Average of total number of interactions (nPLY + nPAU + nSKS) versus number of reproductions (nVis) for all the videos studied in this work during the 5 semesters. The variable nVis is taken by ranges

Full size image

Figure 1 shows a positive relation between the number of interactions and the number of reproductions, i.e., videos that have been visualized more times have more interactions, as expected. The correlation between both variables (R2 = 0.9787) gives ground to this fact.

Influence of the Videos’ Length on the Number of Interactions

Similar to the previous case, the number of interactions could be also affected by the length of the video. So once eliminated the influence of the number of reproductions, we can see in Fig. 2 that the average number of interactions (calculated as the sum of Eqs. 2 to 4) is higher in longer videos. The correlation between both variables (0.9396) gives ground to that fact.

Fig. 2
figure 2

Average of total number of interactions (nPLY + nPAU + nSKS) divided by the number of reproductions (nVis) versus the videos length for all the videos studied in this work during the 5 semesters. The variable length of videos is in seconds taken by range

Full size image

To avoid the effect of the duration of the videos and the number of visualizations in the number of interactions (see “Methodology” section), the variables corresponding to the number of interactions have been calculated, as indicated in Eqs. (2) to (4). We can see an excerpt of such data in Table 4 for all the courses studied. Full table (Table 11) is in the “Appendix” section. Using the data already normalized in this way, we studied the distribution of the data.

Table 4 Excerpt of the average number of reproductions and interactions of each type per semester of each monitored video
Full size table

Studying the Distribution of the Normalized Data

Figures 3 and 4 show the box plots applied to the data collected, first considering all the data as a set for each variable (nVis, nPAU_Norm, nPLY_Norm, and nSKS_Norm) and then, grouped by theory and problem-solving videos for all semesters.

Fig. 3
figure 3

Box plots for the total data set of the characteristics number of reproductions (nVis), number of pauses relativized with respect to the number of reproductions and video duration (nPAU_Norm), number of searches performed relativized with respect to the number of reproductions and duration of the video (nSKS_Norm), and number of starts (plays) made relative to the number of reproductions and duration of the video (nPLY_Norm)

Full size image
Fig. 4
figure 4

Box plots of the different normalized variables according to the different types of videos: theory and problem-solving videos

Full size image

As can be seen in Figs. 3 and 4, the data does not present a normal distribution and show several outliers. These outliers cannot be removed because that could drive to a deletion in cascade. Hence, we have used robust statistic techniques (Mair & Wilcox, 2020) to analyze them to minimize the effect of the outliers in the statistical results.

Results on Normalized Data

In this section, we will expose the results achieved using the data presented in the previous sections.

The Effect of the Length of the Video on the Way Students Interact with Them

As shown in Fig. 2, the duration of the video has an influence in the number of interactions performed. In this section, similarly, the number of interactions normalized of each type divided by video length has been shown. Figure 5 shows the number of normalized interactions for the total of videos and Figs. 6 and 7 the number of normalized interactions in theory and problem-solving videos. The regressions performed to achieve the R2 value have been calculated taking the maximum values of the bars.

Fig. 5
figure 5

Graphical representation of the mean of interactions’ number: nPAU_Norm, nSKS_Norm, and nPLY_Norm of all the semester studied versus each duration range for all the videos

Full size image
Fig. 6
figure 6

Graphical representation of the mean of interactions’ number: nPAU_Norm, nSKS_Norm, and nPLY_Norm of all the semester studied versus each duration range for the theory videos

Full size image
Fig. 7
figure 7

Graphical representation of the mean of nPAU_Norm, nSKS_Norm, and nPLY_Norm of all the semester studied versus each duration range for the problem-solving videos

Full size image

Relation of Videos’ Interaction with Assessment Activities

In Fig. 8, we can see the number of reproductions in the context of the semester. To do so, the figure shows the reproduction of videos per day of semester. There is a figure for each analyzed semester 2017/2018, 2018/2019, and 2019/2020. It must be highlighted that in the first semesters of each year (Fig. 8A, C, E), two courses have been involved: “Physics Foundations of Computer Science” and “Physics I,” while in the second semesters (Fig. 8B, D), only the course “Physics Foundations of Computer Science” is considered. The periods in which the students had to perform the CATs and the final exam have been marked in the graph to provide more context to the data.

Fig. 8
figure 8

Representation of the number of reproductions per day of each semester for Physics Foundation of Computer Science and Physics I between September, 2017 and February, 2020. CATs correspond to continuous assessment delivery

Full size image

Number of Reproductions Initiated by Type of Video

To check if the content of the videos, theory or problem-solving, has any influence in the number of reproductions (nVis), general statistics (mean, standard deviation, and quartiles) were obtained first (see Table 5).

Table 5 Means, standard deviations, and quartiles of the frequency of visualization (reproductions of each type of video divided by the number of videos) of each type available to students per semester classified by type of video
Full size table

In Table 5, the frequency of visualization (number of reproductions / number of videos) data grouped by content (theory or problem-solving) is shown. Then, we performed a robust Student’s t-test for comparing the means of the reproductions of the videos according to the contents, risen the hypotheses as follows:

  • H30 = The average of the reproductions started in the theory videos is the same as the average of the reproductions started in the problem-solving videos.

  • H31 = The average of the reproductions started in the problem-solving videos is different than the average of the reproductions started in the theory videos.

The result of this statistical study is shown in Table 6.

Table 6 Statistical study of the number of reproductions of all the videos in the 5 semesters studied compared by content (theory and problem-solving). Robust Student’s t-test has been applied through the paired Yuen method (Abdullah & Othman, 2012) with a 95% confidence level
Full size table

Since the result shown in Table 6 seems to be in contradiction with the students’ perceptions of our previous work, a set of graphs similar to Fig. 8, but differentiating between theory and problem videos, was performed. Figure 9 shows this distinction that will be analyzed in the “Discussion” section.

Fig. 9
figure 9

Representation of the number of reproductions per day of each semester for Physics Foundation of Computer Science and Physics I between September, 2017 and February, 2020. CATs correspond to continuous assessment delivery. Differentiating the use of the theory videos from those of problem-solving

Full size image

Number of Interactions Made in Each Type of Video

In this section, a similar analysis to the previous section is performed but, in this case, the goal is to check if there are differences between the number of interactions of each type of video (theory and problem-solving). Therefore, in the analysis shown here, general statistics (mean, standard deviation, and quartiles) were obtained first (see Table 7).

Table 7 Means, standard deviations, and quartiles of the reproductions of each type of video divided by the number of videos of that type available to students per semester classified by type of video
Full size table

As in the previous section, Table 8 shows the robust Student’s t-test results for the three types of interactions studied, grouped also by type content (problem-solving or theory). To perform the analysis, the hypotheses taken are:

  • H40 = The mean number of interactions of each type made on the theory videos is the same as the mean number of these interactions made on the problem-solving videos.

  • H41 = The mean number of interactions of each type made on the theory videos is different than the mean number of these interactions made on the problem-solving videos.

Table 8 Number of interactions in all the videos in the 5 semesters studied compared by content (theory and problem-solving). Robust Student’s t-test was applied through the paired Yuen method with a confidence level of 95%
Full size table

We can see in Table 8 that there are significant differences between number of use of pauses between the theory and problem videos.

Results of Grouping and Classifying the Data

Since results related with hypothesis H4 lead to inconclusive results, a clustering study of the interactions by type of videos was carried out, to check whether we could classify the videos of problem-solving and theory into different groups by considering the number of reproductions and number of each kind of interaction. Figure 10 shows the result of the clustering according to their content, based on the number of interactions of each type and the number of reproductions that students carry out with these videos. The trimming height was selected to get two clusters (one per kind of video).

Fig. 10
figure 10

Characterization of the data based on the number of interactions of each type and number of reproductions that the students carry out with the videos. Cutting height established to divide two groups

Full size image

The height of the clustering of Fig. 10 has been selected to be the one that gives two groups of videos. As can be seen, only 3 videos are in cluster 2. Another selection of height was made based on the number of videos, to check if we could get two clearer distinct groups. However, as shown in Fig. 11, similar results are achieved because, even though there are two big clusters, it does not seem that their nature is about the content of videos (theory or problem-solving).

Fig. 11
figure 11

Characterization of the data based on the number of interactions of each type and number of reproductions that the students carry out with the videos. Cutting height established to homogenize number of videos in two clusters

Full size image

In addition, a PCA was performed on these data using the same characteristics, and the results can be seen in Table 9.

Table 9 Data obtained from the PCA characterized by interactions for the theory and problem-solving videos
Full size table

Results of the Number of Interactions by Topic

In this section, we analyze the number of interactions by topic. To make the clustering, the Euclidean distance has been used. To get the final clusters, height was adjusted considering that we are looking for four clusters and then trying to get a homogeneous distribution of videos into clusters. Figure 11 shows both steps taking different heights.

In addition, a PCA was performed on these data using the same characteristics, and the results can be seen in Fig. 13 and Table 10.

Table 10 Data obtained from the PCA characterized by the interactions for the videos by theme using the original data
Full size table

Discussion

Figure 1 and the associated correlation shows a strong positive relation between the number of reproductions and the number of interactions, that is, the bigger the number of reproductions, the larger the number of reproductions. We could consider this affirmation as obvious, but finding it gives confidence on the coherence of the data collected.

The next step has been to divide the whole number of interactions of each video by its number of visualizations. This normalization allows to see the interactions per video, regardless the number of visualizations, and Fig. 2 shows a strong positive correlation between the number of interactions and the length of the video, i.e., the longer the video, the more interactions are found.

To make a deeper analysis on when interactions take place, Fig. 5 shows that the number of interactions per minute reduces as the length of the video increases. These results are compatible with hypothesis H1. This finding is in accordance with previous analyses of the proper length for an educational video and the relation of the increment of the cognitive load and the decrement of the attention attraction as the video length increases (Afify, 2020; Kruger & Doherty, 2016).

Regarding hypothesis H2, Fig. 8 shows that most of the videos were watched next to a deadline of a Continuous Assessment Task, CAT or next to an exam, since the visualization peaks co-occur during deadlines of CAT and final exams. These results are compatible with H2 and with the results obtained in previous works (Perez-Navarro et al., 2021a, b).

The next step is to validate hypotheses H3 and H4. Regarding the number of interactions, Figs. 6 and 7 show some differences between both kind of videos, although they are not conclusive. In the next paragraphs, we will discuss the other elements analyzed to verify or reject the hypotheses.

Tables 6 and 8 show a p-value is lower than 0.05; therefore, we reject the null hypothesis H30 and can state that the frequency the students watch theory videos and problem-solving videos is different. It is in contradiction with the student perceptions of our previous works (Perez-Navarro et al., 2021a, b).

Figure 9 shows that, in fact, the number of reproductions for the problem-solving videos is higher than that of theory videos along the semesters. However, during the semesters, both types of videos are frequently used, and this can create in the students the illusion that both are equally used. It is important to point out that, as shown in Fig. 9, it seems that there is a change of watching tendency in days next to exams or CATs, where the students prefer watching problem-solving videos. Therefore, in this work, we must accept hypothesis H3.

For hypothesis H4, we can only reject the H40 for the case of the interaction “pause.” The mean of the number of this interaction cannot be considered equal for theory and problem-solving videos. However, but for the rest of interactions (“seeks” and “plays”), this cannot be rejected.

Since from the robust t-test performed we cannot make any generalization about the differences of the total number of interactions for theory and problem-solving videos, several clustering analyses were performed. Figures 10 and 11 show that most of the theory and problem-solving videos went into cluster 2, so we could not get a clear distinction between those videos.

To confirm the results of the clustering, a PCA was performed. As shown in Table 9, for both cases, theory and problem-solving videos, we see that 70% of the variance is due to component 1 (0.7) and 20% due to component 2 (0.2). Thus, we can state that there are no significant differences between the number of interactions, when they are considered as a whole, between theory and problem-solving videos, and therefore, hypothesis H4 is rejected which is compatible with students’ perception found in previous works (Perez-Navarro et al., 2021a, b).

Finally, to assess hypothesis H5, we perform a PCA and clustering analysis considering the topics taught in those videos. Figure 12 shows that most of the videos are in cluster 1, and therefore, there is no difference between them. However, cutting the dendrogram in order to get a more homogenous distribution, most of the videos of Circuits and Electrostatics appear in a single cluster, 2, while most of the videos of Magnetism go to cluster 4 and most of the videos of Mechanics are distributed among clusters 1 and 4. Figure 13 also shows a change of behavior on the use of the videos between the topics, and in Table 10, we can see that component 2 represents the 93.6% and the 91.3% of the accumulative variance for Mechanics and Magnetism, respectively, while component 3 represents the 93.0% of the accumulative variance for Circuits and Electrostatics. This change of behavior could be due to the own idiosyncrasy of the proper topics themselves that are those where students need more capacity for abstraction thinking (Faulconer et al., 2018; Tiruneh et al., 2017). Thus, although further analysis should be done to confirm this statement, these results show that there are some differences among topics and therefore are compatible with hypothesis 5.

Fig. 12
figure 12

Dendrogram resulting from the clustering of the videos, considering exclusively the interactions of each type normalized by the number of reproductions and the duration of the video by thematic of the videos

Full size image
Fig. 13
figure 13

Graphic representation of the first two components with respect to the interactions with the video studied by theme using the original data

Full size image

To go deeper into the differences, looking at Fig. 8, we can see that the number of visualizations increases just before every CAT. However, in semester 1, we see some peaks between CAT 4 and the exams, and an important peak just before the exams. That behavior is very different from the graphic corresponding to semester 2. The rational of this difference may be due to the differences in calendar: CAT 4 in semester 1 is delivered just before Christmas, when students face 2 weeks of holidays, and exams come just after them; however, in semester 2, there are no holidays among CAT 4 and exams. Thus, in semester 2, study is probably more uniform than in semester 1, when students stop for holidays and start to study just after them. However, we can see that during Christmas period, there are still some peaks.

Another important element that we can see in Fig. 8 is about the size of the peaks. In Physics Foundation of Computer Science, CAT 1 is about photonics, CAT 2 is about circuits, CAT 3 is about Electrostatics, and CAT 4 is about Magnetism; and in Physics I, CAT 1 is about Mechanics, CAT 2 is about Electrostatics, and CAT 3 is about Magnetism (CAT 4 is about Termodinamics but there are no videos of this topic). In all the cases, we see a first clear peak for CAT 1, although the shape in semester 2 is wider than in semester 1. Probably students in the second semester have already the rhythm of studying and start learning before. CAT 2 has an important peak in semester 1: this is the first CAT with Electromagnetism (in Physics I) and the next peak, we have also a high peak, which is the first peak of Electromagnetism for Physics Foundation of Computer Science. This behavior is compatible with the cluster analysis performed, whose results have shown that Electrostatic and Magnetostatic play an important role.

The importance that Electrostatic has in both courses can be seen also from the number of visualizations. Looking at Table 11, we can see that the four more visualized videos are the following: Electrostatic force and field calculation (192, Problem), F and E calculation (162, Problem), Electrostatic field and force (142, Theory), and The electric charge (135, Theory). They are videos of Electrostatic. We have to go until position 30 to find the first video of magnetism in number of visualizations: Calculation of the magnetic field created by an infinite wire in all space (55, Problem). The reason for so few visualizations of magnetism is that this topic is given at the last part of the course, when many students have already abandoned the course. In fact, the peak corresponding to a CAT with electrostatic is usually higher. However, if we look for the 10 videos with more plays normalized, we see that 9 correspond to magnetism. That means that although the number of visualizations is not so high, students look at them deeply.

On the other hand, if we look at the number of pauses, among the 20 videos with more pauses, 16 are videos of problems. That is compatible with the use explained by students in previous research (Perez-Navarro et al., 2021a). There, students claimed that, when watching a video of problem-solving, they usually stop the video and try to solve the problem by themselves, and thereafter resume the video to check whether their work is right.

Comparing semester 1 and semester 2, we can see those peaks in semester 1 are higher (with the exception of the peak of CAT 1). In semester 1, we have two courses instead of one. However, Physics I students are approximately half the number of students in Physics Foundation of Computer Science, and the peaks in CAT 2 and CAT 3 are more than double in Fig. 8A, C, than in B and D. Only CAT 3 between C and D is not so high. This could be explained by a higher interest of students in Telecommunication in watching videos, or because CAT 2 corresponds to Electrostatics for students of Telecommunication, that is one of the most watched elements. The case that is different is the peak corresponding to CAT 1 in semester 2. It is higher that CAT 1 peak in semester 1, although there is only one course. To analyze this element, extra work needs to be developed.

In Fig. 8E, the broad band of CAT 3 is wider than on the other cases, although the shape is similar in all the curves of all the semester 1 and all the curves in semester 2. However, there are different behaviors between semester 1 and semester 2. We have explained the different behaviors at the end of the semesters because of Christmas, but the rest of the differences need more work. In semester 2, there is Easter week, but this cannot explain so many differences.

Conclusions

This paper presents a learning analytic analysis to see how students consume videos in a Physics course in engineering degrees, taking into account its particularities in type (problem-solving or theory) and content (Mechanics, Electrostatics, Circuits, and Magnetism). The current work improves current literature by analyzing not only students’ behavior when consuming educational videos but by also contrasting these results with the perception of the students in the same context (Perez-Navarro et al., 2021a, b). Therefore, the paper goes further of the typical learning analytics results by assessing whether the way students consume videos aligns with their perception and learning methodology.

When analyzing the data collected from student interactions, the first conclusion obtained is that the duration of the videos affects the number of interactions that students carry out with them (direct relationship) and the frequency of interactions (inverse relationship). On the other hand, observing the frequency/period relation of the visualizations of the videos, it has been possible to verify that students use the videos as a resource to prepare and address their assessment activities and exams in Physics.

In addition, significant differences have been found in the frequency of use of theory videos compared to problem-solving videos. Therefore, it can be concluded that students use problem-solving videos more actively than theory videos. However, no significant differences were found in the way of interacting with theory videos compared to problem-solving videos. Therefore, it can be concluded that students use these two types of videos in a similar way. Nevertheless, if we look only to the videos with more interactions, most of them are problems videos. 

Finally, a significant change in behavior has been observed in the case of circuits and a slight change in mechanics. Therefore, it can be concluded that the different interactions explain the variance and could mean a change in the behavior of the students with the videos according to their theme.

As a future work, we plan to analyze how each single video is consumed and where there are peaks of interactions with students.

Limitations of This Work

Due to the privacy reasons, the interactions are anonymous and are not related to any user neither to the session of users.

The study has been performed in the context of a Physics course in an engineering degree of an online university. Even though previous studies show that results can be generalized in blended learning (Perez-Navarro et al., 2021b), current results cannot be generalized in other contexts without further analysis.