1 Introduction

Mobile device usage is the fastest growing technology trend in industry [1]. The affordance of mobile devices, the convenience of accessing information whenever needed, and hence facilitate contextual learning, make mobile devices an attractive solution for education. Students are already using their mobile devices for learning [34] and several Bring Your Own Device (BYOD) initiatives for educational purposes have been reported in the literature [26, 42]. However, mobile devices’ heterogeneity “impose non-trivial challenges to the Internet multimedia services” [44]. Despite of the known challenges [44], and limited knowledge on how to design for mobile learning [63], existing research has also reported positive results when using mobile devices to facilitate learning [7, 20, 30, 41]. At the same time video based learning is increasing in popularity [15]. Video based learning is considered to be the most effective way of delivering the educational content to mobile devices [27]. Video is considered to be “casual and conversational”, a “primary form of communication” between many young people [4]. In the context of mobile video applications, video quality is considered to be “a critical factor” in their adoption [23], due to both teachers and students losing interest when presented with low quality video [35]. Therefore it is also an important factor to be considered for mobile video based learning.

There is also little knowledge on how to design effective mobile learning courses in general [56, 63] and for mobile video learning in particular [19]. Therefore more research is needed on how to design mobile learning courses, especially those that make use of video, and this research aims to address this gap. Moreover, there is a need for a better understanding on how the quality of the video content (understood in this context as the quality of the encoding) affects learning and perceived quality. We focus on video as opposed to other content (e.g. audio), as this is more likely to be adapted when delivered over the Internet. Video adaptation is provided either by the content providers (e.g. Netflix allows its users to select a lower video quality to decrease data consumption [43]) or it could be imposed at the network providers [55]. Most of the times the users do not have control over the adaptation process as it is done in the background. Video adaptation is likely to affect also mobile learners as mobile networks have limited bandwidth and mobile data billing plans are capped [39] due to increases in video traffic [58, 59].

In this research we study how the video adaptation to the device characteristics affects the learning and learner perceived video quality across different video content categories. We also aim to determine whether there is a certain video content category that is less affected by the adaptation. Although some studies have looked into how the video adaptation could affect the battery lifetime [22, 32], cost of delivery [40] or learning [31], to the best of our knowledge this is the first study which looks into whether certain categories of mobile video content could be more suitable for mobile learning. The results of this study could help to better understand how to design video based learning activities facilitated by mobile devices.

The rest of this paper is organised as follows. Section 2 presents the related work in the areas as follows: Section 2.1 introduces existing work on video based learning facilitated by mobile devices; Section 2.2 expands on the existing work on learning with devices of different screen sizes and resolutions; and Section 2.3 briefly introduces existing work on video adaptation on mobile learning. Section 3 presents the existing classification of the educational video content and justifies the classification used in this study. Section 4 presents the study set-up and methodology, the participants and the results of the evaluation. Section 5 discusses the results and Section 6 presents the conclusions of the study.

2 Related work

2.1 Video based learning

Study habits are changing: YouTube became one of the top search engines [11] and students are using video as a first point of reference when they are looking for information related to a topic [18]. In education, video learning can be used to promote reflective enquiry [3], to scaffold the scientific concepts [12], or as a cost effective means of facilitating access to education for as many students as possible [65]. Also, according to the cognitive theory of multimedia learning [29], presenting content both audio and visually can result in better learning, by increasing the capacity to remember and the transfer of information.

Video has been used in mobile learning systems for different purposes such as: to enable access to education to a large number of students [9, 62, 65], to improve interactivity [64], to compensate for the small size of mobile devices [17], to investigate different solutions for mobile learning educational content [27, 49], or to investigate solutions for creating educational content which will be suitable for any device, regardless that it is a mobile device or a desktop computer [10].

Existing studies have looked into the most effective media for video based learning [27, 53, 63]. For example, Macdonald and Chiu [27] have investigated the viability of using mobile learning for workplace learners. They did a pilot study which contained different media: text (as a PDF), video and audio. The content was both stored on the mobile phone, and part of it was streamed. The results of the study have shown that video was the most effective way of delivering content for mobile devices. At the opposite side were text files that were found cumbersome to use, and students reported eye strains while reading. These results are similar with [53] but [63] also found out that this depends on the location; that mobile learners would prefer power point presentations and e-books at office [63], whereas the videos would be preferred when students learn at home or when learning in a cafe. However, none of the above studies have looked at what type of video educational content might be more suitable for mobile devices, which is what this study addresses.

2.2 Learning across multiple screen and resolution devices

Maniar et al. [28] use three devices to compare how learning effectiveness is affected by different screen sizes. They used a “small screen” device (1.65 inches, 128 × 96 pixels), “medium screen” device (2.28 inches, 320 × 240 pixels) and “large screen” device (3.78 inches, 320 × 240 pixels). A single video is used in the study to determine whether the device screen size has effects on learning. The results have shown that all students improve their learning outcome regardless of device, but students who used large and medium screen devices learned significantly more than those who use a small screen device. No statistically significant difference was obtained for the subjective opinions of participants using the three mobile devices.

Kim and Kim [25] looked into how three different screen sizes (small - 320 × 240; medium - 480 × 320; and large - 600 × 800) and two categories of content text only and text and image affect vocabulary learning. The results show that the larger the screen size the more effectively the students learn, but little difference has been noted between the category of content used and learning effectiveness. As opposed to this study, our work focuses on video based learning.

Al Ghandi et al. [2] compare the effect of three screen sizes in the context of a mobile health application. The study concludes that screen size does not have significant difference in the user retention and understanding of content structure, but it affected the perceived clarity of information, time and ability to recall information.

In an experimental study conducted by Sanchez and Goolsbee [50] examined text size and screen size on reader retention. The participants were divided into two groups: participants using a large desktop and participants reading from a virtual small screen. They were given three separate expository texts. They completed a recall test to measure their retention of each text. The study concluded that the small devices negatively affect retention of information.

As opposed to the above studies we will look at different categories of video content (the video content used in the above studies focuses on a single video category). This study also addressed the perceived video quality aspect which has not been considered in the above studies.

2.3 Video adaptation in mobile learning

Video content, when used in a learning context, is typically provided as a single video version delivered to all the students [40]. This version is created in such a way that will cover most of the students’ devices and network requirements [40]. Few studies have looked into the video adaptation for mobile learning purposes [33]. Moldovan et al. [33] presents an overview of the opportunities and challenges to use video adaptation in mobile learning with the aim of improving learners’ battery lifetime. This kind of adaptation has been shown that it could be done without affecting the learning achievements [32].

Other studies have looked into the learners willingness to pay for video content quality could be determined [38] and whether the adaptation performed with the aim of reducing the cost for those not willing to pay as much affect information assimilation or perceived video quality [40]. The results have shown that learners who are not willing to pay as much for the video quality could save money without necessarily affecting their learning and perceived quality with the algorithm proposed in [40].

Karadimce and Davcev [24] look at the mobile learners’ cognitive styles with the aim of reducing the bandwidth required to deliver the content. Ghinea and Chen [14] have shown that cognitive styles could influence video color choice. This research, similar with the findings from [36] has shown that the users do not always chose the optimum video quality in learning contexts, or when they look at movie trailers and documentaries [51].

The research presented in this article also considers video adaptation. However it will focus on a different aspect than the previous research by addressing how certain categories of video affect learning and perceived video quality.

3 Educational content categories

Video content for educational purposes has been divided by various researchers in different categories [13, 31, 40]. In [13] the author describes the format of video learning objects and argues that the video learning objects could be divided into four categories:

  1. 1.

    Mini-Lecture format: characterized by the presence of one or more teachers speaking into a webcam.

  2. 2.

    Interview format: similar to the interviews presented on television, where one or more participants answer question(s).

  3. 3.

    Demonstration format: the video is characterized by showing something rather than telling something.

  4. 4.

    Scenario format: characterized by filming people in real setting scenarios.

Moldovan et al. [31] propose to characterize educational video content across six categories as presented below. This classification has been made based on the analysis of over 1500 educational videos.

  1. 1.

    Animations: computer generated clips that may present recordings of virtual worlds.

  2. 2.

    Demos: described as a multimedia clip in which a person shows how to do certain practical things.

  3. 3.

    Documentaries: video clips characterized as having a higher number of scene changes comprising the clip. They are intended to show some aspects of reality such as outdoors locations.

  4. 4.

    Presentations: defined as a multimedia clip in which the lecturer and the accompanying slides or blackboard are presented.

  5. 5.

    Screencasts: the clip consists of video sequences in which the computer screen is recorded.

  6. 6.

    Slideshows: the clip consists of video sequences in which the computer screen is recorded.

The last category is more comprehensive than the first category and it could be argued that the categories presented in [13] could be included in [31], where:

  • Mini-Lecture format and Interview format could be both included under the Presentations in [31].

  • Demonstration maps on the Demos category in [31].

  • Scenario format into the Documentaries in [31].

A similar categorization with [31] of video content has been used in [40] with the only difference that the latest one contains also a seventh category called Interview. This makes a more inclusive category. In this case the Mini-Lecture format and the Interview format could be mapped to two different categories. This last category includes both elements from [13] and [31]. As a result it gives a more comprehensive description of the educational video content and could provide better insights on how content type affects user experience and learning. Therefore it will be used in this study.

4 Study

The aim of the study is to determine whether the learners can study across different categories of adapted video content and how certain educational video content categories affect perceived quality.

4.1 Set-up and methodology

4.1.1 Video content

As discussed in Section 3, we aim at selecting a variety of educational content. We used the seven categories educational content classification proposed in [40] as it was more comprehensive than other categories. As a result we selected: (1) video presenting a recoding of a slideshow, (2) a screencast describing how to use a program, (3) a presentation during which a person speaks with the help of some slides playing in the background, (4) a lab demo presenting a recording of an experiment, (5) an interview in which two persons discuss, (6) a documentary, and (7) an animation. The content was selected to meet the requirements of various audiences from domains such as: business, science, astronomy, ecology, environment, learning resources, literature. The variety of educational content was selected in order to eliminate any possible individual differences (e.g. interest in the subject matter), but also to allow for a range of videos, that have different temporal characteristics, characteristics which influence user perceived quality. Figure 1 shows screenshots of the videos used during the study.

Fig. 1
figure 1

Screenshots of the video sequences used in the study, divided by their content category: a slideshow, b screencast, c presentation, d lab demo, e interview, f documentary and g animation

Full size image

4.1.2 Video quality evaluation

Video quality was measured by following the guidelines provided by the International Telecommunication Union [21]. We used the same guidelines in our study to organise the quasi-experimental study which assessed subjective video quality testing. The assessment of the video quality was performed after the video was watched, by using the Mean Opinion Score (MOS) [48]. We use the absolute category rating [21] in our evaluation. The participants were asked to rate a video on a scale from 1 to 5. The labels from Table 1 were provided to the participants when they were rating the videos.

Table 1 Quality and impairment scales according to ITU [21]
Full size table

4.1.3 Learning achievements

Learning achievement is defined in the context of this study as the quantity of knowledge the learner has accumulated as a result of watching the video. Learning achievement was measured using pre and post video watching questionnaires, which addressed the content presented in the video. The questionnaire was previously used in a different study [40] and follows the questionnaires design recommendation presented in [46]. The role of the pre-test is to assess whether the user could have known the answers before seeing the educational content.

4.1.4 Methodology

A quasi-experimental study [6] was organised and designed by following the International Telecommunication Union (ITU) recommendation [21, 47]. During the study the participants had to watch the seven different educational video content categories on two mobile devices: a “small screen device” - Samsung Europe and a “large screen device” - Google Nexus One (see Table 2 for more details on the devices’ characteristics). All videos were displayed on a mobile device and they were embedded in a webpage.

Table 2 Characteristics of the mobile devices
Full size table

We used a single stimulus test [21] to ensure that the study is as close to a real life learning scenario as possible. Among the methods available for assessing the multimedia quality, Absolute Category Ranking (ACR) [21] was chosen. This method was considered the most suitable since it does not imply viewing two sequences in parallel, or having a reference among the clips. If the subjects can see both the reference and the adapted version either in parallel or at different point in times, the learning achievements would have been affected.

The participants were provided with instructions on what they were expected to do during the experimental study including the fact that they were expected to rate only the video quality and not its content. The instructions were provided in writing but also reinforced orally by the researcher presented during the study. According to ITU-T recommendations [21, 47] we organized a training session before the actual study. A separate video clip was provided to the participants (and discarded from the analysis of the results) for training purposes. In this way, the participants could familiarize themselves with the study set-up and could ask questions in order to avoid any confusions later on in the study. After watching each video, the users were asked to rate each video using the Mean Opinion Score (MOS) scale [48].

A within-subject design [8] was followed through the study. This has the advantage of reducing the variance associated with the individual differences. To avoid that the learning achievements results being affected by the participants watching both video versions, we use a similar video to be displayed on the “small screen device” as on the “large screen device”. Both versions of the video were selected from a longer educational video.

The participants were also asked to complete a pre-test to evaluate their knowledge level on different areas covered by the video clips. The results of the pre-test were used to assess whether the participants knew the information presented in the video before watching them and the post test was used to assess participants knowledge after watching the videos. The participants were asked to rate the quality of each video and answer questions related to the learning outcome immediately after seeing the video. The pre-test was delivered at the beginning and after the training session.

The Student t-test [66] was used to measure if there is a statistical significant differences between the student pre and post knowledge, and measuring the differences in video quality between the “small” and “large” screen device.

4.1.5 Video encoding

The videos used in this research were downloaded at high quality from iTunes U[niversity]. The encoding of the original video content used for the study was H.264/MPEG-4 AVC. The videos were adapted to meet the mobile device requirements, following the upper threshold video bit rate recommendations from [39]. As a result the original video bit rate and resolution were changed to the values presented in Table 3). As the audio content plays an important component in understanding the content and improving the user experience [16], but also because this content is not usually adapted, it was maintained at the original quality. Several video sequences were extracted from the original video in order to meet the existing recommendations on video length when learning for mobile devices [5, 53, 61].

Table 3 Video encoding for the two devices
Full size table

4.2 Participants

Fifty-four participants watched the videos on each device. The participants volunteered to take part in the study and they were not reimbursed for their participation. Their ages ranged from 19 to 57 years old, with an average age of 46 years old. A total of 72 % of the participants were male and 28 % of the participants were females.

4.3 Results

4.3.1 Learning achievement

In order to determine whether the participants could learn regardless of the quality level, we compared the pre and post participants answers to the questionnaire. We found that a statistical significant difference was obtained regardless of the mobile device when a confidence interval of 95 % was considered for statistical significance. This happens regardless of the mobile device or content type. More details (e.g. mean, standard deviation, t-value and p-value) are presented in Table 4 for the “small” screen device and Table 5 for the “large” screen device.

Table 4 Details regarding the statistical significant difference between the pre and post knowledge on the “small” screen device across different content categories
Full size table
Table 5 Details regarding the statistical significant difference between the pre and post knowledge on the ”large” screen device across different content categories
Full size table

4.3.2 Overall perceived video quality

A statistically significant difference was obtained between the participants’ perceived quality for the small video screen device and the large video screen device (p < 0.01, C.I. = 99 %, STD = 0.56). On average, the video quality for the small device was perceived as being of a lower quality (average MOS score 3.79) than the video quality on the larger device (average MOS score 4.5). On the small device the video quality is rated between a Poor video quality (2) and a Excellent video quality (5) and on the large device between a Fair video quality (3) and an Excellent video quality (5), depending on the content category (see Fig. 2 for the average MOS values obtained for each device depending on the video content category).

Fig. 2
figure 2

Average MOS values on the two mobile devices across different content types

Full size image

4.3.3 Content category

A statistically significant difference (see Table 6 for more details) between how the quality was rated on a smaller screen device and the one on a larger screen device was obtained for five of the seven categories: slideshow, screencast, presentation, documentary and animation. For all of these categories the quality on the larger devices was perceived as being better than the one on the smaller device (see Fig. 2 for the average MOS values on each device). For the other two categories, lab demo and interview, no statistically significant difference was obtained although also in this case, the quality on the smaller device was perceived as being lower than on a larger device.

Table 6 Details regarding the statistical significant difference between two different devices across different content categories
Full size table

It is worth mentioning that the quality of the slideshow recording on the video devices was rated poorly compared with other video categories (2.70 on average for the small device which is labelled lower than fair quality and 3.40 for the larger device which is slightly better than fair quality), regardless of the device used. Both the interview and the animation category was rated high across both mobile devices (both rated on average over 4 (good) regardless of the mobile device).

5 Discussion

This study has shown that the students can learn regardless of the content category used in the study. However, the MOS results show that the perceived quality differs depending on the content use. The findings are summarised in Table 7, where the second and the third column presents the average MOS scores for the small screen device, and large screen device respectively, and the last column shows whether or not the difference is statistical significant. Table 1 expands on the labels assigned to each of the MOS scales.

Table 7 Summary of video quality results
Full size table

The results show that for two of the video categories: lab demo and interview, the participants did not have a statistical significant difference in rating their perceived quality when using two videos of different qualities. This might suggest that these two types of video could be the most suitable when this kind of adaptation is to be performed in order to compensate for the mobile device limitations, network characteristics, or limited battery life. Ultimately the content chosen on a mobile learning course, would depend on the user needs and the subject taught [57] and not all the content could be expected to match these two categories. However, whenever possible, mobile content designers could take the results of this study into account when the provided mobile video content matches the above categories.

The results also show that the quality of video content categorised as slideshow was rated lower as opposed to the other categories on both devices. This is probably due to the fact that the slides contained text that makes is more difficult to see on mobile devices. Video content categorised as interview and animation was rated high across both mobile devices, which might be due to the low movement in the interview recording and quantization in the animation clip. Both these factors are known to affect video quality [54]. Outside of the educational domains, better perceived video quality has been obtained when degrading the video quality for cartoons (which have a similar content with animation) as compared with sport [45, 60] or panorama videos [60]. News type content, which is also similar in limited movement as an interview in our scenario, has found also to obtain better MOS scores than the sport and panorama videos when degraded [60].

This study also shows that participants could learn with the provided videos regardless of the adaptation and video category, which confirms previous results that have shown that it is possible to adapt the video content and not to affect learning achievements [32, 40].

The findings of this study also suggest that the results could be considered in different video adaptation mechanisms, either by the educational content providers or mobile network operators. Educational content providers might use them to improve either the perceived quality of the provided content or to reduce the traffic to their servers. For example, educational content that fits the interview or animation category could have its quality reduced more than other types of content without affecting the learner quality as much. This leads to a lower size video clip, and hence traffic to their server. This adaptation can also be provided optionally for the learners who access the educational content from a mobile network and may prefer to use a lower quality video content in order to save data [37].

Mobile network operators could use the results of this study if they wish to save bandwidth in an effort to avoid congestion or to increase the number of users served. They can provide adaptive video content that takes into account account how the different educational content categories could affect the learner experience and perform the adaptation based on them. Hence by using a more aggressive adaptive strategy on the three categories (lab demo, interview, and animation) that affect the learner experience the least and performing a more gentle adaptation on the other ones, they could save both the bandwidth but also to optimise the learner quality of experience.

There are also several limitations and possible future directions that emerge as a result of the study and need further exploration. The number of participants was well above the number considered by the ITU-T [47] necessary for video quality evaluation. Although this was done to ensure the validity of the study, recent research has shown that other factors (e.g. culture and personality [52]) could affect the perceived video quality. Therefore, determining whether these study results are still suitable including other factors and across different cultures is an avenue to be considered in the future research. Experimental approach followed in this study has the advantage of controlling external factors that could influence the results. However, students do not always study in controlled environment. The different context in which the learner study could affect the perceived video quality (e.g. different lightening or different devices from which the content is accessed).

This study provides guidelines on how the learner experience is affected by the adaptation of different video content types. It has shown that certain video content does not affect the learner perceived quality as much as the other ones. As such it provides a future research direction for video learning by looking whether there is a threshold after which the learner quality of experience or learning is affected by the adaptation. This could furthermore lead to best practices on how to design video learning material but also on benchmarking indicators.

6 Conclusions

This study assessed how different video educational content categories affect learning and perceived video quality. In order to do so we presented the results of adapting video quality to fit the characteristics of two different mobile devices. Videos from seven different educational content categories have been used in the study. The results have shown that the participants could learn regardless of the educational content type or of the adaptive video version used in the study. The results show that although the decrease in the video quality between the smaller device and the larger device was noticeable to the participants for most of the categories, for two of the video categories lab demo and interview, no statistically significant difference was obtained in perceived video quality between the two devices. We have also shown that although a statistically significant difference was obtained for the rest of the categories, interview and the animation category were rated high on the MOS scale, while the slideshow was rated low regardless of the mobile device used.