We made a static prototype (see Fig. 1) based on the lecture interface of XuetangX, which is one of the most popular MOOC platforms in China. We illustrated the interface in the following scenarios: (1) when the user enters the interface, (2) when the user posts a comment or thread, or takes a note, (3) when the user modifies the filter settings, (4) when the user goes see the detailed content of a thread, and (5) when the user switches the sidebar tabs to see his/her notes. To initially evaluate the design, explore how it can enhance interaction, and improve the design, we interviewed 4 teachers, 8 teaching assistants, and 7 learners.
The teachers and TAs were invited through XuetangX. We tried to interview instructors from diverse subjects. The 4 teachers from Tsinghua University taught Introduction to modern biology, C++ programming, Principle of Marxism, and Medical parasitology respectively in XuetangX. The 8 teaching assistants from Tsinghua University worked for the following courses in XuetangX: Financial analysis, C++ programming, Principle of circuits, Linear algebra, Great art, Zi Zhi Tong Jian (History as a Mirror). This is medical science, and Listening and speaking of daily English. The 7 learners were recruited from a WeChat group of active users of XuetangX and the researchers’ personal social networks. Their age ranged from 18 to 28 (M = 21.6, SD = 3.3). Most of them were students (from high school students to graduated students) except one who worked in a HR department. They had been learning MOOCs for from 2 months to 4 years. All of them had the experience of learning MOOCs on XuetangX.
All participants were interviewed in the quiet environments. Six of the seven students were interviewed by QQ (a video chatting tool). Other participants were interviewed face to face. First, we gathered the participant’s demographic information and MOOC experience. Then we introduced our purpose, showed the prototype of DanMOOC, and introduced the features. Then we interviewed him/her about (1) how DanMOOC may affect the interaction and learning performance, (2) how the flying overlaid comments may distract the learning, and (3) the effectiveness of notes. We also interviewed learners about their opinions of design details (e.g., how to present expanded content and replies of a thread). Each of the semi-structured interviews lasted about 30 min. All interviews were audio-recorded and transcribed into text for further analysis.
Most participants were satisfied with the system and they reported the score of satisfaction from 5 to 6 (maximum 7). Five of the seven students would like to try this system.
Interaction of Commenting System.
Most participants agreed that the connection of discussion and content at different video time points may enhance learner-content interaction, promote understanding, and increase learning efficiency. First, the connection may stimulate discussion. Two TAs said that students may miss many instant thoughts, questions or comments if they discuss after finishing viewing the lecture video. A teacher said the synchronized overlaid comments may create an atmosphere of discussion and attract more learners to join in. Second, the connection may make it convenient to discuss or ask questions about content at certain time points in the lecture video. TAs said this feature may reduce their work load of searching students’ questions in the lecture video. Interviewed learners said that these comments and threads provided helps and summaries of questions immediately when they learnt the concept, and therefore the repeatedly asked questions would be reduced. Third, the connection may enrich the lecture content. As TAs said, this synchronized commenting system may provide an entrance from the video to user generated content, expand the lecture content, and promote collaborative learning.
The commenting system may enhance learner-learner interaction. TAs said students may get to know others’ feelings, such as perceived difficulty. Some learners said they could share inspirations with others and the sense of accompany could be enhanced. They could even make new friends and further develop a sense of belonging to the class. Teachers said the overlaid flying comments can create a lively atmosphere similar to the face-to-face class. For example, the teacher of Medical parasitology said the learners would not be scared of the parasites with the company of comments. These merits may increase the completion rate. However, the TA of Linear algebra said that few learners needed to discuss in the Math class and it was hard to type formula immediately during class. Therefore, he thought it was hard to enhance learner-learner interaction.
The commenting system may enhance learner-instructor interaction. Teachers and TAs said that by the amount of comments and threads, they may figure out which parts of the video attracted or confused learners. The system could also help learners correct the mistakes of videos, and make it easier to ask or answer questions located in specific time points. Although the commenting system may provide an approach of learner-instructor communication, the major of participants said that instructors needed effective and efficient ways to learn students’ feedbacks from a potentially large scale of comments and threads. Useful information required to be abstracted and presented in a clear form.
Although the commenting system may enhance interaction, a few participants said that the interaction brought more harms than benefits. Some learners worried that the during class discussion would distract them from focusing on the video content. One teacher and one student said that although the connection of video content and discussion was helpful, the overlaid flying comments were distracting and they preferred other forms such as the list in sidebar. Some participants said the enhanced student-student interaction may increase useless information and therefore decrease the learning efficiency.
Most participants said too many comments overlaid on the screen would distract learning, therefore they appreciated the functions of filter, transparency setting, and hiding.
However, they still concerned about the quality of comments and the visual clutter. First, the commenting system may increase discussions both related and unrelated to the course. Most participants said content-related comments could benefit learning, but they had different opinions about other comments, such as comments for social interaction. Some of them said social comments could attract learners and create an active atmosphere, whereas others said social comments may provide useless and chaotic information for learning. Second, learners of MOOCs usually have different levels of knowledge and skills. Some interviewed learners worried that they were forced to see comments and threads that may be too easy or boring to them, even if the comments were related to the content. Third, in addition to the individual learning habits and knowledge levels, the form of the video also influenced the perceived distraction. Most students and TAs said if there were slides with dense texts in the video, the overlaid comments would be more distractive than that when a teacher was present and speaking in the video.
Only one of the seven learners used Evernote when learning MOOCs. Other learners took notes on paper notebooks when learning MOOCs. Therefore, although most participants said the notes may be convenient, they may not want to use it. The teacher of Medical parasitology also said paper was more suitable for notes of Medical parasitology. The Evernote user in the seven learners and some teachers and TAs said learners may try to take notes in DanMOOC system if they could export their notes organized in a clear layout from the DanMOOC system, for reviewing after class.
Teachers and TAs said that notes may reflect learners’ learning process. For example, playback time points with many notes may indicate that the content there was difficult to many learners. Teachers could find out unexpected difficult points of the lecture and adjust his/her teaching.
In the prototype, the detailed content and following replies of each thread were shown in the area D in Fig. 1. Most participants said that this skip from the view of video to the bottom of the page would interrupt the learning ofthe lecture. In addition, in the most current MOOCs, forum is independent from the lecture videos. Few courses provided the function of discussing a specific video. Therefore, we removed the area for threads to the whole video, and presented the detailed content and following repliesof threads anchored to the timeline directly in the sidebar (see Fig. 5).