Keywords

1 Introduction

Nowadays, the Web and its technologies have revolutionized our vision to deliver courses and learning models to students. As one of the contributions of the Web in education, the Massive Open Online Courses (MOOCs) generate a new digital learning environment, which provide open learning for large scale and support learners to study at their own pace with instructional video clips. Although these online courses and the digital environment show their popular, they are still far away from delivering effective education strategies. One of the issues is on detecting student’s cognitive states (One step further, the affective states) and supplying adaptive instruction in such digital environments. In a real-world classroom, an instructor easily observes the cognitive state of students and adjusts the speed and contents of the lecture. This strategy regains students’ interest and engagement in learning. Current MOOCs platforms do not support cognitive states recognition and miss the adaptive presentation of videos and other learning materials.

Recently, increased researches have focused on physiological measures on cognitive states, among which electroencephalography (EEG) based brain activity measures are able to monitor in a real-time, compared with self-reported assessment and learning performance measures. One current focus on cognitive states is to enhance student engagement, including measuring features, classifying attention levels, and modelling. In the work [2], the passive brain-computer interfaces are leveraged to enhance user engagement and learn how to better deliver the best reading experience. Szafir and Mutlu [3] design and build a system in which a robotic agent informs of real-time measurements of student attention obtained from EEG data employed cues that human instructors use to recapture student attention when it declines. However, in the learning context, the engagement drops possibly due to student’s lack of interest, possibly due to the inappropriate workload levels. The cognitive workload that students experience while learning instructional materials determines success in learning. With the workload real-time assessment tool, the instructor could observe whether the adjustment of strategies is successful when decreasing or increasing the workload levels of student by optimizing the instructions, using more cases to explain, etc. In digital unsupervised environment, the system embedded with detection modules could provide automatic adaption and personalized learning paths for students. Therefore, in the learning context, it is crucial to monitor and analyze the cognitive workload.

In this work, we design and propose a real-time passive Brain-Computer Interaction (BCI) system to continuously monitor the cognitive workload using EEG-based wireless headset Emotiv Epoc+, which is feasible for working in the online digital environment like Massive Open Online Courses (MOOCs). We choose two electrodes to pick up original EEG signals, which are highly relevant to the workload. The current prototype is able to record EEG signals and classify levels of cognitive load when students watching online course videos. This prototype is based on two layers, using machine learning approaches for classification.

2 Related Work

2.1 Cognitive Workload

The cognitive load theory (CLT) built by Sweller [4] indicates that learning happens best under conditions that are in line with learner’s cognitive structure. In this theory, cognitive load refers to the total amount of metal effort being used in the individual cognitive system in a specific working period. The intensity and type of cognitive load that students experience while learning instructional materials determines success in learning. Sweller classified the workload into three types [5], including intrinsic cognitive load (ICL), extraneous cognitive load (ECL) and germane cognitive load (GCL), based the sources of the workload. Even though the nature of cognitive load is unclear and the workload type cannot be observed directly, through assessing workload we could adjust the load to an expected level by the materials difficulty (related to ICL), the representation and forms (related to ECL) and the cases (related to GCL). Brunken et al. [6] classified various methods of assessing cognitive load along two dimensions: objectivity (subjective and objective) and causal relation (indirect and direct). Along to the time when to detect the workload, these measures could be regarded roughly as real-time and non-real-time. Physiological measures often provide a real-time detection, like EEG-based BCI, heart rate, eye movements, etc. Self-reported assessment and learning performance (learning outcome measures, dual-task performance) are usually conducted before or after tasks.

2.2 Brain-Computer Interface on Workload Recognition

Due to its high temporal resolution, EEG is an appropriate tool to record brain activities and recognize patterns in complex cognitive tasks in learning. EEG headsets can be classified casually as medical caps and portable caps. Usually, the medical cap have 128 channels and expensive. It requires a specific environment or lab to conduct the experiment, and well-trained technicians to operate the device. Each procedure takes at least half an hour to prepare the device. Even though the medical cap performs with more accuracy and has more channels, the system using this cap is impractical for a wide use. On the contrary, the portable cap is cheaper and simpler to use.

As shown in Table 1, we survey and list several existing EEG-based cognitive workload recognition systems. The workload is measured by using the improved method in study [7], which has investigated memory workload in the n-back task using wireless EEG signals. This work was based on the Proximal Support Vector Machine (PSVM) algorithm, using signal power features, statistical features, morphological features, and time-frequency features, to distinguish high, medium and low levels. Honal and Schultz [8] employed SVM and Artificial Neural Networks (ANNs) for classification of EEG signals on workload in lecture and meeting scenarios (Table 1).

Table 1. Existing EEG-based cognitive workload recognition systems
Full size table

From these studies [9,10,11,, 12–14], results indicated that increased memory load was associated with increased theta band power in the frontal midline area by Gevins et al. and other researchers. Besides, alpha band activity changes have been observed in the studies of detecting memory load.

3 EEG-Based Brain-Computer Interface to Assess Cognitive Workload

3.1 System Structure

This passive BCI system is designed for using in a ubiquitous environment, which supports monitoring when student learning online at any place. It consists of two layers: the offline layer for training the classifier and the online layer for real-time recognition of workload levels. As shown in Fig. 1, with regards to both layers, the process is based on the machine learning approaches and includes raw EEG signals recording from headset, artifacts removal, feature extraction, and classification. The implementation of machine learning approaches typically requires training. Then the learning model is used for test or prediction. The parameters are trained and accomplished in the offline layer. We will discuss the components in the subsections.

Fig. 1.
figure 1

The passive BCI system architecture for recognizing cognitive load levels in the context of online courses learning

Full size image

3.2 Apparatus

In this work, we leverage the Emotiv Epoc+ to record EEG signals and its SDK to develop related processing models (see Fig. 2). The Epoc+ is low-cost and ubiquitous, which has been proved to be accuracy and feasibility to access cognitive activities. This wireless acquisition device connects to a computer via a USB and records the raw EEG data via a headset. The headset features 14 channels plus 2 references based on the 10–20 format, with a resolution of 128 samples/s. Its low cost, lightness and channels makes it appropriate and competent to work in online learning environment. The Emotiv SDK allows to program to log EEG signals and train for cognitive states recognition. It supports C++ and other common languages like Python, Matlab. In this work, we use Matlab to build and test the offline system. Then we develop the online system in C++.

Fig. 2.
figure 2

The Emotiv Epoc + headset

Full size image

3.3 EEG Data Recording and Denoising

We adapted two electrodes to gather raw signals for gaining a fast processing, including F3 and F4 (see Fig. 1), with two reference channels A1 and A2. F3 and F4 are frontal channels. These two channels are highly related to the cognitive workload. As aforementioned, the theta and alpha waves reflect the variation of workload. Therefore, F3 and F4 are selected for recording EEG.

The raw EEG data are mixed with noises that impact the signal analysis and classification. These noises, namely the artifacts, are generated from the outside of the brain, including eye and body movements, and electronic inference. To remove artifacts, independent component analysis (ICA) has been proved as a feasible method, which attempts to decompose the brain signal into independent non-Gaussian signals, determine the noisy signals and reconstruct the brain signals by excluding those artifacts. However, ICA is highly user dependent and requires operating manually by experts to some extent. Therefore, FastICA and other extended ICA algorithms are used more often in many studies. In our context, we leveraged an automatic ICA-based ADJUST algorithm [15]. ADJUST identifies and removes artifact-related components based on the combined use of spatial and temporal features. In this way, we eliminate the confounding elements introduced by the motor activities.

3.4 Transformation and Feature Extraction

To monitor the workload, it requires obtain features related to theta and alpha. Epoc + headset samples at a rate of 128 Hz and has a bandwidth in the range of 0.2–43 Hz, which could be resolved to delta (0.5–4 Hz), theta (4–8 Hz), alpha (8–13 Hz), and beta (13–30 Hz) waves. The Fast Fourier Transforms (FFT) are used to process the cleaned EEG data and transformed it to theta and alpha bands.

Since the linear feature extraction requires less time to calculate, we employ 6 linear features, that is, mean absolute amplitude, mean square, variance, activity, mobility and complexity. The mean absolute amplitude, mean square and variance are statistical features, which are calculated as below:

$$ {\text{Mean absolute amplitude}} = {\text{E}}\left( {\text{x}} \right) = \frac{1}{\text{N}}\sum\nolimits_{i = 1}^{N} {\left| {x_{i} } \right|} $$
(1)
$$ {\text{Mean square}} = {\text{E}}\left( {{\text{x}}^{ 2} } \right) = \frac{1}{\text{N}}\sum\nolimits_{i = 1}^{N} {x_{i}^{2} } , $$
(2)
$$ {\text{Variance }} = {\text{ E}}\{ ({\text{x}} - {\text{E}}\left( {\text{x}} \right))^{ 2} \} . $$
(3)

The latter three features are based on Hjorth parameters. Hjorth parameters [16] was proposed by Bo Hjorth in 1970, indicating the statistical properties used in signal processing. Activity parameter represents the signal power in frequency domain. It is calculated by the Eq. (3) as below:

$$ {\text{Activity }} = {\text{ var}}\left( {y(t)} \right). $$
(4)

Where y(t) refers to the signal. Mobility is defined as the mean frequency or the proportion of standard deviation of the power spectrum:

$$ {\text{Mobility }} = \sqrt {\frac{{{\text{var}}(y'(t))}}{{{\text{var}}(y(t))}}} . $$
(5)

Complexity parameter indicates the similarity of the signal to a pure sine wave:

$$ {\text{Complexity}} = \frac{mobility(y '(t))}{mobility(y(t))}. $$
(6)

Hjorth parameters require less time and resources to calculate. To sum up, we have 24 features, counting from 6 features for 2 frequency bands and 2 channels.

3.5 Classification

In our study, the sample size is smaller than the feature numbers. Therefore, we employ the Support Vector Machine (SVM) [17] to classify EEG data, based on the statistical learning theory. SVM algorithm has a fast processing ability. As one of the machine learning approaches, it has shown promising empirical results in many practical applications, including text categorization, classification of images, handwritten digit recognition, biological science, etc. The basic idea behind SVM is to find out a maximal margin hyperplane to make the classification. In addition to performing linear classification, SVM could efficiently perform a non-linear classification using the kernel trick. Concerning the kernel function, we employ the linear kernel, since from previous researches, it performs better with EEG data than the polynomial and the Gaussian radial basis function kernel (RBF).

With regards to the classifier in our current prototype, we have two cognitive load level: high level and low level. The high level is related to the video contents requiring more effort to understand. The low level is related to the contents that are easy to understand and master. Therefore, the ground truth labels are gained from the difficulty levels of instruction materials.

4 Experiment and Discussion

We develop our passive BCI system and train it in a lab environment. This system is built based on a laptop, with Windows 10 and 8.00G RAM, a headset of Emotiv Epoc+, and two monitor screens. As shown in Fig. 3, in our experiment, the tasks that subjects perform are watching the clips of online courses videos. Each of the clip are made at about 3 to 4 min. The meta process includes preparing, watching and doing the questionnaire. After the training, the performance and self-report questionnaire is asked to fill to verify the difficulty levels of the video materials. In this training experiment, we build the learning model and training parameters for the prototype.

Fig. 3.
figure 3

The process of training.

Full size image

5 Conclusion and Future Work

This study is motivated due to missing studies on passive BCI system in online courses learning context. In this preliminary work, we design and propose a real-time passive Brain-Computer Interaction (BCI) system to continuously monitor the cognitive workload using EEG-based wireless headset Emotiv Epoc+, which is feasible for working in the online digital environment like Massive Open Online Courses (MOOCs). We choose two electrodes to pick up original EEG signals and decompose the signal to theta and alpha bands. The current prototype is able to record EEG signals and classify levels of cognitive load when students watching online course videos. This prototype is based on two layers, using machine learning approaches for classification.

In our future work, we are going to verify the classifier at first. Then we will seek for more features to increase the accuracy of this BCI system. Finally, it is interesting to know how to distinguish the adjustment of intrinsic cognitive load, extraneous cognitive load and germane cognitive load based on our BCI approach.