1 Introduction

The direct transfer of information between the human brain and the environment are known as brain-computer interfaces (BCIs) [1]. In other words, BCIs evaluate changes in the brain's electrical activity in response to external stimuli or user intents and transform them directly into output commands for controlling a device or application. [2] [3]. Motor imagery, steady-state visual evoked potentials (SSVEPs), P300 and event-related potentials (ERP) are the EEG control signals utilized in the BCI system [4,5,6]. It is clear that MI techniques often yield impressive results when individuals have been properly trained. For a targeted user to handle an MI-based BCI, the targeted user requires several training trails, making the calibration time unacceptable to a realistic model [7]. P300 produces larger average ITRs and does not require training, but it significantly affects the severity and variability of disease performance.. Nonetheless, several investigations have also shown that even individuals with LIS are able to tolerate a BCI based on P300 for an extended period of time. However, in relation to ITR, in nearly all of the earlier results concerning P300-BCI, a healthy individual generates higher bit rates than disabled subjects [8]. The stimulation method is so complicated that patients can not carry out the experimental process. In addition, a large variety of instructions in a BCI system based on P300 enhances the number of trails, which in turn lessens the overall outcomes [7].

The SSVEP could be a significant alternative in the development of effective BCI [9]. It is the normal response from the brain with various frequencies generated due to visual stimulation [10]. In short, when a person looks at the light with a specific frequency, their visual cortex reacts at the same frequency with EEG activity. BCIs use the concept as a gaze-tracking method: several symbols appear on a screen for the user, with a different frequency blinking each symbol. The SSVEP frequency can be calculated by electrodes near the visual cortex to assess which symbol the user is looking at. This may be achieved either in a single stage (the ultimate command is chosen directly from all possible ones), or in multiple stages (a subset of commands is first selected from all possible ones, and the final specific command is then selected from the subset) depending on the number and complexity of the possible commands. SSVEPs are widely employed to send commands to a device in biomechatronics. The user is shown with multiple commands on the screen (e.g., move the robot forward, stop) and chooses one by looking at it. Additionally, the user can choose for not delivering a command by not focusing on the screen. The technology is noninvasive, and quite easy to utilize with little or no training, and the number of possible commands can be extremely large. The main drawbacks are that the symbols must be kept enough apart on the display so that the user does not perceive two flashing lights at the same time and that the various symbols must be sufficiently dissimilar in frequency to be distinguished in the EEG. Due to the improved SNR faster ITR, the purpose of SSVEP-based BCIs is increasing considerably [11]. The key drawback of the SSVEP approach is that the interface must be supplemented with a screen that cannot be configured in all circumstances (for instance, portable devices). It is also likely to be falsely positive because users still see the screen on the side of their view, even when they don't want the device to be controlled [12].

In SSVEP-based BCI systems via PC instruments, the frequency-coding technique has been commonly utilized. Cheng et al. [13] introduced a 12-command (0–9, BACKSPACE and ENTER) frequency-coding SSVEP-oriented BCI telephone framework and a monitor is utilized as a stimulator. Wang et al. [14] also developed to boost information transfer rate (ITRs) for an SSVEP-based frequency coding BCI method based on optimization of channel location. In addition, the SSVEP-based BCI keypad method was employed by Jia et al. [15]. This method consists of a visual stimulator with a light-emitting diode (LED), an EEG signal collection unit and a PC. A frequency coded SSVEP system to control the electric hand prothesis with two axes having four LED stimulators mounted to a prosthesis is another application reported by Müller-Putz and Pfurtscheller [16]. Cecotti [17] introduced an SSVEP-based BCI speller with a self-paced and without calibration in their more recent work. On the LED screen of this speller, there are five flashing command boxes, three of which are used to choose characters, one of which deletes them, and another which returns to the previous action. In recent, SSVEP BCI was implemented to control hand orthosis in the work of Pfurtscheller et al. [18] and Ortner et al. [12]. They also applied two LEDs flashing with two separate frequencies (8 and 13 Hz) as the flickering source to make the system more realistic. However, they did not use a monitor. Note that LED flickers are restricted in their number with a frequency coding. In addition, the EEG signal was calculated using an instrument. This study develops a complete design of a BCI control nursing system for hospital beds in order to make a practical BCI system. This proposed system involves a range of techniques, including (1) frequency with phase coding LED stimulation panel for enhancing the flicker number (2) analogue bio-signal amplifier circuits for SSVEP signal measurement; (3) digital signal processing Field Programmable Gate Array (FPGA); and (4) power electronic H-Bridge motor drives to adjust the attitude of the hospital bed. Therefore, the system contains no PC and/or commercial instruments. Please note that eight LED's are required to evoke SSVEP in the case of the hospital bed care system and massage cushion. In this case, two LED flickers having frequency coding with two different frequencies [18, 12] do not meet the requirement. Consequently, the work designs a total of eight LEDs in combination with the phase coding technique with two different frequencies [19, 20, 21]. Thus, two flickering options could be extended to eight. SSVEP is not as big a biosignal as electrocardiography or electromyography. In the atmosphere of noise and power disturbances, it is not easy to measure SSVEP if it doesn't use the commercial EEG measuring instrument. The SSVEP signal processor is realized by hand-coding with the high-speed integrated circuit-description hardware (VHDL) language and is not used in commercial equipment. Shyu et al. [22] developed an FPGA-based BCI system in which SSVEP-amplifier / filter and ADC circuits can be acquired and quantified. In addition, this research is developing an FPGA to execute the SSVEP signal processor that simplifies the entire BCI framework.

Xing J. et al. [23] proposed a convolutional neural network (CNN) based comparing network that was employed to attain the connection between EEG data and the SSVEP stimulus frequencies. Erkan E. et al. [5] developed a BCI robot control application based on SSVEPs where they utilized minimum energy combination (MEC) and canonical correlation analysis (CCA) techniques are applied to EEG segments containing several signals window lengths for identifying the SSVEPs.

Spuler et al. [24] proposed an averaging method for c-VEP based BCI where multiple trials are described in a row, and the signal is averaged over multiple epochs. A dynamic stopping approach was used to collect epochs until a certain stopping criterion is achieved. The author utilized a technique of correlation, too, since it is more appropriate to stopping dynamically. Lo et al. [25] developed a wireless BCI-control system with SSVEP utilizing a non-contact metal plate electrode and an FFT method. The system’s accuracy and the best ITR were 91.1% and 38.28 bits/min, respectively, while classifying 12 targets. Nguyen T. H. et al. [26] developed SSVEP based speller system where the SSVEP responses were captured through a bipolar single-channel EEG. For SSVEP frequency identification, 1-D CNN was utilized by the authors. A CNN model as a robust SSVEP frequency classifier has been implemented by another work showed in [27]. The study takes a two-dimensional map of SSVEP data as an input (channels x frequencies) to identify up to five SSVEP frequencies with an EEG headset having multi-channel. A novel SSVEP BCI speller for numerical input was employed in [28] in the field of real-time spelling tasks using a sliding control protocol. To boost frequency recognition of SSVEP, Demir et al. [29] developed bio-inspired filter banks (BIFBs). The BIFBs were built to account for the inherent biological SSVEP features, namely frequency selectivity, subject specificity, and harmonic SSVEP responses. They have been used to enhance the class separability in the feature extraction phase.

Among the studies published in the field of BCI-based environment control systems, the maximum accuracy of 96.92% was achieved by Yang et al. [30]. Another study in [31] authors achieved an accuracy of 94.17% with a response time of 5.2 s when two commands were generated utilizing four channels. Most of the above studies can not be referred to as the optimum methods in terms of classification accuracy, the number of commands, ITR and user-friendliness. Hence, a more friendly system is essential with a high performance and various control commands. This study proposed a hybrid BCI-based environment control system that provides 18 control commands through SSVEP and EMG signals. SSVEP signals operate as selection signals, whereas single eye blinks serve as confirmation signals. This study utilizes the common spatial pattern (CSP) to extract the frequency domain feature from the raw signals. Then, the extracted feature has been classified using the k-nearest neighbour (k-NN). The experimental coutcomes indicate that the proposed system could be utilize in daily life for home automation control with a robust performance. The proposed approach provides a novel strategy for a BCI controlled system. Additionally, the proposed approach could provide the utilization of the home facility conveniently for the disable people.

This paper is organized as follows: the paper is initiated by the introduction. The second section discusses the details of experimental design and methods. The results have been achieved in this study are presented in the third section, followed by the discussion and conclusion in the final two sections (Fig. 1).

Fig. 1
figure 1

General architecture of SSVEP based BCI environment control system

Full size image

2 Experimental design and methods

Nowadays, the researchers have carried a wide range of BCI studies (off-line and online manner), and the principle architecture of these studies is almost identical. Besides the variation of EEG control signals, all BCI systems are fundamentally made up of signal acquisition, signal pre-processing, feature extraction, classification and device commands. The present study was initiated by the data collection phase, where subject selection, equipment set-up and protocol design have been carried out. The data collection process was approved by Universiti Malaysia Pahang Research Ethics Committee, Malaysia. In the second phase, the artefact from the collected SSVEP response was removed and then the time window was selected. The feature in the form of CSP was extracted from the artefact free SSVEP data. The extracted feature was classified using the k-NN classifier. The classifier performance was assessed using a wide range of performance evaluation metrics. Finally, a control mechanism that aims to apply for the environmental control system has also been developed. The proposed system can identify 18 commands (i.e., 16 control commands using SSVEP and two commands using EMG. Figure 2 illustrates the complete procedure of the proposed study.

Fig. 2
figure 2

Complete procedure of the proposed study

Full size image

2.1 Experimental set-up and data acquisition

In this experiment, the data acquisition device was a biosignal amplifier (Bioradio, Great Lakes NeuroTechnologies, USA). The complete set-up of BioRadio has been illustrated in Fig. 3. The BioRadio system utilizes electrodes on the scalp and forehead to monitor the average behaviour of millions of brain cells or neurons. This device is capable of capturing ECG, EEG, EOG, EMG and respiration. The device consists of 10 electrodes, and among these electrodes, one electrode is used for ground electrode, and another electrode is utilized for reference. The remaining eight electrodes are employed to capture the physiological signal. In this study, the time series of the electrical brain activity is picked up utilizing eight gold cup EEG electrodes. The electrodes are placed at predefined locations PZ; PO3; PO4; O1; O2; OZ; O9 and FP2 illustrated in Fig. 2. Between electrodes and the scalp, the EEG conductive paste is applied. Electrodes are positioned near the occipital region according to the international 10–20 system. The sampling frequency is fixed at 250 Hz. To capture the data smoothly, software, namely BioCapture™ has been utilized. This software combines the innovative bioinstrumentation hardware, and transducers are combined with a flexible software platform for data visualization, review, annotation, collection and analysis. During data collection, the BioRadio and BioCapture have been paired through Bluetooth illustrated in Fig. 3. Then the EEG data has been collected as a CSV file.

Fig. 3
figure 3

Equipment set-up for data collection

Full size image

During the data acquisition experiment, a subject was asked to focus on the flickering on four different square boxes (100 × 100 pixel), which are stimulated by 6.66 Hz' up', 7.50 Hz' right', 8.57 Hz'down' and 10 Hz'left' on the ultra HD screen (1920 × 1080 resolution). The most influential SSVEP stimulation frequencies lie in between 6.5 and 17.14 Hz [32], 33. The subject is informed by a voice command about the target which has to look (left, right, up, down). Subjects are seated in front of the graphical user interfaces (GUI) screen at a distance of about 50 cm. The GUIs are designed in the MATLAB 2019a by using Psychtoolbox. Figure 4 shows the complete scenery of the data acquisition process where a subject was worn all electrodes and focused on the GUI.

Fig. 4
figure 4

Photograph of the complete data collection system

Full size image

A total of 50 sessions have been conducted during the whole data collection process. Each session lasts 480 s, and in this period, five runs have been conducted. The duration for each run is 55 s when four different SSVEP stimuli have been visualized with a certain time gap illustrated in Fig. 5. Thus, from one session, 20 trials of SSVEP responses have been captured, and the duration of each trial is five seconds.

Fig. 5
figure 5

Demonstration of the time window for the data acquisition system

Full size image

Four SSVEP responses have been evoked from 6.66, 7.50, 8.57 and 10 Hz stimuli. The dataset consists of 1000 trials where each type of SSVEP response has an equal number of trials tabulated in Table 1. In this study, eight healthy subjects with normal or corrected-to-normal vision have been performed. Table 1 listed details about all subjects in terms of age, sex, session and trials. Among all subjects, three of them had previous experience about how to stimulate SSVEP responses. The other five subjects were totally new to the SSVEP-based BCI. Before the experiment, each participant had to read and sign an informed consent document. This study was approved by Universiti Malaysia Pahang Research Ethics Committee, Malaysia.

Table 1 Details about subjects and data collection
Full size table

2.2 Common spatial pattern

Among the widespread feature extraction methods of EEG, this study utilized the CSP approach, which is widely used for feature extraction. Due to the fact that the CSP approach gives the spatial filters, it increases the variation of one class while minimising the variance of the other [34], 35. The distinctions between two classes may be computed using conventional CSP, a multi-class paradigm addition to the original approach in [36].

The CSP algorithm and the two-class problem feature set derivative has been described in this section. For Class 1 and Class 2, K1 and K2 are the estimates of the covariance matrices, respectively. The CSP algorithm uses the two covariance matrices K1 and K2 simultaneously to diagonalize. This can be done mathematically by solving the problem of the eigenvalue decomposition (see Eq. 1).

$$K_{1} P = (K_{1} + K_{2} ){\text{PD}}$$
(1)

where \(P\in {S}^{L\times L}\) is the CSP projection matrix, which yields the features whose variances are optimal for distinguishing two classes of EEG measurements. The rows of W are stationary spatial filters, and the common spatial patterns can be achieved from the rows of \({P}^{-1}D\) that is a diagonal matrix consisting of the eigenvalues of \({K}_{1}\). For each ith trial of the multichannel EEG signal, \({F}_{i}= {S}^{L\times N}\) are translated to a low-dimensional subspace with a projection matrix, \(P.L\) presents the number of EEG channels and the number of samples per channel is denoted by N. The linear transformation of the ith trial is as following Eq. (2),

$$Z_{i} = PF_{i}$$
(2)

where \({Z}_{i}\in {S}^{L\times N}\) represents the spatially filtered signals, which maximize the difference in the variance of two classes of EEG signals. A small subset of L spatially filtered signals is basically utilized for the feature vector information [23], 25. The subset selection is based on m pairs of the first and last rows of \({Z}_{i}\). Let \({Z}_{f}\in {R}^{2m\times N}\) be the first and last rows of Zi, the variance of \({Z}_{f}\) forms the feature vector for the ith trial (see Eq. 3),

$$V_{i} = {\text{log}}\left( {\frac{{var\left( {Z_{f} } \right)}}{{\mathop \sum \nolimits_{i = 1}^{2m} var\left( {Z_{f} } \right)}}} \right)$$
(3)

where \({V}_{i}\in {S}^{2m}\) In the present study, we use \(m=2\) for all experimental data sets.

The multi-class extension to the binary CSP technique initially presented [37] divides the multi-class problem into many multi-class problems.

The multi-class extension incorporates the divide-and-conquer (DC) strategy, the one-versus-all (OVR) strategy, CSP inside the classifier, and pair-wise schemes. [38], 39. We apply the OVR technique to multi-class issues by dividing the k-class problem into a collection of k binary classes and distinguishing each class from the remaining classes. The binary CSP algorithm is used to determine the characteristics that distinguish one class from the others.

Then, the features are concatenated in Eq. (4) to make the feature vector for the ith trial:

$$F_{i} = \left[ {V_{i}^{1} V_{i}^{2} \ldots \ldots V_{i}^{Lc} } \right]^{N}$$
(4)

where \({F}_{i}\in {S}^{1\times \left(2m\times Lc\right) }, {V}_{i}^{k}\in {S}^{2m}\) represents the binary class features for the kth class versus the rest for the ith trial, Lc is presented as the number of classes in the multi-class problem, and the transpose matrix is represented by superscript N.

2.3 K-Nearest Neighbour

The k-NN technique is built on the concept that the features corresponding to the various classes form distinct clusters in feature space. The characteristics that are in close proximity to one another are referred to as neighbours and are therefore grouped together. [40]. KNN algorithms are not widely used in categorising EEG data, most likely because they are considered extremely susceptible to the curse of dimensionality [41]. However, when used in EEG classification with low-dimensional feature vectors, k-NN may prove to be efficient [41]. However, the effectiveness of the KNN classifier depends on some hypermeters of kNN, including the number of neighbors (k), distance functions and weight. To find the most favourable value of K for any particular task classification is still very challenging. However, some characteristics of excellent k-value are- (1) The value of K must be an odd value, (2) Initialize a random K value and start computing, (3) Choosing a small value of K. The distance between points in a feature space has been measured using different distance functions. There are certain other distance functions available for KNN classification, such as Minkowski distance, Euclidean distance, Manhattan distance. The weight may be uniform or distant.

Among these hypermeters, selecting the optimum parameter is a crucial task, and the performance is highly depending on the hyperparameter setting. To select the optimum hyperparameters, we have used the GridSearchCV function, which is built-in scikit-learn. For the given values, GridSearchCV exhaustively evaluates all the possible combinations of parameter values and suggests the best combination. Using this method, we have selected the optimum hyperparameters of k-NN as: the value of k is 9; the distance function is Euclidean; the weight is distance. The Euclidean distance function is calculated using Eq. (5)

$${\text{Distance}} \left( {A,B} \right) = \sqrt {\frac{{\mathop \sum \nolimits_{i = 1}^{m} \left( {x_{i} - y_{i} } \right)^{2} }}{m}}$$
(5)

where, Distance (A, B) is the distance between points A and B in a feature space and m denotes the dimensionality of the feature space.

2.4 Control mechanism of proposed system

The proposed hybrid system consists of SSVEP and EMG data. The SSVEP data is used to select the target, whereas the EMG data is used to confirm or ignore the selected target. Figure 6 shows the complete control mechanism of the proposed hybrid system. The SSVEP data in terms of every two seconds of the time window (i.e. 500 samples) is considered for the selection of each target. After feature extraction, the extracted feature is classified using k-NN. After classification, a pop-up window is appeared to confirm the selected target. Here, the pop-up window will ask the user whether the classifier identifies the correct target or not. To confirm the target selection, the EMG signal in terms of eye blinking has been utilized. To confirm, the user blinks once more or twice to return to the previous menu. When the system detects a single-eye blink, it indicates that the individual has verified their choices. If the system detected a double-eye-blink signal, it considered that the recognised instruction was inconsistent with the subject's purpose. After that, the system would revert to its previous interface and continue gaze control over the stimulus targets. After selecting the appropriate command, participants might then exercise more control over the home automation system.

Fig. 6
figure 6

Complete architecture of control mechanism of proposed hybrid system

Full size image

The proposed BCI-based environment control system's interface was designed using a 4 × 4 categorical system. To begin, the user selects one of four primary categories. The categories were designed based on the most frequent everyday tasks to simplify and improve the user experience. Figure 7 illustrates the major categories, including robotic arm control, wheelchair control, family contact, and home appliance control. Each category is further split into four subcategories, each of which contains the common tasks associated with it. For example, to control the robotic arm, four control commands, including forwarding, backward, right and left, are generated using four SSVEP responses. Users may choose an option by staring at a flicker and confirm their choice with a single eye blink. To revert to the previous menu after making a selection, a double blink is used. Users are provided with pop-up windows for each chosen result. Throughout the trial, all four stimulus buttons were concurrently presented. Users were instructed to choose menu items by directing their attention to the appropriate button.

Fig. 7
figure 7

Interface for the target selection menu

Full size image

2.5 Performance evaluation

The classification accuracy, precision, sensitivity, specificity, recall, F1-score and MCC are utilized to evaluate the performance of this study. The equations of these metrics are mentioned in Eq. (6)–(12).

$${\text{CA}} = \frac{{{\text{TP}} + {\text{TN}}}}{{{\text{TP}} + {\text{FN}} + {\text{TN}} + {\text{FP}}}} \times 100\%$$
(6)
$${\text{Sensitivity}} = \frac{{{\text{TP}}}}{{{\text{TP}} + {\text{FN}}}} \times 100\%$$
(7)
$${\text{Specificity}} = \frac{{{\text{TN}}}}{{{\text{TN}} + {\text{FP}}}} \times 100\%$$
(8)
$${\text{Precision}} = \frac{{{\text{TP}}}}{{{\text{TP}} + {\text{FP}}}} \times 100\%$$
(9)
$${\text{Recall}} = \frac{{{\text{TP}}}}{{{\text{TP}} + {\text{FN}}}}$$
(10)
$$F1 = \frac{{2*{\text{Precision}}*{\text{Recall}}}}{{{\text{Precision}} + {\text{Recall}}}}$$
(11)
$${\text{MCC}} = \frac{{{\text{TP}}*{\text{TN}} - {\text{FP}}*{\text{FN}}}}{{\sqrt {\left( {{\text{TP}} + {\text{FP}}} \right)\left( {{\text{TP}} + {\text{FN}}} \right)\left( {{\text{TN}} + {\text{FP}}} \right)\left( {{\text{TN}} + {\text{FN}}} \right)} }}$$
(12)

where TP = true positive, FN = false negative, TN = true negative and FP = false positive.

In order to estimate the performance of BCI systems, bit rate or information transfer rate (ITR) are most frequently used. Equation (13) presented an equation that is utilized to estimate ITR (in bits/min) [2],

$${\text{ITR}} = (\log_{2} N + P\log_{2} P + \left( {1 - P} \right)\log_{2} \left\{ {\frac{{\left( {1 - P} \right)}}{{\left( {N - 1} \right)}}} \right\} \times \left( \frac{60}{t} \right)$$
(13)

where N denotes the number of classes, P presents the target identification accuracy, and t (seconds/selection) is the average time for a selection.

3 Experimental results

The experimental results of this study have been presented in terms of the performance of the signal processing methods and device command hardware prototype. To obtain the best signal window length, the classification accuracy and ITR of the proposed signal processing approach has been assessed in different signal window lengths. The variation of classification accuracy and system ITR values with respect to the signal window length for each subject are tabulated in Table 2. Moreover, this variation is also graphically illustrated in Fig. 8. Although the signal window lengths for each trial was five seconds, only three seconds of signal window length has been considered during data analysis to achieve higher ITR. The ITR of the proposed approach has been supposed to be superior at 1.5 s of signal window lengths for all subjects. At 1.5 s signal window lengths, the highest ITR (83.12 bit/min) has been obtained by subject-7, whereas the lowest ITR (76.09 bit/min) has been achieved by subject-6. For classification accuracy, the opposite scenery has been observed. In the case of subject-1, subject-2 and subject-3, the highest accuracy of 100% has been recorded at 2 s of the time window. At 3 s signal window lengths, the best classification accuracy has appeared for the remaining subjects. The average ITR of all subjects at the time window of 2, 2.5 and 3 s are 75.18 ± 3.63 (bit/min), 65.84 ± 2.71 (bit/min) and 52.83 ± 1.78 (bit/min). The average classification accuracy of all subjects at the time window of 2 s, 2.5 s and 3 s are 97 ± 2.77, 97.39 ± 2.97 and 98.73 ± 1.86. The differences in classification accuracy in the time window of 2, 2.5 and 3 s are very low. Hence, based on the classification accuracy and ITR listed in Table 2, we have selected a two-second time window for further analysis.

Table 2 Accuracy and ITR of all subjects in different time window lengths
Full size table
Fig. 8
figure 8

Accuracy and ITR of all subjects in different time windows

Full size image

The achieved two traditional BCI performances, accuracy and ITR, has been averaged for all subjects across various signal window lengths and are shown in Fig. 8. The mean accuracy ranges significantly from 69.25% to 98.73% over the entire signal window lengths. In this case, the classification accuracies change proportionally with the signal window lengths. The average accuracies of all subjects are 69.25 ± 2.60, 84.42 ± 2.21, 92.98 ± 3.64, 97 ± 2.77, 97.39 ± 2.97 and 98.73 ± 1.86 at the signal window lengths of 0.5, 1.0, 1.5, 2.0, 2.5 and 3.0 s, respectively. Average ITR also varies considerably with the entire signal window lengths and range from 52.83 (bit/min) to 78.99 (bit/min). The ITRs change proportionally from 0.5 to 1.5 s of the signal window lengths. However, the ITRs change inversely proportional to the signal window lengths after 1.5 s. The average ITRs of all subject are 71.54 ± 3.89 (bit/min), 76.61 ± 2.61 (bit/min), 78.99 ± 3.78 (bit/min), 75.18 ± 3.63 (bit/min), 65.84 ± 2.71 (bit/min) and 52.83 ± 1.78 (bit/min) at the signal window lengths of 0.5, 1.0, 1.5, 2.0, 2.5 and 3.0 s respectively.

We have also analyzed the classifier performance using the combination of all subject’s trials. In this experiment, we have used four different flickering frequencies (6.66, 7.50, 8.57 and 10.00 Hz). During labelling, 6.66, 7.50, 8.57 and 10.00 has been denoted by 1, 2, 3 and 4, respectively. The datasets consist of 1000 observations where class-1, class-2, class-3, and class-4 have 250 observations. The performance of the classifier has been evaluated using the holdout validation technique. In the holdout validation method, the training and testing ratio was 70:30. After dividing the entire data set into training and testing, the number of test trials are 69, 81, 107 and 90 for class-1, class-2, class-3, and class-4. These trials have been utilized to test the trained model. Figure 9 demonstrates the confusion matrix, in which the 347 observations (out of 338 testing observations) have been recognized accurately. In this experiment, we finally achieved is 97.41% testing accuracy. There are some factors that prevented boosting up the classification accuracy to 100%. Some subjects failed to stimulate the flickering frequencies 7.50 and 8.57 Hz, that is, class- 2 and class-3, properly. In class-2 and class-3, the four trials have been misclassified. However, the performance for class-1 and class-4 are very high. All trials for class-1 are accurately identified, whereas only one trial is misclassified in the case of class-4. Besides classification accuracy, we have also assessed the classifier performance using different performance metrics. These performance evaluation metrics including sensitivity, specificity, kappa, precision, F1-score, MCC, AUC. Figure 10 shows the performance of the k-NN classifier in terms of different performance evaluation metrics. The value of sensitivity, specificity, precision, F1-score, MCC, kappa and AUC are 0.9755, 0.9915, 0.9731, 0.9739, 0.9656, 0.9308 and 0.9899 respectively.

Fig. 9
figure 9

Confusion matrix of the testing trial

Full size image
Fig. 10
figure 10

Performance of k-NN classifier in terms of different metrics

Full size image

The proposed system has been tested in a real-time manner by all subjects. Every subject has been performed five trials, and they were asked to stimulate four SSVEP responses to select 16 targets under the main menu and submenu. Moreover, all the subjects were also asked to perform single blink and double blink to confirm and ignore the target selection. Table 3 tabulates the performance of the proposed real-time hybrid environment control system. All subjects except subject-3 and subject-8 performed accurately to operate the hybrid environment control system. The subject-3 and subject-8 have made three misidentifications among 30 targets. Subject-1, subject-2, subject-3 subject-8 made a single mistake among 30 targets, indicating excellent performance in real-time. Subject-4 and Subject-5 have made two mistakes among thirty times which also indicates comparatively good performance.

Table 3 Performance of proposed real-time SSVEP-based environment control system
Full size table

In summary, eight subjects performed 40 trials where each trial consists of 30 control commands. Hence, eight subjects perform a total of 240 control commands. Among 240 controlling commands systems in a real-time manner, the hybrid environment control system has made only 14 wrong recognition and 226 right recognition. This performance is very encouraging and could be implemented in a real BCI environment control system.

4 Discussion

When a patient with neurological disorders needs to turn on or off their home appliances such as lights, air-conditioners, television or other important services, they require support from other people. If no one is around, their needs cannot be easily provided. The above devices must be controlled either manually or with a remote. Thus, impaired patients are unable to function independently, which raises the strain on nursing assistants. Therefore, patients with disabilities cannot work independently, which raises the workload on nursing assistants. An innovative environment control system is designed to allow disabled patients to manage BCI domestic devices to improve the above problem.

We have developed a control mechanism for a hybrid BCI-based environment control system in this study. The proposed system may generate 18 control commands to operate the assistive applications. Among 18 commands, 16 commands are generated through SSVEP response and the remaining two commands are generated using eye blinking. The proposed CSP-KNN approach has classified the SSVEP responses with an accuracy of 97.41%. The proposed architecture may provide an innovative way for physically impaired people to operate different assistive devices utilizing SSVEP and eye blinks. It enables the user to accomplish regular everyday activities utilising just their brain impulses, obviating the need for physical movement.

In reality, BCI's versatility, accessibility and usability must be significantly enhanced in order to construct BCI practical devices for a broad range of users with real-world communication limitations. In order to achieve these aims for BCI, high ITRs and classification accuracy are crucial [42]. As a result, we have evaluated the proposed system's output and then compared it to the output of state-of-the-art environmental control systems in terms of ITR and accuracy. An asynchronous BCI P300 virtual environmental control technique for handling household devices have been documented by Aloise et al. [43]. In the corresponding experiment, the average accuracy of four patients having chronic neurological conditions is 73.7%, with 0.225 false positives (FPs) per minute. With the eleven healthy subjects, we have achieved obtained an average accuracy of 87.5% and a rate of 1.73 FPs per minute in a household automation control experiment that included an asynchronous P300-based BCI. In these two systems, the P300 potential was evoked by a classical row and column pattern. A multifunctional environmental control system, using an asynchronous P300 BCI for smart home management (for example, light control, fan monitoring, radio and multimedia player), cognitive rehabilitation and Web access (for example, web surfing and emailing) was documented by Miralles et al. [44]. In the six-week evaluation, 61 and 72% of the experimental activities were carried out by two subjects with serious brain injuries, respectively. The explanation why the tasks were not completed was because the decision-making threshold has not been achieved. A study [45] suggested a BCI-based environmental control system for event-related potential (ERP) incorporating electrical home equipment, a nursing bed, and an intelligent wheelchair to help paralyzed patients with serious spinal cord injuries (SCI's) on a daily basis. Average accuracy was reached by 89.6% and an average FPs of 0.97 / minute. In Guger et al. [46], 47, the BCI method, which provides a stimulus to the subject and records his response as an input, is noninvasive as well as subject-dependent P300 stimulus oriented. A GUI with different icons that reflect several tasks such as lighting on / off, windows and TV channels opening etc. belonged to the user. This method achieved 30% accuracy in the worst-case scenario with 12 subjects. At the same time, one of the subjects reached 100% accuracy in the best-case scenario. Kim et al. [48], suggested another P300 BCI method for switching TV channels with eight subjects getting a visual stimulus by flashing a green cursor in the upper-left corner of each channel icon. If the channel has been reached on the desired channel, it was considered to be an input for channel selection with a peak in neural signals of the subject. The system achieved an average accuracy of 92.3%. In reference to [22], the average accuracy was recorded 92.5% based on an SSVEP BCI based nursing bed system with 15 healthy subjects.

In the presented study, the highest mean ITR and classification accuracy for all subjects is 78.99 (bit/min) and 98.73%, respectively. Furthermore, 83.12 bit/min was the highest ITR recorded in subject-7. Similarly, the highest classification accuracy of 100% was recorded for subject subject-1, subject-2, subject-4 and subject-6. However, the overall accuracy and ITR for the entire dataset are 97.41% and 77.52 bit/min, respectively. The validation performance of the proposed system in a real-time manner was also very encouraging. Among 240 control commands in a real-time manner, accurate operation of the proposed system has been recorded in 226 commands. Hence, the accuracy of the proposed system in a real-time manner is 94.16%. Unlike conventional BCI systems, the proposed system of environmental control has many advantages, such as (1) a wearable EEG acquisition device with a limited number of electrodes, lightweight and easy setup (2) the flexible operation of the control mechanism of the proposed hybrid BCI-based environmental control system.

Although the performance of some subjects is very impressive, the overall accuracy of the proposed system has not been met 100%. The causes may be induced by lower SNR because of the individual variation caused by the amplitude of SSVEP, stimulus frequency, and electrodeposition. It was reported by Jia et al. [19] that the position of the reference electrode affects the achieved SSVEP. The experimental findings indicate that certain subjects can constantly and clearly evoke the SSVEP, whereas others cannot. Wang et al. [14] and Pastor et al. [49] suggested that with respect to the same flicker frequency, everyone has a different SSVEP response. Thus, it is almost impossible to select a flicker frequency for evoking the same SSVEP response. Therefore, it is necessary to construct an algorithm with all the same SSVEP responses to each flicker. It is essential to remember that this system realization is completely online; that is, before the online experiment, no calibration is done. Moreover, the problems of poor performance for subjects 3, 5 and 8 can be solved by addressing such a calibration session and recommending changing the stimulating frequencies of these subjects. Changing the stimulating frequencies may be suggested for these subjects. A further solution could be the use of the BCI wizard [50] that can automatically define and optimize essential parameters in various BCI methods, not just for a single BCI paradigm.

The traditional BCI performance measurement in bit/min is directly proportional to time and the number of targets. Two ways to gain a higher ITR are also available: (1) reducing the time needed for single command classifications and (2) increasing the numbers of identifiable targets (generally easier). There is no restriction to the standard formula in [2]; it is possible to measure the ITR if only one command passes through the communication channel. With the experimental protocol containing several targets and the aim of classifying only one of them, a high ITR could be achieved after this initial selection, even if it takes a comparatively long time to generate this control. The complexity of calculation ITRs is not new. In a recent study, the mean times for command classification (known as CTI—command transfer interval) has been employed by Shyu et al. [51]. In real-life applications, BCI utilizers with different impairments often requires to apply 1 bit of information at the right time to launch an autonomous sequence.

The proposed framework is simulated in the virtual environment. The real environment control system’s online operation may show slightly lower accuracy due to the lousy human mental state and the signal transmission issue between the controlling commands device and the EEG device. While the physically impaired patients are intended users of BCI technology, experimental data of the proposed research have been obtained from healthy subjects. However, authors in [52] conducted an experiment to control a BCI-based smart home where the users operated four toggle actions. They believed that physically impaired end-users have good motivation and incentives to utilize BCI technology, and they may handle this technology more accurately than the healthy end-users. According to their findings, disabled and healthy people achieved 81 and 77% accuracy, respectively.

5 Conclusion

The study proposes a hybrid real-time BCI based environment control system to assist the physically impaired patients using SSVEP responses and EMG. By avoiding manual operation, patients could easily use the proposed environmental control system by focusing on the flickering visual stimulation generator control icons and generating eye blinking. The proposed study consists of two modules: analyzing SSVEP data and offline assessment of the proposed controlling mechanism. Some factors that affect efficiencies, such as training procedures, human factors, the time behaviour of the complete system, and error recognition and error correction at the early stages of the BCI signal processing, should be considered in future research. In addition, some subjects require a longer time to build up SSVEP energy. Experimental findings indicate that certain subjects can consistently and clearly evoke SSVEP while certain subjects cannot do it. The future study will explore this phenomenon too. The findings of this study have demonstrated the feasibility of the proposed BCI to provide environment controlling assistance in real-life applications with high accuracy and practicality for disabled people. Further research will be conducted to explore the limits of the number of SSVEP frequencies that the proposed BCI can detect to expand its applications in a reasonably high-performance manner.