Soft Computing

, Volume 23, Issue 21, pp 11199–11215 | Cite as

Indoor space target searching based on EEG and EOG for UAV

  • Tianwei ShiEmail author
  • Hong Wang
  • Wenhua Cui
  • Ling Ren
Methodologies and Application


This paper puts forward a noninvasive electrooculography (EOG) and electroencephalogram (EEG)-based hybrid computer interface (HCI) system to implement the indoor target searching in three-dimensional (3D) space for a low-speed multi-rotor aircraft. The HCI system mainly consists of three subsystems, including the interface switching, decision and semi-autonomous navigation. The interface switching subsystem is accomplished by detecting the eyeblink EOG. The continuous wavelet transform is employed to indentify eyeblink features which are used to switch interfaces between horizontal and vertical motor imagery (MI) tasks. The average accuracy of the eyeblink feature detection reaches to 97.95%. The decision subsystem employs the joint regression (JR) model and spectral powers methods to extract the time and frequency domain features of MI tasks by analyzing the left- and right-hand MI EEG. Simultaneously, the support vector machine is applied to accomplish the MI tasks classification and final decision. The average classification accuracy of the HCI system reaches to 93.99%. The semi-autonomous navigation subsystem extracts the environmental features to avoid obstacles semi-automatically in 3D space and provide feasible directions for the decision subsystem. The actual indoor 3D space target searching experiments are put forward to verify the feasibility and adaptation performances of this proposed HCI system.


Electrooculography Electroencephalogram Motor imagery Continuous wavelet transform Joint regression model Spectral powers Support vector machine 



Tianwei Shi has been supported by the Department of Education of Liaoning Province (2017FWDF03), Natural Science Foundation of Liaoning Province of China (20180550567) and University of Science and Technology Liaoning Youth Fund (2017QN05). Wenhua Cui has been supported by the Department of Education of Liaoning Province (2016HZZD05).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animal rights

This article contains the calibration experiments and actual indoor 3D space target searching experiments with human participants performed by the authors. Five males and five females participated in the experiments. This study was approved by the Human Research Protections Program of University of Science and Technology Liaoning and Northeastern University. Simultaneously, it was performed in accordance with the Declaration of Helsinki. All participants were asked to read and sign an informed consent form before participating in the study.


  1. Ahn JW, Ku Y, Kim DY, Sohn J, Kim JH, Kim HC (2018) Wearable in-the-ear eeg system for ssvep-based brain–computer interface. Electron Lett 54(7):413–414CrossRefGoogle Scholar
  2. Beniczky S, Conradsen I, Henning O, Fabricius M, Wolf P (2018) Automated real-time detection of tonic-clonic seizures using a wearable emg device. Neurology 90(5):428–434CrossRefGoogle Scholar
  3. Blankertz B, Mller KR, Curio G, Vaughan TM, Schalk G, Wolpaw JR, Schloegl A, Neuper C, Pfurtscheller G, Hinterberger T, Schroeder M, Birbaumer N (2004) The BCI competition 2003: progress and perspectives in detection and discrimination of EEG single trials. IEEE Trans Biomed Eng 51(6):1044–1051CrossRefGoogle Scholar
  4. Bulling A, Ward JA, Gellersen H, Troster G (2011) Eye movement analysis for activity recognition using electrooculography. IEEE Trans Pattern Anal Mach Intell 33(4):741–753CrossRefGoogle Scholar
  5. Chau AL, Li X, Yu W (2014) Support vector machine classification for large datasets using decision tree and Fisher linear discriminant. Future Gener Comput Syst 36:57–65CrossRefGoogle Scholar
  6. Choi H, Kim Y (2014) Uav guidance using a monocular-vision sensor for aerial target tracking. Control Eng Pract 22(1):10–19CrossRefGoogle Scholar
  7. Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  8. Fattahi D, Nasihatkon B, Boostani R (2013) A general framework to estimate spatial and spatio-spectral filters for EEG signal classification. Neurocomputing 119:165–174CrossRefGoogle Scholar
  9. Fu K, Qu JF, Chai Y, Dong Y (2014) Classification of seizure based on the time-frequency image of EEG signals using HHT and SVM. Biomed Signal Process Control 13:15–22CrossRefGoogle Scholar
  10. Gandolfi M, Formaggio E, Geroin C, Storti SF, Galazzo IB, Bortolami M, Saltuari L, Picelli A, Waldner A, Manganotti P, Smania N (2018) Quantification of upper limb motor recovery and eeg power changes after robot-assisted bilateral arm training in chronic stroke patients: a prospective pilot study. Neural Plast 1:1–15CrossRefGoogle Scholar
  11. Ghasemi JB, Heidari Z, Jabbari A (2013) Toward a continuous wavelet transform-based search method for feature selection for classification of spectroscopic data. Chemometr Intell Lab Syst 127:185–194CrossRefGoogle Scholar
  12. Ghayab HA, Li Y, Siuly S, Abdulla S (2018) Epileptic EEG signal classification using optimum allocation based power spectral density estimation. IET Signal Proc 12(6):1–13 Google Scholar
  13. Heo J, Yoon H, Park KS (2017) A novel wearable forehead EOG measurement system for human computer interfaces. Sensors 17(7):1485CrossRefGoogle Scholar
  14. Hu S, Tian Q, Cao Y (2013) Motor imagery classification based on joint regression model and spectral power. Neural Comput Appl 23:1931–1936CrossRefGoogle Scholar
  15. Huang Q, He S, Wang Q, Gu Z, Peng N, Kai L, Zhang Y, Shao M, Li Y (2017) An eog-based human–machine interface for wheelchair control. IEEE Trans Biomed Eng 99:1Google Scholar
  16. Jung TP, Humphries C, Lee TW, Makeig S, McKeown MJ, Iragui V, Sejnowski TJ (1998) Extended ICA removes artifacts from electroencephalographic recordings. Adv Neural Inf Process Syst 10:894–900Google Scholar
  17. Kar S, Bhagat M, Routray A (2010) EEG signal analysis for the assessment and quantification of driver’s fatigue. Transp Res Part F Traffic Psychol Behav 13(5):297–306CrossRefGoogle Scholar
  18. Karson CN, Berman KF, Donnelly EF, Mendelson WB, Kleinman JE, Wyatt RJ (1981) Speaking, thinking, and blinking. Psychiatry Res 5(3):243–246CrossRefGoogle Scholar
  19. Kim BH, Kim M, Jo S (2014) Quadcopter flight control using a low-cost hybrid interface with eeg-based classification and eye tracking. Comput Biol Med 51(15):82–92CrossRefGoogle Scholar
  20. Lanillos P, Gan SK, Besada-Portas E, Pajares G, Sukkarieh S (2014) Multi-uav target search using decentralized gradient-based negotiation with expected observation. Inf Sci 282:92–110MathSciNetCrossRefGoogle Scholar
  21. Li J, Yan J, Liu X, Ouyang G (2014) Using permutation entropy to measure the changes in EEG signals during absence seizures. Entropy 16(6):3049–3061CrossRefGoogle Scholar
  22. López A, Ferrero F, Yangüela D, Álvarez C, Postolache O (2017) Development of a computer writing system based on EOG. Sensors 17(7):1505CrossRefGoogle Scholar
  23. Lotte F, Guan C (2011) Regularizing common spatial patterns to improve BCI designs: unified theory and new algorithms. IEEE Trans Biomed Eng 58(2):355–362CrossRefGoogle Scholar
  24. Lotte F, Congedo M, Lecuyer A, Lamarche F, Arnaldi B (2007) A review of classification algorithms for EEG-based brain–computer interfaces. J Neural Eng 4(2):1–13CrossRefGoogle Scholar
  25. Lührs M, Goebel R (2017) Turbo-satori: a neurofeedback and brain–computer interface toolbox for real-time functional near-infrared spectroscopy. Neurophotonics 4(4):041504CrossRefGoogle Scholar
  26. Ma J, Zhang Y, Cichocki A, Matsuno F (2015) A novel EOG/EEG hybrid human–machine interface adopting eye movements and ERPS: application to robot control. IEEE Trans Bio Med Eng 62(3):876–889CrossRefGoogle Scholar
  27. Minati L, Yoshimura N, Koike Y (2016) Hybrid control of a vision-guided robot arm by EOG, EMG, EEG biosignals and head movement acquired via a consumer-grade wearable device. IEEE Access 4:9528–9541CrossRefGoogle Scholar
  28. Moghadamfalahi M, Akcakaya M, Nezamfar H, Sourati J, Erdogmus D (2017) An active RBSE framework to generate optimal stimulus sequences in a BCI for spelling. IEEE Trans Signal Process 65(20):5381–5392MathSciNetCrossRefGoogle Scholar
  29. Nguyen QX, Jo S (2012) Electric wheelchair control using head pose free eyegaze tracker. Electron Lett 48:750–752CrossRefGoogle Scholar
  30. Orfanus D, Freitas ED, Eliassen F (2016) Self-organization as a supporting paradigm for military UAV relay networks. IEEE Commun Lett 20(4):804–807CrossRefGoogle Scholar
  31. Ramli R, Arof H, Ibrahim F, Mokhtar N, Idris MYI (2015) Using finite state machine and a hybrid of EEG signal and EOG artifacts for an asynchronous wheelchair navigation. Expert Syst Appl 42(5):2451–2463CrossRefGoogle Scholar
  32. Riccio A, Leotta F, Bianchi L, Aloise F, Zickler C, Hoogerwerf EJ, Kübler A, Mattia D, Cincotti F (2011) Workload measurement in a communication application operated through a P300-based brain–computer interface. J Neural Eng 8(2):025–028CrossRefGoogle Scholar
  33. Salvo G, Caruso L, Scordo A (2014) Urban traffic analysis through an UAV. Procedia—Soc Behav Sci 111:1083–1091CrossRefGoogle Scholar
  34. Samek W, Vidaurre C, Müller KR, Kawanabe M (2012) Stationary common spatial patterns for brain–computer interfacing. J Neural Eng 9(2):026013CrossRefGoogle Scholar
  35. Shi T, Wang H, Zhang C (2015) Brain computer interface system based on indoor semi-autonomous navigation and motor imagery for unmanned aerial vehicle control. Expert Syst Appl 42(9):4196–4206CrossRefGoogle Scholar
  36. Siebert S, Teizer J (2014) Mobile 3D mapping for surveying earthwork projects using an unmanned aerial vehicle (UAV) system. Autom Constr 41:1–14CrossRefGoogle Scholar
  37. Tuna G, Nefzi B, Conte G (2014) Unmanned aerial vehicle-aided communications system for disaster recovery. J Netw Comput Appl 41:27–36CrossRefGoogle Scholar
  38. Varela G, Caamaño P, Orjales F, Deibe Á, López-Peña F, Duro RJ (2014) Autonomous UAV based search operations using constrained sampling evolutionary algorithms. Neurocomputing 132:54–67CrossRefGoogle Scholar
  39. Vourvopoulos A, Badia SBI, Liarokapis F (2017) EEG correlates of video game experience and user profile in motor-imagery-based brain–computer interaction. Vis Comput 33(4):533–546CrossRefGoogle Scholar
  40. Wu W, Gao X, Hong B, Gao S (2008) Classifying single-trial EEG during motor imagery by iterative spatio-spectral patterns learning (ISSPL). IEEE Trans Biomed Eng 55(6):1733–1743Google Scholar
  41. Xu B, Song A, Wu J (2007) Algorithm of imagined left-right hand movement classification based on wavelet transform and AR parameter model. In: 1st international conference on bioinformatics and biomedical engineering, pp 539–542Google Scholar
  42. Yahyanejad S, Rinner B (2015) A fast and mobile system for registration of low-altitude visual and thermal aerial images using multiple small-scale UAVs. ISPRS J Photogramm Remote Sens 104:189–202CrossRefGoogle Scholar
  43. Youn E, Koenig L, Jeong MK, Baek SH (2010) Support vector-based feature selection using Fisher’s linear discriminant and support vector machine. Expert Syst Appl 37(9):6148–6156CrossRefGoogle Scholar
  44. Zhang C, Wang H, Fu R (2014) Automated detection of driver fatigue based on entropy and complexity measures. IEEE Trans Intell Transp Syst 15(1):168–177CrossRefGoogle Scholar
  45. Zhao Z, Luo H, Song GH, Chen Z, Lu ZM, Wu X (2018) Web-based interactive drone control using hand gesture. Rev Sci Instrum 89(1):014707CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of International Finance and BankingUniversity of Science and Technology LiaoningAnshanChina
  2. 2.Department of Mechanical Engineering and AutomationNortheastern UniversityShenyangChina
  3. 3.Research and Development CenterLiaoning Systemteq Information Technology Co., Ltd.AnshanChina

Personalised recommendations