Electroencephalography as implicit communication channel for proximal interaction between humans and robot swarms
- 1.3k Downloads
Search and rescue, autonomous construction, and many other semi-autonomous multirobot applications can benefit from proximal interactions between an operator and a swarm of robots. Most research on proximal interaction is based on explicit communication techniques such as gesture and speech. This study proposes a new implicit proximal communication technique to approach the problem of robot selection. We use electroencephalography (EEG) signals to select the robot at which the operator is looking. This is achieved using steady-state visually evoked potential (SSVEP), a repeatable neural response to a regularly blinking visual stimulus that varies predictively based on the blinking frequency. In our experiments, each robot was equipped with LEDs blinking at a different frequency, and the operator’s SSVEP neural response was extracted from the EEG signal to detect and select the robot without requiring any conscious action by the user. This study systematically investigates several parameters affecting the SSVEP neural response: blinking frequency of the LED, distance between the robot and the operator, and color of the LED. Based on these parameters, we study two signal processing approaches and critically analyze their performance on 10 subjects controlling a set of physical robots. Our results show that despite numerous artifacts, it is possible to achieve a recognition rate higher than 85 % on some subjects, while the average over the ten subjects was 75 %.
KeywordsHuman–robots interaction EEG SSVEP Emotiv EPOC Thymio robot
Many thanks to Dr. Ricardo Chavarriaga, Dr. Claire Braboszcz, and Dr. Serafeim Perdikis for the constructive discussions about experiments involving EEG; to Dr. Jérôme Scherer and Prof. Marco Picasso for their help on mathematical issues in the signal processing; to the reviewers who contributed with detailed and constructive comments during the submission process; and to all subjects who were available for the experiments. This work was partially supported by the Swiss National Center of Competence in Research “Robotics.”
- Akhtar, A., Norton, J. J., Kasraie, M., & Bretl, T. (2014). Playing checkers with your mind: An interactive multiplayer hardware game platform for brain–computer interfaces. In 36th Annual international conference of the IEEE engineering in medicine and biology society (EMBC), IEEE (pp. 1650–1653).Google Scholar
- Aljshamee, M., Mohammed, M. Q., Malekpour, A., Luksch, P., et al. (2014). Beyond pure frequency and phases exploiting: Color influence in SSVEP based on BCI. Computer Technology and Application, 5(2), 111–118.Google Scholar
- Beverina, F., Palmas, G., & Silvoni, S. (2003). User adaptive BCIs: SSVEP and P300 based interfaces. PsychNology, 1(4), 331–354.Google Scholar
- Bonani, M., Rétornaz, P., Magnenat, S., Bleuler, H., & Mondada, F. (2012). Physical interactions in swarm robotics: The hand-bot case study. In A. Martinoli, F. Mondada, N. Correll, G. Mermoud, M. Egerstedt, M. A. Hsieh, L. E. Parker, & K. Støy (Eds.), Distributed autonomous robotic systems, Springer tracts in advanced robotics (pp. 585–595). Berlin: Springer.Google Scholar
- Cao, T., Wan, F., Mak, P. U., Mak, P. I., Vai, M. I., & Hu, Y. (2012). Flashing color on the performance of SSVEP-based brain–computer interfaces. In Annual international conference of the IEEE engineering in medicine and biology society (EMBC), IEEE (pp. 1819–1822).Google Scholar
- Chua, F., Daftari, A., Alvarez, T., DeMarco, R., Bergen, M., Beck, K., & Servatius, R. (2004). Effects of a single green flash versus a white flash of light on saccadic oculomotor metrics. In Proceedings of the IEEE 30th annual northeast bioengineering conference, IEEE (pp. 9–10).Google Scholar
- Couture-Beil, A., Vaughan, R., & Mori, G. (2010). Selecting and commanding individual robots in a vision-based multi-robot system. In 5th ACM/IEEE international conference on human–robot interaction (HRI) (pp. 355–356). doi: 10.1109/HRI.2010.5453167.
- Faller, J., Müller-Putz, G., Schmalstieg, D., & Pfurtscheller, G. (2010). An application framework for controlling an avatar in a desktop-based virtual environment via a software SSVEP brain computer interface. Presence: Teleoperators and Virtual Environments, 19(1), 25–34. doi: 10.1162/pres.19.1.25.CrossRefGoogle Scholar
- Goodrich, M. A., Pendleton, B., Kerman, S., & Sujit, P. (2013). What types of interactions do bio-inspired robot swarms and flocks afford a human? In Proceedings of robotics science and systems VIII (pp. 105–112).Google Scholar
- Güneysu, A., & Akin, H. L. (2013). An SSVEP based BCI to control a humanoid robot by using portable EEG device. In International conference of the IEEE engineering in medicine and biology society, (Vol. 2013, pp. 6905–6908). doi: 10.1109/EMBC.2013.6611145.
- Hvaring, F. T., & Ulltveit-Moe, A. H. (2014). A comparison of visual evoked potential (VEP)-based methods for the low-cost emotiv EPOC neuroheadset. Technical report, Norwegian University of Science and Technology, NTNU—Trondheim, Trondheim, Norway. http://www.diva-portal.org/smash/get/diva2:751718/FULLTEXT01.pdf.
- Jacobs, G. (2013). SSVEP-based BCI control for navigating a robot. Technical Report. December, Faculteit der Sociale Wetenschappen, Radboud University, Nijmegen, The Netherlands. http://www.ru.nl/publish/pages/641151/jacobs_gj_ba-th-2013.pdf.
- Jian, H. L., & Tang, K. T. (2014). Improving classification accuracy of SSVEP based BCI using RBF SVM with signal quality evaluation. In 2014 International symposium on intelligent signal processing and communication systems (ISPACS), IEEE (pp. 302–306).Google Scholar
- Jones, G., Berthouze, N., Bielski, R., & Julier, S. (2010). Towards a situated, multimodal interface for multiple UAV control. In 2010 IEEE international conference on robotics and automation (ICRA) (pp. 1739–1744). doi: 10.1109/ROBOT.2010.5509960.
- Kirchner, E. A., de Gea Fernandez, J., Kampmann, P., Schröer, M., Metzen, J. H., & Kirchner, F. (2015). Intuitive interaction with robots—technical approaches and challenges. In R. Drechsler & U. Khne (Eds.), Formal modeling and verification of cyber-physical systems: 1st International summer school on methods and tools for the design of digital systems, Bremen, Germany, September 2015, Springer Fachmedien Wiesbaden, Wiesbaden (pp. 224–248). doi: 10.1007/978-3-658-09994-78.
- Liu, Y., Jiang, X., Cao, T., Wan, F., Mak, P. U., Mak, P. I., & Vai, M. I. (2012). Implementation of SSVEP based BCI with emotiv EPOC. In IEEE international conference on virtual environments human–computer interfaces and measurement systems (VECIMS), IEEE (pp. 34–37).Google Scholar
- Martinoli, A., Mondada, F., Correll N., Mermoud, G., Egerstedt, M., Hsieh, M. A, Parker, L. E., & Støy, K. (2012). 10th International symposium on distributed autonomous robotic systems (DARS) (Vol. 83). Springer.Google Scholar
- Monajjemi, V., Wawerla, J., Vaughan, R., & Mori, G. (2013). HRI in the sky: Creating and commanding teams of UAVs with a vision–mediated gestural interface. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 617–623). doi: 10.1109/IROS.2013.6696415.
- Mouli, S., Palaniappan, R., Sillitoe, I. P., & Gan, J. Q. (2013). Performance analysis of multi-frequency SSVEP-BCI using clear and frosted colour led stimuli. In Bioinformatics and bioengineering (BIBE), 2013 IEEE 13th international conference on, IEEE (pp. 1–4).Google Scholar
- Nagi, J., Giusti, A., Gambardella, L. M., & Di Caro, G. (2014). Human–swarm interaction using spatial gestures. In IEEE/RSJ international conference on intelligent robots and systems (IROS 2014) (pp. 3834–3841). doi: 10.1109/IROS.2014.6943101.
- Ortner, R., Guger, C., Prueckl, R., Grünbacher, E., & Edlinger, G. (2010). SSVEP based brain–computer interface for robot control. In International conference on computers for handicapped persons, Springer, Vol. 6180 LNCS (pp. 85–90).Google Scholar
- Pourmehr, S., Monajjemi, V., Vaughan, R., & Mori, G. (2013). “You two! Take off!”: Creating, modifying and commanding groups of robots using face engagement and indirect speech in voice commands. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 137–142). doi: 10.1109/IROS.2013.6696344.
- Renard, Y., Lotte, F., Gibert, G., Congedo, M., Maby, E., Delannoy, V., et al. (2010). OpenViBE: An open-source software platform to design, test, and use brain–computer interfaces in real and virtual environments. Presence: Teleoperators and Virtual Environments, 19(1), 35–53. doi: 10.1162/pres.19.1.35.CrossRefGoogle Scholar
- Riedo, F., Chevalier, M., Magnenat, S., Mondada, F. (2013). Thymio II, a robot that grows wiser with children. In IEEE workshop on advanced robotics and its social impacts (ARSO), IEEE (pp. 187–193). doi: 10.1109/ARSO.2013.6705527.
- Rubenstein, M., Ahler, C., & Nagpal, R. (2012). Kilobot: A low cost scalable robot system for collective behaviors. In IEEE international conference on robotics and automation (ICRA), IEEE (pp. 3293–3298).Google Scholar
- Rzepecki, J., Delcourt, J., Da Silva, M. P., & Le Callet, P. (2012). Virtual interactions: Can EEG help make the difference with real interaction? In Proceedings of the 2012 IEEE international conference on multimedia and expo workshops, ICMEW 2012 (pp. 151–156). doi: 10.1109/ICMEW.2012.33.
- Salehuddin, M., Suprijanto, S., & Muchtadi, F. I. (2011). Prototype design of low cost four channels digital electroencephalograph for sleep monitoring. In 2nd International conference on instrumentation control and automation (ICA) (pp. 188–193). doi: 10.1109/ICA.2011.6130154.
- Stawicki, P., Gembler, F., & Volosyak, I. (2016). Driving a semiautonomous mobile robotic car controlled by an SSVEP-based BCI. Computational Intelligence and Neuroscience. doi: 10.1155/2016/4909685.
- Stytsenko, K., Jablonskis, E., & Prahm, C. (2011). Evaluation of consumer EEG device emotiv EPOC. In MEi: CogSci conference 2011, Ljubljana.Google Scholar
- Trovato, G., Zecca, M., Sessa, S., Jamone, L., Ham, J., Hashimoto, K., et al. (2013). Cross-cultural study on human–robot greeting interaction: Acceptance and discomfort by egyptians and japanese. Paladyn, Journal of Behavioral Robotics, 4(2), 83–93. doi: 10.2478/pjbr-2013-0006.CrossRefGoogle Scholar
- Van Vliet, M., Robben, A., Chumerin, N., Manyakov, N. V., Combaz, A., & Van Hulle, M. M. (2012). Designing a brain–computer interface controlled video-game using consumer grade EEG hardware. In Biosignals and biorobotics conference (BRC), 2012, IEEE (pp. 1–6).Google Scholar
- Volosyak, I., Cecotti, H., & Graser, A. (2009). Optimal visual stimuli on LCD screens for SSVEP based brain–computer interfaces. In Neural Engineering (pp. 447–450).Google Scholar
- Wu, C. H., & Lakany, H. (2013). The effect of the viewing distance of stimulus on SSVEP response for use in brain–computer interfaces. In IEEE international conference on systems, man, and cybernetics (SMC), IEEE (pp. 1840–1845).Google Scholar
- Yanco, H. A., & Drury, J. L. (2004). Classifying human–robot interaction: An updated taxonomy. In IEEE international conference on systems, man and cybernetics, SMC (pp. 2841–2846). doi: 10.1109/ICSMC.2004.1400763.
- Zheng, Y., & Zhang, Y. (2010). An improved segmented match filters with FFT approach for GNSS signal acquisition. In ICCTD 2010—Proceedings of the 2nd international conference on computer technology and development, ICCTD (pp. 425–428). doi: 10.1109/ICCTD.2010.5645833.