Skip to main content
SpringerLink
Log in

Exploring the Recognition of Facial Activities Through Around-The-Ear Electrode Arrays (cEEGrids)

  • Conference paper
  • First Online:
Information Systems and Neuroscience (NeuroIS 2021)

Part of the book series: Lecture Notes in Information Systems and Organisation ((LNISO,volume 52))

Included in the following conference series:

Abstract

NeuroIS scholars increasingly rely on more extensive and diverse sensor data to improve the understanding of information system (IS) use and to develop adaptive IS that foster individual and organizational productivity, growth, and well-being. Collecting such data often requires multiple recording devices, which leads to inflated study cost and decreased external validity due to greater intrusion in natural behavior. To overcome this problem, we investigated the potential of using an around-the-ear electrode array capable of capturing neural and cardiac activity for detecting an additional set of variables, namely facial muscle activity. We find that reading, speaking, chewing, jaw clenching, and six posed emotion expressions can be differentiated well by a Random Forest classifier. The results are complemented by the presentation of an open-source signal acquisition system. Thereby, an economical approach for naturalistic NeuroIS research and artefact development is provided.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    At the time of writing, almost 200 research articles using the OpenBCI system are listed on the manufacturers website: https://docs.openbci.com/citations; Last retrieved 12.04.21.

  2. 2.

    https://github.com/MKnierim/openbci-ceegrids.

References

  1. Xiong, J., Zuo, M.: What does existing NeuroIS research focus on? Inf. Syst. 89, 101462 (2020)

    Article  Google Scholar 

  2. Riedl, R., Fischer, T., Léger, P.-M., Davis, F.D.: A decade of NeuroIS research: progress, challenges, and future directions. ACM SIGMIS Database DATABASE Adv. Inf. Syst. 51, 13–54 (2020)

    Article  Google Scholar 

  3. Riedl, R., Davis, F.D., Hevner, A.R.: Towards a NeuroIS research methodology: intensifying the discussion on methods, tools, and measurement. J. Assoc. Inf. Syst. 15, i–xxxv (2014)

    Google Scholar 

  4. Ortiz de Guinea, A., Titah, R., Léger, P.-M.: Measure for Measure: A two study multi-trait multi-method investigation of construct validity in IS research. Comput. Human Behav. 29, 833–844 (2013)

    Google Scholar 

  5. Léger, P.M., Davis, F.D., Cronan, T.P., Perret, J.: Neurophysiological correlates of cognitive absorption in an enactive training context. Comput. Human Behav. 34, 273–283 (2014)

    Article  Google Scholar 

  6. Debener, S., Emkes, R., De Vos, M., Bleichner, M.: Unobtrusive ambulatory EEG using a smartphone and flexible printed electrodes around the ear. Sci. Rep. 5, 1–11 (2015)

    Article  Google Scholar 

  7. Bleichner, M.G., Debener, S.: Concealed, unobtrusive ear-centered EEG acquisition: ceegrids for transparent EEG. Front. Hum. Neurosci. 11, 1–14 (2017)

    Article  Google Scholar 

  8. Mirkovic, B., Bleichner, M.G., Vos, M.D., Debener, S.: Target Speaker Detection with Concealed EEG Around the Ear. Front. Neurosci. 10, 1–11 (2016)

    Article  Google Scholar 

  9. Pacharra, M., Debener, S., Wascher, E.: Concealed around-the-ear EEG captures cognitive processing in a visual Simon task. Front. Hum. Neurosci. 11, 1–11 (2017)

    Article  Google Scholar 

  10. Bleichner, M.G., Kidmose, P., Voix, J.: Ear-centered sensing: from sensing principles to research and clinical devices. Front. Neurosci. 13, 1437 (2019)

    Article  Google Scholar 

  11. Nogueira, W., et al.: Decoding selective attention in normal hearing listeners and bilateral cochlear implant users with concealed ear EEG. Front. Neurosci. 13, 1–15 (2019)

    Article  Google Scholar 

  12. Garrett, M., Debener, S., Verhulst, S.: Acquisition of subcortical auditory potentials with around-the-ear CEEgrid technology in normal and hearing impaired listeners. Front. Neurosci. 13, 1–15 (2019)

    Article  Google Scholar 

  13. Jaeger, M., Mirkovic, B., Bleichner, M.G., Debener, S.: Decoding the attended speaker from EEG using adaptive evaluation intervals captures fluctuations in attentional listening. Front. Neurosci. 14, 1–16 (2020)

    Article  Google Scholar 

  14. Sterr, A., et al.: Sleep EEG derived from behind-the-ear electrodes (cEEGrid) compared to standard polysomnography: a proof of concept study. Front. Hum. Neurosci. 12, 1–9 (2018)

    Article  Google Scholar 

  15. Mikkelsen, K.B., et al.: Machine-learning-derived sleep–wake staging from around-the-ear electroencephalogram outperforms manual scoring and actigraphy. J. Sleep Res. 28, e12786 (2019)

    Article  Google Scholar 

  16. Wascher, E., et al.: Evaluating mental load during realistic driving simulations by means of round the ear electrodes. Front. Neurosci. 13, 1–11 (2019)

    Article  Google Scholar 

  17. Perusquía-Hernández, M., Hirokawa, M., Suzuki, K.: A wearable device for fast and subtle spontaneous smile recognition. IEEE Trans. Affect. Comput. 8, 522–533 (2017)

    Article  Google Scholar 

  18. Tabar, Y.R., Mikkelsen, K.B., Rank, M.L., Christian Hemmsen, M., Kidmose, P.: Muscle activity detection during sleep by ear-EEG. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, pp. 1007–1010 (2020)

    Google Scholar 

  19. Sawangjai, P., Hompoonsup, S., Leelaarporn, P., Kongwudhikunakorn, S., Wilaiprasitporn, T.: Consumer grade EEG measuring sensors as research tools: a review. IEEE Sens. J. 20, 3996–4024 (2020)

    Article  Google Scholar 

  20. Riedl, R., Minas, R.K., Dennis, A.R., Müller-Putz, G.R.: Consumer-grade EEG instruments: insights on the measurement quality based on a literature review and implications for NeuroIS research. In: Davis, F.D., Riedl, R., vom Brocke, J., Léger, P.-M., Randolph, A.B., Fischer, T. (eds.) NeuroIS 2020. LNISO, vol. 43, pp. 350–361. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60073-0_41

    Chapter  Google Scholar 

  21. Hurst, H.E.: Long-term storage capacity of reservoirs. Trans. Amer. Soc. Civ. Eng. 116, 770–799 (1951)

    Article  Google Scholar 

  22. Petrosian, A.: Kolmogorov complexity of finite sequences and recognition of different preictal EEG patterns. In: Proceedings of the IEEE Symposium on Computer-Based Medical Systems, pp. 212–217 (1995)

    Google Scholar 

  23. Higuchi, T.: Approach to an irregular time series on the basis of the fractal theory. Phys. D Nonlinear Phenom. 31, 277–283 (1988)

    Article  Google Scholar 

  24. Hjorth, B.: EEG analysis based on time domain properties. Electroencephalogr. Clin. Neurophysiol. 29, 306–310 (1970)

    Article  Google Scholar 

  25. Oh, S.-H., Lee, Y.-R., Kim, H.-N.: A novel EEG feature extraction method using Hjorth parameter. Int. J. Electron. Electr. Eng. 2, 106–110 (2014)

    Article  Google Scholar 

  26. Val-Calvo, M., Álvarez-Sánchez, J.R., Ferrández-Vicente, J.M., Fernández, E.: Optimization of real-time EEG artifact removal and emotion estimation for human-robot interaction applications. Front. Comput. Neurosci. 13, 80 (2019)

    Article  Google Scholar 

  27. Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)

    Article  Google Scholar 

  28. Labonté-Lemoyne, É., et al.: Are we in Flow? Neurophysiological correlates of flow states in a collaborative game. In: Proceedings of the 2016 CHI Conference, pp. 1980–1988 (2016)

    Google Scholar 

  29. Frey, J.: Comparison of a consumer grade EEG amplifier with medical grade equipment in BCI applications. In: Proceedings from the International BCI Meeting, pp. 443–452 (2016)

    Google Scholar 

  30. Rashid, U., Niazi, I.K., Signal, N., Taylor, D.: An EEG experimental study evaluating the performance of Texas instruments ADS1299. Sensors 18, 1–18 (2018)

    Article  Google Scholar 

  31. Schaefer, A., Nils, F., Sanchez, X., Philippot, P.: Assessing the effectiveness of a large database of emotion-eliciting films: a new tool for emotion researchers. Cogn. Emot. 24, 1153–1172 (2010)

    Article  Google Scholar 

  32. Holzinger, A.: Interactive machine learning for health informatics: when do we need the human-in-the-loop? Brain Informatics 3(2), 119–131 (2016). https://doi.org/10.1007/s40708-016-0042-6

    Article  Google Scholar 

  33. McDuff, D., Karlson, A., Kapoor, A., Roseway, A., Czerwinski, M.: AffectAura: an intelligent system for emotional memory. In: Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems - CHI ’12, pp. 849–858 (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Karlsruhe Institute of Technology, Institute for Information Systems and Marketing, Karlsruhe, Germany

    Michael T. Knierim & Max Schemmer

  2. University of Essex, Colchester, UK

    Monica Perusquía-Hernández

Authors
  1. Michael T. Knierim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Max Schemmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Monica Perusquía-Hernández
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael T. Knierim .

Editor information

Editors and Affiliations

  1. Texas Tech University, Lubbock, TX, USA

    Fred D. Davis

  2. University of Applied Sciences Upper Austria and University of Linz, Steyr/Linz, Austria

    René Riedl

  3. University of Liechtenstein, Vaduz, Liechtenstein

    Jan vom Brocke

  4. HEC Montréal, Montréal, QC, Canada

    Pierre-Majorique Léger

  5. Kennesaw State University, Kennesaw, GA, USA

    Adriane B. Randolph

  6. Graz University of Technology, Graz, Austria

    Gernot Müller-Putz

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Knierim, M.T., Schemmer, M., Perusquía-Hernández, M. (2021). Exploring the Recognition of Facial Activities Through Around-The-Ear Electrode Arrays (cEEGrids). In: Davis, F.D., Riedl, R., vom Brocke, J., Léger, PM., Randolph, A.B., Müller-Putz, G. (eds) Information Systems and Neuroscience. NeuroIS 2021. Lecture Notes in Information Systems and Organisation, vol 52. Springer, Cham. https://doi.org/10.1007/978-3-030-88900-5_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-88900-5_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-88899-2

  • Online ISBN: 978-3-030-88900-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation