Advertisement

Development of a closed-loop BMI for elbow movement assistance based on kinematical decoding

  • E. Y. VeslinEmail author
  • M. S. Dutra
  • L. Bevilacqua
  • L. S. C. Raptopoulos
  • W. S. Andrade
  • J. G. M. Soares
Technical Paper
  • 37 Downloads

Abstract

A closed-loop brain–machine interface for elbow assistance is proposed in this work. The system decodes flexion and extension movements from noninvasive electroencephalographic signals through Kalman filter and uses them to activate a virtual device. A two-degree-of-freedom control scheme drives the model through a decoded path by generating a set of estimated inputs using differential flatness. These inputs are compensated by a feedback loop when decoding errors or external forces actuate the model. The results provide us an insight into the control architecture performance, which is dependent on the decoding precision. These decoding capabilities can be manipulated through a set of parameter configurations, enhancing the path tracking, or decreasing the decoding error according to their values.

Keywords

Electroencephalography Kalman filter Brain–machine interfaces Decoding Upper member Differential flatness 

Notes

Acknowledgements

The authors would like to thank FINEP, CNPq, FAPERJ, Fundação COPPETEC and DIPPG/CEFET-RJ for supporting our work as well as the students Edwiges Beatriz Coimbra de Souza, Aline Macedo Rocha Rodriguez e André Silva for helping in the EEG data acquisition and Marco Vinicio Chiorri for technical assistance.

References

  1. 1.
    Alazrai R, Alwanni H, Daoud M (2019) EEG-based BCI system for decoding finger movements within the same hand. Neurosci Lett 698:113–120CrossRefGoogle Scholar
  2. 2.
    Baxter BS, Decker A, He B (2013) Noninvasive control of a robotic arm in multiple dimensions using scalp electroencephalogram. In: 2013 6th international IEEE/EMBS conference on neural engineering (NER). IEEE, pp 45–47Google Scholar
  3. 3.
    Besserve M, Jerbi K, Laurent F, Baillet S, Martinerie J, Garnero L (2007) Classification methods for ongoing EEG and MEG signals. Biol Res 40(4):415–437CrossRefGoogle Scholar
  4. 4.
    Bishop G, Welch G (2001) An introduction to the Kalman filter. Proc SIGGRAPH 8:1–41Google Scholar
  5. 5.
    Bradberry TJ, Gentili RJ, Contreras-Vidal JL (2010) Reconstructing three-dimensional hand movements from noninvasive electroencephalographic signals. J Neurosci 30(9):3432–3437CrossRefGoogle Scholar
  6. 6.
    Carmena JM, Lebedev MA, Crist RE, O’Doherty JE, Santucci DM, Dimitrov DF, Patil PG, Henriquez CS, Nicolelis MA (2003) Learning to control a brain–machine interface for reaching and grasping by primates. PLoS Biol 1(2):193–208CrossRefGoogle Scholar
  7. 7.
    De Leva P (1996) Adjustments to Zatsiorsky–Seluyanov’s segment inertia parameters. J Biomech 29(9):1223–1230CrossRefGoogle Scholar
  8. 8.
    Figueiredo R, Carvalho C, Veslin E, Silva F, Campos G, Raptopoulos L, Bevilacqua L, Dutra M, WS A (2018) Desenvolvimento de um estimador para o movimento do cotovelo humano baseado em inteligência artificial e sinais eletromiogr’aficos. In: Proceedings of 2018 ABCM CONEM, X Congresso Nacional de Engenharia Mecânica, ABCM, pp 1–8Google Scholar
  9. 9.
    Fliess M, Lévine J, Martin P, Rouchon P (1995) Flatness and defect of non-linear systems: introductory theory and examples. Int J Control 61(6):1327–1361MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Georgopoulos AP, Kalaska JF, Caminiti R, Massey JT (1982) On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J Neurosci 2(11):1527–1537CrossRefGoogle Scholar
  11. 11.
    Georgopoulos AP, Schwartz AB, Kettner RE (1986) Neuronal population coding of movement direction. Science 233(4771):1416–1419CrossRefGoogle Scholar
  12. 12.
    Hochberg LR, Serruya MD, Friehs GM, Mukand JA, Saleh M, Caplan AH, Branner A, Chen D, Penn RD, Donoghue JP (2006) Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442(7099):164–171CrossRefGoogle Scholar
  13. 13.
    Hochberg LR, Bacher D, Jarosiewicz B, Masse NY, Simeral JD, Vogel J, Haddadin S, Liu J, Cash SS, van der Smagt P, Donoghue J (2012) Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485(7398):372–375CrossRefGoogle Scholar
  14. 14.
    Hogan N, Krebs HI (2004) Interactive robots for neuro-rehabilitation. Restor Neurol Neurosci 22(3–5):349–358Google Scholar
  15. 15.
    Homer ML, Harrison MT, Black MJ, Perge JA, Cash SS, Friehs G, Hochberg LR (2013) Mixing decoded cursor velocity and position from an offline Kalman filter improves cursor control in people with tetraplegia. In: 2013 6th international IEEE/EMBS conference on neural engineering (NER). IEEE, pp 715–718Google Scholar
  16. 16.
    Jerbi K, Vidal J, Mattout J, Maby E, Lecaignard F, Ossandon T, Hamamé C, Dalal S, Bouet R, Lachaux JP et al (2011) Inferring hand movement kinematics from MEG, EEG and intracranial EEG: from brain–machine interfaces to motor rehabilitation. IRBM 32(1):8–18CrossRefGoogle Scholar
  17. 17.
    Kettner RE, Schwartz AB, Georgopoulos AP (1988) Primate motor cortex and free arm movements to visual targets in three-dimensional space. III. Positional gradients and population coding of movement direction from various movement origins. J Neurosci 8(8):2938–2947CrossRefGoogle Scholar
  18. 18.
    Ko CH, Young KY, Huang YC, Agrawal SK (2013) Walk-assist robot: a novel approach to gain selection of a braking controller using differential flatness. IEEE Trans Control Syst Technol 21(6):2299–2305CrossRefGoogle Scholar
  19. 19.
    Korik A, Sosnik R, Siddique N, Coyle D (2018) Decoding imagined 3D hand movement trajectories from EEG: evidence to support the use of mu, beta, and low gamma oscillations. Frontiers Neurosci 12:130CrossRefGoogle Scholar
  20. 20.
    Lalitharatne TD, Yoshino A, Hayashi Y, Teramoto K, Kiguchi K (2012) Toward EEG control of upper limb power-assist exoskeletons: a preliminary study of decoding elbow joint velocities using EEG signals. In: 2012 international symposium on micro-nanomechatronics and human science (MHS). IEEE, pp 421–424Google Scholar
  21. 21.
    Lalitharatne TD, Teramoto K, Hayashi Y, Kiguchi K (2013) Towards hybrid EEG–EMG-based control approaches to be used in bio-robotics applications: current status, challenges and future directions. J Behav Robot 4(2):147–154Google Scholar
  22. 22.
    Levine J (2009) Analysis and control of nonlinear systems: a flatness-based approach. Springer, BerlinzbMATHCrossRefGoogle Scholar
  23. 23.
    Lewis FL, Dawson DM, Abdallah CT (2003) Robot manipulator control: theory and practice. CRC Press, Boca RatonCrossRefGoogle Scholar
  24. 24.
    Li MA, Wang YF, Jia SM, Sun YJ, Yang JF (2019) Decoding of motor imagery EEG based on brain source estimation. Neurocomputing 339:182–193CrossRefGoogle Scholar
  25. 25.
    Li Z, Huang Z, He W, Su CY (2017) Adaptive impedance control for an upper limb robotic exoskeleton using biological signals. IEEE Trans Ind Electron 64(2):1664–1674CrossRefGoogle Scholar
  26. 26.
    Long Y, Du Z, Cong L, Wang W, Zhang Z, Dong W (2017) Active disturbance rejection control based human gait tracking for lower extremity rehabilitation exoskeleton. ISA Trans 67:389–397CrossRefGoogle Scholar
  27. 27.
    Lotte F, Congedo M, Lécuyer A, Lamarche F, Arnaldi B (2007) A review of classification algorithms for EEG-based brain–computer interfaces. J Neural Eng 4(2):R1–R13CrossRefGoogle Scholar
  28. 28.
    Luu TP, He Y, Brown S, Nakagome S, Contreras-Vidal JL (2016) Gait adaptation to visual kinematic perturbations using a real-time closed-loop brain–computer interface to a virtual reality avatar. J Neural Eng 13(3):1–13CrossRefGoogle Scholar
  29. 29.
    Lv J, Li Y, Gu Z (2010) Decoding hand movement velocity from electroencephalogram signals during a drawing task. Biomed Eng Online 9(64):1–21Google Scholar
  30. 30.
    Markus ED, Agee JT, Jimoh AA (2017) Flat control of industrial robotic manipulators. Robot Auton Syst 87:226–236CrossRefGoogle Scholar
  31. 31.
    Martin P, Devasia S, Paden B (1996) A different look at output tracking: control of a VTOL aircraft. Automatica 32(1):101–107zbMATHCrossRefGoogle Scholar
  32. 32.
    Martin P, Murray RM, Rouchon P (2003) Flat systems, equivalence and trajectory generation. Technical report, California Institute of Technology CALTECH,Google Scholar
  33. 33.
    Moorman HG, Gowda S, Carmena JM (2017) Control of redundant kinematic degrees of freedom in a closed-loop brain–machine interface. IEEE Trans Neural Syst Rehabil Eng 25(6):750–760CrossRefGoogle Scholar
  34. 34.
    Moran DW, Schwartz AB (1999) Motor cortical activity during drawing movements: population representation during spiral tracing. J Neurophysiol 82(5):2693–2704CrossRefGoogle Scholar
  35. 35.
    Murray RM, Rathinam M, Sluis W (1995) Differential flatness of mechanical control systems: a catalog of prototype systems. In: ASME international mechanical engineering congress and exposition, pp 1–9Google Scholar
  36. 36.
    Nasr S, Mekki H, Bouallegue K (2019) A multi-scroll chaotic system for a higher coverage path planning of a mobile robot using flatness controller. Chaos Solitons Fractals 118:366–375MathSciNetCrossRefGoogle Scholar
  37. 37.
    Noback CR, Strominger NL, Demarest RJ, Ruggiero DA (2005) The human nervous system: structure and function, 6th edn. Humana Press Inc., TotowaGoogle Scholar
  38. 38.
    Ofner P, Müller-Putz GR (2012) Decoding of velocities and positions of 3D arm movement from EEG. In: 2012 annual international conference of the IEEE engineering in medicine and biology society. IEEE, pp 6406–6409Google Scholar
  39. 39.
    Pfurtscheller G, da Silva FL (1999) Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol 110:1842–1857CrossRefGoogle Scholar
  40. 40.
    Pistohl T, Ball T, Schulze-Bonhage A, Aertsen A, Mehring C (2008) Prediction of arm movement trajectories from ECoG-recordings in humans. J Neurosci Methods 167(1):105–114CrossRefGoogle Scholar
  41. 41.
    Pons JL (2008) Wearable robots: biomechatronic exoskeletons. Wiley, HobokenCrossRefGoogle Scholar
  42. 42.
    Proietti T, Crocher V, Roby-Brami A, Jarrasse N (2016) Upper-limb robotic exoskeletons for neurorehabilitation: a review on control strategies. IEEE Rev Biomed Eng 9:4–14CrossRefGoogle Scholar
  43. 43.
    Rao RP (2019) Towards neural co-processors for the brain: combining decoding and encoding in brain–computer interfaces. Curr Opin Neurobiol 55:142–151CrossRefGoogle Scholar
  44. 44.
    Robinson N, Vinod A (2016) Noninvasive brain–computer interface: decoding arm movement kinematics and motor control. IEEE Syst Man Cybern Mag 2(4):4–16CrossRefGoogle Scholar
  45. 45.
    Robinson N, Guan C, Vinod A (2015) Adaptive estimation of hand movement trajectory in an EEG based brain–computer interface system. J Neural Eng 12(6):1–12CrossRefGoogle Scholar
  46. 46.
    Roy R, Mahadevappa M, Kumar C (2016) Trajectory path planning of EEG controlled robotic arm using GA. Procedia Comput Sci 84:147–151CrossRefGoogle Scholar
  47. 47.
    Ryu JC, Agrawal SK (2011) Differential flatness-based robust control of mobile robots in the presence of slip. Int J Robot Res 30(4):463–475CrossRefGoogle Scholar
  48. 48.
    Shakibaee F, Mottaghi E, Kobravi HR, Ghoshuni M (2019) Decoding knee angle trajectory from electroencephalogram signal using NARX neural network and a new channel selection algorithm. Biomed Phys Eng Express 5(2):1–18CrossRefGoogle Scholar
  49. 49.
    Sim KS, Kiang KDT, You LZ (2016) EEG controlled wheelchair. In: MATEC web of conferences, EDP sciences, vol 51, pp 1–9Google Scholar
  50. 50.
    Soekadar SR, Witkowski M, Vitiello N, Birbaumer N (2015) An EEG/EOG-based hybrid brain–neural computer interaction (BNCI) system to control an exoskeleton for the paralyzed hand. Biomed Eng Biomed Tech 60(3):199–205Google Scholar
  51. 51.
    Tang Z, Zhang K, Sun S, Gao Z, Zhang L, Yang Z (2014) An upper-limb power-assist exoskeleton using proportional myoelectric control. Sensors 14(4):6677–6694CrossRefGoogle Scholar
  52. 52.
    Tayeb Z, Fedjaev J, Ghaboosi N, Richter C, Everding L, Qu X, Wu Y, Cheng G, Conradt J (2019) Validating deep neural networks for online decoding of motor imagery movements from EEG signals. Sensors 19(1):210CrossRefGoogle Scholar
  53. 53.
    Ubeda A, Costa A, Iánez E, Piñuela-Martín E, Márquez-Sánchez E, del Ama AJ, Gil-Agudo Á, Azorín JM (2015) Single joint movement decoding from EEG in healthy and incomplete spinal cord injured subjects. In: 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 6179–6183Google Scholar
  54. 54.
    Van Nieuwstadt MJ (1997) Trajectory generation for nonlinear control systems. PhD thesis, California Institute of TechnologyGoogle Scholar
  55. 55.
    Van Nieuwstadt MJ, Murray RM (1998) Real-time trajectory generation for differentially flat systems. Int J Robust Nonlinear Control IFAC Affil J 8(11):995–1020MathSciNetzbMATHCrossRefGoogle Scholar
  56. 56.
    Veslin E, Dutra M, Bevilacqua L, Raptopoulos L, Andrade W, Pereira A, Fiorani M, Soares J (2017) Classification of right and left elbow movement with lower gamma activity. In: Proceedings of the 24th ABCM international congress of mechanical engineering, pp 1–8Google Scholar
  57. 57.
    Veslin E, Dutra M, Bevilacqua L, Raptopoulos L, Andrade W, Pereira A, Fiorani M, Soares J (2019a) Lower gamma band in the classification of left and right elbow movement in real and imaginary tasks. J Braz Soc Mech Sci Eng 41(91):1–15Google Scholar
  58. 58.
    Veslin E, Dutra M, Bevilacqua L, Raptopoulos L, Andrade W, Soares J (2019b) Decoding elbow movement with Kalman filter using non-invasive EEG. In: 2019 IEEE 2nd Colombian conference on applications in computational intelligence (ColCACI), pp 1–6Google Scholar
  59. 59.
    Veslin Diaz E, Slama J, Dutra M, Lengerke O, Morales Tavera M (2011) Trajectory tracking for robot manipulators using differential flatness. Ingeniería e Investigación 31(2):84–90zbMATHGoogle Scholar
  60. 60.
    Wu W, Black M, Gao Y, Bienenstock E, Serruya M, Donoghue J (2002) Inferring hand motion from multi-cell recordings in motor cortex using a Kalman filter. In: SAB’02-workshop on motor control in humans and robots: on the interplay of real brains and artificial devices, pp 66–73Google Scholar
  61. 61.
    Wu W, Gao Y, Bienenstock E, Donoghue JP, Black MJ (2006) Bayesian population decoding of motor cortical activity using a Kalman filter. Neural Comput 18(1):80–118MathSciNetzbMATHCrossRefGoogle Scholar
  62. 62.
    Xu L, Adler A (2004) An improved method for muscle activation detection during gait. In: 2004 Canadian conference on electrical and computer engineering, vol 1. IEEE, pp 357–360Google Scholar
  63. 63.
    Yuan H, Perdoni C, He B (2010) Relationship between speed and EEG activity during imagined and executed hand movements. J Neural Eng 7(2):1–10CrossRefGoogle Scholar
  64. 64.
    Zhang R, Li Y, Yan Y, Zhang H, Wu S, Yu T, Gu Z (2016) Control of a wheelchair in an indoor environment based on a brain–computer interface and automated navigation. IEEE Trans Neural Syst Rehabil Eng 24(1):128–139CrossRefGoogle Scholar

Copyright information

© The Brazilian Society of Mechanical Sciences and Engineering 2019

Authors and Affiliations

  1. 1.Robotics Laboratory, Department of Mechanical EngineeringFederal University of Rio de JaneiroRio de JaneiroBrazil
  2. 2.Department of Civil EngineeringFederal University of Rio de JaneiroRio de JaneiroBrazil
  3. 3.Federal Center of Technological Education Celso Suckow da Fonseca – CEFET/RJNova IguaçuBrazil
  4. 4.Laboratory of Cognitive Physiology, Institute of Biophysics Carlos Chagas FilhoFederal University of Rio de JaneiroRio de JaneiroBrazil
  5. 5.Rio de JaneiroBrazil

Personalised recommendations