Using Haptic fMRI to Enable Interactive Motor Neuroimaging Experiments

Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 109)


Combining haptics with functional magnetic resonance imaging (Haptic fMRI) has enabled complex motor neuroimaging experiments that non-invasively map real-world motor tasks on to the human brain. The technique’s resolution, fidelity and susceptibility to scanning artifacts, however, have not yet been estimated in a quantitative manner. Here, we demonstrate that unconstrained three degree-of-freedom Haptic fMRI experiments can reliably activate brain regions involved in planning, motor control, haptic perception, and vision. We show that associated neural measurements are reliable, heterogeneous at the millimeter scale, and free from measurable artifacts, and that their anatomical localization is consistent with past neuroscience experiments. In addition, we demonstrate the feasibility of using electromagnetic actuation in Haptic fMRI interfaces to apply high fidelity open-loop three-axis haptic forces (0.5–2N; square or 0.1–65Hz sine waveforms) while maintaining negligible temporal noise in pre-motor, motor, somatosensory, and visual cortex (<1 % of signal). Our results show that Haptic fMRI is a robust and reliable technique for characterizing the human brain’s motor controller.


Haptic fMRI MRI-compatible robotics Motor control Neuroscience Neuroimaging 


  1. 1.
    Logothetis, N.K., Wandell, B.A.: Interpreting the bold signal. Annu. Rev. Physiol. 66, 735–769 (2004)CrossRefGoogle Scholar
  2. 2.
    Logothetis, N.K.: What we can do and what we cannot do with fMRI. Nature 453, 869–878 (2008)CrossRefGoogle Scholar
  3. 3.
    Hale, K.S., Stanney, K.M.: Deriving haptic design guidelines from human physiological, psychophysical, and neurological foundations. IEEE Comput. Graph. Appl. 24, 33–39 (2004)CrossRefGoogle Scholar
  4. 4.
    Kostic, M., Popovic, D., Popovic, M.: Influence of planar manipulandum to the hand trajectory during point to point movement. In: IEEE International Conference on Rehabilitation Robotics, pp. 1–4 (2011)Google Scholar
  5. 5.
    Khatib, O.: Inertial properties in robotic manipulation: an object-level framework. Int. J. Robot. Res. 14, 19–36 (1995)CrossRefGoogle Scholar
  6. 6.
    Rosenbaum, D.: Human Motor Control. Academic Press, San Diego (2009)Google Scholar
  7. 7.
    Meier, J.D., Aflalo, T.N., Kastner, S., Graziano, M.S.A.: Complex organization of human primary motor cortex: a high-resolution fMRI study. J. Neurophysiol. 100, 1800–1812 (2008)CrossRefGoogle Scholar
  8. 8.
    Churchland, M.M., Yu, B.M., Ryu, S.I., Santhanam, G., Shenoy, K.V.: Neural variability in premotor cortex provides a signature of motor preparation. J. Neurosci. 26, 3697–3712 (2006)CrossRefGoogle Scholar
  9. 9.
    Aflalo, T.N., Graziano, M.S.A.: Relationship between unconstrained arm movements and single-neuron firing in the macaque motor cortex. J. Neurosci. 27, 2760–2780 (2007)CrossRefGoogle Scholar
  10. 10.
    Kay, K.N., Winawer, J., Mezer, A., Wandell, B.A.: Compressive spatial summation in human visual cortex. J. Neurophysiol. 110, 481–494 (2013)CrossRefGoogle Scholar
  11. 11.
    Mesa-Munera, E., Ramirez-Salazar, J., Boulanger, P., Bischof, W.F., Branch, J.W.: Estimation of vibration and force stimulus thresholds for haptic guidance in MIS training. Rev. Ing. Biomed. 5, 17–22 (2012)Google Scholar
  12. 12.
    Menon, S., Brantner, G., Aholt, C., Kay, K., Khatib, O.: Haptic fMRI: combining functional neuroimaging with haptics for studying the brain’s motor control representation. In: Proceedings of the 13th Annual Conference of the IEEE Engineering in Medicine and Biology Society, pp. 4137–4142 (2013)Google Scholar
  13. 13.
    Menon, S., Yu, M., Kay, K., Khatib, O.: Haptic fMRI: accurately estimating neural responses in motor, pre-motor, and somatosensory cortex during complex motor tasks. In: Proceedings of the 14th Annual Conference of the IEEE Engineering in Medicine and Biology Society (2014)Google Scholar
  14. 14.
    Menon, S., Quigley, P., Yu, M., Khatib, O.: Haptic fMRI: using classification to quantify task-correlated noise during goal-directed reaching motions. In: Proceedings of the 14th Annual Conference of the IEEE Engineering in Medicine and Biology Society (2014)Google Scholar
  15. 15.
    Vogan, J., Wingert, A., Plante, J., Dubowsky, S., Hafez, M., Kacher, D.: Manipulation in MRI devices using electrostrictive polymer actuators: with an application to reconfigurable imaging coils. In: IEEE International Conference on Robotics and Automation, pp. 2498–2504 (2004)Google Scholar
  16. 16.
    Diedrichsen, J., Hashambhoy, Y., Rane, T., Shadmehr, R.: Neural correlates of reach errors. J. Neurosci. 25, 9919–9931 (2005)CrossRefGoogle Scholar
  17. 17.
    Menon, S., Stanley, A., Zhu, J., Okamura, A., Khatib, O.: Mapping stiffness perception in the brain with an fMRI-compatible particle-jamming haptic interface. In: Proceedings of the 14th Annual Conference of the IEEE Engineering in Medicine and Biology Society (2014)Google Scholar
  18. 18.
    Burdet, E., Gassert, R., Gowrishankar, G., Bleuler, H.: fMRI compatible haptic interfaces to investigate human motor control. Exp. Robot. IX 21, 25–34 (2006)CrossRefGoogle Scholar
  19. 19.
    Klare, S., Peer, A., Buss, M.: Development of a 3 DoF MR-compatible Haptic Interface for Pointing and Reaching Movements. Lecture Notes in Computer Science, vol. 6192, pp. 211–218. Springer, Berlin (2010)Google Scholar
  20. 20.
    Chapuis, D., Gassert, R., Gowrishankar, G., Burdet, E., Bleuler, H.: Investigation of a cable transmission for the actuation of MR compatible haptic interfaces. In: Proceedings of Biomedical Robotics and Biomechatronics, pp. 426–431 (2006)Google Scholar
  21. 21.
    Hribar, A., Koritnik, B., Munih, M.: Phantom haptic device upgrade for use in fMRI. Med. Biol. Eng. Comput. 47, 677–684 (2009)CrossRefGoogle Scholar
  22. 22.
    Massie, T., Salisbury, J.: The phantom haptic interface: a device for probing virtual objects. In: Proceedings of the ASME Winter Annual Meeting, Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, vol. 55(1), pp. 295–300. IOS Press (1994)Google Scholar
  23. 23.
    Khatib, O., Warren, J., Sapio, V.D., Sentis, L.: Human-like motion from physiologically-based potential energies. In: Advances in Robot Kinematics, pp. 149–163. Springer, The Netherlands (2004)Google Scholar
  24. 24.
    Sapio, V.D., Warren, J., Khatib, O., Delp, S.: Simulating the task-level control of human motion: a methodology and framework for implementation. Vis. Comput. 25, 289–302 (2006)Google Scholar
  25. 25.
    Demircan, E., Besier, T., Menon, S., Khatib, O.: Human motion reconstruction and synthesis of human skills. In: Lenarčič, J., Stanisic, M. (eds.) Advances in Robot Kinematics, pp. 283–292. Springer, Berlin (2010)Google Scholar
  26. 26.
    Evarts, E.V.: Relation of pyramidal tract activity to force exerted during voluntary movement. J. Neurophysiol. 31, 14–27 (1968)Google Scholar
  27. 27.
    Georgopoulos, A.P., Kalaska, J.F., Caminiti, R., Massey, J.T.: On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537 (1982)Google Scholar
  28. 28.
    Georgopoulos, A.P., Schwartz, A.B., Kettner, R.E.: Neuronal population coding of movement direction. Science 233, 1416–1419 (1986)CrossRefGoogle Scholar
  29. 29.
    Georgopoulos, A.P., Ashe, J., Smyrnis, N., Taira, M.: The motor cortex and the coding of force. Science 256, 1692–1695 (1992)CrossRefGoogle Scholar
  30. 30.
    Schwartz, A.B., Kettner, R.E., Georgopoulos, A.P.: Primate motor cortex and free arm movements to visual targets in three-dimensional space. I. relations between single cell discharge and direction of movement. J. Neurosci. 8, 2913–2927 (1988)Google Scholar
  31. 31.
    Georgopoulos, A.P., Caminiti, R., Kalaska, J.F.: Static spatial effects in motor cortex and area 5: quantitative relations in a two-dimensional space. Exp. Brain Res. 54, 446–454 (1984)CrossRefGoogle Scholar
  32. 32.
    Kettner, R.E., Schwartz, A.B., Georgopoulos, A.P.: Primate motor cortex and free arm movements to visual targets in three- dimensional space. iii. positional gradients and population coding of movement direction from various movement origins. J. Neurosci. 8, 2938–2947 (1988)Google Scholar
  33. 33.
    Cheney, P.D., Fetz, E.E., Palmer, S.S.: Patterns of facilitation and suppression of antagonist forelimb muscles from motor cortex sites in the awake monkey. J. Neurophysiol. 53, 805–820 (1985)Google Scholar
  34. 34.
    Kakei, S., Hoffman, D.S., Strick, P.L.: Muscle and movement representations in the primary motor cortex. Science 285, 2136–2139 (1999)CrossRefGoogle Scholar
  35. 35.
    Holdefer, R., Miller, L.: Primary motor cortical neurons encode functional muscle synergies. Exp. Brain Res. 146, 233–243 (2002)CrossRefGoogle Scholar
  36. 36.
    Caminiti, R., Johnson, P.B., Urbano, A.: Making arm movements within different parts of space: dynamic aspects in the primate motor cortex. J. Neurosci. 10, 2039–2058 (1990)Google Scholar
  37. 37.
    Reina, G.A., Moran, D.W., Schwartz, A.B.: On the relationship between joint angular velocity and motor cortical discharge during reaching. J. Neurophysiol. 85, 2576–2589 (2001)Google Scholar
  38. 38.
    Fu, Q.G., Suarez, J.I., Ebner, T.J.: Neuronal specification of direction and distance during reaching movements in the superior precentral premotor area and primary motor cortex of monkeys. J. Neurophysiol. 70, 2097–2116 (1993)Google Scholar
  39. 39.
    Churchland, M.M., Santhanam, G., Shenoy, K.V.: Preparatory activity in premotor and motor cortex reflects the speed of the upcoming reach. J. Neurophysiol. 96, 3130–3146 (2006)CrossRefGoogle Scholar
  40. 40.
    Ashe, J., Georgopoulos, A.P.: Movement parameters and neural activity in motor cortex and area 5. Cereb. Cortex 4, 590–600 (1994)CrossRefGoogle Scholar
  41. 41.
    Boynton, G.M., Engel, S.A., Glover, G.H., Heeger, D.J.: Linear systems analysis of functional magnetic resonance imaging in human V1. J. Neurosci. 16, 4207–4221 (1996)Google Scholar
  42. 42.
    Graziano, M.S.A., Aflalo, T.N.: Mapping behavioral repertoire onto the cortex. Neuron 56, 239–51 (2007)CrossRefGoogle Scholar
  43. 43.
    Delp, S.L., Anderson, F.C., Arnold, A.S.: OpenSim: open-source software to create and analyze dynamic simulations of movement. IEEE Trans. Biomed. Eng. 54, 1940–1950 (2007)CrossRefGoogle Scholar
  44. 44.
    Friston, K.J., Ashburner, J.T., Kiebel, S.J., Nichols, T.E., Penny, W.D.: Statistical Parametric Mapping: The Analysis of Functional Brain Images. Elsevier, London (2006)Google Scholar
  45. 45.
    Desikan, R.S., Killiany, R.J., et al.: An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage 31, 968–980 (2006)CrossRefGoogle Scholar
  46. 46.
    Kay, K., Rokem, A., Winawer, J., Dougherty, R., Wandell, B.: GLMdenoise: a fast, automated technique for denoising task-based fMRI data. Front. Neurosci. 7 (2013)Google Scholar
  47. 47.
    Lin, A.L., Fox, P.T., Hardies, J., Duong, T.Q., Gao, J.H.: Nonlinear coupling between cerebral blood flow, oxygen consumption, and ATP production in human visual cortex. Proc. Natl. Acad. Sci. 107, 8446–8451 (2010)CrossRefGoogle Scholar
  48. 48.
    Efron, B., Tibshirani, R.: An Introduction to the Bootstrap, vol. 57. Chapman & Hall, New York (1994)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.A.I. LabStanford UniversityStanfordUSA

Personalised recommendations