Skip to main content

Using Haptic fMRI to Enable Interactive Motor Neuroimaging Experiments

Part of the Springer Tracts in Advanced Robotics book series (STAR,volume 109)

Abstract

Combining haptics with functional magnetic resonance imaging (Haptic fMRI) has enabled complex motor neuroimaging experiments that non-invasively map real-world motor tasks on to the human brain. The technique’s resolution, fidelity and susceptibility to scanning artifacts, however, have not yet been estimated in a quantitative manner. Here, we demonstrate that unconstrained three degree-of-freedom Haptic fMRI experiments can reliably activate brain regions involved in planning, motor control, haptic perception, and vision. We show that associated neural measurements are reliable, heterogeneous at the millimeter scale, and free from measurable artifacts, and that their anatomical localization is consistent with past neuroscience experiments. In addition, we demonstrate the feasibility of using electromagnetic actuation in Haptic fMRI interfaces to apply high fidelity open-loop three-axis haptic forces (0.5–2N; square or 0.1–65Hz sine waveforms) while maintaining negligible temporal noise in pre-motor, motor, somatosensory, and visual cortex (<1 % of signal). Our results show that Haptic fMRI is a robust and reliable technique for characterizing the human brain’s motor controller.

Keywords

  • Haptic fMRI
  • MRI-compatible robotics
  • Motor control
  • Neuroscience
  • Neuroimaging

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-23778-7_7
  • Chapter length: 15 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   129.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-23778-7
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   169.99
Price excludes VAT (USA)
Hardcover Book
USD   219.99
Price excludes VAT (USA)
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Logothetis, N.K., Wandell, B.A.: Interpreting the bold signal. Annu. Rev. Physiol. 66, 735–769 (2004)

    CrossRef  Google Scholar 

  2. Logothetis, N.K.: What we can do and what we cannot do with fMRI. Nature 453, 869–878 (2008)

    CrossRef  Google Scholar 

  3. Hale, K.S., Stanney, K.M.: Deriving haptic design guidelines from human physiological, psychophysical, and neurological foundations. IEEE Comput. Graph. Appl. 24, 33–39 (2004)

    CrossRef  Google Scholar 

  4. Kostic, M., Popovic, D., Popovic, M.: Influence of planar manipulandum to the hand trajectory during point to point movement. In: IEEE International Conference on Rehabilitation Robotics, pp. 1–4 (2011)

    Google Scholar 

  5. Khatib, O.: Inertial properties in robotic manipulation: an object-level framework. Int. J. Robot. Res. 14, 19–36 (1995)

    CrossRef  Google Scholar 

  6. Rosenbaum, D.: Human Motor Control. Academic Press, San Diego (2009)

    Google Scholar 

  7. Meier, J.D., Aflalo, T.N., Kastner, S., Graziano, M.S.A.: Complex organization of human primary motor cortex: a high-resolution fMRI study. J. Neurophysiol. 100, 1800–1812 (2008)

    CrossRef  Google Scholar 

  8. Churchland, M.M., Yu, B.M., Ryu, S.I., Santhanam, G., Shenoy, K.V.: Neural variability in premotor cortex provides a signature of motor preparation. J. Neurosci. 26, 3697–3712 (2006)

    CrossRef  Google Scholar 

  9. Aflalo, T.N., Graziano, M.S.A.: Relationship between unconstrained arm movements and single-neuron firing in the macaque motor cortex. J. Neurosci. 27, 2760–2780 (2007)

    CrossRef  Google Scholar 

  10. Kay, K.N., Winawer, J., Mezer, A., Wandell, B.A.: Compressive spatial summation in human visual cortex. J. Neurophysiol. 110, 481–494 (2013)

    CrossRef  Google Scholar 

  11. Mesa-Munera, E., Ramirez-Salazar, J., Boulanger, P., Bischof, W.F., Branch, J.W.: Estimation of vibration and force stimulus thresholds for haptic guidance in MIS training. Rev. Ing. Biomed. 5, 17–22 (2012)

    Google Scholar 

  12. Menon, S., Brantner, G., Aholt, C., Kay, K., Khatib, O.: Haptic fMRI: combining functional neuroimaging with haptics for studying the brain’s motor control representation. In: Proceedings of the 13th Annual Conference of the IEEE Engineering in Medicine and Biology Society, pp. 4137–4142 (2013)

    Google Scholar 

  13. Menon, S., Yu, M., Kay, K., Khatib, O.: Haptic fMRI: accurately estimating neural responses in motor, pre-motor, and somatosensory cortex during complex motor tasks. In: Proceedings of the 14th Annual Conference of the IEEE Engineering in Medicine and Biology Society (2014)

    Google Scholar 

  14. Menon, S., Quigley, P., Yu, M., Khatib, O.: Haptic fMRI: using classification to quantify task-correlated noise during goal-directed reaching motions. In: Proceedings of the 14th Annual Conference of the IEEE Engineering in Medicine and Biology Society (2014)

    Google Scholar 

  15. Vogan, J., Wingert, A., Plante, J., Dubowsky, S., Hafez, M., Kacher, D.: Manipulation in MRI devices using electrostrictive polymer actuators: with an application to reconfigurable imaging coils. In: IEEE International Conference on Robotics and Automation, pp. 2498–2504 (2004)

    Google Scholar 

  16. Diedrichsen, J., Hashambhoy, Y., Rane, T., Shadmehr, R.: Neural correlates of reach errors. J. Neurosci. 25, 9919–9931 (2005)

    CrossRef  Google Scholar 

  17. Menon, S., Stanley, A., Zhu, J., Okamura, A., Khatib, O.: Mapping stiffness perception in the brain with an fMRI-compatible particle-jamming haptic interface. In: Proceedings of the 14th Annual Conference of the IEEE Engineering in Medicine and Biology Society (2014)

    Google Scholar 

  18. Burdet, E., Gassert, R., Gowrishankar, G., Bleuler, H.: fMRI compatible haptic interfaces to investigate human motor control. Exp. Robot. IX 21, 25–34 (2006)

    CrossRef  Google Scholar 

  19. Klare, S., Peer, A., Buss, M.: Development of a 3 DoF MR-compatible Haptic Interface for Pointing and Reaching Movements. Lecture Notes in Computer Science, vol. 6192, pp. 211–218. Springer, Berlin (2010)

    Google Scholar 

  20. Chapuis, D., Gassert, R., Gowrishankar, G., Burdet, E., Bleuler, H.: Investigation of a cable transmission for the actuation of MR compatible haptic interfaces. In: Proceedings of Biomedical Robotics and Biomechatronics, pp. 426–431 (2006)

    Google Scholar 

  21. Hribar, A., Koritnik, B., Munih, M.: Phantom haptic device upgrade for use in fMRI. Med. Biol. Eng. Comput. 47, 677–684 (2009)

    CrossRef  Google Scholar 

  22. Massie, T., Salisbury, J.: The phantom haptic interface: a device for probing virtual objects. In: Proceedings of the ASME Winter Annual Meeting, Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, vol. 55(1), pp. 295–300. IOS Press (1994)

    Google Scholar 

  23. Khatib, O., Warren, J., Sapio, V.D., Sentis, L.: Human-like motion from physiologically-based potential energies. In: Advances in Robot Kinematics, pp. 149–163. Springer, The Netherlands (2004)

    Google Scholar 

  24. Sapio, V.D., Warren, J., Khatib, O., Delp, S.: Simulating the task-level control of human motion: a methodology and framework for implementation. Vis. Comput. 25, 289–302 (2006)

    Google Scholar 

  25. Demircan, E., Besier, T., Menon, S., Khatib, O.: Human motion reconstruction and synthesis of human skills. In: Lenarčič, J., Stanisic, M. (eds.) Advances in Robot Kinematics, pp. 283–292. Springer, Berlin (2010)

    Google Scholar 

  26. Evarts, E.V.: Relation of pyramidal tract activity to force exerted during voluntary movement. J. Neurophysiol. 31, 14–27 (1968)

    Google Scholar 

  27. Georgopoulos, A.P., Kalaska, J.F., Caminiti, R., Massey, J.T.: On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537 (1982)

    Google Scholar 

  28. Georgopoulos, A.P., Schwartz, A.B., Kettner, R.E.: Neuronal population coding of movement direction. Science 233, 1416–1419 (1986)

    CrossRef  Google Scholar 

  29. Georgopoulos, A.P., Ashe, J., Smyrnis, N., Taira, M.: The motor cortex and the coding of force. Science 256, 1692–1695 (1992)

    CrossRef  Google Scholar 

  30. Schwartz, A.B., Kettner, R.E., Georgopoulos, A.P.: Primate motor cortex and free arm movements to visual targets in three-dimensional space. I. relations between single cell discharge and direction of movement. J. Neurosci. 8, 2913–2927 (1988)

    Google Scholar 

  31. Georgopoulos, A.P., Caminiti, R., Kalaska, J.F.: Static spatial effects in motor cortex and area 5: quantitative relations in a two-dimensional space. Exp. Brain Res. 54, 446–454 (1984)

    CrossRef  Google Scholar 

  32. Kettner, R.E., Schwartz, A.B., Georgopoulos, A.P.: Primate motor cortex and free arm movements to visual targets in three- dimensional space. iii. positional gradients and population coding of movement direction from various movement origins. J. Neurosci. 8, 2938–2947 (1988)

    Google Scholar 

  33. Cheney, P.D., Fetz, E.E., Palmer, S.S.: Patterns of facilitation and suppression of antagonist forelimb muscles from motor cortex sites in the awake monkey. J. Neurophysiol. 53, 805–820 (1985)

    Google Scholar 

  34. Kakei, S., Hoffman, D.S., Strick, P.L.: Muscle and movement representations in the primary motor cortex. Science 285, 2136–2139 (1999)

    CrossRef  Google Scholar 

  35. Holdefer, R., Miller, L.: Primary motor cortical neurons encode functional muscle synergies. Exp. Brain Res. 146, 233–243 (2002)

    CrossRef  Google Scholar 

  36. Caminiti, R., Johnson, P.B., Urbano, A.: Making arm movements within different parts of space: dynamic aspects in the primate motor cortex. J. Neurosci. 10, 2039–2058 (1990)

    Google Scholar 

  37. Reina, G.A., Moran, D.W., Schwartz, A.B.: On the relationship between joint angular velocity and motor cortical discharge during reaching. J. Neurophysiol. 85, 2576–2589 (2001)

    Google Scholar 

  38. Fu, Q.G., Suarez, J.I., Ebner, T.J.: Neuronal specification of direction and distance during reaching movements in the superior precentral premotor area and primary motor cortex of monkeys. J. Neurophysiol. 70, 2097–2116 (1993)

    Google Scholar 

  39. Churchland, M.M., Santhanam, G., Shenoy, K.V.: Preparatory activity in premotor and motor cortex reflects the speed of the upcoming reach. J. Neurophysiol. 96, 3130–3146 (2006)

    CrossRef  Google Scholar 

  40. Ashe, J., Georgopoulos, A.P.: Movement parameters and neural activity in motor cortex and area 5. Cereb. Cortex 4, 590–600 (1994)

    CrossRef  Google Scholar 

  41. Boynton, G.M., Engel, S.A., Glover, G.H., Heeger, D.J.: Linear systems analysis of functional magnetic resonance imaging in human V1. J. Neurosci. 16, 4207–4221 (1996)

    Google Scholar 

  42. Graziano, M.S.A., Aflalo, T.N.: Mapping behavioral repertoire onto the cortex. Neuron 56, 239–51 (2007)

    CrossRef  Google Scholar 

  43. Delp, S.L., Anderson, F.C., Arnold, A.S.: OpenSim: open-source software to create and analyze dynamic simulations of movement. IEEE Trans. Biomed. Eng. 54, 1940–1950 (2007)

    CrossRef  Google Scholar 

  44. Friston, K.J., Ashburner, J.T., Kiebel, S.J., Nichols, T.E., Penny, W.D.: Statistical Parametric Mapping: The Analysis of Functional Brain Images. Elsevier, London (2006)

    Google Scholar 

  45. Desikan, R.S., Killiany, R.J., et al.: An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage 31, 968–980 (2006)

    CrossRef  Google Scholar 

  46. Kay, K., Rokem, A., Winawer, J., Dougherty, R., Wandell, B.: GLMdenoise: a fast, automated technique for denoising task-based fMRI data. Front. Neurosci. 7 (2013)

    Google Scholar 

  47. Lin, A.L., Fox, P.T., Hardies, J., Duong, T.Q., Gao, J.H.: Nonlinear coupling between cerebral blood flow, oxygen consumption, and ATP production in human visual cortex. Proc. Natl. Acad. Sci. 107, 8446–8451 (2010)

    CrossRef  Google Scholar 

  48. Efron, B., Tibshirani, R.: An Introduction to the Bootstrap, vol. 57. Chapman & Hall, New York (1994)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Samir Menon .

Editor information

Editors and Affiliations

Appendix

Appendix

1.1 MRI Protocol

All fMRI scans were conducted at Stanford University’s Center for Cognitive and Neurobiological Imaging on a GE Discovery MR750 3T MRI scanner, with a thirty-two channel Nova Medical head coil. The scan protocol was gradient echo EPI with a 16 cm field of view sampled at a \(64\times 64\) resolution (\(2.5\times 2.5\times 2.5\) mm\(^3\) voxels), a 1.57 s repetition time, a 28 ms echo time, and a 72\(^\circ \) flip angle. Each scan run was preceded by 2nd-order polynomial shimming and was sandwiched by spiral fieldmap scans (\(2.5\times 2.5\times 5\) mm\(^3\) voxels). Fieldmap scans were conducted within 10 s of each scan run’s start and end. After scanning, the fMRI images were slice time corrected, motion corrected (SPM [44]), spatially undistorted using fieldmaps, and analyzed to compute temporal noise-to-signal.

1.2 fMRI Analysis

Temporal noise-to-signal computations used the median neural response distribution obtained by regressing out a line from each voxel’s time series, computing the absolute value of the difference between successive time points, computing the median of these absolute differences, dividing the result by the mean of the original time series, and then multiplying by 100. Cortex segmentation used Freesurfer’s Desikan-Killiany atlas [45]. Surface registration was done using Freesurfer, and all surface images were plotted using Freeview. Freeview smoothed the surface plots while rendering (2 steps).

1.3 Estimating fMRI Impulse Response Time Series and \(R^2\)

fMRI measures changes in blood oxygenation induced by neural metabolic activity [1, 2], which have a slower time course than neural computation and persist long after sensory stimuli and motor tasks terminate. Such persistent responses cause raw fMRI measurements to overlap in experiments where consecutive task conditions are not be separated by large time-intervals. Separating task conditions by large time-intervals, however, makes fMRI runs very long, which can induce a variety of unwanted artifacts related to MRI scanner calibration drift, neural adaptation, or subject attention lapses, microsleep and exhaustion. Instead, we optimized our experiments to ensure reliable motor task execution [13], which caused fMRI measurements for different task conditions to overlap.

We segregated neural activation for individual tasks using a finite impulse response (FIR) model (implemented using GLMdenoise [46]). The FIR model works by associating each task type with a unique time course and segregates time courses while assuming that overlapping responses sum linearly. fMRI signal linearity, however, is an active area of research [1, 41, 47]. As such, we randomized inter-task delays and randomly ordered tasks, which made the model’s time series match anatomical expectations based on past research (see Figs. 2 and 3; read [6] for an overview). When tasks were closely spaced in time, as with planning and motion, this method was noisy. The parts of planning that overlap with motion are thus less reliable and the confidence interval for the planning time series estimates is larger after motion starts (but still above zero; see Fig. 1).

We computed \(95\,\%\) confidence intervals by bootstrapping [48] runs (400 bootstraps), fitting FIR models to each, and taking the median percentile estimates across the estimated bootstrap time series. Finally, we computed \(R^2\) values for each voxel by comparing the time series variance with the variance after regressing out median FIR model estimates.

1.4 Data Collection Protocols

See [13] for precise specifications of the motion protocol. Subjects executed one practice run inside the MRI scanner, and then executed at least eight scan runs (S1, 10; S2, 8). Each run was 630 s long.

The force and visual perception experiment protocol involved fixed duration stimuli instances with visual, motor, or visual and motor sensory input. The experiment was divided into runs, and each run was divided into blocks. During each block, the subject started with their hand at rest. Next, they were instructed to move their hand into free space. After a randomized delay period of 3–5 s, the subject experienced two randomly selected stimuli instances. Each stimulus instance was 3–5 s long and was separated from the other by a randomized delay 3–5 s. Finally, the subject was required to rest their hands for a random time interval (4–20 s), and then restarted the process. The subject executed four scan runs with multiple blocks. Each run was 459 s long.

Force magnitudes were set to evenly spaced directions along the x-y plane, with a magnitude of 1.2 N. The force vectors used were (1.2, 0.0), (0.0, 1.2), (\(-\)1.2, 0.0), (0.0, \(-\)1.2), (0.85, 0.0), (0.0, 0.85), (\(-\)0.85, 0.0), and (0.0, \(-\)0.85).

1.5 Haptic and Force Measurement Details

Haptic experiments were conducted with Haptic fMRI Interface [12], a three degree-of-freedom fMRI-compatible device. All motions were right handed, and the haptic control rate was 350 Hz. The reaching task spanned the entire workspace (see [13] for more details), but avoided arm motion artifacts [14].

Visual stimuli were displayed on a 30 in. diagonal (76 cm, 16 : 10 aspect ratio) flat panel display custom built by Resonance Technology. Subjects viewed visual stimuli through a dual-mirror setup. The visual distance from screen to mirror-2 is 184.4 cm, from mirror-2 to mirror-1 is 6 cm, and from mirror-1 to the eye is about 15 cm, for a total viewing distance of about 205 cm. The visual field of view is about 30\(^\circ \), making each visual checkerboard square span about one and a half degrees of the visual field. The display has a native resolution of \(2560\times 1600\) but stimuli were displayed at \(1280\times 800\). The display has a 7 ms temporal response, and 10-bit color rendering. The maximum luminance of the display is 329 cd/m\(^2\) (red is 88, green is 117, and blue is 124 cd/m\(^2\)).

Forces were measured using a JR3 85M35A-U560 63N4S force sensor. The raw sensor data was sampled at 1 KHz, resampled to match HFI’s control rate, and was finally filtered using a 75 Hz low pass filter to remove high frequency sensor noise.

1.6 Human Subjects

Subjects were healthy right-handed males with no history of motor disorders: S1, 29y, 185lb, 5\(^\prime \)9\({^\prime }{^\prime }\); S2, 19y, 170lb, 6\(^\prime \)2\({^\prime }{^\prime }\); S3, 21y, 160lb, 5\(^\prime \)8\({^\prime }{^\prime }\). Informed consent was obtained in advance on a protocol approved by the Institutional Review Board (IRB) at Stanford University.

Rights and permissions

Reprints and Permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Menon, S., Ganti, H., Khatib, O. (2016). Using Haptic fMRI to Enable Interactive Motor Neuroimaging Experiments. In: Hsieh, M., Khatib, O., Kumar, V. (eds) Experimental Robotics. Springer Tracts in Advanced Robotics, vol 109. Springer, Cham. https://doi.org/10.1007/978-3-319-23778-7_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-23778-7_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-23777-0

  • Online ISBN: 978-3-319-23778-7

  • eBook Packages: EngineeringEngineering (R0)