Spectral hallmark of auditory-tactile interactions in the mouse somatosensory cortex

Zhang, Manning; Kwon, Sung Eun; Ben-Johny, Manu; O’Connor, Daniel H.; Issa, John B.

doi:10.1038/s42003-020-0788-5

Spectral hallmark of auditory-tactile interactions in the mouse somatosensory cortex

Article
Open access
Published: 11 February 2020

Volume 3, article number 64, (2020)
Cite this article

Download PDF

You have full access to this open access article

Communications Biology

Spectral hallmark of auditory-tactile interactions in the mouse somatosensory cortex

Download PDF

4984 Accesses
1 Altmetric
Explore all metrics

Abstract

To synthesize a coherent representation of the external world, the brain must integrate inputs across different types of stimuli. Yet the mechanistic basis of this computation at the level of neuronal populations remains obscure. Here, we investigate tactile-auditory integration using two-photon Ca²⁺ imaging in the mouse primary (S1) and secondary (S2) somatosensory cortices. Pairing sound with whisker stimulation modulates tactile responses in both S1 and S2, with the most prominent modulation being robust inhibition in S2. The degree of inhibition depends on tactile stimulation frequency, with lower frequency responses the most severely attenuated. Alongside these neurons, we identify sound-selective neurons in S2 whose responses are inhibited by high tactile frequencies. These results are consistent with a hypothesized local mutually-inhibitory S2 circuit that spectrally selects tactile versus auditory inputs. Our findings enrich mechanistic understanding of multisensory integration and suggest a key role for S2 in combining auditory and tactile information.

Basic Science: Pain Mechanisms and Pathways

The Superior Colliculus: Cell Types, Connectivity, and Behavior

Article Open access 28 April 2022

What if worms were sentient? Insights into subjective experience from the Caenorhabditis elegans connectome

Article Open access 23 August 2023

Introduction

Our understanding of the external world is derived from what our senses can tell us¹. We can enrich this representation of the environment by combining inputs across disparate sensory modalities². However, maintaining a coherent percept requires active moment-to-moment integration of multisensory inputs along with resolution of inconsistencies between modalities³. How this active process is carried out at the level of neural populations is an open question.

Two stimulus modalities that are deeply entangled are sound and touch. Mechanical forces generated by physical contact not only activate somatosensory mechanoreceptors but also generate acoustic waves that are detected by the machinery of the auditory system⁴. Thus these stimuli are often highly correlated in the environment⁵. A significant behavioral consequence of this interaction is observed when tactile and auditory inputs are not in register: as a human subject touches a surface, if a sound is played that is different from what is expected from the tactile sensation of the surface, the reported roughness of the surface is altered in a manner dependent on the sound frequency⁶. This “parchment-skin illusion” points to the deep bond shared by these modalities and hints at the centrality of stimulus frequency as a parameter that may bind them together^7,8. Consistent with this idea, the secondary somatosensory cortex of mice can respond to sound⁹. Furthermore, retrograde tracing in Mongolian gerbils has revealed direct synaptic projections from primary somatosensory cortex to primary auditory cortex¹⁰ and single unit recordings in behaving monkeys have shown that neurons in anterior parietal areas respond to task-related auditory cues¹¹, hinting at a cohesive sound-touch integration circuit within cortex.

In this study, we leverage these observations to probe the neural correlates of multisensory integration in the neocortex. We aim to determine how stimulus frequency is encoded by neurons in the somatosensory system, whether concurrent sound can influence these responses, and how this information is carried across a population of neurons. Thus, we performed large-scale two-photon Ca²⁺ imaging in multiple cortical regions: the primary (S1) and secondary (S2) somatosensory cortices, the insular somatosensory field (ISF), and the auditory cortex. We find that neurons in S1 and S2 encode the frequency at which a whisker is deflected. Concurrent sound modulates these responses, with responses to low tactile frequencies often completely abolished. In S2, we find a small number of sound-selective neurons whose responses are reciprocally attenuated by high-frequency whisker deflections. These frequency-dependent interactions point towards a spectrally-dependent mutually inhibitory circuit between touch-selective and sound-selective neurons and shed light on the neural circuits that may underlie the computations involved in multisensory integration.

Results

Widefield Ca²⁺ imaging localizes tactile and auditory areas

We performed widefield Ca²⁺ imaging of the left temporal lobe of mouse neocortex to identify regions that respond to either tactile or auditory stimuli individually. Transgenic mice expressing the Ca²⁺-indicator GCaMP6s under pan-neuronal promoters^12,13,14 were implanted with a chronic cranial window exposing up to 5 mm of the left hemisphere^15,16. This approach allowed us to capture the majority of the auditory cortex along with the primary somatosensory (S1), the secondary somatosensory (S2), and the insular somatosensory (ISF; insular somatosensory field) cortices all in the same window. Mice were habituated to head fixation while running on a treadmill. During widefield fluorescence imaging, either tactile or auditory stimuli were delivered to the contralateral target whisker or ear, respectively (Fig. 1a). In response to a sinusoidal 128 Hz stimulus applied to the C2 whisker, transient increases in GCaMP6s fluorescence were observed in somatosensory areas, with smaller responses in the auditory fields (Fig. 1b, c). A spatial map of these changes, which averages the change in fluorescence over 1 s of the response for stimulus frequencies ranging from 36 to 128 Hz (Fig. 1d), highlights three distinct locations that were strongly responsive to tactile stimuli. Based on stereotaxic coordinates (predicted location of C2 barrel of S1 indicated by red cross), the most medial locus was identified as S1, the middle locus corresponded to S2, and the most lateral locus was consistent with the location of ISF identified previously with intrinsic optical imaging¹⁷. In response to auditory stimuli composed of sinusoidal amplitude modulated (SAM; Supplementary Fig. 1) tones, we observed robust fluorescent responses in regions corresponding to the auditory cortex (Fig. 1e, f). SAM tones were chosen to approximate a correspondence between the envelope of auditory stimuli and the frequency of the sinusoidal waveform of the tactile stimuli used throughout this study (in the range of 2 to 128 Hz). Interestingly, we observed a small transient in ISF, consistent with overlap of the nearby insular auditory field (Fig. 1g, h)^17,18.

**Fig. 1: Responses to tactile and auditory stimuli under widefield Ca²⁺ imaging.**

Tuning to the frequency of tactile stimuli in S1 and S2

Having located each cortical area within our cranial window, we next examined the responses of individual layer 2/3 neurons across multiple somatosensory regions. Using two-photon Ca²⁺ imaging, we monitored the activity of hundreds of neurons within a field-of-view (FOV) spanning 465 × 465 μm², a region sufficiently large to include the C2 barrel column in S1 (Fig. 2a)¹⁹. To measure evoked tactile responses, we chose sinusoidal piezo-driven tactile stimuli with frequencies ranging from 2 to 128 Hz presented in random order. The upper limit of 128 Hz was imposed to avoid any piezo-generated sound within the hearing range of mice (see Methods and Supplementary Fig. 1). We found 16% of neurons in S1 were responsive to passive whisker stimulation (598 of 3723 neurons, 16 imaging fields from 8 mice). For comparison, a previous study found 16.8% of layer 2/3 excitatory neurons in the principal column responded to touch while 39¹⁹ or 37%¹⁶ showed task-related tactile responses. Figure 2b displays individual baseline-normalized fluorescent responses \(\left( {\delta \hat F = {\mathrm{\Delta }}F/F_0} \right)\) of two exemplar neurons to tactile stimuli of increasing frequency (multiple trials in gray; ensemble average in black, sorted according to stimulus frequency). A normalized tactile response for each stimulus was then estimated as the ratio of the mean fluorescence change \(\left( {\delta \hat F_{mean}(i)} \right)\) for the given stimulus (i) to the maximal fluorescence change (\({\max } _i {\delta} \hat F_{mean}(i)\)) observed across all stimuli. Plotting the normalized tactile response as a function of frequency revealed a normalized tactile tuning curve (Fig. 2c). The best tactile frequency (BTF, see Methods and Supplementary Fig. 2) served as a convenient measure of the tuning preference of individual neurons. Figure 2d shows the overall distribution of BTF for tactile-responsive neurons in S1. Neurons with low BTFs tended to have biphasic tuning curves while neurons with high BTFs exhibited monotonically increasing responses (Fig. 2c), although we could not exclude the possibility that high-BTF neurons may have responses that decrease at frequencies beyond 128 Hz. In S1, neurons with high BTFs (>60 Hz) outnumbered low-BTF neurons 6-fold in relative prevalence.

**Fig. 2: Responses to tactile stimuli in somatosensory cortices under two-photon Ca²⁺ imaging.**

We further assessed tactile responses of neurons within S2 (Fig. 2e), ~1 mm lateral to S1 based on widefield Ca²⁺ imaging results (Fig. 1d). We found that 9% of all neurons were responsive to tactile stimuli (1313 of 14051 neurons, 67 imaging fields from 19 mice). Again, examination of tactile tuning curves revealed the presence of neurons with varying response profiles (Fig. 2e, f). Neurons with low BTFs were modestly but significantly more prevalent in S2 compared to S1 (19.8% for S2 and 15.9% for S1, p < 0.05 by Chi-square proportion test; Fig. 2g). Neurons with high BTFs nonetheless outnumbered low-BTF neurons by ~4-fold. The coarse similarity in response characteristics of both S1 and S2 neurons indicates that tactile frequency tuning may be a general feature for encoding tactile sensation in multiple regions of the somatosensory cortex.

Concurrent auditory input modulates tactile responses in S2

To identify possible multimodal integration, we explored whether concurrent auditory stimuli could modify tactile responses in the somatosensory cortex, focusing initially on S2 given its putative role in higher-order tactile processing^20,21,22. Accordingly, we measured fluorescence changes of single neuron responses in S2 under three stimulus conditions: (1) tactile stimuli alone, (2) auditory stimuli (SAM tones) alone, and (3) tactile and auditory stimuli presented concurrently. Figure 3 displays responses of three tactile-selective neurons within a single FOV demonstrating various multisensory effects. Neuron 1 exhibited robust responses to high tactile frequencies (black, Fig. 3b) that were enhanced upon concurrent presentation of acoustic stimuli (red, Fig. 3b). The normalized responses obtained with tactile stimuli alone and with concurrent tactile and auditory stimuli are overlaid in Fig. 3c. By contrast, neuron 2, which was also tuned to the high-frequency tactile stimuli, demonstrated inhibition of this response when concurrent auditory stimulation was added (Fig. 3b, c). Finally, neuron 3 exhibited robust responses when probed with tactile stimuli alone (black, Fig. 3b) but was largely silenced when tactile and auditory stimuli were presented concurrently (red, Fig. 3b). All three neurons were nonresponsive to all sound stimuli tested when sound stimuli were presented alone (blue, Fig. 3b). These results illustrate a powerful mode of multisensory processing whereby the response of individual cortical neurons to their primary sensory modality is modulated by a concurrent secondary sensory input. These effects are diverse, allowing for neuron-specific computations of multisensory interactions.

**Fig. 3: Sound-modulated response to tactile stimuli in S2.**

To assess the prevalence of each mode of tactile-auditory integration in S2, we combined results from 67 FOVs across 19 animals. For all tactile-selective neurons in S2, we systematically probed for statistically significant changes in tactile responses by concurrent auditory stimulation (2-way ANOVA with a pre-established criterion of p < 0.05 for significance). We found that most neurons had tactile responses that were either inhibited or unchanged by sound, with neurons tuned to lower frequencies more likely to be inhibited (Fig. 3d). To probe the overall potency of sound modulation, we quantified the net change in aggregate tactile responses (average response elicited across all tactile frequencies) when evoked by concurrent tactile and auditory stimuli (ΔR, see Methods) (Fig. 3e). The relative strength of sound-driven inhibition was most potent in neurons tuned to lower frequencies (BTF up to 60 Hz: ΔR = 43%) in comparison to neurons tuned to higher frequencies (BTF > 60 Hz: ΔR = 31%; p < 0.001 by Wilcoxon rank-sum test) (Fig. 3e). In this manner, sound-driven inhibition appeared to be both more prevalent and more prominent in neurons tuned to lower frequencies, indicating a possible tactile frequency dependence in multisensory interactions.

Dissociation of motor behavior from sound-driven modulation

Ongoing locomotor activity can drastically alter activity in sensory cortices^23,24. To determine whether the observed sound-driven inhibition in S2 may be associated with differences in motor activity and not directly caused by sound stimulation (Fig. 4a), we recorded the animal’s movement velocity on a treadmill during two-photon imaging of neural activity and used a generalized linear model (GLM) to compare the influence of sound stimulation with motor activity on tactile responses. Tactile responses in exemplar neurons were divided into three categories of movement velocity, two categories of sound stimulation (off or on), and four categories of tactile stimulation frequency (Fig. 4b, c). For neuron 1 (Fig. 4b), responses are not appreciably altered by locomotion (compare rows), indicating that sound-driven inhibition of tactile responses cannot be explained by ongoing behavior. For neuron 2 (Fig. 4c), running diminished responses; however, sound also inhibited this neuron’s tactile responses. Using a GLM to quantify the contribution of sound (coefficient c, z-scored) and locomotion (coefficient d, z-scored) to each neuron’s tactile responses (see Methods), we found that neuron 1 was at best weakly inhibited by running velocity (d = −0.3) while neuron 2 was strongly inhibited (d = −2.9) (Fig. 4d). Importantly, in both cases sound is a strong inhibitory factor (c = −3.4 and −3.5, respectively).

**Fig. 4: Sound-driven inhibition of tactile responses is not explained by locomotive behavior.**

Across the population of tactile-responsive neurons in S2, those previously identified as unperturbed by sound (‘no change’) were found to have sound amplitudes near zero in the GLM (‘inhibited’), while those identified as inhibited by sound tended to have large negative values for the sound coefficient (Fig. 4e, f). Interestingly, for sound-inhibited tactile-responsive neurons, neurons tuned to lower tactile frequencies were inhibited by concurrent running behavior while neurons tuned to higher tactile frequencies were facilitated by running (blue curve, Fig. 4g). Across all BTFs, however, the coefficient for sound modulation was consistently of a larger amplitude (red curve, Fig. 4g). While running velocity tended to increase in over half the FOVs in S2 (58 of 108 FOVs, two-sample t-test comparing running during stimulus presentations + /− sound, criteria: p < 0.05), the coefficient for sound modulation was uncorrelated with the change in running behavior (Fig. 4h, p = 0.85 for Pearson’s correlation). In summary, consideration of locomotive behavior does not explain away the contribution of sound to the modulation of tactile responses (Fig. 4i).

Active whisking could also influence the responses of somatosensory neurons to passive whisker stimulation. To test the possibility that the observed sound-driven inhibition can be attributed to an indirect pathway involving active whisking, we transected the buccal branch of the facial nerve in a subset (n = 2) of mice, thus severely diminishing the ability of the mouse to whisk²⁵. We imaged whisker-responsive S2 neurons in these mice before and after cutting the nerve (Supplementary Fig. 3). Dramatic sound-driven inhibition of tactile responses was still observed (Supplementary Fig. 3b, c), and the proportion of neurons exhibiting sound-driven inhibition was preserved (58% before transection and 54% after transection, p > 0.5 by Chi-square proportion test; Supplementary Fig. 3d, e). Thus, active whisking is not crucial to the observed multimodal interactions.

Dependence of integration on modality and sound properties

In a subset (n = 3) of mice, we examined the possibility that these interactions may be generic across sensory modalities by using a number of different auditory stimuli and comparing the resultant changes in tactile responses to those induced by visual stimulation (Fig. 5). Responses are shown for two exemplar neurons. In neuron 1, tactile responses were suppressed by both auditory and visual modalities, while responses in neuron 2 were unchanged by sound but facilitated by visual stimuli (Fig. 5b, c). Across the population of S2 neurons, we found that the effect of sound was similar across the different sounds tested, with similar effects regardless of the specific sound (tone versus noise) or modulation frequency. Roughly 75% of neurons responded consistently. By contrast, light produced quantitatively different effects, with roughly 50% of neurons showing a different pattern of modulation (Fig. 5e). Thus, sound-driven inhibition of tactile responses appears to be modality-specific in some neurons but modality-independent in others. Modulation in this latter group of neurons may be mediated by a general mechanism that is common to visual and auditory stimuli, such as changes in attentional state.

**Fig. 5: Modality-specific multimodal modulation of tactile responses.**

Next, we examined whether sound-driven inhibition of tactile responses may depend on the specific properties of the sound stimulus used. Along with attaining the tactile frequency tuning curves of neurons in the absence and presence of a concurrent sound stimulus (Supplementary Fig. 4a, d), we measured responses to SAM tones in the absence and presence of tactile stimuli while varying either the carrier frequency (Supplementary Fig. 4b, c) or modulation frequency (Supplementary Fig. 4e, f) of the sound. Responses of sound-selective neurons in S2 were strongest for carrier frequencies between 6 and 24 kHz (Supplementary Fig. 4b, c, blue curves), while touch responses in tactile-selective neurons were selectively inhibited by those same carrier frequencies (orange curves). Meanwhile, the population of S2 sound-selective neurons, while individually dependent on modulation frequency (33 of 63 neurons), collectively exhibited only a small increase in their response as modulation frequency increased (Supplementary Fig. 4e, f, blue curves). Touch responses in tactile-selective neurons were individually agnostic to modulation frequency, with only 20 of 259 neurons showing any dependence (Supplementary Fig. 4e, f, orange curves). Yet, as a population, a weak trend was observed, with tactile-selective neurons more strongly inhibited by higher sound-modulation frequencies, which was weakly but significantly anti-correlated to the responses of sound-selective neurons (Supplementary Fig. 4f). These results indicate that the observed sound-driven inhibition of tactile responses in S2 is proportional to the total sound-driven response and is not a peculiarity of the precise frequencies that compose the auditory stimulus.

Concurrent sound modulates tactile responses in S1 and ISF

Primary sensory cortices are believed to be predominantly involved in unimodal processing, but the degree of multimodal interactions has long been debated. As a wealth of studies have illustrated potential multisensory interactions in the primary cortices^26,27,28, we examined whether tactile responses in mouse S1 are modulated by concurrent auditory stimuli. Figure 6b, c shows fluorescence recordings for three exemplar tactile-selective neurons within a single FOV in response to tactile stimuli alone or concurrent tactile and auditory stimuli. Neuron 1 exhibited minimal change to its tactile tuning relation following addition of concurrent sound. By contrast, tactile responses of both neurons 2 and 3 were inhibited by sound. All three neurons were nonresponsive to acoustic stimuli when presented alone (blue traces, Fig. 6b). At the population level, unlike neurons in S2, the tactile responses for a majority of high-BTF (69%) and low-BTF (75%) neurons in S1 were unaltered by auditory stimuli (48 and 28% for S2, respectively; p < 1e-10 by Chi-square proportion test for both categories when compared with S1). Nonetheless, we identified a substantial number of tactile neurons exhibiting sound-driven inhibition and a small number with sound-driven facilitation. The overall proportions of neurons exhibiting sound-modulation of tactile responses is summarized in Fig. 6d. Like sound-modulated neurons in S2, the degree of inhibition was stronger for neurons with lower BTFs (Fig. 6e) (p < 0.01 by Wilcoxon rank-sum test), while the degree of sound-driven facilitation was similar between the two types of tactile neurons (p > 0.1). These results clearly show the presence of sound-driven modulation of neurons in the primary somatosensory cortex.

**Fig. 6: Sound-modulated response to tactile stimuli in S1.**

We also measured response properties of neurons in the ISF, a poorly understood cortical region found lateral to S2. Here, we observed robust responses to tactile stimulation (Supplementary Fig. 5). In contrast to S1 and S2, we found a dearth of low-BTF neurons, with 18-fold more high-BTF neurons (Supplementary Fig. 5d). The percentage of tactile neurons suppressed by auditory stimulation resembled that of S2 (Supplementary Fig. 5e). The presence of similar modes of tactile-auditory interactions across S1, S2, and ISF suggests that multimodal integration occurs across a distributed somatosensory cortical network.

Sound preferentially inhibits low frequency tactile responses

Given the prevalence of sound-driven inhibition of tactile responses observed for neurons in the somatosensory cortex, we next sought to dissect spectral features of multimodal processing. We scrutinized tactile tuning relations in the presence and absence of concurrent auditory stimuli for four neurons with varying BTF within a single FOV in S2. Neuron 1, tuned to tactile frequencies between 8 to 32 Hz, was inhibited across all tactile frequencies (Fig. 7a, b). Interestingly, for neurons 2 and 3, sound-driven inhibition was nearly complete at low tactile frequencies (below 32 Hz) while the high-frequency tactile responses were largely spared. Similar effects were observed for neuron 4, which was responsive to high tactile frequencies, suggesting that sound-driven modulation is exquisitely dependent on the frequency of tactile stimulation: responses to low-frequency tactile stimuli are preferentially suppressed while responses to high-frequency tactile stimuli are relatively preserved.

**Fig. 7: Sound-driven inhibition in S2 depends on tactile frequency.**

To quantify the tactile spectral properties of sound-driven inhibition, we assigned neurons to five groups according to their BTFs (Fig. 7c). Normalized tactile tuning curves (Fig. 7c) in the absence (black) and presence (red) of concurrent sound stimuli were averaged for the five categories. To quantify the strength of sound-driven inhibition, we computed the ratio of normalized tactile responses in the presence of sound stimuli to those in the absence of sound stimuli at each individual tactile frequency (r_sound). Across all groups, r_sound increased monotonically with increasing tactile stimulus frequency. Low frequency tactile responses were strongly diminished (r_sound ~ 0.1) by sound stimuli while high frequency tactile responses were more weakly perturbed (r_sound ~ 0.8) (Fig. 7d). The tactile frequency for half-inhibition increased only weakly as a function of the BTF for the given class of neurons (Fig. 7g). Overall, these results show that sound-driven inhibition depends upon tactile stimulus frequency with low frequency tactile stimuli preferentially suppressed.

To exclude the possibility that the sound-driven inhibition curve is particular to the parameters of the SAM tone, we measured r_sound at different sound modulation frequencies (Fig. 8). Inhibition in individual neurons was preserved across sound modulation frequency (Fig. 8a) and, for the population of sound-inhibited neurons in S2, r_sound curves did not shift (Fig. 8b–e, p = 0.14 for Pearson’s correlation between tactile frequency at which r_sound = 0.5 and the SAM modulation frequency). Because the calculation of r_sound is prone to error for small responses seen with low-frequency tactile stimuli (Fig. 8c), whether r_sound is monotonic or u-shaped could not be firmly determined.

**Fig. 8: Inhibition of S2 tactile responses is invariant to sound modulation frequency.**

This spectral dependence of sound-modulation of tactile responses has two distinct functional consequences. First, the preferential sound-driven inhibition at low tactile frequencies results in a rightward shift in the overall tactile tuning relation for individual neurons. This effect manifests as a positive change in BTF for a majority of neurons (Fig. 7e). Second, the overall inhibition of tactile responses is higher for neurons with low BTF (Fig. 7f).

Touch inhibits responses of sound-selective neurons in S2

A fraction of neurons in S2 that were unresponsive to tactile stimuli were instead responsive to auditory stimuli (1.2% of neurons in S2, 161 neurons across 67 FOVs); for reference, 0.3% of neurons in S1 were sound-selective (12 neurons across 16 FOVs), which was significantly less than in S2 (p < 1e-5 by Chi-square proportion test). We aimed to determine if these auditory responses were influenced by simultaneous tactile stimulation. An exemplar sound-selective neuron in S2 did not respond to tactile stimuli at any frequency (Fig. 9b, black traces) but did respond to sound alone (Fig. 9b, blue trace). When the same sound was paired with different tactile frequencies, the sound response was almost completely abolished at the highest tactile frequencies (Fig. 9b, red traces). The normalized tuning curves demonstrate the tactile frequency-dependent suppression of sound responses in this neuron (Fig. 9c). This trend was observed for the population-averaged tuning curve of sound-selective neurons in S2, with suppression occurring for tactile frequencies above 32 Hz (Fig. 9d). Note that analysis was limited not just to sound-selective neurons (132 neurons in S2) but to neurons selective to the specific auditory stimulus used in the combined stimuli (82 neurons in S2 from 19 FOVs across 7 mice). The suppression was quantified by the response ratio, defined as (r_hi − r_lo)/(r_hi + r_lo), where r_hi is the average response to sounds paired with high frequency (76–128 Hz) tactile stimulation and r_lo for pairing with low frequency (2–8 Hz) tactile stimulation. The distribution of these values for sound-selective S2 neurons (Fig. 9e) indicates that sound responses in many neurons were decreased by high frequency tactile stimuli (31 of 82 neurons met criteria for significant suppression). This suppression was not particular to the choice of SAM tone modulation frequency: the tactile frequency that suppressed the sound response to 1.5x above baseline was flat across a range of modulation frequencies (Fig. 8f, p = 0.66 for Pearson’s correlation).

**Fig. 9: Sound responses in S2 but not in auditory cortex are modulated by touch.**

To ascertain whether this effect was specific to S2, we measured the effect of tactile stimulation on the responses of sound-selective neurons in auditory cortex. The addition of tactile stimulation at any frequency did not change the response of an exemplar neuron to sound (Fig. 9g, h) and, across the population of sound-responsive neurons in auditory cortex, neurons exhibited minimal tactile frequency dependent changes in their sound response (Fig. 9i, j). Of 132 neurons that responded to the auditory stimulus used in the combined stimulus (out of a total of 267 neurons responsive to any auditory stimulus), only 11 met criteria for significant suppression, which was a significantly lower fraction than in S2 (p < 1e-6, Chi-square proportion test). Thus, tactile suppression of auditory responses in S2 is not unique but is much stronger than in auditory cortex.

Spatial layout of functional cell types in S2

Given that auditory responses of sound-selective neurons in S2 are often robustly inhibited at high tactile frequencies, which matches the inflection point of sound-driven suppression of tactile-selective neurons in S2 (Fig. 8e, f), we posit a mutually inhibitory local circuit within S2 that serves to regulate the strength of tactile-auditory inhibition (Fig. 10a). Under this model, sound-selective neurons inhibit tactile responses of touch-selective neurons across the spectrum of whisker deflection frequencies. At low frequencies, this inhibition is nearly total, but at high frequencies the stronger activation of touch-selective neurons as a population and their reciprocal inhibition of the sound-selective population of neurons is enough to tilt the balance towards tactile responses.

**Fig. 10: Spatial layout of multisensory neurons in S2.**

We asked whether S2 exhibited any spatial clustering of neurons based on their functional properties, which could provide indirect evidence for a mutually inhibitory circuit. This patterning should reflect the preference of cortical neurons to connect to their nearest neighbors within 100–200 μm²⁹. In particular, sound-selective neurons that were not modulated by tactile stimuli should be located further away from the C2-whisker-responsive region of S2 than their touch-modulated counterparts. To facilitate comparison of FOVs across mice, we used the local (~100 μm radius) soma-excluded neuropil signal at each pixel in the image to form a map of tactile response strength (Fig. 10b–d). From this map, we could identify the center of the S2 whisker-responsive region (black cross, Fig. 10d), allowing us to register all S2 neurons across all FOVs (Fig. 10e). Neurons are color-coded according to whether they are touch-selective (responsive only to touch; blue circles) or sound-selective (responsive only to sound; orange and hot pink circles). Neurons responsive to both touch alone and sound alone (n = 43) were excluded from this analysis. We found that touch-selective neurons were more clustered (assessed by distance to the center of the S2 whisker-responsive region) than the average neuron in the field (blue and black curves, Fig. 10f). Sound-selective neurons that were not inhibited by touch exhibited a similar distribution to all neurons (orange and black curves, Fig. 10f), indicating that they were equally distributed across a typical FOV and perhaps even partially excluded from the whisker-responsive center of S2. However, when we examined the distribution of sound-selective neurons that showed significant suppression of responses by tactile stimuli, as in Fig. 9b, c, the distribution of distance from the S2 core now closely mirrored that of touch-selective neurons (hot pink and blue curves, Fig. 10f). These results indicate that both touch-selective neurons and touch-inhibited sound-selective neurons are embedded together in a specific region of S2, hinting at a local circuit linking these two functional subtypes. Further corroborating this model, spontaneous correlations between touch-inhibited sound-selective neurons with tactile-responsive neurons was lower than the correlations between non-touch-inhibited sound-selective neurons with tactile-responsive neurons (Fig. 10g), providing indirect evidence of inhibitory connectivity between touch-inhibited sound-selective neurons and local touch-selective neurons in S2.

Discussion

In this work, we asked how information from somatosensory and auditory inputs is integrated in the mouse neocortex. With two-photon Ca²⁺ imaging, we investigated large populations of layer 2/3 neurons across somatosensory and auditory areas with single cell resolution. We found that neurons across somatosensory cortices are tuned to the frequency of tactile stimulation. The addition of concurrent sound resulted in modulation of these tactile responses in both S1 and S2, and this modulation typically manifested as a suppression of the response. Moreover, the degree of suppression depended on tactile frequency, with responses to low frequencies more inhibited than responses to high frequencies. We also identified a population of neurons in S2 responsive to sound but not to touch. Unlike in auditory cortex, sound responses of many (31 of 82) sound-selective neurons in S2 were strongly inhibited by addition of tactile stimuli at high tactile frequencies. These neurons were spatially colocalized with S2 touch-selective neurons.

The detection of the frequency of mechanical vibrations is important for animals to discern surface texture and to handle tools^30,31, and tuning to spectral frequency in the somatosensory system can encode texture information³². In our study, the presence of well-tuned neurons in both S1 and S2 supports the notion that tactile frequency tuning may be a general organizational feature for mouse tactile sensation. The higher proportion of neurons with tuning to lower tactile frequencies in S2 than in S1 may reflect differences in thalamocortical inputs to the two regions. S1 receives strong thalamic drive from the ventral posterior medial nucleus (VPM), while S2 receives a larger share of its thalamocortical input from the posterior medial nucleus (POm)^33,34. Interestingly, although both POm and VPM cells show adaptation, causing decreased response amplitude under high frequency stimulation, POm cells exhibit earlier adaptation than VPM cells³⁵ and as a result are tuned to lower frequencies than VPM cells. Thus, the tuning properties of neurons in S2 may be inherited from the response properties of thalamic neurons, although it could also reflect longer temporal integration windows in higher areas of cortex³⁶.

We found that the addition of an auditory stimulus modulated tactile responses in both S1 and S2, consistent with the sound-driven hyperpolarizing currents previously observed in mouse S1³⁷. This modulation has three notable features: (1) Although a similar proportion of neurons in both S1 and S2 were facilitated by sound, more neurons in S2 were inhibited than in S1 (Figs. 3d and 5d). (2) Inhibition of neurons tuned to low tactile frequencies in both S1 and S2 was more severe than inhibition of neurons to high tactile frequencies in the same regions (Figs. 3e and 5e). (3) Sound-driven suppression in S2 is tactile frequency dependent, with stronger inhibition occurring at lower tactile frequencies (Figs. 7 and 8). Previous studies in human and non-human primates have revealed that multimodal integration improves detection of events in the environment^3,38,39. The optimal integration of competing sensory cues involves dominance of the most reliable sensory cue to minimize variance in the estimation of the true stimulus³. This evaluation of reliability between different sensory cues is a dynamic process, with the weight or value of each stimulus modality being continuously updated³⁹. Low frequency tactile stimulation is potentially less salient of a signal than high frequency tactile stimulation, since it comprises lower velocity whisker motions. Indeed, we observed more suppression of tactile responses at lower tactile stimulus frequencies than at high frequencies (Figs. 7 and 8), indicating that auditory responses are more dominant when tactile stimuli are weak. This result is consistent with the prior observation that, during optimal multimodal integration, the more reliable stimulus modality dominates the response⁴⁰. On the other hand, this frequency-dependent integration is complementary to “inverse effectiveness,” where multimodal integration is largest for weak multimodal stimuli near threshold and decreases with increasing stimulus intensity, as has been reported in the superior colliculus^41,42.

Sound-touch modulations may involve more than just direct interactions between the unimodal stimuli themselves. Attention, arousal, motor behavior, and hidden internal states can be influenced by sensory stimuli and they, in turn, can influence the response to a sensory stimulus^{23,24,43,44,45}. Indeed, multisensory integration, if relevant to behavior, should be associated with a change in the internal state of the animal. Pointing towards this complex interplay of stimuli and internal states, we found that locomotive behavior, while able to influence sensory responses, could not explain sound-driven inhibition of tactile responses on its own (Fig. 4). To untangle these potentially complex interconnections, the underlying cellular and network mechanisms that mediate these interactions need to be uncovered. While our present work focused on neurons in layers 2/3, a fruitful locus of study would be layers 1 and 6, where crossmodal⁴⁶ and neuromodulatory⁴⁷ inputs are known to be stronger and may thus gate sensory inputs and mediate attentional effects.

Previously, it was believed that multimodal influences on activities within classically defined unimodal areas are mediated by feedback from multisensory integration in higher-order cortical regions^48,49. However, human studies using event-related potentials (ERPs) suggest that these multimodal influences may also be carried in the feedforward inputs coming from subcortical regions to unimodal regions^48,50,51. In the present study, we identified a small (1.2%) population of sound-selective neurons within S2 itself. Although prior studies have shown non-matching neurons in primary cortices that respond solely to other sensory modality inputs⁵², the sound-selective neurons we found may play a special computational role in multimodal integration. The sound-driven responses in these neurons were strongly suppressed at high tactile frequencies (Fig. 9a–e), and those inhibited by tactile stimuli neurons are clustered near the center of the whisker-responsive region of S2 (Fig. 10), similar to the spatial organization of non-matching neurons seen in other studies^51,52. The existence of touch-inhibited sound-selective neurons in S2 indicates that they may play a role in local sound-driven suppression observed in tactile-selective neurons of S2. This winner-take-all circuit (Fig. 10a) could dynamically select a stimulus modality at each moment and, under the right conditions, would be consistent with divisive normalization, a model that has been proposed as a driving force behind multisensory interactions^53,54,55.

Methods

Animal surgery and general procedures

All animal procedures were approved by the Johns Hopkins Institutional Animal Care and Use Committee. Imaging experiments used transgenic mice 10–22 weeks of age expressing the genetically encoded Ca²⁺ indicator GCaMP6s⁵⁶ under a pan-neuronal promoter. Most experiments used GP4.12 Thy1-GCaMP6s mice (JAX No. 025776)¹². A small subset of tetO-GCaMP6s mice (JAX No. 024742) were also used for imaging¹⁴. Finally, lox-GCaMP6s mice (JAX No. 024106)¹³ were crossed with Syn1-Cre mice (JAX No. 003966)⁵⁷, resulting in lox-GCaMP6s-Syn1-Cre mice. In total, 18 males and 5 females were imaged for this study. Among them, 17 were GP4.12 Thy1-GCaMP6s mice, 3 were tetO-GCaMP6s mice, and 3 were lox-GCaMP6s-Syn1-Cre mice.

For surgery to implant headposts and chronic imaging windows, general anesthesia was provided by 1–1.5% isoflurane in 0.5 L/min O₂. Body temperature was monitored through a rectal probe and maintained at 37 °C with a heating pad. Lidocaine (20 mg/ml) was injected at the incision site for local anesthesia while carprofen (5 mg per kg, i.m.) and dexamethasone (5 mg per kg, i.m.) were given for pain and inflammation. Normal saline (0.5 ml, i.p.) was given to avoid dehydration. Skin and muscle over the left hemisphere were removed to expose skull overlying auditory and somatosensory cortices and a custom headpost was attached to the skull using UV-cured primer (Kerr OptiBond) and dental cement (Heraeus Charisma A1). A 4–5 mm craniotomy spanning both auditory and somatosensory cortices was performed. After removing the skull, the craniotomy was filled with a transparent silicone gel (3–4680, Dow Corning, Midland, MI)⁵⁸ to help maintain the stability and clarity of the chronic window. Next a 4 or 5 mm (No. 0 or 1) cover glass was secured with dental cement above the window. Mice were given buprenorphine (0.1 to 0.5 mg per kg, s.c.) for pain and allowed to recover from anesthesia on a warmed pad. After surgery, mice were periodically given buprenorphine for pain along with carprofen and dexamethasone. Mice were given 10–14 days to recover before imaging sessions were started. Location of the C2 barrel column in S1 was estimated by stereotaxic measurement of its expected location relative to lambda (1.3 mm posterior and 3.5 mm lateral). This location matched the most medial of the three regions responsive to tactile stimulation as seen with functional widefield imaging (Fig. 1).

In a subset of experiments, to examine whether sound-driven inhibition was dependent on active whisking, the facial nerve was transected. The buccal branch of the facial nerve innervates the mystacial pad musculature⁵⁹, which in turn controls whisker movement⁶⁰. General anesthesia was achieved with isoflurane (1.5% in 0.5 L/min O₂) and body temperature was maintained at 37 °C. Local anesthetic (lidocaine, 20 mg/ml) was injected at the incision site, which was 1–2 mm caudal of the E1 whisker. The surrounding tissue was dissected to reveal the buccal branch of the facial nerve at the junction of the ramus superior buccolabialis and ramus inferior buccolabialis^25,61. Micro scissors were used to transect the nerve just anterior to the junction point and proximal to entering the mystacial pad. Skin closure was achieved with nonabsorbable suture (4–0 Prolene) and Vetbond (3 M). Mice were allowed 2–3 days to recover. The absence of active whisking ipsilateral to the site of the transection was used to assess the success of the surgery. Imaging was performed as described below until active whisking returned, typically 2 weeks after the initial cut.

Experimental setup

During imaging sessions, all mice were awake and head-rotated ~45° to make the imaging plane perpendicular to the microscope objective. Mice were first adapted to running on a custom-built treadmill while head-fixed for 1–2 session of 20 to 40 min each. The treadmill consisted of a spherical ball with a single axis of rotation. Once habituated, imaging sessions began. Mice were able to tolerate head fixation for up to 2 or 3 h. Running speed was recorded with an optical encoder (E5-2500-188-IE-S-H-D-B, US Digital). For whisker stimulation, all whiskers of the right whisker pad were trimmed except for the C2 whisker, facilitating isolated stimulation of a single whisker.

Imaging

For widefield imaging, a white light source (LED Engin LZ1-10CW00) was used for illumination. Illumination light and fluorescence signals were filtered through a GFP filter cube (460/50 excitation, 540/50 emission). A 4× 0.13 NA objective (Olympus) and a Photometrics Evolve 512 Delta camera were used to collect the emitted light. The field of view was roughly 5.5 × 5.5 mm². The frame rate was 20 Hz.

Two-photon Ca²⁺ imaging was performed with an Ultima system (Prairie Technologies) built on an Olympus BX61W1 microscope. A mode-locked laser (Coherent Chameleon XR Ti:Sapphire) tuned to 950 nm was raster scanned at 5 Hz for excitation while emitted GCaMP6s fluorescence was collected through a green filter (525/70 nm). Laser power at the sample was 20–80 mW. Dwell time was set to 2 μs. To increase imaging speed, resolution along the y-axis was reduced by a factor of 4. The final pixel size was 0.9 × 3.6 μm. A 20× 1.0 NA objective (Olympus) was used to yield a 465 × 465 μm² field of view. Imaging depths were between 150–350 μm.

Tactile stimuli

To stimulate the target whisker, a piezoelectric bending actuator (Q220-A4-203YB, Piezo systems, Inc., MA) moving in the rostral-caudal direction was attached to a 2 cm sewing needle and the target whisker was passed through the eye of the needle at a distance of 2 mm from the face. The needle coupled the whisker to the piezo movement. The shape of the eye of the needle allowed the whisker some freedom to move in the medial-lateral direction but not in the rostral-caudal direction. Positioning the needle close to the base of the whisker reduced flexion of the whisker during stimulation, thus minimizing nonlinear mechanical transformations of the sinusoidal driving signal. To facilitate precision in the positioning of the needle, we attached the piezo to a micro-manipulator (MT-XY Compact Dovetail Linear Stage, Newport Corporation). The piezo itself was driven by a voltage driver (MDT694B, Thorlabs) with a 10× gain. Thus, the 0–10 V signal delivered by the DAQ card was expanded to a range of 0–100 V to drive the piezo.

A single tactile stimulus consisted of a 500 ms sinusoidal wave. One tactile trial consisted of a set of sixteen individual stimuli with onsets spaced by 2 s. Frequency of each sinusoidal stimulus was randomized between 2 to 128 Hz, spaced logarithmically. The amplitude of piezo deflection was 0.6 mm. With the needle 2 mm away from skin surface, a 2 Hz stimulus was equivalent to a mean angular speed of 66.8°/sec, calculated as tan⁻¹(0.6 mm/2 mm) × 2 Hz × 2 (one forward and one backward deflection per cycle). Thus, mean angular speed of our tactile stimuli is calculated from frequency as 33.4° per sec × (frequency), where frequency is given in Hz. In this work, we report the stimulus parameters in frequency. The piezo was calibrated monthly by imaging piezo movement to measure maximum deflection displacement and angular velocity. Once angular velocity dropped by more than 20%, the piezo was replaced.

An important consideration in this work was whether the delivery of tactile stimulation with the piezo generated any audible noise within the hearing range of mice. We performed a noise calibration while driving the piezo at frequencies ranging from 2 to 256 Hz. Sound was recorded with a Sokolich probe placed directly next to the piezo. We found that no noise audible to mice (across the mouse hearing range of 3–100 kHz) was generated as long as the frequency of piezo deflection was kept lower than 140 Hz. Above 140 Hz, though, audible sounds were generated (Supplementary Fig. 1d, e). Thus, all stimuli used in this study were limited to a maximum of 128 Hz. In support of this assertion, only 2 out of 2036 neurons recorded in auditory cortex passed criteria for a response to stimulation by the piezo alone. To test the opposite, whether acoustic stimuli could cause any vibration of the piezo, we imaged the piezo through the microscope objective at a frame rate of 363 Hz while playing sounds at the maximum amplitude possible with our speakers. We found no detectable movement of the piezo under imaging.

Auditory stimuli

The imaging set-up was located in a sound-attenuated room (Acoustical Solutions, AudioSeal ABSC-25) with noisy equipment placed outside the room. Noise calibrations showing ambient noise to be outside the mouse hearing range have been previously published¹⁵. Sound stimuli were delivered through a free-field speaker (LCY K100, Ying Tai Corporation) located 15 cm away from the right ear.

The main stimuli used were sinusoidal amplitude modulated (SAM) tones, which were composed of a pure tone (the carrier frequency) with a low frequency amplitude modulation. To search for sound responses, SAM tones were delivered over a wide range of carrier frequencies (3 to 96 kHz, 16 frequencies) or amplitude modulation frequencies (2 to 256 Hz, 8 frequencies). The duration of each stimulus was 500 ms (Supplementary Fig. 1b). We tested a variety of additional sounds in a subset of experiments, including pure tones and broadband noise (Fig. 5). Pure tones were gated by 5 ms squared cosine ramps. White noise was bandlimited between 3 to 96 kHz and, if indicated, was amplitude modulated by a sinusoidal envelope (SAM Noise). Neurons were categorized as sound-responsive if they responded to any of the auditory stimuli tested. All the stimuli were delivered in a random order and with a 2 s interval between onsets. Stimuli were played at −40 to −20 dB attenuation, corresponding to ~60–80 dB SPL using our speaker. Sound levels were chosen by looking for reliable responses in auditory cortical areas as visualized by widefield fluorescence imaging.

Combined stimuli

Combined stimuli consisted of simultaneous presentation of the auditory and tactile stimuli described above. The tactile stimuli were sixteen tactile deflections with different frequencies ranging from 2–128 Hz while a fixed auditory stimulus was presented concurrently with each of the sixteen tactile stimuli. For the auditory stimuli, SAM tones were used for every FOV, with the particular carrier and modulation frequencies chosen such that the stimulus had elicited a robust response in auditory regions during widefield fluorescence imaging (typically between 6–12 kHz for the carrier frequency and 8–64 Hz for the modulation frequency). As the choice of sound could have a bearing on the observed multisensory effects in somatosensory cortex, we also tested pure tones and broadband noise (with or without amplitude modulation) in some experiments (Fig. 5). Each stimulus lasted 500 ms and the tactile and auditory stimuli were co-initiated and co-terminated (Supplementary Fig. 1c). Stimulus order was randomized with a 2 s interval between the onset of each presentation.

Stimulus order

In general, we interleaved blocks of each stimulus condition, for example one block of tactile alone (consisting of 16 piezo frequencies) followed by one block of tactile + auditory (16 piezo frequencies in combination with one fixed sound stimulus). Within each block, the stimuli (such as the 16 piezo frequencies) were always randomized, stimuli lasted 500 ms, and intervals were 2 s. The blocks were interleaved in a semi-random order as chosen by the experimenter, with the goal being to spread each stimulus block across the duration of the imaging session to minimize the effects of habituation and bleaching. At least 4 repeats were attained for each condition, although typically we aimed for 5 to 10 repeats. Whenever responses are shown, all individual trials are included.

Widefield imaging analysis

For widefield imaging, a structured sparse encoding algorithm was used to detect the baseline fluorescence F₀^15,62,63, from which ∆F/F₀ = (F − F₀)/F₀ was calculated for each pixel. To facilitate comparison of changes across different regions of cortex, plotted time series data (Fig. 1b, c, e, f) were zeroed at the start of each stimulus.

Two-photon imaging analysis

We first performed a frame-by-frame image registration to correct for drift in our imaging field⁶⁴. Assisted by a structured sparse coding algorithm⁶², we then performed a semiautomatic segmentation of our movies acquired from a two-photon imaging session to acquire regions-of-interest (ROIs) corresponding to individual neurons. From each ROI, we generated brightness-over-times (BOTs). This process included neuropil subtraction, where we subtracted the signal from pixels within 75 μm of each cell (but excluding pixels containing other cell bodies). Using a low-pass filter that excluded large positive deflections (putative transients), a smooth baseline (F₀) was fit to each BOT, allowing for calculation of a normalized fluorescence signal ∆F/F₀. To increase the temporal accuracy of our Ca²⁺ signal and to decrease the influence of noise, we employed a non-negative deconvolution method with a time constant of 0.7 s to generate a correlate of spike probability⁶⁵, which was then thresholded to estimate events per time bin, which loosely corresponds to the frequency of spikes or firing rates for a given neuron^{66,67,68,69,70,71}. Events per time bin was used for constructing tuning curves and other measures as described in the section of “Response Analysis.” Parameters for choosing neurons in the segmentation algorithm were purposefully set to be sensitive so that no responsive neurons would be missed; conversely, many ROIs correspond to relatively inactive neurons or false detections. These ROIs would not reach significance for being responsive and thus do not alter any of the conclusions or analysis in this work other than the potential for inflating the number of nonresponsive neurons.

Registration

For most mice, the imaging window was initially mapped under two-photon imaging by tiling FOVs across the entire visible window and at multiple depths. Two-photon imaging fields were registered to widefield imaging fields by using vasculature landmarks. Information about the location of auditory (primary auditory cortex, anterior auditory field, and secondary auditory cortex) and somatosensory cortices (S1, S2, and ISF) obtained from widefield imaging was then used to identify the cortical identity of each two-photon imaging field.

Response analysis

Response analysis includes five major parts: (1) event rate normalization; (2) quality measure of responsiveness; (3) best frequency extraction; (4) construction of tuning curves; (5) calculation of sound-driven response change.

For event rate normalization, we used a 600 ms period after the start of a stimulus as our response window. The averaged baseline-corrected signal during that period yielded the event rate or response for each individual stimulus.

To judge whether a neuron is responsive to a given stimulus modality, we developed a metric that combines 3 measures: (1) the p-value of 1-way ANOVA to quantify whether there is a significant effect of stimulus frequency on the response, which indicates a neuron is tuned to stimulus frequency; (2) the total area under the tuning curve, a measure of driven rate; (3) the ratio of activity during the response window to activity outside the response window (Supplementary Fig. 2a, b). Each quantity is passed through a saturating nonlinearity and summed to yield a unitless measure of responsiveness (termed ‘score’ in Supplementary Fig. 2). This ad hoc metric is thresholded to categorize neurons as responsive or not responsive to a particular stimulus type. Individual neurons were scrutinized to test for the adequacy of this approach and the threshold chosen to best match our manual characterization of neurons. We chose this approach because any single measure was inadequate in capturing whether neurons were responsive. For example, measure 1 (1-way ANOVA) does not account for magnitude differences, so a strongly-responsive, broadly-tuned neuron may have an insignificant p-value (Supplementary Fig. 2b, neuron (1) while an unresponsive neuron with incomplete neuropil subtraction may have a significant p-value (Supplementary Fig. 2b, neuron (2). For further analysis, if the neuron responded only to tactile stimuli, we classified it as a tactile-selective neuron. Similarly, if the neuron only responded to any of the sound stimuli we tested, we classified it as a sound-selective neuron. Using a shuffle test applied to neurons from our population (100 shuffles of the times of stimulus presentation), only 1 in 4832 iterations passed criteria for touch selectivity and 1 in 14,980 for sound selectivity. We did find that sound-selective S2 neurons, while not passing criteria individually for being responsive to tactile stimuli, did on average show a small level of activity in response to tactile stimuli alone (black curve, Fig. 9d). This response was weak (~5× smaller than the sound response) and inconsistent, as confirmed by manual inspection of the 82 individual tuning curves, and thus may reflect weak and inconsistent tactile-related activity in these neurons.

A measure of best tactile frequency (BTF) was used to categorize neurons (Supplementary Fig. 2c). We used the average of two different approaches to measure best frequency: (1) the weighted average of all significant responses; (2) the frequency that yielded the maximal response. We chose to optimize for robustness as each of these standard measures on their own is susceptible to pitfalls. The first approach has a slight bias, as neurons tuned to high frequencies will be biased by the weighted mean towards lower frequencies. The second approach can be unreliable for noisy data. Nonetheless, the two measures are highly correlated (r = 0.913, Pearson’s correlation, Supplementary Fig. 2c), and BTFs in over 97% of responsive neurons were less than 1 octave apart by the two measures. Neurons with best frequencies less than or equal to 60 Hz are categorized as low-BTF neurons while neurons with best frequencies greater than 60 Hz are categorized as high-BTF neurons.

To construct tuning curves, we used two different strategies. For characterizing tuning properties of tactile-selective neurons in S1 and S2 (Fig. 2), we normalized the response for different tactile frequencies to the maximum response among all the tactile frequencies. For characterizing the sound modulation effect of response to tactile stimuli, we normalized the response to the maximum response of either tactile stimuli or tactile plus auditory stimuli. And, for experiments where we used a third modality (visual light stimulation in Fig. 5), we normalized the response to the maximum response across the three stimuli (tactile stimuli, tactile stimuli with concurrent sound stimuli, and tactile stimuli with concurrent visual stimuli).

To quantify the sound-driven response change of tactile stimuli, we calculated the ratio of response change (∆R) as \({\mathrm{\Delta }}R = {\sum} {\left( {R_{T + S} - R_T} \right)} /{\sum} {\left( {R_{T + S} + R_T} \right)}\), where R_T is the response to tactile stimuli alone, R_T+S the response to combined tactile plus auditory stimuli, and the summations are over all tactile frequencies tested (2 to 128 Hz).

For the registration of S2 imaging fields shown in Fig. 10, we first formed smoothed neuropil maps by taking a region centered on each pixel and calculating a cell-excluded average of the signal within 100 μm of that pixel. This neuropil BOT signal was processed similarly to the neuronal BOT signal as described above, with the only difference being that neuropil subtraction was not performed on the neuropil signal. The amplitude of the whisker-induced response as quantified by summing the neuropil ∆F/F₀ over a 600 ms window for each stimulus was used to generate an image of tactile response strength. The peak of this map was nominally labeled as the center of the S2 whisker region and used to register FOVs across experiments.

The generalized linear model (GLM) was used to model the response of individual neurons on a single trial basis. The output was taken as the mean response over a 600 ms window for each stimulus, as described above, and then normalized so that the maximum response at any time is equal to one. Dependent variables consisted of tactile stimulus frequency (binned into four separate frequency channels), the presence of sound, and the running velocity. Indicator functions were used for stimulus frequency and the presence of sound, while running velocity was passed through a logarithmic function: log₂(6*v+1)/10, such that the range would fall between 0 to 1 (v never went higher than 170.5 cm/s in our data). Velocity was calculated based on the ball diameter of 6.75 cm and 2500 counts per rotation on the optical encoder. A velocity of 50 cm per s resulted in a value of ~0.8234 fed into the GLM, while a velocity of 0 cm per s gave a value of 0. This transformation function was chosen as it yielded the best model fits for velocity modulated neurons in our population. The logit function was used as the link function, and the MATLAB function glmfit.m was used to perform the fitting. Resulting coefficients were then z-scored by dividing by their associated standard errors. Trials were included for tactile stimulation alone, tactile stimulation with SAM tones, SAM tones alone, and, if performed, “silent” trials with no tactile or auditory stimuli.

Statistics and reproducibility

Sample sizes were chosen based on the consistency of the observed results. Data were excluded if the imaging quality was poor as judged by brightness of the fluorescence signal, ability to observe active transients in individual cells, and general optical sharpness of the two-photon imaging field. Replication was only indirectly tested by using large sample sizes or by varying experimental conditions and observing the results. Individual statistical tests are described where referenced in the manuscript.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Data availability

Data is available upon request from the corresponding author. Image files are stored in tiff format and processed data are stored as MATLAB data structures.

Code availability

Code (written for MATLAB R2014A) is available upon request from the corresponding author.

References

Hobbes, T. Chapter I. Of Sense. Of Man, Being the First Part of Leviathan (1651).
Stein, B. E. & Meredith, M. A. Merging of the Senses (1993).
Ernst, M. O. & Banks, M. S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429–433 (2002).
Article CAS PubMed Google Scholar
Von Bekesy, G. Similarities between hearing and skin sensations. Psychol. Rev. 66, 1–22 (1959).
Article Google Scholar
Lederman, S. J. Auditory texture perception. Perception 8, 93–103 (1979).
Article CAS PubMed Google Scholar
Jousmäki, V. & Hari, R. Parchment-skin illusion: sound-biased touch. Curr. Biol. 8, R190–R191 (1998).
Article PubMed Google Scholar
Crommett, L. E., Pérez-Bellido, A. & Yau, J. M. Auditory adaptation improves tactile frequency perception. J. Neurophysiol. 117, 1352–1362 (2017).
Article PubMed PubMed Central Google Scholar
Yau, J. M., Olenczak, J. B., Dammann, J. F. & Bensmaia, S. J. Temporal frequency channels are linked across audition and touch. Curr. Biol. 19, 561–566 (2009).
Article CAS PubMed PubMed Central Google Scholar
Carvell, G. E. & Simons, D. J. Somatotopic organization of the second somatosensory area (SII) in the cerebral cortex of the mouse. Somatosens. Res. 3, 213–237 (1986).
Article CAS PubMed Google Scholar
Budinger, E., Heil, P., Hess, A. & Scheich, H. Multisensory processing via early cortical stages: connections of the primary auditory cortical field with other sensory systems. Neuroscience 143, 1065–1083 (2006).
Article CAS PubMed Google Scholar
Zhou, Y.-D. & Fuster, J. M. Somatosensory cell response to an auditory cue in a haptic memory task. Behav. Brain Res. 153, 573–578 (2004).
Article PubMed Google Scholar
Dana, H. et al. Thy1-GCaMP6 transgenic mice for neuronal population imaging in vivo. PLoS ONE 9, e108697 (2014).
Article PubMed PubMed Central CAS Google Scholar
Madisen, L. et al. Transgenic mice for intersectional targeting of neural sensors and effectors with high specificity and performance. Neuron 85, 942–958 (2015).
Article CAS PubMed PubMed Central Google Scholar
Wekselblatt, J. B., Flister, E. D., Piscopo, D. M. & Niell, C. M. Large-scale imaging of cortical dynamics during sensory perception and behavior. J. Neurophysiol. 115, 2852–2866 (2016).
Article CAS PubMed PubMed Central Google Scholar
Issa, J. B. et al. Multiscale optical Ca2+ imaging of tonal organization in mouse auditory cortex. Neuron 83, 944–959 (2014).
Article CAS PubMed PubMed Central Google Scholar
Kwon, S. E., Yang, H., Minamisawa, G. & O’Connor, D. H. Sensory and decision-related activity propagate in a cortical feedback loop during touch perception. Nat. Neurosci. 19, 1243–1249 (2016).
Article CAS PubMed PubMed Central Google Scholar
Gogolla, N., Takesian, A. E., Feng, G., Fagiolini, M. & Hensch, T. K. Sensory integration in mouse insular cortex reflects GABA circuit maturation. Neuron 83, 894–905 (2014).
Article CAS PubMed PubMed Central Google Scholar
Sawatari, H. et al. Identification and characterization of an insular auditory field in mice. Eur. J. Neurosci. 34, 1944–1952 (2011).
Article PubMed Google Scholar
Peron, S. P., Freeman, J., Iyer, V., Guo, C. & Svoboda, K. A cellular resolution map of barrel cortex activity during tactile behavior. Neuron 86, 783–799 (2015).
Article CAS PubMed Google Scholar
Carvell, G. E. & Simons, D. J. Biometric analyses of vibrissal tactile discrimination in the rat. J. Neurosci. 10, 2638–2648 (1990).
Article CAS PubMed PubMed Central Google Scholar
Jones, E. G. & Powell, T. P. An anatomical study of converging sensory pathways within the cerebral cortex of the monkey. Brain J. Neurol. 93, 793–820 (1970).
Article CAS Google Scholar
Lemus, L., Hernández, A., Luna, R., Zainos, A. & Romo, R. Do sensory cortices process more than one sensory modality during perceptual judgments? Neuron 67, 335–348 (2010).
Article CAS PubMed Google Scholar
Niell, C. M. & Stryker, M. P. Modulation of visual responses by behavioral state in mouse visual cortex. Neuron 65, 472–479 (2010).
Article CAS PubMed PubMed Central Google Scholar
Lee, S., Kruglikov, I., Huang, Z. J., Fishell, G. & Rudy, B. A disinhibitory circuit mediates motor integration in the somatosensory cortex. Nat. Neurosci. 16, 1662–1670 (2013).
Article CAS PubMed PubMed Central Google Scholar
Krupa, D. J., Matell, M. S., Brisben, A. J., Oliveira, L. M. & Nicolelis, M. A. Behavioral properties of the trigeminal somatosensory system in rats performing whisker-dependent tactile discriminations. J. Neurosci. 21, 5752–5763 (2001).
Article CAS PubMed PubMed Central Google Scholar
Brosch, M., Selezneva, E. & Scheich, H. Nonauditory events of a behavioral procedure activate auditory cortex of highly trained monkeys. J. Neurosci. 25, 6797–6806 (2005).
Article CAS PubMed PubMed Central Google Scholar
Foxe, J. J. et al. Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. J. Neurophysiol. 88, 540–543 (2002).
Article PubMed Google Scholar
Kayser, C., Petkov, C. I., Augath, M. & Logothetis, N. K. Integration of touch and sound in auditory cortex. Neuron 48, 373–384 (2005).
Article CAS PubMed Google Scholar
Carl, H., Tibor, H., Björn, S. & Yuri, Z. Pyramidal cell communication within local networks in layer 2/3 of rat neocortex. J. Physiol. 551, 139–153 (2004).
Google Scholar
Johnson, K. O. The roles and functions of cutaneous mechanoreceptors. Curr. Opin. Neurobiol. 11, 455–461 (2001).
Article CAS PubMed Google Scholar
Manfredi, L. R. et al. Natural scenes in tactile texture. J. Neurophysiol. 111, 1792–1802 (2014).
Article PubMed Google Scholar
Ritt, J. T., Andermann, M. L. & Moore, C. I. Embodied information processing: vibrissa mechanics and texture features shape micromotions in actively sensing rats. Neuron 57, 599–613 (2008).
Article CAS PubMed PubMed Central Google Scholar
Bosman, L. W. J. et al. Anatomical pathways involved in generating and sensing rhythmic whisker movements. Front. Integr. Neurosci. 5, 53 (2011).
Article PubMed PubMed Central Google Scholar
Feldmeyer, D. Excitatory neuronal connectivity in the barrel cortex. Front. Neuroanat. 6 (2012).
Diamond, M. E., Armstrong-James, M. & Ebner, F. F. Somatic sensory responses in the rostral sector of the posterior group (POm) and in the ventral posterior medial nucleus (VPM) of the rat thalamus. J. Comp. Neurol. 318, 462–476 (1992).
Article CAS PubMed Google Scholar
Murray, J. D. et al. A hierarchy of intrinsic timescales across primate cortex. Nat. Neurosci. 17, 1661–1663 (2014).
Article CAS PubMed PubMed Central Google Scholar
Iurilli, G. et al. Sound-driven synaptic inhibition in primary visual cortex. Neuron 73, 814–828 (2012).
Article CAS PubMed PubMed Central Google Scholar
Alais, D. & Burr, D. The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol.14, 257–262 (2004).
Article CAS PubMed Google Scholar
Fetsch, C. R., Turner, A. H., DeAngelis, G. C. & Angelaki, D. E. Dynamic reweighting of visual and vestibular cues during self-motion perception. J. Neurosci. 29, 15601–15612 (2009).
Article CAS PubMed PubMed Central Google Scholar
Battaglia, P. W., Jacobs, R. A. & Aslin, R. N. Bayesian integration of visual and auditory signals for spatial localization. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 20, 1391–1397 (2003).
Article PubMed Google Scholar
Perrault, T. J., Vaughan, J. W., Stein, B. E. & Wallace, M. T. Neuron-specific response characteristics predict the magnitude of multisensory integration. J. Neurophysiol. 90, 4022–4026 (2003).
Article PubMed Google Scholar
Stanford, T. R., Quessy, S. & Stein, B. E. Evaluating the operations underlying multisensory integration in the cat superior colliculus. J. Neurosci. 25, 6499–6508 (2005).
Article CAS PubMed PubMed Central Google Scholar
Shimaoka, D., Harris, K. D. & Carandini, M. Effects of arousal on mouse sensory cortex depend on modality. Cell Rep. 22, 3160–3167 (2018).
Article CAS PubMed PubMed Central Google Scholar
Vinck, M., Batista-Brito, R., Knoblich, U. & Cardin, J. A. Arousal and locomotion make distinct contributions to cortical activity patterns and visual encoding. Neuron 86, 740–754 (2015).
Article CAS PubMed PubMed Central Google Scholar
McGinley, M. J. et al. Waking state: rapid variations modulate neural and behavioral responses. Neuron 87, 1143–1161 (2015).
Article CAS PubMed PubMed Central Google Scholar
Morrill, R. J. & Hasenstaub, A. R. Visual information present in infragranular layers of mouse auditory cortex. J. Neurosci. 38, 2854–2862 (2018).
Article CAS PubMed PubMed Central Google Scholar
Brombas, A., Fletcher, L. N. & Williams, S. R. Activity-dependent modulation of layer 1 inhibitory neocortical circuits by acetylcholine. J. Neurosci. 34, 1932–1941 (2014).
Article CAS PubMed PubMed Central Google Scholar
Foxe, J. J. & Schroeder, C. E. The case for feedforward multisensory convergence during early cortical processing. Neuroreport 16, 419–423 (2005).
Article PubMed Google Scholar
Macaluso, E., Frith, C. D. & Driver, J. Modulation of human visual cortex by crossmodal spatial attention. Science 289, 1206–1208 (2000).
Article CAS PubMed Google Scholar
Giard, M. H. & Peronnet, F. Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J. Cogn. Neurosci. 11, 473–490 (1999).
Article CAS PubMed Google Scholar
Stein, B. E. & Stanford, T. R. Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 9, 255–266 (2008).
Article CAS PubMed Google Scholar
Wallace, M. T., Ramachandran, R. & Stein, B. E. A revised view of sensory cortical parcellation. Proc. Natl Acad. Sci. USA 101, 2167–2172 (2004).
Article CAS PubMed PubMed Central Google Scholar
Carandini, M. & Heeger, D. J. Normalization as a canonical neural computation. Nat. Rev. Neurosci. 13, 51–62 (2011).
Article PubMed PubMed Central CAS Google Scholar
Ohshiro, T., Angelaki, D. E. & DeAngelis, G. C. A normalization model of multisensory integration. Nat. Neurosci. 14, 775–782 (2011).
Article CAS PubMed PubMed Central Google Scholar
Ohshiro, T., Angelaki, D. E. & DeAngelis, G. C. A neural signature of divisive normalization at the level of multisensory integration in primate cortex. Neuron 95, 399–411.e8 (2017).
Article CAS PubMed PubMed Central Google Scholar
Chen, T.-W. et al. Ultrasensitive fluorescent proteins for imaging neuronal activity. Nature 499, 295–300 (2013).
Article CAS PubMed PubMed Central Google Scholar
Zhu, Y. et al. Ablation of NF1 function in neurons induces abnormal development of cerebral cortex and reactive gliosis in the brain. Genes Dev. 15, 859–876 (2001).
Article CAS PubMed PubMed Central Google Scholar
Jackson, N. & Muthuswamy, J. Artificial dural sealant that allows multiple penetrations of implantable brain probes. J. Neurosci. Methods 171, 147–152 (2008).
Article CAS PubMed PubMed Central Google Scholar
Knutsen, P. M., Pietr, M. & Ahissar, E. Haptic object localization in the vibrissal system: behavior and performance. J. Neurosci. 26, 8451–8464 (2006).
Article CAS PubMed PubMed Central Google Scholar
Haidarliu, S., Simony, E., Golomb, D. & Ahissar, E. Muscle architecture in the mystacial pad of the rat. Anat. Rec. Hoboken NJ 2007 293, 1192–1206 (2010).
Google Scholar
Dörfl, J. The innervation of the mystacial region of the white mouse: A topographical study. J. Anat. 142, 173–184 (1985).
PubMed PubMed Central Google Scholar
Haeffele, B., Young, E. & Vidal, R. Structured low-rank matrix factorization: Optimality, algorithm, and applications to image processing. in Proceedings of the 31st International Conference on Machine Learning (ICML-14) 2007–2015 (2014).
Issa, J. B., Haeffele, B. D., Young, E. D. & Yue, D. T. Multiscale mapping of frequency sweep rate in mouse auditory cortex. Hear. Res. 344, 207–222 (2017).
Article PubMed Google Scholar
Guizar-Sicairos, M., Thurman, S. T. & Fienup, J. R. Efficient subpixel image registration algorithms. Opt. Lett. 33, 156–158 (2008).
Article PubMed Google Scholar
Vogelstein, J. T. et al. Fast nonnegative deconvolution for spike train inference from population calcium imaging. J. Neurophysiol. 104, 3691–3704 (2010).
Article PubMed PubMed Central Google Scholar
Bonin, V., Histed, M. H., Yurgenson, S. & Reid, R. C. Local diversity and fine-scale organization of receptive fields in mouse visual cortex. J. Neurosci. 31, 18506–18521 (2011).
Article CAS PubMed PubMed Central Google Scholar
Froudarakis, E. et al. Population code in mouse V1 facilitates readout of natural scenes through increased sparseness. Nat. Neurosci. 17, 851–857 (2014).
Article CAS PubMed PubMed Central Google Scholar
Hofer, S. B. et al. Differential connectivity and response dynamics of excitatory and inhibitory neurons in visual cortex. Nat. Neurosci. 14, 1045–1052 (2011).
Article CAS PubMed PubMed Central Google Scholar
Ko, H. et al. Functional specificity of local synaptic connections in neocortical networks. Nature 473, 87–91 (2011).
Article CAS PubMed PubMed Central Google Scholar
Petreanu, L. et al. Activity in motor-sensory projections reveals distributed coding in somatosensation. Nature 489, 299–303 (2012).
Article CAS PubMed PubMed Central Google Scholar
Rajasethupathy, P. et al. Projections from neocortex mediate top-down control of memory retrieval. Nature 526, 653–659 (2015).
Article CAS PubMed PubMed Central Google Scholar

Download references

Acknowledgements

We thank David T. Yue for conceiving of this project and for providing the scientific foundation for this work. We also wish to thank Eric D. Young and Dwight E. Bergles for comments on the manuscript, Benjamin D. Haeffele for experimental set-up and guidance on image processing, Gordon F. Tomaselli for guidance and support, Jeffrey M. Yau and Sascha du Lac for valuable discussions, Ingie Hong for assistance with assembly of the head-fixed awake preparation, Travis Babola for assistance with computer hardware, Terry Shelley for fabrication of stereotaxic equipment, Loren Looger and the GENIE project for supplying transgenic GCaMP6s mice, and members of the O’Connor lab and the Calcium Signals Lab for helpful discussions. This work was supported by grants from the Kleberg Foundation, the Whitehall Foundation, the Klingenstein Fund, and the NIH (NIMH R01MH065531, NINDS R01NS073874, NINDS P30NS050274, NINDS R01NS089652).

Author information

Manning Zhang
Present address: Department of Biomedical Engineering, Washington University in St. Louis, St. Louis, MO, 63130, USA
Sung Eun Kwon
Present address: Department of Molecular, Cellular and Developmental Biology, University of Michigan, Ann Arbor, MI, 48109, USA
Manu Ben-Johny
Present address: Department of Physiology and Cellular Biophysics, Columbia University, New York, NY, 10032, USA
John B. Issa
Present address: Department of Neurobiology, Northwestern University, Evanston, IL, 60201, USA

Authors and Affiliations

Department of Biomedical Engineering, The Johns Hopkins University School of Medicine, Baltimore, MD, 21205, USA
Manning Zhang, Manu Ben-Johny & John B. Issa
Solomon H. Snyder Department of Neuroscience, The Johns Hopkins University School of Medicine, Kavli Neuroscience Discovery Institute, and Brain Science Institute, Baltimore, MD, 21205, USA
Sung Eun Kwon & Daniel H. O’Connor

Authors

Manning Zhang
View author publications
You can also search for this author in PubMed Google Scholar
Sung Eun Kwon
View author publications
You can also search for this author in PubMed Google Scholar
Manu Ben-Johny
View author publications
You can also search for this author in PubMed Google Scholar
Daniel H. O’Connor
View author publications
You can also search for this author in PubMed Google Scholar
John B. Issa
View author publications
You can also search for this author in PubMed Google Scholar

Contributions

M.Z. performed all experiments and collected all data with assistance from S.E.K. and J.B.I. All authors contributed to experimental design. D.H.O. and J.B.I. supervised the project. M.Z., M.B.J. and J.B.I. processed and analyzed the data with assistance from D.H.O. All authors contributed to the writing and editing of the manuscript with M.Z. and M.B.J. assembling the initial draft and M.Z., D.H.O. and J.B.I. revising the final version.

Corresponding author

Correspondence to John B. Issa.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Reporting Summary

Peer Review File

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Cite this article

Zhang, M., Kwon, S.E., Ben-Johny, M. et al. Spectral hallmark of auditory-tactile interactions in the mouse somatosensory cortex. Commun Biol 3, 64 (2020). https://doi.org/10.1038/s42003-020-0788-5

Download citation

Received: 07 November 2019
Accepted: 22 January 2020
Published: 11 February 2020
DOI: https://doi.org/10.1038/s42003-020-0788-5
Springer Nature Limited

Spectral hallmark of auditory-tactile interactions in the mouse somatosensory cortex

Abstract

Similar content being viewed by others

Basic Science: Pain Mechanisms and Pathways

The Superior Colliculus: Cell Types, Connectivity, and Behavior

What if worms were sentient? Insights into subjective experience from the Caenorhabditis elegans connectome

Introduction

Results

Widefield Ca2+ imaging localizes tactile and auditory areas

Tuning to the frequency of tactile stimuli in S1 and S2

Concurrent auditory input modulates tactile responses in S2

Dissociation of motor behavior from sound-driven modulation

Dependence of integration on modality and sound properties

Concurrent sound modulates tactile responses in S1 and ISF

Sound preferentially inhibits low frequency tactile responses

Touch inhibits responses of sound-selective neurons in S2

Spatial layout of functional cell types in S2

Discussion

Methods

Animal surgery and general procedures

Experimental setup

Imaging

Tactile stimuli

Auditory stimuli

Combined stimuli

Stimulus order

Widefield imaging analysis

Two-photon imaging analysis

Registration

Response analysis

Statistics and reproducibility

Reporting summary

Data availability

Code availability

References

Acknowledgements

Author information

Authors and Affiliations

Contributions

Corresponding author

Ethics declarations

Competing interests

Additional information

Supplementary information

Supplementary Information

Reporting Summary

Peer Review File

Rights and permissions

About this article

Cite this article

Share this article

Search

Navigation

Widefield Ca²⁺ imaging localizes tactile and auditory areas