Electrophysiological correlates of listening effort: neurodynamical modeling and measurement

Abstract

An increased listing effort represents a major problem in humans with hearing impairment. Neurodiagnostic methods for an objective listening effort estimation might support hearing instrument fitting procedures. However the cognitive neurodynamics of listening effort is far from being understood and its neural correlates have not been identified yet. In this paper we analyze the cognitive neurodynamics of listening effort by using methods of forward neurophysical modeling and time-scale electroencephalographic neurodiagnostics. In particular, we present a forward neurophysical model for auditory late responses (ALRs) as large-scale listening effort correlates. Here endogenously driven top–down projections related to listening effort are mapped to corticothalamic feedback pathways which were analyzed for the selective attention neurodynamics before. We show that this model represents well the time-scale phase stability analysis of experimental electroencephalographic data from auditory discrimination paradigms. It is concluded that the proposed neurophysical and neuropsychological framework is appropriate for the analysis of listening effort and might help to develop objective electroencephalographic methods for its estimation in future.

Appendix

A1: Map of the gain parameters to the hearing path

The corticothalamic feedback dynamics in our model is represented by three different gains which we map to the hearing path in the following.

Gain G1: The auditory cortex projects indirectly to thalamic reticular nucleus (TRN) by means of axon-collaterals of corticothalamic projections. Additionally the TRN receives inhibitory input from dorsal thalamic nuclei. Thus, the TRN provides an inhibitory influence on the specific thalamus cores, namely the medial geniculate body (MGB) in the case of auditory evoked potentials. The target of TRN projections are the ventral and the medial subnuclei of MGB. The ventral subnucleus (VMGB) is specific for auditory processing, while the medial subnucleus (MMGB) receives also information from non-auditory pathways. The VMGB projects to anterior auditory field (AAF), the posterior auditory field (PAF) and the primary (A1) auditory cortex, the MMGB projects to the ipsilateral parts of the primary (A1), and the secondary (A2) auditory cortex, and to the ipsilateral posterior PAF and the anterior AAF auditory fields.

Gain G2: The auditory cortex projects directly to all the subnuclei of MGB, namely VMGB, MMGB, and the dorsal geniculate body (DMGB), which also gets informational input from earlier stages of the auditory pathway. Back projection to the cortex occurs as described above plus efferent projections from the dorsal subnucleus (DMGB) to the auditory cortex.

Gain G3: As described above the auditory cortex projects indirectly to the TRN by means of axon-collaterals of corticothalamic projections. The TRN has no efferent fibres projecting towards the auditory cortex, but is part of a thalamocortical feedback loop. The TRN receives additional input by axon-collaterals of thalamocortical projections. Due to its inhibitory influence on specific thalamus cores (i.e. MGB) the TRN can directly regulate information flow from thalamus to cortical areas.

A2: Wavelet and gabor frame phase synchronization

Wavelet Phase Synchronization Stability: We follow our definition of the WPSS as in Strauss et al. (2008a). Let $L^2({\mathbb{R}})$ denote the Hilbert space of all square integrable functions, i.e., $L^2({\mathbb{R}})=\{f : {\mathbb{R}} \mapsto {\mathbb{C}} : \int_{{\mathbb{R}}} |f(\mu)|^2 \hbox{d}mu < \infty\}.$ Let further ψ_a,b(·) = |a|^−1/2ψ((· − b)/a) where $\psi \in L^2({\mathbb{R}})$ is the wavelet with $0< \int_{{\mathbb{R}}} |\Uppsi (\omega)|^2|\Uppsi(\omega)|^{-1} \hbox{d}\omega < \infty\; (\Uppsi (\omega)$ is the Fourier transform of the wavelet), and $a,b\in {\mathbb{R}}, a\neq 0$. The wavelet transform

$$ {\mathcal{W}}_{\psi}:L^2({\mathbb{R}})\longrightarrow L^2({\mathbb{R}}^2,{\frac{\hbox{d} a\hbox{d} b}{a^2}}) $$

(2)

of a signal $x \in L^2({\mathbb{R}})$ with respect to the wavelet ψ is given by the inner L²-product $({\mathcal{W}}_{\psi}x)(a,b)=\langle x,\psi_{a,b} \rangle_{L^2}.$ In the following, we restrict our interest to discrete time systems and signals such that all signals are represented by sequences. For the sake of a handy notation, we denote the index of the individual sequence elements as argument in square brackets. Let $\ell^2$ denote the Hilbert space of all square summable sequences, i.e., $\ell^2=\ell^2({\mathbb{Z}})=\{{\bf x} : {\mathbb{Z}} \mapsto {\mathbb{C}} : \sum_{m\in {\mathbb{Z}}} |x[m]|^2 < \infty\}.$ To compute the WPSS for such sequences in $\ell^2$ rather than continuous time signals in $L^2({\mathbb{R}})$, we used the discretization scheme of Misiti et al. (2000).

Tight Gabor Frames: We restrict our interest in the following further to time-invariant systems of the form $\varphi_{m,n}[{\cdot}]=\varphi_{m}[{\cdot-}\alpha n], \quad n\in{\mathbb{Z}}, m=0,1,\ldots,M-1, \alpha\in {\mathbb{N}}_{> 0}$ where $\varvec{\varphi}_{m}\in \ell^2.$ A set $\{\varvec{\varphi}_{m,n} : m,n\in {\mathbb{Z}}, \varvec{\varphi}_{m,n} \in \ell^2\}$ is called a frame for $\ell^2$ if

$$ A||{\bf x}||_{\ell^2}^2 \leq \sum_{m,n\in {\mathbb{Z}}} |\langle {\bf x}, \varvec{\varphi}_{m,n} \rangle_{\ell^2}|^2 \leq B ||{\bf x}||_{\ell^2}^2, \quad \forall {\bf x} \in \ell^2. $$

(3)

For A = B the frame is called a tight frame for $\ell^2$ and we have the expansion ${\bf x} = A^{-1} \sum_{m,n\in {\mathbb{Z}}} \langle {\bf x}, \varvec{\varphi}_{m,n} \rangle_{\ell^2} \varvec{\varphi}_{m,n}.$ If $||\varvec{\varphi}_m||_{\ell^2}^2=1 \forall m\in {\mathbb{Z}}$ and A = 1 we obtain orthonormal expansions and for A > 1 the expansion becomes overcomplete and A reflects its redundancy. Two frames $\{\varvec{\varphi}_{m,n} : m,n\in {\mathbb{Z}}\}$ and $\{\tilde{\varvec{\varphi}}_{m,n} : m,n\in {\mathbb{Z}}\}$ for the Hilbert space $\ell^2$ are called dual frames if ${\bf x}=\sum_{m,n\in {\mathbb{Z}}} \langle {\bf x}, \varvec{\varphi}_{m,n} \rangle \tilde{\varvec{\varphi}}_{m,n}, \forall {\bf x} \in \ell^2.$ A Gabor system $(\varvec{\varphi},\alpha,M^{-1})$ for $\ell^2$ is defined as

$$ \varphi_{m,n}[{\cdot}]=e^{2 \pi \imath m {\cdot M}^{-1}} \varphi[{\cdot-}\alpha n], $$

(4)

i.e, the system represents a family of sequences which are generated by one particular sequence due to modulation and translation. A Gabor system that is also a frame for $\ell^2$ is called a Gabor Frame for $\ell^2.$ For αM ⁻¹ > 1 the system is undersampled and cannot be a basis or a frame for $\ell^2.$ For αM ⁻¹ = 1 we have the critically sampled case and, if the Gabor system represents a frame, it is also a basis. For α M ⁻¹ < 1 we have the oversampled case and the Gabor system cannot be a basis but a frame.

Tight Gabor frames have been constructed in Søndergaard (2007), Strohmer (1999). For a tight Gabor frame $\{\varvec{\varphi}_{m,n} : m,n\in {\mathbb{Z}},\ \varvec{\varphi}_{m,n} \in \ell^2\}$ and an index set ${\mathcal{I}}=\{0,1,\ldots,M-1\}$ , we define the analysis operator by

$$ ({\mathcal{G}}_{\varphi} {\bf x}) [m,n]=C[m,n]= \langle {\bf x}, \varvec{\varphi}_{m,n} \rangle_{\ell^2}. $$

(5)

We define synthesis operator with respect to the tight Gabor frame by

$$ x[\cdot] =\left({\mathcal{G}}^*_{\varphi}C[m,n]\right) [\cdot] = \sum_{m,n\in{\mathbb{Z}}} C[m,n] \varphi_{m,n}[\cdot]. $$

(6)

The described Gabor decompositions can also efficiently be implemented by oversampled uniform band discrete Fourier transform filter banks as shown in Bölcskei et al. (1998).

A3: Mathematical model of corticothalamic feedback

What follows is based on the large-scale evoked potential theory proposed in Robinson et al. (1997, 2005) and Rennie et al. (2002) adapted to our discussion in sections “A probabilistic auditory scene analysis framework” and “Corticofugal modulation and auditory late responses”.

According to Freeman (1991), within a neuron the relationship between the rate of incoming pulses from excitatory or inhibitory neurons Q _ae or Q _ai and their corresponding soma potentials V _e or V _i , can be obtained by an impulse response equation of the form,

$$ V_{e,i}(r,t)=g\int\limits_{-\infty}^{t}w(t-t')Q_{ae,ai}(r,t')dt', $$

(7)

where w(u) is a causal weight function with a characteristic width that satisfies

$$ \int\limits_{0}^{\infty}w(u)du=1. $$

As stated in Robinson et al. (1997), a suitable choice for w(u) is given by

$$ w(u)=\left\{ \begin{array}{ll} {\frac{\alpha \beta}{\beta -\alpha}} (e^{-\alpha u}-e^{-\beta u}), & \quad \beta\neq \alpha\\ \alpha^{2}ue^{-\alpha u}, &\quad \alpha=\beta \end{array} \right. $$

(8)

for u > 0, where α and β represent the rise and decay times of the cell-body potential produced by an impulse at a dendritic synapse. By combining Eqs. 7 and 8 for the case β ≠ α, the mean field soma potential V _a (a = e, i excitatory and inhibitory), representing the synaptic inputs from various types of afferent neurons that are summed after being filtered and smeared out in time as a result of receptor dynamics and passage trough the dendritic tree, is governed by the following equation

$$ D_{\alpha}V_{a}(r,t)=\sum_{b}N_{ab}S_{ab}\phi_{b}(r,t-\tau_{ab}), $$

(9)

where

$$ D_{\alpha}={\frac{1}{\alpha\beta}}{\frac{d^{2}}{dt^{2}}}+({\frac{1}{\alpha}} +{\frac{1}{\beta}}){\frac{d}{dt}}+1, $$

(10)

N _ab is the average number of synapses from neurons of type b = e, i, s on neurons of type a = e, i, where s stands for subcortical, S _ab represents the magnitude of postsynaptic potentials, ϕ _b represents fields of incoming pulses and τ _ab are synaptic time delays.

In accordance to Rennie et al. (2002), the mean firing rate Q _a related to cell-body potential V _a is given by a sigmoidal-type function

$$ Q_{a}={\frac{Q_{\hbox {max}}}{1+exp\{-[V_{a}(r,t)-\theta_{a}]/\sigma_{a}\}}}, $$

(11)

where θ _a is the mean firing threshold of neurons of type a, σ _a is the standard deviation of this threshold in the neural population, and Q _max is the maximum attainable firing rate.

An assumption that reflects the large-scale effect of neural populations is that each part of the corticothalamic system produces a field ϕ _a of pulses, which travels at a velocity v _a through axons with a characteristic range r _a . Approximately such pulses propagate according to the damped-wave equation

$$ \left({\frac{1}{\gamma_{a}^{2}}}{\frac{\partial^{2}}{\partial t^{2}}}+ {\frac{2}{\gamma_{a}}}{\frac{\partial}{\partial t}} + 1-r_{a}^{2}\nabla^{2}\right)C_{a}(r,t)=Q_{a}, $$

(12)

where γ _a  = v _a /r _a . The simulation of auditory evoked cortical streams S is achieved by using a thalamic stimulus A _n (k, w) of angular frequency w and wave vector k, which for convenience is chosen to be a unit Gaussian in time and space (centered at t _0s and r _0s with standard deviations t _s and r _s ), together with a function C _l (k, w) which refers to an excitatory cortical activity of a short-range neural population with local connectivity, to define a corticothalamic transfer function

$$ \begin{aligned} {\frac{C_{l}(k,w)}{A_{n}(k,w)}}\,=\,&A{\frac{L_{l}L_{s}} {(1-L_{l}I_{ll}-L_{i}I_{ii})}}{\frac{e^{iwt_{d}}/2} {(1-L_{s}L_{r}(G3))}}\\ &\times[1 +{\frac{1}{(k^{2}r_{e}^{2}+q^{2}r_{e}^{2})}} ({\frac{L_{e}I_{ee}}{(1-L_{l}I_{ll}-L_{i}I_{ii})}}\\ &+{\frac{L_{e}L_{s}((G2)+(G1)L_{r})e^{iwt_{d}}} {(1-L_{l}I_{ll}-L_{i}I_{ii})(1-L_{s}L_{r}(G3))}}], \end{aligned} $$

in which t _d represents the time delay between thalamus and cortex, A represents an amplitude scaling factor, G1, G2, and G3 are the relevant corticofugal and intrathalamic gains, L _z represents a dendritic transfer function that exerts a low pass filtering effect as given by

$$ L(w)=(1-{\frac{i\omega}{{\mathcal{N}}_{1}}})^{-1}(1-{\frac{i\omega} {{\mathcal{N}}_{2}}})^{-1}, $$

(13)

where ${\mathcal{N}}_{1}=\alpha,\eta_{1}$ and ${\mathcal{N}}_{2}=\beta,\eta_{2}$, and I _xy stands for secundary gain factors. The indices for the parameters x, y, z = e, i, l, s, n, r denote e excitatory, i inhibitory, l excitatory cortical neurons with local axons, s excitatory thalamic neurons in specific and secondary relay nuclei, n excitatory afferents to thalamus, and r inhibitory thalamic reticular neurons. The response to the thalamic stimulus is given by

$$ \Updelta(r,w)=\int\limits_{0}^{\infty}{\frac{k dk}{2\pi}}{\frac{C_{l}} {A_{n}}}e^{-1/4k^{2}r_{s}^{2}}J_{0}(k|r-r_{0s}|), $$

(14)

and the desired evoked potential streams can be obtained by using

$$ R(r,t)={\mathcal{F}}^{-1}\{e^{-1/2\omega^{2}t_{s}^{2}}e^{i\omega t_{0s}}\Updelta(r,w)\}(r,t), $$

(15)

here J ₀ denotes the bessel function of first kind, and ${\mathcal{F}}^{-1}$ denotes the inverse Fourier transform, see Rennie et al. (2002) for more details.