Computing an optimal time window of audiovisual integration in focused attention tasks: illustrated by studies on effect of age and prior knowledge

Abstract

The concept of a "time window of integration" holds that information from different sensory modalities must not be perceived too far apart in time in order to be integrated into a multisensory perceptual event. Empirical estimates of window width differ widely, however, ranging from 40 to 600 ms depending on context and experimental paradigm. Searching for theoretical derivation of window width, Colonius and Diederich (Front Integr Neurosci 2010) developed a decision-theoretic framework using a decision rule that is based on the prior probability of a common source, the likelihood of temporal disparities between the unimodal signals, and the payoff for making right or wrong decisions. Here, this framework is extended to the focused attention task where subjects are asked to respond to signals from a target modality only. Evoking the framework of the time-window-of-integration (TWIN) model, an explicit expression for optimal window width is obtained. The approach is probed on two published focused attention studies. The first is a saccadic reaction time study assessing the efficiency with which multisensory integration varies as a function of aging. Although the window widths for young and older adults differ by nearly 200 ms, presumably due to their different peripheral processing speeds, neither of them deviates significantly from the optimal values. In the second study, head saccadic reactions times to a perfectly aligned audiovisual stimulus pair had been shown to depend on the prior probability of spatial alignment. Intriguingly, they reflected the magnitude of the time-window widths predicted by our decision-theoretic framework, i.e., a larger time window is associated with a higher prior probability.

Appendices

Appendix 1: probability of integration P(I)

The peripheral processing times V for the visual and A for the visual stimulus have an exponential distribution with parameters λ _V and λ _A , respectively. That is,

$$ \begin{aligned} f_V(t)&=\lambda_V e^{-\lambda_V t} \\ f_A(t)&=\lambda_A e^{-\lambda_A t} \end{aligned} $$

for t ≥ 0, and f _V (t) = f _A (t)≡ 0 for t < 0. The corresponding distribution functions are referred to as F _V (t) and F _A (t).

The visual stimulus is the target and the auditory stimulus is the nontarget. By definition,

$$ \begin{aligned} P(I)& = Pr(A + \tau < V < A + \tau + \omega) \\ & = \int\limits_0^\infty f_A(x)\{F_V(x+\tau + \omega)-F_V(x+\tau)\} dx, \\ \end{aligned} $$

where τ denotes the SOA value and ω is the width of the integration window. Computing the integral expression requires that we distinguish between three cases for the sign of τ + ω:

• (i) τ < τ + ω < 0

  $$ \begin{aligned} P(I)& = \int\limits_{-\tau - \omega}^{-\tau} \lambda_A e^{-\lambda_A x} \{1-e^{-\lambda_V(x+\tau+\omega)}\} {\hbox {d}}x \\ &\quad+ \int\limits_{-\tau}^\infty \lambda_A e^{-\lambda_A x}\{e^{-\lambda_V(x+\tau)}-e^{-\lambda_V(x+\tau+\omega)}\} {\hbox {d}}x \\ &=\frac{\lambda_V}{\lambda_V+\lambda_A}\, e^{\lambda_A \tau}(-1+e^{\lambda_A \omega}); \end{aligned} $$

• (ii) τ < 0 < τ + ω

  $$ \begin{aligned} P(I)&=\int\limits_0^{-\tau} \lambda_A e^{-\lambda_A x} \left\{1-e^{-\lambda_V(x+\tau+\omega)}\right\} {\hbox {d}}x \\ &\quad + \int\limits_{-\tau}^\infty \lambda_A e^{-\lambda_A x}\left\{e^{-\lambda_V(x+\tau)}-e^{-\lambda_V(x+\tau+\omega)}\right\} {\hbox {d}}x \\ &=\frac{1}{\lambda_V+\lambda_A}\, \left\{\lambda_A\left(1-e^{-\lambda_V(\omega+\tau)}\right) +\lambda_V(1-e^{\lambda_A \tau})\right\}; \end{aligned} $$

• (iii) 0 < τ < τ + ω

  $$ \begin{aligned} P(I)&=\int\limits_0^\infty \lambda_A e^{-\lambda_A x}\{e^{-\lambda_V(x+\tau)}-e^{-\lambda_V(x+\tau+\omega)}\} {\hbox {d}}x \\ &=\frac{\lambda_A}{\lambda_V+\lambda_A} \{e^{-\lambda_V \tau} -e^{-\lambda_V(\omega + \tau)} \}. \end{aligned} $$

Appendix 2: asymmetric-Laplace density

The asymmetric-Laplace density AL(θ, κ, σ) is defined as (cf. Kotz et al. 2001, p. 137)

$$ f(t| \theta, \sigma,\kappa)= \frac{\sqrt{2}}{\sigma} \frac{\kappa}{1+\kappa^2}\left\{\begin{array}{ll}\exp\left( -\frac{\sqrt{2}\kappa}{\sigma}\,|x-\theta| \right)&\hbox{if}\,t \geq \theta,\\ \exp\left(-\frac{\sqrt{2}}{\sigma \kappa}\,|x-\theta| \right)&\hbox{if}\,t<\theta, \end{array}\right. $$

where θ is location parameter, κ = 1 implies symmetry, and σ is a scale parameter. In deriving the distributional form used in the text, the following parameter mappings have been made:

$$ \sigma^2=\frac{2}{\lambda_V \lambda_A}; \quad \kappa^2=\lambda_V/\lambda_A;\quad \theta=0. $$