Maximizing noise energy for noise-masking studies
Noise-masking experiments are widely used to investigate visual functions. To be useful, noise generally needs to be strong enough to noticeably impair performance, but under some conditions, noise does not impair performance even when its contrast approaches the maximal displayable limit of 100 %. To extend the usefulness of noise-masking paradigms over a wider range of conditions, the present study developed a noise with great masking strength. There are two typical ways of increasing masking strength without exceeding the limited contrast range: use binary noise instead of Gaussian noise or filter out frequencies that are not relevant to the task (i.e., which can be removed without affecting performance). The present study combined these two approaches to further increase masking strength. We show that binarizing the noise after the filtering process substantially increases the energy at frequencies within the pass-band of the filter given equated total contrast ranges. A validation experiment showed that similar performances were obtained using binarized-filtered noise and filtered noise (given equated noise energy at the frequencies within the pass-band) suggesting that the binarization operation, which substantially reduced the contrast range, had no significant impact on performance. We conclude that binarized-filtered noise (and more generally, truncated-filtered noise) can substantially increase the energy of the noise at frequencies within the pass-band. Thus, given a limited contrast range, binarized-filtered noise can display higher energy levels than Gaussian noise and thereby widen the range of conditions over which noise-masking paradigms can be useful.
KeywordsMasking Noise External noise paradigm Binary noise Filtered noise
Noise-masking experiments are widely used to investigate visual functions (Allard, Faubert, & Pelli, 2015; Lu & Dosher, 2008; Pelli & Farell, 1999; Pelli, 1981). Masking occurs when the noise noticeably impairs the observer’s performance, but under some conditions, performance remains unaffected even when the noise approaches 100 % contrast. For instance, consider using a noise-masking paradigm to investigate the internal factors limiting contrast sensitivity (e.g., Pelli & Farell, 1999). The noise energy required to noticeably impair the detection threshold is minimal at middle spatial frequencies and gradually increases at low and high spatial frequencies. At low spatial frequencies, the impact of noise is attenuated by sparser and larger receptive fields integrating (~averaging) the noise over large areas and thereby weakening its masking strength (Raghavan, 1995). At high spatial frequencies, the impact of the noise is attenuated by the modulation transfer function of the eye reducing the effective contrast of the stimulus (Campbell & Gubisch, 1966) and therefore requiring high noise energy to noticeably impair performance. As a result, the frequency range over which noise can effectively impair performance is limited by the maximal external noise energy that can be displayed, that is, without exceeding 100 % contrast. Furthermore, the noise energy required to impair detection also increases when reducing luminance intensities, especially at high spatial frequencies (Raghavan, 1995). Displaying more noise energy would widen the frequency and luminance range over which noise-masking paradigms could be implemented. To extend the usefulness of noise-masking paradigms, the present study developed a new noise maximizing the displayable energy.
Many noise-masking paradigms assume, usually implicitly, that the same underlying processing strategy operates in absence and presence of noise. But recent studies by Allard and colleagues (Allard & Cavanagh, 2011; Allard & Faubert, 2013, 2014a, 2014b; Allard, Renaud, Molinatti, & Faubert, 2013) suggested that this noise-invariant processing assumption can be violated when using some types of noise. For instance, contrast detection is known to be immune to crowding, but adding noise that is spatiotemporally localized to the target (i.e., at the potential target locations and turn on and off with target) made a detection task vulnerable to crowding, whereas noise that is spatiotemporally extended (i.e., full-screen, continuously displayed dynamic noise) did not (Allard & Cavanagh, 2011). These results suggest that the processing strategy in localized noise involved processes vulnerable to crowding, whereas the processing strategy in absence of noise and in extended noise did not. The aim of the present study was to increase the noise-masking strength without triggering a change in processing strategy.
To avoid triggering a change in processing strategy, Allard and colleagues (Allard & Cavanagh, 2011; Allard & Faubert, 2013, 2014a, 2014b; Allard et al., 2013) recommended to use full-screen, continuously displayed, dynamic noise. Unfortunately, using dynamic noise resampled at a high temporal rate instead of static noise tends to reduce the masking strength of noise due to temporal integration (i.e., averaging) occurring early in the visual system. The use of dynamic noise can therefore reduce the range of conditions over which noise-masking paradigms can be usefully implemented (i.e., noticeably impair performance). The constraint of using dynamic noise further emphasizes the need to maximize the displayable noise energy.
A typical way to modify Gaussian noise to increase its effective masking power is to concentrate its energy to frequencies relevant to the processing of the stimulus (Pelli, 1981; Solomon & Pelli, 1994; Stromeyer & Julesz, 1972). Another method simply consists in sampling the noise from a binary distribution instead of Gaussian distribution (e.g., Allard et al., 2013). The present study combined these two approaches to further increase masking power.
The distribution that maximizes the noise energy given a limited contrast range is a binary distribution in which one of two values is randomly selected independently for each sample (left image in the second row of Fig. 1). Given no correlation across samples, binary noise is white (flat energy spectrum) and its energy can also be defined by Eq. (1). Thus, binary noise has the same expected energy level at all frequencies as Gaussian noise given the same variance (left graph in third row of Fig. 1). The variance of binary noise is equal to the variance of Gaussian noise (σ 2) when the two values from which the samples are drawn are ±σ (left graph in bottom row of Fig. 1).
For many experiments, the ideal noise would have a flat energy spectrum over all frequencies. However, to have the same energy level across an infinite range of frequencies, such an ideal noise would require infinitely small samples (e.g., w, h and d infinitely small) and its energy would therefore also be infinitely small for any finite sample variance (Pelli, 1981). In practice, a noise can have a flat energy spectrum over a finite range of frequencies and the maximal displayable noise energy can be increased by concentrating the noise energy over a narrower range of frequencies.
The simplest way of increasing the energy level is to decrease the spatial and/or temporal resolution of the display (e.g., center image in the top row of Fig. 1 in which each noise check size is set to 4×4 pixels rather than 1×1 pixels). For the same variance, reducing the resolution of the display increases the noise energy (Eq. 1) and reduces the upper frequency limit of the noise (black curve in top center graph in Fig. 1) so it can be used when the noise at these frequencies is not relevant to the task. Nevertheless, a drawback of low-resolution noise is that it introduces apparent edges between noise checks forming a grid as it can be seen in the center image in the top row of Fig. 1. Because these apparent edges may have undesirable effects (e.g., Harmon & Julesz, 1973), it is safer to avoid them as their presence could potentially interfere with the processing of the target.
An alternative method that does not introduce artificial edges consists in filtering the noise to remove frequencies that are not relevant to the task (right image in the top row and black curve in the right graph in third row of Fig. 1). Indeed, some frequencies can be removed to reduce the contrast range without affecting performance. For instance, removing the frequencies outside ±1 octave from the spatial frequency of a sine-wave target does not affect detection threshold (Pelli, 1981; Stromeyer & Julesz, 1972). Filtering out information irrelevant to the task (i.e., does not affect performance) is an efficient way of reducing the noise contrast (e.g., Gaussian distribution narrower for filtered noise compared to Gaussian noise, black curves in right and left graphs in bottom row of Fig. 1, respectively). As a result, at equal contrast ranges, filtered noise would have higher energy at the frequencies within the pass-band.
The rationale of the present study was to combine the two approaches described above (i.e., binary noise and non-white noise) to further increase the energy of the noise at the frequencies relevant to the task. For low-resolution noise, the method simply consists in sampling noise elements from a binary distribution instead of a Gaussian distribution (center images in top and second rows of Fig. 1). Low-resolution binary noise is not novel as it has been used before (e.g., Allard et al., 2013). However, the drawback of using low-resolution noise remains: it artificially introduces apparent edges between noise checks. Alternatively, the present study combined binary noise with filtered noise (right image in second row of Fig. 1). Binarizing the noise after the filtering operation substantially increases the energy level for equal contrast range. In other words, the binarized-filtered noise requires much less contrast than (unbinarized-)filtered noise to reach the same energy level at the frequencies within the pass-band (right graphs in third and bottom rows of Fig. 1).
Binarizing filtered noise changes the profile of the spectral density function as it introduces energy at frequencies that were filtered out (e.g., right graph in third row of Fig. 1). However, given that the energy at those frequencies is irrelevant (completely removing them should not affect performance), this small gain in energy should also be negligible for masking experiments. More importantly, the binarizing operation does not affect the expected constant spectral density across the frequencies within the pass-band (right graph in third row of Fig. 1).
Experiment: Binarized-filtered noise vs Gaussian noise
Binarized-filtered noise requires less contrast than Gaussian noise to display the same expected energy at the frequencies relevant to the task so both noises are expected to have the same masking strength. The main aim of this experiment was to empirically verify this prediction. If the performance in binarized-filtered noise differs from the one in Gaussian noise (given equated energy levels at frequencies within the pass-band), then the binarized-filtered noise cannot be considered equivalent to the Gaussian noise. On the other hand, if binarized-filtered noise has the same expected noise energy at the frequencies relevant to the task as Gaussian noise and has the same masking strength, then binarized-filtered noise can be considered as equivalent to Gaussian noise.
Four naïve observers and one of the authors participated in this study. They had normal or corrected-to-normal vision.
Stimuli were presented on a 22.5-inch LCD monitor designed for psychophysics (VIEWPixx) with a refresh rate of 120 Hz. At the viewing distance of 1 m, the spatial resolution of the display was 64 pixels/degree of visual angle. The monitor was the only source of light in the room. The output intensity of each color gun was linearized psychophysically using a homemade program.
Stimuli and procedure
The detection task was implemented using a two-interval forced-choice procedure with an interstimulus interval of 500 msec. The noise was continuously displayed, refreshed at every frame and covered the entire screen. The signal was a 4 cycles-per-degree vertical grating presented in only one of the two 500-msec intervals. The spatial window of the signal had a diameter of 1 degrees plus a half-cosine soft edge of 0.25 degrees. A 500-msec sound was audible during each of the two intervals and the task consisted in determining if the target was present at the during the first or second sound by pressing one of two keys.
Contrast thresholds were measured using a 3down1up staircase procedure (Levitt, 1971) with step size of 0.1 log and was interrupted after 12 inversions. Threshold estimation of a staircase was set as the geometric mean of the last 8 inversions. The 10 conditions were blocked and performed in a pseudo-random order. Each staircase was performed 5 times so that each threshold was set as the geometric mean of the 5 staircases.
Results and discussion
The second important outcome was that the filtering operation had no impact when the filter bandwidth was at least 2 octaves (Fig. 5), that is, 1 octave above and below the signal spatial frequency, which is consistent with previous findings (Pelli, 1981; Stromeyer & Julesz, 1972). Note that statistically, the two-way ANOVA showed a simple main effect of filter bandwidth (F(4,16) = 46.4, p < .001), which can be explained by the lower contrast thresholds when many frequencies were filtering out (e.g., <2 octaves bandwidth filters). Nevertheless, when measuring contrast detection threshold in noise, the same performance was observed whether the noise was white or had energy only 1 octave above and below the spatial frequency of the target. This suggests that, for a contrast detection task, removing the noise outside this frequency range reduces the contrast range of the noise without affecting performance.
Taken together, these two outcomes suggest that binarized-filtered noise at ±1 octave around the signal frequency was equivalent to Gaussian noise. Given that the filtering and binarizing operations substantially reduced the noise contrast range without affecting performance (given equated noise energy at frequencies within the pass-band), increasing the contrast of binarized-filtered would display higher noise energy at the relevant frequencies.
The advantage of binarized-filtered noise is that it uses a narrower contrast range than Gaussian noise for a given masking strength, which enables to display higher noise energy (by increasing noise contrast). A potential issue with this noise is that the use of only two luminance intensities introduces sharp edges (e.g., Fig. 1, bottom right). Because the position of these edges randomly varies over time, they are less salient than the edges for the low-resolution noises presented above (e.g., see Movie 1 in supplementary material). Although these edges could potentially have an undesirable effect, the experiment above rather suggests that these edges had a negligible impact for this contrast detection task. Nevertheless, apparent edges could potentially cause binarized-filtered noise not to be equivalent to Gaussian noise. The present section shows that the visibility of these apparent edges can be substantially attenuated with the small cost of slightly decreasing noise energy (given equated noise contrast). Thus, even though the potential drawback of the binarization operation (i.e., introducing sharp edges) was empirically found to be negligible, the present section nevertheless shows that it can be substantially attenuated.
Above, to compare the energy of binary noise and Gaussian noise (Fig. 2), the Gaussian noise was truncated at various truncation thresholds. Figure 2 shows that, for equated contrast ranges, the energy of truncated Gaussian noise approaches the one of binary noise as the truncation threshold approaches 0. This is because at an infinitely small truncation threshold, truncated Gaussian noise is equivalent to binary noise. Thus, by varying the truncation value and equating the noise energy at frequencies within the pass-band, truncated Gaussian noise gradually varies from binary noise (truncation threshold = 0) to Gaussian noise (truncation threshold = ∞).
By combining two methods of increasing noise energy given a limited contrast range (namely, filtered noise and binary noise), the present study developed a noise, namely truncated-filtered noise, equivalent to Gaussian noise with respect to a given task, but requiring less contrast to be displayed (see Appendix for detailed algorithm and Matlab code). Truncated-filtered noise was found to have the same masking strength as Gaussian noise when having the same expected energy at frequencies relevant to the task, which required less contrast. Thus, this new noise enables to display higher noise energy and thereby widen the range of conditions under which noise-masking paradigms can be effectively used. For instance, the amount of noise required to affect performance is known to gradually increase as luminance intensity is reduced (Pelli, 1981). To illustrate the usefulness of this method, we evaluated, for the same stimulus as in the experiment above, the lowest luminance intensity at which the maximal displayable noise could noticeably affect performance (i.e., increase detection threshold by a factor of at least 2). The lowest luminance intensities at which Gaussian, binary, filtered (±1 octave) and truncated-filtered (±1 octave, truncation = 1 sd) noises could noticeably affect performance were found to be about 350, 39, 16 and 4 td, respectively. This simple example shows that truncated-filtered noise can be a useful tool to widen the range of conditions under which noise can be effectively be used. Note that maximizing noise energy could also be useful for other types of paradigms requiring strong masking, such as continuous flash suppression (Tsuchiya & Koch, 2005).
Truncated-filtered noise varies, depending on the truncation threshold, along a continuum between binarized-filtered noise (truncation threshold = 0) and filtered noise (truncation threshold = ∞, i.e., no truncation). Empirically, results showed that when equating energy at the frequencies relevant to the task, the same performance was observed for the two extreme truncation thresholds (i.e., binarized-filtered noise and filtered noise). This suggests that truncating the noise at any truncation threshold (while equating noise energy at frequencies within the pass-band) is a useful way of reducing the contrast range of the noise without affecting its masking strength.
The advantage of a low truncation threshold is that it reduces the contrast range required to display a given energy level. An apparent drawback is that it artificially introduces edges as for binarized-filtered noise (i.e., truncation threshold = 0, Fig. 1). Truncating the noise at ±1 sd, was found, in the example above (center image of Fig. 6), to substantially reduce the appearance of the edges and reduce the energy (given equated total contrast range) by a factor of only 1.4 relative to binarized-filtered noise. We can also note that using a truncation threshold of ±2 sd compared to untruncated-filtered noise (fourth vs last image of Fig. 6) has little impact on the noise appearance and provides the advantage of having a noise that covers a smaller and well-defined contrast range. However, a truncation threshold at ±2 sd reduced energy (given equated total contrast range) by about 2.8 times relative to binarized-filtered noise. Ultimately, choosing the truncation threshold depends on the experimental paradigm and its constraint (e.g., contrast available and potential issue of apparent edges), but the truncated-filtered noise at ±1 sd appears to be a good compromise, as it requires little additional contrast and substantially reduces the appearance of edges.
For truncated-filtered noise to be equivalent to Gaussian noise, they should have the same masking strength. The current study suggests that truncating (or even binarizing) filtered noise has no impact on performance given equated noise energy at frequencies within the pass-band (see Appendix for equating noise energy). Thus, truncated-filtered noise is equivalent to Gaussian noise, if the filtering operation has no impact on performance. In the current study, the noise was filtered according to the spatial frequency, but it could also be filtered along any other dimension such as temporal frequency or orientation. A priori, however, it is not possible to know what information can be filtered out without causing a noticeable change in performance as it depends on which information is relevant to the visual system for the given task. The processing involved in a typical contrast detection task, for instance, is known to be narrowly tuned in the spatial frequency domain, which explains why a narrow filter (e.g., ±1 octave of the signal spatial frequency) has no effect on performance as observed here and elsewhere (Pelli, 1981; Stromeyer & Julesz, 1972). But for filtering along other dimensions, the filter that can be used without affecting performance depends on the tuning of the processing for the given task and cannot be known a priori. In sum, the filter that can be used to avoid reducing masking strength cannot be known a priori as it depends on the processing properties relevant to the given task. Obviously, truncated-filtered noise suffers from the same limitations as filtered noise. Nevertheless, once a filter is chosen (and assumed or shown not to have any impact on performance when applied to Gaussian noise), the current study suggests that the noise can be truncated in order to further increase energy and thereby extend the range of conditions over which noise-masking paradigms can be useful. See the Appendix for Matlab code to generate truncated-filtered noise filtered along the spatial frequency, orientation or/and temporal frequency dimensions.
A common use of external noise is to quantify the performance of human observers relative to the performance of an ideal observer (Gold, Abbey, Tjan, & Kersten, 2009; Kersten & Mamassian, 2010). An ideal observer often has a perfect performance in noiseless condition (e.g., an infinitely small contrast detection threshold), but under noisy conditions, the optimal performance is limited even when using optimally all the available information. If the human performance is close to the ideal performance (e.g., Allard & Cavanagh, 2012; Allard & Faubert, 2013; Baldwin, Baker, & Hess, 2016), then this means that the observer efficiently integrates all the necessary information to perform the task. Otherwise, some information must be lost, deteriorated or not optimally integrated. Although truncated-filtered noise may be equivalent to Gaussian noise for a human observer, they may not be equivalent for an ideal observer, which may use additional information only available in truncated-filtered noise (e.g., detect a signal when the luminance of any pixel exceed, even by an infinitely small amount, the contrast range of the truncated noise). Thus, at first sight, using truncated-filtered noise seems to compromise the comparison with the ideal observer even if truncated-filtered noise is equivalent to Gaussian noise for the human observer. However, a simple way to prevent the ideal observer from using additional information only available in truncated-filtered noise is to compare the human performance in truncated-filtered noise with the ideal performance in Gaussian noise. Indeed, given that the human performance in truncated-filtered noise is equivalent to the one in Gaussian noise, the human performance in Gaussian noise can be estimated and compared with the ideal performance. Thus, the use of truncated-filtered noise instead of Gaussian noise does not compromise the comparison with the ideal performance given that both noises are equivalent for human observers. Such a comparison can be performed by quantifying the noise energy of Gaussian and truncated-filtered noise at the unfiltered frequencies (see Appendix for a detailed algorithm and Matlab code).
The current method should not be confused with methods improving the contrast resolution of digital displays, such as the Noisy-bit (Allard & Faubert, 2008) or bit-stealing (Tyler, 1997) methods. These methods aim at overcoming practical limitations of the digital display (smallest contrast displayable), whereas the current study rather deals with the theoretical limit of 100 % contrast displayable. Thus, these two kinds of method address distinct displayable contrast limitations (smallest and highest displayable contrast) and can be used conjointly if needed.
In sum, the current study developed truncated-filtered noise by combining two methods to increase noise energy at frequencies relevant to the task without increasing noise contrast. This novel noise enables to extend the use of noise-masking experiments over a wider range of conditions.
Thanks to Daphné Silvestre for helpful comments. This research was supported by ANR – Essilor SilverSight Chair.
- Allard, R., & Cavanagh, P. (2011). Crowding in a detection task: external noise triggers change in processing strategy. Vision Research, 51(4), 408–416. Retrieved from http://ovidsp.ovid.com/ovidweb.cgi?T=JS&CSC=Y&NEWS=N&PAGE=fulltext&D=medl&AN=21185855
- Allard, R., & Cavanagh, P. (2012). Different processing strategies underlie voluntary averaging in low and high noise. Journal of Vision, 12(11). doi: 10.1167/12.11.6
- Allard, R., & Faubert, J. (2013). Zero-dimensional noise is not suitable for characterizing processing properties of detection mechanisms. Journal of Vision, 13(10). doi: 10.1167/13.10.25
- Allard, R., & Faubert, J. (2014a). Motion processing: The most sensitive detectors differ in temporally localized and extended noise. Frontiers in Psychology, 5. doi: 10.3389/fpsyg.2014.00426
- Allard, R., & Faubert, J. (2014b). To characterize contrast detection, noise should be extended, not localized. Frontiers in Psychology, 5. doi: 10.3389/fpsyg.2014.00749
- Allard, R., Faubert, J., & Pelli, D. G. (2015). Editorial: Using visual noise to reveal the computations underlying perception. Frontiers in Psychology, 6(1707). doi: 10.3389/fpsyg.2015.01707
- Campbell, F., & Gubisch, R. (1966). Optical quality of the human eye. The Journal of Physiology, 558–578. doi: 10.1113/jphysiol.1966.sp008056
- Kersten, D., & Mamassian, P. (2010). Ideal observer theory. In Encyclopedia of Neuroscience (pp. 89–95). doi: 10.1016/B978-008045046-9.01435-2
- Levitt, H. (1971). Transformed up-down methods in psychoacoustics. Journal of the Acoustical Society of America, 49(2), Suppl 2:467+.Google Scholar
- Pelli, D. G. (1981). The effects of visual noise. Department of Physiology. Cambridge University, Cambridge.Google Scholar
- Raghavan, M. (1995). Sources of visual noise. Syracuse, NY: Syracuse University.Google Scholar