Abstract
Equiluminant stimuli help assess the integrity of colour perception and the relationship of colour to other visual features. As a result of individual variation, it is necessary to calibrate experimental visual stimuli to suit each individual’s unique equiluminant ratio. Most traditional methods rely on training observers to report their subjective equiluminance point. Such paradigms cannot easily be implemented on pre-verbal or non-verbal observers. Here, we present a novel Pupil Frequency-Tagging Method (PFTM) for detecting a participant’s unique equiluminance point without verbal instruction and with minimal training. PFTM analyses reflexive pupil oscillations induced by slow (< 2 Hz) temporal alternations between coloured stimuli. Two equiluminant stimuli will induce a similar pupil dilation response regardless of colour; therefore, an observer’s equiluminant point can be identified as the luminance ratio between two colours for which the oscillatory amplitude of the pupil at the tagged frequency is minimal. We compared pupillometry-based equiluminance ratios to those obtained with two established techniques in humans: minimum flicker and minimum motion. In addition, we estimated the equiluminance point in non-human primates, demonstrating that this new technique can be successfully employed in non-verbal subjects.
Similar content being viewed by others
Introduction
Perception of the chromatic properties of light is essential for adaptive visual behaviours in many animals1, including human and non-human primates2. While most eutherian mammals are dichromats, about 30–40 million years ago, a mutation of the X chromosome-linked gene for the long wavelength-sensitive cone pigment resulted in trichromatic vision in catarrhine primates (the family that includes old-world monkeys, apes, and humans). Substantial individual variation in cone distributions and the subsequent processing across the visual cortex inevitably leads to substantial subjective differences in the perceptual properties of colour3,4.
One such property that differs among individuals is the response of the luminance pathways to different chromatic inputs. This response is proportional to a stimulus’ objective luminance as defined by luminous intensity per unit area (candelas per square meter, cd/m2), but the proportion varies across individuals. In experiments of colour perception, it is essential to set the colours of the stimuli to subjectively equal luminance levels to better isolate the visual system’s elements sensitive to colour. For example, equiluminant colour stimuli optimally isolate midget cells’ relative contribution to the parvocellular layers of the lateral geniculate nucleus (sensitive to chromatic contrast) in constructing visual perception5,6. However, there is substantial individual variation in the luminance response to stimuli of the same objective luminance value7. As a result, an individual’s unique equiluminance ratio must be determined in order to minimise response in the observer’s luminance pathway. While there are several established methods of evaluating equiluminance ratios8, most require the cooperation and understanding of the individual being tested. They are less effective for assessing the equiluminance ratio of nonverbal or preverbal populations, such as non-human primates or human infants9. Optokinetic nystagmus (OKN) has been used to measure the equiluminance point in human infants, amphibians, and fish10,11,12. However, the OKN reflex critically depends on the processing of motion, which has a dominant input from magnocellular pathways that provide weak contributions to colour pathways13,14. Additionally, because similar oculomotor structures control the OKN and ocular fixation, the OKN can be differentially suppressed via the frontal eye fields during attentive fixation in human and non-human primates15,16. This voluntary control of the OKN response makes its use problematic for populations such as non-human primates with extensive training in fixation.
There is a growing appreciation of how, in addition to classical reflexive mechanisms, subjective cognitive processing also modifies pupil diameter17,18. Perhaps the clearest example of this comes from studies showing that learned categories (i.e. “the Sun is very bright”)19,20,21 or illusions of brightness in humans22, and even rodents23, can change pupil size. Additionally, there is considerable evidence that the pupillary response is sensitive to inputs from the extrastriate colour pathways in human and non-human primates24,25,26,27. While the pupillary response to colour can be impacted by attention28, this is not the global antagonistic impact attention has on the OKN reflex. Therefore, we developed a new method of measuring equiluminance ratios that provides an objective measure in verbal and nonverbal populations. This uses a low-cost eye tracking device along with rapid and straightforward signal processing methods. Pupil diameter is mainly driven by perceived luminance independently of colour22 so that a flickering stimulus that alternates between two colours will cause the pupil diameter to oscillate at the same frequency as the luminance variation in this stimulus (hereto forth, the “tagging frequency”). With this Pupil Frequency-Tagging Method (PFTM) the amplitude of the oscillations at the tagging frequency will be proportional to the perceived luminance difference between the two colours—as long as the frequency is slow enough to drive a reliable pupillary response (< 2 Hz)29 and fast enough to minimise input from the melanopsin pathway that also drives the pupil response for prolonged changes (< 1 Hz)30,31 (noting that the melanopsin response has little to no impact on colour perception)32. The pupil oscillations will be at a minimum when the two alternating colours have matched perceived luminance. Therefore, an observer’s unique equiluminance point can be identified as the luminance ratio between two colours at which the Fast Fourier Transform (FFT) amplitude at the tagging frequency is minimal.
Results
Exemplar PFTM measurements in human
We first sought to measure whether the PFTM stimulus successfully drove oscillations in pupil diameter that were synchronous with the frequency of the alternations of two different luminances of the visual stimuli. The stimulus itself comprised a 20° smoothed central disc that changed between two stimuli (either two luminance values of the same colour, or two luminance values of two different colours) every 0.283 s, presented against a physically equiluminant grey background (Fig. 1a). As the participant maintained fixation, pupil diameter was recorded with an eye-tracker with sample times aligned to stimulus onset. Figure 1b shows an example of the raw pupil diameter plotted against time: the main features of the pupillary stimulus-locked oscillation that vary with luminance are a mean baseline change, a variable oscillatory amplitude and a phase reversal for luminances above and below a minimal amplitude.
We first used the same colour (e.g. a fixed red and variable red) to verify that the pupillary oscillation showed a minimum at the fixed luminance. In all cases we found that the oscillation in the raw pupil diameter disappeared only when the fixed and variable colour were identical. We next wanted to test how the oscillation varied when testing a different fixed and variable colour. A full example of the analysis necessary to quantify a perceptual minimum between two different colours are shown in Figs. 2, 3 and 4. They demonstrate pupil diameter oscillations, amplitude extracted via the FFT transform, and normalised amplitude at the tagging frequency plotted across all variable luminances, respectively. For this human exemplar (participant 2), we held red fixed at 21.72 cd/m2, and varied green in ten equally spaced steps between 0 and 41.2 cd/m2 (Fig. 2a). We chose 5 repeated trials of each luminance to minimise task duration, leading to a total task time of 5 min and 25 s.
The stimulus oscillation is illustrated by alternating grey bands and stimulus icons in Fig. 2b. Pupil changes are slow, and so we excluded the first second after stimulus onset then used the mean amplitude of the measurement range to correct each conditions response before averaging. The critical features of the pupil oscillation were the minimum in amplitude as a function of the variable luminance and a clear reversal in the oscillatory phase, with an almost complete 180° inversion of phase between the darkest and lightest values of the variable test colours.
The FFT can extract amplitude and phase measurements from a time–series signal. We first plotted the FFT amplitudes across all frequencies ranging from 0 to 7 Hz. 0 Hz represents the mean baseline change, and in this case, we baseline-corrected the pupil diameter signals resulting in no values at 0 Hz. The tagging frequency was 1.77 Hz, and we can see an amplitude peak at that frequency whose value changes across luminance. The amplitudes (normalised to the largest amplitude) for each luminance at the tagging frequency are shown in Fig. 4. We predicted that the amplitude of pupillary oscillations should be minimised when the two stimuli had a similar luminance.
We noted a clear change in normalised amplitude with luminance, and the minimum was observed at 22.89 cd/m2 of green (red luminance denoted by dotted line). This denoted a clear shift from the physical equiluminant point. We also noted that the phase transition started after 18.31 and ended at 32.04. The best-fitting spline had its minimum more closely aligned to the mid-point of the phase transition, and we therefore used the fitted curve to estimate the minimum ratio, in this participant giving a value of 0.79.
Comparisons of PFTM with flicker and motion tasks in humans
Previous methods that estimate a participant’s equiluminance point include tasks in which they must subjectively decide (using either an alternate forced choice or a matching paradigm) on a stimulus minimum.
For example, in a heterochromatic minimum flicker task (Fig. 5a), a participant can change the luminance of one test colour as a square wave reverses colour contrast over time to minimise the perception of flicker. In a minimum motion task, the participant can choose the direction of motion (for example, using a leftwards or rightwards 2-alternate forced choice [2-AFC]) and the point at which leftward and rightward choices occur equally is taken to be the perceptual minimum (Fig. 5b). We used a matching paradigm for our minimum flicker and a randomised 2-AFC task for minimum motion.
The minimum-flicker, minimum-motion and PFTM settings for red versus green for the eight participants are shown in the three columns of Table 1, respectively. We visualised these values using a raincloud plot (Fig. 6a) and as a correlation matrix (Fig. 6b). Overall, no significant differences existed between any group means (Flicker: Motion p = 0.69/BF10 = 0.36; Flicker: PFTM p = 0.46/BF10 = 0.95; Motion: PFTM p = 0.36/BF10 = 0.7; Holm post-hoc corrected repeated-measures ANOVA / Bayesian paired t-tests).
Testing for bivariate normality suggested that only Flicker:Motion violated the assumption of normality requiring the use of a nonlinear correlation test (Shapiro–Wilk = 0.789, p = 0.022). The correlation between Flicker: Motion was weakly positively correlated without reaching significance (one-tailed Spearman ρ = 0.402, effect size = 0.427, p = 0.161). We found that Flicker:PFTM exhibited the strongest significant positive correlation (one-tailed Pearson r = 0.727, effect size = 0.922, p = 0.021), and Motion:PFTM showed a moderate positive correlation that did not reach significance (one-tailed Pearson r = 0.585, effect size = 0.67, p = 0.064). We also performed pairwise reliability plots (Fig. 6c) that suggested that there are no substantial differences in agreement across the three methods (Flicker: Motion − 0.13 ≺ 0.01 ≻ 0.15 | Flicker:PFTM − 0.13 ≺ − 0.03 ≻ 0.07 | Motion:PFTM − 0.2 ≺ − 0.04 ≻ 0.12 |− 1.96 SD ≺ mean ≻ 1.96 SD). Overall, the values were within the previously reported range for colour normal observers, and the weak correlation between minimum flicker and motion has been previously described8,33.
Measuring PFTM in non-human primates
It is often optimal for non-verbal subjects like children or non-human primates for test sessions to contain independent trials, where positive feedback and encouragement (in children) and fruit or food rewards (non-human primates) can be assigned for correct behaviour. Previous studies have shown that pupil responses to colour in human and non-human primates exhibit similar properties25. To test this, we used our PFTM protocol with four non-human primates that had previously been trained to keep their eyes steady. They were required to fixate a central spot for 3.5 s while the stimulus oscillated between the fixed and a variable luminance at the tagging frequency. We first used the same colour to test that the PFTM could identify a clear minimum. Given our subject’s daily trial completion rates, we increased the sampled luminance density and used a greater number of repeated trials at each luminance.
We first sought to confirm that the PFTM method could drive robust pupil signals with features similar to those observed in humans using a same-colour test. Figure 7a plots control data for the condition where both the fixed and variable luminance were in the green channel. The fixed luminance was set at 11.04 cd/m2, and 21 variable values were linearly spaced between 1.6 and 33.05 cd/m2. Ten repeated trials were taken for each variable luminance value (210 trials total), and all trial placements were fully randomised and interleaved. We baseline-corrected using the pre-stimulus mean amplitude (around 0 s) to visualise any potential baseline changes during visual stimulation. After an initial response latency, the pupil diameter increases (signifying dilation caused by a lower perceived brightness) or decreases (signifying constriction caused by a higher perceived brightness) with a baseline offset positively correlated with the difference between the fixed and variable values. After about 0.7 s, the signal oscillated with a period matched to the stimulus reversals. In addition, the phase of the signal peaks for variable values lower than 11.04 cd/m2 reverses for variable values higher than 11.04 cd/m2. This confirmed that the pupil signals in non-human primate had the same features as those observed in humans.
To quantify this, we computed the FFT and focused on the frequencies at the tagging frequency (Fig. 7b). As we did not remove the baseline offset, there is substantial signal at 0 Hz, which demonstrates the mean offset from the baseline, and the largest values are those with the greatest differential to the fixed value. There is also a clear peak at the tagging frequency, whose magnitude is again positively correlated with the difference between fixed and variable values. To better measure these relationships, we replotted the FFT amplitude data for the mean (0 Hz) and tagging frequency against variable stimulus luminance (Fig. 7c). There is a clear minimum at the equiluminant 11.04 cd/m2 point for the amplitude extracted at the tagging frequency (dark red curve). The curve plotted for the amplitude extracted for the mean (0 Hz dark blue) also shows a minimum; however, it is at 12.61 cd/m2, missing the equiluminant point, and exhibits a somewhat broader tuning relationship across luminance (it was common that the mean amplitude was not a reliable indicator of an equiluminant point). Finally, the phase calculated at the tagging frequency shows the reversal identified in Fig. 7a, going from around − 30° at luminances lower than, to around 140° for luminances higher than the equiluminant point.
We next held red as the fixed colour and varied green around this point (as in the human experiments) to test our four participants. We found clear minima that differed from the physical luminance (CIE) matches (Table 2). The ratio values we observed were qualitatively within the range of that seen for the human participants.
Our non-human participants could maintain fixation for 3–4 s, but not all participants may be able to maintain fixation for this length of time. We were therefore interested in seeing whether a shorter data collection can yield a measurable minimum using a post-hoc analysis. Figure 8 shows the pupil data for participant Fanfan comparing the raw data (upper) and FFT amplitude and phase (lower) for a single cycle (Fig. 8a) or 5 cycles (Fig. 8b).
Although the error bars are larger for the single cycle and phase transition more variable, the overall minimum defined by amplitude at the tagging frequency is preserved. We can therefore trade-off fixation length against noise depending on the participant’s ability to maintain fixation. We also tested downsampling the raw pupil data to 120 Hz, considering eye-trackers with lower sampling rates, which had no impact on the FFT results (data not shown).
Discussion
We successfully identified the equiluminance point of participants by analysing pupil oscillations at the tagging frequency (1.77 Hz) and compared these values to measurements obtained using traditional photometric tasks. Our results show that the amplitude of pupil dilation oscillations in the tagging frequency band closely corresponds to the difference in luminance values when a single colour was used in the flickering stimuli and that these luminance-induced pupil responses are reduced at equiluminance when two colours are used. This measurement was significantly correlated to a previously well–established minimum flicker measurement. Consequently, PFTM enables measurement of equiluminance at colour exchange rates far below the higher temporal rates used in the flicker and motion tasks. Pupillary Frequency-Tagging has a significant advantage over established subjective techniques by objectively measuring an obligatory, automated response (pupil dilation) rather than relying on subjective responses about the appearance of the stimulus.
The pattern of ratio values for the two subjective methods compared to the PFTM shown in Fig. 6 show fair, but not exact correlation. Two accounts of these differences are worth discussing. First, it is important to note that the stimuli used in the PFTM, the minimum flicker, and minimum motion tests differ in several respects. Firstly, the temporal frequencies used are different across the three tasks. Changes in temporal frequency can profoundly affect neuronal responses in the retina and at cortical stages, which could thus account for some of the observed differences. Second, and perhaps more importantly, the circuits underlying the autonomic pupillary response and subjective responses differ. The “fast” circuit involved in the pupil constriction response—from the retina to the iris sphincter bypasses the thalamocortical pathway. This is not the case with subjective pupil responses that reflect the processing of downstream cortical neurons along the visual processing pathways from V1 to V4 and beyond24,25,27,34,35. At these higher levels of processing, neurons have an exquisite sensitivity to colour contrast, and colour/luminance perception results from complex interactions that initiate within the magnocellular, parvocellular and koniocellular cells from V1 onwards36. These distinctions in the relative contributions of the neural pathways may account for the some of the differences found here. However, it is known that feedback from frontal and occipital cortices via the locus coeruleus (LC), superior colliculus (SC), and hypothalamic orexin neurons do ultimately target the same nuceli18,37,38,39. It is also worth noting that the Spearman correlation between minimum motion and minimum flicker (both subjective methods) were the lowest among the three methods (Fig. 6), therefore any such anatomical pathway differences cannot fully explain differences in the perceptual equiluminant point between methods.
Equiluminant stimuli are often used to reduce the relative contributions of magnocellular cells to visual processing to study the contributions of the parvocellular cells “in isolation” (though noting the contested effects on parvo vs. magno pathways)40. A critical question arises concerning the best strategy to do so. At first, reducing the overall contribution of magnocellular cells at the retinal level may seem optimal, as cortical processing builds upon retinal inputs. One should be cautious however, in part because this view relies on underlying assumptions concerned with the projections of RGCs to the pupillary circuits that differ from their projections to the cortex. In this respect, recent detailed publications41,42 reveal the complex interactions within the retina before projecting to the pretectal olivary nucleus (PON) and Edinger-Westphal nucleus (EWN, that project to the ciliary ganglion driving the iris sphincter). The dominant view is that intrinsically photosensitive retinal ganglion cells (ipRGCs) containing melanopsin43 are the dominant drive to pupillary circuits for stimuli with temporal frequencies at 1 Hz or lower30. Our stimuli were modulated at 1.77 Hz. Interactions of ipRGCs with rods, cones, and RGCs have been consistently reported and can account for the pupillary responses to coloured stimuli not directly stimulating ipRGCs (as the absorption spectrum of melanopsin lies in the blue range—short wavelength) at lower temporal rates. Although it is out of the scope of this report to describe these circuits in detail, one should keep in mind that dedicated circuits exist to drive pupillary responses that differ from those driving cortical neurons through the thalamocortical pathways.
We used a method-of-constants presentation method for this study, but preliminary experiments also tested continual stimulation44 with no trial-by-trial structure. Subjects continued fixation and could blink etc. while the variable luminance ramped from a minima to maxima. By using post-hoc signal processing, we could also successfully extract the equiluminant ratio (data not shown). An alternative paradigm could use an adaptive staircase45; and given the computational simplicity to run the FFT in real time an adaptive staircase driven by the FFT amplitude of the previous trial could significantly reduce the time it takes to find the equiluminant point. Likewise, a heuristic that uses the phase-transition to objectively weight the minimum-amplitude should converge more quickly on the minimum than a fitted curve alone. Measurements using PFTM should extend to blue and yellow, and the use of alternative colour spaces. Finally, we expect that exploring the spatial extent of the PFTM stimulus, given the known fall-off of colour-sensitive cells in the retina, could further optimise the PFTM technique in future experiments.
A primary predicted application of the Pupil Frequency-Tagging Method described here is as a tool to assess colour blindness based on equiluminance settings7. It is equally as applicable to cooperative human adults as to infants and non-human animal subjects, improving previous colour blindness detection techniques and providing an opportunity for novel research trajectories. Additionally, PFTM is helpful as a method to validate that stimuli used in studies of colour perception are truly equiluminant. This is a critical variable to control for, as equiluminant stimuli ensure that the effects of colour perception are better isolated from those of luminance. For example, dominant theories suggest that the saccadic inhibition (SI) effect, observed when a visual mask or “distractor” placed over a saccadic target consequently inhibits saccadic movements, is driven by superior collicular pathways46,47. While the SC and its afferent pathways were not thought to represent colour stimuli directly48,49,50, recent studies in non-human primates using physically equiluminant colour change detection does exhibit SC activity in non-human primates (which they speculated to come from V4 or IT)51,52. In both this task and the SI task, it would be beneficial that the colours used were perceptually matched to ensure that the visual circuit cannot identify the target and produce saccades or modify behaviour based on residual luminance signals alone. Studies that test neurodevelopmental changes of perception in human or non-human infants are another ideal target53, and PFTM would be a valid method to assess the infants’ equiluminance ratio to provide optimal chromatic control. Adding an attention grabbing smooth pursuit target could aid in keeping infants engaged during pupillary frequency tagging.
In the future, PFTM may be used by technology companies marketing TrueTone displays on their devices. With the continued refinement of the technology, it may be possible to find a user’s exact equiluminance point with a simple test of less than a minute and then build this luminance ratio into the display algorithms of the device. A screen that is personalised to the colour perception of each user will allow for a more pleasant user experience. This could be especially helpful to colour-blind users, for whom the display can be programmed for optimal contrast based on their unique equiluminance values.
Methods
Human participants
The experiments on the three methods of measuring equiluminance were run on 8 participants (5 male and 3 female), including two of the authors. All participants gave informed consent in writing prior to participation and the protocols for the study were approved by the ethical committee of the Institute of Neuroscience, Chinese Academy of Sciences, in accordance with the Declaration of Helsinki. They all reported normal colour vision and normal or corrected-to-normal acuity.
Non-human participants
Four adult rhesus macaques (male Macaca mulatta, 6–8 years old, weighing 4.5–10.9 kg) participated in this study. All experimental procedures were approved by the Animal Committee of the Institute of Neuroscience, Chinese Academy of Sciences, and were in accordance with the US National Institutes of Health Guide for the Care and Use of Laboratory Animals.
Apparatus
Stimuli were presented on a linearised Display++ (Cambridge Research Systems, Cambridge, UK) at a frame rate of 120 Hz, a resolution of 1920 × 1080 pixels, and a distance of 67 cm (human, yielding a size of 60.8° × 34.2°) or 72 cm (non-human, yielding a size of 56.9° × 31.8°). We measured 40 samples across the full range of luminance output and CIE spectral characteristics for the red, green and blue channels of the display using a SpectroCal2 spectrophotometer (Cambridge Research Systems, Cambridge, UK). All experiment code and stimulus generation was written using Opticka54 and the Psychophysics toolbox (PTB, V3.0.17)55, running under MATLAB 2021a and Ubuntu Linux 20.04. All stimuli used 32-bit floating point calculations internally and 14-bits per channel of colour output via the Color++ mode of the Display++. Eye position and pupil diameter were measured from the right eye using an Eyelink 1000 (SR Research, Ontario, Canada) at a sample rate of 1000 Hz.
The Eyelink records the size of the pupil in arbitrary units. We followed the instructions supplied by SR research to generate a scaling factor for a known physical diameter in millimetres. We performed this for both the human and NHP setups separately. Knowing the distance between the infrared camera and the participant, we used an 8 mm black target as an artificial pupil at the same position as the participant’s eye to record the arbitrary units for this known diameter target. The recorded data from the artificial pupil was then used to calculate a scaling factor, θ, using: \(\theta =8{\text{mm}}\div \sqrt{units}\). We then used θ to calculate our participant’s pupil diameter using: \(diameter=\theta \times \sqrt{data}\).
To ensure precise temporal fidelity of stimulus presentation, we used a photodiode recorded via a LabJack T4 to confirm that temporal intervals were accurate and without significant variance. We also used PTB’s flip command to log every frame’s VBL timestamp and calculate any missed frames, to confirm no frames were dropped. The MATLAB code, including friendly GUIs to run experiments for minimum motion, minimum flicker and PFTM methods, as well as analyse the data, is available at <https://github.com/iandol/equiluminance>.
Minimum flicker
We used a 15° square of grating presented against a grey background (physically equiluminant with the fixed luminance). Subjects were asked to fixate a small central cross, and maintain fixation within a 2° radius during stimulus presentation, but as this was a matching task we did not enforce fixation. A red/green square-wave grating of 0.25 cycles/° counterphase flickered (red and green exchanged places) at 15 Hz (30 reversals per second). The red and green were the isolated red and green of the display monitor (red and green CIE x/y coordinates were 0.172/0.738 and 0.675/0.323, respectively) and varied only in intensity. The stimulus profile was a square wave in space and time33. The red intensity was fixed to 21.04 cd/m2, and the green intensity varied to find the minimum flicker point. The maximum green luminance was 103 cd/m2. Before the main trials began, participants were allowed to practice the tasks to become familiar with how their perception of the stimuli changed as they reached their equiluminance point. Participants used key presses to increase or decrease the green luminance of the stimuli. The green luminance value was recorded when the participants achieved minimum flicker. The green value was then randomised, and the same procedure was repeated. Each participant performed five such repeats.
Minimum motion
We used a 15° square of grating presented against a grey background (physically equiluminant with the fixed luminance). Subjects were asked to initiate fixation on a small central cross for 0.3 s and then maintain fixation within a 2° radius during stimulus presentation; failure to maintain fixation aborted that trial. Four repeating frames generated motion to the left for red darker than green and in the opposite direction for red lighter than green (Fig. 5b). Subjects used the leftward or rightward arrows to choose the direction in a 2-alternate forced choice. When the luminances were matched for the participant, the motion was nulled, and choices were random around 50%. As with the minimum flicker, the minimum motion stimulus had a fixed red luminance of 21.04 cd/m2. The yellow was always the mixture of the red and green of the red/green grating at half amplitude (\(\left(R+G\right)\div 2\)) to keep the mean luminance matched between the red/green and yellow gratings. This was confirmed using our spectrophotometer. The light yellow was 20% more luminous than this \(\left(R+G\right)\div 2\) mixture and the dark yellow 20% less luminous. A method-of-constant-stimuli paradigm with 10 repeats was utilised, and psychometric fitting was performed using maximum likelihood estimation with a Quick function (Eq. 1 using the Palamedes toolbox56 and MATLAB 2021a):
The Quick function estimates the threshold at 50% of the psychometric curve.
Pupil frequency-tagging method
Each trial was initiated by the participant voluntarily foveating (2° radius) a fixation cross57 presented against a grey background (physically equiluminant with the fixed value colour). After the initial fixation was maintained for 0.8–1.0 s, a coloured disc 20° in diameter was presented while the participant continued to maintain fixation for a further 3.5 s. The edges of the disc were smoothed using 0.3° hermite interpolation window (OpenGL GLSL smoothstep function). Single-colour tasks (where the fixed and variable parameters had the same colour) were used to confirm that the paradigm was effective. In this task there is one step in the middle of the range in which the circle and background are identical, resulting in a solid screen that should produce no pupil oscillations in the tagging frequency band. For measurement of perceptual equiluminance, the luminance of one colour (red) was fixed and the luminance of a second colour (green) chosen from five to twenty linearly spaced luminance steps centered around the physical luminance value of the fixed colour. The duration of each colour presentation in the alternating sequence was 0.283 s, yielding a 1.77 Hz tagging frequency (Fig. 1). The luminance for red was usually fixed at 21.04 cd/m2. The grey background was displayed full-screen for 2 s between trials to stabilise the pupil diameter before the subsequent trial initiated. Blocks of trials consisted of the same fixed colour which alternated with the variable colour in a fully randomised order with at least 5 repeated presentations of each variable luminance condition.
For analysis, the pupil diameter time series was time-locked to stimulus onset. We first used a 30 ms gaussian smoothing function on the raw pupil data (smoothdata in MATLAB 2021a). Two different methods were then employed for baseline correction. The first involved calculating the mean pupil diameter between − 0.1 ↔ + 0.1 s for each individual trial and subtracting it from that individual trial’s data trace (pre-stimulus baseline correction). The second involved calculating the mean diameter from the same time range used for the subsequent FFT for each individual trial and subtracting it from that individual trial’s data trace (mean-baseline correction). Either of these were performed before averaging the trials for each condition. In general, mean-baseline correction resulted in slightly better signal:noise when calculating the FFT, but the differences were minimal (our analysis code computed both to check any differences manually). Next, a discrete Fourier transform was computed from the averaged time-locked signal. To measure the FFT amplitude at the 1st harmonic, the time window must be an integer multiple of the frequency time-period (e.g. \(0.567\times \rho\), where ρ is the number of repeat oscillation periods one wishes to measure). We offset the starting time-point \(t1\) manually to where the oscillation was stable (given the onset lag) and the phase of the darkest and lightest variable luminances were diverging so that measured phase was relative to that point. For example, if the start time was: \(t1=0.3\) and \(\rho =4\) cycles were used, then \(t2=t1+\left(0.567\times 4\right)=2.568\) and the time range for analysis is therefore \(\left[t1,t2\right]=\left[0.3, 2.568\right]\). We computed the mean and first harmonic at the tagged frequency and calculated the signal’s phase at the tagged frequency across all variable luminance steps. This was used to compute an interpolated tuning curve fit using a smoothing spline (fit function and the smoothingspline model type in MATLAB 2021a curve fitting toolbox), whose minimum is equal to the minimum FFT amplitude at the tagging frequency. We utilised the phase transition to confirm the minimum value. At this minimum, the changing colour is considered the closest perceptual match to the fixed colour.
Pupil frequency-tagging method applied to non-human primates: to assess PFTM in non-human primates, the paradigm detailed above was adapted to give positive rewards (preferred fruit juice delivered via a peristaltic pump controlled via an Arduino) after successfully initiating and maintaining fixation during the trial. All four non-human primate participants had been previously well trained to fixate on a central fixation spot (a standard procedure for most non-human primate research involving visual stimulation). The paradigm used the same spatial and temporal stimulus structure as that used with the human participants: a central disc and background in which a fixed and variable colour reversed lasting 3.5 s in each trial (except for one participant who could only fixate for 3 s). The luminance for red was usually fixed at 20.24 cd/m2 as we used a different Display++ monitor with a slightly different overall luminance. Given that non-human participants worked for reward, we could test more steps of the variable luminance and more repeated trials for each luminance could be collected. The analysis of the pupil signals was identical to the human data.
Statistical analysis
Statistical analysis was performed with JASP58. We checked for bivariate normality of the correlation using the Shapiro–Wilk test, equality of variance using Levene’s test, and the post-hoc analysis of the repeated-measures ANOVA was corrected using Holm’s method. Group comparisons were visualised using the Raincloud plot library in R59.
Data availability
The datasets used and/or analyzed during the current study available from the corresponding author on reasonable request.
References
Cuthill, I. et al. The biology of color. Science 357, eaan0221. https://doi.org/10.1126/science.aan0221 (2017).
Mollon, J. Tho’ she kneel’d in that place where they grew... The uses and origins of primate colour vision. J. Exp. Biol. 146, 21–38. https://doi.org/10.1242/jeb.146.1.21 (1989).
Brainard, D. H. Color and the cone mosaic. Annu. Rev. Vis. Sci. 1, 519–546. https://doi.org/10.1146/annurev-vision-082114-035341 (2015).
Bosten, J. M. Do you see what i see? Diversity in human color perception. Annu. Rev. Vis. Sci. 8, 101–133. https://doi.org/10.1146/annurev-vision-093020-112820 (2022).
De Valois, R. L., Cottaris, N. P., Elfar, S. D., Mahon, L. E. & Wilson, J. A. Some transformations of color information from lateral geniculate nucleus to striate cortex. J. Opt. Soc. Am. A: 97, 4997–5002. https://doi.org/10.1073/pnas.97.9.4997 (2000).
Hubel, D. & Livingstone, M. Color and contrast sensitivity in the lateral geniculate body and primary visual cortex of the macaque monkey. J. Neurosci. 10, 2223–2237. https://doi.org/10.1523/jneurosci.10-07-02223.1990 (1990).
Cavanagh, P. & Anstis, S. The contribution of color to motion in normal and color-deficient observers. Vis. Res. 31, 2109–2148. https://doi.org/10.1016/0042-6989(91)90169-6 (1991).
He, J., Taveras Cruz, Y. & Eskew, R. Methods for determining equiluminance in terms of l/m cone ratios. J. Vis. 20, 22. https://doi.org/10.1167/jov.20.4.22 (2020).
National Research Council (US) Committee on Vision. Procedures for Testing Color Vision: Report of Working Group 41 (National Academies Press (US), 1981). https://doi.org/10.17226/746.
Brown, A., Lindsey, D., McSweeney, E. & Walters, M. Infant luminance and chromatic contrast sensitivity: Optokinetic nystagmus data on 3-month-olds. Vis. Res. 35, 3145–3160. https://doi.org/10.1016/0042-6989(95)00017-t (1995).
Cavanagh, P., Anstis, S. & Mather, G. Screening for color blindness using optokinetic nystagmus. Invest. Ophthalmol. Vis. Sci. 25, 463–466 (1984).
Mueller, K. P. & Neuhauss, S. C. F. Quantitative measurements of the optokinetic response in adult fish. J. Neurosci. Methods 186, 29–34. https://doi.org/10.1016/j.jneumeth.2009.10.020 (2010).
Livingstone, M. & Hubel, D. Segregation of form, color, movement, and depth: Anatomy, physiology, and perception. Science 240, 740–749. https://doi.org/10.1126/science.3283936 (1988).
Nassi, J. & Callaway, E. M. Parallel processing strategies of the primate visual system. Nat. Rev. Neurosci. 10, 360–372. https://doi.org/10.1038/nrn2619 (2009).
Izawa, Y. & Suzuki, H. Suppressive control of optokinetic and vestibular nystagmus by the primate frontal eye field. J. Neurophysiol. 124, 691–702. https://doi.org/10.1152/jn.00015.2020 (2020).
Mastropasqua, A., Dowsett, J., Dieterich, M. & Taylor, P. Right frontal eye field has perceptual and oculomotor functions during optokinetic stimulation and nystagmus. J. Neurophysiol. 123, 571–586. https://doi.org/10.1152/jn.00468.2019 (2020).
Mathôt, S. Tuning the senses: How the pupil shapes vision at the earliest stage. Annu. Rev. Vis. Sci. 6, 433–451. https://doi.org/10.1146/annurev-vision-030320-062352 (2020).
Joshi, S. & Gold, J. Pupil size as a window on neural substrates of cognition. Trends Cogn. Sci. 24, 466–480. https://doi.org/10.1016/j.tics.2020.03.005 (2020).
Binda, P., Pereverzeva, M. & Murray, S. Pupil constrictions to photographs of the sun. J. Vis. 13, 8. https://doi.org/10.1167/13.6.8 (2013).
Naber, M. & Nakayama, K. Pupil responses to high-level image content. J. Vis. 13, 7. https://doi.org/10.1167/13.6.7(2013).
Castellotti, S., Conti, M., Feitosa-Santana, C. & Del Viva, M. Pupillary response to representations of light in paintings. J. Vis. 20, 14. https://doi.org/10.1167/jov.20.10.14 (2020).
Suzuki, Y., Minami, T., Laeng, B. & Nakauchi, S. Colorful glares: Effects of colors on brightness illusions measured with pupillometry. Acta Psychol. 198, 102882. https://doi.org/10.1016/j.actpsy.2019.102882 (2019).
Vasilev, D., Raposo, I. & Totah, N. Brightness illusions evoke pupil constriction preceded by a primary visual cortex response in rats. Cereb. Cortex 33, 7952–7959. https://doi.org/10.1093/cercor/bhad090 (2023).
Heywood, C. A., Nicholas, J. J., LeMare, C. & Cowey, A. The effect of lesions to cortical areas V4 or AIT on pupillary responses to chromatic and achromatic stimuli in monkeys. Exp. Brain Res. 122, 475–480. https://doi.org/10.1007/s002210050536 (1998).
Gamlin, P., Zhang, H., Harlow, A. & Barbur, J. Pupil responses to stimulus color, structure and light flux increments in the rhesus monkey. Vis. Res. 38, 3353–3358. https://doi.org/10.1016/s0042-6989(98)00096-0 (1998).
Carle, C., James, A. & Maddess, T. The pupillary response to color and luminance variant multifocal stimuli. Investig. Ophthalmol. Vis. Sci. 54, 467–475. https://doi.org/10.1167/iovs.12-10829 (2013).
Sabeti, F. et al. Comparing multifocal pupillographic objective perimetry (mfPOP) and multifocal visual evoked potentials (mfVEP) in retinal diseases. Sci. Rep. 7, 45847. https://doi.org/10.1038/srep45847 (2017).
Rosli, Y. et al. Retinotopic effects of visual attention revealed by dichoptic multifocal pupillography. Sci. Rep. 8, 2991. https://doi.org/10.1038/s41598-018-21196-1 (2018).
Naber, M., Alvarez, G. & Nakayama, K. Tracking the allocation of attention using human pupillary oscillations. Front. Psychol. 4, 919. https://doi.org/10.3389/fpsyg.2013.00919 (2013).
Joyce, D. S., Feigl, B., Cao, D. & Zele, A. J. Temporal characteristics of melanopsin inputs to the human pupil light reflex. Vis. Res. 107, 58–66. https://doi.org/10.1016/j.visres.2014.12.001 (2015).
Carle, C. F., James, A. C., Kolic, M., Essex, R. W. & Maddess, T. Blue multifocal pupillographic objective perimetry in glaucoma. Investigat. Opthalmol. Vis. Sci. 56, 6394–6403 (2015).
Woelders, T., Allen, A. E. & Lucas, R. J. Melanopsin enhances image persistence. Curr. Biol. https://doi.org/10.1016/j.cub.2023.10.039 (2023).
Anstis, S. & Cavanagh, P. A Minimum Motion Technique for Judging Equiluminance Color Vision (Academic Press, 1983).
Vandenbroucke, A., Fahrenfort, J., Meuwese, J., Scholte, H. S. & Lamme, V. A. F. Prior knowledge about objects determines neural color representation in human visual cortex. Cereb. Cortex 26, 1401–1408. https://doi.org/10.1093/cercor/bhu224 (2014).
Bannert, M. M. & Bartels, A. Decoding the yellow of a gray banana. Curr. Biol. 23, 2268–2272. https://doi.org/10.1016/j.cub.2013.09.016 (2013).
Horwitz, G. Signals related to color in the early visual cortex. Annu. Rev. Vis. Sci. 6, 287–311. https://doi.org/10.1146/annurev-vision-121219-081801 (2020).
Joshi, S., Li, Y., Kalwani, R. & Gold, J. Relationships between pupil diameter and neuronal activity in the locus coeruleus, colliculi, and cingulate cortex. Neuron 89, 221–234. https://doi.org/10.1016/j.neuron.2015.11.028 (2016).
Ebitz, R. & Moore, T. Selective modulation of the pupil light reflex by microstimulation of prefrontal cortex. J. Neurosci. 37, 5008–5018. https://doi.org/10.1523/JNEUROSCI.2433-16.2017 (2017).
Grujic, N., Tesmer, A., Bracey, E., Peleg-Raibstein, D. & Burdakov, D. Control and coding of pupil size by hypothalamic orexin neurons. Nat. Neurosci. 26, 1160–1164. https://doi.org/10.1038/s41593-023-01365-w (2023).
Skottun, B. On using isoluminant stimuli to separate magno- and parvocellular responses in psychophysical experiments-a few words of caution. Behav. Res. Methods 45, 637–645. https://doi.org/10.3758/s13428-012-0290-1 (2013).
Fain, G. & Sampath, A. P. Rod and cone interactions in the retina. F1000 Res. 7, 657. https://doi.org/10.12688/f1000research.14412.1 (2018).
Pickard, G. & Sollars, P. Intrinsically photosensitive retinal ganglion cells. Rev. Physiol. Biochem. Pharmacol. 162, 59–90. https://doi.org/10.1007/112_2011_4 (2012).
Lucas, J. & Schmidt, T. Cellular properties of intrinsically photosensitive retinal ganglion cells during postnatal development. Neural Dev. 14, 8. https://doi.org/10.1186/s13064-019-0132-2 (2019).
Huk, A., Bonnen, K. & He, B. J. Beyond trial-based paradigms: Continuous behavior, ongoing neural activity, and natural stimuli. J. Neurosci. 38, 7551–7558. https://doi.org/10.1523/jneurosci.1920-17.2018 (2018).
Prins, N. The psi-marginal adaptive method: How to give nuisance parameters the attention they deserve (no more, no less). J. Vis. 13, 3. https://doi.org/10.1167/13.7.3 (2013).
Reingold, E. & Stampe, D. Saccadic inhibition in voluntary and reflexive saccades. J. Cognit. Neurosci. 14, 371–388. https://doi.org/10.1162/089892902317361903 (2002).
Sparks, D., Rohrer, W. & Zhang, Y. The role of the superior colliculus in saccade initiation: A study of express saccades and the gap effect. Vis. Res. 40, 2763–2777. https://doi.org/10.1016/s0042-6989(00)00133-4 (2000).
Marrocco, R. & Li, R. Monkey superior colliculus: Properties of single cells and their afferent inputs. J. Neurophysiol. 40, 844–860. https://doi.org/10.1152/jn.1977.40.4.844 (1977).
Schiller, P., Malpeli, J. & Schein, S. Composition of geniculostriate input ot superior colliculus of the rhesus monkey. J. Neurophysiol. 42, 1124–1133. https://doi.org/10.1152/jn.1979.42.4.1124 (1979).
Schiller, P. & Malpeli, J. Properties and tectal projections of monkey retinal ganglion cells. J. Neurophysiol. 40, 428–445. https://doi.org/10.1152/jn.1977.40.2.428 (1977).
Herman, J. P. & Krauzlis, R. J. Color-change detection activity in the primate superior colliculus. eNeuro 4. https://doi.org/10.1523/ENEURO.0046-17.2017 (2017).
White, B., Boehnke, S., Marino, R., Itti, L. & Munoz, D. Color-related signals in the primate superior colliculus. J. Neurosci. 29, 12159–12166. https://doi.org/10.1523/JNEUROSCI.1986-09.2009 (2009).
Skelton, A., Maule, J. & Franklin, A. Infant color perception: Insight into perceptual development. Child Dev. Perspect. 16, 90–95. https://doi.org/10.1111/cdep.12447 (2022).
Andolina, I. M. Opticka: Behavioural research software. Zenodo https://doi.org/10.5281/zenodo.8412813 (2023).
Kleiner, M. et al. What’s new in psychtoolbox-3. Perception 36, 1 (2007).
Prins, N. & Kingdom, F. A. A. Applying the model-comparison approach to test specific research hypotheses in psychophysical research using the palamedes toolbox. Front. Psychol. 9, 1250. https://doi.org/10.3389/fpsyg.2018.01250 (2018).
Thaler, L., Schütz, A. C., Goodale, M. A. & Gegenfurtner, K. R. What is the best fixation target? The effect of target shape on stability of fixational eye movements. Vis. Res. 76, 31–42. https://doi.org/10.1016/j.visres.2012.10.012 (2013).
JASP Team. JASP open source statistical software (version 0.18.0). https://jasp-stats.org (2023).
Allen, M. et al. Raincloud plots: A multi-platform tool for robust data visualization. Wellcome Open Res. 4, 63. https://doi.org/10.12688/wellcomeopenres.15191.2 (2021).
Acknowledgements
We would like to thank Jean Lorenceau for comments and advice on the manuscript. This work was supported by the following grants: National Science and Technology Innovation 2030 Major Program 2022ZD0204600 (to W.W.), National Natural Science Foundation of China grants 32070992 and 32150410370 (to I.M.A.) and 32100805 (to Y.L.), Shanghai Municipal Science and Technology Major Project 2018SHZDZX05 (to I.M.A.).
Author information
Authors and Affiliations
Contributions
Y.L. collected and analyzed non-human primate data, and contributed to review and editing; B.W.M. collected preliminary data and contributed to review and editing; X.W. collected and analyzed the human data, and contributed to review and editing; P.M.D. collected and analyzed preliminary data; W.W. provided funding and contributed to review and editing; I.M.A provided funding, wrote the experimental control and analysis software and prepared the figures. P.C.; P.P.; P.M.D.; and I.M.A. conceptualised the study, discussed analysis and wrote and revised the manuscript.
Corresponding authors
Ethics declarations
Competing interests
P.M.D and P.P are co-funder of P3lab (a clinical setting exploring pupil responses). Y.L.;B.W.M.; X.W.; W.W.; P.C. and I.M.A. declare that they have no competing interest.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Liu, Y., Mahony, B.W., Wang, X. et al. Assessing perceptual chromatic equiluminance using a reflexive pupillary response. Sci Rep 14, 2420 (2024). https://doi.org/10.1038/s41598-024-51982-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-024-51982-z
- Springer Nature Limited