A total of 97 participants were tested in Chennai, the capital city of Tamil Nadu state in Southern India. The participants were divided into three groups: illiterate (N = 34, mean age = 36.0 years) participants who did not know to read and write Tamil and had not attended any formal education, high-literate participants (N = 30, mean age = 37.4 years) who had completed at least 10 years of formal education and could read and write Tamil, and a third, low-literate category (N = 30, mean age = 36.3 years) who had completed only primary education and dropped out during middle school. The participants were recruited through an NGO that works to support the development of the urban poor in Chennai. The participants were matched for age and socioeconomic status. A compensation of 2400 INR (approximately 30 Euros) was given to the participants for taking part in the research. Three participants did not complete Experiment 1 because they elected to stop taking part in the test battery at an earlier stage.
Common illusions were chosen as stimuli for this experiment. Some were illusions that were used in study done by Luria (1976). For this study, the stimuli were divided into two categories based on the type of illusion. Shape illusions included visual illusions based on the length or size judgment of the objects presented (e.g. the Müller-Lyer and the Poggendorff illusions, see Appendix for full list of stimuli), we included sixteen stimuli of this type. The second category consisted of illusions based on color (e.g. the checkerboard illusion, see Appendix for full list of stimuli), we included six stimuli of this type.
Control images for each of the color and shape illusions were created by removing the factors causing the illusions. For instance, in the Müller-Lyer illusions the two lines of equal length were shown without the arrows at the end of the lines that typically cause the illusion of length of these lines. For three shape illusions, no control stimuli were included because no appropriate control images could be constructed.
All materials, data and scripts are freely available in an OSF repository: https://osf.io/p38ny/
Design and procedure
Stimuli were presented in random order on the screen of a laptop computer. Each stimulus picture was followed by a question prompt regarding the illusion/control image that the participants saw. These were presented orally (prerecorded by a native speaker of Tamil) to ensure it was comprehensible for all participants, including the illiterates. Question prompts were framed as yes or no questions, which participants could answer by pressing color-coded buttons on the laptop keyboard (green for “yes” and red for “no”). For instance, for the Müller-Lyer Illusion the question prompt was: are the two horizontal lines of the same length? Care was taken to make sure that the control image and illusion image had comparable question prompts.
Response accuracy across the two types of illusions (color and shape) and trials (illusion and control) for the three literacy groups is reported in Fig. 1 (top row). Response time is reported in Fig. 2 (top row).
The response accuracies plotted in Fig. 1 are the raw, uncorrected participant responses. However, these raw accuracies misrepresent the behavioral data we recorded in an important way: in the illusion condition, we score participant responses as “correct” if they correspond with the ground truth (i.e. if a given trial features a visual illusion that makes two line segments of equal length seem unequal in length, we score the “equal” response as correct, and the “unequal” response as incorrect). This means that if a participant perceives the illusion (as intended), their response will be scored as incorrect, despite being the modal (or “normal”) response. This choice might seem natural (or simply arbitrary) at first glance, but as a consequence, deviation from the norm (the modal response, i.e. perceiving the illusion) looks like improved accuracy. This is sort of technically correct, but crucially it is not conceptually correct: an illiterate participant might perform closer to chance on both illusion and control trials because they find the testing setting distracting (or find interacting with a laptop computer difficult), but while this will look like decreased accuracy in control trials, it will look like improved accuracy in illusion trials (the effect reported by Luria, 1976). This is an illusory effect, the participant is simply performing closer to chance on both types of trials, but in order to correct the misperception (and to allow us to model participant variability correctly in our statistical modeling) we need to flip the scoring on the illusion trials. In effect, we will score perceiving the illusion (and therefore giving the technically incorrect but modal or “normal” response) as correct, rather than incorrect. These rectified accuracy scores are presented in Fig. 3 (but note that this only affects the illusion condition, not the control trials). Correcting this misrepresentation allows us to better interpret participant responses, both when visually presenting the aggregate data and when modeling them statistically.
For both the accuracy and response time data we specified Bayesian (generalized) linear mixed effects models using the BAMBI package (Yarkoni & Westfall, 2016), we used ADVI (Kucukelbir et al., 2017) to initialize a NUTS sampler (Hoffman & Gelman, 2014), both implemented in PyMC3 (Salvatier et al., 2016), to draw 6000 MCMC samples across three chains (after first drawing and discarding 6000 warmup samples) for each of these models and visualized the results using Arviz (Kumar et al., 2019).
The accuracy model treats each trial as a Bernoulli trial, with probability of success predicted by illusion condition (illusion versus control), illusion type (color versus shape), reading score, and all possible interactions between these three predictors. We specify a complex, but not quite maximal random effects structure, informed by theoretically plausible sources of random variability. Note that the inclusion of item-level random effects allows for unbiased estimation of effect sizes despite unbalanced numbers of color and shape illusions and the absence of control stimuli for three of the illusions. (Excluding the three illusions for which no control stimuli were presented—rather than accounting for them with random effects—did not meaningfully alter any of the effects estimates.)
95% compatibility intervals (also known as credible intervals, highest density intervals, etc.) of the posterior estimates are reported as a forest plot in Fig. 4.
We interpret these coefficient estimates as follows:
There is a positive intercept, so overall participants answered above chance across all trials.
Illusion condition: the mean estimate is around -0.6, but with a wide compatibility interval. This reflects that illusion questions are generally answered closer to chance than control questions (meaning participants overall perceive illusions, but they are more likely to “not perceive” an illusion than to incorrectly answer a control trial).
Illusion type: mean estimate is close to zero with a wide compatibility interval. This means there was no meaningful difference in base rate correct responses between shape and color trials.
Reading score: the mean estimate is not huge, at around 0.4, but with a narrow interval, reflecting strong evidence that there is a small positive effect of reading score on the overall chance of answering correctly, as expected.
Illusion condition: illusion type:
An essentially zero mean estimate and wide interval indicate that the difference between illusion and control trials does not meaningfully differ between shape and color trials.
Illusion condition: reading score:
This is the key coefficient, because it indicates whether reading score affects the difference between illusion and control trials, i.e. that the mean estimate is around -0.3 and the interval overlaps with zero indicates that while reading score does improve the overall chance of answering correctly (see point 4 in this list) we cannot be sure that it improves that chance specifically in the illusion condition.
Illusion type: reading score:
This reflects that there was no meaningful effect (mean effect estimate around -0.25, with compatibility interval overlapping zero) of literacy on the difference between shape and color trials.
Illusion condition: illusion type: reading score:
This three way interaction is estimated at zero, with large uncertainty. The effect of literacy does not appear to vary by both illusion versus control and color versus shape illusion conditions in any meaningful way.
The response time model is a Gaussian model, predicting the log-transformed response times from the same predictors used in the accuracy model. Random effects structure is identical to the structure used in the accuracy model. None of the claims in the prior literature are about the speed with which illiterates perceive (or do not perceive) illusions, so this model mostly serves to confirm (or potentially complicate) the conclusions from our accuracy model. 95% compatibility intervals of the posterior estimates are reported as a forest plot in Fig. 5.
The results are generally consistent with the estimates from the accuracy model. Illusions take a little longer to recognize than controls, which is unsurprising and consistent with them being answered closer to chance level (see accuracy model). In contrast to the accuracy model, higher reading scores are not associated with shorter RTs overall, meaning that if illiterate participants found the task harder to perform (as we concluded from their answering more accurately, per the accuracy model) this did not result in them answering any slower. The interactions are all close to zero, broadly consistent with the accuracy model.
Experiment 1 was part of a larger test battery that was administered to Tamil participants of varying literacy status. Results obtained in other experiments in the test battery warranted replication, and so for reasons largely unrelated to the experiments reported here, additional participants were recruited to perform certain parts of the test battery, including the task reported here as Experiment 1. This follow-up allowed us to collect additional evidence to examine whether the interaction between literacy and visual illusion condition (the mean estimated effect size of which was around -0.3 in Experiment 1, although the compatibility interval was fairly wide and included zero) is non-zero, as claimed by Luria (1976). For the sake of clarity and transparency, we report this follow-up group of participants here as a separate experiment (Experiment 2), but also conduct a statistical analysis using pooled data from both experiments.