Psychonomic Bulletin & Review

, Volume 25, Issue 1, pp 472–481 | Cite as

A critical reexamination of doing arithmetic nonconsciously



A recent study claimed to have obtained evidence that participants can solve invisible multistep arithmetic equations (Sklar et al., 2012). The authors used a priming paradigm in which reaction times to targets congruent with the equation’s solution were responded to faster compared with incongruent ones. We critically reanalyzed the data set of Sklar et al. and show that the claims being made in the article are not fully supported by the alternative analyses that we applied. A Bayesian reanalysis of the data accounting for the random variability of the target stimuli in addition to the subjects shows that the evidence for priming effects is less strong than initially claimed. That is, although Bayes factors revealed evidence for the presence of a priming effect, it was generally weak. Second, the claim that unconscious arithmetic occurs for subtraction but not for addition is not supported when the critical interaction is tested. Third, the data do not show well-established features of numerosity priming as derived from V-shaped response time curves for prime-target distances. Fourth, we show that it is impossible to classify reaction times as resulting from congruent or incongruent prime-target relationships, which should be expected if their results imply that participants genuinely solve the equations on each trial. We conclude that the claims being made in the original article are not fully supported by the analyses that we apply. Together with a recent failure to replicate the original results and a critique of the analysis based on regression to the mean, we argue that the current evidence for unconscious arithmetic is inconclusive. We argue that strong claims require strong evidence and stress that cumulative research strategies are needed to provide such evidence.


Interocular suppression Unconscious processing Reproducibility 



P.M. was supported by the Research Fund Flanders (FWO Vlaanderen) through a doctoral fellowship. GH is supported by the German Research Foundation (grant HE 6244/1-2).

Author contributions

P.M. and G.H. designed research; P.M. analyzed data; P.M. and G.H. wrote the paper.


  1. Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4), 390–412. doi: 10.1016/j.jml.2007.12.005 CrossRefGoogle Scholar
  2. Bachmann, T., Breitmeyer, B., & Ogmen, H. (2007). The experimental phenomena of consciousness: A brief dictionary. Oxford: Oxford University Press.Google Scholar
  3. Barbot, A., & Kouider, S. (2011). Longer is not better: Nonconscious overstimulation reverses priming influences under interocular suppression. Attention, Perception & Psychophysics, 74, 174–184. doi: 10.3758/s13414-011-0226-3 CrossRefGoogle Scholar
  4. Breitmeyer, B. G. (2015). Psychophysical “blinding” methods reveal a functional hierarchy of unconscious visual processing. Consciousness and Cognition, 35, 234–250. doi: 10.1016/j.concog.2015.01.012 CrossRefPubMedGoogle Scholar
  5. Clark, H. H. (1973). The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. Journal of Verbal Learning and Verbal Behavior, 12(4), 335–359. doi: 10.1016/S0022-5371(73)80014-3 CrossRefGoogle Scholar
  6. Core Team, R. (2014). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. Retrieved from Google Scholar
  7. Dehaene, S., Molko, N., Cohen, L., & Wilson, A. J. (2004). Arithmetic and the brain. Current Opinion in Neurobiology, 14(2), 218–224. doi: 10.1016/j.conb.2004.03.008
  8. Dehaene, S., Charles, L., King, J.-R., & Marti, S. (2014). Toward a computational theory of conscious processing. Current Opinion in Neurobiology, 25, 76–84. doi: 10.1016/j.conb.2013.12.005 CrossRefPubMedGoogle Scholar
  9. Doyen, S., Klein, O., Simons, D. J., & Cleeremans, A. (2014). On the other side of the mirror: Priming in cognitive and social psychology. Social Cognition, 32(Supplement), 12–32. doi: 10.1521/soco.2014.32.supp.12 CrossRefGoogle Scholar
  10. Etz, A., & Vandekerckhove, J. (2016). A Bayesian perspective on the reproducibility project: Psychology. PLoS ONE, 11(2), e0149794. doi: 10.1371/journal.pone.0149794 CrossRefPubMedPubMedCentralGoogle Scholar
  11. Fogelson, S. V., Kohler, P. J., Miller, K. J., Granger, R., & Tse, P. U. (2014). Unconscious neural processing differs with method used to render stimuli invisible. Frontiers in Psychology, 5, 601. doi: 10.3389/fpsyg.2014.00601 CrossRefPubMedPubMedCentralGoogle Scholar
  12. Franz, V. H., & von Luxburg, U. (2014). Unconscious lie detection as an example of a widespread fallacy in the neurosciences. arXiv:1407.4240. Retrieved from
  13. Franz, V. H., & von Luxburg, U. (2015). No evidence for unconscious lie detection: A significant difference does not imply accurate classification. Psychological Science. doi: 10.1177/0956797615597333 PubMedGoogle Scholar
  14. Gelman, A., & Stern, H. (2006). The difference between “significant” and “not significant” is not itself statistically significant. The American Statistician, 60(4), 328–331. doi: 10.1198/000313006X152649 CrossRefGoogle Scholar
  15. Goodman, S. N., Fanelli, D., & Ioannidis, J. P. A. (2016). What does research reproducibility mean? Science Translational Medicine, 8(341). doi: 10.1126/scitranslmed.aaf5027
  16. Hassin, R. R. (2013). Yes it can: On the functional abilities of the human unconscious. Perspectives on Psychological Science, 8(2), 195–207. doi: 10.1177/1745691612460684 CrossRefPubMedGoogle Scholar
  17. Hedger, N., Adams, W. J., & Garner, M. (2015). Fearful faces have a sensory advantage in the competition for awareness. Journal of Experimental Psychology: Human Perception and Performance. doi: 10.1037/xhp0000127 Google Scholar
  18. Hesselmann, G., & Knops, A. (2014). No conclusive evidence for numerical priming under interocular suppression. Psychological Science. doi: 10.1177/0956797614548876
  19. Hesselmann, G., & Moors, P. (2015). Definitely maybe: Can unconscious processes perform the same functions as conscious processes?. Frontiers in Psychology, 6(584). doi: 10.3389/fpsyg.2015.00584
  20. Hesselmann, G., Darcy, N., Sterzer, P., & Knops, A. (2015). Exploring the boundary conditions of unconscious numerical priming effects with continuous flash suppression. Consciousness and Cognition, 31, 60–72. doi: 10.1016/j.concog.2014.10.009 CrossRefPubMedGoogle Scholar
  21. Ioannidis, J. P. A., Allison, D. B., Ball, C. A., Coulibaly, I., Cui, X., Culhane, A. C., & van Noort, V. (2009). Repeatability of published microarray gene expression analyses. Nature Genetics, 41(2), 149–155. doi: 10.1038/ng.295 CrossRefPubMedGoogle Scholar
  22. Iqbal, S. A., Wallach, J. D., Khoury, M. J., Schully, S. D., & Ioannidis, J. P. A. (2016). Reproducible research practices and transparency across the biomedical literature. PLoS Biology, 14(1), e1002333. doi: 10.1371/journal.pbio.1002333 CrossRefPubMedPubMedCentralGoogle Scholar
  23. Judd, C. M., Westfall, J., & Kenny, D. A. (2012). Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem. Journal of Personality and Social Psychology, 103(1), 54–69. doi: 10.1037/a0028347 CrossRefPubMedGoogle Scholar
  24. Karpinski, A., Yale, M., & Briggs, J. C. (2016). Unconscious arithmetic processing: A direct replication. European Journal of Social Psychology. doi: 10.1002/ejsp.2175 Google Scholar
  25. Kim, C.-Y., & Blake, R. (2005). Psychophysical magic: Rendering the visible “invisible.”. Trends in Cognitive Sciences, 9(8), 381–388. doi: 10.1016/j.tics.2005.06.012 CrossRefPubMedGoogle Scholar
  26. Knops, A. (2016). Probing the neural correlates of number processing. The Neuroscientist. doi: 10.1177/1073858416650153 PubMedGoogle Scholar
  27. Koch, C., Massimini, M., Boly, M., & Tononi, G. (2016). Neural correlates of consciousness: Progress and problems. Nature Reviews Neuroscience, 17(5), 307–321. doi: 10.1038/nrn.2016.22 CrossRefPubMedGoogle Scholar
  28. Leek, J. T., & Jager, L. R. (2016). Is most published research really false? bioRxiv, 50575. doi: 10.1101/050575
  29. Ludwig, K., & Hesselmann, G. (2015). Weighing the evidence for a dorsal processing bias under continuous flash suppression. Consciousness and Cognition, 35, 251–259. doi: 10.1016/j.concog.2014.12.010 CrossRefPubMedGoogle Scholar
  30. Meyer, D. E., & Schvaneveldt, R. W. (1971). Facilitation in recognizing pairs of words: Evidence of a dependence between retrieval operations. Journal of Experimental Psychology, 90(2), 227–234.CrossRefPubMedGoogle Scholar
  31. Moors, P., Boelens, D., van Overwalle, J., & Wagemans, J. (2016). Scene integration without awareness: No conclusive evidence for processing scene congruency during continuous flash suppression. Psychological Science, 27(7), 945–956. doi: 10.1177/0956797616642525 CrossRefPubMedGoogle Scholar
  32. Moors, P., Wagemans, J., & de-Wit, L. (2016). Faces in commonly experienced configurations enter awareness faster due to their curvature relative to fixation. PeerJ, 4. doi: 10.7717/peerj.1565
  33. Morey, R. D., Rouder, J. N., Love, J., & Marwick, B. (2015). BayesFactor: 0.9.12-2 CRAN [Zenodo]. Retrieved from doi: 10.5281/zenodo.31202
  34. Morey, R. D., Romeijn, J.-W., & Rouder, J. N. (2016). The philosophy of Bayes factors and the quantification of statistical evidence. Journal of Mathematical Psychology, 72, 6–18. doi: 10.1016/ CrossRefGoogle Scholar
  35. Naccache, L. (2009). Priming. In T. Bayne, A. Cleeremans, & P. Wilken (Eds.), The Oxford companion to consciousness (pp. 533–536). Oxford: Oxford University Press.Google Scholar
  36. Nieuwenhuis, S., Forstmann, B. U., & Wagenmakers, E.-J. (2011). Erroneous analyses of interactions in neuroscience: A problem of significance. Nature Neuroscience, 14(9), 1105–1107. doi: 10.1038/nn.2886 CrossRefPubMedGoogle Scholar
  37. Nuijten, M. B., Hartgerink, C. H. J., van Assen, M. A. L., Epskamp, S., & Wicherts, J. M. (2015). The prevalence of statistical reporting errors in psychology (1985-2013). Behavior Research Methods. doi: 10.3758/s13428-015-0664-2 PubMedPubMedCentralGoogle Scholar
  38. Phillips, N. (2016). yarrr: A companion to the e-book YaRrr!: The Pirate’s Guide to R. R package version 0.0.5. Retrieved from
  39. Reynvoet, B., Brysbaert, M., & Fias, W. (2002). Semantic priming in number naming. The Quarterly Journal of Experimental Psychology, 55(4), 1127–1139. doi: 10.1080/02724980244000116 CrossRefPubMedGoogle Scholar
  40. Ric, F., & Muller, D. (2012). Unconscious addition: When we unconsciously initiate and follow arithmetic rules. Journal of Experimental Psychology: General, 141(2), 222–226. doi: 10.1037/a0024608 CrossRefGoogle Scholar
  41. Roggeman, C., Verguts, T., & Fias, W. (2007). Priming reveals differential coding of symbolic and non-symbolic quantities. Cognition, 105(2), 380–394. doi: 10.1016/j.cognition.2006.10.004 CrossRefPubMedGoogle Scholar
  42. Rouder, J. N., Engelhardt, C. R., McCabe, S., & Morey, R. D. (2016). Model comparison in ANOVA. Psychonomic Bulletin & Review, 23(6), 1779–1786. doi: 10.3758/s13423-016-1026-5 CrossRefGoogle Scholar
  43. RStudio Team. (2015). RStudio: Integrated development environment for R (Version 0.99.441). Boston: RStudio, Inc.Google Scholar
  44. Shanks, D. R. (2016). Regressive research: The pitfalls of post hoc data selection in the study of unconscious mental processes. Psychonomic Bulletin & Review, in press.Google Scholar
  45. Singmann, H., Bolker, B., Westfall, J., & Aust, F. (2016). afex: Analysis of Factorial Experiments. R package version 0.16-1. Retrieved from
  46. Sklar, A. Y., Levy, N., Goldstein, A., Mandel, R., Maril, A., & Hassin, R. R. (2012). Reading and doing arithmetic nonconsciously. Proceedings of the National Academy of Sciences, 109(48), 19614–19619. doi: 10.1073/pnas.1211645109 CrossRefGoogle Scholar
  47. Soto, D., & Silvanto, J. (2014). Reappraising the relationship between working memory and conscious awareness. Trends in Cognitive Sciences, 18(10), 520–525. 10.1016/j.tics.2014.06.005.CrossRefPubMedGoogle Scholar
  48. Stein, T., & Sterzer, P. (2014). Unconscious processing under interocular suppression: Getting the right measure. Frontiers in Psychology, 5, 387. doi: 10.3389/fpsyg.2014.00387
  49. Stein, T., Kaiser, D., & Peelen, M. V. (2015). Interobject grouping facilitates visual awareness. Journal of Vision, 15(8), 10. doi: 10.1167/15.8.10 CrossRefPubMedGoogle Scholar
  50. Sterzer, P., Stein, T., Ludwig, K., Rothkirch, M., & Hesselmann, G. (2014). Neural processing of visual information under interocular suppression: A critical review. Frontiers in Psychology, 5, 453. doi: 10.3389/fpsyg.2014.00453 CrossRefPubMedPubMedCentralGoogle Scholar
  51. Tsuchiya, N., & Koch, C. (2005). Continuous flash suppression reduces negative afterimages. Nature Neuroscience, 8(8), 1096–1101. doi: 10.1038/nn1500 CrossRefPubMedGoogle Scholar
  52. Tsuchiya, N., Koch, C., Gilroy, L. A., & Blake, R. (2006). Depth of interocular suppression associated with continuous flash suppression, flash suppression, and binocular rivalry. Journal of Vision, 6(10), 1068–1078. doi: 10.1167/6.10.6 CrossRefPubMedGoogle Scholar
  53. Van Opstal, F., Gevers, W., De Moor, W., & Verguts, T. (2008). Dissecting the symbolic distance effect: Comparison and priming effects in numerical and nonnumerical orders. Psychonomic Bulletin & Review, 15(2), 419–425.CrossRefGoogle Scholar
  54. Wolsiefer, K., Westfall, J., & Judd, C. M. (2016). Modeling stimulus variation in three common implicit attitude tasks. Behavior Research Methods Google Scholar
  55. Yang, E., Brascamp, J., Kang, M.-S., & Blake, R. (2014). On the use of continuous flash suppression for the study of visual processing outside of awareness. Frontiers in Psychology, 5, 724. doi: 10.3389/fpsyg.2014.00724 PubMedPubMedCentralGoogle Scholar
  56. Yuval-Greenberg, S., & Heeger, D. J. (2013). Continuous flash suppression modulates cortical activity in early visual cortex. The Journal of Neuroscience, 33(23), 9635–9643. doi: 10.1523/JNEUROSCI.4612-12.2013 CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2017

Authors and Affiliations

  1. 1.Laboratory of Experimental Psychology, Department of Brain & CognitionUniversity of Leuven (KU Leuven)LeuvenBelgium
  2. 2.Visual Perception Laboratory, Department of Psychiatry and PsychotherapyCharité-UniversitätsmedizinBerlinGermany

Personalised recommendations