Advertisement

Comment on “Quantifying the informational value of classification images”: A miscomputation of the infoVal metric

  • Mathias SchmitzEmail author
  • Marine Rougier
  • Vincent Yzerbyt
Article

Abstract

Brinkman et al. (2019) recently introduced an innovative metric—infoVal—to assess the informational value of classification images (CIs) relative to a random distribution. Although this measure constitutes a valuable tool to distinguish random from nonrandom CIs, we identified two noteworthy discrepancies between the mathematical formalization of the infoVal metric and the authors’ computation. Specifically, the computation was based on the one norm instead of the Euclidean norm, and the k constant was omitted in the denominator of the ratio that produces infoVal. Accordingly, the simulations and experimental results reported by Brinkman et al. do not build on the correct infoVal computation but on a biased index. Importantly, this discrepancy in the computation affects the statistical power and Type I and error rate of the metric. Here we clarify the nature of the discrepancies in the computation and run Brinkman et al.’s Simulation 1 anew with the correct values, to illustrate their consequences. Overall, we found that relying on the miscomputed infoVal metric can lead to misguided conclusions, and we urge researchers to use the correct values.

Keywords

infoVal Reverse correlation Classification images rcicr package 

Notes

Acknowledgements

This work was supported by the Fonds de la Recherche Scientifique (FNRS), grant 1.A393.17, awarded to M.S., and by FNRS grant 1.B347.19, awarded to M.R.

Open Practices Statement

The data, R code, and material are publicly available on Open Science Framework (https://osf.io/fbduk/?view_only=e3c04e549f844facb109200dcba7a0db).

Supplementary material

13428_2019_1295_MOESM1_ESM.docx (1 mb)
ESM 1 (DOCX 219 kb)

References

  1. Brinkman, L., Goffin, S., van de Schoot, R., van Haren, N. E. M., Dotsch, R., & Aarts, H. (2019). Quantifying the informational value of classification images. Behavior Research Methods. Advance online publication. doi: https://doi.org/10.3758/s13428-019-01232-2 CrossRefGoogle Scholar
  2. Dotsch, R. (2017). rcicr: Reverse-correlation image-classification toolbox (R package version 0.4.0). Retrieved from https://rdrr.io/cran/rcicr/
  3. Dotsch, R., Wigboldus, D. H. J., Langner, O., & van Knippenberg, A. (2008). Ethnic out-group faces are biased in the prejudiced mind. Psychological Science, 19, 978–980. doi: https://doi.org/10.1111/j.1467-9280.2008.02186.x CrossRefPubMedGoogle Scholar
  4. Mangini, M., & Biederman, I. (2004). Making the ineffable explicit: Estimating the information employed for face classifications. Cognitive Science, 28, 209–226. doi: https://doi.org/10.1016/j.cogsci.2003.11.004 CrossRefGoogle Scholar
  5. Microsoft & R Core Team. (2017). Microsoft R Open (Version 3.5.3). Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://mran.microsoft.com/ Google Scholar
  6. R Core Team. (2015). R: A language and environment for statistical computing (Version 3.5.3). Vienna, Austria: R Foundation for Statistical Computing. Retrieved from www.R-project.org Google Scholar
  7. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22 1359–1366. doi: https://doi.org/10.1177/0956797611417632 CrossRefPubMedGoogle Scholar

Copyright information

© The Psychonomic Society, Inc. 2019

Authors and Affiliations

  1. 1.Université catholique de LouvainLouvain-la-NeuveBelgium

Personalised recommendations