Comment on “Quantifying the informational value of classification images”: A miscomputation of the infoVal metric
Brinkman et al. (2019) recently introduced an innovative metric—infoVal—to assess the informational value of classification images (CIs) relative to a random distribution. Although this measure constitutes a valuable tool to distinguish random from nonrandom CIs, we identified two noteworthy discrepancies between the mathematical formalization of the infoVal metric and the authors’ computation. Specifically, the computation was based on the one norm instead of the Euclidean norm, and the k constant was omitted in the denominator of the ratio that produces infoVal. Accordingly, the simulations and experimental results reported by Brinkman et al. do not build on the correct infoVal computation but on a biased index. Importantly, this discrepancy in the computation affects the statistical power and Type I and error rate of the metric. Here we clarify the nature of the discrepancies in the computation and run Brinkman et al.’s Simulation 1 anew with the correct values, to illustrate their consequences. Overall, we found that relying on the miscomputed infoVal metric can lead to misguided conclusions, and we urge researchers to use the correct values.
KeywordsinfoVal Reverse correlation Classification images rcicr package R
This work was supported by the Fonds de la Recherche Scientifique (FNRS), grant 1.A393.17, awarded to M.S., and by FNRS grant 1.B347.19, awarded to M.R.
Open Practices Statement
The data, R code, and material are publicly available on Open Science Framework (https://osf.io/fbduk/?view_only=e3c04e549f844facb109200dcba7a0db).
- Dotsch, R. (2017). rcicr: Reverse-correlation image-classification toolbox (R package version 0.4.0). Retrieved from https://rdrr.io/cran/rcicr/