Acuity VEP: improved with machine learning

Abstract

Purpose

Acuity-VEP approaches basically all use the information obtained across a number of check sizes (or spatial frequencies) to derive a measure of acuity. Amplitude is always used, sometimes combined with phase or a noise measure. In our approach, we employ steady-state brief-onset low-contrast checkerboard stimulation and obtain amplitude and significance for six different check sizes, yielding 12 numbers. The rule-based “heuristic algorithm” (Bach et al. in Br J Ophthalmol 92:396–403, 2008. https://doi.org/10.1136/bjo.2007.130245) is successful in over 95% with a limit of agreement (LoA) of ± 0.3LogMAR between behavioral and objective acuity for 109 cases. We here aimed to test whether machine learning techniques with this relatively small dataset could achieve a similar LoA.

Methods

Given recent advances in machine learning (ML), we applied a wide class of ML algorithms to this dataset. This was done within the “caret” framework of R using altogether 89 methods, of which rule-based and multiple regression approaches performed best. For cross-validation, using a jackknife (leave-one-out) approach, we predicted each case based on an ML model having been trained on all remaining 108 cases.

Results

The ML approach predicted visual acuity well across many different types of ML algorithms. Using amplitude values only (discarding the p values) improved the outcome. Nearly half of the tested ML algorithms achieved an LoA better than the heuristic algorithm; several “Random Forest”- or “multiple regression”-type algorithms achieved an LoA of below ± 0.3. In the cases where the heuristic approach failed, acuity was predicted successfully. We then applied the ML model trained with the Bach et al. [1] dataset to a new dataset from 2018 (78 cases) and found both for the heuristic algorithm and for the ML approach an LoA of ± 0.259, a nearly one-line improvement.

Conclusions

The ML approach appears to be a useful alternative to rule-based analysis of acuity-VEP data. The achieved accuracy is comparable or better (in no case the ML-based acuity differed more than ± 0.29 LogMAR from behavioral acuity), and testability is higher, nearly 100%. Possible pitfalls are examined.

Appendix

See Table 1.