Abstract
An astute reader will note that the title is a direct quote from Tukey’s seminal text Exploratory Data Analysis (p. 56, Tukey, 1977). In this book, Tukey lays out the foundation for leveraging visual tools to facilitate exploration and understanding of data and statistical models of data. As cognitive science moves toward establishing robust practices for modeling, building on the foundational recommendations by Lee et al. (Computational Brain and Behavior, 2019), I argue that we must make visualization one of our cornerstones.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The IEEE VIS communities have developed multiple languages and libraries that may be useful to cognitive modelers, for ambitious learners, check out languages such as Vega and Vega-Lite (https://vega.github.io/vega/, Satyanarayan et al. 2015, 2016) or the JavaScript library D3 (https://d3js.org/, Bostock et al. 2011).
I note, too, that there are subgroups of visualization researchers emphasizing the creating of visualization tools specifically for behavioral data that may be useful for cognitive modelers to obtain and use without reinventing them, for example, there is a dedicated community developing techniques and tools for visualizing eye tracking data, ETVIS (https://etra.acm.org/2019/etvis.html)
References
Birnbaum, M.H. (1974). Reply to the devil’s advocates: don’t confound model testing and measurement. Psychological Bulletin, 81(11), 854.
Bostock, M., Ogievetsky, V., Heer, J. (2011). D 3 data-driven documents. IEEE Transactions on Visualization and Computer Graphics, 17(12), 2301–2309.
Dasgupta, A., & Kosara, R. (2010). Pargnostics: screen-space metrics for parallel coordinates. IEEE Transactions on Visualization and Computer Graphics, 16(6), 1017–1026.
Draper, G.M., Livnat, Y., Riesenfeld, R.F. (2009). A survey of radial methods for information visualization. IEEE Transactions on Visualization and Computer Graphics, 15(5), 759–776.
Kafadar, K. (2011). Exploratory data analysis: statistically sound “data mining” for large,complex data sets. Workshop on Computational Methods for Complex Data Analysis, Dayton, Ohio.
Lee, M.D., Criss, A.H., Devezer, B., Donkin, C., Etz, A., Leite, F.P., Vandekerckhove, J. (2019). Robust modeling in cognitive science. Computational Brain & Behavior, https://doi.org/10.1007/s42113-019-00029-y.
McClelland, J.L. (2009). The place of modeling in cognitive science. Topics in Cognitive Science, 1(1), 11–38.
Navarro, D.J., Myung, J., Pitt, M., Woojae, K. (2003). Global model analysis by landscaping. In R. Alterman, & D. Kirsh (Eds.) Proceedings of the twenty-fifth annual conference of the cognitive science society, pp. 851–856. Routledge.
Navarro, D.J., Pitt, M.A., Myung, I.J. (2004). Assessing the distinguish ability of models and the informativeness of data. Cognitive Psychology, 49(1), 47–84.
Pitt, M.A., Kim, W., Navarro, D.J., Myung, J.I. (2006). Global model analysis by parameter space partitioning. Psychological Review, 113(1), 57–83.
Ratcliff, R., & Smith, P.L. (2004). A comparison of sequential sampling models for two-choice reaction time. Psychological Review, 111(2), 333–367.
Satyanarayan, A., Moritz, D., Wongsuphasawat, K., Heer, J. (2016). Vega-lite: a grammar of interactive graphics. IEEE Transactions on Visualization and Computer Graphics, 23(1), 341–350.
Satyanarayan, A., Russell, R., Hoffswell, J., Heer, J. (2015). Reactive vega: a streaming dataflow architecture for declarative interactive visualization. IEEE Transactions on Visualization and Computer Graphics, 22 (1), 659–668.
Steegen, S., Tuerlinckx, F., Vanpaemel, W. (2017). Using parameter space partitioning to evaluate a model’s qualitative fit. Psychonomic Bulletin & Review, 24(2), 617–631.
Tukey, J.W. (1977). Exploratory data analysis. Reading: Addison-Wesley Publishing Company.
Wilkinson, L. (2005). The grammar of graphics. Berlin: Springer.
Acknowledgments
The author thanks G. Gunzelmann and two anonymous reviewers for their helpful comments on this discussion.
Funding
This work was financially supported by a seedling grant from the 711th Human Performance Wing Chief Scientist.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Disclaimer
The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Government.
Rights and permissions
About this article
Cite this article
Blaha, L.M. We Have Not Looked at Our Results Until We Have Displayed Them Effectively: a Comment on Robust Modeling in Cognitive Science. Comput Brain Behav 2, 247–250 (2019). https://doi.org/10.1007/s42113-019-00059-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42113-019-00059-6