Abstract
Signal detection theory (SDT; Tanner & Swets in Psychological Review 61:401–409, 1954) is a dominant modeling framework used for evaluating the accuracy of diagnostic systems that seek to distinguish signal from noise in psychology. Although the use of response time data in psychometric models has increased in recent years, the incorporation of response time data into SDT models remains a relatively underexplored approach to distinguishing signal from noise. Functional response time effects are hypothesized in SDT models, based on findings from other related psychometric models with response time data. In this study, an SDT model is extended to incorporate functional response time effects using smooth functions and to include all sources of variability in SDT model parameters across trials, participants, and items in the experimental data. The extended SDT model with smooth functions is formulated as a generalized linear mixed-effects model and implemented in the gamm4 R package. The extended model is illustrated using recognition memory data to understand how conversational language is remembered. Accuracy of parameter estimates and the importance of modeling variability in detecting the experimental condition effects and functional response time effects are shown in conditions similar to the empirical data set via a simulation study. In addition, the type 1 error rate of the test for a smooth function of response time is evaluated.
This is a preview of subscription content, log in via an institution to check access.
Notes
We use the term functional to refer to the intrinsic structure of the data rather than their explicit form (Ramsay & Silverman, 2005, p. 38).
The 28 image groups are: baby, backpack, banana, belt, bird, boot, box, chair, desk, dog, flag, grapes, hair, hat, jacket, juice, pants, paper, pie, pig, ring, shirt, shoe, skirt, sock, swords, tree, watch.
There are three identifiable parameterizations of smooth functions of \(RT_{lji}\): (a) the mean level of \(\texttt{isold}_{lji}\) and a smooth function of \(RT_{lji}\) for each level of \(\texttt{isold}_{lji}\), (b) a smooth function of \(RT_{lji}\) and a smooth function for the differences in the effect of response time by \(\texttt{isold}_{lji}\), and (c) the mean level of \(\texttt{isold}_{lji}\), a smooth function of \(RT_{lji}\), and a smooth function for the differences in the effect of response time by \(\texttt{isold}_{lji}\). In this current study, we chose the third parameterization to estimate the fixed d parameter (\(\mu ^{d}\)), a smooth function of \(RT_{lji}\) for the \(c_{lji}\) parameter, and a smooth function of \(RT_{lji}\) for the \(d_{lji}\) parameter.
“0” in the superscripts of \(\mu ^{c0}\) and \(\mu ^{d0}\) indicates “null,” which means that they are estimated without random effects and without experimental condition effects.
With ordered factors in R (e.g., ordered.disOLD= as.ordered(factor.disOLD) for a isold covariate), the interactions are not strictly interactions as in the analysis of variance (ANOVA) but as regression coefficients of the covariates.
Summations start with 2 because of the identification constraints.
References
Agresti, A. (2002). Categorical data analysis (2nd ed.). Wiley.
Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412. https://doi.org/10.1016/j.jml.2007.12.005
Baayen, R. H., Vasishth, S., Kliegl, R., & Bates, D. (2017). The cave of shadows: Addressing the human factor with generalized additive mixed models. Journal of Memory and Language, 94, 206–234. https://doi.org/10.1016/j.jml.2016.11.006
Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67, 1–48.
Benjamin, A. S. (2001). On the dual effects of repetition on false recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27, 941–947. https://doi.org/10.1037/0278-7393.27.4.941
Benjamin, A. S. (2013). Where is the criterion noise in recognition? (Almost) everyplace you look: Comment on Kellen, Klauer, and Singmann (2012). Psychological Review, 120, 720–726. https://doi.org/10.1037/a0031911
Benjamin, A. S., Diaz, M., & Wee, S. (2009). Signal detection with criterion noise: Applications to recognition memory. Psychological Review, 116, 84–115. https://doi.org/10.1037/a0014351
Besson, G., Ceccaldi, M., Didic, M., & Barbeau, E. J. (2012). The speed of visual recognition memory. Visual Cognition, 20, 1131–1152. https://doi.org/10.1080/13506285.2012.724034
Bolsinova, M., De Boeck, P., & Tijmstra, J. (2017). Modeling conditional dependence between response time and accuracy. Psychometrika, 82, 1126–1148. https://doi.org/10.1007/s11336-016-9537-6
Bolsinova, M., & Molenaar, D. (2018). Modeling nonlinear conditional dependence between response time and accuracy. Frontiers in Psychology, 9, 1525. https://doi.org/10.3389/fpsyg.2018.01525
Brown, S., & Heathcote, A. (2008). The simplest complete model of choice response time: Linear ballistic accumulation. Cognitive Psychology, 57, 153–178. https://doi.org/10.1016/j.cogpsych.2007.12.002
Chen, H., De Boeck, P., Grady, M., Yang, C.-L., & Waldschmidt, D. (2018). Curvilinear dependency of response accuracy on response time in cognitive tests. Intelligence, 69, 16–23. https://doi.org/10.1016/j.intell.2018.04.001
Cho, S.-J., Partchev, I., & De Boeck, P. (2012). Parameter estimation of multiple item profiles models. British Journal of Mathematical and Statistical Psychology, 65, 438–466. https://doi.org/10.1111/j.2044-8317.2011.02036.x
De Boeck, P., & Jeon, M. (2019). An overview of models for response times and processes in cognitive tests. Frontiers in Psychology, 10, 102. https://doi.org/10.3389/fpsyg.2019.00102
DeCarlo, L. T. (1998). Signal detection theory and generalized linear models. Psychological Methods, 3, 186–205. https://doi.org/10.1037/1082-989X.3.2.186
DeCarlo, L. T. (2010). On the statistical and theoretical basis of signal detection theory and extensions: Unequal variance, random coefficient, and mixture models. Journal of Mathematical Psychology, 54, 304–313. https://doi.org/10.1016/j.jmp.2010.01.001
DeCarlo, L. T. (2011). Signal detection theory with item effects. Journal of Mathematical Psychology, 55, 229–239. https://doi.org/10.1016/j.jmp.2011.01.002
DeCarlo, L. T. (2021). On joining a signal detection choice model with response time models. Journal of Educational Measurement. https://doi.org/10.1111/jedm.12300
Finch, W. H., & Finch, M. H. (2018). A simulation study evaluating the generalized additive model for assessing intervention effects with small samples. Journal of Experimental Education, 86(4), 652–670. https://doi.org/10.1080/00220973.2017.1339010
Frenkel, T. I., Lamy, D., Algom, D., & Bar-Haim, Y. (2009). Individual differences in perceptual sensitivity and response bias in anxiety: Evidence from emotional faces. Cognition and Emotion, 23, 688–700. https://doi.org/10.1080/02699930802076893
Goldhammer, F., Steinwascher, M. A., Kroehne, U., & Naumann, J. (2017). Modeling individual response time effects between and within experimental speed conditions: A GLMM approach for speeded tests. British Journal of Mathematical and Statistical Psychology, 70, 238–256. https://doi.org/10.1111/bmsp.12099
Jaeger, T. F. (2008). Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language, 59, 434–446. https://doi.org/10.1016/j.jml.2007.11.007
James, A. N., Fraundorf, S. H., Lee, E. K., & Watson, D. G. (2018). Individual differences in syntactic processing: Is there evidence for reader-text interactions? Journal of Memory and Language, 102, 155–181. https://doi.org/10.1016/j.jml.2018.05.006
Kang, I., De Boeck, P., & Partchev, I. (2022). A randomness perspective on intelligence processes. Intelligence, 91, 101632. https://doi.org/10.1016/j.intell.2022.101632
Kang, I., De Boeck, P., & Ratcliff, R. (2022). Modeling conditional dependence of response accuracy and response time with the diffusion item response theory model. Psychometrika. https://doi.org/10.1007/s11336-021-09819-5
Krauss, R. M., & Weinheimer, S. (1964). Changes in reference phrases as a function of frequency of usage in social interaction: A preliminary study. Psychonomic Science, 1, 113–114. https://doi.org/10.3758/BF03342817
MacLeod, C. M., Gopie, N., Hourihan, K. L., Neary, K. R., & Ozubko, J. D. (2010). The production effect: Delineation of a phenomenon. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 671–685. https://doi.org/10.1037/a0018785
Marra, G., & Wood, S. N. (2012). Coverage properties of confidence intervals for generalized additive model components. Scandinavian Journal of Statistics, 39(1), 53–74. https://doi.org/10.1111/j.1467-9469.2011.00760.x
Martínez, E., Mollica, F., & Gibson, E. (2022). Poor writing, not specialized concepts, drives processing difficulty in legal language. Cognition, 224, 105070. https://doi.org/10.1016/j.cognition.2022.105070
Parasuraman, R., Masalonis, A. J., & Hancock, P. A. (2000). Fuzzy signal detection: Basic postulates and formulas for analyzing human and machine performance. Human Factors, 42, 636–659. https://doi.org/10.1037/a0019737
R Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria.https://www.R-project.org/.
Ramsay, J. O., & Silverman, B. W. (2005). Functional data analysis (2nd ed.). Springer.
Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59–108. https://doi.org/10.1037/0033-295X.85.2.59
Ratcliff, R., Smith, P. L., & McKoon, G. (2015). Modeling regularities in response time and accuracy data with the diffusion model. Current Directions in Psychological Science, 24, 458–470. https://doi.org/10.1177/0963721415596228
Ratcliff, R., Thapar, A., & McKoon, G. (2006). Aging and individual differences in rapid two-choice decisions. Psychonomic Bulletin & Review, 13, 626–635. https://doi.org/10.3758/bf03193973
Rodríguez, G. (2007). Lecture notes on generalized linear models. Downloaded from https://data.princeton.edu/wws509/notes/
Rouder, J. N., & Lu, J. (2005). An introduction to Bayesian hierarchical models with an application in the theory of signal detection. Psychonomic Bulletin and Review, 12, 573–604. https://doi.org/10.3758/bf03196750
Rouder, J. N., Lu, J., Sun, D., Morey, R., & Naveh-Benjamin, M. (2007). Signal detection models with random participant and item effects. Psychometrika, 72, 621–642. https://doi.org/10.1007/s11336-005-1350-6
Ryskin, R., Benjamin, A. S., Tullis, J., & Brown-Schmidt, S. (2015). Perspective-taking in comprehension, production, and memory: An individual differences approach. Journal of Experimental Psychology: General, 144, 898–915. https://doi.org/10.1037/xge0000093
Schnipke, D. L., & Scrams, D. J. (2002). Exploring issues of examinee behavior: Insights gained from response-time analyses. In C. N. Mills, M. T. Potenza, J. J. Fremer, & W. C. Ward (Eds.), Computer-based testing: Building the foundation for future assessments (pp. 237–266). Lawrence Erlbaum Associates Publishers.
Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464.
Silverman, B. W. (1985). Some aspects of the spline smoothing approach to nonparametric regression curve fitting. Journal of the Royal Statistical Society, Series B, 47(1), 1–21. https://doi.org/10.1111/j.2517-6161.1985.tb01327.x
Slamecka, N. J., & Graf, P. (1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning and Memory, 4, 592–604. https://doi.org/10.1037/0278-7393.4.6.592
Tanner, W. P., & Swets, J. A. (1954). A decision-making theory of visual detection. Psychological Review, 61, 401–409. https://doi.org/10.1037/h0058700
Vaida, F., & Blanchard, S. (2005). Conditional Akaike information for mixed-effects models. Biometrika, 92, 351–370. https://doi.org/10.1093/biomet/92.2.351
van der Linden, W. J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72, 287–308. https://doi.org/10.1007/s11336-006-1478-z
van der Linden, W. J. (2009). Conceptual issues in response-time modeling. Journal of Educational Measurement, 46, 247–272. https://doi.org/10.1111/j.1745-3984.2009.00080.x
Wickelgren, W. A. (1972). Trace resistance and the decay of long-term memory. Journal of Mathematical Psychology, 9, 418–455. https://doi.org/10.1016/0022-2496(72)90015-6
Wickelgren, W. A. (1977). Speed-accuracy tradeoff and information processing dynamics. Acta Psychologica, 41, 67–85.
Wieling, M. (2018). Analyzing dynamic phonetic data using generalized additive mixed modeling: A tutorial focusing on articulatory differences between L1 and L2 speakers of English. Journal of Phonetics, 70, 86–116. https://doi.org/10.1016/j.wocn.2018.03.002
Wood, S. N. (2019). Package ‘mgcv’: Mixed GAM computation vehicle with automatic smoothness estimation. https://cran.r-project.org/web/packages/mgcv/mgcv.pdf
Wood, S. N., & Scheipl, F. (2020). gamm4: Generalized additive mixed models using ‘mgcv’ and ‘lme4’. R package version 0.2-6. https://CRAN.R-project.org/package=gamm4
Wood, S. N. (2004). Stable and efficient multiple smoothing parameter estimation for generalized additive models. Journal of the American Statistical Association, 99(467), 673–686. https://doi.org/10.1198/016214504000000980
Wood, S. N. (2006). Low-rank scale-invariant tensor product smooths for generalized additive mixed models. Biometrics, 62(4), 1025–1036. https://doi.org/10.1111/j.1541-0420.2006.00574.x
Wood, S. N. (2013). On p-values for smooth components of an extended generalized additive model. Biometrika, 100(1), 221–229. https://doi.org/10.1093/biomet/ass048
Wood, S. N. (2017). Generalized additive models: An introduction with R (2nd ed.). CRC Press.
Wood, S. N., Pya, N., & Säfken, B. (2016). Smoothing parameter and model selection for general smooth models. Journal of the American Statistical Association, 111(516), 1548–1563. https://doi.org/10.1080/01621459.2016.1180986
Wright, D. B., Horry, R., & Skagerberg, E. M. (2009). Functions for traditional and multilevel approaches to signal detection theory. Behavior Research Methods, 41, 257–267. https://doi.org/10.3758/BRM.41.2.257
Yoon, S. O., Benjamin, A. S., & Brown-Schmidt, S. (2016). The historical context in conversation: Lexical differentiation and memory for the discourse history. Cognition, 154, 102–117. https://doi.org/10.1016/j.cognition.2016.05.011
Yoon, S. O., Benjamin, A. S., & Brown-Schmidt, S. (2021). Referential form and memory for the discourse history. Cognitive Science. https://doi.org/10.1111/cogs.12964
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Edward Ip was the handling ARCS Editor for this paper. This research was supported in part by Grant No. SES-1851690 from the National Science Foundation (NSF) to Sun-Joo Cho, Sarah Brown-Schmidt, and Paul De Boeck. Nothing in this article necessarily reflects the positions or policies of the agency.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Cho, SJ., Brown-Schmidt, S., Boeck, P.D. et al. Incorporating Functional Response Time Effects into a Signal Detection Theory Model. Psychometrika 88, 1056–1086 (2023). https://doi.org/10.1007/s11336-023-09906-9
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-023-09906-9