Calculation of signal detection theory measures

Other Articles
  • 8.6k Downloads

Abstract

Signal detection theory (SDT) may be applied to any area of psychology in which two different types of stimuli must be discriminated. We describe several of these areas and the advantages that can be realized through the application of SDT. Three of the most popular tasks used to study discriminability are then discussed, together with the measures that SDT prescribes for quantifying performance in these tasks. Mathematical formulae for the measures are presented, as are methods for calculating the measures with lookup tables, computer software specifically developed for SDT applications, and general purpose computer software (including spreadsheets and statistical analysis software).

References

  1. Achenbach, T. M., &Edelbrock, C. S. (1983).Manual for the child behavior checklist and revised child behavior profile. Burlington: University of Vermont, Department of Psychiatry.Google Scholar
  2. Ahroon, W. A., Jr., &Pastore, R. E. (1977). Procedures for computingd′ andβ.Behavior Research Methods & Instrumentation,9, 533–537.Google Scholar
  3. Alf, E. F., &Grossberg, J.M. (1987). DORF2R. BAS: Analyzing signaldetection theory rating data in the BASIC programming language.Behavior Research Methods, Instruments, & Computers,19, 475–482.CrossRefGoogle Scholar
  4. Banks, W. P. (1970). Signal detection theory and human memory.Psychological Bulletin,74, 81–99.CrossRefGoogle Scholar
  5. Brophy, A. L. (1985). Approximation of the inverse normal distribution function.Behavior Research Methods, Instruments, & Computers,17, 415–417.CrossRefGoogle Scholar
  6. Brophy, A. L. (1986). Alternatives to a table of criterion values in signal detection theory.Behavior Research Methods, Instruments, & Computers,18, 285–286.CrossRefGoogle Scholar
  7. Centor, R. M. (1985). A Visicalc program for estimating the area under a receiver operating characteristic (ROC) curve.Medical Decision Making,5, 139–148.PubMedCrossRefGoogle Scholar
  8. Centor, R. M., &Schwartz, J. S. (1985). An evaluation of methods for estimating the area under a receiver operating characteristic (ROC) curve.Medical Decision Making,5, 149–156.PubMedCrossRefGoogle Scholar
  9. Cradit, J. D., Tashchian, A., &Hofacker, C. F. (1994). Signal detection theory and single observation designs: Methods and indices for advertising recognition testing.Journal of Marketing Research,31, 117–127.CrossRefGoogle Scholar
  10. Craig, A. (1979). Nonparametric measures of sensory efficiency for sustained monitoring tasks.Human Factors,21, 69–78.PubMedGoogle Scholar
  11. Craig, A. (1985). Field studies of human inspection: The application of vigilance research. In S. Folkard & T. H. Monk (Eds.),Hours of work (pp. 133–145). Chichester, U.K.: Wiley.Google Scholar
  12. Craven, B. J. (1992). A table ofd′ forM-alternative odd-man-out forced-choice procedures.Perception & Psychophysics,51, 379–385.CrossRefGoogle Scholar
  13. Davies, D. R., &Parasuraman, R. (1982).The psychology of vigilance. London: Academic Press.Google Scholar
  14. Donaldson, W. (1993). Accuracy ofd′ andA′ as estimates of sensitivity.Bulletin of the Psychonomic Society,31, 271–274.Google Scholar
  15. Dorfman, D. D. (1982). RSCORE II. In J. A. Swets & R. M. Pickett,Evaluation of diagnostic systems: Methods from signal detection theory (pp. 212–232). New York: Academic Press.Google Scholar
  16. Dorfman, D. D., &Berbaum, K. S. (1995). Degeneracy and discrete receiver operating characteristic rating data.Academic Radiology,2, 907–915.PubMedCrossRefGoogle Scholar
  17. Elliott, P. B. (1964). Tables ofd′. In J. A. Swets (Ed.),Signal detection and recognition by human observers (pp. 651–684). New York: Wiley.Google Scholar
  18. Erdelyi, M. H. (1974). A new look at the New Look: Perceptual defense and vigilance.Psychological Review,81, 1–25.PubMedCrossRefGoogle Scholar
  19. Freeman, P. R. (1973).Tables of d′ and β. Cambridge: Cambridge University Press.Google Scholar
  20. Gardner, R. M., &Boice, R. (1986). A computer program to generate signal-detection theory values for sensitivity and response bias.Behavior Research Methods, Instruments, & Computers,18, 54–56.CrossRefGoogle Scholar
  21. Gardner, R. M., Dalsing, S., Reyes, B., &Brake, S. (1984). Table of criterion values (β) used in signal detection theory.Behavior Research Methods, Instruments, & Computers,16, 425–436.CrossRefGoogle Scholar
  22. Garland, H. L. (1949). On the scientific evaluation of diagnostic procedures.Radiology,52, 309–328.PubMedGoogle Scholar
  23. Goldstein, E. B. (1996).Sensation and perception (4th ed.). New York: Brooks Cole.Google Scholar
  24. Green, D. M., &Moses, F. L. (1966). On the equivalence of two recognition measures of short-term memory.Psychological Bulletin,66, 228–234.PubMedCrossRefGoogle Scholar
  25. Green, D. M., &Swets, J. A. (1966).Signal detection theory and psychophysics. New York: Wiley.Google Scholar
  26. Grier, J. B. (1971). Nonparametric indexes for sensitivity and bias: Computing formulas.Psychological Bulletin,75, 424–429.PubMedCrossRefGoogle Scholar
  27. Hacker, M. J., &Ratcliff, R. (1979). A revised table ofd′ form-alternative forced choice.Perception & Psychophysics,26, 168–170.CrossRefGoogle Scholar
  28. Hautus, M. (1995). Corrections for extreme proportions and their biasing effects on estimated values ofd′.Behavior Research Methods, Instruments, & Computers,27, 46–51.CrossRefGoogle Scholar
  29. Hershman, R. L., &Small, D. (1968). Tables ofd′ for detection and localization.Perception & Psychophysics,3, 321–323.CrossRefGoogle Scholar
  30. Hodos, W. (1970). Nonparametric index of response bias for use in de tection and recognition experiments.Psychological Bulletin,74, 351–354.CrossRefGoogle Scholar
  31. Hutchinson, T. P. (1981). A review of some unusual applications of signal detection theory.Quality & Quantity,15, 71–98.CrossRefGoogle Scholar
  32. Ingham, J. G. (1970). Individual differences in signal detection.Acta Psychologica,34, 39–50.PubMedCrossRefGoogle Scholar
  33. Kaplan, H. L., Macmillan, N. A., &Creelman, C. D. (1978). Tables ofd′ for variable-standard discrimination paradigms.Behavior Research Methods & Instrumentation,10, 796–813.CrossRefGoogle Scholar
  34. Klatzky, R. L., &Erdelyi, M. H. (1985). The response criterion problem in tests of hypnosis and memory.International Journal of Clinical & Experimental Hypnosis,33, 246–257.CrossRefGoogle Scholar
  35. Macmillan, N. A. (1993). Signal detection theory as data analysis method and psychological decision model. In G. Keren & C. Lewis (Eds.),A handbook for data analysis in the behavioral sciences: Methodological issues (pp. 21–57). Hillsdale, NJ: Erlbaum.Google Scholar
  36. Macmillan, N. A., &Creelman, C. D. (1990). Response bias: Characteristics of detection theory, threshold theory, and “nonparametric” indexes.Psychological Bulletin,107, 401–413.CrossRefGoogle Scholar
  37. Macmillan, N. A., &Creelman, C. D. (1991).Detection theory: A user’s guide. Cambridge: Cambridge University Press.Google Scholar
  38. Macmillan, N. A., &Creelman, C. D. (1996). Triangles in ROC space: History and theory of “nonparametric” measures of sensitivity and bias.Psychonomic Bulletin & Review,3, 164–170.CrossRefGoogle Scholar
  39. Macmillan, N. A., &Kaplan, H. L. (1985). Detection theory analysis of group data: Estimating sensitivity from average hit and false-alarm rates.Psychological Bulletin,98, 185–199.PubMedCrossRefGoogle Scholar
  40. McGowan, W. T., III, &Appel, J. B. (1977). An APL program for calculating signal detection indices.Behavior Research Methods & Instrumentation,9, 517–521.Google Scholar
  41. McNicol, D. (1972).A primer of signal detection theory. London: Allen & Unwin.Google Scholar
  42. Miller, J. (1996). The sampling distribution ofd′.Perception & Psychophysics,58, 65–72.CrossRefGoogle Scholar
  43. Naylor, J. C., &Lawshe, C. H. (1958). An analytical review of the experimental basis of subception.Journal of Psychology,46, 75–96.CrossRefGoogle Scholar
  44. Nelson, T. O. (1984). A comparison of current measures of the accuracy of feeling-of-knowing predictions.Psychological Bulletin,95, 109–133.PubMedCrossRefGoogle Scholar
  45. Nelson, T. O. (1986). ROC curves and measures of discrimination accuracy: A reply to Swets.Psychological Bulletin,100, 128–132.PubMedCrossRefGoogle Scholar
  46. Pollack, I., &Norman, D. A. (1964). A nonparametric analysis of recognition experiments.Psychonomic Science,1, 125–126.Google Scholar
  47. Rey, J. M., Morris-Yates, A., &Stanislaw, H. (1992). Measuring the accuracy of diagnostic tests using receiver operating characteristic (ROC) analysis.International Journal of Methods in Psychiatric Research,2, 39–50.Google Scholar
  48. Richards, B. L., &Thornton, C. L. (1970). Quantitative methods of calculating thed′ of signal detection theory.Educational & Psychological Measurement,30, 855–859.CrossRefGoogle Scholar
  49. Richardson, J. T. E. (1994). Continuous recognition memory tests: Are the assumptions of the theory of signal detection really met?Journal of Clinical & Experimental Neuropsychology,16, 482–486.CrossRefGoogle Scholar
  50. Smith, J. E. K. (1982). Simple algorithms for M-alternative forcedchoice calculations.Perception & Psychophysics,31, 95–96.CrossRefGoogle Scholar
  51. Smith, W. D. (1995). Clarification of sensitivity measureA′.Journal of Mathematical Psychology,39, 82–89.CrossRefGoogle Scholar
  52. Snodgrass, J. G., &Corwin, J. (1988). Pragmatics of measuring recognition memory: Applications to dementia and amnesia.Journal of Experimental Psychology: General,117, 34–50.CrossRefGoogle Scholar
  53. Snodgrass, J. G., Levy-Berger, G., &Haydon, M. (1985).Human experimental psychology. New York: Oxford University Press.Google Scholar
  54. Stanislaw, H. (1995). Effect of type of task and number of inspectors on performance of an industrial inspection-type task.Human Factors,37, 182–192.CrossRefGoogle Scholar
  55. Stanislaw, H. (1996). Relative versus absolute judgments, and the effect of batch composition on simulated industrial inspection. In R. J. Koubek & W. Karwowski (Eds.),Manufacturing agility and hybrid automation: I (pp. 105–108). Louisville, KY: IEA Press.Google Scholar
  56. Stanislaw, H., &Todorov, N. (1992). Documenting the rise and fall of a psychological theory: Whatever happened to signal detection theory? (Abstract).Australian Journal of Psychology,44, 128.Google Scholar
  57. Swets, J. A. (1973). The relative operating characteristic in psychology.Science,182, 990–1000.PubMedCrossRefGoogle Scholar
  58. Swets, J. A. (1986). Form of empirical ROCs in discrimination and diagnostic tasks: Implications for theory and measurement of performance.Psychological Bulletin,99, 181–198.PubMedCrossRefGoogle Scholar
  59. Swets, J. A. (1988a). Measuring the accuracy of diagnostic systems.Science,240, 1285–1293.PubMedCrossRefGoogle Scholar
  60. Swets, J. A. (Ed.) (1988b).Signal detection and recognition by human observers. New York: Wiley.Google Scholar
  61. Swets, J. A., &Pickett, R. M. (1982).Evaluation of diagnostic systems: Methods from signal detection theory. New York: Academic Press.Google Scholar
  62. Thomas, E. A. C., &Hogue, A. (1976). Apparent weight of evidence, decision criteria, and confidence ratings in juror decision making.Psychological Review,83, 442–465.CrossRefGoogle Scholar
  63. Zelen, M., &Severo, N. C. (1972). Probability functions. In M. Abramowitz & I. A. Stegun (Eds.),Handbook of mathematical functions with formulas, graphs, and mathematical tables (pp. 925–973). Washington, DC: National Bureau of Standards.Google Scholar

Copyright information

© Psychonomic Society, Inc. 1999

Authors and Affiliations

  1. 1.Department of PsychologyCalifornia State University, StanislausTurlock
  2. 2.Macquarie UniversitySydneyAustralia

Personalised recommendations