Abstract
When measuring thresholds, careful selection of stimulus amplitude can increase efficiency by increasing the precision of psychometric fit parameters (e.g., decreasing the fit parameter error bars). To find efficient adaptive algorithms for psychometric threshold (“sigma”) estimation, we combined analytic approaches, Monte Carlo simulations, and human experiments for a one-interval, binary forced-choice, direction-recognition task. To our knowledge, this is the first time analytic results have been combined and compared with either simulation or human results. Human performance was consistent with theory and not significantly different from simulation predictions. Our analytic approach provides a bound on efficiency, which we compared against the efficiency of standard staircase algorithms, a modified staircase algorithm with asymmetric step sizes, and a maximum likelihood estimation (MLE) procedure. Simulation results suggest that optimal efficiency at determining threshold is provided by the MLE procedure targeting a fraction correct level of 0.92, an asymmetric 4-down, 1-up staircase targeting between 0.86 and 0.92 or a standard 6-down, 1-up staircase. Psychometric test efficiency, computed by comparing simulation and analytic results, was between 41 and 58 % for 50 trials for these three algorithms, reaching up to 84 % for 200 trials. These approaches were 13–21 % more efficient than the commonly used 3-down, 1-up symmetric staircase. We also applied recent advances to reduce accuracy errors using a bias-reduced fitting approach. Taken together, the results lend confidence that the assumptions underlying each approach are reasonable and that human threshold forced-choice decision making is modeled well by detection theory models and mimics simulations based on detection theory models.
Similar content being viewed by others
Notes
The Cramér-Rao bound—named for Harold Cramér and C.R. Rao, two of the first to derive it—defines the theoretic lower bound for the variance of estimated parameters. More can be learned at http://en.wikipedia.org/wiki/Cramer-Rao_bound.
Indeed, Wetherill (1963) stated that “ we need the asymptotic properties of various strategies.”
References
Agrawal Y, Bremova T, Kremmyda O, Strupp M, MacNeilage PR (2013) Clinical testing of otolith function: perceptual thresholds and myogenic potentials. J Assoc Res Otolaryngol 14:905–915
Benson AJ, Spencer MB, Stott JR (1986) Thresholds for the detection of the direction of whole-body, linear movement in the horizontal plane. Aviat Space Environ Med 57:1088–1096
Benson AJ, Hutt EC, Brown SF (1989) Thresholds for the perception of whole body angular movement about a vertical axis. Aviat Space Environ Med 60:205–213
Butler JS, Smith ST, Campos JL, Bulthoff HH (2010) Bayesian integration of visual and vestibular signals for heading. J Vis 10:23
Chaudhuri SE, Merfeld DM (2013) Signal detection theory and vestibular perception: III. Estimating unbiased fit parameters for psychometric functions. Exp Brain Res 225:133–146
Chaudhuri SE, Karmali F, Merfeld DM (2013) Whole-body motion-detection tasks can yield much lower thresholds than direction-recognition tasks: implications for the role of vibration. J Neurophysiol 110:2764–2772
Cornsweet T (1962) The staircrase-method in psychophysics. Am J Psychol 75:485–491
Cramér H (1946) Mathematical methods of statistics. Princeton University Press, Princeton
Crane BT (2012a) Fore-aft translation aftereffects. Exp Brain Res 219:477–487
Crane BT (2012b) Roll aftereffects: influence of tilt and inter-stimulus interval. Exp Brain Res 223:89–98
Dixon WJ, Mood AM (1948) A method for obtaining and analyzing sensitivity data. J Am Stat Assoc 43:109–126
Efron B, Tibshirani RJ (1994) An introduction to the bootstrap. CRC Press, New York
Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415:429–433
Firth D (1993) Bias reduction of maximum likelihood estimates. Biometrika 80:27–38
Fisher RA (1922) On the mathematical foundations of theoretical statistics. Philos Trans R Soc Lond Ser A 222:309–368
Fisher RA (1925) Theory of statistical estimation. Proc Cambridge Philos Soc 22:700–725
Fletcher H (1923) Physical measurements of audition and their bearing on the theory of hearing. Bell Syst Tech J 2:145–180
Garcia-Perez MA (1998) Forced-choice staircases with fixed step sizes: asymptotic and small-sample properties. Vision Res 38:1861–1881
Garcia-Perez MA (2000) Optimal setups for forced-choice staircases with fixed step sizes. Spat Vis 13:431–448
Garcia-Perez MA, Alcala-Quintana R (2007) Bayesian adaptive estimation of arbitrary points on a psychometric function. Br J Math Stat Psychol 60:147–174
Garcia-Perez MA, Alcala-Quintana R (2009) Empirical performance of optimal Bayesian adaptive estimation. Span J Psychol 12:3–11
Garcia-Perez MA, Alcala-Quintana R (2013) Shifts of the psychometric function: distinguishing bias from perceptual effects. Q J Exp Psychol (Hove) 66:319–337
Grabherr L, Nicoucar K, Mast FW, Merfeld DM (2008) Vestibular thresholds for yaw rotation about an earth-vertical axis as a function of frequency. Exp Brain Res 186:677–681
Green DM (1990) Stimulus selection in adaptive psychophysical procedures. J Acoust Soc Am 87:2662–2674
Guedry FE, Collins WE, Sheffey PL (1961) Perceptual and oculomotor reactions to interacting visual and vestibular stimulation. Percept Mot Skills 12:307–324
Haburcakova C, Lewis RF, Merfeld DM (2012) Frequency dependence of vestibuloocular reflex thresholds. J Neurophysiol 107:973–983
Hall JL (1968) Maximum-likelihood sequential procedure for estimation. J Acoust Soc Am 44:370
Hall JL (1981) Hybrid adaptive procedure for estimation of psychometric functions. J Acoust Soc Am 69:1763–1769
Hartmann M, Haller K, Moser I, Hossner EJ, Mast FW (2014) Direction detection thresholds of passive self-motion in artistic gymnasts. Exp Brain Res 232:1249–1258
Harvey LO (1986) Efficient estimation of sensory thresholds. Behav Res Methods InstrumComput 18:623–632
Kaernbach C (1991) Simple adaptive testing with the weighted up-down method. Percept Psychophys 49:227–229
Kaernbach C (2001) Slope bias of psychometric functions derived from adaptive data. Percept Psychophys 63:1389–1398
Karmali F, Lim K, Merfeld DM (2014) Visual and vestibular perceptual thresholds each demonstrate better precision at specific frequencies and also exhibit optimal integration. J Neurophysiol 111:2393–2403
Kolev O, Mergner T, Kimmig H, Becker W (1996) Detection thresholds for object motion and self-motion during vestibular and visuo-oculomotor stimulation. Brain Res Bull 40:451–457
Kollmeier B, Gilkey RH, Sieben UK (1988) Adaptive staircase techniques in psychoacoustics: a comparison of human data and a mathematical model. J Acoust Soc Am 83:1852–1862
Kontsevich LL, Tyler CW (1999) Bayesian adaptive estimation of psychometric slope and threshold. Vision Res 39:2729–2737
Leek MR (2001) Adaptive procedures in psychophysical research. Percept Psychophys 63:1279–1292
Leek MR, Hanna TE, Marshall L (1992) Estimation of psychometric functions from adaptive tracking procedures. Percept Psychophys 51:247–256
Levitt HCCH (1971) Transformed up-down methods in psychoacoustics. J Acoust Soc Am 49:467–477
Lewis RF, Priesol AJ, Nicoucar K, Lim K, Merfeld DM (2011) Abnormal motion perception in vestibular migraine. Laryngoscope 121:1124–1125
Lim K, Merfeld DM (2012) Signal detection theory and vestibular perception: II. Fitting perceptual thresholds as a function of frequency. Exp Brain Res 222:303–320
MacNeilage PR, Banks MS, DeAngelis GC, Angelaki DE (2010) Vestibular heading discrimination and sensitivity to linear acceleration in head and world coordinates. J Neurosci 30:9084–9094
McCullagh P, Nelder JA (1989) Generalized linear models. Chapman and Hall, London
Merfeld DM (2011) Signal detection theory and vestibular thresholds: I. Basic theory and practical considerations. Exp Brain Res 210:389–405
Nakayama K, Tyler CW (1981) Psychophysical isolation of movement sensitivity by removal of familiar position cues. Vision Res 21:427–433
Pentland A (1980) Maximum likelihood estimation: the best PEST. Percept Psychophys 28:377–379
Rao CR (1945) Information and the accuracy attainable in the estimation of statistical parameters. Bull Calcutta Math Soc 37:81–89
Roditi RE, Crane BT (2012a) Directional asymmetries and age effects in human self-motion perception. J Assoc Res Otolaryngol 13:381–401
Roditi RE, Crane BT (2012b) Suprathreshold asymmetries in human motion perception. Exp Brain Res 219:369–379
Shen Y, Dai W, Richards VM (2015) A MATLAB toolbox for the efficient estimation of the psychometric function using the updated maximum-likelihood adaptive procedure. Behav Res Methods 47:13–26
Soyka F, Robuffo GP, Beykirch K, Bulthoff HH (2011) Predicting direction detection thresholds for arbitrary translational acceleration profiles in the horizontal plane. Exp Brain Res 209:95–107
Taylor MM (1971) On the efficiency of psychophysical measurement. J Acoust Soc Am 49(Suppl 8):505–508
Taylor MM, Creelman CD (1967) PEST: efficient estimates on probability functions. J Acoust Soc Am 41:782–787
Treutwein B (1995) Adaptive psychophysical procedures. Vision Res 35:2503–2522
Treutwein B, Strasburger H (1999) Fitting the psychometric function. Percept Psychophys 61:87–106
Valko Y, Priesol AJ, Lewis RF, Merfeld DM (2012) Contributions of the vestibular labyrinth to human whole-body motion discrimination. J Neurosci 32:13537–13542
Van Trees HL, Bell KL, Tian Z (2013) Detection estimation and modulation theory, detection, estimation, and filtering theory. Wiley, New York
Watson AB, Pelli DG (1983) QUEST: a Bayesian adaptive psychometric method. Percept Psychophys 33:113–120
Watt RJ, Andrews DP (1981) APE: adaptive probit estimation of psychometric functions. Curr Psychol Rev 1:205–213
Wetherill GB (1963) sequential estimation of quantal response curves. J R Stat Soc B 25:1–48
Wichmann FA, Hill NJ (2001a) The psychometric function: I. Fitting, sampling, and goodness of fit. Percept Psychophys 63:1293–1313
Wichmann FA, Hill NJ (2001b) The psychometric function: II. Bootstrap-based confidence intervals and sampling. Percept Psychophys 63:1314–1329
Zupan LH, Merfeld DM (2008) Interaural self-motion linear velocity thresholds are shifted by roll vection. Exp Brain Res 191:505–511
Acknowledgements
We appreciate the participation of our anonymous subjects. Marcos Mattana assisted with data collection. We thank Bob Grimes and Wangsong Gong for technical support. Harvard Orchestra provided high-capacity computation for simulations. This research was supported by the National Institute on Deafness and Other Communication Disorders (NIDCD) grants R01-DC04158, R56-DC12038 and R03-DC013635. Some of this work was previous described in an academic report of a Harvard School of Engineering and Applied Sciences undergraduate senior design project: Chaudhuri SE (2011) Protocol Design for Characterizing Vestibular Thresholds. The authors declare no competing financial interests.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Karmali, F., Chaudhuri, S.E., Yi, Y. et al. Determining thresholds using adaptive procedures and psychometric fits: evaluating efficiency using theory, simulations, and human experiments. Exp Brain Res 234, 773–789 (2016). https://doi.org/10.1007/s00221-015-4501-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-015-4501-8