Skip to main content
SpringerLink
Log in

Quantifying accuracy on distance estimation tasks: A Monte Carlo study

  • Original Manuscript
  • Published:
Behavior Research Methods Aims and scope Submit manuscript

Abstract

An important aspect of perceptual learning involves understanding how well individuals can perceive distances, sizes, and time-to-contact. Oftentimes, the primary goal in these experiments is to assess participants’ errors (i.e., how accurately participants perform these tasks). However, the manner in which researchers have quantified error, or task accuracy, has varied. The use of different measures of task accuracy, to include error scores, ratios, and raw estimates, indicates that the interpretation of findings depends on the measure of task accuracy utilized. In an effort to better understand this issue, we used a Monte Carlo simulation to evaluate five dependent measures of accuracy: raw distance judgments, a ratio of true to estimated distance judgments, relative error, signed error, and absolute error. We simulated data consistent with prior findings in the distance perception literature and evaluated how findings and interpretations vary as a function of the measure of accuracy used. We found there to be differences in both statistical findings (e.g., overall model fit, mean square error, Type I error rate) and the interpretations of those findings. The costs and benefits of utilizing each accuracy measure for quantifying accuracy in distance estimation studies are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1 
Fig. 2 
Fig. 3 
Fig. 4 
Fig. 5 
Fig. 6 
Fig. 7 
Fig. 8 
Fig. 9 
Fig. 10 
Fig. 11
Fig. 12 
Fig. 13
Fig. 14 

Similar content being viewed by others

Data availability

The R code and data are available at Open Science Framework: https://osf.io/7hxrs/

References

  • Adams, H., Narasimham, G., Rieser, J., Creem-Regehr, S., Stefanucci, J., & Bodenheimer, B. (2018). Locomotive recalibration and prism adaptation of children and teens in immersive virtual environments. IEEE Transactions on Visualization and Computer Graphics, 24(4), 1408–1417. https://doi.org/10.1109/TVCG.2018.2794072

    Article  PubMed  Google Scholar 

  • Akers, R., & Campbell, F. L. (1970). Size and the administrative component in occupational associations. The Pacific Sociological Review, 13(4), 241–251. https://doi.org/10.2307/1388455

    Article  Google Scholar 

  • Bollen, K. A., & Ward, S. (1979). Ratio variables in aggregate data analysis: Their uses, problems, and alternatives. Sociological Methods & Research, 7(4), 431–450. https://doi.org/10.1177/004912417900700405

    Article  Google Scholar 

  • Buck, L. E., Young, M. K., & Bodenheimer, B. (2018). A comparison of distance estimation in HMD-based virtual environments with different HMD-based conditions. ACM Transactions on Applied Perception, 15(3), 1–15. https://doi.org/10.1145/3196885

    Article  Google Scholar 

  • Certo, S. T., Busenbark, J. R., Kalm, M., & LePine, J. A. (2020). Divided we fall: How ratios undermine research in strategic management. Organizational Research Methods, 23(2), 211–237. https://doi.org/10.1177/1094428118773455

    Article  Google Scholar 

  • Day, B., Ebrahimi, E., Hartman, L. S., Pagano, C. C., Robb, A. C., & Babu, S. V. (2019). Examining the effects of altered avatars on perception-action in virtual reality. Journal of Experimental Psychology: Applied, 25(1), 1–24. https://doi.org/10.1037/xap0000192

    Article  PubMed  Google Scholar 

  • Feldstein, I. T., Kölsch, F. M., & Konrad, R. (2020). Egocentric distance perception: A comparative study investigating differences between real and virtual environments. Perception, 49(9), 940–967. https://doi.org/10.1177/0301006620951997

    Article  PubMed  Google Scholar 

  • Firebaugh, G., & Gibbs, J. P. (1986). Using ratio variables to control for population size. Sociological Methods & Research, 15(1–2), 101–117.

    Article  Google Scholar 

  • Fox, J., & Weisberg, S. (2019). An R companion to applied regression (3rd ed.). Sage.

    Google Scholar 

  • Geuss, M. N., Stefanucci, J. K., Creem-Regehr, S. H., & Thompson, W. B. (2012). Effect of viewing plane on perceived distances in real and virtual environments. Journal of Experimental Psychology: Human Perception and Performance, 38(5), 1242–1253. https://doi.org/10.1037/a0027524

    Article  PubMed  Google Scholar 

  • Guth, D. (1990). Space saving statistics: An introduction to constant error, variable error, and absolute error. Peabody Journal of Education, 67(2), 110–120. https://doi.org/10.1080/01619569009538684

    Article  Google Scholar 

  • Hancock, G. R., Butler, M. S., & Fischman, M. G. (1995). On the problem of two-dimensional error scores: Measures and analyses of accuracy, bias, and consistency. Journal of Motor Behavior, 27(3), 241–250.

    Article  PubMed  Google Scholar 

  • Hartman, L. (2018). Perception-action system calibration in the presence of stable and unstable perceptual perturbations. All Dissertations. 2144. https://tigerprints.clemson.edu/all_dissertations/2144

  • Henry, F. M. (1975). Absolute error vs “E” in target accuracy. Journal of Motor Behavior, 7(3), 227–228.

    Article  CAS  PubMed  Google Scholar 

  • Horswill, M. S., Helman, S., Ardiles, P., & Wann, J. P. (2005). Motorcycle accident risk could be inflated by a time to arrival illusion. Optometry and Vision Science, 82(8), 740–746. https://doi.org/10.1097/01.opx.0000175563.21423.50

    Article  PubMed  Google Scholar 

  • Interrante, V., Ries, B., & Anderson, L. (2006). Distance perception in immersive virtual environments, revisited. IEEE Virtual Reality Conference (VR 2006), 3–10. https://doi.org/10.1109/VR.2006.52

  • Kelly, J. W., Hammel, W. W., Siegel, Z. D., & Sjolund, L. A. (2014). Recalibration of perceived distance in virtual environments occurs rapidly and transfers asymmetrically across scale. IEEE Transactions on Visualization and Computer Graphics, 20(4), 588–595. https://doi.org/10.1109/TVCG.2014.36

    Article  PubMed  Google Scholar 

  • Kiefer, R. J., Flannagan, C. A., & Jerome, C. J. (2006). Time-to-collision judgments under realistic driving conditions. Human Factors: The Journal of the Human Factors and Ergonomics Society, 48(2), 334–345. https://doi.org/10.1518/001872006777724499

    Article  Google Scholar 

  • Kunz, B. R., Creem-Regehr, S. H., & Thompson, W. B. (2015). Testing the mechanisms underlying improved distance judgments in virtual environments. Perception, 44(4), 446–453. https://doi.org/10.1068/p7929

    Article  PubMed  Google Scholar 

  • Long, S. B. (1980). The continuing debate over the use of ratio variables: Facts and fiction. Sociological Methodology, 11, 37–67. JSTOR. https://doi.org/10.2307/270859

  • Loomis, J. M., & Knapp, J. M. (2003). Visual perception of egocentric distance in real and virtual environments. In Virtual and Adaptive Environments: Applications, Implications, and Human Performance Issues (Vol. 11, pp. 21–46).

  • Mark, L. S. (1987). Eyeheight-scaled information about affordances: A study of sittting and stair climbing. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 361–370.

    CAS  PubMed  Google Scholar 

  • Marsaglia, G. (1965). Ratios of normal variables and ratios of sums of uniform variables. Journal of the American Statistical Association, 60(309), 193–204.

  • Marsaglia, G. (2006). Ratios of normal variables. Journal of Statistical Software, 16, 1–10.

    Article  Google Scholar 

  • McCullagh, P., & Nelder, J. A. (1989). Generalized linear models (2nd ed.). Chapman & Hall.

    Book  Google Scholar 

  • Messing, R., & Durgin, F. H. (2005). Distance perception and the visual horizon in head-mounted displays. ACM Transactions on Applied Perception, 2(3), 234–250. https://doi.org/10.1145/1077399.1077403

    Article  Google Scholar 

  • Mohler, B. J., Creem-Regehr, S. H., & Thompson, W. B. (2006). The influence of feedback on egocentric distance judgments in real and virtual environments. In Proceedings of the 3rd symposium on Applied perception in graphics and visualization (pp. 9–14). https://doi.org/10.1145/1140491.1140493

  • Newell, K. M. (1976). More on absolute error, etc. Journal of Motor Behavior, 8(2), 139–142. https://doi.org/10.1080/00222895.1976.10735064

    Article  CAS  PubMed  Google Scholar 

  • Ogawa, N., Narumi, T., & Hirose, M. (2019). Virtual hand realism affects object size perception in body-based scaling. 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 519–528. https://doi.org/10.1109/VR.2019.8798040

  • Osborne, J. W., & Waters, E. (2002). Four assumptions of multiple regression that researchers should always test. Practical Assessment, Research, and Evaluation, 8(2), 1–5.

    Google Scholar 

  • Plumert, J. M., Kearney, J. K., Cremer, J. F., & Recker, K. (2005). Distance perception in real and virtual environments. ACM Transactions on Applied Perception, 2(3), 18.

    Article  Google Scholar 

  • R Core Team (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/

  • Ritz, C., & Streibig, J. C. (2008). Nonlinear regression with R. New York, NY: Springer New York.

  • Rosopa, P. J., Schaffer, M. M., & Schroeder, A. N. (2013). Managing heteroscedasticity in general linear models. Psychological Methods, 18(3), 335–351. https://psycnet.apa.org/doi/https://doi.org/10.1037/a0032553

  • Rosopa, P. J., Brawley, A. M., Atkinson, T. P., & Robertson, S. A. (2019). On the conditional and unconditional type I error rates and power of tests in linear models with heteroscedastic errors. Journal of Modern Applied Statistical Methods, 17(2), 2–26.

    Article  Google Scholar 

  • Rusch, M. L., Schall, M. C., Lee, J. D., Dawson, J. D., Edwards, S. V., & Rizzo, M. (2016). Time-to-contact estimation errors among older drivers with useful field of view impairments. Accident Analysis & Prevention, 95, 284–291. https://doi.org/10.1016/j.aap.2016.07.008

    Article  Google Scholar 

  • Sahm, C. S., Creem-Regehr, S. H., Thompson, W. B., & Willemsen, P. (2005). Throwing versus walking as indicators of distance perception in similar real and virtual environments. ACM Transactions on Applied Perception, 2(1), 35–45. https://doi.org/10.1145/1048687.1048690

    Article  Google Scholar 

  • Schmidt, R. A. (1975). A schema theory of discrete motor skill learning. Psychological Review, 82(4), 225–260. https://doi.org/10.1037/h0076770

    Article  Google Scholar 

  • Schmidt, R. A., & Lee, T. D. (2005). Motor control and learning: A behavioral emphasis (4th ed.). Human Kinetics.

  • Schutz, R. W., & Roy, E. A. (1973). Absolute error: The devil in disguise. Journal of Motor Behavior, 5(3), 141–153. https://doi.org/10.1080/00222895.1973.10734959

    Article  CAS  PubMed  Google Scholar 

  • Schwartz, M. (1999). Haptic perception of the distance walked when blindfolded. Journal of Experimental Psychology: Human Perception and Performance, 25(3), 852–865. https://doi.org/10.1037/0096-1523.25.3.852

    Article  CAS  PubMed  Google Scholar 

  • Sidaway, B., Fairweather, M., Sekiya, H., & Mcnitt-Gray, J. (1996). Time-to-collision estimation in a simulated driving task. Human Factors: The Journal of the Human Factors and Ergonomics Society, 38(1), 101–113. https://doi.org/10.1518/001872096778940813

    Article  Google Scholar 

  • Siegel, Z. D., & Kelly, J. W. (2017). Walking through a virtual environment improves perceived size within and beyond the walked space. Attention, Perception, & Psychophysics, 79(1), 39–44. https://doi.org/10.3758/s13414-016-1243-z

    Article  Google Scholar 

  • Solini, H., & Andre, J. (2020). Time-to-arrival estimations to simulated pedestrians. Accident Analysis & Prevention, 146, 105739. https://doi.org/10.1016/j.aap.2020.105739

    Article  Google Scholar 

  • Solini, H. M., Bhargava, A., & Pagano, C. C. (2021). The effects of testing environment, experimental design, and ankle loading on calibration to perturbed optic flow during locomotion. Attention, Perception, & Psychophysics, 83(1), 497–511.

    Article  Google Scholar 

  • Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Pearson.

    Google Scholar 

  • Thompson, W. B., Willemsen, P., Gooch, A. A., Creem-Regehr, S. H., Loomis, J. M., & Beall, A. C. (2004). Does the quality of the computer graphics matter when judging distances in visually immersive environments? Presence: Teleoperators and Virtual Environments, 13(5), 560–571. https://doi.org/10.1162/1054746042545292

  • van der Hoort, B., & Ehrsson, H. H. (2016). Illusions of having small or large invisible bodies influence visual perception of object size. Scientific Reports, 6(1), 34530. https://doi.org/10.1038/srep34530

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  • Van Domelen, D. R. (2019). dvmisc: Convenience functions, moving window statistics, and graphics. R package version 1.1.4. https://CRAN.R-project.org/package=dvmisc

  • Willemsen, P., Gooch, A. A., Thompson, W. B., & Creem-Regehr, S. H. (2008). Effects of stereo viewing conditions on distance perception in virtual environments. Presence: Teleoperators and Virtual Environments, 17(1), 91–101. https://doi.org/10.1162/pres.17.1.91

  • Wiseman, R. M. (2009). On the use and misuse of ratios in strategic management research. In D. D. Bergh & D. J. Ketchen (Eds.), Research Methodology in Strategy and Management (Vol. 5, pp. 75–110). Emerald Group Publishing Limited. https://doi.org/10.1108/S1479-8387(2009)0000005004

  • Withagen, R., & Michaels, C. F. (2004). Transfer of calibration in length perception by dynamic touch. Perception & Psychophysics, 66(8), 1282–1292. https://doi.org/10.3758/BF03194998

    Article  Google Scholar 

  • Witmer, B. G., & Kline, P. B. (1998). Judging perceived and traversed distance in virtual environments. Presence: Teleoperators and Virtual Environments, 7(2), 144–167. https://doi.org/10.1162/105474698565640

  • Witmer, B. G., & Sadowski, W. J. (1998). Nonvisually guided locomotion to a previously viewed target in real and virtual environments. Human Factors: The Journal of the Human Factors and Ergonomics Society, 40(3), 478–488. https://doi.org/10.1518/001872098779591340

    Article  Google Scholar 

  • Woltman, H., Feldstain, A., MacKay, J. C., & Rocchi, M. (2012). An introduction to hierarchical linear modeling. Tutorials in Quantitative Methods for Psychology, 8(1), 52–69.

    Article  Google Scholar 

  • Ziemer, C. J., Branson, M. J., Chihak, B. J., Kearney, J. K., Cremer, J. F., & Plumert, J. M. (2013). Manipulating perception versus action in recalibration tasks. Attention, Perception, & Psychophysics, 75(6), 1260–1274. https://doi.org/10.3758/s13414-013-0473-6

    Article  Google Scholar 

  • Ziemer, C. J., Plumert, J. M., Cremer, J. F., & Kearney, J. K. (2009). Estimating distance in real and virtual environments: Does order make a difference? Attention, Perception, & Psychophysics, 71(5), 1095–1106. https://doi.org/10.3758/APP.71.5.1096

    Article  Google Scholar 

Download references

Funding

Not applicable

Author information

Authors and Affiliations

  1. Department of Psychology, Clemson University, 418 Brackett Hall, Clemson, SC, USA

    Hannah M. Solini, Elenah B. Rosopa, Patrick J. Rosopa & Christopher C. Pagano

Authors
  1. Hannah M. Solini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elenah B. Rosopa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patrick J. Rosopa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christopher C. Pagano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hannah M. Solini.

Ethics declarations

Conflicts of interest

The authors declare no potential conflicts of interest

Ethics approval

Not applicable

Consent to participate

Not applicable

Consent for publication

Not applicable

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table

Table 10 Fictitious data
Full size table

10

Table

Table 11  Subset of the simulation results for the relative error and signed error approaches where σε was 0.5, sample size was 1000, and the environment was VR
Full size table

11

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Solini, H.M., Rosopa, E.B., Rosopa, P.J. et al. Quantifying accuracy on distance estimation tasks: A Monte Carlo study. Behav Res (2024). https://doi.org/10.3758/s13428-024-02353-z

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.3758/s13428-024-02353-z

Keywords

Search

Navigation