Abstract
Common methods for Bayesian prior elicitation call for expert belief in the form of numerical summaries. However, certain challenges remain with such strategies. Drawing on recent advances made in graphical inference, we propose an interactive method and tool for prior elicitation in which experts express their belief through a sequence of selections between pairs of graphics, reminiscent of the common procedure used during eye examinations. The graphics are based on synthetic datasets generated from underlying prior models with carefully chosen parameters, instead of the parameters themselves. At each step of the process, the expert is presented with two familiar graphics based on these datasets, billed as hypothetical future datasets, and makes a selection regarding their relative likelihood. Underneath, the parameters that are used to generate the datasets are generated in a way that mimics the Metropolis algorithm, with the experts’ responses forming transition probabilities. Using the general method, we develop procedures for data models used regularly in practice: Bernoulli, Poisson, and Normal, though it extends to additional univariate data models as well. A free, open-source Shiny application designed for these procedures is also available online, helping promote best practice recommendations in myriad ways. The method is supported by simulation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Graph 12 displays the real data.
Additional variances were considered, and the conclusions discussed in the following sections (as well as the proposal standard deviation used) are robust across reasonable ranges of \(\sigma ^2\).
References
Blair, S. (2017). Contributions to the theory and practice of prior elicitation in biopharmaceutical research (Unpublished doctoral dissertation). Baylor University.
Buja, A., Cook, D., Hofmann, H., Lawrence, M., Lee, E.-K., Swayne, D. F., & Wickham, H. (2009). Statistical inference for exploratory data analysis and model diagnostics. Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 367(1906), 4361–4383.
Casement, C. J., & Kahle, D. J. (2018). Graphical prior elicitation in univariate models. Communications in Statistics-Simulation and Computation, 47(10), 2906–2924.
Chang, W., Cheng, J., Allaire, J., Sievert, C., Schloerke, B., Xie, Y., & Borges, B. (2021). shiny: Web application framework for R. Retrieved from https://CRAN.R-project.org/package=shiny (R package version 1.6.0). Accessed 24 Sept 2021.
Garthwaite, P. H., Kadane, J. B., & O’Hagan, A. (2005). Statistical methods for eliciting probability distributions. Journal of the American Statistical Association, 100(470), 680–701.
Jones, G., & Johnson, W. O. (2014). Prior elicitation: interactive spreadsheet graphics with sliders can be fun, and informative. The American Statistician, 68(1), 42–51.
Kahle, D., Stamey, J., Natanegara, F., Price, K., & Han, B. (2016). Facilitated prior elicitation with the wolfram CDF. Biometrics & Biostatistics International Journal, 3(6), 1–6.
Keren, G. (1991). Calibration and probability judgements: Conceptual and methodological issues. Acta Psychologica, 77(3), 217–273.
Lichtenstein, S., & Fischhoff, B. (1977). Do those who know more also know more about how much they know? Organizational Behavior and Human Performance, 20(2), 159–183.
Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1977). Calibration of probabilities. In H. Jungermann & G. De Zeeuw (Eds.), The state of the art. In Decision making and change in human affairs (pp. 275–324). Springer.
Morita, S., Thall, P. F., & Müller, P. (2008). Determining the effective sample size of a parametric prior. Biometrics, 64(2), 595–602.
Morris, D. E., Oakley, J. E., & Crowe, J. A. (2014). A web-based tool for eliciting probability distributions from experts. Environmental Modelling & Software, 52, 1–4.
Oakley, J., & O’Hagan, A. (2010). SHELF: The Sheffield elicitation framework (version 2.0). School of Mathematics and Statistics, University of Sheffield.
O’Hagan, A., Buck, C. E., Daneshkhah, A., Eiser, J. R., Garthwaite, P. H., Jenkinson, D. J., & Rakow, T. (2006). Uncertain judgements: Eliciting experts’ probabilities. Wiley.
Oskamp, S. (1965). Overconfidence in case-study judgments. Journal of Consulting Psychology, 29(3), 261.
R Core Team. (2021). R: A language and environment for statistical computing. Vienna, Austria. Retrieved from https://www.R-project.org/. Accessed 24 Sept 2021.
Resnick, S. I. (2013). A probability path. Springer Science & Business Media.
Robert, C. P., & Casella, G. (2004). Monte Carlo statistical methods (2nd ed.). Springer.
Ruckdeschel, P., Kohl, M., Stabla, T., & Camphausen, F. (2006). distrEx: S4 classes for distributions (Vol. 6) (No. 2).
Su, C.-L. (2006). BetaBuster. Retrieved from https://cadms.vetmed.ucdavis.edu/diagnostic/software (Center for Animal Disease Modeling and Surveillance, University of California, Davis). Accessed 24 Sept 2021.
Wickham, H., Cook, D., Hofmann, H., & Buja, A. (2010). Graphical inference for infovis. IEEE Transactions on Visualization and Computer Graphics, 16(6), 973–979.
Funding
No funding was received to assist with the preparation of this manuscript.
Author information
Authors and Affiliations
Contributions
The authors report they each contributed substantially to this work.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
Ethics approval
Not applicable
Consent to participate
Not applicable
Consent for publication
Both authors agreed with the content and gave explicit consent to submit the manuscript.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Casement, C.J., Kahle, D.J. The phoropter method: a stochastic graphical procedure for prior elicitation in univariate data models. J. Korean Stat. Soc. 52, 60–82 (2023). https://doi.org/10.1007/s42952-022-00189-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42952-022-00189-x