Abstract
Background and Objective
Bayesian methods can be used to elicit experts’ beliefs about the clinical value of healthcare technologies. This study investigates a belief–elicitation method for estimating diagnostic performance in an early stage of development of photoacoustic mammography (PAM) imaging versus magnetic resonance imaging (MRI) for detecting breast cancer.
Research Design
Eighteen experienced radiologists ranked tumor characteristics regarding their importance to detect malignancies. With reference to MRI, radiologists estimated the true positives and negatives of PAM using the variable interval method. An overall probability density function was determined using linear opinion pooling, weighted for individual experts’ experience.
Result
The most important tumor characteristics are mass margins and mass shape. Respondents considered MRI the better technology to visualize these characteristics. Belief elicitation confirmed this by providing an overall sensitivity of PAM ranging from 58.9 to 85.1 % (mode 75.6 %) and specificity ranging from 52.2 to 77.6 % (mode 66.5 %).
Conclusion
Belief elicitation allowed estimates to be obtained for the expected diagnostic performance of PAM, although radiologists expressed difficulties in doing so. Heterogeneity within and between experts reflects this uncertainty and the infancy of PAM. Further clinical trials are required to validate the extent to which this belief–elicitation method is predictive for observed test performance.
Similar content being viewed by others
References
IJzerman MJ, Steuten LMG. Early assessment of medical technologies to inform product development and market access: a review of methods and applications. Appl Health Econ Health Policy. 2011;9(5):331–47.
Vallejo-Torres L, Steuten LMG, Buxton MJ, et al. Integrating health economics modeling in the product development cycle of medical devices: a Bayesian approach. Int J Technol Assess Health Care. 2008;24(4):459–64.
Garthwaite AJ, Kadane JB, O’Hagan A. Statistical methods for eliciting probability distributions. J Am Stat Assoc. 2005;100(470):680–701.
Spiegelhalter DJ, Abrams KR, Myles JP. Bayesian approaches to clinical trials and health-care evaluation. New York: Wiley; 2004.
O’Hagan A, Buck CE, Daneshkhah A, et al. Uncertain judgements: eliciting experts’ probabilities (statistics in practice). Chichester: Wiley; 2006.
Hiance A, Chevret S, Lévy V. A practical approach for eliciting expert prior beliefs about cancer survival in phase III randomized trial. J Clin Epidemiol. 2009;62(4):431–7.
Johnson SR, Tomlinson GA, Hawker GA, et al. A valid and reliable belief elicitation method for Bayesian priors. J Clin Epidemiol. 2010;63(4):370–83.
Leal J, Wordsworth S, Legood R, et al. Eliciting expert opinion for economic models: an applied example. Val Health. 2007;10(3):195–203.
Bojke L, Claxton K, Bravo-Vergel Y, et al. Eliciting distributions to populate decision analytic models. Val Health. 2010;13(5):557–64.
Soares MO, Bojke L, Dumville J, et al. Methods to elicit experts’ beliefs over uncertain quantities: application to a cost effectiveness transition model of negative pressure wound therapy for severe pressure ulceration. Stat Med. 2011;30(19):2363–80.
Johnson SR, Tomlinson GA, Hawker GA, et al. Methods to elicit beliefs for Bayesian priors: a systematic review. J Clin Epidemiol. 2010;63(4):355–69.
Jose J, Manohar S, Kolkman RG, et al. Imaging of tumor vasculature using Twente photoacoustic systems. J Biophotonics. 2009;2(12):701–17.
Piras D, Wenfeng X, Steenbergen W, et al. Photoacoustic imaging of the breast using the Twente Photoacoustic Mammoscope: present status and future perspectives. IEEE J Sel Top Quant. 2010;16(4):730–9.
Mammacarcinoom: Landelijke richtlijn met regionale toevoegingen. Versie 1.1 Nationaal Borstkankeroverleg Nederland (NABON), Vereniging voor Integrale Kankercentra (VIKC), Kwaliteitsinstituut voor de Gezondheidszorg CBO, Amsterdam, The Netherlands; 2008. http://www.oncoline.nl/uploaded/FILES/mammacarcinoom/Richtlijn%20Behandeling%20van%20het%20Mammacarcinoom%20oktober%202005.pdf
Gigerenzer G, Gaissmaier W. Heuristic decision making. Annu Rev Psychol. 2011;62(1):451–82.
Cooke R. Experts in uncertainty: opinion and subjective probability in science. Oxford: Oxford University Press; 1991.
Knol A, Slottje P, van der Sluijs J, et al. The use of expert elicitation in environmental health impact assessment: a seven step procedure. Environ Health. 2010;9(1):19.
ACR BI-RADS- MRI Lexicon Classification Form. Reston: American College of Radiology; 2003.
Gibbs P, Liney GP, Lowry M, et al. Differentiation of benign and malignant sub-1 cm breast lesions using dynamic contrast enhanced MRI. Breast. 2004;13(2):115–21.
Nunes LW, Schnall MD, Orel SG. Update of breast MR imaging architectural interpretation model. Radiology. 2001;219(2):484–94.
Bluemke DA, Gatsonis CA, Chen MH, et al. Magnetic resonance imaging of the breast prior to biopsy. JAMA. 2004;292(22):2735–42.
Bone B, Aspelin P, Bronge L, et al. Sensitivity and specificity of MR mammography with histopathological correlation in 250 breasts. Acta Radiol. 1996;37(2):208–13.
Peterson C, Miller A. Mode, median, and mean as optimal strategies. J Exp Psychol. 1964;68(4):363–7.
van Dorp RJ, Kotz S. A novel extension of the triangular distribution and its parameter estimation. J R Stat Soc Ser D-Sta. 2002;51(1):63–79.
Miglioretti DL, Gard CC, Carney PA, et al. When radiologists perform best: the learning curve in screening mammogram interpretation. Radiology. 2009;253(3):632–40.
Liberman L, Benton CL, Dershaw DD, et al. Learning curve for stereotactic breast biopsy: how many cases are enough? Am J Roentgenol. 2001;176(3):721–7.
Spiegelhalter DJ, Myles JP, Jones DR, et al. An introduction to bayesian methods in health technology assessment. BMJ. 1999;319(7208):508–12.
Girling A, Young T, Brown C, et al. Early-stage valuation of medical devices: the role of developmental uncertainty. Val Health. 2010;13(5):585–91.
Hilgerink MP, Hummel MJM, Manohar S, et al. Assessment of the added value of the Twente Photoacoustic Mammoscope in breast cancer diagnosis. Med Devices. 2011;4:107–15.
Acknowledgments
We would like to thank the radiologists Frank van den Engh, Roland Bezooijen, and Magreet van der Schaaf for providing their input and comments. Furthermore, we would like to thank all radiologists that participated in this study. We would like to thank Srirang Manohar for providing the information about PAM.
There were no sponsors involved in this research and there is no conflict of interest. The submitted manuscript has not been published elsewhere and no funding was received.
Authors’ contribution
WH: design of study, and responsible for data collection and analysis, writing.
LB: design of study, review of expert consultation approach, review of paper.
LS: review of design and data collection, interpretation of findings, review of paper.
MIJ: initiated the study, design and study approach, review of results and paper, responsible for overall content.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Haakma, W., Steuten, L.M.G., Bojke, L. et al. Belief Elicitation to Populate Health Economic Models of Medical Diagnostic Devices in Development. Appl Health Econ Health Policy 12, 327–334 (2014). https://doi.org/10.1007/s40258-014-0092-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40258-014-0092-y