We agree with Dr. Shibata that education that incorporates the principles we discussed in our article will contribute to diagnostic excellence in medicine.1,2 We also agree that spectrum effect3 can be a potential threat to external validity (generalizability) in studies of diagnostic accuracy, and that this is equally important to consider as potential threats to internal validity, such as verification bias and biases related to reference test selection and interpretation.4 We used the term “data quality issues” to refer to both types of validity in our article.
We do have one point of disagreement with Dr. Shibata, who advocates for an alternative likelihood ratio heuristic, first proposed by McGee,5 in which several easy-to-remember likelihood ratios are scaled to approximate changes in probability of disease. In our view, the major limitation of using this rule is that it becomes more inaccurate as pretest probability approaches the extremes (0% or 100%). Even within the pretest probability range that Shibata suggests (10%–90%), the heuristic can misguide clinicians. For example, if pretest probability is 70%, then after a test result with LR = 5, the heuristic suggests a definitive diagnosis (heuristic: 70% + 30% = 100% posttest probability). However, the true posttest probability is 92%, which is significantly below certainty. We caution against assuming that patients with disease probabilities outside of the 10%–90% range do not require additional testing, because decision-making thresholds vary based on the disease, test, and treatment being considered,6 as well as patient values and preferences. Furthermore, it is common for patients with pretest probability < 10% to undergo further testing (e.g., D-dimer testing in patients with low probability of pulmonary embolism based on a clinical prediction rule; cardiac troponin testing in patients with low, but not negligible, pretest probability of myocardial infarction; and most screening programs, to name a few).
Like with our criticism of other heuristics that have been used to teach diagnostic reasoning, we fear that learners too often remember the rule but not the caveats, which leads to mistakes. In addition, the proposed heuristic is simply unnecessary because it requires the same amount of information input—pretest probability and likelihood ratio—as does use of a nomogram or online calculator, which are more accurate.2 Therefore, we continue to advocate for use of Bayes’ rule as the only rule when it comes to probabilistic diagnostic reasoning.
References
Shibata M. Letter to the Editor: Moving Past SpPin and SnNout. J Gen Intern Med. 2023.
Fischer BG, Evans AT. SpPin and SnNout are not enough. It’s time to fully embrace likelihood ratios and probabilistic reasoning to achieve diagnostic excellence [published online ahead of print, 2023 Apr 3]. J Gen Intern Med. 2023. https://doi.org/10.1007/s11606-023-08177-5
Mulherin SA, Miller WC. Spectrum bias or spectrum effect? Subgroup variation in diagnostic test evaluation. Ann Intern Med. 2002;137(7):598–602.
Furukawa TA, Strauss SE, Bucher HC, Agoritsas T, Guyatt G. Chapter 18: Diagnostic tests. In: Guyatt G, Rennie D, Meade MO, Cook DJ, eds. Users’ Guides to the Medical Literature: a Manual for Evidence-Based Clinical Practice. 3rd ed. New York: McGraw-Hill Education; 2015:345–57.
McGee S. Simplifying likelihood ratios. J Gen Intern Med. 2002;17(8):646–9.
Pauker SG, Kassirer JP. The threshold approach to clinical decision making. N Engl J Med. 1980;302(20):1109–17.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they do not have a conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Fischer, B.G., Evans, A.T. Response to Shibata: Moving Past SpPin and SnNout. J GEN INTERN MED 38, 2838 (2023). https://doi.org/10.1007/s11606-023-08242-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-023-08242-z