Improving the Decision Support in Diagnostic Systems Using Classifier Probability Calibration

Abstract

In modern medical diagnoses, classifying a patient’s disease is often realized with the help of a system-aided symptoms interpreter. Most of these systems rely on supervised learning algorithms, which can statistically extend the doctor’s logic capabilities for interpreting and examining symptoms, thus supporting the doctor to find the correct diagnosis. Besides, these algorithms compute classifier scores and class labels that are used to statistically characterize the system’s confidence level on a patient’s type of disease. Unfortunately, most classifier scores are based on an arbitrary scale but not uniformed, thus the interpretations often lack of clinical significance and evaluation criterion. Especially combining multiple classifier scores within a diagnostic system, it is essential to apply a calibration process to make the different scores comparable.

As a frequently used calibration technique, we adapted isotonic regression for our medical diagnostic support system, to provide a flexible and effective scaling process that consequently calibrates the arbitrary scales of classifiers’ scores. In a comparative evaluation, we show that our disease diagnostic system with isotonic regression can actively improve the diagnostic result based on an ensemble of classifiers, also effectively remove outliers from data, thus optimize the decision support system to obtain better diagnostic results.