Encyclopedia of Machine Learning and Data Mining
pp 18
Classifier Calibration
 Peter A. FlachAffiliated withDepartment of Computer Science, University of Bristol Email author
Abstract
Classifier calibration is concerned with the scale on which a classifier’s scores are expressed. While a classifier ultimately maps instances to discrete classes, it is often beneficial to decompose this mapping into a scoring classifier which outputs one or more realvalued numbers and a decision rule which converts these numbers into predicted classes. For example, a linear classifier might output a positive or negative score whose magnitude is proportional to the distance between the instance and the decision boundary, in which case the decision rule would be a simple threshold on that score. The advantage of calibrating these scores to a known, domainindependent scale is that the decision rule then also takes a domainindependent form and does not have to be learned. The bestknown example of this occurs when the classifier’s scores approximate, in a precise sense, the posterior probability over the classes; the main advantage of this is that the optimal decision rule is to predict the class that minimizes expected cost averaged over all possible true classes.The main methods to obtain calibrated scores are logistic calibration, which is a parametric method that assumes that the distances on either side of the decision boundary are normally distributed and a nonparametric alternative that is variously known as isotonic regression, the pool adjacent violators (PAV) method or the ROC convex hull (ROCCH) method.
Inside
Reference Work Entry Metrics
Version History
Date: 2016 (Latest)
History 2016 (Latest)
Reference tools
Other actions
 Title
 Classifier Calibration
 Reference Work Title
 Encyclopedia of Machine Learning and Data Mining
 Pages
 pp 18
 Copyright
 2016
 DOI
 10.1007/9781489975027_9001
 Online ISBN
 9781489975027
 Publisher
 Springer US
 Copyright Holder
 Springer Science+Business Media New York
 Topics
 Industry Sectors
 eBook Packages
 Editors

 Claude Sammut ^{(1)}
 Geoffrey I. Webb ^{(2)}
 Editor Affiliations

 1. Engineering (CSE), University of New South Wales School of Computer Science &
 2. Software Engineering, Monash University School of Computer Science &
 Authors

 Peter A. Flach ^{(3)}
 Author Affiliations

 3. Department of Computer Science, University of Bristol, Woodland Road, BS8 1UB, Bristol, UK
Continue reading...
To view the rest of this content please follow the download PDF link above.