Abstract
Closely related to the problem of the reliability presented in the previous chapter is the problem of determining if the difference between multiple performance evaluation measurements is statistically significant. If the interest is in the statistical significance of differences between performance evaluation scores of machine learning models, the discussion of the previous chapter showed that there are two major sources of randomness that need to be respected: one is the randomness of the test data sample on which the models are to be compared. The other is the inherent randomness of the machine learning procedure, exemplified by meta-parameter variations.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Riezler, S., Hagmann, M. (2022). Significance. In: Validity, Reliability, and Significance. Synthesis Lectures on Human Language Technologies. Springer, Cham. https://doi.org/10.1007/978-3-031-02183-1_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-02183-1_4
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-01055-2
Online ISBN: 978-3-031-02183-1
eBook Packages: Synthesis Collection of Technology (R0)eBColl Synthesis Collection 11