Skip to main content

Table 1 Performance metrics

From: Sorting Through the Safety Data Haystack: Using Machine Learning to Identify Individual Case Safety Reports in Social-Digital Media

Name Value
True positive (tp) No. of true positivesa
True negative (tn) No. of true negativesa
False positive (fp) No. of false positivesa
False negative (fn) No. of false negativesa
Accuracy (Acc) \(\frac{{{\text{tp}} + {\text{tn}}}}{{{\text{tp}} + {\text{fp}} + {\text{tn}} + {\text{fn}}}}\)
Gwet AC1 \(\frac{{{\text{Acc}} - e(\pi )}}{1 - e(\pi )}\)
e(π) \(\left\{ {\frac{{(2{\text{tp}} + {\text{fp}} + {\text{fn}})/2}}{{{\text{tp}} + {\text{fp}} + {\text{tn}} + {\text{fn}}}}} \right\}^{2} + \left\{ {\frac{{(2{\text{tn}} + {\text{fp}} + {\text{fn}})/2}}{{{\text{tp}} + {\text{fp}} + {\text{tn}} + {\text{fn}}}}} \right\}^{2}\)
Area under the curve Trapezoidal method [30]
  1. SME subject matter expert
  2. aTrue positive, negative, etc are based on SME-determined ‘ground truth’