Appendix
A summary of the statistical techniques used to derive Spiegelhalter–Knill–Jones weightings is outlined by Seymour et al. [13].
To employ the Spiegelhalter–Knill–Jones method, which is an extension of logistic regression, all explanatory variables as well as the outcome variable must be binary. The LAg assay is the outcome variable and a cut-off of ODn ≤ 1.5 is used, with those ≤ 1.5 regarded as being recent infections. Algorithm 15.1 is presented in the data set as a binary variable (recent or non-recent), so no division or recoding is needed. A cut-off for the ARCHITECT S/CO of 250 was used in this study, with values < 250 regarded as recent infections.
Briefly as outlined by Seymour et al. [13], the mathematical derivation of Spiegelhalter–Knill–Jones weightings:
$${\text{Posterior odds }}={\text{ prior odds }} \times {\text{ LR of variable 1 }} \times {\text{ LR of variable 2 }} \times {\text{ }} \cdots {\text{ }} \times {\text{ LR of variable }}n,$$
where
-
(a)
the posterior odds are the predicted odds of the outcome in an individual;
-
(b)
the prior odds are the odds of the outcome in the population under study;
-
(c)
LR is the likelihood ratio:
$${\text{LR }}={\text{ }}\left( {{\text{sensitivity}}} \right)/\left( {{\text{1}} - {\text{specificity}}} \right){\text{ for a positive test result }}\left( {{\text{in this study}},{\text{ recenct}}} \right),{\text{ OR}}$$
$${\text{LR }}={\text{ }}\left( {{\text{1}} - {\text{sensitivity}}} \right)/\left( {{\text{specificity}}} \right){\text{ for a negative test result }}\left( {{\text{in this study}},{\text{ non}} - {\text{recent}}} \right).$$
$${\text{1}}00 \times {\text{ln}}\left( {{\text{posterior odds}}} \right){\text{ }}={\text{ 1}}00 \times {\text{ln}}\left( {{\text{prior odds}}} \right){\text{ }}+{\text{ 1}}00 \times {\text{ln}}\left( {{\text{LR of variable 1}}} \right){\text{ }}+{\text{ 1}}00 \times {\text{ln}}\left( {{\text{LR of variable 2}}} \right){\text{ }}+{\text{ }} \ldots {\text{ }}+{\text{ 1}}00 \times {\text{ln}}\left( {{\text{LR of variable }}n} \right).$$
$${\text{Total score }}\left( T \right){\text{ }}={\text{ starting score }}+{\text{ weight of evidence of variable 1 }}+{\text{ weight of evidence of variable 2 }}+ \cdots +{\text{ weight of evidence of variable }}n.$$
-
This equation assumes that the variables are independent of one another, which is unrealistic in many cases, so the Spiegelhalter–Knill–Jones method calculates adjusted weights to account for dependence. Adjusted weights are obtained by entering the raw weights of evidence as independent variables in a logistic regression equation. This produces the final form of the predictive equation:
$${\text{Total score }}\left( T \right){\text{ }}={\text{ starting score }}+{\text{ adjusted weight of evidence for variable 1 }}+{\text{ adjusted weight of evidence for variable 2 }}+ \cdots +{\text{ adjusted weight of evidence for variable }}n.$$
$${\text{Predicted probability }}\left( \% \right)~={\text{ }}T{\text{ }}={\text{ 1}}00{\text{ln}}\left( {p/{\text{1}} - p} \right).$$
Therefore, Ln(p/1 − p) = T/100:
$$\begin{gathered} (p/p - 1)={e^{(T/100)}} \hfill \\ P={e^{(T/100)}}(1 - p)={e^{(T/100)}} - {e^{(T/100)}}p \hfill \\ P+{e^{(T/100)}}p={e^{(T/100)}} \hfill \\ P(1+{e^{(T/100)}})={e^{(T/100)}} \hfill \\ P={e^{(T/100)}}/(1+{e^{(T/100)}}) \hfill \\ {\text{Predicted}}\;{\text{probability}}\;(\% )={e^{(T/100)}}/(1+{e^{(T/100)}}) \times 100 \hfill \\ \end{gathered}$$
The application of the Spiegelhalter–Knill–Jones method for the two variables studied (Algorithm 15.1 and ARCHITECT S/CO) is shown below (Table 3).
Table 3 Derivation of scoring system based on Spiegelhalter–Knill–Jones method
LR and Crude weights were calculated as shown below. Logistic coefficients were obtained by logistic regression analysis of the raw data using SPSS 22.
Individual markers
ARCHITECT S/CO versus LAg as the gold standard
Sensitivity = 21/30 = 70%.
Specificity = 112/118 = 94.9%.
The observed risk of being recent when ARCHITECT < 250 (suggests recency) is 21/27 = 77.8% (as shown in Table 2 in the manuscript).
Likelihood ratios for recency:
Sensitivity/100 − Specificity = 70.0/100 − 94.9 = 70.0/5.1 = 13.73.
Ln13.73 = 2.62.
2.62 × 100 = 262 = weighting to be applied to a recent test.
Likelihood ratios for non- recency:
100 − Sensitivity/Specificity = 100 − 70.0/94.9 = 30.0/94.9 = 0.316.
Ln0.316 = − 1.152.
− 1.152 × 100 = − 115.2 = weighting to be applied to a non-recent test.
Algorithm 15.1 versus LAg as the gold standard
Sensitivity = 39/76 = 51.3%.
Specificity = 323/345 = 93.6%.
The observed risk of being recent when Algorithm 15.1 suggests that recency is 39/61 = 63.9% (as shown in Table 2 in the manuscript).
Likelihood ratios for recency:
Sensitivity/100 − Specificity = 51.3/100 − 93.6 = 51.3/6.4 = 8.016.
Ln8.016= 2.081.
2.081 × 100 = 208 = weighting to be applied to a recent test.
Likelihood ratios for non-recency:
100 − sensitivity/specificity = 100 − 51.3/93.6 = 48.7/93.6 = 0.520.
Ln0.520= − 0.654.
− 0.654 × 100 = − 65.4 = weighting to be applied to a non-recent test.
Predictive analysis for the two markers—ARCHITECT S/CO and Algorithm 15.1
ARCHITECT S/CO (recent) and Algorithm 15.1 (recent) versus LAg as the gold standard
See Table 4.
$$\begin{gathered} {\text{Predicted probability }}\left( \% \right)~={\text{ 1}}00{\text{ln}}\left( {p/{\text{1}} - p} \right){\text{ }}={\text{ }} - {\text{117 }}+{\text{ }}\left( {0.00{\text{9}} \times {\text{1}}00} \right){\text{ }}\left( {{\text{262}}} \right){\text{ }}+{\text{ }}\left( {0.0{\text{12}} \times {\text{1}}00} \right){\text{ }}\left( {{\text{2}}0{\text{8}}} \right) \hfill \\ ~={\text{ }} - {\text{117 }}+{\text{ 235}}.{\text{8 }}+{\text{ 249}}.{\text{6 }}={\text{ 368}}.{\text{4}} \hfill \\ \end{gathered}$$
$$\begin{gathered} {\text{Ln}}\left( {p/{\text{1}} - p} \right)~~~={\text{ 3}}.{\text{684}} \hfill \\ \left( {p/{\text{1}} - p} \right)~~~~~~~={\text{ 39}}.{\text{815}}. \hfill \\ \end{gathered}$$
Table 4 Crosstabulation of LAg with ARCHITECT (S/CO < 250)
Predicted risk of being recent (as measured by the LAg as gold standard) if both tests suggest the recent infection = 39.815/1 + 39.805 = 39.815/40.815 = 97.6% (as shown in Table 2 in the manuscript).
ARCHITECT S/CO (non-recent) and Algorithm 15.1 (non-recent) versus LAg as the gold standard
See Table 5.
$$\begin{gathered} {\text{Predicted probability }}\left( \% \right)~={\text{ 1}}00{\text{ln}}\left( {p/{\text{1}} - p} \right){\text{ }}={\text{ }} - {\text{117 }}+{\text{ }}\left( {0.00{\text{9 }} \times {\text{1}}00} \right){\text{ }}\left( { - {\text{115}}} \right){\text{ }}+{\text{ }}\left( {0.0{\text{12 }} \times {\text{ 1}}00} \right){\text{ }}\left( { - {\text{65}}} \right) \hfill \\ ={\text{ }} - {\text{117 }}+{\text{ }}\left( { - {\text{1}}0{\text{3}}.{\text{5}}} \right){\text{ }}+{\text{ }}\left( { - {\text{78}}} \right){\text{ }}={\text{ }} - {\text{298}}.{\text{5}} \hfill \\ {\text{Ln}}\left( {p/{\text{1}} - p} \right)~~~={\text{ }} - {\text{2}}.{\text{985}} \hfill \\ \left( {p/{\text{1}} - p} \right)~~~~~~~={\text{ }}0.0{\text{5}}0{\text{5}}. \hfill \\ \end{gathered}$$
Table 5 Crosstabulation of LAg with Algorithm 15.1
Predicted risk of being non-recent (as measured by the LAg as gold standard) if both tests suggest non-recent infection = 0.0505/1 + 0.0505 = 0.0505/1.0505 = 4.8% (as shown in Table 2 in the manuscript).