Inter-rater agreement in pPOSSUM scores of geriatric trauma patients: a prospective evaluation

Abstract

Purpose

Risk prediction models are widely used in the perioperative setting to identify high-risk patients who may benefit from additional care and to aid clinical decision-making. pPOSSUM is such a prediction model, however, little is known about the inter-rater agreement when scoring subjective parameters. This study assessed the inter-rater agreement between clinicians of different specialties and work-level when scoring 30 clinical case reports of geriatric hip fracture patients with pPOSSUM.

Methods

Eighteen clinicians of the department of Surgery (three specialists, four residents), Anaesthesiology (four specialists, two residents) and Emergency Medicine (three specialists, two residents) who were familiar with the pPOSSUM scoring system were asked to calculate the scores. The kappa statistic and the statistical method of Fleiss were used to analyse inter-rater agreement.

Results

The response rate was 100%. Among surgeons, Anaesthesiologists and Emergency department doctors (ED), the overall mean kappa values were 0.42, 0.08 and 0.20, respectively. Among surgery, anaesthesiology and ED residents the overall mean kappa values were 0.21, 0.33 and 0.37, respectively. Within the department of Surgery, Anaesthesiology and Emergency Medicine the overall mean kappa values were 0.23, 0.12 and 0.22, respectively. An overall mean kappa value of 0.19 was seen among all specialists. All residents had an overall mean kappa value of 0.21 and all clinicians had an overall mean kappa value of 0.21.

Conclusion

The overall inter-rater agreement of clinicians and interdisciplinary agreement when scoring geriatric hip fracture patients with pPOSSUM was low and prone to subjectivity in our study. A higher work-experience level did not lead to better agreement. When pPOSSUM is calculated without clinical assessment by the same clinician, caution is advised to prevent over-reliance on the pPOSSUM risk prediction model.

Level of evidence

III.

Appendices

Appendix A

pPOSSUM assessment tool.

	Score
	1	2	4	8
Physiological parameters
Age	≤ 60	61–70	≥ 71
Cardiac function	No failure	Diuretic. digoxin. anti-angina or hypertensive therapy	Peripheral oedema or warfarin therapy	Raised central venous pressure or cardiomegaly
Respiratory function	No dyspnoea	Dyspnoea on exertion. mild obstructive airway disease	Limiting dyspnoea or moderate obstructive airway disease	Dyspnoea at rest (rate ≥ 30/min) fibrosis or consolidation on x-ray
Electrocardiogram	normal		Atrial fibrillation (rate 60–90)	Any abnormal rhythm or ≥ 5 ectopics/minute or Q waves or ST/T changes
Systolic blood pressure (mmHg)	110–130	131–170. 100–109	≥ 171. 90–99	≤ 89
Pulse rate (rate/minute)	50–80	81–100. 40–49	101–120	≥ 121. ≤ 39
Haemoglobin (g/dl)	13–16	11.5–12.9. 16.1–17.0	10.0–11.4	≤ 9.9. ≥ 18.1
White blood cells (109 cells/L)	4–10	10.1–20. 3.1–4.0	≥ 2.0. ≤ 3.0
Urea (mmol/L)	≤ 7.5	7.6–10	10.1–15.0	≥ 15.1
Sodium (mmol/L)	≥ 136	131–135	126–130	≤ 125
Potassium (mmol/L)	3.5–5.5	3.2–3.4. 5.1–5.3	2.9–3.1. 5.4–5.9	≥ 2.8. ≥ 6.0
Glasgow coma scale	15	12–14	9–11	≤ 8
Operative parameters
Operation type	Minor	Moderate	Major	Complex major
Number of procedures	1		2	> 2
Operative blood loss (ml)	≤ 100	101–500	501–999	≥ 1000
Peritoneal contamination	None	Minor (serous fluid)	Local pus	Free bowel content
Malignancy status	None	Minor (serous fluid)	Nodal metastasis	Distant metastasis
Mode of surgery	elective		urgent	Emergency (within 2 h)

pPOSSUM portsmouth physiological and operative severity score for the enumeration of mortality

Appendix B

pPOSSUM scoring form.

pPOSSUM Portsmouth Physiological and Operative Severity Score for the enumeration of Mortality, Rx prescription, JVP jugular venous pressure, COAD chronic obstructive airway disease, AF atrial fibrillation, CEPOD confidential enquiry into perioperative deaths