Inter-rater Reliability of the Dysphagia Outcome and Severity Scale (DOSS): Effects of Clinical Experience, Audio-Recording and Training

Abstract

The Dysphagia Outcome and Severity Scale (DOSS) is widely used to measure dysphagia severity based on videofluoroscopy (VFSS). This study investigated inter-rater reliability (IRR) of the DOSS. It also determined the effect of clinical experience, VFSS audio-recording and training on DOSS IRR. A quantitative prospective research design was used. Seventeen speech and language pathologists (SLPs) were recruited from an acute teaching hospital, Dublin (> 3 years’ VFSS experience, n = 10) and from a postgraduate dysphagia programme in a university setting (< 3 years’ VFSS experience; n = 7). During testing, participants viewed eight VFSS clips (5 with audio-recording). Each VFSS clip was independently rated using the DOSS scale. Four weeks later, the less experienced group attended a 1-h training session on DOSS rating after which DOSS IRR was re-tested. Cohen’s kappa co-efficient was used to establish IRR. IRR of the DOSS presented only fair agreement (κ = 0.36, p < 0.05). DOSS IRR was significantly higher (κ = 0.342) within the more experienced SLP group, compared to the less experienced SLP group (κ = 0.298) (p < 0.05). DOSS IRR was significantly higher in VFSS clips with audio-recording (κ = 0.287) compared to VFSS clips without audio-recording (κ = − 0.0395) (p < 0.05). IRR of the DOSS pre-training (κ = 0.328) was significantly better comparing to post-training (κ = 0.218) (p < 0.05). Findings raise concerns as the DOSS is frequently used in clinical practice to capture dysphagia severity and to monitor changes.