Reliability and Validity of Cervical Auscultation: A Controlled Comparison Using Video fluoroscopy

Abstract

Cervical auscultation is experiencing a renaissance as an adjunct to the clinical swallowing assessment. It is a controversial technique with a small evidence base. We have aimed to establish whether cervical auscultation interpretation is based on the actual sounds heard or, in practice, influenced by information gleaned from other aspects of the clinical assessment, medical notes, or previous knowledge. We sought to determine (a) rater reliability and its impact on the clinical value of cervical auscultation and (b) how judgments compare with the “gold standard”: videofluoroscopy. Swallow sounds were computer recorded via a Littmann stethoscope. Sounds were sampled from 10 healthy control swallows with no aspiration/penetration and 10 patient swallows with aspiration/penetration, all recorded during simultaneous videofluoroscopy. The system generated sound quality similar to “live” bedside listening, a feature rarely seen in cervical auscultation studies. The 20 sound clips were classified as “normal” or “abnormal” by 19 volunteer speech–language pathologists with experience in cervical auscultation. After at least four weeks, 11 of these judges rated the sounds rerandomized on a new CD. Intrarater reliability kappa ranged from −0.12 to 0.71. Individual reliability did not correlate with years of experience, practice pattern, or frequency of use. Interrater reliability kappa = 0.17. Comparison with radiologically defined aspiration/penetration yielded 66% specificity, 62% sensitivity, and majority consensus gave 90% specificity, 80% sensitivity. There was a significant relationship between individual reliability and true positive rate (r_s = 0.623, p = 0.040). The reliability of individual judges varied widely and thus, inevitably, agreement between judges was poor. Validity is dependent upon reliability: Improving the poor raters would improve the overall accuracy of this technique in predicting abnormality in swallowing. The group consensus correctly identified 17 of the 20 clips so we may speculate that the swallow sound contains audible cues that should in principle permit reliable classification.

Appendices

Appendix 1

Questions Relating to Rater Experience

Q1. How many years have you been working in the field of dysphagia years

Q2. How many years have you been performing Cervical Auscultation?

□ Not performing Cervical Auscultation

□ Training course experience only

□ Less than 1 year

□ 1-2 years

□ 2-3 years

□ 3-5 years

□ 5-10 years

□ 10+ years

Q3. What has been your level of training?

□ No training whatsoever

□ Limited informal “on the job” training

□ Self education

□ Established work based “in-service” training program

□ Attended Cervical Auscultation training course

Please specify

□ Other

Please specify

Q4. On average, how many examinations with Cervical Auscultation do you perform per week?

□ I do not perform Cervical Auscultation at my facility

□ Less than 1

□ 1-2

□ 2-3

□ 3-5

□ 5-10

□ 10+

Q5. How are you using Cervical Auscultation at your facility? (Tick all applicable)

□ Ad-hoc service/nothing formal in place

□ Occasional assessments only

□ Cervical Auscultation assessment and diagnostic clinic for inpatients and outpatients

□ Therapy and biofeedback

□ On hospital wards, for example ITU

□ Other

Please specify

Q6. What is the procedure of Cervical Auscultation administration?

□ No formal protocol/administration procedure

□ Therapists involved have their own protocol/set of procedures

□ Department staff has developed some protocol/set of procedures, although this is not rigidly adhered to

□ Department staff has developed a protocol with strict guidelines for administration procedures, consistencies and volumes of bolus

□ Protocol guidelines have been taken from literature and modified to suit the departmental requirements

□ Other

Please specify

Q7. How are you rating Cervical Auscultation at present?

□ No formal rating scale in place/therapist’s own narrative comments

□ Department rating scale (non-standardized)

□ Department rating scale (standardized/some validity and reliability measures taken)

□ Published rating scale from literature, for example:

Please specify

□ Modified published rating scale

Please specify

□ Other

Please specify

Q8. How would you rate your level of experience?

□ No experience

□ Novice/just starting

□ Skilled novice/still need supervision

□ Skilled clinician ready for independence

□Working independently but with frequent peer review by senior staff

□ Independent and confident in own abilities

□ Specialist in this field

Q9. What is the practice pattern for Cervical Auscultation?

□ Perform Cervical Auscultation at all swallow examinations but do not record findings

□ Perform Cervical Auscultation at all swallow examinations and record in SLT notes

□ Perform Cervical Auscultation at all swallow examinations and record in all notes

□ Perform Cervical Auscultation only at initial swallow examination but do not record findings

□ Perform Cervical Auscultation only at initial swallow examinations and record in SLT notes

□ Perform Cervical Auscultation only at initial swallow examinations and record in all notes

□ Perform Cervical Auscultation only when unsure of clinical findings but do not record findings

□ Perform Cervical Auscultation only when unsure of clinical findings and record in SLT notes

□ Perform Cervical Auscultation only when unsure of clinical findings and record in all notes

□ Perform Cervical Auscultation only when suspect silent aspiration but do not record findings

□ Perform Cervical Auscultation only when suspect silent aspiration and record in SLT notes

□ Perform Cervical Auscultation only when suspect silent aspiration and record in all notes

□ Not performing Cervical Auscultation

□ Other

Please specify

Q10. Who participates in the Cervical Auscultation assessment?

□ 1 SLT experienced in Cervical Auscultation

□ 2 SLTs experienced in Cervical Auscultation

□ 1 SLT experienced and 1 SLT training in Cervical Auscultation

□ 1 SLT and 1 student

□ Other Please specify

Q11. Aetiology of patient(s) receiving Cervical Auscultation? (Tick all applicable)

□ Head and neck cancer/ENT

□ Stroke

□ Pulmonary/COPD

□ General medical

□ Closed head injury

□ Dementia/Alzheimer’s

□ Neurosurgery

□ Progressive neurological disorder Please specify

□ Neurological developmental, i.e., adults with learning disability

□ Psychological component

□ Other, nonspecific swallow complaint Please specify

Appendix 2

Instructions for Raters

• 1. Before listening to the CD, could you please fill out the demographics questionnaire relating to relevant experience and current practice in your facility. Please add any additional information you feel is relevant when asked to specify, and if you select the category Other, please clarify with a short comment.

• 2. The sound quality is best if you listen through headphones. You can use a PC or a standard CD player but the speakers are often not too good and we have optimized the sound quality to try to replicate the use of a standard stethoscope. Make sure the shuffle switch is not on, so the swallows are played in order 1–20. Each track is approximately 8 seconds long with a silence of 12 seconds to space the tracks. The swallow may occur anywhere in the 8 seconds. The microphone was very sensitive and so you will often hear voices; please try to ignore these and concentrate on the swallow sounds.

• 3. Try not to spend too much time deliberating over answers as in real life you would only hear a swallow once. Listen to each track as often as you feel is necessary. I must highlight that the completion of the questionnaire and the rating is an individual task, please do not confer with colleagues. The project involves interrater reliability not group consensus.

• 4. Tick or cross the box to rate each swallow as definitely normal or probably normal or probably abnormal or definitely abnormal. Please write down any comments that you feel are relevant, e.g., judgments you would make at bedside based on what you heard, bolus type, aspiration/penetration/pooling/residue. etc.

• 5. Please attempt every question, do not leave any blank.

  If a question states tick all applicable, please select more than one answer if you feel it is relevant.

• 6. Although the questionnaire is based on ticking a box, please do quality judgments or list alternatives that are not provided.

• 7. Please, return your questionnaire and rating form, using the envelope provided, to Newcastle University. The due date for returning them is Friday 31 May. If you have any queries, please do not hesitate to contact me at Newcastle University by telephoning 0191 222 6279 or via e-mail: paula.leslie@ncl.ac.uk. If you do not want to give your details that is fine. All data is confidential and will be made anonymous for analysis and publication.

• 8. Ethical Approval and Consent

  Written informed consent was obtained for all participants involved in the study. For those patients with language disorders, consent was obtained from the next of kin and the researcher explained the study as far as possible to the patient with gesture and modified language. The Joint Ethics Committee for Newcastle and North Tyneside Health Authority, the University of Newcastle, and the University of Northumbria at Newcastle granted ethical approval for the study.