Abstract
Background
Blind, randomised trials are conceived as the gold standard in clinical research, but this ideal, in its strict sense, can rarely be achieved in surgical settings. One way to strengthen the study design is to check for observer bias in the assessment and evaluation of surgical outcome.
Method
In a randomised, prospective trial comparing nasogastric versus gastrostomy tubes the primary endpoint was the subjective inconvenience induced by the tube system and was assessed in the context of a standardised face-to-face interview. These interviews were tape-recorded on a pocket memo. Two independent raters listened to these interviews and judged—on the basis of how the interviewer formulated the questions—which treatment arm they thought the patients were assigned to and how confident they were in their judgement.
Results
The overall proportion of correct judgements was 50.5% for rater 1 and 53.2% for rater 2. In other words, both judgement performances were not greater than chance. Nevertheless, the raters’ confidence in their judgements increased significantly (P<0.05) in the course of the rating procedure, whereas the actual proportion of correct judgements did not. There was no overlap between the two raters [kappa = 0.022, not significant (NS)] and between actual group assignment and both raters’ judgements (kappa = 0.012, NS and kappa = 0.110, NS).
Conclusion
The two independent raters were not able to detect systematic variations in the interviewing style that were contingent on treatment arm assignment. This gives further credence to the results of the randomised trial showing greater patient-reported discomfort and inconvenience with the nasogastric tube than with the gastrostomy tube. The present report describes a feasible method to monitor subtle biases that may occur in trial settings. This helps to strengthen the design of randomised clinical trials in surgery.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Pocock SJ (1983) Blinding and placebos. Clinical trials. Wiley, Chichester New York Brisbane Toronto Singapore, pp 90–99
Friedman LM, Furberg CD, DeMets DL (1982) Fundamentals of clinical trials. Wright–PSG, Boston Bristol London
McPherson K, Britton AR, Wennberg JE (1997) Are randomized controlled trials controlled? Patient preferences and unblind trials. J R Soc Med 90:652–656
Lorenz W, et al. (1994) Incidence and clinical importance of perioperative histamine release: randomised study of volume loading and antihistamines after induction of anaesthesia. Lancet 343:933–940
Lorenz W (1999) Blinding in surgery. Paper presented at the SACRE meeting, Hamburg
Russell RCG, Johnson AG, Lorenz W (1999) Blinding in surgical trials. Paper presented at the SACRE meeting, Hamburg
Majeed AW, et al. (1996) Randomised, prospective, single-blind comparison of laparoscopic versus small-incision cholecystectomy. Lancet 347:989–994
Nilsson G, Larsson S, Johnsson F (2000) Randomized clinical trial of laparoscopic versus open fundoplication: blind evaluation of recovery and discharge period. Br J Surg 87:873–878
Moseley JB, et al. (2002) A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med 347:81–88
Swank DJ, et al. (2003) Laparoscopic adhesiolysis in patients with chronic abdominal pain: a blinded randomised controlled multi-centre trial. Lancet 361:1247–1251
Lorenz W, et al. (1999) Second step: testing—outcome measurements. World J Surg 23:768–780
Koller M, Lorenz W (2002) Quality of life: a deconstruction for clinicians. J R Soc Med 95:481–488
Rosenthal R, Rubin DB (1978) Interpersonal expectancy effects: the first 345 studies. Behav Brain Sci 3:377–415
Beecher HK (1961) Surgery as placebo. JAMA 176:1102–1107
Johnson AG (1994) Surgery as a placebo. Lancet 344:1140–1142
Rosenthal R (1966) Blind and minimized contact. In: Rosenthal R (ed) Experimenter effects in behavioral research. Appleton-Century-Crofts, New York, pp 367–379
Hoffmann S, et al. (2001) Nasogastric tube versus gastrostomy tube for gastric decompression in abdominal surgery: a prospective, randomized trial comparing patients’ tube-related inconvenience. Langenbecks Arch Surg 386:402–409
Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20:37–46
Bühl A, Zöfel P (2000) SPSS version 10: Einführung in die moderne Datenanalyse unter Windows. Addison-Wesley, Münich
Troidl H, et al. (1987) Quality of life: an important endpoint both in surgical practice and research. J Chronic Dis 40:523–528
Blazeby JM (2001) Measurement of outcome. Surg Oncol 10:127–133
Westhoff G (1993) Handbuch psychosozialer Messinstrumente. Hogrefe, Göttingen
Bowling A (2001) Measuring disease. A review of disease-specific quality of life measurement scales. Open University Press, Buckingham
Koller M, et al. (1996) Symptom reporting in cancer patients: the role of negative effect and experienced social stigma. Cancer 77:983–995
Sudman S, Bradburn N, Schwarz N (1996) Thinking about answers: the application of cognitive processes to survey methodology. Jossey-Bass, San Francisco
Schwarz N (1994) Judgment in a social context: biases, shortcomings, and the logic of conversation. Adv Exp Soc Psychol 26:123–162
Martin DK, McKneally MF (1998) Qualitative research. In: Troidl H, et al. (eds) Surgical research. Basic principles and clinical practice, 3rd edn. Springer, New York pp 235–241
Nies C, et al. (2001) Outcome nach minimal-invasiver Chirurgie. Qualitative Analyse und Bewertung der klinischen Relevanz von Studienendpunkten durch Patient und Arzt. Chirurg 72:19–29
Dey I (1957) Qualitative data analysis. Routledge, London
Festinger L (1957) A theory of cognitive dissonance. Stanford University Press, Stanford
Koller M, Lorenz W (2002) Chirurgisches Entscheiden und Handeln: Erklärungen und Forschungsperspektiven der Sozialpsychologie. Chirurg 73:846–854
Acknowledgements
Preparation of the manuscript was facilitated by a grant from the German Ministry of Health, FB 2-43332-70/6. We thank our independent raters and gratefully acknowledge the computer support of Susanne Hainbach.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Koller, M., Hoffmann, S., Rothmund, M. et al. Checking for interviewer bias in outcome assessment: a method for strengthening the design of prospective, randomised trials in surgery. Langenbecks Arch Surg 389, 92–96 (2004). https://doi.org/10.1007/s00423-003-0428-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00423-003-0428-9