Langenbeck's Archives of Surgery

, Volume 389, Issue 2, pp 92–96 | Cite as

Checking for interviewer bias in outcome assessment: a method for strengthening the design of prospective, randomised trials in surgery

  • M. Koller
  • S. Hoffmann
  • M. Rothmund
  • W. Lorenz
  • U. Plaul
Original Article
First Online:
Received:
Accepted:

Abstract

Background

Blind, randomised trials are conceived as the gold standard in clinical research, but this ideal, in its strict sense, can rarely be achieved in surgical settings. One way to strengthen the study design is to check for observer bias in the assessment and evaluation of surgical outcome.

Method

In a randomised, prospective trial comparing nasogastric versus gastrostomy tubes the primary endpoint was the subjective inconvenience induced by the tube system and was assessed in the context of a standardised face-to-face interview. These interviews were tape-recorded on a pocket memo. Two independent raters listened to these interviews and judged—on the basis of how the interviewer formulated the questions—which treatment arm they thought the patients were assigned to and how confident they were in their judgement.

Results

The overall proportion of correct judgements was 50.5% for rater 1 and 53.2% for rater 2. In other words, both judgement performances were not greater than chance. Nevertheless, the raters’ confidence in their judgements increased significantly (P<0.05) in the course of the rating procedure, whereas the actual proportion of correct judgements did not. There was no overlap between the two raters [kappa = 0.022, not significant (NS)] and between actual group assignment and both raters’ judgements (kappa = 0.012, NS and kappa = 0.110, NS).

Conclusion

The two independent raters were not able to detect systematic variations in the interviewing style that were contingent on treatment arm assignment. This gives further credence to the results of the randomised trial showing greater patient-reported discomfort and inconvenience with the nasogastric tube than with the gastrostomy tube. The present report describes a feasible method to monitor subtle biases that may occur in trial settings. This helps to strengthen the design of randomised clinical trials in surgery.

Keywords

Quality of life Clinical trials Surgery Bias reduction Experimenter bias 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

Preparation of the manuscript was facilitated by a grant from the German Ministry of Health, FB 2-43332-70/6. We thank our independent raters and gratefully acknowledge the computer support of Susanne Hainbach.

References

  1. 1.
    Pocock SJ (1983) Blinding and placebos. Clinical trials. Wiley, Chichester New York Brisbane Toronto Singapore, pp 90–99Google Scholar
  2. 2.
    Friedman LM, Furberg CD, DeMets DL (1982) Fundamentals of clinical trials. Wright–PSG, Boston Bristol LondonGoogle Scholar
  3. 3.
    McPherson K, Britton AR, Wennberg JE (1997) Are randomized controlled trials controlled? Patient preferences and unblind trials. J R Soc Med 90:652–656PubMedGoogle Scholar
  4. 4.
    Lorenz W, et al. (1994) Incidence and clinical importance of perioperative histamine release: randomised study of volume loading and antihistamines after induction of anaesthesia. Lancet 343:933–940CrossRefPubMedGoogle Scholar
  5. 5.
    Lorenz W (1999) Blinding in surgery. Paper presented at the SACRE meeting, HamburgGoogle Scholar
  6. 6.
    Russell RCG, Johnson AG, Lorenz W (1999) Blinding in surgical trials. Paper presented at the SACRE meeting, HamburgGoogle Scholar
  7. 7.
    Majeed AW, et al. (1996) Randomised, prospective, single-blind comparison of laparoscopic versus small-incision cholecystectomy. Lancet 347:989–994PubMedGoogle Scholar
  8. 8.
    Nilsson G, Larsson S, Johnsson F (2000) Randomized clinical trial of laparoscopic versus open fundoplication: blind evaluation of recovery and discharge period. Br J Surg 87:873–878CrossRefPubMedGoogle Scholar
  9. 9.
    Moseley JB, et al. (2002) A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med 347:81–88CrossRefPubMedGoogle Scholar
  10. 10.
    Swank DJ, et al. (2003) Laparoscopic adhesiolysis in patients with chronic abdominal pain: a blinded randomised controlled multi-centre trial. Lancet 361:1247–1251CrossRefPubMedGoogle Scholar
  11. 11.
    Lorenz W, et al. (1999) Second step: testing—outcome measurements. World J Surg 23:768–780PubMedGoogle Scholar
  12. 12.
    Koller M, Lorenz W (2002) Quality of life: a deconstruction for clinicians. J R Soc Med 95:481–488PubMedGoogle Scholar
  13. 13.
    Rosenthal R, Rubin DB (1978) Interpersonal expectancy effects: the first 345 studies. Behav Brain Sci 3:377–415Google Scholar
  14. 14.
    Beecher HK (1961) Surgery as placebo. JAMA 176:1102–1107Google Scholar
  15. 15.
    Johnson AG (1994) Surgery as a placebo. Lancet 344:1140–1142CrossRefPubMedGoogle Scholar
  16. 16.
    Rosenthal R (1966) Blind and minimized contact. In: Rosenthal R (ed) Experimenter effects in behavioral research. Appleton-Century-Crofts, New York, pp 367–379Google Scholar
  17. 17.
    Hoffmann S, et al. (2001) Nasogastric tube versus gastrostomy tube for gastric decompression in abdominal surgery: a prospective, randomized trial comparing patients’ tube-related inconvenience. Langenbecks Arch Surg 386:402–409PubMedGoogle Scholar
  18. 18.
    Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20:37–46Google Scholar
  19. 19.
    Bühl A, Zöfel P (2000) SPSS version 10: Einführung in die moderne Datenanalyse unter Windows. Addison-Wesley, MünichGoogle Scholar
  20. 20.
    Troidl H, et al. (1987) Quality of life: an important endpoint both in surgical practice and research. J Chronic Dis 40:523–528PubMedGoogle Scholar
  21. 21.
    Blazeby JM (2001) Measurement of outcome. Surg Oncol 10:127–133PubMedGoogle Scholar
  22. 22.
    Westhoff G (1993) Handbuch psychosozialer Messinstrumente. Hogrefe, GöttingenGoogle Scholar
  23. 23.
    Bowling A (2001) Measuring disease. A review of disease-specific quality of life measurement scales. Open University Press, BuckinghamGoogle Scholar
  24. 24.
    Koller M, et al. (1996) Symptom reporting in cancer patients: the role of negative effect and experienced social stigma. Cancer 77:983–995PubMedGoogle Scholar
  25. 25.
    Sudman S, Bradburn N, Schwarz N (1996) Thinking about answers: the application of cognitive processes to survey methodology. Jossey-Bass, San FranciscoGoogle Scholar
  26. 26.
    Schwarz N (1994) Judgment in a social context: biases, shortcomings, and the logic of conversation. Adv Exp Soc Psychol 26:123–162Google Scholar
  27. 27.
    Martin DK, McKneally MF (1998) Qualitative research. In: Troidl H, et al. (eds) Surgical research. Basic principles and clinical practice, 3rd edn. Springer, New York pp 235–241Google Scholar
  28. 28.
    Nies C, et al. (2001) Outcome nach minimal-invasiver Chirurgie. Qualitative Analyse und Bewertung der klinischen Relevanz von Studienendpunkten durch Patient und Arzt. Chirurg 72:19–29PubMedGoogle Scholar
  29. 29.
    Dey I (1957) Qualitative data analysis. Routledge, LondonGoogle Scholar
  30. 30.
    Festinger L (1957) A theory of cognitive dissonance. Stanford University Press, StanfordGoogle Scholar
  31. 31.
    Koller M, Lorenz W (2002) Chirurgisches Entscheiden und Handeln: Erklärungen und Forschungsperspektiven der Sozialpsychologie. Chirurg 73:846–854Google Scholar

Copyright information

© Springer-Verlag 2004

Authors and Affiliations

  • M. Koller
    • 1
  • S. Hoffmann
    • 2
  • M. Rothmund
    • 2
  • W. Lorenz
    • 1
  • U. Plaul
    • 2
  1. 1.Institute of Theoretical SurgeryPhilipps-University MarburgMarburgGermany
  2. 2.Department of Visceral, Thoracic and Vascular SurgeryPhilipps-University MarburgMarburgGermany

Personalised recommendations