Skip to main content
SpringerLink
Log in

Die Validität von Routinedaten zur Qualitätssicherung

Eine qualitative systematische Übersichtsarbeit

The validity of routine data on quality assurance

A qualitative systematic review

  • Originalien
  • Published:
Der Chirurg Aims and scope Submit manuscript

Zusammenfassung

Hintergrund

Die Bewertung der Qualität ärztlichen Handelns ist eine legitime gesellschaftliche Forderung. Dabei wird weltweit nach verlässlichen Methoden der Qualitätsmessung gesucht. Häufig wird dabei die Qualität aus Abrechnungsdaten (sog. Routinedaten) ermittelt. In Deutschland hat die AOK dieses Verfahren umgesetzt.

Fragestellungen

1) Wie wird das AOK-Qualitätssystem von chirurgischen Chefärzten eingeschätzt? 2) Wie ist die Validität von Qualitätsaussagen, die aus Abrechnungsdaten hergeleitet werden?

Material und Methoden

Die vorliegende Arbeit wird in Anlehnung an das PRISMA-Statement für eine qualitative systematische Übersichtsarbeit erstellt. Zur Beantwortung der 1) Fragestellung führte der Berufsverband der Deutschen Chirurgen zwei Umfragen durch. Zur Beantwortung der 2) Frage erfolgte eine strukturierte Literaturrecherche in Anlehnung an das PICO-Format. Zusätzlich wurden zahlreiche Webseiten kontaktiert.

Ergebnisse

Insgesamt 95 % der antwortenden chirurgischen Chefärzte sind der Meinung, dass die AOK-Methodik, aus Abrechnungsdaten Qualitätsaussagen zu treffen, nicht objektiv ist. Ein Drittel wurde konkret falsch beurteilt. Aus der Literaturrecherche geht hervor, dass für AOK-Indikatoren inklusive des Elixhauser-Risikoscores, keine Validierung vorliegt. Insgesamt ist die Sensitivität von Indikatoren in der Literatur schlecht, wenn man eine gute Sensitivität als ≥ 80 < 90 % definiert (AQUA-Institut).

Schlussfolgerung

Qualitätsaussagen basierend allein auf Abrechnungsdaten sind nicht verlässlich.

Abstract

Background

The assessment of the quality of medical practice is a legitimate requirement by society. Reliable methods for measurement of the quality of performance are sought worldwide. Quality is often quantified by using administrative data and in Germany this method has been implemented by the health insurance company AOK.

Objectives

(1) How is the AOK quality system rated by senior consultant surgeons? (2) How valid are quality statements derived from administrative data?

Methods

This article was compiled following the PRISMA (i.e. preferred reporting items for systematic reviews and meta-analyses) statement for qualitative systematic reviews. In order to answer the first question the Professional Association of German Surgeons (Berufsverband der Deutschen Chirurgen) initiated two surveys and to answer the second question a structured literature search following the PICO (i.e. patient problem or population, intervention, comparison control or comparator and outcomes) format was initiated. In addition numerous websites were contacted.

Results

Of the responding senior consultant surgeons 95 % considered that the AOK method of quality measurement by administrative data is not objective. One third was definitely wrongly classified. The literature search revealed that no validation data exist for the AOK indicators, including the Elixhauser comorbidity risk score. Altogether, the sensitivity of indicators is poor when good sensitivity is defined by the Institute for Applied Quality Improvement and Research in Health Care (AQUA Institute) as ≥ 80 < 90 %.

Conclusions

Quality statements resulting from administrative data alone are unreliable.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Abb. 1

Literatur

  1. AOK-Bundesverband, FEISA, HELIOS Kliniken, WIDO: Qualitätssicherung der stationären Versorgung mit Routinedaten (QSR). Abschlussbericht. Bonn 2007 http://www.qualitaetssicherung-mit-routinedaten.de/imperia/md/qsr/publikationen/wido_kra_qsr-abschlussbericht_0407.pdf. Zugegriffen: 30. Nov. 2014

  2. Arah OA, Klazinga NS (2004) How safe is the safety paradigm? Qual Saf Health Care 3:226–232

    Article  Google Scholar 

  3. Aylin P, Bottle A, Majeed A (2007) Use of administrative data or clinical databases as predictors of risk of death in hospital: comparison of models. BMJ. doi:10.1136/bmj.39168.496366.55

  4. Berwick D (2015) Measuring surgical outcomes for improvement. Was Codman wrong?. JAMA 313:469–470

    Article  PubMed  CAS  Google Scholar 

  5. Best WR, Khuri SF, Phelan M, Hur K, Henderson WG, Demakis JG, Daley J (2002) Identifying patient preoperative risk factors and postoperative adverse events in administrative databases: results from the Department of Veterans Affairs National Surgical Quality Improvement Program. J Am Coll Surg 3:257–266

    Article  Google Scholar 

  6. Borzecki AM, Kaafarani H, Cevasco M (2011) How valid is the AHRQ Patient Safety Indicator „postoperative hemorrhage or hematoma“? J Am Coll Surg 6:946–953.e1–2

    Article  Google Scholar 

  7. Charlson M, Szatrowsk T, Peterson J, Gold J (1994) Validation of a comorbidity index. J Clin Epidemiol 47:1245–1251

    Article  PubMed  CAS  Google Scholar 

  8. Committee on quality of health care in America (2000) To err is human. In: Kohn L, Corrigan M, Donaldson M (Hrsg) Building a safer health system. National Academy Press, Washington DC

    Google Scholar 

  9. Committee on quality of health care in America (2001) Crossing the quality chasm. A new health system for the 21st century. National Academy Press, Washington DC

    Google Scholar 

  10. Donabedian A (1966) Evaluating the quality of medical care. Milbank Mem Fund 3(Suppl):166–206

    Article  Google Scholar 

  11. Elixhauser A, Steiner C, Harris DR, Coffey RM (1998) Comorbidity measures for use with administrative data. Med Care 1:8–27

    Article  Google Scholar 

  12. Etzioni D, Wasif N, Dueck AC, Cima RR, Hohmann SF, Naessens JM, Mathur AK, Habermann E (2015) Association of hospital participation in a surgical outcomes monitoring program with inpatient complications and mortality. JAMA 313:505–511

    Article  PubMed  CAS  Google Scholar 

  13. Groene O, Skau, Jutta KH, Frolich A (2008) An international review of projects on hospital performance assessment. Int J Qual Health Care 3:162–171

  14. Gute Praxis Sekundäranalyse (GPS) – Leitlinien und Empfehlungen 3. Fassung 2012 http://www.gesundheitsforschung-bmbf.de/_media/GPS.pdf. Zugegriffen: 30. Nov. 2014

  15. Iezzoni LI, Foley SM, Heeren T, Daley J, Duncan CC, Fisher ES, Hughes J (1992) A method for screening the quality of hospital care using administrative data: preliminary validation results. QRB Qual Review Bull 11:361–371

    Google Scholar 

  16. Jakob J, Marenda D, Sold M, Schluter M, Post S, Kienle P (2014) Quality of documentation of intraoperative and postoperative complications. Chirurg 8:705–710

    Article  Google Scholar 

  17. Kaafarani MA, Borzecki AM, Itani K (2011) Validity of selected patient safety indicators: opportunities and concerns. J Am Coll Surg 6:924–934

    Article  Google Scholar 

  18. Ketelaar A, Faber MJ, Flottorp S, Rygh LH, Deane HO, Eccles MP (2011) Public release of performance data in changing the behaviour of healthcare consumers, professionals or organisations. Cochrane Database Syst Rev 11:CD004538

    PubMed  Google Scholar 

  19. Koch CG, Li L, Hixson E, Tang A, Phillips S, Henderson JM (2012) What are the real rates of postoperative complications: elucidating inconsistencies between administrative and clinical data sources. J Am Coll Surg 5:798–805

    Article  Google Scholar 

  20. Kostuj T, Smektala R (2010) Quality assurance using routine data. Is outcome quality now measurable? Unfallchirurg 113(12):1047–1048, 1050–1052

    Article  PubMed  CAS  Google Scholar 

  21. Krumholz HM, Lin Z, Normand S-LT (2013) Measuring hospital clinical outcomes. BMJ 346:f620

    Article  PubMed  Google Scholar 

  22. Lawson E, Louie R, Zingmond D, Brook R, Hall B, Han L, Rapp M, Ko C (2012) A comparison of clinical registry versus administrative claims data for reporting of 30-day surgical complications. Ann Surg 6:973–981

    Article  Google Scholar 

  23. Leal JR, Laupland KB (2010) Validity of ascertainment of co-morbid illness using administrative databases: a systematic review. Eur J Clin Microbiol Infect Dis 6:715–721

    Article  Google Scholar 

  24. Liberati A, Altman D, Tetzlaff J et al (2009) The PRISMA statement for reporting systematic reviews and meta-analysis of studies that evaluate healthcare interventions: explanations and elaboration. BMJ 339:b2700

    Article  PubMed  PubMed Central  Google Scholar 

  25. Lilford R, Pronovost P (2010) Using hospital mortality rates to judge hospital performance: a bad idea that just won’t go away. BMJ 340:c2016

    Article  PubMed  Google Scholar 

  26. McGinn TG, Guyatt GH, Wyer PC, Naylor CD, Stiell IG, Richardson WS (2000) Users’ guides to the medical literature: XXII: how to use articles about clinical decision rules. Evidence-Based Medicine Working Group. JAMA 1:79–84

    Article  Google Scholar 

  27. Mull H, Borzecki A, Loveland S et al (2014) Detecting adverse events in surgery: comparing events detected by the Veterans Health Administration Surgical Quality Improvement Program and the Patient Safety Indicators. Am J Surg 4:584–595

    Article  Google Scholar 

  28. Osborne N, Nocholas L, Ryan A, Thumma J, Dimick J (2015) Association of hospital participation in a quality reporting program with surgical outcomes and expenditures for medicare beneficiaries. JAMA 313:496–504

    Article  PubMed  CAS  PubMed Central  Google Scholar 

  29. Pine M, Jordan HS, Elixhauser A, Fry D, Hoaglin D, Jones B, Meimban R, Warner D, Gonzales J (2007) Enhancement of claims data to improve risk adjustment of hospital mortality. JAMA 297:71–76

    Article  PubMed  CAS  Google Scholar 

  30. Quan H, Eastwood C, Cunningham C, Liu M, Flemons W, De Coster C, Ghali W (2013) Validity of AHRQ patient safety indicators derived from ICD-10 hospital discharge abstract data (chart review study). BMJ Open 3:e003716

    Article  PubMed  PubMed Central  Google Scholar 

  31. Quan H, Li B, Saunders LD, Parsons GA, Nilsson CI, Alibhai A, Ghali WA (2008) Assessing validity of ICD-9-CM and ICD-10 administrative data in recording clinical conditions in a unique dually coded database. Health Services Res 4:1424–1441

    Article  Google Scholar 

  32. Romano P, Mull H, Rivard P et al (2009) Validity of selected AHRQ patient safety indicators based on VA National Surgical Quality Improvement Program data. Health Services Res 44:182–204

    Article  Google Scholar 

  33. Romano P, Roos L, Luft H, Jollis J, Doliszny K (1994) A comparison of administrative versus clinical data: coronary artery bypass surgery as an example. J Clin Epidemiol 47:249–260

    Article  PubMed  CAS  Google Scholar 

  34. Rosen AK, Itani MF, Cevasco M et al (2012) Validating the patient safety indicators in the Veterans Health Administration: do they accurately identify true safety events? Med Care 1:74–85

    Article  Google Scholar 

  35. So L, Beck CA, Brien S, Kennedy J, Feasby TE, Ghali WA, Hude Q (2010) Chart documentation quality and its relationship to the validity of administrative data discharge records. Health Informatics 2:101–113

    Article  Google Scholar 

  36. Stekeler-Weithofer (2011) Qualität/Quantität. In: Sandkühler H (Hrsg) Enzyklopädie Philosophie- Meiner F, Hamburg, S 2184, 2188

    Google Scholar 

  37. Utter GH, Zrelak PA, Baron R, Tancredi DJ, Sadeghi B, Geppert JJ, Romano PS (2013) Detecting postoperative hemorrhage or hematoma from administrative data: the performance of the AHRQ Patient Safety Indicator. Surgery 5:1117–1125

    Article  Google Scholar 

  38. www.qualitätssicherung-mit-routinedaten.de. Zugegriffen: 30. Nov. 2014

Download references

Author information

Authors and Affiliations

  1. Asklepios Klinik Langen, Röntgenstr 20, 63225, Langen, Deutschland

    E. Hanisch & A. Buia

  2. Heilig-Geist-Hospital Bingen, Bingen, Deutschland

    T. F. Weigel

  3. Berufsverband der Deutschen Chirurgen, Langenbeck-Virchow-Haus, Berlin, Deutschland

    H.-P. Bruch

Authors
  1. E. Hanisch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. T. F. Weigel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Buia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. H.-P. Bruch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to E. Hanisch.

Ethics declarations

Interessenkonflikt

E. Hanisch, T. F. Weigel, A. Buia und H.-P. Bruch geben an, dass kein Interessenkonflikt besteht.

Dieser Beitrag beinhaltet keine Studien an Menschen oder Tieren.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hanisch, E., Weigel, T., Buia, A. et al. Die Validität von Routinedaten zur Qualitätssicherung. Chirurg 87, 56–61 (2016). https://doi.org/10.1007/s00104-015-0012-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00104-015-0012-1

Schlüsselwörter

Keywords

Search

Navigation