Quality indicators for mental healthcare in the Danube region: results from a pilot feasibility study


Quality indicators are vital for monitoring the transformation of institution-based mental health services towards the provision of person-centered mental healthcare. While several mental healthcare quality indicators have been identified as relevant and valid, their actual usability and utility for routine monitoring healthcare quality over time is significantly determined by the availability and trustworthiness of the underlying data. In this feasibility study, quality indicators that have been systematically identified for use in the Danube region countries of Bulgaria, the Czech Republic, Hungary, and Serbia were measured on the basis of existing mental healthcare data in the four countries. Data were collected retrospectively by means of the best available, most standardized, trustworthy, and up-to-date data in each country. Out of 21 proposed quality indicators, 18 could be measured in Hungary, 17 could be measured in Bulgaria and in the Czech Republic, and 8 could be measured in Serbia. The results demonstrate that a majority of quality indicators can be measured in most of the countries by means of already existing data, thereby demonstrating the feasibility of quality measurement and regular quality monitoring. However, data availability and usability are scattered across countries and care sectors, which leads to variations in the quality of the quality indicators themselves. Making the planning and outputs of national mental healthcare reforms more transparent and evidence-based requires (trans-)national standardization of healthcare quality data, their routine availability and standardized assessment, and the regular reporting of quality indicators.

This is a preview of subscription content, access via your institution.


  1. 1.

    ANQ-Nationaler Verein für Qualitätsentwicklung in Spitälern und Kliniken (2019) Psychiatrie. https://www.anq.ch/de/fachbereiche/psychiatrie/. Accessed 26 June 2019

  2. 2.

    Armesto SG, Medeiros H, Wie L. Information availability for measuring and comparing quality of mental health care across OECD countries. DELSA/ELSA/WD/HTP(2008)2. https://www.oecd.org/health/health-systems/41243838.pdf. Accessed 6 Aug 2008

  3. 3.

    Bitter I, Kurimay T (2012) State of psychiatry in Hungary. Int Rev Psychiatry 24(4):307–313

    Article  Google Scholar 

  4. 4.

    Dimova A, Rohova M, Moutafova E, Atanasova E, Koeva S, Panteli D, van Ginneken E (2012) Bulgaria: Health system review. Health Syst Transit 14(3):1–186. https://www.euro.who.int/__data/assets/pdf_file/0006/169314/E96624.pdf. Accessed 2 July 2018

  5. 5.

    Dobiášová K, Tušková E, Hanušová P, Angelovská O, Ježková M (2016) The development of mental health policies in the Czech Republic and Slovak Republic since 1989. Cent Eur J Public Policy 10:35–46

    Article  Google Scholar 

  6. 6.

    Großimlinghaus I, Falkai P, Gaebel W, Janssen B, Reich-Erkelenz D, Wobrock T, Zielasek J (2013) Entwicklungsprozess der DGPPN-Qualitätsindikatoren. Nervenarzt 84:350–365

    Article  Google Scholar 

  7. 7.

    Großimlinghaus I, Falkai P, Gaebel W, Hasan A, Jänner M, Janssen B, Reich-Erkelenz D, Grüber L, Böttcher V, Wobrock T, Zielasek J (2015) Erhebung von Qualitätsindikatoren anhand von Routinedaten. Darstellung eines Machbarkeitstests in zehn Fachkliniken für Psychiatrie und Psychotherapie. Nervenarzt 86:1393–1399

    Article  Google Scholar 

  8. 8.

    Hermann R, Mattke S (2004) Selecting indicators for the quality of mental health care at the health systems level in OECD countries. OECD Health Technical Papers, No. 17, OECD Publishing. https://doi.org/10.1787/388745076135

  9. 9.

    Höschl C, Winkler P, Pec O (2012) The state of psychiatry in the Czech Republic. Int Rev Psychiatry 24(4):278–285

    Article  Google Scholar 

  10. 10.

    IQTiG (2017) Aktualisierung und Erweiterung des QS-Verfahrens Versorgung von volljährigen Patienten und Patientinnen mit Schizophrenie, schizotypen und wahnhaften Störungen. Abschlussbericht. Stand: 22. Dezember 2017.https://iqtig.org/downloads/berichte/2017/IQTIG_QS-Verfahren-Schizophrenie_Abschlussbericht_2017-12-22.pdf. Accessed 23 June 2019

  11. 11.

    JCAHO-Joint Commission on Accreditation of Healthcare Organizations (1989) Characteristics of clinical indicators. Qual Rev Bull 163:6401–6406

    Google Scholar 

  12. 12.

    Kilbourne A, Beck K, Spaeth-Rublee B, Ramanuj P, O’Brien RW, Tomoyasu N, Pincus HA (2018) Measuring and improving the quality of mental health care: a global perspective. World Psychiatry. 17:30–38

    Article  Google Scholar 

  13. 13.

    Krupchanka D, Winkler P (2016) State of mental healthcare systems in Eastern Europe: do we really understand what is going on? B J Psych Int 13(4):96–99

    Google Scholar 

  14. 14.

    Lecic Tosevski D, Pejovic Milovancevic M, Popovic Deusic S (2007) Reform of mental health care in Serbia: ten steps plus one. World Psychiatry 6:115–117

    PubMed  PubMed Central  Google Scholar 

  15. 15.

    Lehmann I, Chisholm D, Hinkov H, Höschl C, Kapócs G, Kurimay T, Lecic-Tosevski D, Nakov V, Winkler P, Réthelyi JM, Zielasek J, Gaebel W (2018) Development of quality indicators for mental healthcare in the Danube region. Psychiatr Danubina 30:197–206

    Article  Google Scholar 

  16. 16.

    MacDonald AJD, Fugard AJB (2015) Routine mental health outcome measurement in the UK. Int Rev Psychiatry 27(4):306–319

    Article  Google Scholar 

  17. 17.

    Mainz J (2003) Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care 15(6):523–530

    Article  Google Scholar 

  18. 18.

    Ministry of Health of the Czech Republic (2013) Psychiatric care reform strategy. MHCR, Prague

    Google Scholar 

  19. 19.

    Ministry of Health, Republic of Serbia (2007) Strategy for development of mental health care. Republic of Serbia, Ministry of Health

    Google Scholar 

  20. 20.

    Mitchell J (2018) Mental health quality indicators: background and secondary definitions. Scottish Government. https://www.gov.scot/binaries/content/documents/govscot/publications/factsheet/2018/09/mental-health-quality-indicators-background-secondary-definitions/documents/00540532-pdf/00540532-pdf/govscot%3Adocument/00540532.pdf. Accessed 16 Oct 2019

  21. 21.

    Pincus HA, Hudson Scholle S, Spaeth-Rublee B, Hepner KA, Brown J (2016) Quality measures for mental health and substance use: gaps, opportunities, and challenges. Health Affairs 35:1000–1008

  22. 22.

    Reiter A, Fischer B, Kötting J, Geraedts M, Jäckel WH, Barlag H, Döbler K (2018) QUALIFY: instrument for the assessment of quality indicators. Version 1.0 (English). Düsseldorf, BQS gGmbH. Retrieved on 3rd August 2018. https://www.researchgate.net/publication/267256474_QUALIFY_Instrument_for_the_Assessment_of_Quality_Indicators

  23. 23.

    Spaeth-Rublee B, Pincus HA, Huynh PT, IIMHL Clinical Leaders Group (2010) Measuring quality of mental health care: a review of initiatives and programs in selected countries. Can J Psychiatry 55:539–548

    Article  Google Scholar 

  24. 24.

    WHO (1997) Quality Assurance in Mental Health Care – Checklists and Glossaries. Division of Mental Health and Prevention of Substance Abuse, 1 and 2. http://apps.who.int/iris/handle/10665/66931. Accessed 7 April 2020

  25. 25.

    WHO (2013) Mental Health Action Plan 2013—2020. Geneva: World Health Organization https://www.who.int/mental_health/publications/action_plan/en/. Accessed 26 June 2018

  26. 26.

    WHO Regional Office for Europe (2013) European Mental Health Action Plan 2013–2020. Copenhagen: World Health Organization https://www.euro.who.int/__data/assets/pdf_file/0020/280604/WHO-Europe-Mental-Health-Acion-Plan-2013-2020.pdf. Accessed 3 Jan 2018

  27. 27.

    WHO (2018) Mental Health Atlas 2017. https://www.who.int/mental_health/evidence/atlas/mental_health_atlas_2017/en/. Accessed 20 June 2018

  28. 28.

    Winkler P, Koeser L, Kondrátová L, Broulíková HM, Páv M, Kališová L et al (2018) Cost-effectiveness of care for people with psychosis in the community and psychiatric hospitals in the Czech Republic: an economic analysis. Lancet Psychiatry 5(12):1023–1031

    Article  Google Scholar 

  29. 29.

    Winkler P, Krupchanka D, Roberts T, Kondratova L, Machů V, Höschl C, Sartorius N, Van Voren R, Aizberg O, Bitter I, Cerga-Pashoja A (2017) A blind spot on the global mental health map: a scoping review of 25 years' development of mental health care for people with severe mental illnesses in central and eastern Europe. Lancet Psychiatry 4(8):634–642

    Article  Google Scholar 

  30. 30.

    Winkler P, Mladá K, Krupchanka D, Agius M, Ray MK, Höschl C (2016) Long-term hospitalizations for schizophrenia in the Czech Republic 1998–2012. Schizophr Res 175(1–3):180–185

    Article  Google Scholar 

Download references


This study was funded by the German Ministry for Education and Research (BMBF) under the title: “Development and Implementation of Quality Indicators for Mental Healthcare in the Danube Region” (DAQUMECA, project ID 01DS17020).

Author information



Corresponding author

Correspondence to Wolfgang Gaebel.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Additional information

Communicated by Andrea Schmitt.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gaebel, W., Lehmann, I., Chisholm, D. et al. Quality indicators for mental healthcare in the Danube region: results from a pilot feasibility study. Eur Arch Psychiatry Clin Neurosci (2020). https://doi.org/10.1007/s00406-020-01124-z

Download citation


  • Quality
  • Mental healthcare
  • Quality indicator
  • Quality assurance
  • Mental healthcare reform