Skip to main content


Log in

The REFINEMENT Glossary of Terms: An International Terminology for Mental Health Systems Assessment

  • Original Article
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript


Comparing mental health systems across countries is difficult because of the lack of an agreed upon terminology covering services and related financing issues. Within the European Union project REFINEMENT, international mental health care experts applied an innovative mixed “top-down” and “bottom-up” approach following a multistep design thinking strategy to compile a glossary on mental health systems, using local services as pilots. The final REFINEMENT glossary consisted of 432 terms related to service provision, service utilisation, quality of care and financing. The aim of this study was to describe the iterative process and methodology of developing this glossary.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others


  • Becker, T., & Kilian, R. (2006). Psychiatric services for people with severe mental illness across western Europe: What can be generalized from current knowledge about differences in provision, costs and outcomes of mental health care?. Acta Psychiatrica Scandinavica. Supplementum, 113(429), 9–16. doi:10.1111/j.1600-0447.2005.00711.x.

    Article  Google Scholar 

  • Bernstein, R. (2011). Drop that pipette: Science by design. Cell, 147(3), 496–497. doi:10.1016/j.cell.2011.10.010.

    Article  CAS  PubMed  Google Scholar 

  • Bramesfeld, A., Amaddeo, F., Caldas-de-Almeida, J., Cardoso, G., Depaigne-Loth, A., Derenne, R., Donisi, V., Jørgensen, M., Lindelius, B., Lora, A., Mainz, J., Mulder, C. L., Szecsenyi, J., & Killaspy, H. (2016). Monitoring mental healthcare on a system level: Country profiles and status from EU countries. Health Policy, 120(6), 706–717. doi:10.1016/j.healthpol.2016.04.019.

    Article  PubMed  Google Scholar 

  • BRidging Information and Data Generation for Evidence-based Health policy and research project (BRIDGE Health). (2017). Homepage official website. Retrieved Sept 04, 2017 from

  • Cacace, M., Ettelt, S., Mays, N., & Nolte, E. (2013). Assessing quality in cross-country comparisons of health systems and policies: towards a set of generic quality criteria. Health Policy, 112(1–2), 156–162. doi:10.1016/j.healthpol.2013.03.020.

    Article  PubMed  Google Scholar 

  • Chow, W. S., & Priebe, S. (2016). How has the extent of institutional mental healthcare changed in Western Europe? Analysis of data since 1990. BMJ Open, 6(4), e010188. doi:10.1136/bmjopen-2015-010188.

    Article  PubMed  PubMed Central  Google Scholar 

  • European Community Treaty. (1998). A rticle 152. Amsterdam: European Community Treatment. Retrieved Sept 13, 2017, from ECT:

  • European Health Care Outcomes, Performance and Efficiency (EuroHOPE). (2017). Homepage official website. Retrieved Sept 04, 2017, from

  • European Network For Mental Health Service Evaluation (ENMESH). (2017). Homepage official website. Retrieved Sept 04, 2017, from

  • Fakhoury, W., & Priebe, S. (2002). The process of deinstitutionalization: an international overview. Current Opinion in Psychiatry, 15(2), 187. Retrieved March 2002 from

  • Goodyear-Smith, F. (2017). Collective enquiry and reflective action in research: Towards a clarification of the terminology. Family Practice, 34(3), 268–271. doi:10.1093/fampra/cmw098.

    PubMed  Google Scholar 

  • Haro, J. M., Ayuso-Mateos, J. L., Bitter, I., Demotes-Mainard, J., Leboyer, M., Lewis, S. W., et al. (2014). ROAMER: roadmap for mental health research in Europe. International Journal of Methods in Psychiatric Research, 23(Suppl 1), 1–14. doi:10.1002/mpr.1406.

    Article  PubMed  Google Scholar 

  • Johnson, S., Kuhlmann, R., & EPCAT Group European Psychiatric Assessment Team. (2000). The European Service Mapping Schedule (ESMS): Development of an instrument for the description and classification of mental health services. Acta Psychiatrica Scandinavica. Supplementum, 405, 14–23. Retrieved Sept 13, 2017, from

  • Johnson, S., Salvador-Carulla, L., & EPCAT group. (1998). Description and classification of mental health services: A European perspective. European Psychiatry: The Journal of the Association of European Psychiatrists, 13(7), 333–341. doi:10.1016/S0924-9338(99)80699-3.

    Article  CAS  Google Scholar 

  • Kiivet, R., Sund, R., Linna, M., Silverman, B., Pisarev, H., & Friedman, N. (2013). Methodological challenges in international performance measurement using patient-level administrative data. Health Policy, 112(1–2), 110–121. doi:10.1016/j.healthpol.2013.05.011.

    Article  PubMed  Google Scholar 

  • Killaspy, H., White, S., Dowling, S., Krotofil, J., McPherson, P., Sandhu, S., Arbuthnott, M., Curtis, S., Leavey, G., Priebe, S., Shepherd, G., & King, M. (2016). Adaptation of the quality indicator for rehabilitative care (QuIRC) for use in mental health supported accommodation services (QuIRC-SA). BMC Psychiatry, 16, 101. doi:10.1186/s12888-016-0799-4.

    Article  PubMed  PubMed Central  Google Scholar 

  • Knapp, M., & McDaid, D. (2007). The mental health economics european network. Journal of Mental Health, 16(2), 157–165. doi:10.1080/09638230701279840.

    Article  Google Scholar 

  • Lukersmith, S., Millington, M., & Salvador-Carulla, L. (2016). What is case management? a scoping and mapping review. International Journal of Integrated Care, 16(4), 2. doi:10.5334/ijic.2477.

    Article  PubMed  PubMed Central  Google Scholar 

  • Minsky, S., & Lloyd, P. (1996). The use of unique client identifiers in state mental health service systems. Administration and Policy in Mental Health, 23(3), 231–252. doi:10.1007/BF02108322.

    Article  Google Scholar 

  • Muijen, M. (2012). The state of psychiatry in Europe: Facing the challenges, developing consensus. International Review of Psychiatry, 24(4), 274–277. doi:10.3109/09540261.2012.697264.

    Article  PubMed  Google Scholar 

  • Priebe, S., Frottier, P., Gaddini, A., Kilian, R., Lauber, C., Martínez-Leal, R., Munk-Jørgensen, P., et al. (2008). Mental health care institutions in nine European countries, 2002 to 2006. Psychiatric Services, 59(5), 570–573. doi:10.1176/ps.2008.59.5.570.

    Article  PubMed  Google Scholar 

  • Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., et al. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health, 38(2), 65–76. doi:10.1007/s10488-010-0319-7.

    Article  PubMed  Google Scholar 

  • REFINEMENT Group. (2014a). The refinement decision support toolkit for improving the financing of mental health care. Retrieved Sept 04, 2017, from

  • REFINEMENT Group. (2014b). The refinement results. Retrieved Sept 04, 2017, from

  • REFINEMENT Group. (2014c). The refinement glossary. Retrieved Sept 04, 2017, from

  • Salvador-Carulla, L., Alvarez-Galvez, J., Romero, C., Gutiérrez-Colosía, M. R., Weber, G., McDaid, D., et al. (2013). Evaluation of an integrated system for classification, assessment and comparison of services for long-term care in Europe: The eDESDE-LTC study. BMC Health Services Research, 13, 218. doi:10.1186/1472-6963-13-218.

    Article  PubMed  PubMed Central  Google Scholar 

  • Salvador-Carulla, L., Amaddeo, F., Gutiérrez-Colosía, M. R., Salazzari, D., Gonzalez-Caballero, J. L., Montagni, I., et al. (2015). Developing a tool for mapping adult mental health care provision in Europe: The REMAST research protocol and its contribution to better integrated care. International Journal of Integrated Care, 15, e042.

    PubMed  PubMed Central  Google Scholar 

  • Salvador-Carulla, L., Dimitrov, H., Weber, G., McDaid, D., Venner, B., Sprah, L., et al. (Eds.). (2011). DESDE-LTC: Evaluation and classification of services for long-term care in Europe. Barcelona: PSICOST and CatalunyaCaixa.

    Google Scholar 

  • Salvador-Carulla, L., Fernandez, A., Madden, R., Lukersmith, S., Colagiuri, R., Torkfar, G., & Sturmberg, J. (2014). Framing of scientific knowledge as a new category of health care research. Journal of Evaluation in Clinical Practice, 20(6), 1045–1055. doi:10.1111/jep.12286/full.

    Article  PubMed  Google Scholar 

  • Salvador-Carulla, L., & González-Caballero, J. L. (2010). Assessment instruments in mental health: description and metric properties. In G. Thronicroft & M. Tansella (Eds.), Mental health outcome measures (pp. 28–62). London: RCPsych Publications.

    Google Scholar 

  • Salvador-Carulla, L., Salinas-Pérez, J. A., Martín, M., Grané, M.-S., Gibert, K., Roca, M., et al. (2010). A preliminary taxonomy and a standard knowledge base for mental-health system indicators in Spain. International Journal of Mental Health Systems, 4, 29. doi:10.1186/1752-4458-4-29.

    Article  PubMed  PubMed Central  Google Scholar 

  • Schulz, S., Suntisrivaraporn, B., Baader, F., & Boeker, M. (2009). SNOMED reaching its adolescence: ontologists’ and logicians’ health check. International Journal of Medical Informatics, 78(Suppl 1), 86–94. doi:10.1016/j.ijmedinf.2008.06.004.

    Article  Google Scholar 

  • Spaeth-Rublee, B., Pincus, H. A., Huynh, P. T., & IIMHL Clinical Leaders Group, Mental Health Quality Indicator Project. (2010). Measuring quality of mental health care: a review of initiatives and programs in selected countries. Canadian Journal of Psychiatry. Revue Canadienne de Psychiatrie, 55(9), 539–548. doi:10.1177/070674371005500902.

    Article  PubMed  Google Scholar 

  • Straßmayr, C., Berger, M., Endel, F., & Katschnig, H. (2016). Ensuring interoperability of data on psychiatric rehospitalisation derived from large electronic administrative health care registries across six European countries. European Journal of Public Health, 26(1), 229–230. doi:10.1093/eurpub/ckw170.059.

    Google Scholar 

  • Taylor, M. J., McNicholas, C., Nicolay, C., Darzi, A., Bell, D., & Reed, J. E. (2013). Systematic review of the application of the plan–do–study–act method to improve quality in healthcare. BMJ Quality & Safety. doi:10.1136/bmjqs-2013-001862.

    Google Scholar 

  • The EuroREACH project. (2017). Homepage official website. Retrieved Sept 04, 2017, from

  • Toth, F. (2016). Classification of healthcare systems: Can we go further?. Health Policy, 120(5), 535–543. doi:10.1016/j.healthpol.2016.03.011.

    Article  PubMed  Google Scholar 

  • Wendt, C. (2009). Mapping European healthcare systems: A comparative analysis of financing, service provision and access to healthcare. Journal of European Social Policy, 19(5), 432–445. doi:10.1177/0958928709344247.

    Article  Google Scholar 

  • Wodchis, W. P., Dixon, A., & Anderson, G. M. (2015). Continuing efforts to integrate care can benefit from cross-jurisdictional comparisons. International Journal of Integrated Care, 15, e012.

    PubMed  PubMed Central  Google Scholar 

  • World Health Assembly. (2008). Medium-term strategic plan 20082013. Geneva: World Health Assembly. Retrieved Apr 4, 2011, from WHO:

  • World Health Organization. (2004). A glossary of terms for community health care and services for older persons. Kobe: World Health Organization. Retrieved Sept 13, 2017, from WHO:

  • World Health Organization. (2005). Assessment instrument for mental health systems. Geneva: World Health Organization. Retrieved Sept 13, 2017, from WHO:

  • World Health Organization. (2007). Multilingualism: plan of action: report by the Secretariat. Geneva: World Health Organization. Retrieved Apr 19, 2007, from WHO:

  • World Health Organization. (2017). Online glossary terminology information system. Retrieved Sept 04, 2017, from

  • Zeilinger, E., Brehmer, B., Seyrlehner, D., Poole, M., Salvador-Carulla, L., & Weber, G., for the eDESDE-LTC Group. (2011). eDESDE-LTC: Quality Assessment and Evaluation Package. Jerez: PSICOST and Telnet. Retrieved Sept 04, 2017, from

Download references


The Refinement Project has been funded by the European Commission Seventh Framework Programme ([FP7/2007–2013] [FP7/2007–2011]) under grant agreement n° 261459. The members of the Refinement Group are: Florian Endel, Gisela Hagmair, Heinz Katschnig, Sonja Scheffel, Christa Strassmyr, Barbara Weibold (Austria); Tihana Matosevic, David McDaid, A-La Park (England); Taina Ala-Nikkola, Niklas Grönlund, Irja Hemmilä, Jutta Järvelin, Grigori Joffe, Raija Kontio, Maili Malin, Petri Näätänen, Sami Pirkola, Minna Sadeniemi, Eila Sailas, Marjut Vastamäki, Kristian Wahlbeck (Finland); Ane Auraaen, Matthias Brunn, Karine Chevreul, Isabelle Durand-Zaleski, Thomas Lefèvre, Morgane Michel, Amélie Prigent, Marie-Amélie Vinet (France); Francesco Amaddeo, Gaia Cetrano, Valeria Donisi, Laura Grigoletti, Ilaria Montagni, Laura Rabbi, Damiano Salazzari, Federico Tedeschi (Italy); Ovynd Hope, Brigitte Kalseth, Jorid Kalseth, Jon Magnussen (Norway); Lucian Albu, Nona Chiriac, Mugur Ciumageanu, Dan Ghenea, Simona Musat, Cristian Oana, Carmen Pauna, Alexandru Paziuc, Raluca Sfetcu (Romania); Juan M. Cabases, Jordi Cid, Ana Fernandez, Andrea Gabilondo, Carlos García, Juan Luis González-Caballero, Mencía R. Gutiérrez- Colosía, Alvaro Iruin, Esther Jorda, Cristina Molina, Emma Motrico, Carlos Pereira, Bibiana Prat, Jose Alberto Salinas, Luis Salvador-Carulla, Jose Juan Uriarte (Spain). Collaborators and observers from public health agencies: Mental Health Departments of Bizkaia and Gipuzkoa, Pais Vasco (Spain), Institut d’Assistència Sanitària; Girona (Spain). The first author, Ilaria Montagni, acknowledges the Excellence Cluster “Health Determinants in Societies” HEADS of the University of Bordeaux, managed by the ANR, under the “Future Investments” program in the framework of the Bordeaux “Initiatives d’excellence” (IdEx) program, grant number [ANR-10-IDEX- 03-02] for providing funding for her research studies.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Ilaria Montagni.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants performed by any of the authors.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 32 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Montagni, I., Salvador-Carulla, L., Mcdaid, D. et al. The REFINEMENT Glossary of Terms: An International Terminology for Mental Health Systems Assessment. Adm Policy Ment Health 45, 342–351 (2018).

Download citation

  • Published:

  • Issue Date:

  • DOI: