Background

In the past 20 years, healthcare quality initiatives were mainly related to six quality domains as defined by the Institute of Medicine (IOM): patient-centredness, timeliness, efficiency, effectiveness, safety and equity [1]. Recently, Lachman and colleagues reflected on the relevance of IOM’s quality domains and suggested a multidimensional quality model that includes new domains. The revised domains reflect the changing worldview of quality management [2, 3], such as ecology [4] and transparency [5]. Lachman's new quality model extends the domain of person-centredness by recognising the patient’s kin and healthcare professionals as persons with fundamental needs embodied in every other quality domain. Kin involvement is increasingly being seen as an individual component of quality initiatives that can lead to improved patient outcomes [6,7,8]. Emphasis is placed on including their experiences as an important knowledge source for quality purposes [9]. Moreover, research has shown that quality of care (QoC) and patient safety are related to professionals’ characteristics, such as a negative association with burnout [10, 11], and that their working environment should be monitored [12]. The incorporation of care for professionals has been reinforced by the transition from the Triple Aim to the Quintuple Aim for improving healthcare, with an emphasis on healthcare equity [13]. To conclude, integrating experiential knowledge of patients, kin and professionals about QoC for patients and kin as well as for professionals is recognised as important considering the trend towards value-based, co-produced quality management systems.

In order to effectively co-produce an organisation-wide quality management system, it is essential to approach QoC multidimensionally and integrate it from a multistakeholder perspective [14,15,16]. Many instruments have been developed to measure experiences of QoC [12, 17,18,19,20,21,22,23,24,25,26] and quantifying them has become widespread [27]. Nevertheless, existing instruments have focused on a particular stage of a patients’ hospital journey from admission [17] to hospital discharge [18], on a specific disease, e.g. in cardiology care [19], on certain quality domains, e.g. such as communication and coordination of care [20] or on including only patients [21, 22], kin [23] or professionals [12, 24,25,26]. An instrument that captures organisation-wide experiences would provide a comprehensive healthcare quality assessment whose results can catalyse meso- and macro-level quality management, such as prioritising quality improvement efforts based on multistakeholder experiences. Such an instrument, that encompasses all quality domains [2] and integrates patients’, kin’s and professionals’ perspectives on these domains, is currently lacking. The absence of experiences from other quality domains, such as Lachman’s core values and catalysts, which has been highlighted as desired quality outcomes in previous research [28,29,30,31,32], leads currently to a non-comprehensive view on QoC in hospitals. In addition, blind spots from other stakeholders’ experiences prevent hospitals from creating a quality management system that creates value for all. Measuring QoC multidimensionally from a multistakeholder perspective is fundamental for hospitals to gain a deeper understanding of experiences. Though, no studies have so far constructed a bipartite, organisation-wide instrument measuring both healthcare quality for ‘patients and kin’ and how the organisation cares for its ‘professionals’ in a methodologically sound way that involved patients, kin and professionals. Results of such a validated instrument will facilitate co-production of a sustainable, organisation-wide quality management system in which all stakeholders’ values are central. In conclusion, we need a valid instrument encompassing quality multidimensionally in terms of care for patients, kin and for professionals and integrating multistakeholder perspectives, i.e. with patients, kin and professionals as key stakeholders in quality management. To address this research gap, we aimed to develop and validate an instrument to measure experiences of healthcare quality multidimensionally from a multistakeholder perspective.

Methods

Design

A multi-method approach was used to develop and validate a rigorous instrument [33]. Development started by establishing the content and face validity, followed by testing the construct and criterion validity as well as the reliability using a cross-sectional survey design in 17 Flemish (Belgian) hospitals (Fig. 1). Data were collected between May 2021 and June 2022 via an online survey, the Flanders Quality Model (FlaQuM)-Quickscan. A parallel process evaluation was organised to identify effective strategies in instrument distribution by conducting semi-structured interviews with healthcare quality managers [34].

Fig. 1
figure 1

Instrument development and assessment of psychometric properties

Full size image

Instrument development and psychometric properties

Content validity

Content validity, also known as theoretical analysis, referred to the adequacy with which a measure assesses the domain of interest, i.e. that the items capture the relevant experience of the target population being examined [33]. First, our research group conducted four focus groups with patients and kin (n = 35 in total, npatients = 23 and nkin = 12) to gain a deeper understanding of key attributes of QoC relevant to them [35]. ‘Kin’ refers to the wider social construct around the people involved in receiving and providing care [2]. Kin is also known as caregivers, as used in other international publications [36, 37]. Caregiver refers to someone who takes care of a person who is young, old, ill, or disabled, i.e. having an illness, injury, or condition that makes it difficult for them to do some things that other people do, either as a family member or friend, or as a job [38]. As the word ‘kin’ is used in Lachman’s original multidimensional quality model [2], this term is also used in this manuscript to decrease the risk of confusion between the instrument and Lachman’s model [2]. Focus group results were mainly related to the quality domains ‘Partnership and co-production’, ‘Dignity and respect’ and ‘Effectiveness’. Technical quality domains were linked to organisational aspects of care in terms of staffing levels and time. A theory-based, inductive interpretation of patients’ and kin’s experiential knowledge during these focus groups resulted in the validation of Lachman’s multidimensional quality model. This model, that was developed by QoC experts using deductive reasoning based on expertise of healthcare researchers and professionals’ knowledge over the past 20 years, served as a conceptual framework for the development of the instrument [2]. Second, to design instrument items, three quality and patient participation experts (two are postdoctoral fellows, one with specific expertise in patient participation and empowerment and one with additional experience as member of a patient association in a Flemish hospital, and one is staff member specialised in patient participation in a Flemish hospital), were involved to consider content relevance of instrument items and to ensure operational ‘fit’ with the theoretical meaning of quality domains. An expert advisory panel (n = 10), consisting of the instrument’s target population (patients, kin and different types of professionals) provided input on the completeness, relevance, structure and usability of items. Based on their feedback, instrument items were revised.

Face validity

Face validity, which is defined as the appropriateness of instrument items to the intended construct [33], was obtained through a pilot test by a multistakeholder group (n = 41), including patients (n = 5), their kin (n = 5) and professionals (n = 31). The latter were hospital board members (n = 4), executives (n = 11), healthcare quality managers (n = 4), physicians (n = 6), nurses (n = 5) and medical secretary (n = 1). In addition to scoring instrument items, they were asked to give feedback about appropriateness of instrument items, wording clarity, overall instrument design and navigation. The pilot test results were used to develop an updated version of the instrument.

Description and scoring of the instrument

The instrument, hereinafter referred to as FlaQuM-Quickscan, is designed to mirror patients’, kin’s and professionals’ experiences of QoC through two parts that measure identical quality domains from different care perspectives [See Additional file 1]. The first part aims to explore perspectives on ‘Healthcare quality for patients and kin’, the second part on ‘Healthcare quality for professionals’, i.e. how the hospital cares for their professionals. Patients, kin and professionals were asked to complete both instrument parts. Each part includes 15 items, measuring exactly the same domains, i.e. those of the multidimensional quality model [2], three general items, two of which (the overall quality score and recommendation score) are based on international [39] and Belgian questionnaires [40] and one (the intention-to-stay score) was included because of the importance of this topic in the healthcare landscape and the current shortage of professionals, and sociodemographic questions. The 15 items reflecting quality domains were divided into four subscales: person- and kin-centred care (2 items), catalysts (3 items), technical domains (6 items) and core values (4 items). Each item was rated on a 11-point Likert-type scale reflecting the respondent’s level of disagreement or agreement with the item statement [score from “0” (strongly disagree) to “10” (strongly agree)]. The three general items started with the overall quality assessment of received care (in part 1) and the overall quality assessment of the hospital as employer (in part 2) [score from “0” (worst possible quality) to “10” (best possible quality)]. The second general item concerned the willingness to recommend the hospital to family and friends for receiving care (in part 1) or to work as an employee (in part 2) [score from “0” (definitely no) to “10” (definitely yes)]. The last general item reflected on respondents’ intention-to-stay in the next year to receive care (in part 1) or to work as employee (in part 2) [score from “0” (definitely no) to “10” (definitely yes)]. Demographic items included respondent groups (patients, kin or different professional groups), gender and age. The instrument language was Dutch.

Multi-centre testing: setting and participants

This study took place in a convenience sample of 17 hospitals in Flanders (Belgium), which are implementing a new Flanders Quality Model (FlaQuM). FlaQuM focuses on developing a sustainable quality management system and encompasses 3 pillars: 1) “thinking” based on a quality vision model [2]; 2) “doing” by focusing on the implementation of a co-creation roadmap [14] and 3) “learning and innovating” from social capital in inter-hospital collaboratives [41]. The FlaQuM-Quickscan is part of pillar 1. Patients and their kin who had a consultation, treatment or admission in one of the included hospitals were invited to complete the FlaQuM-Quickscan. Dutch-speaking participants (patients, kin and professionals) of at least 18 years old were invited to complete the FlaQuM-Quickscan online. A FlaQuM Coordinator, i.e. the local healthcare quality manager, for each hospital was responsible for distributing the survey link for their hospital. The link to the electronic survey was provided by the University of Leuven and all the response data flowed to the university database. Each hospital invited patients, kin, or its professional staff to complete the survey, whether by way of e-mail, website, or a limited, local hospital portal. Only fully completed instruments (part 1 and part 2) were included in this study. In line with recommendations, a minimum sample size of 300 patients and kin and 300 professionals was considered acceptable for testing the FlaQuM-Quickscan validation [42].

Descriptive statistics

Descriptive analyses of sociodemographic data delineated frequencies across type of respondents and their characteristics (gender and age). Descriptive analyses for each of the 15 items reflecting quality domains and for the three general items included average, percentage distribution of scores on the 11-point Likert scale and percentage of scores between 0–5, between 6–7 and between 8–10. The Kolmogorov–Smirnov test, a test to assess whether two samples have the same statistical distribution, was used to compare percentage distributions of scores on the 11-point Likert scale between patients/kin and professionals. The t-test, a test to assess differences between two independent groups, was used to compare averages of the 15 items and three general items scored by patients/kin and professionals. The level of significance was set to p < 0.05. The descriptive analyses were generated using the SAS software, Version 9.4 of the SAS System for Windows.

Construct validity

First, confirmatory factor analysis (CFA) was performed to evaluate the tetra-dimensional structure of the FlaQuM-Quickscan (person- and kin-centred care, catalysts, technical domains and core values) defined a priori by the multidimensional quality model [2]. We assessed whether the hypothesised subscales of part 1 and 2 are conceptualized as such by patients, kin and professionals. Second, independent clusters (ICM)-CFA was used to assess measurement invariance across type of respondents and their characteristics. By doing so, the model fit across groups of respondents could be evaluated. To start, model fit was assessed in each group by conducting single-group CFA to investigate whether the established dimensionality of the instrument fit the two stakeholder groups separately: patients/kin and professionals [43]. Next, multiple group ICM-CFA was conducted to assess various types of invariance [44]. Configural invariance relates to showing the same pattern of associations between items and factors and the same number of factors. Factor loadings and thresholds are free across groups. Evidence of scalar invariance is a requirement for drawing meaningful comparisons across groups [44]. All items were continuous for all described steps. For multiple‐group ICM‐CFA, weighted least squares estimation with delta parameterization was used. In multiple‐group analyses, factor variances and latent means were fixed to be 1 and 0, respectively, for identification purposes [45]. Model fit evaluation was based on internationally recognised cut-off criteria [46] and Chen’s [47] allowed changes in fit indices when studying invariance for the Comparative Fit Index (CFI) (ranges between 0 and 1; reasonable if > 0.90 and very good if > 0.95), the Tucker–Lewis index (TLI) [48] (ranges between 0 and 1; reasonable if > 0.90 and very good if > 0.95), and the Root Mean Square Error of Approximation (RMSEA) [49] (ranges between 0 and 1; good fit if < 0.1). Mplus version 7.1 was used to estimate factor analytic models [45].

Criterion validity

Criterion validity, defined as the degree of a relationship between a given test score and performance on another measure [33], was assessed by determining the degree of Pearson’s correlation between the 15-item instrument and the three general items (overall quality score, recommendation score and intention-to-stay score) for each instrument part. Coefficients exceeding r = 0.3 were considered as meaningful [50]. As no other instrument was available to measure patients’, kin’s and professionals’ experiences of QoC as defined by Lachman’s multidimensional model, scores on general items were treated as a substitute for a gold standard with which the instrument items were correlated. The general items have been found to relate well to quality domains [51, 52]. The overall quality score and recommendation score are also used for public reporting of patient experiences via the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) in the United States [39] and in Belgium [40]. The intention-to-stay score received international attention because of the increasing shortage of healthcare professionals [53] and is used in Belgium as a smoke signal for policymakers and managers [54].

Reliability

To obtain reliability for the FlaQuM-Quickscan, the internal consistency was measured using the Cronbach’s alpha for both instrument parts, the subscales and for both stakeholder groups (patients/kin and professionals), with a coefficient ≥ 0.7 considered to be good [33].

Process evaluation

The process evaluation aimed to identify effective strategies to communicate the FlaQuM-Quickscan, to distribute it hospital-wide and to motivate patients, kin and professionals to complete. This evaluation started in three pilot hospitals by taking observation notes from all activities related to its aim. Qualitative, thematic analysis of notes led to lessons learned for other hospitals. Based on these lessons, a topic list and interview guide were developed to conduct semi-structured interviews with healthcare quality managers of the 17 included hospitals. This manager leads the overall coordination of instrument distribution in their hospital. All interviews were audio recorded. The rapid identification of themes from audio recordings (RITA) was used as a first data analysis step [55]. RITA allows for expeditious identification of themes in qualitative data while minimizing the loss of information. Next, thematic analysis was used to inductively analyse the data and to gradually develop and refine insights into effective strategies [56]. Research team (all authors) discussions enabled interpretation of the data and identification of strategies. The data analysis was performed in MS Excel.

Ethical considerations

Ethical approval was obtained from all local ethics committees of participating hospitals. All respondents (focus groups, FlaQuM-Quickscan and process evaluation) provided informed consent. All methods were carried out in accordance with the Declaration of Helsinki guidelines and regulations.

Results

Developed instrument ‘FlaQuM-Quickscan’

By involving experts in the development of quality items and through pilot testing by a multi-stakeholder group, the content and face validity of the instrument and instrument items was ensured. During the development steps, the number of items remained the same, but the wording in item statements was adjusted based on feedback. The FlaQuM-Quickscan contains two parts (part 1 ‘Healthcare quality for patients and kin’ and part 2 ‘Healthcare quality for professionals’). Each part includes 15 quality items and three general items.

Multi-centre testing: sample

In total, 13,615 respondents (NPatients/kin = 5,891 and NProfessionals = 7,724) completed the FlaQuM-Quickscan. The respondents’ characteristics are shown in Table 1. Among patients and kin, 56.4% were female and 32.9% were aged 51–65. Among professionals, 40.8% were nurses, 75.3% were female, and 48.2% were aged 31–50.

Table 1 Characteristics of respondents
Full size table

Descriptive results

Descriptive results of 15 items of the multidimensional quality model and the three general items are analysed for part 1 ‘Healthcare quality for patients and kin’ and part 2 ‘Healthcare quality for professionals’ [see Additional file 2]. For part 1, averages of items varied between 7.7 (‘Kin-centred care’ and ‘Eco-friendly’) and 8.7 (‘Equity’ and ‘Kindness with compassion’) and between 5.9 (‘Eco-friendly’) and 8.3 (‘Equity’) scored by patients/kin and professionals, respectively. The item with the lowest average was the same as the one with the highest percentage of scores between 0–5 (‘Eco-friendly’ scored by patients/kin and professionals) and vice versa for the highest percentage of scores between 8–10. For part 2, averages of items varied between 7.6 (‘Kin-centred care’, ‘Resilience’, ‘Partnership and co-production’) and 8.3 (‘Kindness with compassion’) and between 5.8 (‘Resilience’, ‘Efficient’, ‘Accessible and timely’ and ‘Partnership and co-production’) and 8.0 (‘Equity’) scored by patients/kin and professionals, respectively. The items with the lowest average were the same as the ones with the highest percentage of scores between 0–5 (‘Partnership and co-production’ scored by patients and kin, ‘Accessible and timely’ and ‘Partnership and co-production’ scored by professionals) and vice versa for the highest percentage of scores between 8–10. For all items, percentage distributions of scores for each item and averages on items scored by patients and kin were significantly different from those scored by professionals, except for the general item ‘Intention-to-stay’ in instrument part 2.

Construct validity

The hypothesised dimensionality of part 1 ‘Healthcare quality for patients and kin’ and part 2 ‘Healthcare quality for professionals’ were evaluated separately (Table 2). The hypothesised subscales of both instrument parts were conceptualized as such by patients, kin and professionals. Moreover, the ICM-CFA and the multiple group ICM-CFA showed good to very good fit for the data for the respondent groups in both instrument parts [see Additional file 3]. The FlaQuM-Quickscan allowed for cross-group comparison between patients, their kin and professionals.

Table 2 Confirmatory factor analyses
Full size table

Criterion validity

All correlation coefficients exceeded the 0.3 criterion. For part 1, associations of items-to-overall-quality-score ranged from 0.545 to 0.802 and from 0.373 to 0.713 responded by patients/kin and professionals respectively (Table 3). Associations of items-to-recommendation-score ranged from 0.494 to 0.790 and from 0.326 to 0.671 responded by patients/kin and professionals respectively. Associations of items-to-intention-to-stay-score ranged from 0.468 to 0.759 and from 0.309 to 0.608 responded by patients/kin and professionals respectively. The association of the item ‘Eco-friendly’ and the three general items of both parts, responded by patients/kin as well as professionals, is assessed as the lowest, except for the item ‘Equity’ responded by professionals in part 2. The association of the item ‘Dignity and respect’ and each general item of both parts and responded by patients/kin and professionals is considered the highest. For part 2, associations of items-to-overall-quality-score ranged from 0.697 to 0.812 and from 0.438 to 0.822 responded by patients/kin and professionals respectively. Associations of items-to-recommendation-score ranged from 0.654 to 0.777 (scored by patients/kin) and from 0.434 to 0.781 (scored by professionals). Associations of items-to-intention-to-stay-score ranged from 0.633 to 0.729 (scored by patients/kin) and from 0.417 to 0.637 (scored by professionals).

Table 3 Item-to-general-items correlations
Full size table

Reliability

For part 1, the Cronbach’s alphas were 0.967 and 0.957 scored by patients/kin and professionals, respectively. The Cronbach’s alphas for the subscales were ranging from 0.828 to 0.937 (Table 4). For part 2, the Cronbach’s alphas were 0.981 and 0.947 scored by patients/kin and professionals, respectively. The Cronbach’s alphas for the subscales were ranging from 0.857 to 0.945.

Table 4 Internal consistency
Full size table

Process evaluation

In three pilot hospitals, presentations at committees, leaflets, paper posters and screensavers in waiting rooms were used to communicate about the FlaQuM-Quickscan. Healthcare quality managers, job students and volunteers were actively distributing the FlaQuM-Quickscan with a QR-code and tablets on which respondents could immediately complete it. In these hospitals, an individualised approach to explain FlaQuM-Quickscan objectives, to describe the added value of both instrument parts and to support in the online navigation was observed to be most effective. Based on the analysis of the researchers’ observation notes, a clear introduction and instructions on how to complete this mirror instrument emerged as essential. In part 1 of the FlaQuM-Quickscan, professionals without experience as a patient in that hospital were asked to imagine what it would be like to be a patient there. In part 2 of the FlaQuM-Quickscan, patients and kin that were not employed in that hospital, were asked to score the items based on what they could feel, hear and experience during their hospital contact. These lessons learned were presented to healthcare quality managers of included hospitals before the FlaQuM-Quickscan distribution was launched in their hospital. Interviews with 17 healthcare quality managers revealed that all hospitals explained FlaQuM-Quickscan objectives and added value on meetings with employees and used a personalised poster in the hospital’s language, leaflet or screensaver to communicate to all healthcare stakeholders, including patients and kin.

Six hospitals published an article in their hospital magazine and three hospitals launched an introduction video. For distribution, social media or internal webpages were used by three hospitals towards patients and kin and by nine hospitals towards professionals. Moreover, all hospitals used e-mail addresses of professionals to contact them and one hospital used text messages to reach patients. Additionally, to motivate patients, kin and professionals, all hospitals used an individualised approach with a job student or volunteer motivating respondents hospital-wide to complete the FlaQuM-Quickscan. In eleven hospitals they used tablets for immediate instrument completion. Moreover, hospitals received weekly feedback about the number of respondents for each type of respondent, which motivated them to focus on reaching lower response groups.

Discussion

This study described a multi-step approach to develop and validate an instrument that measures experiences of QoC multidimensionally [2] from an integrated multistakeholder perspective, i.e. patients, kin and professionals. The goal of this mirror instrument is to measure patients’, kin’s and professionals’ experiences of quality in terms of care for patients and their kin (instrument part 1) and for professionals (instrument part 2). The FlaQuM-Quickscan is the first to provide a comprehensive, non-disease-specific assessment of QoC for both patients/kin and professionals. A mirror instrument has been used extensively in health services research to study different perspectives, e.g. to mirror experiences of different stakeholder groups, such as patients and professionals [17, 57, 58], or to mirror experiences of one stakeholder group focusing on different care perspectives [43]. The uniqueness of the FlaQuM-Quickscan is that all stakeholders complete both instrument parts, which implicates that patients and kin have to imagine how the hospital cares for professionals and vice versa. Mirroring experiences is substantially supported by theoretical models [2, 59] describing that experiential knowledge of patients and kin may differ from the gaps experienced and preferences held by professionals and vice versa. Integrating different perspectives gives the opportunity to analyse discrepancies and to foster an in-depth discussion to gain a deeper understanding on QoC [59]. The complementarity of quantitative and qualitative results to define QoC priorities, reinforce an integrated, well-informed approach towards quality management.

The validation of the FlaQuM-Quickscan started by conducting focus groups [35] and involving an expert advisory panel to establish content validity, followed by obtaining face validity through a pilot test in a multistakeholder group. Subsequent validation steps focused on a series of factor analytic models assessing multidimensionality and measurement invariance. The hypotheses to divide each instrument part in four subscales, as a priori defined in Lachman’s model, were confirmed in our multicentre study. This dimensionality fitted our stakeholder groups of patients/kin and professionals separately. Multiple group analyses showed a well-fitting model for both groups and allowed comparison across various types of respondents and their characteristics (gender and age). We assumed that respondents can only score on domains experienced by themselves, but based on validity tests we can conclude that items of each instrument part separately had the same meaning for each type of respondent. The criterion validity tests revealed that the majority of items demonstrated strong correlations with overall quality assessment of respondents, thus appearing to measure QoC and nothing else. Consistent with other research [60], the core value ‘Dignity and respect’ showed the highest correlation with the overall quality assessment in both instrument parts and for both stakeholder groups (patients/kin and professionals). Therefore, despite the generally accepted measurement of technical quality aspects, from a patients’, kin’s and professionals’ view the emphasis has to be on interpersonal, relational, interprofessional and behavioural aspects in quality management [12, 26, 31, 32]. The Cronbach’s alpha coefficients revealed good internal consistency for both instrument parts. These values are excellent in comparison with earlier studies that demonstrated lower range rates for instruments measuring healthcare quality experiences of patients [19, 22], kin [23] and professionals [12, 24, 26]. In conclusion, this validated instrument can facilitate co-production of a sustainable, multidimensional quality management system in which all stakeholders’ values are central.

Our process evaluation emphasised the need for an individualised approach in communicating and distributing the FlaQuM-Quickscan and in motivating stakeholders to share their QoC experiences. Although the domain ‘Eco-friendly’ is a maturing quality attribute receiving growing research attention [4], it correlates the lowest of all quality domains in our study. In the current paradigm of youth awareness for environmental conditions and climate targets, the domain may be correlated differently by younger respondents in our sample. Moreover, despite including health equity in the Quintuple Aim [13], the domain has the second lowest association with overall quality assessment. This may be due to the inclusion of only Dutch-speaking respondents in our sample. The FlaQuM-Quickscan can be expanded to include information on cultural backgrounds and socio-economic demographics of respondents.

FlaQuM-Quickscan results at meso- or micro-level can be used by hospitals to build a shared quality vision and to define related aims (FlaQuM pillar 1). In practice, the discrepancies between the experiences of patients, kin and professionals as well as the differences between results of FlaQuM-Quickscan part 1 ‘Healthcare quality for patients and kin’ and FlaQuM-Quickscan part 2 ‘Healthcare quality for professionals’ can be used for this vision development. The brief tool can be used to develop a monitoring and transparent feedback system, as guided in the co-creation roadmaps towards sustainable QoC (FlaQuM pillar 2) [14]. As shown in our study, monitoring quality multidimensionally implies a focus on technical experiences and soft skills. Education programmes are increasingly focusing on soft skills such as leadership and teamwork as important factors contributing to quality improvement [25, 61]. Hospital human resources departments can use FlaQuM-Quickscan results to improve patient, kin and employer experience [13]. Moreover, the FlaQuM-Quickscan could be expanded to include items concerning care pathways, protocols or procedures as well as the quality of communication between patient and provider. In addition to in-hospital QoC management, benchmark reports can be shared to learn during inter-hospital learning collaboratives (FlaQuM pillar 3). In conclusion, the FlaQuM-Quickscan will be useful to researchers, healthcare managers, hospitals’ executives and policymakers. In future research, variation in experiences within and between stakeholder groups and hospitals can be examined to identify quality priorities at management, Executive and Board levels and to co-produce future quality initiatives. Additionally, associations of experiences and respondents’ demographic variables will be researched. When data from repeated measurements become available, longitudinal invariance and impact of quality initiatives on FlaQuM-Quickscan scores must be studied to explore the sensitivity of the instrument.

Strengths and limitations

A major strength of this study is the evidence-based, stepwise development of this new instrument in a multi-centre setting of 17 hospitals and a parallel process evaluation. The sample of patients, kin and professionals consisted of a female/male ratio that is similar to other healthcare studies [18, 62]. Inclusion criteria were only restricted by age, which might lead to a generalisability of results in hospital settings. Quality is addressed multidimensionally in each instrument part, which are validated separately and can be used to mirror results of both parts and of both perspectives. Subscales or individual quality domains can be used on their own. Because the validation of this multidimensional instrument is complex, with analyses per respondent group and per FlaQuM-Quickscan part, and because previous studies used different types of analyses based on the characteristics of their developed instrument, this study did not make a statement on the comparison of our validation results with those of other instruments. The approach of the FlaQuM-Quickscan is efficient (not time-consuming), feasible and therefore useful for formal quality improvement methods that put patients’, kin’s and professionals’ experiences central. Although this instrument has been developed in Flanders, the method of the FlaQuM-Quickscan could be applied in all healthcare settings in an international perspective. Potential limitations of this study are the cross-sectional design and the self-administrating instrument completion. Further testing of psychometric properties, such as content validity index and convergent validity, is preferable. Evaluation of the FlaQuM-Quickscan in other languages, different countries and in the wider context of healthcare systems, such as in primary care settings, will be the focus of future research. Additionally, within the methods of this study we were not able to match patients, kin and professionals around individual patient cases. Future studies should focus on matched analysis and on understanding differences between experiences of the different stakeholders.

Conclusions

Based on a multi-method approach to establish content and face validity followed by the assessment of construct validity, criterion validity as well as the reliability, the FlaQuM-Quickscan is considered as valid to measure and mirror experiences of QoC multidimensionally from a multistakeholder perspective, i.e. patients, kin and professionals. The FlaQuM-Quickscan measures ‘Healthcare quality for patients and kin’ (part 1) and ‘Healthcare quality for professionals’ (part 2). Each instrument part contains 15 quality items, reflecting quality domains, and 3 general items. The power of this new instrument is its ability to mirror experiences from patients, kin and professionals, providing unique and detailed data to design a sustainable quality management system in hospitals. Continuous monitoring of stakeholders’ experiences can serve as a catalyst for quality improvement. Future research will assess the transferability to other healthcare systems, examine between-stakeholder group and between-hospitals variation and support to set national quality priorities.