Introduction

Several studies indicate that the forensic environment currently holds more people with mental health problems than any other institution; between 15 and 26% of people who have been in prison have been diagnosed with a mental health problem1. Another study indicates that one in seven people admitted to prison worldwide has suffered from a serious mental disorder2.

In Spain, the magnitude of this phenomenon has been described on several occasions. The most representative results were those from the study “Prevalence of mental disorders in Spanish prisons”. These results show that the prevalence mental health problem in a penitentiary environment is 84.4%3. To this, Tort et al. indicate that we must add the "factors related to one's own incarceration that have an influence on their health situation."4.

Consequently, there is a clear need to evaluate the quality of medical care provided by the professionals in penitentiary psychiatric units. In forensic psychiatry there is a growing interest in the evaluation of quality of life and care as an outcome measure5 and also as a variable in the evaluation of the likelihood of criminal recidivism6.

'Quality of medical care' was defined by Donabedian as "that which is expected to provide the user with the maximum and most complete well-being after assessing the balance of gains and losses that can accompany all its parts"7. Quality of medical care can be understood as a multidimensional concept based on three interrelated elements: professional care, interpersonal relationships and comforts of the environment8. One element to be taken into account in relation to the perception of the quality of medical care is job satisfaction. Numerous studies have emphasized the importance of this relationship9.The NTP 394 general satisfaction scale: Overall Job Satisfaction created by Warr10 is the most widely used in Spain and subsequently validated in Spanish by Pérez and Fidalgo11.

Measuring and improving the quality of health system care is of global interest12. Despite this, there is a lack of cross-cultural comparative studies on the perception of patients and professionals regarding quality of care13,14 and even fewer in the forensic field. This is mainly due to an absence of standardised psychiatric instruments for the measurement of medical care.

A systematic psychometric review was published in 201814 in which 22 instruments on quality and satisfaction with mental health care according to professionals and patients were analyzed. This study shows that the instruments with the best psychometric properties most appropriate for the analysis of quality of care are: The Psychiatric Out-Patient Experiences Questionnaire (POPEQ)15, The Questionnaire of Experiences of General Practitioners (GPEQ)16, the Quality Indicator for Rehabilitative Care (QuIRC)17, Combined Assessment of Psychiatric Environments (CAPE)18 and the Quality in Psychiatric Care-Forensic In-Patient (QPC-FIP)19. The systematic review emphasises that the QPC instrument is the most suitable instrument for the measurement of quality of care in comparison with the other instruments mentioned above, as each of these instruments is specific to a given setting and population. In contrast, the QPC has been validated in both professional and patient versions and in different settings (community, hospital and forensic), allowing for a comprehensive assessment of quality of care and comparison of quality perception according to professionals and patients.

The QPC instrument was developed from the phenomenographic study by Schröder, Ahlström and Larsso20 which focused on the search for the dimensions that characterize the quality of care received in various areas of mental health care: hospital admission, community care and forensic psychiatry units. Each of the areas has developed a specific, self-administered instrument to assess quality of care from the perspective of the professionals and the people served20.

In Spain, the respective instruments for hospital professionals21, inpatients22 and outpatient staff23 have been validated. Within the scope that concerns us, the QPC-FIPS (Quality of Psychiatric Care-Forensic Inpatient Staff), QPC version for professionals in the field of forensic psychiatry, has not been validated in Spanish.

The current version of the QPC-FIPS scale is Swedish24 and the first cross-cultural adaptation is a Danish version25. Thus, it is considered necessary to carry out this adaptation to assess the quality of care provided in prison mental health units with the aim of improving such care and contributing to the comparison of results in different countries.

In this context, the objective of this study was to adapt the QPC-FIPS instrument to Spanish and analyze its reliability and validity.

Methods

Participants

The sample consisted of professionals from different disciplines who work in forensic mental health units (nursing, psychiatry, psychology, social education, nursing assistance, social work, and occupational therapy), who participate voluntarily. Having less than six month’s experience in the area of mental health was established as an exclusion criterion. Non-probability convenience sampling was used.

A total of 153 professionals participated, of whom 61.4% were women and 38.6% were men. The mean age was 39.92 ± 9.83 years, while the mean number of years worked in forensic units was 7.10 ± 4.94 years. The sample consisted of a variety of professional categories, with 34.6% nursing assistants, 22.9% occupational therapists, 21.6% nurses, 9.2% psychologists, 7.8% psychiatrists, and 3.9% social workers. Some 85% of the professionals stated that they would define their working environment as good or very good and 15% as bad or very bad. A total of 41.2% stated that they often or always have the opportunity to work on improvement of quality of medical care, 51.0% sometimes and 7.8% rarely or never.

Instruments

Professional perception of quality care was obtained using QPC-FIPS-Spanish. This version is an adaptation of original questionnaire QPC-FIPS24.

The Quality of Psychiatric Care-Forensic Inpatient Staff (QPC-FIPS)24, described in the introduction section, is a self-administered questionnaire consisting of 34 items and measures 7 dimensions of quality of care: encounter (8 items), participation (8 items), discharge (3 items), support (4 items), Secluded environment (2 items), secure environment (3 items) and forensic specific (6 items). Each item is related to the statement "I consider that…" and responses are based on 4-point Likert scales, where 1 corresponds to "strongly disagree" and 4 to "strongly agree". All items also have the option "does not correspond".

To analyze convergent validity, the NTP 394 general satisfaction scale: Overall Job Satisfaction created by Warr et al.10 was used. This scale, which evaluates working conditions (intrinsic and extrinsic), consists of 15 items evaluated on a scale of 1 to 7 (1 “very dissatisfied” and 7 “very satisfied”)11.

Procedure

The translation and back-translation process was carried out following the Standards for Educational and Psychological Testing26.

First, the original version was translated into Spanish by two independent native-speaker translators who had no knowledge of the instrument or the aims of the study. A group of experts comprising a specialist in psychometrics, three quality experts, five nurses specializing in mental health, a psychologist and a psychiatrist specializing in forensic medicine previously assessed semantic equivalence (grammatical difficulties in translation, equivalent meaning of words), and idiomatic (contextualization of the text, colloquialisms) and conceptual equivalence. The translation and back-translation process did not present any major difficulties. After the back-translation process, the expert committee suggested modifying some terms in items 30 and 33 to adapt them to the Spanish context, respecting the semantic meaning of the original version. Item 30 after back-translation of the Swedish version indicated: "the staff helps patients, if they wish, to present their wishes and their case before the administrative court"24, it was considered appropriate to modify it in the Spanish version to "users can access the judge through their lawyer or through the legal advice service”, changing the term "patients" to "users” since in Spain this term is widely used. Item 33 in the Swedish version indicated: "patients receive help from staff to elaborate their offence" for "during the stay in the prison the professionals help the user to talk about the offence he/she committed"24, so the term "patients" was replaced by "users" and "staff" by "professionals". Also "talking about the offence" was chosen instead of "elaborating the offence"24. Finally, inclusive non-discriminatory language was used when referring to gender.

A pilot cognitive test was administered to 30 professionals to assess comprehension and completion time. The test included an open-ended question asking participants to indicate whether any item might present difficulties in comprehension. The average completion time of the questionnaire ranged between 15 and 20 min and no item presented comprehension difficulties. Following the debriefing, it was not considered necessary to make any changes to either format of content. The Spanish adapted version of the QPC-FIPS scale was configured by the same number of items and dimensions as the original, with the final version in Spanish named QPC_FIPS-Spain.

The adaptation and psychometric evaluation process was carried out in four forensic mental health units at Parc Sanitari Sant Joan de Déu, Barcelona, Spain. Data collection was carried out between February, 2019 and December, 2021.

To analyze temporal stability, it was estimated that a minimum of 60 professionals would be needed to detect an intraclass correlation coefficient (ICC) around 0.70 between two administrations of the instrument, assuming a confidence level of 95% and a power of 80% in a bilateral comparison22.

Data analysis

The SPSS Statistics program version 26 was used for analyses, along with EQS program version 6.2 for the confirmatory factor analysis (CFA)27.

Construct validity was analyzed through confirmatory factor analysis (CFA) with estimated parameters using the method of least squares (LS) which is similar to maximum-likelihood (ML) but estimates patterns of relationships between variables by minimising the sum of squares of the deviation between the hypothesised and observed model. LS performs better than ML with small samples and provides a better estimate when there is a violation of normal distribution assumptions28. The criteria for a good fit were Bentler Bonnet Normed Fit Index (BBNFI), Bentler Bonnet Non-Normed Fit Index (BBNNFI), Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI), Comparative Fit Index (CFI) > 0.9029,30,31; the ratio of chi square to degrees of freedom (χ2/df) < 332. The value of the Root Mean Square Error of Approximation (RMSEA) and the Standarized Root Mean-Square (SRMR) and Root mean-square residual (RMR) was ≤ 0.0833,34.

The General Satisfaction Scale NTP 394: Overall Job Satisfaction was used for convergent validity11; an analysis of the Spearman’s correlation coefficient was performed. Furthermore, as an additional method, a Spearman’s correlation analysis was also performed between the factors of the QPC-FIPS instrument with the aim of verifying the Fayers & Machin35 hypothesis which indicates that the correlation was higher between each factor and the general scale than the correlations between the subscales.

To evaluate the internal consistency of the instrument at a general level and for each of the factors, Cronbach's alpha was used and ordinal Alpha, with a value greater than or equal to 0.7036 considered adequate reliability. Temporal stability or test–retest reliability was evaluated after 7–14 days through the ICC in a sample of 77 professionals. A value greater than or equal to 0.70 was considered an indicator of good agreement36. In addition, composite reliability was calculated. Item analyses included the calculation of item means, standard deviations, and corrected total item correlation.

Ethical aspects of research

This study was approved by the research ethics committee at the Sant Joan de Déu Foundation, under CEIC code PIC-73-18. All research was conducted in accordance with relevant guidelines and regulations. All participants were informed about the objective of the study and gave their written consent to participate voluntarily and anonymously.

Results

Construct validity

Confirmatory factor analysis was used to verify the internal structure of the instrument. The CFA shows a chi-square value of (χ2 = 1115.705; df = 506; p < 0.0001). The GFI and AGFI show an adjustment > 0.90; the ratio between chi square and degrees of freedom (χ2/df) was 2.20, that is, achieving an index less than 3, and the value ≤ 0.08 was also obtained for the RMR, SRMR and RMSEA. All index showed a reasonable adjustment. Table 1 shows the fit of the model.

Table 1 Goodness-of-fit indices for the confirmatory model QPC-FIPS Spanish.
Full size table

The adaptation in Spanish reproduces the structure of the instrument in its original language where the different items are grouped into 7 factors. Figure 1 shows the saturation of all items. All saturations were equal to or greater than 0.40.

Figure 1
figure 1

Factor loadings derived from the LS estimation (least squares).

Full size image

Convergent validity

Convergent validity was analyzed using Spearman’s correlation coefficient with the NTP 394 general satisfaction scale: Overall Job Satisfaction11. The correlation obtained was rho = 0.438 (p < 0.0001). Table 2 shows the correlations between factors and with the total instrument score. F1."Encounter" and F2." Participation" correlated most strongly with the total instrument (rho = 0.886 and rho = 0.834 respectively).

Table 2 Correlations among the subscales of the SPANISH QPC-FIPS.
Full size table

Reliability

Table 3 shows the description of each of the items. The mean value of the items ranged from 2.27 to 3.71 (standard deviation from 0.54 to 1.0). On the other hand, the corrected total item correlation was higher than 0.20 in all items. Nor does instrument reliability improve if any of the items are eliminated.

Table 3 Descriptive Statistics of the Items of the QPC-FIPS Scale and Cronbach’s alpha.
Full size table

Cronbach's alpha of the internal consistency coefficient of the total instrument scored high. However, it should be noted that three of the seven factors (F3.“Discharge”, F5.“Secluded Environment” and “F6.Secure Enviroment”) obtained lower values. Table 4 shows the results.

Table 4 Spanish QPC-FIPS.
Full size table

The ICC analysis showed that the test–retest reliability was 0.802 (95% CI: 0.689–0.874; n = 77), and this value was greater than. 70 in all instrument factors except F1."Encounter" and F4." Support" with values of 0.641 and 0.605, respectively (Table 4).

Discussion

The objective of this study was to adapt the Quality in Psychiatric Care Inpatient Staff (QPC-FIPS) instrument to Spanish, as well as analyzing its reliability and validity.

The process of translation and back-translation from the original version24 did not present any major difficulties. The expert committee suggested modifying two elements to improve comprehension in their cross-cultural adaptation. These results are consistent with what is specified in the Danish adaptation25 of the instrument, therefore, it is demonstrated that the construct of quality in care in penitentiary mental health units has a similar meaning to that proposed in the Swedish or Danish context.

In relation to the sample used, it is worth noting the smaller number of participants (153) in our study compared to the original Swedish version (348)24 and the Danish adaptation (630)25; although the percentage proportion of women (60%) and men (40%) with a similar mean age and representation of the different professional categories in a similar way is maintained. It should be noted that for the Spanish adaptation of the QPC-IPS instrument the sample was similar (163) and adequate psychometric properties were also obtained21.

At the psychometric level, the results of the Spanish adaptation of the QPC-FIPS were adequate. Regarding construct validity, the CFA showed a significant chi-square value (χ2 = 1115.705; df = 506; p < 0.0001); It should be noted that the GFI and AGFI values were ≥ 0.90. The ratio between chi square and degrees of freedom (χ2/df) was ≤ 3. The RMSEA, SRMR and RMR index values indicated values ≤ 0.08. results which are very similar to the Danish25 and Swedish versions24 of QPC-FIPS and Spanish adaptation of QPC-OPS23 and QPC-IPS21.

The convergent validity of the instrument was calculated using the Spearman’s correlation ratio between the QPC-FIPS and the NTP 394 general satisfaction scale: Overall Job Satisfaction. The correlation obtained was positive. These data were similar to those reported in the results of the QPC-OPS (rho = 0.31)23 and QPC-IPS (rho = 0.58)21 validation studies. Furthermore, as an additional method, a Spearman’s correlation analysis was also performed between the factors of the QPC-FIPS questionnaire. It was observed that, in the vast majority of cases, the correlation coefficients between the QPC-FIPS factors were moderate (0.40–0.60). While it is true that a greater degree of correlation between the correlation coefficients is observed in the Danish25 or Swedish version24, we noted that the correlation was higher between each factor and the general scale than the correlations between the subscales, confirming the Fayer and Machin hypothesis35.

With respect to reliability, the internal consistency of the instrument was evaluated at a general level, showing a Cronbach's alpha of 0.91, which is similar to that obtained with the original version (0.94)24, the Danish adaptation (0.93)25. and of the Spanish adaptations of the QPC-IPS (0.92)21 and QPC-OPS(0.88)23. Regarding the internal consistency of each of the factors, it should be noted that in each of the versions of the QPC-FIPS created so far, Swedish, Danish and Spanish, the factors with the lowest score are F3. "Discharge", F5. "Secluded environment" and F6. "Secure Environment"; with F7 "Forensic specific"24 added to this list from the Danish version25. The ICC analysis showed that the test–retest reliability was adequate at 0.80 (95% CI: 0.689–0.874; n = 77). These results were similar to those reported in validations of QPC versions that have reported the ICC: QPC-IPS 0.91 (95% CI: 0.87–0.94; n = 92)21 and QPC-OPS 0.84 (95% CI: 0.790–0.888; n = 157)23.

When analyzing the psychometric measures of the QPC-IPS-Spanish scale, validated in the hospital context, we found that the factors indicated above: F3. "Discharge", F5. "Secluded environment" and F6. "Secure Environment" also show a lower score21. Finally, the psychometric properties of this instrument also agree with those obtained in the adaptation of QPC_OPS-Spanish scale, validated at the community level23.

Limitations

One limitation was the sample size used in this study. Since the sample is a factor that interacts with aspects of the design such as the communality of the data and the number of variables per factor, following the recommendations a sample size of 200 subjects is required when we have communalities between 0.40 and 0.70, and the number of variables per factor is 3–4 items37. In this study, the sample was slightly less than 200 so the data should be interpreted with caution. However, recruitment itself was a limitation due to the limited number of professionals working in the field of forensic psychiatry. However, this limitation is mitigated by the comparison of similar psychometric outcomes in the original studies24 as well as those carried out in our context with mental health professionals from other healthcare settings21.

In terms of convergent validity, one limitation is the use of NTP 394 general satisfaction scale: Overall Job Satisfaction11. This is a questionnaire that allows us to assess overall job satisfaction. It may be that this instrument is not the most suitable for analysing this validity, but it has been translated and validated in Spanish and is one of the most widely used instruments. It was selected because although the two instruments do not assess exactly the same thing, the literature indicates that professionals who report higher perceptions of quality of care are those who report higher job satisfaction9.

Steps should be taken in future research to avoid the present limitations.

Conclusion

The results obtained indicate that the psychometric properties in terms of temporal stability and internal consistency are adequate and that the factorial structure of the Spanish version of the QPC-FIPS is equivalent to the original Swedish version24, demonstrating that the concept of quality perceived by professionals at forensic psychiatry units in Spain fits the concept of their counterparts in Sweden24. Likewise, the factorial loads, such as the homogeneity of the dimensions, were also similar to those of the Swedish version. The Spanish version of QPC-FIPS is the first instrument that allows assessment of the perceptions of forensic psychiatry professionals in Spain regarding the care they provide.