Introduction

Various approaches have been initiated to improve quality of care [1]. As generally accepted quality performance measurements were lacking, an international project defined a set of quality of healthcare indicators [2], including patient-centered care with empowerment of the patient and family. Consequently, patient satisfaction was then gradually recognized as an important tool to evaluate healthcare systems [3, 4].

Over the past few decades, integration of patient and family perspectives in clinical practice evolved slowly. There were some early initiatives concentrating on the needs of patient and family members, but the evaluation tools used did not always take into account their experiences [5–8]. Researchers started to develop instruments to explore user experiences with intensive care a few years ago, but only a few reliable family satisfaction surveys claim to be driven by patient and family experiences [9–13]. It would seem that continuous assessment of patient satisfaction as a quality performance indicator has not yet been widely accepted [14].

Similar to adult intensive care units, PICUs tend to concentrate on clinical outcome parameters such as mortality, morbidity, length of stay or survival outcomes to justify quality of care. Emphasizing partnership between parents and healthcare professionals, the PICUs in The Netherlands have recognized parental satisfaction as a quality performance indicator. This collaborative decision resulted in the multicenter Empowerment of Parents in the Intensive Care (EMPATHIC) study, a multiphase project designed to develop and implement a PICU parent satisfaction instrument. The study started with identifying satisfaction items described in the literature [15]. The next phase was a descriptive study on perceptions of nurses and physicians working in the PICUs [16]. In the present study we explored how parents perceived the identified satisfaction with care issues. The objectives were twofold: (1) to assess perceptions of parents who had experienced a PICU admission of their child on parental satisfaction issues, and (2) to perform factor analysis to evaluate the satisfaction items within domains.

Methods

The study was designed as a multicenter prospective cohort study. Seven of the eight multidisciplinary PICUs in The Netherlands participated in the study. All PICUs are level III tertiary referral centers with the capacity to provide transport facilities, (non)invasive ventilation, support multisystem failure management, postsurgical care, and other complex procedures and interventional care. Number of beds per PICU ranges from 8 to 24. The annual total number of admissions is approximately 4,500. In 2005 the median age of admitted children was 1.8 years (P25–75 0.3–7.8 years), the median length of stay was 3 days (P25–75 2–6 days), and 55.8% were mechanically ventilated [17]. One 14-bed PICU, with approximately 600 yearly admissions, did not participate because the parents were already involved in another study. The study protocol was approved initially by the medical ethical review board of the Erasmus Medical Center in Rotterdam and subsequently by the review boards of the participating centers.

Participants

The study population consisted of parents or legal caretakers of children discharged from a PICU in the period July through October 2007. No minimum stay was defined, because the overall aim of the project was to develop a satisfaction instrument for all parents. Excluded from the study were parents whose child had died, either during PICU admission or after discharge from the PICU, and parents of children who had been readmitted in the study period (Fig. 1). The rationale for excluding parents of children who had died was the assumption that other research methods would be more appropriate for this group [18–20]. In the eventuality of transferral from one to another PICU, the parents’ opinion was sought for the first admission only.

Fig. 1
figure 1

Flowchart of the inclusion of study participants

Full size image

Ethnicity was determined by checking the child’s first and family name using the combined name method [21]. Ethnicity was categorized into Dutch and non-Dutch.

Questionnaire

In a previous Delphi study, a self-administered questionnaire had been developed from literature data and opinions of nurses and physicians in eight PICUs [16]. The questionnaire included 78 items of satisfaction with care in five domains: Information, Care and Cure, Organization, Parental Participation, and Professional Attitudes. Based on the empirical data of the nurses and physicians, factor analysis showed that 72 items had standardized factor loadings above 0.50. The Cronbach’s α of the domains varied from 0.74 to 0.92.

Parents received a letter explaining the aim and content of the study 2–3 weeks after discharge. Attached were the above-mentioned questionnaire, a consent form, and a post-paid reply envelope. Parents were asked to rate the importance of the items. The rating scale was a ten-point scale ranging from 1, completely unimportant, to 10, extremely important. An open-ended question was included asking for any comments. As The Netherlands is a multicultural society, the questionnaire was available, apart from in Dutch, in Arabic, Turkish, and English. Parents could obtain translated versions by returning a reply slip in three languages included in the invitation letter. The Dutch and English versions were also available online at www.empathic.nl through an individual code provided in the invitation letter.

Statistical analysis

Descriptive statistics were used to calculate mean scores and standard deviations of the satisfaction items for ranking the importance of the satisfaction items. Importance was defined as the highest mean and the smallest standard deviation.

Each center provided data of the participating children via the Pediatric Intensive Care Evaluation database. Anonymity was protected by coding. Distribution-free tests, i.e., the Mann–Whitney test for two independent samples and the χ 2 test, served to compare characteristics of respondents and nonrespondents.

The empirical data of the parents were subjected to factor analysis aimed at identifying and estimating the dimensional structure and importance of individual items within that structure. Confirmatory factor analysis (CFA) was chosen instead of explorative factor analysis because the latter has limitations in statistical testing. In addition, the adequacy of model fit could be tested on statistical plausibility. The performance measures applied were the χ 2 test of model fit and the χ 2 test of model of fit for the baseline model. Values of χ 2, P values, and degrees of freedom (df) were scrutinized. For an adequate model fit the χ 2 test of model fit had to be nonsignificant, which meant that the identified model adequately represented the interrelationship of data, the so-called observed correlation matrix. The reproduced correlation matrix, calculated from the identified model, must closely correspond to the observed correlation matrix. The ratio of \( \frac{{\chi^{2} }}{df} < 1.5 \) would represent a good model fit. Other tests used for the model fit were: comparative fit index (preferably CFI >0.95), Tucker–Lewis index (preferably TLI >0.95), root-mean-square error of approximation (preferably RMSEA ≈ 0.05), and the weighted root-mean-square residual (preferably WRMR <1.00) [22].

As a measure of relative importance of the individual items, the standardized factor loadings, theoretically varying from −1 to +1, are presented. Reliabilities of the domains were estimated by Cronbach’s α, with theoretically a maximum value of 1.0.

The data were analyzed using SPSS (version 15, Chicago, USA) and the statistical modeling program Mplus (version 5, 2007, Los Angeles, USA).

Results

In the 4-month data collection period 1,177 children were discharged alive from the participating PICUs. Parents of 1,042 children were eligible to participate in the study. A total of 559 parents (54%) completed the questionnaire (Fig. 1). The response rate per center ranged between 49% and 60%. No more than 34 parents (3%) completed the questionnaire online. Only three requests were received for translated versions: one for the Arabic version and two for the Turkish version. Most children were Dutch (n = 805, 77.3%). In the non-Dutch group (n = 237), 47 (4.5%) were Turkish, 70 (6.7%) were Moroccan, 43 (4.1%) were Surinamese, and 77 (7.4%) were categorized as “other”. Characteristics of the children in the response and nonresponse groups are presented in Table 1. Significant differences between these groups were noted for ethnicity, PICU admission (unplanned versus planned), and number of ventilation days.

Table 1 Characteristics of the children
Full size table

Table 2 presents the mean score and standard deviation (SD) for the individual items; they are ranked per domain on the highest mean and lowest standard deviation. The item “Caregivers give the highest priority to the child’s health” was ranked as most important. Parents felt an organizational issue to be least important: “A locker on the PICU is available for all parents.” For 17 satisfaction items the mean score was below eight. This cut-off point was determined from the empirical data due to lack of scientific rationale. The standard deviations of these 17 items exceeded 1.65 (Table 2). Six were related to information giving, in particular the accessibility and manner of information giving. The mean scores of the domains were: Information 8.64 (SD 0.86); Care and Cure 8.92 (SD 0.88); Organization 8.36 (SD 1.07); Parental Participation 8.39 (SD 1.11); Professional Attitude 8.72 (SD 0.91).

Table 2 Mean, standard deviation, standardized factor loadings, and reliability estimates of the satisfaction items
Full size table

Parents of the non-Dutch group rated two cultural-related items with higher means than did the parents of the Dutch group. The first item concerned the use of interpreters or interpreter-telephone services: mean 8.09, SD 2.24 versus mean 7.86, SD 2.26 (P = 0.43). The second item was the alertness of the cultural background of child and parents; mean 8.19, SD 1.71 versus mean 7.71, SD 2.04 (P = 0.03).

Altogether 203 (36.3%) parents commented in response to the open-ended question. Of these, 37 comments were words of thank to the PICU team. The remaining comments were analyzed by two researchers (J.M.L., B.E.-S.) comparing the parental narratives with the topics of the questionnaire. No additional items were identified from the qualitative data. Comments generally complemented the items in the questionnaire. For example, “What surprised us was the limited space around the bed, and a lot of, in our view, moving of the beds. As parents you soon feel yourself in the way. Annoying were the noise and open discussions across the whole unit about patients” [respondent 233-038]. This quote was categorized to four satisfaction items related to bed space, noise levels, unnecessary discussions at the bed side, and privacy protection.

Confirmatory factor analysis showed that four items did not fit the empirical structure of the domains to which the items belonged. These items were removed from the empirical structure for further analysis. Furthermore, four domains appeared to be unidimensional. The Information domain was two dimensional, with items specifically related to the dimensions Care and Accessibility (Table 2).

Although the model fits of the factor structures of the domains were weak (Table 3), the standardized factor loadings of the individual items were adequate with values above 0.50 except for one item, “Caregivers daily inform parents about the child’s care and treatment” (0.38). Reliability was estimated for both the items and the domains (Tables 2, 4). On the domain level, the reliabilities (estimated by Cronbach’s α) for five of the six domains were above 0.80. The sixth domain, Information-Accessibility, had a Cronbach’s α of 0.79, which might still be qualified as acceptable. The corrected item-total correlations for the domains varied from 0.4 to 0.83. As a measure of internal consistency, the values of Cronbach’s α were above 0.85, with the exception of all items in the domain Information-Accessibility (Table 2).

Table 3 Performance of the models
Full size table
Table 4 Descriptives and reliability estimates for domains
Full size table

Discussion

Intensive care professionals are increasingly concerned about patient and family-centered care and involving family members in the care of the patient. In addition, the past decade showed an increase in explorative studies on perceptions of patient and family member. Both have led to the issuing of clinical practice guidelines on family-centered care in intensive care units [23]. It is debatable whether healthcare providers have truly gone along with these recommendations [6, 24]. Assessment of parental experiences would elucidate the extent to which family-centered care corresponds to the parental expectations. Standardized parental satisfaction outcomes may then ultimately provide interventions to improve clinical practice. The parents in our study classified the importance of satisfaction measures, providing insight into their perceptions and experiences.

Several studies have provided state-of-the-art knowledge of parental needs in the PICU, as established by the modified 45-item Critical Care Family Needs Inventory [25–28]. The top 20 needs in these studies tend to focus on information provision, such as information on illness, progress, prognosis, and “knowing what is being done” [29]. Similar findings were reported in a systematic review of 115 articles on families with critically ill children [30]. Nevertheless, the authors concluded that most reports were anecdotal, often had small sample size, and that some showed methodological flaws.

Our data set of 78 satisfaction items covered a wider range of care aspects than only the parental needs. The large sample in our study rated several items in all domains very highly, indicating that issues other than just receiving adequate information at the right moment are important. Not surprisingly, the child’s care is of importance to them, as well as behavior and attitude of the healthcare professionals. Some of these issues emerged also from a recent qualitative study among 20 parents of hospitalized children: 11 in a PICU and 9 in a general ward [31]. Parents of children in the PICU had more stress, and this affected their psychological outcomes. Importantly, stress factors were related to the severity of the child’s illness, varying from admission to post discharge. This implication strengthens our aim to develop a core set of satisfaction items that covers the entire PICU stay, including the discharge process.

The widely used Parental Stressor Scale: PICU, developed by Carter and Miles [32], is a valuable tool to study parental stressors. Two small studies using this instrument demonstrated that the child having to undergo procedures or tests, the child having an endotracheal tube, and overall experiences in the PICU are the most important stressors for mothers (n = 31) and fathers (n = 15) [33, 34]. Furthermore, particularly mothers stated that receiving information from too many professionals is highly stressful. In this respect, the Creating Opportunities for Parent Empowerment (COPE) program offers a structured information intervention and directs parental participations [35]. In the study by Melnyk et al., mothers in the COPE program group reported significant less stress [35]. Regrettably, satisfaction was not considered as an outcome measure. For deeper understanding of parental experiences of a PICU admission, it might be advisable to combine a satisfaction survey with a stressor scale. Findings from such an approach might guide professionals towards combined interventions aimed at decreasing stress levels among parents, possibly resulting in higher satisfaction outcomes.

Availability of a valid and reliable satisfaction instrument is likely to contribute to general acceptance of parental satisfaction as a quality performance outcome. Measuring parental satisfaction and evaluating outcomes among several PICUs will, in all probability, provide opportunities for quality programs based on best practices. In other words, PICUs with low satisfaction ratings on certain care aspects may learn from a PICU with high satisfaction ratings on these aspects. Via these best practices PICUs are able to work on continuous collaborative quality improvement.

The limitations of the study should be addressed. First, parents’ characteristics were not taken into account, albeit on purpose. The overall aim was to develop a parental satisfaction instrument for the general population in the PICU. Additional questions about family composition and characteristics would have raised the already high number of items in the survey and could have reduced the response rate. This argument is counteracted, however, by a study from Jenkinson and colleagues that reported no significant difference in response rate between a satisfaction survey with 15 items and one with 108 items [36].

Second, the satisfaction items had been generated from the existing literature and the opinions of PICU professionals. It could be argued that, in the initial stage of the developmental process, an explorative study among parents might have been appropriate to identify the satisfaction items. Such a qualitative study might have resulted in other satisfaction items. We believe, however, that the approach we opted for created a firm enough basis for further development.

Third, the overall response rate of 54% is fairly low. Comparable studies using data collection via mail obtained response rates between 60% and 70% [37–39]. Still, we made some efforts to achieve a high response rate, i.e., translating the questionnaire, providing an online submission option, and sending a reminder. The face-to-face approach might be a better method in future studies, since it is associated with a significant higher response compared with mailing the survey [38].

In conclusion, a large group of parents of children admitted to a PICU in The Netherlands provided their perceptions on satisfaction with care topics based on their experiences. These valuable perceptions will make it possible to further develop a satisfaction instrument specifically for the PICU and based on the empowerment of parents. The empirical structure of the satisfaction items and domains can be considered adequate and the reliabilities of the domains are of high level. The results provide a scientific basis for further modification of the instrument. Redundant items that measure the same concept might be eliminated, leaving a still reliable and valid satisfaction instrument for PICU services.

Future research should also include open-ended questions asking parents for suggestions on how care could be improved. These important qualitative data can clarify and complement the quantitative results, and the synergy of these data makes it possible to identify areas to improve clinical practice [9, 40]. Furthermore, it might be recommended to design satisfaction instruments that allow to discriminate groups of parents such as parents of different cultural backgrounds. Also, parents of children hospitalized for postoperative care are assumed to have different experiences than parents of a child admitted with a life-threatening condition. Finally, parents of different cultural backgrounds hardly made use of one of the translated versions of questionnaires. Different approaches such as giving the questionnaire face-to-face in their own language might be recommended to increase their participation.

Ultimately, the instrument should be able to provide parents with a tool to collaborate with healthcare professionals in quality improvement of PICU services.