Intensive Care Medicine

, Volume 37, Issue 2, pp 310–318

Construction and psychometric testing of the EMPATHIC questionnaire measuring parent satisfaction in the pediatric intensive care unit

  • Jos M. Latour
  • Johannes B. van Goudoever
  • Hugo J. Duivenvoorden
  • Marcel J. I. J. Albers
  • Nicolette A. M. van Dam
  • Eugenie Dullaart
  • Marc van Heerde
  • Marjorie de Neef
  • Carin W. M. Verlaat
  • Elise M. van Vught
  • Jan A. Hazelzet
Open AccessPediatric Original

DOI: 10.1007/s00134-010-2042-y

Cite this article as:
Latour, J.M., van Goudoever, J.B., Duivenvoorden, H.J. et al. Intensive Care Med (2011) 37: 310. doi:10.1007/s00134-010-2042-y
  • 961 Views

Abstract

Purpose

To construct and test the reliability and validity of the EMpowerment of PArents in THe Intensive Care (EMPATHIC) questionnaire measuring parent satisfaction in the pediatric intensive care unit (PICU).

Methods

Structured development and psychometric testing of a parent satisfaction-with-care instrument with the results of two cohorts of parents (n = 2,046) from eight PICUs in the Netherlands.

Results

In the first cohort, 667/1,055 (63%) parents participated followed by 551/991 (56%) parents in the second cohort. The empirical structure of the instrument was established by confirmatory factor analysis with the first sample of parents confirming 65 statements within five theoretically conceptualized domains: information, care and cure, organization, parental participation, and professional attitude. The standardized factor loadings were greater than 0.40 in 63 statements. Cronbach’s α, a measure of reliability, per domain ranged from 0.73 to 0.93 in both cohorts with no significant difference documenting the reliability over time. Beside rigorous content and face validity, the congruent validity of the instrument showed adequate correlation with four gold standard questions measuring overall satisfaction. The non-differential validity was confirmed with no significant differences between the population characteristics and the domains, except that parents with a child for a surgical admission were more satisfied on information issues.

Conclusions

The final EMPATHIC questionnaire incorporates 65 statements. The empirical structure of the satisfaction statements and domains was satisfactory. The reliability and validity proved to be adequate. The EMPATHIC questionnaire is a valid quality performance indicator to measure quality of care as perceived by parents.

Keywords

Family satisfactionParentsCareReliabilityValiditySurveyQuality of care

Introduction

Performance measures of a pediatric intensive care unit (PICU) have traditionally been documented by clinical and physical parameters of critically ill children [13]. Others have focused on long-term outcomes of children after PICU treatment to evaluate pediatric intensive care, in particular health-related quality of life [4, 5]. Nevertheless, healthcare professionals are not only responsible for healthy outcomes of children. Parents too are part of the integrated care system and they have a right to receive the best care. Looking from a quality of care perspective, parent satisfaction is an important area to evaluate the desirable outcome of healthcare performances [6]. Given the increased awareness of consumer-driven healthcare systems, it is not surprising that patient and family satisfaction is increasingly accepted as an important quality indicator in critical care [7]. It is expected that an increasing number of PICUs already use satisfaction surveys to document their quality of care. Whether these surveys are of an acceptable academic standard is unknown. The reality remains that only two parent satisfaction questionnaires for PICU services are documented in the literature [8, 9].

With respect to the empowerment of parents who experienced a PICU admission of their child, their knowledge is valuable for documenting and improving quality of care. Therefore, a large group of parents were consulted in developing a parent satisfaction instrument together with the Dutch PICU nurses and physicians [10, 11]. Their views on satisfaction-with-care topics were the foundation to create a set of quality indicators measuring the perceived care by the parents. The next step and the aim of this study were to construct and evaluate the reliability and validity of the EMpowerment of PArents in THe Intensive Care (EMPATHIC) questionnaire measuring parent satisfaction.

Methods

This multicenter survey study was designed to assess the psychometric properties of the EMPATHIC questionnaire. All eight PICUs in the Netherlands participated in the study. Data were collected during two cohort periods of 4 months between February and September 2009. The medical ethical review board of the Erasmus Medical Center in Rotterdam approved the study followed by approval of the review boards of the participating medical centers.

Questionnaire construction

The item generation and selection of the questionnaire followed a structured process. The first steps concerned item generation. A literature review was performed to generate items from the existing satisfaction surveys [12]. The following step was a Delphi study to identify the importance of the items among 269 PICU nurses and physicians including the identification of possible new items [10]. This study resulted in 78 individual satisfaction-with-care items. Subsequently, a convenience sample of parents (n = 559) recruited from seven PICUs was asked to rate the importance of these 78 items [11]. Also, in-depth interviews with 41 parents were held to explore their experiences. Qualitative analysis revealed 63 subthemes divided into 6 themes. Following the item generation steps, item selection was based on the priorities of the opinions of parents and healthcare professionals. Of the generated items, 60 were found to be most relevant for measuring parent satisfaction with care. The items were rephrased into statements appealing to parents. Several items (n = 18) were considered important to differentiate between nurses and physicians. For example, the item about emotional support given to parents was rephrased into two statements; the first measuring the nurses’ emotional support and the second measuring the physicians’ emotional support. The draft questionnaire contained a demographic section to obtain characteristics of the child and its family, the 78 statements, four control questions about overall satisfaction-with-care, and free space was made available for the parents to write comments on their experiences. A 6-point scale, from 1 ‘certainly no’ to 6 ‘certainly yes’, was used for the statements. Parents could also tick a box if the statement was not applicable. This draft was introduced to eight nurses and eight physicians of the EMPATHIC study group and to ten parents. The aim was to investigate whether the statements were understandable and clearly formulated. The feedback resulted in some textual adjustments to the statements. The questionnaire was then translated into Arabic and Turkish by registered translators since a considerable number of parents might have difficulty in reading and understanding the Dutch questionnaire.

Statistical analyses

Descriptive statistics were used to calculate mean scores and standard deviations of the statements. Before analyzing the psychometric properties of the instrument, four negatively formulated statements were deviated from the other statements. Two statements were deleted because more than 75% of the parents scored these as not applicable leaving 72 statements for analysis. For each statement principal component analysis for the non-numerical data was performed to determine the scale value of the answer alternative ‘not applicable’. Two-dimensional plots of the individual statements revealed that this value is closely related to the highest score on the 6-point scale. Therefore, it was justified to impute the answer alternative ‘not applicable’ to the highest value (in this case 6), meaning that parents who ticked the answer alternative ‘not applicable’ belong to the highest category of the 6-point scale.

Structural equation modeling is an advanced multivariate analysis technique using various analysis methods to identify, test, and estimate the relationship between study variables. Confirmatory factor analysis (CFA) is tailored to unraveling the empirical structure of the interrelationship of the 72 statements. The final model was based on both theoretical and statistical plausibility. The measures applied in this study were χ2 test of model fit, and the ratio of \( {\frac{{\chi^{2} }}{df}}\, < \,3 \) represents a good model fit. Other tests used for the model fit were: comparative fit index (preferably CFI ≥ 0.95), Tucker–Lewis index (preferably TLI ≥ 0.95), root mean square error of approximation (preferably RMSEA <0.08), and the weighted root mean square residual (preferably WRMR < 0.90) [13].

Reliability in this study refers to the internal consistency of the statements within the instrument. Internal consistency reliability measures were performed on the individual statements of the domains identified. On domain level the Cronbach’s α was assumed to be satisfactory with values between 0.70 and 1.00. In order to get insight into the stability of the findings over time, the means of the domains at two measurement moments were tested for difference using the Levene’s test for equality of variances and t test for equality of means. If no statistical difference occurred, it was considered that the reliability over time was adequate.

Validity refers to the degree in which an instrument measures what it is intended to measure. Content validity and face validity were ensured as described by the questionnaire construction above. Congruent validity was assessed by using the Spearman’s rank correlation for estimating the relationship between the statements on domain level and four overall satisfaction-with-care scales. Furthermore, the non-differential validity referring to variables, which were assumed to have non-differential effects, was calculated by using the effect size of standardized mean difference (Cohen’s d) between the demographic variables and the domains. The effect size is small with a value of 0.20, medium with 0.50, and large with greater than 0.80 [14].

All significance testing was set at P ≤ 0.05 (two-tailed). SPSS (version 15, Chicago, USA) and the statistical modeling program Mplus (version 5, 2007, Los Angeles, USA) were used for analysis.

Participants

The sample size was aimed at 600 parents at least for adequate psychometric analysis and a second cohort of 600 parents to test the stability of the instrument over time. Parents whose child was discharged alive from a PICU were recruited. If a child had one or more PICU admissions in the study period, the parents received only one questionnaire at the first admission. Parents whose child had died were excluded because in the Netherlands these parents receive an invitation to meet with the PICU physician and nurse 6 weeks after their child’s death. During this meeting all aspects of treatment and care are evaluated. Two to three weeks after PICU discharge parents received an invitation letter, including the information about the study, the EMPATHIC questionnaire, a consent form to be signed, and a prepaid reply envelope. One reminder was sent after 3 weeks if no response was received. Ethnicity of the parents was determined by asking the cultural background of the family in the demographic section of the questionnaire. In the analysis ethnicity was categorized into Dutch and non-Dutch.

Results

In the first 4-month data collection period the questionnaire was mailed to 1,055 parents and 667 (63%) parents responded. In the second 4-month period 551/991 (56%) parents responded. Between both cohorts of parents, the questionnaire was completed by almost similar percentages of mothers (61.5 vs. 64.3%), fathers (11.9 vs. 11.4%), both mother and father (24.3 vs. 23.2%), and others (2.3 vs. 1.1%). The characteristics of the children and parents of the two cohorts are presented in Table 1 and they are not significantly different.
Table 1

Characteristics of children and parents

 

Cohort 1

Cohort 2

P value

n

 

n

 

Age in months; median (P25–75)

660

30 (7–114)

539

36 (7–120)

0.59a

Length of stay in days; median (P25–75)

656

3 (2–7)

537

3 (2–7)

0.42a

PICU admission

662

 

546

 

0.61b

 Unplanned

365

55.1%

293

53.7%

 

 Planned

297

44.9%

253

46.3%

 

Type of admission

652

 

534

 

0.65b

 Surgical

359

55.1%

301

56.4%

 

 Medical

293

44.9%

233

43.6%

 

Mechanical ventilation required

661

 

541

 

0.77b

 Yes

405

61.3%

336

62.1%

 

 No

256

38.7%

205

37.9%

 

Family culture

664

 

551

 

0.33b

 Dutch

576

86.7%

487

88.4%

 

 Other

88

13.3%

64

11.6%

 

PICU pediatric intensive care unit

aMann–Whitney U test

bChi-square test; total numbers of respondents vary due to missing data

During the process of structural equation modeling, the CFA showed that seven statements did not fit into the empirical structure of the domains. These statements where therefore removed from further analysis. Overall, the performance of the model fit of the remaining 65 statements in the five domains was adequate (Table 2). The standardized factor loadings of the statements were above 0.40 except for two statements (Table 3). The statement about the physician’s knowledge of the child’s medical history had a standardized factor loading of 0.39 and the statement about having an assigned physician during PICU stay had a standardized factor loading of 0.36.
Table 2

Performance of the models

Domains

No. of statements

n

Chi-square test of model fit

Value

df

P value

CFI

TLI

RMSEA

WRMR

Information

9

667

83.60

17

<0.01

0.97

0.99

0.08

0.82

Care and cure

30

667

454.74

99

<0.01

0.95

0.98

0.07

1.09

Parental participation

8

667

55.64

12

<0.01

0.99

0.99

0.07

0.75

Organization

6

655

18.50

8

0.02

0.99

1.00

0.05

0.46

Professional attitude

12

667

141.26

27

<0.01

0.98

0.99

0.08

0.94

Range of scoring scale was 1–6

df degrees of freedom, CFI comparative fit index, TLI Tucker–Lewis index, RMSEA root mean square error of approximation, WRMR weighted root mean square residual

Table 3

Mean, standard deviation, and standardized factor loadings of satisfaction statements (n = 667)

 

\( \hat{\mu } \)

\( \hat{\sigma } \)

Standardized factor loadings

Information

 We were always informed right away when our child’s physical condition worsened

5.60

0.86

0.63

 We had daily talks about our child’s care and treatment with the nurses

5.50

0.95

0.69

 Our questions were clearly answered by the nurses

5.48

0.88

0.72

 Our questions were clearly answered by the doctors

5.42

1.00

0.78

 The doctor clearly informed us about the consequences of our child’s treatment

5.33

1.13

0.75

 We received clear information about the examinations and tests

5.32

1.03

0.84

 We were given clear information about our child’s disease

5.30

1.20

0.61

 We received understandable information about the effects of the drugs

5.06

1.29

0.66

 We had daily talks about our child’s care and treatment with the doctors

4.87

1.56

0.63

Care and cure

 When our child’s condition worsened, action was immediately taken by the doctors

5.79

0.56

0.65

 When our child’s condition worsened, action was immediately taken by the nurses

5.79

0.59

0.70

 During acute situations there was always a nurse to support us

5.64

0.82

0.67

 The team was alert to the prevention and treatment of pain in our child

5.60

0.76

0.73

 The doctors and nurses are real professionals; they know what they are doing

5.59

0.72

0.78

 Our child’s comfort was taken into account by the nurses

5.58

0.81

0.80

 The team had a common goal: the best care and treatment for our child and ourselves

5.56

0.76

0.91

 Our child’s needs were well responded to by the nurses

5.55

0.83

0.78

 Attention was paid to our child’s developmental level by the nurses

5.53

0.94

0.78

 The team was helpful to our child and to us

5.52

0.88

0.89

 Our own needs were well responded to by the nurses

5.49

0.86

0.80

 Our child’s needs were well responded to by the doctors

5.49

0.90

0.70

 Attention was paid to our child’s developmental level by the doctors

5.47

1.03

0.68

 Every day we knew who was responsible for our child, regarding the nurses

5.46

1.10

0.58

 Our child’s comfort was taken into account by the doctors

5.44

0.91

0.71

 The correct medication was always given on time

5.39

1.03

0.70

 We were well supported emotionally by the nurses

5.38

1.07

0.69

 Our own needs were well responded to by the doctors

5.37

1.00

0.71

 At admission our child’s medical history was known by the doctors

5.37

1.25

0.39

 The doctors and nurses worked closely together

5.34

0.93

0.66

 Transferral of care from the PICU staff to colleagues in the pediatric ward had gone well

5.17

1.30

0.58

 We were well prepared for our child’s discharge by the nurses

5.14

1.30

0.56

 At admission our child’s medical history was known by the nurses

5.12

1.41

0.41

 We were well prepared for our child’s admission by the nurses

5.11

1.47

0.41

 We were well supported emotionally by the doctors

5.07

1.32

0.62

 We were well prepared for our child’s admission by the doctors

5.04

1.49

0.42

 We were well prepared for our child’s discharge by the doctors

4.87

1.53

0.50

 During our child’s stay we were assigned to a first responsible nurse

4.77

1.76

0.47

 Every day we knew who was responsible for our child, regarding the doctors

4.62

1.72

0.53

 During our child’s stay we were assigned to one and the same doctor

4.25

1.96

0.36

Parental participation

 We had confidence in the doctors

5.63

0.75

0.63

 We had confidence in the nurses

5.58

0.85

0.63

 Even during intensive procedures we could always stay close to our child

5.57

0.94

0.65

 We were encouraged to stay close to our child

5.35

1.09

0.77

 Before discharge the care for our child was once more discussed with us by the nurses

5.15

1.29

0.62

 We were actively involved in decision-making on care and treatment of our child

5.14

1.22

0.76

 Before discharge the care for our child was once more discussed with us by the doctors

4.91

1.54

0.58

 During our stay the staff regularly asked for our experiences

4.52

1.59

0.60

Organization

 The IC unit could easily be reached by telephone

5.63

0.77

0.70

 The visiting hours were flexible

5.60

0.96

0.50

 The team worked efficiently

5.54

0.74

0.86

 There was enough space around our child’s bed

5.39

0.99

0.80

 The IC unit was clean

5.39

1.00

0.73

 Noise in the IC unit was muffled as well as possible

4.95

1.37

0.64

Professional attitude

 Our child’s health always came first for the nurses

5.68

0.70

0.74

 Our child’s health always came first for the doctors

5.68

0.72

0.71

 At admission we felt welcome

5.61

0.83

0.74

 The team showed respect for our child and for us

5.58

0.76

0.89

 There was a pleasant atmosphere among the staff

5.54

0.73

0.86

 The team worked hygienically

5.52

0.86

0.70

 We received sympathy from the nurses

5.50

0.87

0.74

 In spite of the workload, sufficient attention was paid to our child and to us by the nurses

5.49

0.87

0.78

 The team respected the privacy of our child and of us

5.30

1.05

0.69

 Nurses and doctors always introduced themselves by name and function

5.26

1.20

0.68

 In spite of the workload, sufficient attention was paid to our child and to us by the doctors

5.25

1.05

0.66

 We received sympathy from the doctors

5.21

1.18

0.58

Range of scoring scale was 1–6

\( \hat{\mu } \), mean; \( \hat{\sigma } \), standard deviation

The reliability estimates of the individual statements and domains were sufficient. On the domain level the reliability estimates, Cronbach’s α, were between 0.73 and 0.92 in the first cohort of parents. These estimates were also calculated among the second cohort of parents and showed similar outcomes from 0.73 to 0.93 (Table 4). At different time moments the two cohorts did not significantly differ on the equal variances for differences of the means on domain level and differences of variances (Table 4). Additionally, the standardized mean differences on domain level, Cohen’s d, were calculated between the two cohorts: information −0.02, care and cure −0.09, parental participation −0.12, organization −0.12, and professional attitude −0.11. Overall, the reliability and stability of the EMPATHIC questionnaire over time provided empirical evidence that the reliability of the instrument was adequate.
Table 4

Descriptives, reliability estimates, and testing on domains of two cohorts

Domains (statements)

Cohort 1 (n = 667)

Cohort 2 (n = 551)

Levene’s test differences on variances

T test differences on means

\( \hat{\mu } \)

\( \hat{\sigma } \)

α

\( \hat{\mu } \)

\( \hat{\sigma } \)

α

F

P value

P value

Information (9)

5.32

0.72

0.84

5.33

0.68

0.81

1.19

0.28

0.84

Care and cure (30)

5.32

0.60

0.92

5.35

0.61

0.93

1.19

0.28

0.35

Parental participation (8)

5.23

0.73

0.77

5.30

0.71

0.76

0.11

0.73

0.13

Organization (6)

5.42

0.63

0.73

5.47

0.61

0.73

1.56

0.21

0.19

Professional attitude (12)

5.47

0.58

0.88

5.50

0.57

0.88

0.94

0.33

0.30

P value is two-tailed; range of scoring scale was 1–6

\( \hat{\mu } \), mean; \( \hat{\sigma } \), standard deviation; α, Cronbach’s alpha on standardized items

The EMPATHIC questionnaire showed sufficient correlation with the four statements of overall satisfaction as an indicator of congruent validity (Table 5). The non-differential validity of the instrument was assessed by calculating the effect size of domains and various population variables. The four tested variables of the five domains did not show significantly differences (Table 6). The parents of a child with a surgical procedure admission were more satisfied about the information statements than parents whose child had a medical PICU admission, Cohen’s d 0.29, P < 0.001.
Table 5

Congruent validity of scales assessed using Spearman’s rank correlation

 

n

Would suggest PICU to others

Would come back again if needed

Overall satisfaction with physicians

Overall satisfaction with nurses

Information

650

0.435

0.422

0.504

0.364

Care and cure

642

0.522

0.527

0.542

0.472

Parental participation

651

0.478

0.475

0.465

0.423

Organization

651

0.523

0.508

0.392

0.401

Professional attitude

651

0.588

0.583

0.575

0.519

Correlation is significant at 0.01 (two-tailed)

Table 6

Non-differential validity, differences between domains and PICU characteristics

 

Yes

No

Cohen’s d

P value

n

\( \hat{\mu } \)

\( \hat{\sigma } \)

n

\( \hat{\mu } \)

\( \hat{\sigma } \)

Mechanical ventilation

 Information

397

5.27

0.76

251

5.40

0.65

−0.18

0.07

 Care and cure

391

5.31

0.62

249

5.34

0.55

−0.05

0.83

 Parental participation

397

5.23

0.75

251

5.24

0.70

−0.02

0.91

 Organization

396

5.41

0.67

252

5.42

0.57

−0.01

0.53

 Professional attitude

397

5.48

0.60

251

5.45

0.56

0.05

0.32

Unplanned admission

 Information

358

5.28

0.73

289

5.37

0.71

−0.13

0.06

 Care and cure

354

5.30

0.60

286

5.35

0.60

−0.07

0.24

 Parental participation

357

5.25

0.73

291

5.21

0.73

0.06

0.38

 Organization

357

5.42

0.61

291

5.42

0.65

0.01

0.97

 Professional attitude

357

5.49

0.57

291

5.45

0.59

0.08

0.46

Admission for surgery

 Information

351

5.41

0.67

286

5.20

0.78

0.29

<0.001

 Care and cure

348

5.35

0.60

284

5.28

0.60

0.11

0.11

 Parental participation

351

5.24

0.75

287

5.21

0.71

0.03

0.44

 Organization

352

5.42

0.64

287

5.42

0.62

0.00

0.92

 Professional attitude

352

5.47

0.61

286

5.47

0.55

0.00

0.39

Dutch culture

 Information

562

5.31

0.71

87

5.38

0.80

−0.09

0.11

 Care and cure

558

5.32

0.57

83

5.33

0.77

−0.01

0.09

 Parental Participation

563

5.23

0.70

86

5.27

0.88

−0.06

0.07

 Organization

564

5.42

0.62

85

5.39

0.69

0.05

0.97

 Professional attitude

563

5.47

0.56

86

5.45

0.72

0.05

0.20

Range of scoring scale was 1–6

\( \hat{\mu } \), mean; \( \hat{\sigma } \), standard deviation; Cohen’s d, standardized mean difference; P value, Mann–Whitney test (two-tailed)

Discussion

The strength of this study lies in the rigorous development of the satisfaction instrument through a convenience sample of parents and PICU healthcare professionals. Psychometric testing of the EMPATHIC questionnaire revealed that this instrument is reliable and valid for measuring parent satisfaction of pediatric intensive care services. The 65 statements representing five domains provide a comprehensive conceptualization of parent satisfaction. The relevance of this study is not limited to the development of an acceptable and feasible quality performance indicator for PICUs. Healthcare departments are increasingly demanding satisfaction outcomes of individual healthcare services above and beyond general hospital satisfaction measures [1517]. Besides, healthcare insurance companies increasingly demand patient satisfaction data to fund hospitals [18]. This study provides a valuable contribution relevant to PICU colleagues in other countries and cultures. The increased need for validated instruments to collect data of patient or family experiences is evident. The challenge is to construct and design research instruments for use in study populations with different cultural backgrounds. The EMPATHIC questionnaire showed that there were no significant differences between Dutch parents and parents with a non-Dutch background. However, these results can not be generalized when using the EMPATHIC questionnaire in other countries with other languages. Although its reliability and validity has been proven, thorough translation of the instrument is needed to provide evidence that the meaning of the translated statements is equivalent to the original statements [19]. For this translation process, a framework has been described of 10 consecutive steps, based on a review of 12 guidelines for translation and cultural adaptation, to ensure principles of good practice [20].

Only two validated parent satisfaction instruments have been documented in the literature for PICU settings [8, 9]. The instrument developed in the UK assessed the face and content validity although no reliability testing was performed [8]. Despite the proper testing of the reliability and validity of the 23-item instrument developed in the USA, the authors stated that a limitation of this instrument is its restricted number of items [9]. The authors stated that their instrument was not intended to be ‘all-inclusive’ but that it was rather directed to PICU care only. However, it is debatable whether 23 statements are sufficient to reflect PICU services. In contrast, after its validation process the EMPATHIC questionnaire included 65 statements. In this perspective, two issues are important to address: the number of statements needed to measure the complexity of PICU services and the length of the questionnaire related to the response rates. The latter is addressed in a review of 210 patient satisfaction studies. In 125 studies the number of items used in the survey was reported and ranged from 1 to 361 items [21]. Of these, there was a weak negative correlation between the number of items and the response rate. A similar effect occurred between the Picker Patient Experience (PPE-15) questionnaire embedded in a four-page survey with 31 questions including demographic details and a 12-page survey with 108 questions [22]. The analyses revealed that the length of the questionnaire did not lead to reduced response rates and there was no effect on the quality of data in terms of percentages of items completed. Therefore, the 65-statement EMPATHIC questionnaire seems feasible as it resulted in response rates of 63 and 56% in the two measurement periods. Incidentally, these response rates can be considered acceptable for satisfaction surveys because the mean initial response rate was 72.1% (SD = 19.8) in 124 satisfaction studies [21]. The other issue to be addressed is the question of how many statements are needed to measure parent satisfaction for PICU services. Basically, an instrument should include items on all factors of importance to the trait under study. This refers to the content validity of an instrument. A review of ten satisfaction instruments designed for PICU, neonatal ICU, adult ICU, and pediatric wards documented a range of 15–45 items [12]. The strength of the EMPATHIC questionnaire lies in the item generation and item selection phase where a large number of parents and healthcare professionals were consulted to identify the most important items [10, 11]. The items were converted into statements and theoretically conceptualized in domains related to family-centered care concepts [23]. It is, therefore, recognized that the 65 statements of the EMPATHIC questionnaire reflect the most important issues of the PICU care.

Two limitations of the psychometric testing of the EMPATHIC questionnaire need to be addressed. The first concern is the test–retest reliability. This refers to administering the same instrument to the same respondents at two different moments in time to estimate its stability over time. A high correlation between the two measurement periods refers to a good test–retest reliability. It was decided not to burden the parents with two questionnaires in a short period of time. Therefore, as it was assumed that neither the care and the treatment nor the population of parents would change within 8 months, two cohorts of parents were included to test the stability of the findings over time.

The second limitation is the lack of criterion validity testing of the instrument. This validity refers to association of the results between the tested instrument and another validated instrument measuring the related concept. Although one parent satisfaction instrument for the PICU was tested to be valid [9], its limitations to use it as the gold standard were based on a single center study, the limited items of the instrument, and the validity tested on a small group of 40 parents of which 27 responded. For that reason, congruent validity was chosen to examine part of the validity of the EMPATHIC questionnaire by testing the statements on four generally accepted gold standard questions measuring overall satisfaction.

Despite the recognition that satisfaction outcome measures are increasingly used as quality performance indicators [24, 25], the reality is that there is a paucity of PICU parent satisfaction instruments in the literature. However, it is expected that many PICUs have developed and carry out their own unit-based satisfaction survey. Therefore, and in conclusion, this study provides a reliable and valid parent satisfaction instrument that is feasible to administer to parents. The found empirical structure of the satisfaction-with-care statements can be considered adequate. The proven statistical evidence of the statements is of importance in order to be able to apply the EMPATHIC questionnaire in other PICUs. Nevertheless, including statements of clinical relevance specific to local PICU settings might be considered. Ultimately, using a validated instrument contributes to the empowerment of parents to work collaboratively with nurses and physicians on PICU quality of care.

Acknowledgments

We are grateful to the parents for consenting to participate in this study and providing their valuable opinions. Beatrix Elink Schuurman is gratefully acknowledged for allocating the data into SPSS. This study was partially supported by a grant, SNO-T-0501114, from the Nuts OHRA foundation. The sponsor was not involved in the study design, data collection, analysis, the manuscript, and in the decision to submit the manuscript for publication.

Conflict of interest

The authors have no financial interest or any other conflict of interest to declare.

Open Access

This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Copyright information

© The Author(s) 2010

Authors and Affiliations

  • Jos M. Latour
    • 1
    • 2
  • Johannes B. van Goudoever
    • 2
    • 3
    • 4
  • Hugo J. Duivenvoorden
    • 5
  • Marcel J. I. J. Albers
    • 6
  • Nicolette A. M. van Dam
    • 7
  • Eugenie Dullaart
    • 8
  • Marc van Heerde
    • 9
  • Marjorie de Neef
    • 10
  • Carin W. M. Verlaat
    • 11
  • Elise M. van Vught
    • 12
  • Jan A. Hazelzet
    • 1
  1. 1.Division of Pediatric Intensive Care, Department of PediatricsErasmus MC, Sophia Children’s HospitalRotterdamThe Netherlands
  2. 2.Division of Neonatology, Department of PediatricsErasmus MC, Sophia Children’s HospitalRotterdamThe Netherlands
  3. 3.Department of PediatricsAMC, Emma Children’s HospitalAmsterdamThe Netherlands
  4. 4.Department of PediatricsVU University Medical CenterAmsterdamThe Netherlands
  5. 5.Department of Medical Psychology and Psychotherapy, Netherlands Institute for Health ScienceErasmus University Medical CenterRotterdamThe Netherlands
  6. 6.Pediatric Intensive Care Unit, Beatrix Children’s HospitalUniversity Medical Center GroningenGroningenThe Netherlands
  7. 7.Pediatric Intensive Care UnitLeiden University Medical CenterLeidenThe Netherlands
  8. 8.Pediatric Intensive Care UnitMaastricht University Medical CenterMaastrichtThe Netherlands
  9. 9.Department of Pediatric Intensive CareVU University Medical CenterAmsterdamThe Netherlands
  10. 10.Pediatric Intensive Care UnitAMC, Emma Children’s HospitalAmsterdamThe Netherlands
  11. 11.Pediatric Intensive Care UnitUniversity Medical Center St RadboudNijmegenThe Netherlands
  12. 12.Pediatric Intensive Care UnitUniversity Medical Center Utrecht, Wilhelmina Children’s HospitalUtrechtThe Netherlands