Introduction

The pathophysiology of childhood constipation consists of multiple factors. Because in the vast majority of patients no evidence can be found of an inflammatory, anatomic, metabolic or neoplastic process that explains the subject’s symptoms, these patients are considered to have a functional disorder [5]. Functional constipation is a worldwide digestive problem in children and adults [28, 36]. Studies in the USA and the Netherlands showed that 10–45% of the children visiting pediatric gastroenterologists have constipation-related complaints [1, 7, 22, 35].

Functional constipation often is multifactorial and can have potential invalidating consequences such as frequent absence from school, social desolation and feelings of depression [5]. However, evidence on the optimal care for patients with functional constipation is scarce. Therefore, a multidisciplinary group of professionals in the Netherlands took the initiative to develop a national guideline on the diagnosis, treatment and follow-up of children with functional constipation.

To improve care, insight into actual care and adherence to the guideline is necessary. Actual care can be measured with quality indicators, which can indirectly lead to insight into the process of quality improvement in care [8, 12]. Quality indicators are ‘measurable elements of practice performance for which there is evidence or consensus that they can be used to assess the quality of care’ [2], and they are often translated out of recommendations from relevant clinical guidelines and literature. Quality indicators aim to detect the quality of care either in the structure or the process of medical care or in the outcome of delivered care [21]. Structure indicators assess health system characteristics such as number of staff and supplies, whereas process indicators refer to what professionals did for the patient and how well it was done. The outcome indicators assess outcomes of delivered care, which ideally can be expressed in the five Ds (dead, disease, discomfort, disability and dissatisfaction) [19]. To our knowledge, no quality indicators have yet been developed for children with functional constipation.

This paper reports on the development of quality indicators for care delivered to children with functional constipation, based on the Dutch evidence-based multidisciplinary guideline ‘Functional constipation in children between 0 and 18 years’ [5]. These indicators can be helpful in improving care for children with functional constipation. Actual care can be assessed, and low scores of the indicators point out that improvement activities are needed.

Methods and Results

The basis for the development of the quality indicators was the multidisciplinary evidence-based guideline ‘Functional constipation in children between 0 and 18 years’. The RAND-modified-Delphi method, a procedure that combines evidence from the guideline with expert opinions, was used to reach consensus [3, 4, 30]. The steps taken have been visualized in Fig. 1 and described below.

Fig. 1
figure 1

Process of quality indicator selection

Full size image

Step 1. Extraction and classification of recommendations from the guideline

Two researchers (MH and JS) extracted all 84 recommendations from the guideline and classified these recommendations into four dimensions: diagnosis (N = 23), medical treatment (N = 22), non-medical treatment (N = 15) and referral (N = 24).

Step 2. Written appraisal of recommendations by the expert panel

A representative panel of 21 experts (e.g. general practitioners, pediatric gastroenterologists (including co-authors MB and MT), primary health care doctors, pediatricians, clinical epidemiologists, a pediatric physiotherapist, a pediatric surgeon, a psychologist, the chair of the patient association, a hospital pharmacist and a nurse consultant, who had been involved in the guideline development, was also approached to participate in the development of the quality indicators. The panel was asked to score the recommendations on the relevance to the quality of care for children with functional constipation. A nine-point Likert scale ranging from 1 (hardly relevant) to 9 (extremely relevant) was used to rate the recommendations. A category ‘could not assess’ was also available. Besides, the panel was asked to give their top ranking: the best recommendation by dimension (diagnosis, medical treatment, non-medical treatment and referral) and the three overall most relevant recommendations for the quality of care (top-three ranking). Remarks about the recommendations were also allowed.

Step 3. Processing the results

Subsequently, the results were analyzed using a standardized consensus tool, and the recommendations were rated as valid if they matched three of our criteria. The first two criteria include that the recommendation was rated with a median score of eight or nine and that there was agreement among the ratings of the independent panel members. Agreement was defined as the case in which 70% or more of the scores was in the top tertile (scores 7, 8 or 9) of the scale and the other 30% or less of the scores was divided over the remaining two tertiles. These criteria were deduced from the two of Campbell’s criteria [3].

Earlier research on indicator development showed that using only the criteria mentioned above, often does not provide enough discrimination [3, 4]. Therefore, a third criterion was added: the recommendation should be in the top ranking for 20% of the scores [16, 23]. Points were awarded according to the panels’ best recommendation by dimension and the top-three ranking. A recommendation that was first mentioned in the top three was given four points, the second one was given three points and the third one two points. Recommendations that were selected as best recommendation in one of the four dimensions (without being mentioned in the top three) were given one point. These points were converted into percentages based on the number of experts that scored that recommendation and the related maximum score.

An access-based consensus tool combined the three criteria as described above and converted them into three categories: ‘selected’, ‘equivocal’ or ‘non-selected’. In this way, recommendations that met all three criteria were classified as ‘selected’, those who met at least one and a half criteria as ‘equivocal’ and the remaining recommendations as ‘non-selected’ (Table 1). The consensus tool listings were the input for the next round, a consensus meeting.

Table 1 Example of consensus tool methodology to select recommendations and quality indicators, based on a selection of recommendations
Full size table

Step 4. Consensus meeting of the expert panel

All members of the expert panel were invited to the consensus meeting to discuss the results of step 3. All recommendations were discussed with special attention to the ‘equivocal’ ones. Apart from the relevance to the quality of care for children with functional constipation, the recommendations were also assessed on their measurability and their ‘room for improvement’. This consensus meeting, chaired by two of the authors (MO and JS), resulted in a set of consensus-based recommendations that best reflects the quality of care for children with functional constipation according to the expert panel.

Step 5.Translation of the recommendations into indicators and final comments of the expert panel

The chosen recommendations were translated into quality indicators, which means they were described as numerators and denominators (in the case of process and outcome indicators) or questions that could be answered with ‘yes’ or ‘no’ (in the case of structure indicators). The numerator represents the proportion of the patient population that applied the criteria as stated in the indicator. The term denominator represents the patient population to which the criteria (as stated in the indicator) should be applied. The quality indicators were e-mailed to all members of the expert panel for final comments and approval.

Step 6. Practice test to assess the measurability of the indicators

The indicators were tested on their measurability. Four professionals participated in the practice test: a general practitioner, a pediatric gastroenterologist and two primary health care doctors. All four professionals were asked to assess the measurability of the set of indicators based on data from ten randomly selected children with functional constipation. If data needed for an indicator could be collected by searching medical records or a patient survey, this was referred to as ‘measurable’. The results of the practice test were evaluated with the four professionals individually by means of a semi-structured interview. Based on these results, the definitive set of quality indicators was determined.

Results

The 84 recommendations extracted from the guideline ‘Functional constipation in children between 0 and 18 years’ were scored by 16 of the 21 experts involved in the development of the guideline (five experts did not return the questionnaire). Twenty-seven of the recommendations met at least one criterion as described in step 3: 24 were classified as ‘equivocal’ and three were classified as ‘selected’ (Results are shown in the online appendix).

Eleven of the 21 experts were present at the consensus meeting. In advance, all experts were given the opportunity to comment on the results per e-mail. All 84 recommendations were discussed during the consensus meeting according to the consensus tool results. Eleven recommendations were selected for inclusion in the core set of the recommendations: one on diagnosis, eight on medical treatment, two on non-medical treatment and none on referral. Due to the overlap between some of the recommendations, these were combined into a definitive set of seven quality indicators (Table 2). The set includes one structure indicator on patient information and six process indicators. These process indicators dealt with the topics ‘concerning diagnosis based on the ROME III criteria’, ‘medical treatment of polyethylene glycol and lactulose’ and ‘patient contact during and after medical treatment’.

Table 2 Selected quality indicators concerning quality of care for functional constipation in children
Full size table

All indicators were tested in the practice test and evaluated by four professionals. All indicators were found to be measurable, although there were some barriers. The Rome III criteria (indicator 1) were not explicitly used in a checklist, but were mostly written down in the medical record during a patient’s visit with complaints. Therefore, data collection was found to be time-consuming. Another issue was the lack of information on the whole set of Rome III criteria. Besides, patient information as described in indicator 2 is mostly not kept at the professional’s office, but available via other resources, such as the internet or patient organizations. These other resources are often not used by patients through lack of awareness. Unfortunately, it was not possible to measure whether professionals provided their patients with this information. In contrast, the indicators concerning medication use (indicators 3, 4 and 5) were well-registered and could be easily found in the medical records. However, general practitioners did not always register the duration of the medication but only the dosage, which made it more time-consuming to collect the data. Contact with the patient 1 to 2 weeks after starting treatment (indicator 6) and 2 months after stopping treatment (indicator 7) was not yet a routine for most professionals, but these indicators were considered measurable if consultation dates had been documented.

None of the quality indicators were excluded after the practice test. The final set of quality indicators for care delivered to children with functional constipation consisted of seven quality indicators (Table 2).

Discussion

In this study, the quality indicators have been developed based on the recommendations of the Dutch evidence-based multidisciplinary guideline, ‘Functional constipation in children between 0 and 18 years’. Assessing the quality of care using these quality indicators is essential for improving the quality of health care in children with functional constipation.

Clinical indicators must be developed and tested with scientific rigor in a transparent process. Although some good examples exist of sets of indicators that have been developed for the diagnosis, treatment and follow-up of patients with a specific disease or type of care [15, 16, 23, 27], these do not exist for children with functional constipation. Frequently used methods to assess the value of potential quality indicators are the Delphi technique and RAND appropriateness method [2]. In our study, we used the RAND-modified Delphi technique, with a nine-point Likert scale. Previous research showed that the method of rating recommendations on a Likert scale is reliable for the selection of indicators [18, 33]. The reliability of a consensus procedure is only moderate [34], but the reproducibility can be improved by choosing a high cut-off value and a top ranking. In the present study, the cut-off value of the potential quality indicators was a median score of eight or higher on a nine-point scale and the recommendation should be in the top ranking for >20% of the scores [16, 23].

A multidisciplinary team of experts is required for the selection of quality indicators, to make sure that all aspects of the quality of care are discussed. All disciplines involved in health care for children with constipation were represented by one or more professionals in our expert team. Five experts did not fill out the questionnaire and ten did not attend the consensus meeting. This is not likely to disturb our results, because all experts were given the opportunity to comment on the results before and after the consensus meeting (which was done by a few experts). Furthermore, all specialisms were represented among the 16 experts that returned the questionnaire, and we believe that a response rate of 76% (16/21) for filling out a questionnaire (84 recommendations) is quite high.

Indicators based on guidelines mainly lead to process indicators and structure indicators. We used the guideline as a starting point for the development of clinical indicators for patients with functional constipation. By following this procedure, the final set of indicators includes only structure indicators (e.g. Availability of a brochure on pediatric constipation in the consulting-room) and process indicators (e.g. Percentage of patients with constipation that received lactulose as initial or maintenance treatment). This phenomenon has also been identified in similar selection procedures [9, 23, 24]. Optimal care according to the guidelines should lead to better outcomes of care, although outcome indicators themselves are not part of the guidelines. Therefore, outcome indicators do not originate from our development procedure and adding a set of outcome indicators, such as mortality, morbidity, quality of life and patient satisfaction, could be considered.

Although process indicators are better suited for quality improvement than outcome indicators, insurers, policy makers and consumers are usually more interested in outcome measures. However, outcome measures have major disadvantages: they usually have a low incidence or prevalence and therefore need long periods of observation; they are difficult to control because they are also influenced by lifestyle choices of patients, compliance and health status; and they are heavily confounded, for example by disease stage [9, 20, 29]. Process and structure indicators, on the other hand, are easy to measure, do mostly not require case mix adjustment and are therefore considered more valuable for quality improvement programs, when compared with outcome measures.

The development of clinical indicators should be followed by a practice test. During the practice test for the indicators for childhood functional constipation, it became clear that not all information needed to measure the indicators was easily accessible. Collecting the information needed was time-consuming for professionals, which is an often heard criticism. The time spent on data collection could better be spent on patient care. An electronic medical record with standardized templates could minimize this burden. At best, information on the indicators is routinely collected so that instant feedback is possible, which can lead to the continuous improvement activities.

Indicators give insight into determinants and variation in actual care. This is needed to target the improvement strategy [2, 6, 26, 31, 32]. Evidence suggests that educational outreach visits, education meetings, workshops and audit- and feedback-based quality indicators can be effective in changing health care practice [10, 17, 25]. Grol et al. showed that studies using feedback reports combined with other implementation strategies, such as the use of education and quality improvement plans, were most effective in improving quality of care [11, 13, 14]. Therefore, we suggest the development of an implementation strategy, including at least audit and feedback, in order to implement the guideline ‘Functional constipation in children between 0 and 18 years’ and its quality indicators.

In conclusion, based on evidence from the literature, expert opinions and a systematic methodology, we have developed a set of process and structure indicators to monitor the quality of health care for children with functional constipation. This set of quality indicators could be helpful during the implementation of the evidence-based multidisciplinary guideline ‘Functional constipation in children between 0 and 18 years’. This development procedure could serve as an example for others in their efforts to implement guidelines into practice.