1 Introduction

According to the United Nations Convention of the Right of the Child (CRC), all children have the right to participate in procedures that affect them, including children involved in the child protection system (UN Committee on the Rights of the Child 1989). According to the General Comment 12 of the CRC, meaningful participation of children should consist of four key elements (UN Committee on the Rights of the Child 2009). Initially, children should be provided with all necessary information to develop an informed view. Secondly, children should be provided with opportunities that enable them to express their views. Further, children’s voices should be given a true weight in decision-making processes. Lastly, children should receive feedback on how their views are incorporated in the final decision. Providing children with feedback ensures that children’s participation is not just a formality but aims to ensure their views are actually taken into account in decision-making processes. The CRC emphasizes that all children, irrespective of their developmental age, have the rights to participate in procedures that affect them. However, it is important to execute participatory practices in such a way that it fits the developmental level of the child. This implies child protection professionals should communicate with children such a way that fits their developmental level (UN Committee on the Rights of the Child 2009).

Previous research has revealed that children’s participation in child protection practice does not always correspond with the CRC and policy guidelines. For example, children are not always adequately informed or they are only asked for their opinions when decisions have already been made, which does not enable them to truly engage and influence decision-making processes (van Bijleveld et al. 2014). This lack of correspondence needs our attention, since children’s participation has been linked to positive outcomes (Kriz and Roundtree-Swain 2017). For instance, children’s participation leads to better care decisions and well-tailored interventions (Vis et al. 2011); in the reunification process participation seems to contribute to its success and stability (Balsells et al. 2017); participation is generally associated with higher levels of well-being (Lloyd and Emerson 2017); and participation potentially enhances children’s development as it prepares them for their role as active citizens in society (Dedding et al. 2013). In addition, practices of non-participation seem associated with negative outcomes for young people who are in touch with the child protection system. Firstly, public inquiries into child protection services (see for instance Lord Laming, 2003) suggest that the lack of participation of children in decision-making processes might contribute to practice failures. Atwool (2006) examines children’s participation in decision-making from the perspective of New Zealand’s indigenous children, suggesting that the cultural disconnection of children in care may have been avoided had children been consulted and their viewsrespected. Woolfson et al. (2010) report that a lack of participatory practices within the child protection systems provides children with insecure and anxious feelings because they do not know what to expect. Similarly, Leeson (2007) reports that not being involved in decision-making resulted into feelings of being powerless. Furthermore, Cossar et al. (2016) illustrate that engaging in formal child protection conferences may result in feelings of disempowerment because children feel unprepared to take part in these meeting or did not understand what is discussed. Lastly, when children are not involved in decision-making processes, it may lead to distrusting professionals in the child protection system (Bouma 2019). An explanation why child protection practice may not consistently ensure children’s right to participate, is that child protection systems seem mostly focused on protecting children, which is also a key right for young people (Davidson et al. 2017). Peleg (2018) argues how children’s rights focused on the their protection may tension with rights that ensure the autonomy of children, including participation. Paternalistic views on children oftentimes use ‘the best child’s best interest’ principle to limit the autonomy of the child which eradicates chances for children’s participation. This phenomenon is illustrated in several studies where child protection professionals state that children are ‘vulnerable’ and do not want to ‘burden’ them with ‘adult issues’ (e.g., Bouma 2019; Heimer and Palme 2016; Pölkki et al. 2012). These paternalistic views may limit professionals’ inclination to involve young people in decision-making processes in the child protection system.

Considering the link between children’s participation and positive outcomes and the discrepancies between participation policies and practices, we aim to validate the Meaningful Participation Assessment Tool (MPAT) to encourage future studies on the outcomes of children’s participation in child protection practice. The tool aims to capture the degree in which child protection professionals seem to enable children to participate in child protection investigations and is based on the Model of Meaningful Participation (Bouma et al. 2018; see Fig. 1).

Fig. 1
figure 1

Model of Meaningful Participation (retrieved with permission from Bouma 2019)

Full size image

1.1 What is Participation?

The concept of children’s participation has developed over the past decades and was initially shaped by a hierarchal perspective on participation. Early models of children’s participation were usually founded on the participation ladder of Arnstein (1969), which describes how governmental institutions can realize citizens’ participation in several ways. The ladder symbolizes Arnstein’s hierarchical view on participation, as she viewed that some forms were more participatory compared to others and thus were placed higher on the ladder. For instance, she gradually ordered informing, consulting, and delegating power to the people on the different rungs of the ladder, implying that informing is viewed as less participatory. Inspired by Arnstein, several scholars have developed hierarchal models on children’s participation where some forms of participation were viewed as more participatory compared to others (e.g., Hart 1992; Reynaert et al. 2009; Thoburn et al. 1995). For instance, Hart’s (1992) ladder of children’s participation ranks involving children in decision-making processes higher compared to informing children. More recently, hierarchal perspectives on participation have received critique because it was argued that all forms of participation – from informing people to involve them in decision-making processes – are important (e.g., Charles and Haines 2014). This has led to the development of non-hierarchal frameworks of children’s participation (e.g., Hart 2008; Mannion 2003).

1.2 The Model of Meaningful Participation and the Meaningful Participation Assessment Tool (MPAT)

In this study, we investigate the psychometric properties of the MPAT. MPAT is founded on the Model of Meaningful Participation, which was developed based on an extensive literature review on participation and has a non-hierarchal perspective on participation (Bouma et al. 2018; see Fig. 1). This model was originally founded to evaluate policies of child protection. It includes three core dimensions - informing, hearing, and involving - which are described as the core aspects of participation in previous scientific literature and the General Comment of Article 12 on the CRC (Berrick et al. 2015; UN Committee on the Rights of the Child 2009). The informing dimension stresses that children can be informed about several aspects concerning their involvement with child protection agencies: their right to grow up in safety (a) and their right to participate (b). Then, children can be informed about several aspects about the participation possibilities (c), consequences of participation (d), participation process (e), focus and aim of participation (f), and the potential impact of participation (g). In addition, children can be informed about the content of the investigation (h). Lastly, children can receive information about decisions that are made (i) and how their views were given weight in the decision-making process (j). The hearing dimension implies that children can be provided with opportunities to express their views (a); that information can be gathered from children (b); that professionals should be willing to listen to children (c); and should have dialogues with children (d). It is suggested that children could be heard in individual meetings so that they can express themselves more freely (e). The final dimension, involving, states that children’s views about decisions should be heard before decisions are made (a), that they should be involved in such decision-making processes (b), and that the child’s perspective should be considered (c). The dimensions and items of the Model of Meaningful Participation can be translated into different operationalizations of participation, which together form the MPAT (see Table 1 for shortened items; full overview of items is provided by Appendix 1). The MPAT consists of 13 items that represent different ways in how child protection professionals can enable children to participate in child protection investigations. The third and fourth author of this paper (Name and Name) were involved in the development of the Model of Meaningful Participation (Bouma et al. 2018). In collaboration with all the authors of the latter publication and this publication, we developed the items of MPAT.

Table 1 Overview of items of the Model of Meaningful Participation (Bouma et al. 2018) and the Meaningful Participation Assessment Tool
Full size table

1.3 Context: The Dutch Child Protection System

This study is conducted within the landscape of the Dutch child protection system (López López et al. 2019). A main characteristic of the Dutch child protection system is that there are two child protection agencies that investigate suspicions of child maltreatment. Everyone who suspects that a child in the Netherlands is being maltreated can report to the AMHK (in Dutch: Advies- en Meldpunt Huiselijk Geweld en Kindermishandeling; English translation: the Advice and Reporting Centre for Domestic Violence and Child Maltreatment). Subsequently, the AMHK can decide to conduct a child protection investigation or directly refer the child and their family to voluntary support services. If the AMHK assesses that an involuntary support measure is necessary to protect the child’s safety (i.e., a supervision order or an out-of-home placement), the case can be referred to the RvdK (In Dutch: Raad voor de Kinderbescherming; English translation: Child Protection Board), which is the only child protection agency that can recommend these measures. Subsequently, the RvdK can refer cases to the Juvenile Court, who have the authority to legally enforce involuntary child protection measures. Whereas everyone can report to the AMHK, only designated institutions can report to the RvdK, such as the AMHK itself, the municipalities, and certain certified youth care institutions who are performing child protection measures. Another key characteristic of the Dutch child protection system is that it is more fragmented compared to other countries’ systems (López López et al. 2019). In case a Dutch child or a family is referred to a certain type of services or child protection measures, other agencies in the child welfare system will provide these services or execute child protection measures. This implies that Dutch child protection agencies are foremost responsible for investigating and assessing children’s home situations and for making recommendations regarding support or child protection measures, whereas they do not provide support or execute measures.

1.4 Factors that May Influence Children’s Participation

Both institutional policies and characteristics of children may influence the degree in which children are enabled to participate in child protection investigations. Bouma et al. (2018) report that Dutch child protection institutions keep several age limits for children’s participation. For instance, the AMHK states that children who have the minimum age of six should be heard by child protection professionals and that children above 12 years old have to be informed. Although the RvdK does not have age limits to see or hear children (i.e., all children have to be seen or heard), they have certain guidelines about from which age on children can read the investigation report; 12 till 15-year-olds are supposed to read specific parts of the investigation report such as the report of their conversation with the professional, whereas children from 16 years old should be allowed to read the full investigation report. Lastly, Eriksson (2009) describes how gendered views on boys and girls might lead towards increased participation of girls, as girls are generally perceived as more mature and relationship oriented compared to boys.

1.5 Research Questions

In this study, we investigate the psychometric properties of the MPAT. We formulate the following research questions: 1) What is the intercoder reliability of MPAT items?; 2) What is the internal validity of the MPAT?; and 3) What is the external validity of MPAT? The last research question will be answered by comparing MPAT scores for subgroups based on child protection institution, age of the child, and gender of the child. Higher MPAT scores for groups for we expect higher participation rates (RvdK cases, older children, and girls) would suggest that the MPAT is externally valid.

2 Methods

This study is embed within the research project Hestia, an international comparative study on child protection policy and practice in the Netherlands, Germany, and England (for more information see www.projecthestia.com or Middel et al. 2020). The study reported in this paper study is part of the quantitative case file study of Hestia and solely reports on a subsample of data that is obtained at Dutch child protection agencies.

2.1 Sample

For the case file study of the Hestia research project, we aimed to collect data from approximately 400 children who lived in four different administrative child protection regions in the Netherlands. In each of the regions, we randomly draw 100 children to be part of our sample. 80% of these children were reported solely to the AMHK and 20% of these children were reported to the RvdK (10% of these children were initially reported to the AMHK and referred to the RvdK and 10% of the children were reported directly reported RvdK without interference of the AMHK). While the RvdK processed cases at the child level, the AMHK handled cases at the family level, which meant that AMHK cases could focus on multiple children instead of one child. If the latter occurred, we randomly chose one child to take part in the Hestia sample. For the overall Hestia sample, we included children who were not yet born (e.g., due to concerns about the child that started during pregnancy) until children who were 17 years old. We included cases that were reported to agencies between the 1st of March and the 31st of August 2015, a period following the introduction of the Youth Act in January 2015 (López López et al. 2019). We selected cases that were reported to child protection agencies due to suspicions of maltreatment or concerns regarding the child rearing capacities of the parents. We excluded cases that were closed due to within country move of the family which resulted into a transfer of the case to another regional office. The entire Dutch Hestia sample consisted of 398 children. For the current study, we only included children who were at least 4 years old which resulted in a sample size of 292 children.

Table 2 provides an overview of the characteristics of the sample. The average age of children was 9.92 years old (SD = 4.13) and 52.1% of the children were teenagers. A slight majority of the children were male (53.1%). The gender of one child was different than female or male (0.3%). Children were reported to child protection agencies due to suspicions of different types of maltreatment: physical abuse (14.1%), neglect (44.4%), emotional maltreatment (11.5%), witnessing domestic violence (39.5%), and sexual abuse or exploitation (4.1%).

Table 2 Characteristics of the sample
Full size table

2.2 Measurement

2.2.1 Qualitative Participation Descriptions

We read case files at child protection agencies. Our coding scheme contained three main open questions regarding children’s participation: 1) What is documented in the case with regard to informing the child and the subsequent procedures?; 2) What is documented in the case about children’s participation between the referral and investigation?; and 3) What is documented in the case about participation of the child with regard to the decision? The supplementary materials (Online Resource 1) provide a detailed overview of the additional coding instructions and sub-questions that were provided to our coders. The answers to these three open questions resulted into qualitative participation descriptions for all children in the sample.

2.2.2 MPAT

We coded all qualitative participation descriptions using the 13 MPAT items (See Appendix 1; See supplementary materials for entire coding scheme, Online Resource 2). We coded whether the several types of participation were present according to the case file or whether the type of participation was absent in the case file. When participation was absent, we coded whether there was no documentation about this type of participation or whether it was explicitly documented that this type of participation did not happen.

2.2.3 Case and Organization Characteristics

We collected contextual information on the cases such as the child protection organization that handled the case (i.e., AMHK, RvdK), age of the child (i.e., <10 years old, ≥10 years old), and gender.

2.3 Procedure

Six researchers (bachelor and master students and PhD researchers – including the first author) from the Faculty of Behavioral and Social Sciences at the University of Groningen collected data at child protection agencies from May 2016 till January 2017. Previously, they received a training manual with relevant reading materials, familiarized themselves with the coding scheme by coding mock cases, and received on-​the-​job coaching to get familiar with the structure of case files. They wrote qualitative participation descriptions when collecting data at the child protection agencies’ offices. They recorded data in an online coding scheme.

After the research team collected data at child protection agencies, they completed the MPAT for each child in our sample based on the qualitative participation descriptions. The tool was completed by two researchers who were also involved in collecting data at child protection agencies (Name and Name).

The Ethics Committee Pedagogical & Educational Sciences at the University of Groningen approved the research procedure of the research project Hestia in March 2015.

2.4 Analyses

2.4.1 Intercoder Reliability

We assessed the intercoder reliability by Cohen’s Kappa coefficients and proportion agreement ratings (Landis and Koch 1977). We assessed reliability based on the two coders’ judgements whether participation was present or absent in the case file (i.e., we merged scores for no explicit documentation of participation and codes that contained explicit documentation that participation did not take place). We selected all 196 case files that were not directly referred to agencies that provide support or child protection measures, as we expected these files contained more information regarding children’s participation. These 196 participation descriptions were coded by both coders. Subsequently, we computed intercoder reliability coefficients in SPSS (IBM 2017). Potential disagreements in coding of the initial 196 files were verbally discussed to reach consensus about the correct scoring procedures. After we concluded that we had reached at least substantial intercoder reliability for all items (i.e., Cohen’s Kappa < .60), the remaining part of the data (n = 96) were coded by the first author.

2.4.2 Construct Validity

We determined the construct validity of the MPAT by conducting a confirmatory Mokken Scale Analysis (MSA; conducted in MSPWIN 5.0, Molenaar and Sijtsma 2000). We assessed validity by contrasting two coding options, which were “participation present” (1) and “participation absent” (0; there was either no documentation of participation or there was explicit documentation that participation did not happen). MSA is an item response technique that aims to capture underlying latent traits by ordering items from ‘easy’ (often positively scored) to ‘difficult’ (often negatively scored). In the current study, the latent trait is the degree in which child protection professionals seem to enable children to participate in child protection investigations. Difficult items are generally positively scored by individuals with high scores on the latent trait. If difficult items are scored, this generally implies that the easier items on the scale are scored as well. MSA is a technique that is based on the general principles of Item Response Theory (IRT; Molenaar and Sijtsma 2000) and has four key assumptions: 1) unidimensionality, meaning that the items share one underlying latent trait; 2) local independence, implying that items are stochastically independent from each other; 3) monotonicity, meaning that the item response curves do not decrease; and 4) non-intersecting item response functions (IRFs), which implies that item ranking order of the probability that items are achieved is independent on a person’s value of the latent trait. We included all items that were at least coded once to be part of the MSA. Then, we checked whether H-values of individual items reached at least .30. We used the H-coefficient of the scale to assess its strength. Then, we computed the scale’s Rho reliability coefficient to asses reliability.

Subsequently, we investigated the monotonicity and non-intersecting IRFs assumptions of the Mokken model by inspecting criterion (crit) values in MSP. Crit values below 40 indicate no violations of assumptions, while higher values between 40 and 80 suggest a minor violation of the assumptions, whereas crit values above 80 indicate a serious violation. Crit values above 80 were reported. We used the rest score method to assess the non-intersecting IRFs assumption. In addition, we visually inspected IRFs that were obtained by using the Mokken package in R (van der Ark et al. 2018).

Another preferable characteristic of item response models is that IRFs for the latent trait are independent of other characteristics. Therefore, we explored sample independence for different subgroups based on gender, age, and child protection institution by using the test option in MSPWIN 5.0. We investigated item ordering for different subgroups and explored potential violations for the monotonicity and non-intersection assumption for each subgroup specifically by examining crit values.

2.4.3 External Validity

In case of a sample independent scale, the external validity can be assessed by comparing differences in scores of groups and evaluating whether these differences are in line with the researchers’ expectations. We investigated the external validity of MPAT by comparing different subgroups based on age, child protection institution, gender on participation sum scores (i.e., adding the amount of times that participation was explicitly mentioned in the case files), using t-tests in R (R Core Team 2014). We conducted unpooled t-tests when standard deviations of subgroups differed substantially and visualized the results using the ggplot package in RStudio (Wickham et al. 2019). We drew positive conclusions regarding MPAT’s external validity when we found higher MPAT scores for groups for which we expected higher participation scores based on scientific literature (i.e., higher scores for girls compared to boys, higher scores for older children compared to younger children, and higher scores for RvdK cases compared to AMHK cases).

3 Results

3.1 Intercoder Reliability MPAT

Proportion agreement ranged between .89 and 1 for all the items of MPAT. Four items reached close to perfect Cohen’s Kappa intercoder reliability scores (.81–1.00; item 5, 6, 8, 9). For four other items substantial agreement was reached (0.61–0.80; item 4, 10, 11, 14; Landis and Koch 1977). For the other items (item 1, 2, 3, 7, 12), no Cohen’s Kappa coefficients were computed, since there was perfect agreement that all items should be scored with “no participation”, meaning that these particular types of participation were not observed in the case files (See Table 3).

Table 3 Overview of items, including reliability statistics (Cohen’s kappa and proportion agreement; N = 196) and overview of participation scores in the overall dataset (N = 292)
Full size table

3.2 Descriptive Statistics

We coded all qualitative participation descriptions using the MPAT. Table 3 provides an overview of the participation scores for each of the items. As regards to the informing dimension, we were mostly unable to find evidence whether children received information about their right to grow up without violence, their right to participate, the participation process, or how their views were given weight in the decision-making process (item 1, 2, 3, 7). However, some case files revealed that children (11–12%) received information about the content of the report or information about the investigation (item 4, 5). The most common type of information that children received related to the decision that was made (19%; item 6).

The most prevalent way in which child protection professionals enable children to participatory practices is seeing and hearing children. Results show that 39% of the children are seen (item 8) whereas 32% also seem to be heard about aspects that are relevant to the child protection investigation (item 9). According to the case files, 15% of the children had an individual meeting with a child protection professional (item 10).

When focusing on the dimension involving, it turns out that children’s views on decisions are heard and explicitly documented in 12% of the files (item 11). Documentation about children’s presence or a child’s representative during the decision-making meeting was never mentioned (item 12). In a similar vein, only one case file explicitly reports how the child’s view was considered in the decision-making process (item 13).

3.3 Mokken Scale Analysis

The MSA resulted in a scale of 8 items. Item 1 (informing – right to grow up without violence), item 2 (informing – right to participation), item 3 (informing – participation process), item 7 (informing – whether and how perspectives of child were given weight in decision-making process), and item 12 (involving – child or representative present at decision-making meeting) were never scored and were therefore not included in the scale. This suggests that child protection professionals do not seem to enable children to participate in these ways. The scale is presented ordered from the most difficult item to the easiest item in Table 4. The scale has an overall H-coefficient of .70, indicating a strong scale (Molenaar and Sijtsma 2000), and a reliability of Rho = .89. H-coefficients of individual items ranged between .53 (item 10 – individual meeting) and .92 (item 9 – Conversation with child about topics regarding the child protection investigation). The hardest item of the scale was item 13 (M = .00; Views child considered in decision-making meeting) and the easiest item was 8 (M = .39; Seeing the child). There were no indications that the assumptions of monotonicity and non-intersecting IRFs were seriously violated. The IRFs of the individual items are shown by Fig. 2.

Table 4 Results of the Mokken scale analysis of the Meaningful Participation Measurement Tool (MPAT)
Full size table
Fig. 2
figure 2

IRFs for all items of the MPAT ordered from most difficult to most easy

Full size image

The MSA provided a clear order of item difficulty which provides information about how professionals are more or less able to enable different forms of children’s participation during child protection investigations. Our results show that children who do participate in child protection investigations are most often seen (item 8, M = .39, H = .91) or heard about child protection topics (item 9, M = .39, H = .92) by child protection professionals. Then, children most prevalently receive information about the decision that has been made (item 6, M = .19, H = .63). Subsequently, children seem slightly less likely to be heard about their views on (potential) decisions (item 11, M = .15, H = .63) or have an individual meeting (item 10, M = .15, H = .53) with the professional. Even less frequently, children are receiving information about the content of the report (item 4, M = .12, H = .59) and the child protection investigation (item 5, M = .11, H = 66). Lastly, available information in the case file suggests that child protection professionals were less inclined to incorporate the view of children in decision-making processes compared to all other MPAT items on the scale (item 13, M = .00, H = .82).

3.3.1 Sample Independence

We tested the scale for different subgroups based on child protection institution, age, and gender. Scale H-coefficients for all subgroups ranged between H = .53–.73 (see Table 5). Crit values indicated no or minor violations in equal item ordering for subgroups based on institution and gender. However, violations appeared in equal item ordering for subgroups based on age for items 6 (crit = 90) and 10 (crit = 118). It appeared that item 6 (Information about outcomes of decision-making process) was a relatively easy item for teenagers, while it was a relatively hard item for younger children. Even though item 10 (individual meeting) was more reported for teenagers compared to younger children, the item was relatively more easy to score for younger children than teenagers compared to the other items on the scale.

Table 5 Item ordering, H coefficients and mean for subgroups
Full size table

Then, we tested the monotonicity and non-intersection IRFs assumptions for the separate subgroups. No violations were shown with regard to the monotonicity assumption in any of the subgroups. However, several signs of serious violations appeared with regard to the non-intersecting IRFs assumption for cases referred to the RvdK cases (item 5, crit = 108; item 10, crit = 124), teenagers (item 6, crit = 87), and females (item 6, crit = 95).

3.4 External Validity

We investigated differences in MPAT sum scores for subgroups using t-tests (Table 6; Fig. 3). In line with our expectations, RvdK cases (M = 3.66, SD = 2.19) scored higher compared to AMHK cases (M = 0.88, SD = 1.56), t = −9.08, p < .001. Furthermore, teenagers (M = 2.09, SD = 2.37) participated more compared to younger children (M = 0.72, SD = 1.24), t = −6.26, p < .001. Lastly, we found higher participation for girls (M = 1.76, SD = 2.14) compared to boys (M = 1.15, SD = 1.90), t = 2.54, p = 0.11. The fact that we found significantly higher MPAT scores for RvdK cases, teenagers, and girls – as we hypothesized – suggests that the tool may be externally valid.

Table 6 Average MPAT scores for subgroups based in child protection institution, age, and gender, including results t-test to investigate for differences between subgroups
Full size table
Fig. 3
figure 3

MPAT scores for subgroups based on child protection institution, age, and gender

Full size image

4 Discussion

4.1 Reflection on Main Findings

The aim of this study was to investigate the psychometric properties of the Meaningful Participation Assessment Tool (MPAT), in particular its reliability, internal validity, and external validity. The items of the MPAT seem reliable, as we obtained sufficient intercoder reliability values. In addition, the items of MPAT formed a strong Mokken Scale (H = .70). We need to point out here that the assumption of local independence was violated for item 9: if item 11 (Child’s views on decision heard) was coded, this would imply that item 9 was coded too. This may have resulted in a slightly inflated H-value of the scale. No serious violations appeared with regard to the monotony and non-intersection IRFs assumption for the entire sample. However, there was some evidence that the non-intersection assumption of some items was violated for RvdK cases, teenagers, and girls. The ordering of items did not differ substantially for subgroups based on institution and gender, which points to sample independence of the MPAT. However, there were indications that item ordering differed for teenagers and younger children, which suggests that item difficulty may be dependent on the child’s age. Lastly – when comparing subgroups based on institution, age, and gender – the tool suggests to be externally valid because higher participation scores were obtained for cases that were handled at the RvdK, older children, and girls as we expected. However, we should be cautious in drawing conclusions with regard to external validity based on the comparison of subgroups based on age because we cannot assure that the MPAT is sample independent with respect to children’s age.

It stands out that the ordering of participation items in our study seems to contradict the ordering of previous hierarchal frameworks of participation. Frameworks developed by Arnstein (1969) and Hart (1992) rate informing lower on the participation ladder than hearing or consulting people. However, our study suggests that child protection workers are more inclined to hear or consult children whereas it appears to be more difficult to them to provide them with information. This finding can be explained by findings from Bouma (2019) who states that policies concerning hearing children seem more prevalent compared to policies on informing and involving children. In addition, Bouma reports that child protection professionals themselves often operationalize the concept of participation as hearing children as they emphasize that conversations with children provide them with information that is relevant for the investigation. It can be hypothesized that the overemphasis on hearing children to retrieve information from them results into the relative neglect of the other forms of participation. The finding that children do not always receive information is concerning, because receiving information is a prerequisite for meaningful participation (UN Committee on the Rights of the Child 2009). Even though participation is generally suggested to be associated with positive outcomes for children, such as increased levels of confidence and self-esteem (Duncan 2019), it is also suggested that participation without receiving relevant information can be harmful to children. Not receiving information will likely result in a lack of understanding of the child protection procedure, which contributes to feelings of insecurity (Duncan 2019; Woolfson et al. 2010). Furthermore, a lack of information inhibits children to participate in decision-making processes, because they need information to feel confident when actually making decisions (Leeson 2007).

Moreover, the degree in which child protection professionals enable children to participate seems to depend on the age of the child. While it appears that child protection professionals are relatively eager to inform teenagers about decisions and outcomes of the child protection investigation and the corresponding item was rated as the third most easy, this scarcely appears to happen for children below the age of ten, where the corresponding item was rated as the second hardest item.

We validated the MPAT based on a research sample of children aged between 4 and 17 years old. We would like to underline that this does not imply that only children from the age of four should participate in child protection investigations. The Committee on the Rights of the Child argues that children of all ages should participate within child protection and rejects the use of age limits for participation (UN Committee on the Rights of the Child 2009). Very young children who have not yet acquired conversational skills are able to express themselves non-verbally, which implies that child protection professionals can enable young children to participate by observing their behaviors (UN Committee on the Rights of the Child 2009).

4.2 Strengths and Limitations

A strength of our study is that child protection workers were not informed about the study beforehand and that we as researchers did not intervene in child protection investigations. By our non-intrusive method we were able to capture processes as they are, without causing observer bias (Fraenkel et al. 2012). Moreover, our sample can be considered as representative of child protection case files in the Netherlands, as we sampled case files at different agencies and regions.

A limitation of our study may be that we acquired data from child protection case files, which may be incomplete or inaccurate, because the quality and completeness of the files may differ from one professional to another due to factors such as the professional’s orderliness or workload (Cockburn 2000). The MPAT aims to measure the degree in which child protection professionals enable children’s participation in child protection investigations. However, since we have investigated participation using case file data, it could be argued that we de facto developed a tool that measures the degree in which child protection professionals document participation in their case files. We encountered a large extent of missing participation data, which is a common issue when conducting case file research (Witte 2020). In many instances there was no explicit document on whether certain types of participation occurred. We have made the assumption that absence of participation descriptions in case files implied that participation did not happen. We need to note that this is a precarious assumption, because it could be that participation was just not recorded in case files (Witte 2020). It can be hypothesized that child protection workers do not report implicit acts or “common sense” decisions, which might include informing children or involving them in decisions.

Another limitation of our study is that we did not involve children in the development of the MPAT. Charles, Haines, and young co-researchers (Charles and Haines 2014) evaluated several children’s participation frameworks developed by adults and concluded that these models, for instance Hart’s (1992) Ladder of Participation, were adult-centered and that the complicated terminology used created a barrier to the use the frameworks.

Lastly, the results of this study are based on data from the Dutch child protection system which has several unique features compared to other child protection regimes (e.g., López López et al. 2019; Lisa Merkel-Holguin et al. 2019). This might be a threat to the generalizability of our findings to other national child protection systems.

4.3 Recommendations for Future Research

To further assess the validity of MPAT, we suggest that the tool can be completed by children to investigate whether their experiences of participation result in a similar ranking order compared to our study, in which the MPAT was coded by researchers based on case files. However, before this step can be taken, we suggest that first children evaluate the face and content validity of the MPAT, as it may be possible that a children’s version of MPAT might need adjustments to make it suitable for children’s use.

4.4 Implication for Practice

We recommend the use of both the MPAT and the Model of Meaningful Participation (Bouma et al. 2018) to further develop child protection policies and to evaluate practices related to children’s participation. Our study has shown that child protection professionals do not seem able to facilitate participation for all children and that in particular some types of participation, such as informing, are relatively hard to achieve in child protection practice. This suggests that there is some room to improve the degree in which professionals can enable children to participate. Policymakers could analyze which types of participation are reflected in current policies and what types of participation should be implemented to a larger extent in future policies. Professionals may use MPAT to reflect on their participatory practices at the case level.

In line with Balsells et al. (2017), we agree that participation should be an aspect of evaluation of child protection practice because the relationship with clients and professionals is an important indicator of quality within the child welfare system (Munro 2004; van Yperen et al. 2008). Therefore, we suggest that the MPAT may be used for auditing purposes, aiming to analyze processes within child protection agencies (Munro 2004). Auditors could score the items of MPAT to investigate the degree in which children participate for a random sample of cases that have been handled within the child protection agency by accessing information in case files. If the MPAT appears to be suitable for use with children or when the MPAT children’s version is developed, also children could score the MPAT items to investigate how children perceive their participation. However, we also believe that auditing measures should be implemented with caution because auditing also have several limitations (for a critical analysis see Munro 2004). An important limitation could be that auditing may result into perverse effects, implying that professionals may focus too much on meeting targets and scoring high on indicators, while neglecting the impact on the service user (Munro 2004).