Service Provision by Child Protection Services — Exploring Variability at Case and Agency Levels in a Norwegian Sample

In this study, we look at the decision to provide services following a child protection investigation in Norway. More specifically, we want to look at variability at the case level and at the agency level in order to identify factors that may impact a decision to provide services. The study was designed as a cross-sectional archive study that was carried out retrospectively. Included predictors for service provision were presence of abuse, and risk factors related to child, parental and family function. Agency predictors were referral rate and metropolitan location. Variation in service provision was sectioned into case level (N = 883) variation and agency level (N = 16) variation. Five nested multilevel models were estimated. Service provision for families following a child protection investigation was on average 38.4% in the sample. Among the 16 agencies, it varied between 21.9 and 60.0%. This variation is mostly explained by case variables. The strongest were physical abuse (OR = 1.37) and neglect (OR = 1.18). There is evidence for agency differences with respect to the types of cases being referred. Differences in agency thresholds for service provision when controlling for the referral reason seem to be quite small.


Introduction
In a systematic review of factors that determine decision-making in child protection investigations (Lauritzen et al. 2018), the existing research literature was categorized according to four dimensions of the decision-making ecology, i.e. case factors, individual factors, organizational factors and external factors. Although there is theoretical reason to assume that these factors operate together in determining the decision, most studies look only at variables from one of the factor domains. One reason for this may be that there are significant methodological difficulties associated with conducting studies that consider the many different levels that make up the Decision-Making Ecology (DME) (Baumann et al. 2014).
In this study, we look at the decision to provide services following a child protection investigation in Norway. More specifically, we want to look at variability at the case level and at the agency level simultaneously and identify factors that may impact a decision to provide services. In the Norwegian context, this is of interest because criticism has been raised that variability in closure rates are too high among agencies. This has been taken as evidence that the child protection legislation is practiced differently, and that there is need for more uniform standards. The rationale being that equal cases should be treated equally across all parts of the country.

Child protection decision-making within the Norwegian context
In 2021, there were 244 child protection agencies in Norway. Of these did 63 agencies cover several smaller municipalities, 156 covered only the municipality in which the agency was located, and 20 agencies were covering the three major cities. Forty agencies had less than five employees while the largest ones had about 100 employees. Hence, there is a large variance in size/number of professionals across CWPS agencies.
Most child welfare and protection cases start with the Child Welfare and Protection Services (CWPS) receiving a referral that a child is living in difficult conditions. Norway has a mandated reporting system and about 80% of the reports come from a mandated reporter, most often a teacher, a health professional or the police. The rest come from parents, relatives and other non-professionals. The CWPS then reviews and assesses the report of concern, which leads to the report either being screened out or the opening of an investigation. After the investigation, the case can then be closed, or a decision action by CWPS is to provide services. Finally, it must be determined what type of intervention is needed. This means that in Norwegian CWPS work there are three timepoints of decision-making, and the path to service provision is a three-step process. Figure 1 illustrates the assessment process of the accepted referrals of reported concern. In Norway, there is a low threshold for opening an investigation and most cases are thus investigated (80.1%). According to Ellingsen et al. (2015), there are significant variations between municipalities and districts in Norway in terms of the extent to which reports of concern are dismissed or investigated further (screened 1 3 Service Provision by Child Protection Services -Explorin… in). In their study of caseworkers' referrals, they found that the dismissal rate between the local CWPS agencies that participated varied between 12 and 40%, where the national average is approximately 20% (Ellingsen et al. 2015).
If after an investigation the CWPS concludes that there are causes for concern, services may be provided. In Norway, this happens in about 38% of the assessed cases on average. In 2020, the closure rates varied between 0 and 85% among Norwegian CWPS agencies (Statistics Norway 2022). Not much is however known about what may explain this variation in Norway. It is therefore of interest to examine more closely if there are different types of concerns being reported to the different CWPS agencies or if the variability is explained by agency characteristics.
The outcome measure in this study is service provision during or after an investigation. The rationale for using this as an outcome is that we want to study the differences in decision-making among agencies. As illustrated in Fig. 1, the services provided are usually some form of consultation, economical or practical support at home. In a minority of cases (1.8%), the CWPS concludes that the child needs to be removed from home (Statistics Norway 2022). This illustrates an important key feature of the Norwegian CWPS system, it is highly geared towards the provision of voluntary support and prevention for children and families at risk.
The alternative to service provision is closure of the case. Administrative data on reasons for case closures are not very detailed in Norway. However, when we break down the reasons for closures used by Statistics Norway (2022), 49.9% of the investigations were closed because they were not considered by CWPS to meet the statutory requirements of the law. In cases where statutory requirements are met, 9.0% were still closed because the family did not agree to voluntary service provision. In these cases, the requirements for mandatory service provision or removal of the child were presumably not met. Then there was a small proportion of the cases (2.8%) that were closed because the family moved to a different jurisdiction. Hence, 1 3 voluntary service provision by CWPS indicates the confluence of two conditions; (i) the problem being considered serious enough by the CWPS and (ii) the proposed service being accepted by the family. Service provision or closure of the case is the outcome of the CWPS decision-making process. It should be noted though that service provision is not necessarily a good measure of improved outcome for children and families. There may be many reasons for this, for example may services offered be inappropriate or ineffective. Rustad et al. (2022) found that in about a quarter of the cases reported to CWPS in Norway (26.6%), the family had previously received services from the CWPS. We cannot, however, safely conclude from this that the service was ineffective. There are two main reasons for this; (i) as time went by, the problem may have become more serious, and (ii) the new referral may include a different problem. In the international research literature, the main factors associated with re-referrals are parental difficulties, in particular drug abuse (Forrester 2007;DePanfilis & Zuravin 1999). This is an indication that for certain types of problems, CWPS services in and on itself will not always be sufficient in order to achieve longterm improvements. In general, there is scarcity of well-designed effectiveness studies regarding the outcomes of voluntary support measures provided by child protection services.

Theory of decision-making
Many factors have an impact on the service provision decision-making process. Decision-making processes in child welfare and protection are complex. The complexity is evident by the multifaceted problems and resources that characterize children and families that come into contact with CWPS. Baumann et al. (2014) developed a theoretical framework where knowledge derived from decision-making theory is applied specifically in a CWP context. This is called the Decision-Making Ecology (DME) (Fig. 2). When the CWPS makes decisions, the outcome of the decision depends on case characteristics related to the child's situation, the parents' ability to care for the child and the resources in the local community. In addition, another factor of importance is how the information is interpreted and understood by the individual caseworker. Decisions related to child welfare service provision are made in an organization where the processes are regulated by the law and governed by available resources. Thus, both organizational and other external factors have an impact on the decisions that are made.

Literature review
Out of the 34 studies reviewed by Lauritzen et al. (2018), 14 had looked at case factors, 11 had examined variables associated with the social worker, and 9 had addressed organizational factors. The review concluded that case factors that predict service provision vary across studies and that organizational factors seem more important than individual worker factors. For example, Bywaters (2015) looked at how external factors of low income and the prevalence of crime and social problems in a community is associated with inequities in services provision. Studies that have taken a multilevel approach to the study of service provision are mostly based upon use of administrative data. Jud et al. (2012) applied the DME framework in their classification of study variables. They found that in Canada (n = 15,980) service provision was predicted by a range of case characteristics, most notably sexual abuse (OR = 3.01) and exposure to interpersonal violence (OR = 2.99). At the agency level regional differences were identified but were not explained by metropolitan location nor proportion of ethnic minority reports. The authors concluded that the variation at the agency level remains less understood than the impact of case characteristics. Using the same DME framework Font and Maguire-Jack (2015, 2019) investigated substantiation and out of home removal in an American sample (N = 5873). They found that the proportion of Hispanic origin families in the county predicted reduced chance of substantiation. They did not identify any significant county level predictors for removal from home. At the case level, child protection service history and parental mental health/substance abuse problems were predictive of removal, but parenting skills, child's special needs or economic hardship or domestic violence were not. Placement decisions and first nations disparities were investigated in another Canadian study (N = 1304) (Fluke et al. 2010). Here, the proportion of indigenous families investigated within the agency, were predictive of out of home placement (OR = 4.1) Metropolitan location, staff vacancies and worker position were not. At the case level, removal was predicted by physical (OR = 1.7) or mental harm (OR = 2.3) and the presence of three or more concerns related to parental functioning (OR = 2.0). A fourth study also based upon the Canadian incidence study (Fallon, et al. 2015) investigated the effect of clinical and organizational characteristics on the decision to place aboriginal children in out of home placements (N = 1710). In that study, the proportion of aboriginal investigations was a significant agency level predictor for removal, but aboriginal identity at the case level was not. Additionally, emotional harm but not physical harm was indicative of placement 1 3 likelihood. The authors concluded that there were clear signs of indigenous overrepresentation, which was explained by agency level factors, but that the causal mechanisms behind this is not fully understood though inequities in resources amongst agencies services indigenous populations was offered as a hypothesized explanation. Across two reports from of the Ontario Incidence Study, Smith et al. 2017; found that organizations that were set up to provide an array of services, as well as those that had service specialist functions, were more likely to provide services.
Another report based on data from the Canadian Incidence Study (Lwin et al. 2018) found that there was large persistent variation in worker level decisions to provide services. Furthermore, higher levels of service provision were partially explained by the presence of employee support programs.
Middel and her colleagues (2020) using the Hestia study child welfare case data from England, Germany, and Netherlands found that both the presence of mother perpetrators and immigrant status intersecting with a perpetrating father increased the likelihood of service provision in England, but not in the other two countries. Portmann et al. (2022) used data from the Optimus study in Switzerland to analyze case (N = 4735) and organizational level (N = 222) predictors for child abuse reporting. The results did show associations between child neglect reporting rates and vacant housing rates, social welfare rates, and single-parent household rates.
Apart from these articles based on (i) the Canadian Incidence Study, (ii) the Ontario Incidence Study and the (iii) US second cohort of the National Survey of Child and Adolescent Well-Being from 2009, (iv) the Hestia study, and (v) the Optimus study, we are not aware of any other studies looking at both agency and case level factors in CWPS decision-making. Little is thus known about how external factors influence service provision outside of North America.

Aims of the study
The main aim of this study was to determine the role of external factors in explaining variability of service provision among Norwegian CWP agencies. The following research questions were explored: 1) How much of the variability in service provision is attributable to case factors (level 1) versus agency related factors (level 2)?
2) What are the main variables that explain variability in service provision within and between agencies?

Design and sample
The study was designed as a cross-sectional case review archive study that was carried out retrospectively. For this study, a random sample of cases was drawn from the archives in 16 child protection agencies comprising the four different regions of Norway. The agencies were (i) six districts from the three major cities in Norway with a population ranging from 190,000 to 680,000, (ii) six regional cities with a population ranging from 20,000 to 80,000 and (iii) four agencies from smaller towns and rural areas with a population between 10,000 and 20,000. A total of 1365 cases were randomly drawn from all referrals that had been registered in the period January 2015 to December 2017. The number of cases from each agency varied between 50 and 150 depending on the size of the agency. The reason for this is that the size of the agencies differed considerably, and we wanted the number of cases drawn from each agency have about the same proportion of the total available sample. For this study, we only included the cases which were subject to a full CWPS investigation and had a concluding report on file. The study sample was thus 883 cases from 16 agencies. The remaining 482 cases had either been screened out at an early stage of the investigation process (n = 242, 17.7%) or were missing a concluding report (n = 240, 17.6%) and could therefore not be included. We tested if there were differences in service provision between included cases (38.4% rate of service provision, N = 883) and cases that were excluded due to missing data (44.6%, N = 240). The difference was not significant (X 2 = 3.02 (1), p = 0.082). We therefore assume that data were missing at random. There were 54% boys among the participants and the mean age was 8.6 years (SD = 5.0). There were higher proportions of families with immigrant background in the major cities (54.5%) compared to the smaller cities and rural areas (30.0%). In a total of 40.8% of the cases, the family had immigrant background. Following the investigation, 61.6% (n = 544) of the cases were closed, 2.2% (n = 19) were concluded with a decision to remove the child out-of-home and the rest (36.2% (n = 320) were concluded with offering voluntary in-home service.

Ethics and procedures
The study protocol was subject to review of research ethics by the Norwegian Council for Patient Confidentiality in Research, and review of data handling procedures by the Norwegian Centre for Research Data. Access to case files were granted to the researchers through a legal decision made by the Norwegian Directorate for Children and Family Affairs, which exempted the participating CWPS agencies from confidentiality. The license to collect and store data was issued by The Norwegian Data Protection Authority.
The researchers were given access to the casefiles and to electronic systems for record-keeping by the CWPS agency. All case files were coded on site at the agency through an electronic web-based data entry form that was developed specifically for this purpose. The data were encrypted and subsequently transported to a secure sandbox zone approved for storage of sensitive client data.

Measures
An online instrument for data collection was developed and tested. After two revisions, the final interrater agreement was 90.8%. In health research, interrater agreement over 80% is generally considered acceptable (McHugh 2012).

Case level data (level 1)
The coding form information collected included (i) child and family characteristics, (ii) whether or not physical abuse was substantiated, (iii) number and types of risk factors that were identified in the investigation report (iii) and the decision to provide services or to close the case.
Information about child and family consisted of child age, child gender and immigrant background. Immigrant background was defined as the child and/or one parent being born outside Norway. The number of prior referrals of the child to the CWPS was recorded.
Substantiated abuse was defined as physical abuse with or without visible damage and was coded as present or absent.
Substantiated risk factors were coded as present or absent and categorized into 4 groups for which a sum score was calculated.
Child Function consisted of these 9 items: non age-adequate development, mental health problems, crime/substance abuse, externalizing behavior problems, functioning in school/kindergarten, emotional problems, social problems with peers, social problems with adults, conflict with adults. Each of the items were coded as present = 1 or absent = 0 and a sum of all present problems was calculated.
Neglect consisted of these four items: deficiencies in parental stimulation and guidance, basic care for the child, parents' emotional availability, parents' protection of the child. Each of the items were coded as present = 1 or absent = 0 and a sum of all present problems was calculated.
Caregiver function consisted of these six items: somatic heath, psychological health, substance abuse, crime, exhaustion, parental conflict. Family risk consisted of these six items: economy, housing, work situation, stressful life events within family, social network, social integration. Each of the items were coded as present = 1 or absent = 0 and a sum of all present problems was calculated.

Agency level data (Level 2)
Agencies were categorized as residing within a metropolitan area (N = 6) or not (N = 10). The main difference between metropolitan area agencies and the others is that within a metropolitan area, there are several agencies within one city. In smaller cities and towns, i.e. the others, there is only one agency serving the whole population.
Referral rates expressed as referrals per 1000 children residing within the municipality were collected from statistics Norway.
Analyses were carried out in Mplus. First, variation in service provision and in the independent predictors were sectioned into level one variation and level two variation by estimating a two-level unconditional model for each of the variables. Intraclass correlations were calculated and recorded. For further analysis of variation in service provision level one predictors were centered. Five nested models were estimated by first adding fixed effects for the control variables, secondly the Service Provision by Child Protection Services -Explorin… fixed effect for physical abuse was added and thirdly the fixed effects for the risk factors were added. Changes in model fit were measured as deviance of the minus two times log likelihood and tested by the chi-square distribution. For the final model, random slopes were added for each of the risk factors. Due to restrictions in the number of free parameters, the risk factors were added to the model one by one and then removed if no significant improvement in model fit was found. In the final model only the random effect for child problems was retained. Model improvement for each of the sequences is shown in Table 1.

Results
Service provision for families following a child protection investigation occurred on average in 38.4% of the included cases. Among the 16 agencies, it varied between 21.9 and 60.0% and those differences between agencies were significant (x 2 = 28.16 (15), p = 0.021). In the metropolitan agencies, services were provided in 33.2% of the cases and in agencies from smaller cities service provision was at 42.5%. The difference was significant (x 2 = 8.046 (1), p = 0.005). The referral rates ranged from 3.0 to 7.0 referrals per thousand children (M = 4.76, SD = 1.14). The mean referral rates were 4.8 in the metropolitan agencies and 4.7 per thousand in the other agencies, the difference was not significant (t = 1.37 (881), p = 0.17). At the case level, service provision was much more likely if child abuse had been substantiated. The sum of other types of substantiated risks were also associated with increased likelihood of service provision, as was younger child age. Family immigrant background and the number of previous referrals were not significantly associated with service provision in bi-variate analysis (Table 2).
By testing an unconditional model for service provision, we found that only about 1.4% of the variation in service provision was attributable to agency level factors (level 2). The variance term for the intercept was not significant (p = 0.21), i.e. there is substantially more variability at the case level than at the agency level. We therefore must conclude that the main reason for the rather large differences between agencies in terms of the proportion of cases that receive services, are attributable to differences in case characteristics in populations being served. We did however find significant interagency variance in all the risk factor scales and in the families' immigrant status (Table 3). Not surprisingly, the proportion of immigrant families were much higher in the metropolitan areas (54.5%) compared to the other parts of the country (30.0%). The difference was significant (x 2 = 54.3(1), p < 0.001).
By adding fixed effects predictors to the model (Table 4), we found that within agency variability was significantly explained by child age, substantiated child abuse, neglect and the number of risk factors related to child functioning, caregiver functioning and family risk factors. Between agency variability was significantly explained by referral rates and location within a metropolitan area.
In the final model, the model fit improved significantly by including a random slope effect for the number of risks related to child functioning (Table 5). This final model explained 66.7% of the between agency variation and 37.3% of the within agency variation. The results show that service provision is more

3
Service Provision by Child Protection Services -Explorin… likely when child abuse is substantiated (OR = 1.37) and that there is a cumulative effect of the other risk factors. When the agency is within a smaller city, there is about 8% higher chance for service provision compared to an agency residing within a metropolitan area, all other factors being equal. When there are more referrals per child per thousand within the geographical area that is covered by the agency, the chance for service provision is decreased. Comparing the agency with lowest referral rate (RR) per thousand child (RR = 3.0) with the agency with the highest referral rate (RR = 7.0) indicates that the chance of service provision in the low RR agency is about 4% higher (OR 95% CI = 1.01-1.10) all other factors being equal.

Discussion
We found that the variability in service provision following a CWPS investigation is mainly attributable to differences in case characteristics and not to agency level factors. We did not find evidence to support a claim that equal cases are treated differently among agencies with respect to service provision. The rather large variation in service provision rates that was observed (from 21.9 to 60.0%) must thus be assumed to be mainly caused by differences in the types of problems that are identified during the investigation of cases. There are surely many agency characteristics unaccounted for in this study. However, the point that we are trying to make here is that the variation does not predominantly lie at the agency level. Almost all the variation is at the case level. Thus, adding more agency level predictors should not be expected to improve the model very much. We need also to point out, that we did study differences that occur in service provision after the case was assessed by social workers. It is therefore still possible that the social workers' assessments as well as their judgement of risk leading up to the decision to substantiate, may differ. Albeit we did not study this directly, there is an indication that this may be the case because there were significant agency level differences in substantiation of child functioning problems, neglect problems, caregiver functioning and family risk. This may be supportive of the hypothesis that what constitutes a serious problem, is not the same in all places and at all times, even though service provision for substantiated cases may be (Wolock et al. 2001;Križ and Skivenes 2013). The largest difference among agencies was in the substantiation of problems related to child functioning. We assume that for many of the problems within this category such as a mental health problems, externalizing problems, emotional problems or social problems, the social workers substantiation decision might be influenced not only by the etiology and clinical significance of the problem itself. But that this may also depend upon the other types of help and services that are available within that community. For example, the CWPS may see a child's mental health problem as more worrisome if nobody else is providing assistance, compared to if the child is receiving adequate help elsewhere. Christiansen et al. (2019) noted that in a small rural community where specialized services are not readily available, the CWPS has been seen to take on cases and provide services for families based not only on the case risk, but that they also take into consideration the other types of services that are available. This hypothesis is further supported by the results from our final model, where the variability at the agency level is partly explained by metropolitan location. In these urban areas, other specialized services for children with health and social problems are known to be more easily available, compared to more rural areas. However, with only 16 agencies in the analysis, we failed to statistically confirm the random slope effect for child functioning problems. To the best of our knowledge, the potential moderating effects of other types of services upon a CWPS decision have never been tested. For physical abuse, there were no agency differences in substantiation rates. We believe this indicates that variability in social workers judgement and the impact of other resources within the community may be more pronounced at the low-risk welfare spectrum of problems than when outright threats to a child's health and safety are more apparent.
Both referral rates and closure rates at the agency level vary significantly and they do seem to be related. The mechanism behind this however is still not well understood. Whether the significance of higher referral rates relative to the child population is indicative of increased pressure upon the services available, or if it means that more low-risk cases are referred, we cannot say for certain. It is also possible that agencies, by their response threshold, over time shape the reporting patterns within a local community. If so, the thresholds for service provision may change over time as referrals and available services, balance against each other. It would therefore be instructive to look more closely at referral and service patterns from a longitudinal perspective to better understand what causes agency differences. The strongest predictors for case level variability in this study were substantiated physical abuse (OR = 1.37). This is similar to the results from Jud et al. (2012) who found a slightly higher effect of child abuse (OR = 1.79) upon the decision to remove a child from the home. However, two of the other DME oriented studies we reviewed found no such significant effect (Fallon et al. 2015;Fluke et al. 2010). It should be noted though that while those other studies only looked at removal decisions, the decisions in our study were predominantly to provide voluntary services, and in only a small proportion of the cases was the child removed from home. The results are therefore not readily comparable. We believe that a removal decision rarely is justified with only one type of substantiated problem, i.e. the abuse would have to be very serious for removal to occur if there were no other problems present. This may help explain why the effect of abuse may be different in decisions about removal than in decisions about provision of home-based services. Removal can be seen as a high-risk decision in which the consequences are serious for both the child and the family, as well as the CWPS agency, if mistakes are made. Voluntary counseling and home-based services on the other hand carries risk mainly if they are insufficient to protect the child from future abuse or neglect. It is therefore worrisome that so little research exists on the long-term preventive effects of voluntary home-based services. Treating and preventing child abuse is considered one of the core duties of the CWPS in Norway. Thus, services may be offered in such cases even if the evidence of abuse is uncertain or not present. Neglect problems, family problems and child health problems on the other hand may also elicit CWPS services but rarely in and on its own. Christiansen et al. (2019) found that service provision was most likely when child problems were combined with reduced parental capacity and other risk factors. Services were not more likely to be provided if there was only a problem related to child functioning, then the family are often referred to other health or social services.
Previous studies have identified the proportion of ethnic minority population to be predictive of agency thresholds (Fluke et al 2010;Middel et al 2020). In this study, we found that immigrant status as a case level predictor was not a significant variable in our models. This means that families with immigrant background were no more or less likely to receive services given that the risks were the same. Still ethnic minorities are overrepresented within the CWPS system in Norway as in most other places. A different study would be needed to disentangle what may be a complex relationship between minority status at the case level and its relationship with minority population as a community level factor.
We believe that service provision is, at least to some degree, driven by the availability of services and the workers belief that the available service will be helpful. Parenting problems are one of the few areas where CWPS has access to good evidence-based interventions such as Parent Management Training (Ogden and Hagen 2008), and The Incredible years program (Fossum et al. 2014) which are both available in many but not all local communities in Norway. It is possible that service provision as a response to parenting problems vary depending on the availability of effective parenting programs.

Implications
The study carries implications for policymakers, social work practice and for future research. While not definitive, our study indicates that policy makers do not need to worry overmuch about difference in rates of service provision among CWPS agencies in Norway. The idea that the proportion of families that are given assistance should be about the same everywhere, as a prerequisite for equal service availability, seems to be built on two flawed assumptions. These are (i) that every agency receives referrals regarding the same types of cases and (ii) that assistance from other health and social services is equally available everywhere. This study draws into question whether assumption number one is correct.
There is large variability in the types and seriousness of cases referred to CWPS. This creates space to explore possible promotion of the focus on policy and practice to further develop differentiated responses to referrals, rather than on standardizing procedures. One aim of differentiating responses to referrals would be to produce somewhat similar decision outcomes across similar cases. In considering needed research it would be useful to focus on the long-term impacts of CWPS decisions in screening of cases and provision of services for children's health and safety and to determine if agency level indicators are determinative of more successful long-term protection of children.

Limitations
Because we only had 16 agencies in the study, we were restricted to estimating 15 parameters in the final model. It is possible, and likely, that more of the variability could be explained if more agencies were in the study and thus allowing more predictors and improving our estimate of agency level variability. It is also possible that larger inter-agency variation may exist between some of the agencies not included in the study. Unfortunately, we were not able to account for variability at the individual caseworker level since we did not know the identity of the social worker. We should however note that this effect would have been immensely difficult to isolate because (i) many social workers are usually involved in one case given the sampling density of case records, and (ii) because supervisors and other management are also involved in decisions about service provision. On another note, we believe that it is possible and very likely that the records of substantiated and unsubstantiated risks are a function of the quality of the investigation. The investigation report may not be a perfect representation of facts of the case. It is however not easy to correct for the quality of the assessment in statistical analysis. It would require a measure based on clearly specified criteria of what constitutes good accuracy, quality practice, and breadth of information collection in a CWPS investigation. On a final note, we acknowledge that it is important to also take into account the prevalence of the respective problems in the population served by the agency, before definitive conclusions can be made about what is appropriate levels of service provision. Unfortunately, we do not have good municipality level prevalence data for child abuse and neglect problems in Norway.

Conclusions
Although rates of service provision differ significantly between agencies, this variability is mostly explained by case level factors. Most notably, physical child abuse. There is however also a cumulative effect of other identified problems related to parenting, caregiver functioning, family risk and child health problems. The reason why some agencies close more cases than others are either because less serious cases are reported or because agencies have different thresholds for recognition of a problem. Once a problem is identified, there is only small agency variation in service provision. Most of that variation was explained by geographical location within a metropolitan area and hence possible difference in the availability and accessibility of other services, and the overall referral rate.