Skip to main content
SpringerLink
Log in

Agreement on Child Maltreatment Decisions: A Nonrandomized Study on the Effects of Structured Decision-Making

  • Original Paper
  • Published:
Child & Youth Care Forum Aims and scope Submit manuscript

Abstract

Background

Practitioners investigating cases of suspected child maltreatment often disagree whether a child is subject to or at risk of abuse or neglect in the family and, if so, what to do about such abuse or neglect. Structured decision-making is considered to be a solution to the problem of subjective judgments and decisions.

Objective

This study investigates the effects of ORBA, a method for structured decision-making in Advice and Reporting Centres for Child Abuse and Neglect (ARCCAN), on interrater agreement of judgments and decisions.

Methods

Two groups of ARCCAN practitioners, one trained in using ORBA and one untrained, used a questionnaire to make judgments and decisions on the same case vignettes. Interrater agreement on the judgments was obtained by calculating the percentage of agreement, intra class correlation, and the Kappa coefficient.

Results

Both ORBA trained and untrained practitioners showed little agreement on judgments and decisions, except for the judgment on child maltreatment substantiation, for which trained practitioners showed fair agreement. Agreement among trained and untrained practitioners only differed for some judgments and decisions, and differences were not always in the same direction.

Conclusions

This result indicates no convincing evidence that structured decision-making leads to better agreement on decisions concerning child abuse and neglect. Recommendations for improvements in uniform decision-making and further research are given.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. A child maltreatment definition is established in the Law on Child and Youth Care Child, but offers no exact judgment and decision making rules: child maltreatment is any act or series of acts of physical, psychological or sexual commission or omission by a parent or other caregiver that results in harm, potential for harm, or threat of harm to a child.

References

  • Arad-Davidzon, B., & Benbenishty, R. (2008). The role of workers’ attitudes and parent and child wishes in child protection workers’ assessments and recommendation regarding removal and reunification. Children and Youth Services Review, 30, 107–121.

    Article  Google Scholar 

  • Armitage, E., Boffa, J., & Parker, B. (1999). Victorian risk framework: A guided professional judgment approach to risk assessment in child protection. Victoria: Department of Human Services.

    Google Scholar 

  • Baartman, H. (2009). Het begrip kindermishandeling: Pleidooi voor een herbezinning en voor bezonnen beleid. [The concept child maltreatment: Plea for a reconsideration and for well thought out policy]. Driebergen-Rijsenburg: Augeo Foundation.

    Google Scholar 

  • Baeten, P. (2009). Protocol van handelen Advies- en Meldpunten Kindermishandeling bij vermoedens van kindermishandeling in relaties van afhankelijkheid en onvrijheid. [Protocol of action of the ARCCAN with suspicions of child maltreatment in relations of dependence and lack of freedom]. Utrecht: MOgroep.

  • Baird, C., & Wagner, D. (2000). The relative validity of actuarial- and consensus-based risk assessment systems. Children and Youth Services Review, 22, 839–871.

    Article  Google Scholar 

  • Baumann, D. J., Dalgleish, L., Fluke, J., & Kern, H. (2011). The decision-making ecology. American Humane Association. Retrieved from www.americanhumane.org.

  • Belia, S., Fidler, F., Williams, J., & Cumming, G. (2005). Researchers misunderstand confidence intervals and standard error bars. Psychological Methods, 10, 389–396.

    Article  PubMed  Google Scholar 

  • Benbenishty, R. (1992). An overview of methods to elicit and model expert clinical judgment and decision making. Social Service Review, 66, 598–616.

    Article  Google Scholar 

  • Benbenishty, R., Osmo, R., & Gold, N. (2003). Rationales provided for risk assessments and for recommended interventions in child protection: A comparison between Canadian and Israeli professionals. British Journal of Social Work, 33, 137–155.

    Article  Google Scholar 

  • Berben, E. G. M. J. (2000). Als iedereen hetzelfde was… Indicatiestelling in de jeugdzorg. [If everybody would be the same… Assessment of youth care]. Maastricht: Shaker Publishing B.V.

    Google Scholar 

  • Bolton, A., & Lennings, C. (2010). Clinical opinions of structured risk assessments for forensic child protection: The development of a clinically relevant device. Children and Youth Services Review, 32, 1300–1310.

    Article  Google Scholar 

  • Britner, P. A., & Mossler, D. G. (2002). Professionals’ decision-making about out-of-home placements following instances of child abuse. Child Abuse and Neglect, 26, 317–332.

    Article  PubMed  Google Scholar 

  • Center, Children’s Research. (1999). The improvement of child protective services with structured decision making. The CRC model. Madison: Children’s Research Center.

    Google Scholar 

  • Cumming, G., & Finch, S. (2005). Inference by eye: Confidence intervals and how to read pictures of data. American Psychologist, 60, 170–180.

    Article  PubMed  Google Scholar 

  • Cyr, C., Euser, E. M., Bakermans-Kranenburg, M. J., & Van IJzendoorn, M. H. (2010). Attachment security and disorganization in maltreating and high-risk families: A series of meta-analyses. Development and Psychopathology, 22, 87–108.

    Article  PubMed  Google Scholar 

  • Dalgleish, L. (1997). Risk assessment and decision making in child protection. Brisbane: University of Queensland, Department of Psychology.

    Google Scholar 

  • De Bruyn, E. E. J., Ruijssenaars, A. J. J. M., Pameijer, N. K., & Van Aarle, E. J. M. (2003). De diagnostische cyclus: Een praktijkleer. [The diagnostic cycle: A practice theory]. Leuven/Leusden: Acco.

    Google Scholar 

  • De Kwaadsteniet, L., Bartelink, C., Witteman, C., Ten Berge, I., & Van Yperen, T. (2013). Improved decision making about suspected child maltreatment: Results of structuring the decision process. Children and Youth Services Review, 35, 347–352.

    Article  Google Scholar 

  • De Kwaadsteniet, L., Hagmayer, Y., Krol, N., & Witteman, C. (2010). Causal client models in selecting effective interventions: A cognitive mapping study. Psychological Assessment, 22, 581–592.

    Article  PubMed  Google Scholar 

  • Department of Health. (2000). Framework for the assessment of children in need and their families. http://www.dh.gov.uk.

  • Drury-Hudson, J. (1999). Decision making in child protection: The use of theoretical, empirical and procedural knowledge by novices and experts and implications for fieldwork placement. British Journal of Social Work, 29, 147–169.

    Article  Google Scholar 

  • Dumbrill, G. C. (2005). Child welfare in Ontario: Developing a collaborative intervention model. Toronto: Ontario Association of Children’s Aid Societies.

    Google Scholar 

  • Eells, T. C., Lombart, K. G., Kendjelic, E. M., Turner, L. C., & Lucas, C. P. (2005). The quality of psychotherapy case formulations: A comparison of expert, experienced and novice cognitive-behavioral and psychodynamic therapists. Journal of Consulting and Clinical Psychology, 73, 579–589.

    Article  PubMed  Google Scholar 

  • Fearon, R. P., Bakermans-Kranenburg, M. J., Lapsley, A., & Roisman, G. (2010). The significance of insecure attachment and disorganization in the development of children’s externalizing behaviour: A meta-analytic study. Child Development, 81, 435–456.

    Article  PubMed  Google Scholar 

  • Folger, S. F., & Wright, M. (2013). Altering risk following child maltreatment: Family and friend support as protective factors. Journal of Family Violence, 28, 325–337.

    Article  Google Scholar 

  • Forrester, D., Kershaw, S., Moss, H., & Hughes, L. (2008). Communication skills in child protection: How do social workers talk to parents?. Child & Family Social Work, 13, 41–51.

    Article  Google Scholar 

  • Gambrill, E. (2005). Critical thinking in clinical practice: Improving the quality of judgments and decisions. Hoboken, NJ: Wiley.

    Google Scholar 

  • Gambrill, E., & Shlonsky, A. (2000). Risk assessment in context. Children and Youth Services Review, 22, 813–837.

    Article  Google Scholar 

  • Garb, H. N. (2005). Clinical judgment and decision making. Annual Review of Clinical Psychology, 1, 67–89.

    Article  PubMed  Google Scholar 

  • Gold, N., Benbenishty, R., & Osmo, R. (2001). A comparative study of risk assessments and recommended interventions in Canada and Israel. Child Abuse and Neglect, 25, 607–622.

    Article  PubMed  Google Scholar 

  • Goldstein, H., & Healy, M. J. R. (1995). The graphical presentation of a collection of means. Journal of the Royal Statistical Society, 158, 175–177.

    Article  Google Scholar 

  • Hodgkinson, G. P., Bown, N. J., Maule, A. J., Glaister, K. W., & Pearman, A. D. (1999). Breaking the frame: An analysis of strategic cognition and decision making under uncertainty. Strategic Management Journal, 20, 977–985.

    Article  Google Scholar 

  • Juffer, F. (2010). Beslissingen over kinderen in problematische opvoedingssituaties. Inzichten uit gehechtheidsonderzoek. [Deciding on children in problematic parenting situations]. Den Haag: Raad voor de Rechtspraak.

    Google Scholar 

  • Kaplan, S. J., Pelcovitz, D., & Labruna, V. (1999). Children and adolescent abuse and neglect research: A review of the past 10 years. Part I: Physical and emotional abuse and neglect. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 1214–1222.

    Article  PubMed  Google Scholar 

  • Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.

    Article  PubMed  Google Scholar 

  • Leeb, R. T., Paulozzi, L., Melanson, C., Simon, T., & Arias, I. (2008). Child maltreatment surveillance: Uniform definitions for public health and recommended data elements, version 1.0. Atlanta, GA: Centers for Disease Control and Prevention, National Center for Injury Prevention and Control.

    Google Scholar 

  • Léveille, S., & Chamberland, C. (2010). Toward a general model for child welfare and protection services: A meta-evaluation of international experiences regarding the adoption of the framework for the assessment of children in need and their families (FACNF). Children and Youth Services Review, 32, 929–944.

    Article  Google Scholar 

  • Lindsey, D. (1992). Reliability of the foster care placement decision: A review. Research on Social Work Practice, 2, 65–80.

    Article  Google Scholar 

  • Mezzich, A. C., Mezzich, J. E., & Coffman, G. A. (1985). Reliability of DSM-III vs. DSM-II in child psychology. Journal of the American Academy of Child Psychiatry, 24, 273–280.

    Article  PubMed  Google Scholar 

  • Munro, E. (1996). Avoidable and unavoidable mistakes in child protection work. British Journal of Social Work, 26, 793–808.

    Article  Google Scholar 

  • Munro, E. (1998). Improving social workers’ knowledge base in child protection work. British Journal of Social Work, 28, 89–105.

    Article  Google Scholar 

  • Munro, E. (1999). Common errors of reasoning in child protection work. Child Abuse and Neglect, 23, 745–758.

    Article  PubMed  Google Scholar 

  • Munro, E. (2002). Effective child protection. London: Sage.

    Google Scholar 

  • Munro, E. (2008). Effective child protection. London: Sage.

    Google Scholar 

  • Oosterman, M., Schuengel, C., Slot, N. W., Bullens, R. A. R., & Doreleijers, T. A. H. (2007). Disruptions in foster care: A review and meta-analysis. Children and Youth Services Review, 29, 53–76.

    Article  Google Scholar 

  • Osmo, R., & Benbenishty, R. (2004). Children at risk: Rationales for risk assessments and interventions. Children and Youth Services Review, 26, 1155–1173.

    Article  Google Scholar 

  • Pameijer, N. K., & Van Beukering, J. T. E. (2004). Handelingsgerichte diagnostiek: Een praktijkmodel voor diagnostiek en advisering bij onderwijsleerproblemen. [Functional assessment: A practice model for assessment and advice with learning problems]. Leuven/Voorburg: Acco.

    Google Scholar 

  • Perry, B. D. (2002). Childhood experience and the expression of genetic potential: What childhood neglect tells us about nature and nurture. Brain and Mind, 3, 79–100.

    Article  Google Scholar 

  • Perry, B. D. (2009). Examining child maltreatment through a neurodevelopmental lens: Clinical applications of the neurosequential model of therapeutics. Journal of Loss and Trauma, 14, 240–255.

    Article  Google Scholar 

  • Pijnenburg, H. M. (1996). Psychodiagnostic decision-making within clinical conferences. Exploring a domain. Dissertation, NICI, Nijmegen.

  • Prins, D. C. M. (2011). Een onderzoek naar de ORBA-werkwijze: Onderzoek, Risicotaxatie en Besluitvorming van de Advies- en Meldpunten Kindermishandeling. [A study about the ORBA method: Investigation, Risk Assessment and Decision-making of the ARCCANs]. Master thesis, University of Utrecht, Utrecht.

  • Regehr, C., Bogo, M., Shlonsky, A., & LeBlanc, V. (2010). Confidence and professional judgment in assessing children’s risk of abuse. Research on Social Work Practice, 20, 621–628.

    Article  Google Scholar 

  • Schuerman, J., Rossi, P. H., & Budde, S. (1999). Decisions on placement and family preservation: Agreement and targeting. Evaluation Review, 23, 599–618.

    Article  PubMed  Google Scholar 

  • Shlonsky, A., & Wagner, D. (2005). The next step: Integrating actuarial risk assessment and clinical judgment into an evidence-based practice framework in CPS case management. Children and Youth Services Review, 27, 409–427.

    Article  Google Scholar 

  • Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 68, 420–428.

    Article  Google Scholar 

  • Spitzer, R. L., Endicott, J., & Robins, E. (1978). Research diagnostic criteria. Archives of General Psychiatry, 35, 773–782.

    Article  PubMed  Google Scholar 

  • Spitznagel, E. L., & Helzer, J. E. (1985). A proposed solution to the base rate problem in the kappa statistic. Archives of General Psychiatry, 42, 725–728.

    Article  PubMed  Google Scholar 

  • Stals, K. (2012). De cirkel is compleet: Onderzoek naar succesvolle implementatie van interventies in de jeugdzorg. [The circle is complete: Research on successful implementation of interventions in youth care]. Dissertation, University of Utrecht, Utrecht.

  • Ten Berge, I. J. (1998). Besluitvorming in de kinderbescherming. De ontwikkeling en evaluatie van een checklist voor de beoordeling van meldingen bij de raad voor de kinderbescherming. [Decision-making in child protective services. The development and evaluation of a checklist for decision-making at Child Protective Services intake]. Dissertation, Eburon, Delft.

  • Ten Berge, I. J., & Bartelink, C. (2014). Zicht op veiligheid: Veiligheids- en risicotaxatie in de zorg voor jeugd [View of safety: Safety and risk assessment in child and youth care]. In P. Goudena, R. de Groot, & J. Janssens (Eds.), Orthopedagogiek: State of the art [Orthopedagogy: State of the art] (pp. 171–186). Antwerpen/Apeldoorn: Garant.

    Google Scholar 

  • Ten Berge, I. J., Bartelink, C., & De Kwaadsteniet, L. (2011). Beter beslissen over kindermishandeling: Onderzoek naar de effecten van gestructureerde oordeelsvorming middels ORBA. [Better deciding on child maltreatment: Research of the effects of structured decision-making by means of ORBA]. Utrecht: Nederlands Jeugdinstituut.

    Google Scholar 

  • Ten Berge, I. J., & Vinke, A. (2006a). Beslissen over vermoedens van kindermishandeling: Eindrapport project ORBA. [Deciding on suspicions of child maltreatment: Final report on ORBA project]. Utrecht: NIZW Jeugd.

    Google Scholar 

  • Ten Berge, I. J., & Vinke, A. (2006b). Beslissen over vermoedens van kindermishandeling: Handreiking en hulpmiddelen voor het Advies- en Meldpunt Kindermishandeling. [Deciding on suspicions of child maltreatment: Practice manual and tools for the Advice and Reporting Centres of Child Abuse and Neglect]. Utrecht/Woerden: NIZW Jeugd/Adviesbureau Van Montfoort.

  • Turnell, A., & Edwards, S. (1999). Signs of safety: A solution and safety oriented approach to child protection casework. New York: Norton.

    Google Scholar 

  • Van der Horst, F. C. P., & Van der Veer, R. (2009). Changing attitudes towards the care of children in hospital: A new assessment of the influence of the work of Bowlby and Robertson in the UK, 1940–1970. Attachment & Human Development, 11, 119–142.

    Article  Google Scholar 

  • Van IJzendoorn, M. H. (2008). Opvoeding over de grens: Gehechtheid, trauma en veerkracht. Amsterdam: Boom Academic.

    Google Scholar 

  • Van Montfoort, A. J. (2004). Beslissen over interventies in gezinnen. Reader bij de basiscursus voor nieuwe medewerkers Advies- en meldpunten kindermishandeling. [Deciding on interventions with families. Reader for the elementary course for new practitioners of the Advice and Reporting Centres of Child Abuse and Neglect]. Woerden: Adviesbureau van Montfoort.

  • Van Yperen, T. A. (1990). Multi-axiale classificatie van specifieke ontwikkelingsstoornissen: Een studie over as II van het MAC. [Multi-axial classification of specific developmental disorders: A study on axis II of the MAS]. Dissertation, Rijksuniversiteit Leiden, Leiden.

  • Wagner, D., Johnson, K., & Caskey, R. (2001). Evaluation of Michigan’s foster care structured decision making case management system. Madison: Children’s Research Center.

    Google Scholar 

Download references

Acknowledgments

This study was funded by The Netherlands Organisation for Health Research and Development (NOHRD). The NOHRD examined the social and scientific relevance and feasibility of the study. It played no part in the study and the report.

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical standard

The manuscript does not contain clinical studies or patient data.

Author information

Authors and Affiliations

  1. Netherlands Youth Institute, Catharijnesingel 47, Postbox 19221, 3501 DE, Utrecht, The Netherlands

    C. Bartelink, T. A. van Yperen & I. J. ten Berge

  2. Department of Pedagogy and Educational Sciences, University of Groningen, Grote Rozenstraat 38, 9712 TJ, Groningen, The Netherlands

    T. A. van Yperen

  3. Faculty of Social Sciences, Behavioural Science Institute, Radboud University Nijmegen, Montessorilaan 3, 6525 HR, Nijmegen, The Netherlands

    L. de Kwaadsteniet & C. L. M. Witteman

Authors
  1. C. Bartelink
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. T. A. van Yperen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. I. J. ten Berge
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. L. de Kwaadsteniet
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. C. L. M. Witteman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to C. Bartelink.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bartelink, C., van Yperen, T.A., ten Berge, I.J. et al. Agreement on Child Maltreatment Decisions: A Nonrandomized Study on the Effects of Structured Decision-Making. Child Youth Care Forum 43, 639–654 (2014). https://doi.org/10.1007/s10566-014-9259-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10566-014-9259-9

Keywords

Search

Navigation