The Development and Validation of a Child Safeguarding in Sport Self-assessment Tool for the Council of Europe

Abuse and interpersonal violence threaten participant safety and trust in sport. Many political and project activities have sought to promote safeguarding policies within national sporting structures. Despite this, implementation of safe sport policy measures has been lacklustre, and policy guidance is often disparate and sometimes contradictory. Against this background, the Council of Europe initiated the development of a safeguarding in sport self-assessment tool to assist national sport authorities in this crucial area. This tool addresses some of the gaps within safe sport policy guidance by summarising current good practices and offering policy guidance and legitimation. In the following Innovations article, we present our work developing the safeguarding self-assessment tool for national sport organisations in collaboration with the Council of Europe.

1 3 et al., 2015). Complementing the above, several studies (e.g. Bisgaard & Støckel, 2019;Bjørnseth & Szabo, 2018;Cense & Brackenridge, 2001;Hartill, 2017;Parent & Demers, 2011) have further detailed risk and protective factors, as well as the meticulous ways in which grooming may unfold for children and young people within sport. Amongst others, significant power imbalances between victim and perpetrator, as well as lack of awareness concerning what constitutes sexual abuse, are important risk factors at the individual level (Bjørnseth & Szabo, 2018;Cense & Brackenridge, 2001;Vertommen et al., 2016). At the organisational level, negative views of protection, lack of resources, and lack of pre-employment screening are also noteworthy risk factors (Parent & Demers, 2011).
Against this background, European and international resolutions have urged member states, sport federations, and non-governmental organisations (NGOs) to provide clear safeguarding policies and develop procedures to keep sportspeople, especially children and young people, safe in sport. In particular, the Council of Europe (CoE) has taken an active role in shaping child safeguarding policy, both in sport and beyond. Since 2008, the sport policy activities of the CoE have been part of the Enlarged Partial Agreement on Sport (EPAS). EPAS is a platform for sportrelated intergovernmental cooperation and currently includes 41 member states as well as numerous European sport organisations (Council of Europe, 2020a). However, the CoE's sport activities predate the founding of EPAS. In May 2000, European ministers responsible for sport assembled in Bratislava and adopted the Resolution on the prevention of sexual harassment and abuse of women, young people and children in sport. Through this, member states were called upon to "prepare a national policy which would make a clear statement about the absolute need to safeguard and promote the welfare of children, young people and women in sport" and "agree that the implementation of the policy within the context of an overall framework of support and protection for children, young people and women in sport" (Council of Europe, 2000). Similarly, the European Sports Charter calls upon signatories to implement measures to safeguard sportspeople from exploitation and abuse (Council of Europe, 2001. More recently, EPAS has become increasingly active through several education and advocacy projects, including Start to Talk (STT), Pro Safe Sport (PSS), Pro Safe Sport + (PSS +), and Child Safeguarding in Sport (CSiS) (Council of Europe, 2020b, 2020c. A resulting question from these political and project activities has been how to best design and implement safeguarding policies within national sporting structures. Over the past decade, numerous organisations (e.g. International Olympic Committee, 2018; International Safeguarding Children in Sport Working Group, 2016) and researchers (e.g. Gurgis & Kerr, 2021;Mountjoy et al., 2015) have begun addressing the question. Though this body of work proposes various common policy measures and tools, complexities and contradictions remain. This disparate and sometimes conflicting guidance may thus limit policy development and implementation. Indeed, the adequate implementation of safe sport policy measures has been lacking in many European nations (e.g. Kerr et al., 2014;Parent & Hlimi, 2013;Rulofs et al., 2019). In particular, European sport organisations have been slow or reluctant to implement safeguarding principles (Hartill, 2017;Lang, 2014;Mergaert et al., 2016).

The Development and Validation of a Child Safeguarding in Sport…
In line with its project activities, EPAS initiated the development of a safeguarding in sport self-assessment tool to assist national authorities, such as sport ministries, sport confederations, Olympic committees, or other sport federations, to assess their organisational capacity related to safe sport and identify areas for improvement. This tool addresses some of the gaps within safe sport policy guidance by summarising current good practices, providing policy guidance, and supporting critical self-evaluation.
In the following Innovations article, we present our work developing the safeguarding self-assessment tool for national sport organisations in collaboration with EPAS. This tool considers a variety of perspectives as it merges key themes and recommendations from a range of self-assessment tools and policy documents and is further validated through expert interviews. Moving forward, we will first present the step-by-step methodology used to develop the tool. Then, an overview of the self-assessment tool will be presented. Finally, we will look ahead and contemplate some future directions in the development of safe sport policy.

Document Search
A search of policy documents and self-assessment tools related to safeguarding in sport and in general was conducted. To source these materials, the websites of Founder Members and Pioneer Organisations from the International Safeguards for Children in Sport were searched for relevant, English-language policy/self-assessment materials on safeguarding in sport (see International Safeguarding Children in Sport Working Group, 2016). No geographical restrictions were imposed, and this ensured that we included a comprehensive range of important national and international stakeholders. Other relevant sport and non-sport resources were also searched, including the GIZ Sport for Development Toolkit, the Olympic World Library, the Council of Europe, and the National Society for the Prevention of Cruelty to Children (NSPCC). Documents were included for analysis if they presented policy recommendations for sport organisations related to safeguarding or featured some form of policy self-assessment tool.
Based on this process, 11 documents were included, leading to a list of 223 selfassessment items or recommendations. A complete list of included documents is presented in Table 1.

Cataloguing and Questionnaire Development
Once the materials were collected, all main recommendations and self-assessment items were catalogued and deductively organised into the nine pillars identified within the CoE's recent policy consultation activities. Currently, the CoE is engaging in awareness and consultation activities with several European countries and is collaboratively developing tailored policy roadmaps for individual countries (Council of Europe, 2020b). Within these roadmaps, nine core pillars for policy action are identified, and we will detail these later. For now, suffice it to say that these pillars generally align with those present in the International Safeguards for Children and have been developed as part of the Child Safeguarding in Sport project together with the European Union (Council of Europe, 2020b; International Safeguarding Children in Sport Working Group, 2016).
Following this cataloguing, common themes were identified and brought together to form the first draft of the self-assessment tool. Two authors coded each recommendation/item into one of the nine pillars, and the authorship group as a whole then reviewed this coding. Once the external items were organised within the pillars, we merged similar items to formulate a summarised list of items. We also edited descriptions of the nine pillars as needed to reflect the breadth of items identified through the cataloguing.

Validation Interviews
Following the initial questionnaire development, we led one-on-one expert interviews to pilot and refine the tool. This allowed for the collection of feedback from a broad range of experts while also generating acceptance and buy-in for the proposed tool. In the end, we interviewed seven experts from a variety of related fields, including higher education (n = 2), non-governmental organisations (n = 4), and government (n = 1). The first and second authors each conducted two interviews, while the third author conducted three.
For this discussion, a critical realist interview process was deployed (Pawson, 1996;Verkooijen et al., 2020). This meant that the item under investigation (in The Development and Validation of a Child Safeguarding in Sport… this case, the self-assessment questionnaire) was "open for inspection in a way that allows the respondent to make an informed and critical account of them" (Pawson, 1996). Thus, the self-assessment questionnaire was presented to the experts, and they were invited to critically evaluate the pillars and items of the tool. In particular, the discussion sought feedback on (1) the completeness of the tool, (2) the wording and clarity of the different self-assessment questions, and (3) any potential additions or modifications. Notes were taken during and after the interviews, and interviewees also provided written feedback to the authors. Following the interviews, the authorship group met to discuss and summarise the findings, as well as establish key areas for improvement. In particular, the feedback received focused on ensuring that protection measures did not solely focus on athletes but on all sportspeople and that safeguarding was embedded in all facets of an organisation. Based on this, the self-assessment tool was reviewed and refined, leading to the final version of the self-assessment tool presented below.

Self-assessment Tool
In the end, 85 self-assessment items were retained across the nine pillars. The items themselves take the form of binary statements that can be answered with yes, partially, or no. This simple response scheme was chosen as it is easy to apply and can help provide a reasonably reliable measurement of safeguarding policy through a set of predefined criteria. Furthermore, in many cases, the indicators are progressive and build on previous indicators. In Table 2, we provide a description of these nine pillars and some sample indicators.
As alluded to above, this tool is meant to be used by national sport organisations. These sport organisations can include national sport authorities (e.g. national Olympic committees, ministries, national sport confederations) or national sport federations (i.e. federations responsible for a single sport or a group of sports in their country). No scoring scheme has been developed for this. Though we recognise that, in certain contexts, some pillars may be more important than others, this tool should encourage organisational learning and support a culture of safeguarding. Thus, we advocate that each pillar be equally weighted and that organisations completing the self-assessment receive a "traffic light" rating for each area. More detailed scores are not necessarily needed as we do not want to create competition or scorekeeping between organisations but rather promote learning, reflection, and improvement. In that spirit, it is also recommended that organisations complete the self-assessment every year.

Future Directions
This tool is meant to apply across the Council of Europe's member states and to a wide variety of sport organisations. However, we recognise that not all items will apply across all states or organisations to the same extent. Furthermore, there will Arrangements are in place for all sportspeople, especially children and young people, to be provided with advice and support, including information on where to go for help if they are experiencing abuse.
Internal and public resources are also in place for all sportspeople and people responsible for safeguarding in sport when seeking advice and support. This can include nominating designated support officers, providing internal information resources, developing public online materials, and more My organisation has designated at least two people (one male, one female) who act as designated safeguarding officers responsible for safeguarding issues My organisation provides public online resources regarding safeguarding and protection, including guidelines, policies, literature, and other guidance 1 3 The Development and Validation of a Child Safeguarding in Sport…

Education and training framework
Everyone with responsibilities for other sportspeople, especially for children and young people, has access to information, education, and training to help them fulfil their roles in safeguarding in and through sport. These include staff, volunteers, officials, administrators, and coaches. In addition, those with specialist roles, such as designated safeguarding officers, should receive more specific, higher-level training. Education activities can include mandatory training, workshops, further education programmes, and more. This education should ultimately help the organisation promote a values-based safeguarding framework My organisation provides safeguarding education that covers a wide range of topics related to safeguarding, including healthy eating, bullying, burnout, injuries, sexual abuse, harassment, and doping My organisation provides training to all sportspeople, including children and young people, on what is appropriate and inappropriate behaviour towards each other 6. Minimum operating standards (e.g. relating to organisational or operational processes) There is a set of minimum operating standards for safeguarding and protecting all sportspeople, especially children and young people, in and through sport. These should be mandatory and comply with national legislation. Meeting, maintaining, and embedding them should be a condition for receiving funding from the national authority or sport federation. These standards should build on existing globally tested standards and be developed in partnership with sports bodies My organisation requires all associated coaches, officials, staff, athletes, and volunteers to sign and adhere to our code of conduct My organisation restricts funding to member organisations/clubs that adhere and implement our code of conduct Regular monitoring and evaluation systems should be implemented to ensure that action plans are in place and are meeting agreed outputs and outcomes, including in relation to the number of reports/cases/ incidents, adherence to minimum standards, and adherence/implementation of guidelines. Data from the M&E system is shared with researchers and communicated to the public and ultimately helps support organisational learning and improvement My organisation has systems in place to regularly monitor and evaluate the effectiveness of our policy, code of conduct, systems, support, guidelines, partnerships, training, and recruitment My organisation has a person responsible for the monitoring and evaluation of safeguarding guidelines and policies 1 3 The Development and Validation of a Child Safeguarding in Sport… always be a need to ensure that the recommendations embedded in the self-assessment tool are adequately adapted to the legal, social, and cultural reality of a given context. And there is a need for constant, continuous dialogue to support responsiveness and improvements in safeguarding. As recent years have shown, the conditions around the delivery and implementation of sporting activities can change rapidly and dramatically. Finally, we recognise that many items here focus on a harm reduction view of safeguarding, though there are increasing calls for safeguarding to encourage values like inclusion, safety, fairness, accessibility, and human rights (Gurgis & Kerr, 2021). This is not because we do not endorse a more holistic, values-based view of safeguarding but rather because of the inherent complexity of representing this view through dichotomous indicators. Our work attempted to collate and summarise current good practices in the field, which limited our ability to develop more open-ended, value-based indicators. As such, we encourage practitioners and policymakers to work towards developing a more values-based, comprehensive, proactive culture of safeguarding. To do so, future policy recommendations should make a concerted effort to move beyond mere harm reduction and ensure that they actively include the voices of children and youth.
In short, we wholeheartedly encourage sport implementers and policymakers to engage with this tool, reflect on the impact of their chosen policies, and continue developing new ideas and good practices for safeguarding in sport. Sport can be a crucial vehicle for health, wellbeing, and societal development. Safe and inclusive sport is an essential prerequisite to tap into that potential.