Functioning, as defined by the International Classification of Functioning, Disability and Health (ICF), encompasses all areas of an individual’s life, extending beyond specific areas such as ‘adaptive functioning’ or ‘cognitive functioning’ (World Health Organization (2007)). Instead, it includes a range of areas of activity and participation: (a) learning and applying knowledge, (b) general tasks and demands, (c) communication, (d) mobility, (e) self-care, (f) domestic life, (g) interpersonal interactions and relationships, (h) major life areas, and (i) community, social, and civic life. The ICF also conceptualises functioning as resulting from the interplay between body structures and functions, activities and participation, environmental factors, and personal factors (World Health Organization, 2007). The assessment of functioning is of increasing importance for individuals with disabilities and the systems and services aiming to support them (Madden & Bundy, 2019). In the Australian context, functional capacity assessment is central to accessing funding, with limitations in functional capacity being a key criterion in accessing the National Disability Insurance Scheme (National Disability Insurance Agency (2019)). Children with neurodevelopmental conditions (NDCs) are currently one of the largest populations accessing disability services in Australia (National Disability Insurance Agency, 2021). Additionally, the assessment of functioning is central to the diagnostic process for NDCs, as functional impairment is a diagnostic criteria for neurodevelopmental disorders within the Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 2022; World Health Organization, 2021).

There are many published tools (e.g., surveys, interviews, structured observations) available to clinicians for the assessment of different functioning domains in children; however, these primarily focus on the activity and participation domain of the ICF to the exclusion of body functions and environmental factors (D’Arcy et al., 2021). These tools also have varying levels of psychometric properties and clinical utility in children with NDCs, which may be related to the paucity of tools developed specifically for this population (McConachie et al., 2015; Smart, 2006). While clinicians’ decision making has been explored in some contexts (Kramer et al., 2009), there has been limited exploration as to what clinicians look for when choosing an assessment of functioning tool, particularly for individuals with NDCs. Additionally, a recent national survey of assessment of functioning practices identified that there is wide variability in the purpose, context, and processes used in the assessments of functioning for individuals with NDCs (D’Arcy, et al., 2022). Currently, some of the most common tools used in the Australian context include the Vineland Adaptive Behavior Scales (Sparrow et al., 2016), the Adaptive Behavior Assessment System (Harrison & Oakland, 2015), and the Pediatric Evaluation of Disability Inventory (Haley et al., 2019).

Understanding clinicians’ priorities and considerations when selecting assessment of functioning tools can potentially inform the format and development of new tools, identify gaps between current and best practice, and inform policy, education, and practice changes. Elucidating the decision making of experienced clinicians when selecting these tools can further assist novice clinicians in developing their understanding of the clinical reasoning behind selecting assessment tools. Therefore, this study sought to describe Australian clinicians’ considerations when selecting an assessment of functioning tool for use with children with NDCs.

Methods

Participants

Any medical or allied health clinician involved in assessing the functioning of children with NDCs in Australia was eligible to participate. A total of 45 clinicians participated in the study, 20 via workshops (44%) and 25 via survey (56%). The majority being female (n = 39, 87%) and occupational therapists (n = 18, 40%). In addition to occupational therapists, the sample included psychologists (n = 10, 22%), paediatricians (n = 4, 9%), speech pathologists (n = 4, 9%), a general practitioner (n = 1, 2%), and physiotherapist (n = 1, 2%). Participants were represented all Australian states, primarily Western Australia (n = 16, 36%), New South Wales (n = 13, 30%), and Victoria (n = 10, 23%). The majority of participants had at least a master’s degree (n = 24, 53%), and the sample had a median of 10 years of experience (range: 1–40 years). Participants’ demographics are outlined in Table 1.

Table 1 Clinician demographics
Full size table

Procedure

Convenience sampling was used, with clinicians recruited through social media, organisations’ mailing lists and professional networks until data saturation had been reached (Portney & Watkins, 2009). After registering their interest in participating through an online survey, and providing informed consent, participants were invited to participate in one of the available workshops. The use of workshops and a survey allowed a broader range of participants to participate, with the qualitative approach best suited to inductive exploration of a topic (Brooks et al., 2015). This study was undertaken as part of a larger program of research evaluating specific assessment of functioning tools. However, the above question was asked without reference to any specific tool, and prior to questions relating to specific tools. Four in-person and two online workshops were conducted. A self-directed survey version of the workshop was also available for participants who were not able to or did not want to participate in a workshop. Participants were asked to write their most important considerations or characteristics on post-it notes during the workshops, while survey respondents were provided with a free-text box.

Measures

Demographic information was collected using an online survey for both workshop and survey participants. Demographic information collected included age, postcode of practice, field and level of qualifications, and experience assessing functioning for individuals with NDCs. In addition to demographic information, the responses to the question “When choosing an assessment of functioning tool, what are the most important considerations or characteristics?” were collected as part of the interview schedule in the workshops, or via a free-text question in the survey. This allowed an understanding of the considerations and tool characteristics clinicians valued in assessments of functioning.

Data Analysis

Demographic information was analysed descriptively, including counts, percentages, medians, and ranges (Portney & Watkins, 2009). The qualitative data were entered into NVivo software (Q S R International Pty Ltd., 2018) and analysed via template analysis, without the use of a priori themes (Brooks et al., 2015; King, 2012). Initially, the first author inductively generated codes and the initial coding template, then the second author (who was also present during the workshops and had reviewed survey data), reviewed the coding template and provided feedback. The coding template was then revised based on the second author’s feedback, and then re-applied to the data by the first author. The coding template was reviewed a third time by the second author before final themes were confirmed. After the coding was complete, a representative quote was selected for each sub-theme (Brooks et al., 2015).

Results

From the survey and workshop responses, the considerations raised were grouped into five themes: easy, feasible, fair, holistic, and useful. Table 2 presents the themes and sub-themes (coding template), including exemplar quotes.

Table 2 Themes identified
Full size table

Easy

Clinicians frequently mentioned a tool’s ease of use in relation to administration, scoring, and interpretation. Considerations were raised related to both the clinician administering the tool and the respondents (e.g., caregivers, teachers, and the child being assessed). Clinicians felt it was important for a tool to be “easy for the parent to fill in”, and where relevant to its design, to be “engaging to [the] child”. The resulting report should also be “easy to go through with the [child] and family”. Features increasing a tool’s ease of use included the ability to email forms to caregivers to complete online, and automatically generated reports.

Feasible

In determining if a tool was feasible to use, time and cost were also frequently reported as important considerations. Cost included consideration of both the “out-of-pocket cost[s]” of purchasing an assessment tool and the clinician’s time in administering, scoring, and interpreting the tool. However, time-related considerations also included the clients’ needs, with one participant mentioning, “[The] duration of the assessment—can the child get through the assessment in one session or should it be broken up?”. Additionally, clinicians also considered their own training, qualifications, and experience to determine if the tool was appropriate or feasible for them to administer.

Fair

The reliability and accuracy of a tool was also mentioned as an important characteristic, usually in general terms. Specifically clinicians desired a tool to have inter-rater reliability, so that it is “’fair’ when administered by different professionals”. It was also valuable for a tool to be standardised and have norms, allowing comparison of a child to their peers. It was specifically noted that a tool such standardisation and norming have been established with populations relevant to the child being assessed. Clinicians also desired that a tool fit with other characteristics of the child to be assessed, including their age, diagnosis, cultural background, communication level, and presenting difficulties.

Holistic

The content of assessment tools was frequently discussed, with participants preferring comprehensive tools assessing a child’s functioning in detail (“[Is the tool] effective to understand the child and family in enough detail to support them?”), and breadth (“[covering] all areas of the child”). Clinicians specifically noted that “skill areas relevant to [the] population” should be covered. Assessing the role of the environment in a child’s functioning was also specifically mentioned as important, with the tool needing to be “meaningful to everyday function and participation” and to “capture daily functional needs of families”.

Useful

Clinicians considered the design and outcomes of an assessment tool to determine the usefulness of a tool to their clinical practice. They noted whether or not “[the tool] translate[s] to clinically useful goals and considerations” and “[if it] aligns with [the] DSM-5”. In addition to their immediate clinical practice, clinicians also considered the alignment between a tool and the systems in which they practiced. These systems included their specific practice context (e.g., “team protocol”), the tool’s “acceptability to funding bodies” such as the NDIS, its adherence to practice standards, and if it provided continuity across disability, health, and education sectors.

Discussion

The themes identified in this study highlight that clinicians consider a range of characteristics when selecting an assessment of functioning tool for children with NDCs, including if the tool in question is easy, feasible, fair, holistic, and useful. These considerations align with published frameworks, including Smart’s (2006) multi-dimensional model of clinical utility and the consensus-based standards for the selection of health measurement instruments taxonomy (COSMIN; Mokkink et al., 2010), used widely across various health and disability contexts. The common tool characteristics included in these existing frameworks and the themes identified in this work (e.g., inter-rater reliability, content validity, appropriateness, and feasibility) suggest that these frameworks could be used in future evaluations of the appropriateness of assessment tools assessing the functioning of children with NDCs.

While the COSMIN does not differentiate between the importance of different psychometric properties, the literature evaluating assessment tools focuses predominantly on reliability (McConachie et al., 2015). However, the themes identified in this study highlight that Australian clinicians prioritise validity, particularly content validity. While content validity, the extent to which a tool reflects the concept(s) it aims to evaluate, is prima facie in establishing psychometric properties, it is often assumed and rarely evaluated in assessment tools, particularly those aiming to assess functioning (Price et al., 2018; Wales et al., 2016). While the ICF provides a comprehensive benchmark for evaluating the content validity of assessment tools relating to functioning (World Health Organization, 2007), ICF Core Sets enable examination of the content validity of functional assessment tools designed for specific populations and age ranges (Bölte et al., 2019). Previous research using these standards has identified that existing tools have varying levels of content validity, with most neglecting to assess environmental factors (Chien et al., 2014; D’Arcy et al., 2021).

Systemic factors such as caseloads and time pressures also impact clinicians’ choice of functional assessment, leading to tools being chosen for their expediency and ease of use (D’Arcy et al., 2022). While the response time burden tools placed on families was included in this theme, it appears the importance placed on administration time was underpinned by pressures within the disability sector. This stands in contrast to best practice principles which point to the importance of client characteristics, their goals, and therapeutic and medical history, rather than systemic resource demands in driving the choice of functional assessment instruments in clinical practice (Bagatell et al., 2013; Kramer et al., 2009).

When selecting an assessment tool, in addition to the content validity and expediency of tools, clinicians considered tools’ appropriateness to a range of factors in the clinical context. These included the characteristics of the child being assessed, the clinician’s own experience and qualifications and the purpose of their assessment. Clinical requirements, frameworks used, and the purposes of conducting assessments of functioning vary widely within both inter-state and international contexts (Bagatell et al., 2013; D’Arcy et al., 2022). In Australia, the majority of disability supports are provided through the NDIS, which requires evidence of difficulty in functional capacity for eligibility (Commonwealth of Australia, 2013). However, clinicians highlight that there is a lack of guidance from the NDIS as to what is required in an assessment of functioning, negatively impacting their attempts to select the most appropriate tool for the context (D’Arcy et al., 2022). In the field of NDCs, functional assessment tools are frequently utilised in contexts and with populations beyond those for which they were designed. This is cause for concern as this calls into question the validity of these evaluations (Wales et al., 2016). Due to the complex interaction between client characteristics, context requirements and clinician knowledge and abilities, a ‘one size fits all’ approach to assessing functioning in NDCs is not recommended (Kramer et al., 2009).

Limitations and Future Research

The present study focussed on identifying the ‘most important characteristics or considerations’ to Australian clinicians in assessing the functioning of children with NDC’s and did not explore the preferences or the clinical reasoning behind clinicians’ choice of assessment tools. Further work is required to understand how the different characteristics identified in this study, such as the content validity, time requirement or scoring options of a tool, influence the choice of a specific tool in clinical practice. More detailed qualitative workshops would be beneficial, as would the use of Q methodology to determine if there are separate viewpoints on the topic once enough is known about the topic to create a concourse (Watts & Stenner, 2012). The characteristics of Australian assessors of functioning for children with NDCs are currently unavailable, other than those included in D’Arcy et al (2022); therefore, it is unknown if this sample represents the majority of clinicians assessing functioning for children with NDCs. The high representation of occupational therapists and psychologists is not unexpected, as these professions historically are the professions that focus on holistic functioning and adaptive behaviour. However, the inclusion of a wider range of professions in the sample would have strengthened the results.