Background

The increasing rates of long-term sickness absence and work disability in an ageing population have obliged several countries to shift their focus from providing long-term disability benefits and social protection programmes to promoting the work reintegration of people with partial or residual work capacity [1,2,3]. By introducing policy reforms, many countries have shifted their focus away from assessing disability on predominantly medical grounds to the assessment of the remaining work capacity of disability benefit claimants [4].

Several countries have developed new assessment instruments over the last ten years to assess individuals’ abilities to participate in the labour market actively, to assess barriers which may restrict work participation, and to indicate directions interventions may take to overcome barriers for work participation [4, 5]. These new assessment instruments have shifted from predominantly focusing on loss of physical and/or mental functioning, towards assessment of work capacity from a holistic perspective, i.e. the ability to participate actively in the labour market from physical, mental, social, and societal perspectives. Instead of the traditional disability assessment instruments, new instruments should not only assess limitations in activities [6], but also incorporate personal and environmental factors [1] which could mitigate limitations in activities when appropriate adjustments are applied. Although a biopsychosocial approach [3, 4, 6] has been integrated into many of these instruments, the literature about the validity of these instruments is limited.

Recognizing the interaction of activity limitations with the particular requirements of the individual’s work context led to the development of a novel approach for work capacity assessments by the Dutch Social Security Institute, the Institute for employee benefits schemes (UWV). The Social Medical Work Capacity instrument, SMWC, was developed by a panel of experts of the UWV (e.g. staff members, labour experts, and insurance physicians) and based on the International Classification of Functioning, Disability and Health (ICF) [7]. It has been developed to help the UWV professionals asssessing a clients’ ability to participate in work and to provide indications and/or advice for reintegration support to optimize the use of available potential and finding a good jobmatch [6, 8, 9]. The instrument was pilot tested in practice and showed that professionals using the instrument were positive as it provides a structure for describing the clients work capacity and their possibilities to participate in work [10, 11]. However, the professionals also critizised the large amount of items making the instrument timely in its use. Providing a better evidence base for the content of the instrument could improve utility of this new instrument in practice. Therefore, insight is needed whether all included items are relevant.

To examine the relevance of items needed to assess claimants’ remaining work capacity, it is important to evaluate the content validity of the SMWC, i.e. the degree to which the content of the instrument is an adequate reflection of the construct to be measured [12, 13], and to evaluate whether all items are relevant and comprehensive for the construct to be measured [14]. To validate the content of the SMWC, ICF core sets can be of potential benefit to determining these factors. They provide a minimum standard for the assessment and reporting of functioning and health [15]. Each ICF core set includes a selection of essential categories from the full ICF classification considered most relevant for describing the functioning and environmental factors of a person with a specific health condition or in a specific healthcare context. ICF core sets are frequently used in daily practice by clinicians and other professionals for the assessment and reporting of functioning and health [15].

The overall aim of the present study was to examine the content validity of SMWC by comparing the content of the instrument with ICF core sets.

Method

The Social Medical Work Capacity instrument (SMWC)

The Social Medical Work Capacity instrument, SMWC, was developed by a group of experts working as professionals at the Dutch Social Security Institute, the Institute for employee benefits schemes (UWV). The instrument is designed to guide social security professionals in taking a biopsychosocial approach when creating an overview of a person’s work capacity and what is needed to find a good jobmatch [8, 9]. The 129 items of SMWC related to 95 2nd level ICF categories, of which 54 from the Body Functions, 35 from Activities and Participation, and six from Environmental factors. With the exception of Chapter 6 (domestic life) and Chapter 9 of Activities and Participation (Community, social and civic life), all Chapters of the Classification of Body Functions and the Classification of Activities and Participation are represented in the SMWC. The SMWC does not include categories from the Classification of Body Structures. The ICF categories of the Classification of Environmental factors are mostly related to the work environment, such as climate, light, sound, vibration, and air quality (Chapter 1 and 2). Some ICF categories were further specified in the SMWC to provide more detail of work capacity items which is needed to exploit this capacity in actual work. Supplementary Table S2 presents a full overview of included ICF categories in the SMWC.

Procedure

First, we conducted a systematic search to identify relevant articles on relevant core sets for the working age population. Second the content of these core sets were mapped to the content of the SMWC. Next, we compared the content of the SMWC with the ICF-core sets.

ICF Core Sets

Medline, PsycINFO (both using Ebsco), and Web of Science were searched using the terms ‘disability evaluation’, ‘work capacity’, and ‘work ability’ combined with search terms identifying assessment instruments (including questionnaires) and ICF core sets [15]. The full search strategy can be found in Supplementary Table S1. The databases were searched for articles published between January 2000 and July 2018. Although the ICF was published in 2001, the year 2000 was also included as there was already a draft version of the ICF available.

Selection of Articles

Articles were included if they described the development of an ICF core set and presented final results. Letters to the editor, guidelines, editorials, book Chapters, dissertations, conference proceedings, design papers or case reports were excluded. Core sets were included if they were designed for the assessment of the functioning of working age (18–65) people with a specific disease or for their assessment in a work-related setting. A core set was excluded if the context precluded work, e.g. in acute or post-acute settings and geriatric settings. Additionally, core sets were excluded if they were developed in a too specific setting (e.g. applicable in a specific country). The ICF Research Branch website was checked for completeness [16]. A first selection based on title and abstract was conducted by two independent reviewers. When the reviewers could not reach consensus, a third reviewer was consulted. When the title and abstract did not provide enough information to decide if the inclusion criteria were met, the article was included for full-text screening. Disease-specific core sets are grouped into disease groups in line with the ICD-10 [17] and in accordance with the most prevalent diseases of people claiming disability benefits.

Data Extraction

First, data regarding core set, study aim, number of ICF categories included, and methods used were extracted from the included full-text articles by two reviewers. The methodological quality of the core set development was described taking ‘the guide on how to develop an ICF core set’ by Selb et al. [15] as the gold standard. When this gold standard was not applied in the development, the method used was described. Second, all ICF categories included in the core sets were registered. Data extraction was limited to the second level order of ICF categories. Figure 1 shows the hierarchical structure of the ICF classification. To allow for comparison with ICF core sets, the SMWC was compared on 2nd level ICF categories, collapsing items of 3rd and 4th level under the related 2nd level category.

Fig. 1
figure 1

The hierarchical structure of the ICF

Full size image

To structure the results, the included 31 disease-specific core sets were grouped into disease groups in line with the ICD-10 [17] and in accordance with the most prevalent diseases of people claiming disability benefits [18, 19]. Two work-related core sets completed the total inclusion of 33 core sets.

Content Comparison of SMWC with ICF Categories

First, to evaluate whether the SMWC contains all relevant items for the purpose of holistic work capacity assessments, a comparison was made with the two work-related core sets as these core sets are related to the construct of the SMWC.

Second, to evaluate whether the SMWC is comprehensive and thus covers all relevant aspects of the construct to be measured and whether all included items of the SMWC are relevant for the construct, we compared the content of the SMWC to the content of all retrieved ICF core sets, going beyond a specific work-related focus. This allowed for comparing the SMWC with core sets developed for reporting on functioning and health, and may lead to identification of common indicators across disease specific core sets which are possibly relevant to include in the new instrument. We used a relevance ranking by calculating the relative frequency of each ICF category within the disease groups. Scores of 0% indicated that an ICF category was not included in any core set, and scores of 100% indicated that an ICF category was included in all core sets of that particular disease group. Presentation of this relevance ranking was restricted to scores higher than 70%, a rather arbitrary cut-off. Subsequently, these common indicators were compared to the content of the work capacity instrument.

Results

Search

The combined searches yielded 3376 hits (1950 in Medline/PubMed, 637 in PsycINFO, 789 in Web of Science). After removal of duplicates, a total of 2267 abstracts were identified and 277 full-text articles on core sets were read. Forty-five articles described the development of a core set and presented the final results. Of these, 33 articles met the inclusion criteria and were included. A reference check of the included articles and a check of the ICF Research Branch website did not identify additional articles or core sets. However, our search retrieved three core sets that were not included on the website. Figure 2 depicts how the core sets were selected, and Table 1 provides a description of the characteristics of the included articles and two work-related and 31 disease specific core sets.

Fig. 2
figure 2

Flowchart of ICF core set inclusion

Full size image
Table 1 Overview of included articles and their core sets, aggregated by disease group
Full size table

Content Comparison of SMWC with ICF Categories

Work-Related Core Sets

The two work-related core sets are the Vocational Rehabilitation core set and the Disability Evaluation core set (see Text Box 1 for a further description). The SMWC, existing of 129 categories, and the two work-related core sets overlap on 47 categories (36% of the SMWC), mainly in Chapters Mental functions (b1), Learning and applying knowledge (d1), General tasks and demands (d2), Mobility (d4), Interpersonal interactions (d7), and Natural environment and human made changes to environment (e2) (see Table 2; Supplemental Figure S1). As well as the SMWC, both work-related core sets do not include categories from the Classification of Body Structures. The SMWC overlaps on 17 categories with the Disability Evaluation core set (13.2% of the SMWC) and 46 with the Vocational Rehabilitation core set (35.7% of the SMWC). A total of 54 ICF categories are included in the SMWC but not in any work-related core set (41.9% of the SMWC), of which N = 37 are from the Body functions, reflecting mainly physical and mental functions. In turn, the work related core sets contain 44 ICF categories not included in the SMWC, with the majority from the component of Environmental factors (N = 28). For instance, the Vocational Rehabilitation core set includes Environmental factors within the four ICF Chapters Products and technology (e1), Support and relationships (e3), Attitudes (e4), and Services, systems and policies (e5), which are all not included in SMWC (see Table 2). The Disability Evaluation core set does not include any Environmental factors because no consensus could be reached during its development on which factor to include [11]. See Supplementary Fig. 1 for an overview of overlap between the SMWC and the two work-related core sets on the ICF components (Table 3).

Table 2 Content comparison SMWC and work-related core-sets
Full size table
Table 3 Relevance ranking ICF categories in the disease specific core sets
Full size table

Disease-Specific Core Sets

The 33 disease-specific core sets were grouped into musculoskeletal conditions, cardiovascular and respiratory conditions, neurological conditions, mental conditions, and cancers, see Text Box 2 for a further description and grouping. First, when looking at the distribution of included ICF categories across the disease groups, the ICF categories are more or less equally divided over the Body Functions, Activities and Participation and Environmental factors, while 7.4% are from the Body structures.

The distribution of ICF categories across the ICF components in the SMWC differs from the distribution across disease-specific core sets, with 52.5% from Body Functions, 41.4% from Activities and Participation, and 6.1% from Environmental factors. No categories from the Body Structures are included, see Fig. 3. ICF categories with relative frequencies above 70% are in the Body Functions (N = 6), Activities and Participation (N = 14), and Environmental factors (N = 11), see Text Box 1. When comparing the content of the SMWC with the disease specific core sets on Chapter level, we see overlap in 10 ICF categories with high relative frequencies (> 70%) that are included in most disease specific core sets and the SMWC. Of these, four categories are from the Body functions and six from the Activities and Participation component, see Table 3. Highly frequent ICF categories in the disease specific core sets that are not included in the SMWC are related to social factors, e.g. friends, family and colleagues, factors related to taking care of oneself, e.g. washing, eating, caring for body parts, doing housework, and related to health professionals and systems.

Fig. 3
figure 3

Overview of the distribution of identified ICF categories within the included disease-specific core sets (aggregated by disease group), the work-related core sets and the SMWC over the ICF classifications

Full size image

Discussion

The aim of this study was to examine the content validity of the SMWC by comparing its content with ICF core sets. Comparison of the SMWC with the included work-related and disease specific core sets showed that the SMWC covers most of the relevant items on Body functions and Activities and Participation, however, most of the Environmental factors were lacking.

The relative strong focus on Body functions and Activities and Participation level may be due to the legal context in which the SMWC was developed and used. The SMWC was developed to provide a holistic view of work capacity, including medical history taking and attention to activity limitations and participation restrictions, influencing this capacity [8, 9]. Because of the legal constraints, the assessment is highly protocolized, leaving limited room to take personal and environmental factors into account. This might explain the scarse inclusion of these additional factors in the SMWC. When the outcomes are to be used to provide a holistic assessment of a persons’ residual work capacity and what is needed to find a good jobmatch, the content of the SMWC may therefore not be comprehensive enough. In line with the ICF framework, a dynamic interaction between health and personal and environmental factors are likely to have a direct or indirect influence on a persons’ work capacity [20, 21].

Work-related factors included in the SMWC are in particular assessing the physical work environment (e.g. heat, sound, air quality) and physical job demands, i.e. work endurance, working hours, and level of work exertion. Factors on psychosocial job demands (e.g. job content, decision authority, supervisor and colleagues support) are lacking. With the purpose of jobmatching in mind, additional environmental factors of the work place were included during the SMWC development, additional to the ICF. These factors are based on the content of currenty used methods for work capacity assessments, and relate to, for example work endurance [23]. However, to achieve a jobmatch, not only information about the person and hypothetical workplace factors are needed, but also knowledge about the physical and psychosocial job demands. Since the importance of work-related factors in the assessment of work capacity has long been recognised [22, 23], and that both physical and psychosocial job demands are predictors for work participation [24, 25], it is strongly recommended to extent the SMWC with this type of work characteristics in the work capacity assessment.

Highly frequent social factors, e.g. friends and family, and factors related to taking care of oneself, e.g. washing, eating, caring for body parts, doing housework, illustrate that ICF categories related to the social context might also be relevant to include in the SMWC. This is in line with findings of a recent systematic review showing the relevance of including the social context for work capacity. They concluded that several cognitive behavioural factors of significant others (like friends or family) can facilitate or hinder work participation [26]. When asked, insurance physicians also considered the context of community life, social life and civic life in addition to disease related factors and functions and structures as important factors for work capacity assessments [65].

Strengths and Limitations

A strength of this study is the use of ICF core sets in examining the content validity of the SMWC, which is in line with recent recommendations by the WHO and others to use the ICF in work disability assessments [22, 27]. Using the ICF framework to evaluate the content validity of the SMWC is a strong and novel approach and allows for a more structured assessment in comparison to expert judgements [14]. The ICF framework provides a holistic view of the person and provides a unified language for expressing these assessments, and core-set development is often standardized and published in peer reviewed publications. An additional strength is the systematical approach in identifying ICF core sets in the literature.

Some limitations should also be reported. The ICF does not operationalize personal factors and lacks specific work-related environmental factors [28,29,30,31]. Information about the work context or personal factors might provide valuable information relevant for work capacity evaluation as they possibly act as barriers or facilitators for work participation and are currently not included in our overview. Second, there is criticism regarding the development of core sets as they have a biomedical connotation, while the aim of the ICF is a biopsychosocial approach [15].

Implications for Research and Practice

The SMWC was developed to guide the social security experts in taking a biopsychosocial approach when creating an overview of a person’s work capacity and what is needed to find a good jobmatch. However, we showed that the instrument still has a focus on Body functions and Activities and Participation, and could be further developed by including additional factors to take into account the home situation (e.g. attitudes and relationships with friends and family), personal care (e.g. washing and doing housework), and workplace factors. Comparisons with disease specific core sets showed additional blind spots in the SMWC content. Further research could also focus on a more tailored use of the SMWC for specific diseases or underlying illness. The content comparisons with the disease specific core sets could therefore be a starting point for selection of relevant content. In addition, more research is needed to identify additional items in particular focussing on the work context, i.e. the implications of functioning problems for work opportunities, the barriers to participation in work, and the workplace adjustments or interventions required to overcome these barriers and achieve a good jobmatch. Work endurance, dealing with different types of working hours, level of exertion, estimating own options, overseeing the consequences of own actions, and achieving workpace are some examples of work related items that are found in the SMWC and not present in the ICF framework [51] and therefore also not identified in the core sets of our review. Additionally, aspects of the psychosocial work environment are important factors for finding a good jobmatch. It is therefore recommended to add these factors to the SMWC and possibly to the ICF framework, see also table S2. To further develop and improve precision and practical use of the SMWC, tailored subsets of the instrument should be identified together with insurance physicians and labor experts, combined with existing literature on barriers and facilitators in the work context or personal factors in various disease groups.

Conclusion

The SMWC content seems relevant, but needs to be more comprehensive for the purpose of use in work capacity assessments, as it has a relatively strong focus on body functions and activities and participation. To better achieve it’s goal in taking a biopsychosocial approach when creating an overview of a person’s work capacity and what is needed to find a good jobmatch, it is recommended to extend the instrument by adding personal and environmental factors, such as social factors and domestic factors, as well as more specific work related factors. To improve the use of the SMWC in practice, it is recommended to select the relevant disease-specific categories out of the comprehensive instrument, to aid a tailored use of the instrument.