Introduction

For people with limited capability for work (LCW) due to physical or mental disability, having a job contributes to health, well-being and personal growth, and to important values such as inclusion in society and full and equal enjoyment of all human rights. Many countries, including the Netherlands, ratified the United Nations Convention on the Rights of Persons with Disabilities and aim to stimulate the inclusion of people with disability in the labour market [1]. However, despite formal policies, employment rates lag behind and part time, temporary contracts are the norm [2, 3]. In the Netherlands, approximately 1.8 million people experience LCW [4]. Around 50% of the people with capability for work are working, of which only 60% are still in employment after one year [5]. This is alarming, since many people with LCW are able to work and eager to do so [6].

Research on sustainable employability (SE) of people with LCW is crucial for a movement towards more sustainable participation for all. SE refers to an individual’s chance to find and maintain work [7, 8]. Despite its importance, research on SE has not yet specifically targeted people with LCW, which consists of a heterogeneous group of individuals who experience functional limitations (such as limitations in cognition, communication, social interactions, etc.) which may overlap between different disabilities. People with LCW may experience a combination of functional limitations, and their severity varies from person to person. These functional limitations limit the ability of this target group to engage in (regular) paid work without additional support or accommodations. Studies on SE that do exist for people with LCW typically consider a small array of indicators (e.g. job satisfaction, work motivation and work engagement [9, 10]) and do not allow for a comprehensive understanding of SE, as this construct is multidimensional in nature.

While integrative measurement instruments for SE are available for a general population [11, 12], recent calls stress the importance of developing instruments that are specifically designed for people with LCW [13]. Accessible questionnaires with clear instructions and items that are tailored to the abilities and experience of people with LCW are crucial for participation and valid results. Research conducted with similar groups showed that the experiential knowledge of employees with LCW can contribute to the adaptation and validation of questionnaires [14, 15]. With appropriate, adapted, instruments, we will gain a better understanding of how to create opportunities for sustainable employment for this large, important and diverse group of people.

This study aims to develop and validate a questionnaire to measure the SE for people with LCW. Specifically, we draw on the validated measurement instrument for SE by Fleuren et al. [12] as used among a general working population. Additionally, the questionnaire includes person-job fit (PJ fit) and work-related sense of coherence (Work-SoC) as close antecedents of SE. This paper firstly presents a participatory adaptation process to develop a measurement instrument for SE that is suitable for people with LCW. To ensure face validity of the resulting instrument, we use cognitive interviews and include a co-researcher with LCW in our research team. Secondly, we assess the dimensionality and validity of the tailored questionnaire using confirmatory factor analysis on self-reported questionnaire data of a large sample of people with LCW. Specifically, we include people with LCW to whom the Dutch Participation act applies and who work for a governmental organisation that facilitates a diverse range of paid jobs for this target group. With this approach, the present article provides a validated questionnaire to assess the SE of people with LCW, thereby enabling more profound research on their SE. Moreover, the current approach provides an extensive illustration of how existing instruments to measure complex constructs can be adapted to suit specific populations and settings.

Sustainable Employability

In this paper, we define SE as follows: ‘that an individual’s ability to function at work and in the labor market, or their ‘employability’, is not negatively, and preferably positively affected by that individual’s employment over time’ [8]. This definition operationalises SE as an individual and formative construct, with a set of complementary indicators of functioning in terms of health, well-being and competence that should ideally be considered over time. Indicators in the health domain include perceived health, work ability (the ability to work given personal health status) and measures of fatigue (typically captured as the need for recovery and/or fatigue). These indicators are complemented by job satisfaction and motivation to work to cover occupational well-being, as well as perceived employability, skill gap and job performance as indicators of competence. Several other definitions of SE exist (e.g. [11, 16, 17]), but these definitions suffer from shortcomings that the current definition resolves (see [8] for an in-depth discussion).

Other approaches to SE offer theoretically relevant antecedent constructs that enrich our understanding of SE. The most cited approach to SE, the capability approach, is based on the notion that generating value through work is central to SE [11]. Its premise is that if workers can achieve value at work, they will be sustainably employable. This approach has raised considerable criticism because it simultaneously treats and defines SE as a characteristic of the job and the employee [18]. To gain a better understanding of individual SE and how this is affected by work, we need instruments that enable us to disentangle the value of work from an individual’s SE. The salutogenic perspective offers means to do so, as it underscores the importance of an individual’s experience of work as meaningful, manageable and comprehensible—i.e. work-related sense of coherence (Work-SoC) [19]. Strong Work-SoC has been associated with well-being (e.g. fatigue) and positive work outcomes (e.g. work involvement) in normal work populations [19, 20]. As such, we consider Work-SoC as a valuable way of operationalizing the idea of achieving value in work in the context of SE, acknowledging it as close antecedent factor of individual SE.

The ability to generate value through work depends on the interaction between individuals and their work environment. Therefore, we expand the questionnaire also with person-job (PJ fit). PJ fit describes to what extent the abilities and characteristics of a person align with (the tasks of) their job. PJ fit consists of two dimensions. The demands-ability fit refers to extent to which an employee has the skills, knowledge and capabilities to accomplish the requirements of the job. The needs-supplies fit represents the extent to which work accommodates an individual’s needs [20]. A strong PJ fit is positively related to SE indicators in general work populations (e.g. job satisfaction and performance [20]) and has been used as an indicator of SE [16]. For people with LCW, an association is shown with exhaustion and work engagement [21], although instruments for regular work populations were used. Literature further indicates a relation with job satisfaction for this target group, although the evidence is limited [22]. By including PJ fit, we recognize its relevance for meaningful work experiences and individual SE.

Methods

Overview

The study consisted of two phases. The first phase developed and improved the face validity of the predefined questionnaire for SE using cognitive interviews and a participatory approach. The second phase involved administering the adapted questionnaire to people with LCW to assess its reliability and factor structure. Informed consent of participants was asked prior to research activities, and the study was approved by the Wageningen Scientific Ethical committee.

Setting

This study was part of a larger longitudinal study on the effects of work in natural environments on the health and SE of people with LCW in the Netherlands. This study was carried out at a central governmental organisation in the Netherlands that works closely with other governmental departments (e.g. the Dutch State Forestry Service or the Dutch Tax Administration) to provide paid work for people with LCW, who meet the LCW criteria of the Dutch Participation Act. Employees carry out work in, e.g. forestry, facility management or archive editing and receive practical and social support from their work supervisor. The organisation aims for job security for people with LCW, providing the opportunity for permanent employment after an initial probationary period.

Initial Questionnaire

The initial questionnaire contained 56 items, including scales regarding SE, PJ fit and Work-SoC (41 items). When possible, the authors used existing and validated scales for the main concepts. If the scales were not available in Dutch, they were translated forward and backward to maintain their original meaning [23]. The scales are displayed in Table 1. In Phase I, the full questionnaire (including informed consent, socio-demographic factors and work-related control variables) was tested and adjusted where necessary, because complex introductions or too many (difficult) items can lead to response burden. Response burden may affect the responses of participants, influencing the validity of the main scales [24]. The items capturing SE, PJ fit and Work-SoC can be found in Supplement A.

Table 1 Scales used for the initial questionnaire
Full size table

Phase I: Improving Face Validity and Comprehensibility

Study Design

Phase I consisted of a participatory approach involving active participation of people with LCW [32]. First, people with LCW participated in cognitive interviews [33] that aimed to systematically investigate and improve the clarity, comprehensibility and content validity of the questionnaires. During the cognitive interviews, participants provided feedback and identified potential improvements. This method has been used successfully with similar target groups before [14, 15]. Second, the research team included an employee with LCW as co-researcher (AdJ), following guidelines for inclusive research [34], who actively contributed to the interpretation of the cognitive interviews results and the design of the final questionnaire (e.g. item and instruction formulations). Here explicit roles included reflecting and offering a sounding board for the researchers, designing and writing for new questions and procedures and co-deciding on adjustments.

Participants

Six employees with LCW were included in the cognitive interviews. Participants were recruited using convenience sampling at one work site within the overall project. All participants were men, and their ages ranged from 18 to 33 years. Their educational level ranged from (special) primary education to post-secondary vocational education (see Table 2). All participants experienced LCW due to mental or physical health disabilities, among which mild intellectual disabilities, severe dyslexia and/or physical and mental problems. Although only male participants were included in this stage, they experienced different limitations and represented a broad spectrum of people with LCW. The co-researcher was female and provided input in the interpretation of the cognitive interview results.

Table 2 Demographics of participants in Phase I & II
Full size table

Procedure

During the cognitive interviews, the principal investigator (SRH) asked individual participants to complete the questionnaire digitally on Microsoft Forms, to read the questions and their responses out loud and to explain how they decided to respond. This provided first insights about the comprehensibility of items. The principal researcher then asked additional probing questions, to identify both expected and unexpected challenges [14, 33]. The interview guide was based on the Question Appraisal System (QAS-99) by Willis [33] and contained instructions for the cognitive interviews, definitions of the intended meaning of items and probing questions. When problems were identified, participants were asked about potential improvements. Interviews lasted between 39 min and 1 h. Three rounds of two cognitive interviews each were conducted iteratively. The interviews were audio-recorded, and additional notes were made. The interviews were transcribed intelligent verbatim (see Fig. 1 for a visualisation of the process).

Fig. 1
figure 1

Process of Phase I

Full size image

Analysis & Adjustment of Questionnaire

First, the principal investigator identified and clustered problems and suggestions for improvement. This process resulted in an overview containing the initial intended meanings of items, identified problems within the QAS-99 classification and participant suggestions for improvements. Subsequently, the co-researcher read the overview in silence, while the principal researcher was present for emerging questions. The co-researcher noted salient issues and additional suggestions for improvement.

After the analysis, the principal researcher and the co-researcher discussed the results and decided with consensus which suggestions, changes, additions and deletions would improve the questionnaire for people with LCW. Instances of doubt on improvements were recorded by the principal researcher. Finally, the adjusted concept questionnaire was discussed among the researchers (SRH, BPIF, LV) to check for scientific rigour and legal boundaries and to resolve instances of doubt.

Phase II: Test for Construct Validity

Procedure & Participants

The adapted questionnaire was administered to employees of the governmental organisation that participated in the longitudinal study. The questionnaire was introduced to employees by their supervisors, who explained the purpose of the study to their team and facilitated the introduction game. After this, supervisors shared the questionnaire link with their employees, who could voluntarily decide whether to participate. Participants were asked to complete the questionnaire independently (if possible) during working hours. In total, 248 employees participated in the study. Their average age was 33.9 (SD = 12,40) years (n = 248). The majority of participants identified as male (n = 187), and attended post-secondary education (n = 126) (Table 2).

Analysis

To assess dimensionality and validity of the tailored questionnaire, confirmatory factor analysis (CFA) with Mplus 7 (Muthén & Muthén, Los Angeles) was applied (1) to test the factor structure of each multiple-item scale with three or more items for the indicators of SE and (2) to test the overall models for SE, PJ fit and Work-SoC. To test the overall model of SE, indicator scales of which a low score indicated high employability were reverse-coded to ensure that a high score would contribute positively to SE. Of interest in these analyses were model fit and factor loadings. Regarding model fit, the goodness-of-fit was based on the Comparative Fit Index (CFI) and the Tucker-Lewis Index (TLI) (≥ 0.90), the root mean square error of approximation (RSMEA) (≤ 0.08) and the assessment of factor loadings (≥ 0.3) [35]. The fit of the model was evaluated by the overall fit based on a majority of the fit indices. The weighted least squares mean and variance adjusted (WLSMV) estimator was used because inspection of the data suggested that the response distributions were not normal [36]. For completeness, we also estimated the internal consistency of the scales using Cronbach’s alpha, where α ≥ 0.7 indicates acceptable reliability.

Results

Phase I: Improving Face Validity and Comprehensibility

In total, 173 experienced problems were identified (Table 3). 144 adjustments were made to the questionnaire by modifying, adding or deleting an item, a response category or an instruction. These adjustments led to the final version of the questionnaire (57 items), including items for SE indicators (20), PJ fit (6), Work-SoC (9) and socio-demographic questions and work-related control variables (22). The total number of items increased due to additional socio-demographic and work-related control variables. The total number of items to measure SE, Work-SoC and PJ Fit decreased. Overall, there was a decrease in identified problems after each cognitive interview round. Most problems occurred in the QAS-99 components ‘clarity’, ‘reading’ and ‘response categories’. The results of Phase I for the QAS-99 categories are presented in Table 3, and the final items for SE, PJ Fit and Work-SoC are presented in Supplement A.

Table 3 Overview of identified problems and adjustments
Full size table

Clarity

Regarding clarity, 51 problems were identified, mostly due to difficult or unfamiliar words (such as the Dutch word ‘vervoering’/ ‘ecstasy’ from ‘Utrecht Work Engagement Scale—9’) or technical terms (like ‘production standard’). These issues were addressed by adding examples, using more familiar language, removing items, and experimenting with shorter/longer versions of the scales (which led to the use of ‘Utrecht Work Engagement Scale—3’ [37]).

Reading

Participants experienced 47 problems reading the items or interpreting these correctly, due to uncertainty about how or what to read and missing information. Examples include long introduction texts, missing information and items that start with one sentence followed by several statements, causing the cued context to be lost (i.e. with the items from the original Work-SoC questionnaire: ‘I find my work …. Manageable/meaningful/predictable’). Text length was adjusted, and complete sentences were added for each statement.

Response Categories

Regarding ‘response categories’, 36 problems occurred. Exemplary was the Work-SoC scale that originally contained nine items consisting of two opposite terms with a 7-point Likert scale in between. Participants were unsure whether the 7-point scale was supposed to represent a gradual shift from one term to the other. Additionally, other scales were scored using agreement-oriented 5-point Likert scales while yet others used 7-point scales or used temporal classifications for scoring, e.g. ‘once a week’. This confused participants and required them to seriously concentrate while completing the questionnaire. These problems were addressed by experimenting with new layouts and adjusting response categories when possible to a 5-point Likert scale ranging from ‘totally disagree’ to ‘totally agree’ with labels for each scale point in between.

Assumptions

Thirteen problems arose due to incorrect assumptions by the researchers about the context and work situation of participants. For example, one item addressed the extent to which the participant meets the performance standard of their job [29]. However, in the current work situation of participants, work was actually adjusted to individual capacity instead of a performance standard. In these cases, items were reframed to better suiting descriptions, e.g. ‘I do my work well’.

Instructions

Problems with ‘instructions’ were identified eight times and included complexities in introductions, instructions or explanations. For example, when a subscale was longer than one page, the guiding text was repeated on the next page, causing confusion (‘we just completed these questions’). The problems were addressed by clustering subscales on one page or explicitly referring to a continuation of questions for a specific topic.

Knowledge and Memory

Eight problems occurred because participants did not know or had trouble remembering specific information. For example, recalling the specific month in which participants started working at their current job proved difficult. If possible, items that led to these errors were eliminated or cued with examples.

Sensitivity and Bias

Seven problems were identified with respect to sensitive language or content. Items or topics could make respondents uncomfortable or experience (negative) emotions, for example items regarding educational background. The answer possibility ‘I prefer not to say’ was added.

Questionnaire Administration Procedures

After the second cognitive interview round, it became evident that linguistic modifications couldn’t address challenges such as difficulty remembering dates or emotional responses. Based on suggestions from the cognitive interviews and the collaboration with the co-researcher, we decided to familiarise participants with these items before administering the questionnaire. To achieve this, we created an introductory game, as games can help people to reflect on challenging topics [34]. The game consists of a twelve-sided die with both challenging and easy questionnaire items. The game is designed as a team effort and is meant to be played in a team of employees with LCW. Employees take turns rolling the die and answer the questions themselves or ask a colleague to do so. Before administering the questionnaire, supervisors of these employees introduce the purpose of the study and facilitate the game. They guide the conversation and help to interpret the items.

To assess the game’s effectiveness, we invited the complete work-site team that provided the convenience sample for the cognitive interviews before the third cognitive interview round. This team (n = 8) comprised the work supervisor, four previous cognitive interview participants, two employees who did not yet participate in cognitive interviews and the co-researcher. The researcher was present and available for questions. Analysis of cognitive interview round 3 showed that participants recognised the items introduced by the game and that difficulties related to remembering answers and emotional charge had decreased. Consequently, the game was integrated into the questionnaire procedure of the final questionnaire.

Phase 2: Validity Assessment Results

The results show a generally good fit for most of the scales for the SE indicators (Table 4), except for external employability (CFI = 0.95; TLI = 0.85; RMSEA = 0.303 [CI 90% 0.206–0.414]) and work engagement (CFI = 0.91; TLI = 0.74; RMSEA = 0.533 [CI 90% 0.432–0.642]). The RMSEA did not meet the cut-off criteria for any SE indicator scale. However, this failure to meet the cut-off criteria should be interpreted with caution, as RMSEA can falsely indicate a poor fit due to low degrees of freedom [38], which was the case in this study (Table 4). For all indicator scales, the factor loads for all items were adequate (Supplement B) and the internal reliability proved to be acceptable-to-good (α = 0.72–0.83).

Table 4 Results validity and reliability assessment of multiple-item indicators of SE, Work-SoC and PJ fit and the overall model for SE, including descriptive statistics of all SE indicators
Full size table

The overall hypothesised SE model, based on the subscales for the indicators, was found to have a generally good fit (CFI = 0.91; TLI = 0.88; RSMEA = 0.061 [CI 90% 0.052–0.069]). Factor loadings proved to be adequate (Supplement B). However, the cognitive interviews suggested that in the specific context of this research, internal employability could rather be an indicator of organisational policy than the actual internal employability of the individual. An exploration of a competing model (Supplement B) revealed that the final questionnaire with the structure of the hypothesised factor and without the internal employability item provided a better and adequate model fit (CFI = 0.93; TLI = 0.92; RSMEA = 0.054 [CI 90% 0.044–0.063]).

Regarding the antecedents of SE, the results show an acceptable-to-good overall fit and internal consistency for both PJ fit (CFI = 0.96; TLI = 0.93; RMSEA = 0.286 [CI 90% 0.249–0.324], α = 0.90) and Work-SoC (CFI = 0.98; TLI = 0.97; RMSEA = 0.077 [CI 90% 0.053–0.102]; α = 0.73). Factor loadings were sufficient for all PJ fit and Work-SoC items (Supplement A).

Discussion

This study aims to validate a questionnaire to measure SE, PJ fit and Work-SoC for people with LCW. The study was conducted at a central governmental organisation that works closely with governmental departments to provide paid work for people with LCW, who meet the LCW criteria of the Dutch Participation Act. First, the results show that the participatory validation process contributed to the improvement of the face validity of items for SE, PJ fit and Work-SoC for people with LCW, as demonstrated by the number and nature of adjustments. Second, the study demonstrates that the tailored scales for SE, PJ fit and Work-SoC are valid and reliable instruments to measure SE, PJ fit and Work-SoC for people with LCW.

The cognitive interview results show that face validity benefited most from improving the clarity of the questions, the answer categories, the accompanying instructions and the procedure. Based on the results and the cut-off scores of this study, the final questionnaire had good psychometric properties and can be used for future research on SE, PJ fit and Work-SoC for people with LCW. These results are in line with recent studies that developed other questionnaires for people with disability [14, 15]. Noteworthy is that the ‘Utrecht Work Engagement Scale—3’, which was not adjusted due to copyrights, appears to have the weakest fit results of the indicators examined for SE. However, previous studies on the Utrecht Work Engagement scales also showed weak goodness-of-fit statistics in general populations [39].

Regarding the Work-SoC scale, it is noticeable that factor loadings for item 4 (λ = 0.477) and item 9 (λ = 0.328) are relatively low. This suggests that the subscales for manageability (item 4) and comprehensibility (item 9) are not unidimensional. Based on our results, it is difficult to distinguish whether these findings are due to the specific target group or to dimensionality issues of the subscales. Studies in general working populations have also shown problems with other items and the dimensionality of subscales [40, 41].

The current study underlines the importance of clarity of questions, response options and instructions, and feasible procedures. Furthermore, the results show the importance of contexts for the validity of measurement scales [42], as illustrated by the item on internal employability in this study. This item was understood as intended and reflected the possibility of a participant to keep their job at their current employer. However, this seemed to be due to organisational policy rather than to reflect the employability of an individual. Removing this item led to a better model fit. While these findings are presented in the light of validity for people with LCW, one could argue that unambiguous measurement instruments, based on correct assumptions about target groups and their context, are beneficial for broader research populations.

Strengths and Limitations

A major strength of this study is the participatory approach. Cognitive interviews and the cooperation with a co-researcher allowed for the systematic observation and discussion of problems in order to optimise the questionnaire and its procedures. In line with other participatory studies, the results showed that incorporating experiential knowledge positively impacted the suitability and validity of the questionnaire [14, 15, 43]. Additionally, the participatory approach evoked co-learning and empowerment processes, as is recognised in other inclusive research settings [44, 45]. For example, the co-researcher experienced opportunities to gain new skills and knowledge, and researchers became more aware of the context and implications of their activities. In addition, the iterative approach of this study allowed to adapt administration procedures with additional features, reducing the burden on participants and contributing to validity.

A first important limitation of our study is the specificity of the employment setting, which limits the generalisability of our results to different settings or professions. Second, we aimed to develop and validate a questionnaire on SE for people with LCW in general. Given the desired heterogeneity of our sample that matches the target population, it might be relevant to check if different subgroups (e.g. based on gender, age, education level or mental/physical condition(s)) show measurement invariance [46]. However, despite our relatively large sample size (i.e. N = 248; for this target population obtaining larger samples is challenging), subgroups became too small to perform adequate SEM with invariance tests in this study. Future studies should further explore if invariance exists, specifically by establishing the stability of the reflective measurement models of the scales used and the formative indicator-to-construct paths across subgroup categories. A third apparent limitation resides in the relatively small convenience sample consisting exclusively of males in Phase I. However, this sample was diverse regarding experienced disabilities and the co-researcher could supplement the female perspective. The method allowed for in-depth reflections, thereby ‘guiding “best informed” design decisions’ [47], and there were no indications that a lack of diversity in perspectives biased our findings. Nonetheless, future studies may want to include more (e.g. gender-) diverse samples. A fourth consideration is that, ideally, longitudinal studies should be conducted given SE’s longitudinal nature. As noted in a similar cross-sectional validation study on SE, the present cross-sectional study offers an important basis for doing so among people with LCW [12]. In sum, the aforementioned limitations emphasise the importance of replication studies among diverse settings, samples and timeframes. The questionnaire developed in the present study offers means to do so.

Implications for Occupational Science and Practice

The current study provides first a validated questionnaire to measure SE, PJ fit and Work-SoC for people with LCW. The study’s participatory approach is an example of how researchers can tailor questionnaires to specific target groups, in specific settings. This example highlights the importance of participatory collaboration in psychological research [48]. Second, the validated questionnaire enables researchers to measure SE across a diverse range of people with physical and mental disability, while acknowledging the antecedent importance of aligning work with individual needs, capacities and values [7, 16]. However, when this questionnaire is used in another country or setting, it is recommended to reapply cognitive interviews with diverse samples to make sure that face validity is not lost in translation.

Furthermore, the findings raise new questions about the measurement of Work-SoC for the target group and broader research populations. The study found low factor loadings for the comprehensibility and manageability dimensions. These have proven difficult to discriminate in general populations [40, 41]. However, specific items may have limited success in capturing comprehensibility and manageability for people with LCW. Further research is needed to explore the underlying factor structure of Work-SoC in different populations. Nevertheless, at this moment, the tailored Work-SoC scale serves as the best instrument to measure Work-SoC for people with LCW in a Dutch context.

Future studies demonstrating the broader validity of the questionnaire can contribute to further insight into what makes work ‘work’ for whom and when. This knowledge can improve the development and evaluation of supportive work environments for people with LCW, thereby contributing to sustainable work participation.

Conclusion

The participatory validation process carried out within a governmental employment setting for people with LCW resulted in a validated and comprehensive questionnaire to measure SE, PJ fit and Work-SoC for this target group, which enables research into the development of their SE. This knowledge can be used to contribute to a more inclusive society with meaningful occupations for people with LCW.