Quality appraisal of workers’ wellbeing measures: a systematic review protocol
- 167 Downloads
Measuring wellbeing has never been so important. With the rapid growth of workplace wellbeing interventions, determining their effectiveness is not only good science but also good practice. A wide variety of wellbeing measures exist in the literature but it is not always clear what they are measuring, nor which measures best meet study objectives. This study seeks to identify the most valid and reliable measure/s of workers’ wellbeing.
Measures will be included if they were (1) designed for measuring workers’ wellbeing and (2) available in English. We will use a three-staged electronic search strategy to identify studies that include measures that meet the inclusion criteria: (1) electronic bibliographic databases for published work, (2) reference lists of studies with included measures, and (3) the reference list of previously published reviews. The following electronic bibliographic databases will be searched: OVID: psycINFO, psycTESTS, Cochrane library, AMED, Health and Psychosocial instruments; PubMed; PubPsych; Europe PMC; Scopus and Google Scholar. Database key search terms will include [wellbeing OR “well-being”] AND [employee* OR worker* OR staff OR personnel], and a validated search filter will be applied for the measurement properties. The methodological quality of the included studies will be assessed and rated. Then, this quality assessment of the included studies will be considered in the quality assessment of the measurement instruments. Finally, recommendations for the most appropriate instrument to measure workers’ wellbeing will be reported.
This systematic review will evaluate the quality of instruments that measure workers’ wellbeing. The findings of this review will improve clarity for researchers and clinicians in the appropriate instrument selection in the measurement of workers’ wellbeing.
Systematic review registration
KeywordsWork wellbeing Employee wellbeing Measures Systematic review Quality appraisal
COnsensus-based Standards for the selection of health status Measurement INstruments framework
Grading of Recommendations Assessment, Development, and Evaluation
Preferred Reporting Items for Systematic review and Meta-Analysis
Preferred Reporting Items for Systematic review and Meta-Analysis Protocols
Patient-Reported Outcome Measures
Work-related quality of life
With the rapid growth of workplace wellbeing interventions, determining their effectiveness is not only good science but also good practice. Wellbeing research continues to evolve; thus, it is likely that there will be advances in measurement. With the growing list of wellbeing measures (e.g., see [1, 2]), identifying and selecting the most appropriate instruments for effectiveness evaluations in the workplace have never been so important. Whilst a wide variety of wellbeing measures exist in the literature, it is not always clear what they are measuring, nor which measures best meet study objectives. For example, a recent systematic review of longitudinal studies that investigated workers’ wellbeing constructs found the majority of the 40 identified studies focussed on illbeing, or the “negative side” of employee wellbeing (e.g., burnout ). This systematic review protocol, and subsequent systematic review, will provide a unique opportunity to provide future rigorous updates as the work wellbeing science grows. The study will improve clarity for researchers and clinicians in the appropriate instrument selection in the measurement of workers’ (or employees’) wellbeing.
The construct of workers’ wellbeing is described as rich and multifaceted, with key features scaffolding individual, team and organisational levels, inclusive of factors that transcend work (the role), workers (the individuals and teams), and workplaces (organisations) . The factors associated with wellbeing differ for different occupational groups . For professionals, the greatest amount of variance in job satisfaction is due to the five factors of work-life balance, satisfaction with education, being engaged, and experiencing meaning, purpose and autonomy . For labourers, these factors were work-life balance, being absorbed, experiencing meaning and purpose, feeling respected, and having self-esteem . For nurses, these factors included workplace characteristics, the ability to cope with changing demands and feedback loops . The largely Western theoretical models and definitions of work wellbeing are also varied [7, 8, 9]. Key factors are thought to include subjective wellbeing, including job satisfaction, attitudes and affect; eudiamonic wellbeing including engagement, meaning, growth, intrinsic motivation and calling; and social wellbeing such as quality connections and satisfaction with co-workers . Laine and Rinne  add to these factors in their ‘discursive’ definition which encompasses healthy living/working, work/family roles, leadership/management styles, human relations/social factors, work-related factors, working life uncertainties, and personality/individual factors. Work-related quality of life (WRQoL) adds further factors, including general wellbeing, home-work interface, job and career satisfaction, control at work, working conditions, and stress at work . Given the breadth of these factors, and the disparity in theoretical models and definitions of what workers’ wellbeing is, selecting instruments for the measurement of workers’ wellbeing is challenging. The most appropriate instrument to measure the construct may require a selection of unidimensional (sub) scales, similar to the measurement of wellbeing  and WRQoL . It is expected that two different instruments that are intended to measure the same construct of “workers’ wellbeing” should correlate. Thus, we will test the a priori hypothesis: instruments intending to assess the same construct of “workers’ wellbeing” will be strongly positively correlated. For this review, the aim is to evaluate the measurement properties of instruments that measure the broader construct of workers’ wellbeing (e.g., the Workplace Well-being Index [14, 15]). Any identifiable sub-scales within the instruments will be individually reported. Specifically, the objectives are to (1) systematically identify studies that measure workers’ wellbeing, (2) critically appraise the methodological quality of the studies, (3) critically appraise the workers’ wellbeing instrument properties and, (4) recommend the most appropriate instruments to measure workers’ wellbeing.
This review protocol followed the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols 2015 checklist (PRISMA-P; [16, 17]). Any protocol amendments will be documented in the systematic review.
Review inclusion and exclusion criteria
Types of instruments
Eligible workers’ wellbeing data collection instruments include interviewer-administered, self-administered, or computer-administered. Examples include an online survey, a written questionnaire completed by a worker, or a worker’s responses to an interviewer’s questions.
Eligible studies will be those published as a full-text original article that (1) use a data collection instrument to investigate workers’ wellbeing (e.g., survey, interview or questionnaire) and (2) report measurement properties to enable reviewers to determine reliability (internal consistency, reliability, and measurement error/s), validity (content, criterion, and construct), responsiveness, and interpretability.
Settings and participants
This review will identify wellbeing outcome measures for workers’. Wellbeing measures may have been applied to any workplace (a workplace is defined as a place where a worker goes to carry out work ).
Types of wellbeing outcome measures
Instruments measuring workers’ wellbeing as an outcome will be eligible for inclusion. As a consequence of the disparate theoretical views and definitions of both wellbeing [19, 20, 21] and work wellbeing [7, 8, 9, 11, 22], for the purpose of this review, the terms used are very specific. For a measure to be included, the term ‘wellbeing’ must be specifically stated as either ‘wellbeing’, ‘well-being’ or ‘well being’. The term ‘workers’ must be specifically stated as either ‘employee*’, ‘worker*’, ‘staff’ or ‘personnel’.
Studies published in languages other than English will be excluded. Abstracts, books, theses and conference proceedings will be excluded.
The following electronic bibliographic databases will be searched: OVID: psycINFO, psycTESTS, Cochrane library, AMED, Health and psychosocial instruments; PubMed; PubPsych; Europe PMC; Scopus and Google Scholar. No date range will be applied.
A three-staged search strategy will be used to identify studies that include measures meeting the inclusion criteria: (1) electronic bibliographic databases for published work, (2) reference lists of studies with included measures, and (3) the reference list of previously published reviews.
Database key search terms will include [wellbeing OR ‘well-being’] AND [employee* OR worker* OR staff OR personnel]. Search terms for measurement properties of measurement instruments will draw from the ‘precise search filter for measurement properties’ and ‘exclusion filter’  (see sample search strategy, Additional file 1).
References identified in the search strategy will be exported to EndNote X8 bibliographic software, and duplicates will be removed
Titles and abstracts will be screened by two independent reviewers. The full-text documents of these potentially relevant studies will then be independently screened against the eligibility criteria by two reviewers. Any disagreement will be resolved through consensus of the wider research team. Findings from the search will be presented in a Preferred Reporting Items for Systematic review and Meta-Analysis (PRISMA) flow chart .
Data collection process
Data will be extracted into Microsoft Excel 2016 tables adopted from the COnsensus-based Standards for the selection of health status Measurement Instruments (COSMIN) methodology user guide . Example tables which include variables to be extracted are presented in Additional file 1. The data tables will be checked for accuracy and completeness by a second reviewer.
Assessment of the methodological quality of the included studies
The methodological quality of each single study on a measurement property will be assessed using the COSMIN risk of bias checklist for systematic reviews of Patient-Reported Outcome Measures (PROMs; ). For this review, in accordance with the COSMIN methodology recommendation, the term ‘Patient’ in ‘PROMs’ is considered synonymous with the population group for this study, ‘Worker’. The COSMIN checklist includes 10 boxes, two for content validity, three for internal structure, and five for the remaining measurement properties of reliability, measurement error, criterion validity, hypotheses testing for construct validity and responsiveness . In terms of content validity, the COSMIN criteria and rating system for evaluating the content validity of PROMs will be used. Then two reviewers will independently rate each study’s quality for each measurement property using the COSMIN checklist 4-point rating scale as either very good, adequate, doubtful or inadequate . A third reviewer will be consulted if any disagreement arises.
Evaluation of the study results against criteria for good measurement properties
The quality of the measurement instruments will be rated as either sufficient, insufficient or indeterminate against the recently updated and published (see Table 2, p. 7; ) criteria of good measurement properties .
Data will be presented in summary of findings tables such as those presented in Additional file 1. The results from different studies on each measurement property will be quantitatively pooled in a meta-analysis, for example, by calculating weighted means and confidence intervals. If the data do not support meta-analysis, they will be summarised, for example, by providing ranges for interpretability values and percentages of confirmed hypotheses for construct validity.
Confidence in cumulative evidence
The evidence will be summarised and the quality graded using four of the five factors in the Grading of Recommendations Assessment, Development, and Evaluation (GRADE; ) approach: risk of bias (i.e., methodological quality of the studies), inconsistency (i.e., unexplained inconsistency of results across studies), imprecision (i.e., total sample size of the available studies), and indirectness (i.e., evidence from different populations than the population of interest in the review) . Results of all available studies on each measurement property will be quantitatively pooled or qualitatively summarised and compared against the criteria for good measurement properties to determine whether overall the measurement property of the measure is sufficient, insufficient, inconsistent, or indeterminate. If ratings per study are all sufficient (or all insufficient), results can be statistically pooled and the overall rating will be deemed sufficient (or insufficient) according to the criteria of good measurement properties. If results are inconsistent, this will be explored to determine explanations for inconsistency. Where explanations are determined, overall ratings will be provided for relevant subgroups with consistent results. Where no explanation is determined, the overall rating will be inconsistent. Where there is insufficient information available, the overall rating will be indeterminate. Overall ratings of each measurement property and the grading for the quality of the evidence will indicate reviewer confidence that pooled results and overall ratings are trustworthy. Where the overall rating for a specific measurement property is indeterminate, there will be no grading of the quality of the evidence as the quality of the measure cannot be judged. The quality assessment will be undertaken by two reviewers independently, and a third reviewer will be consulted in the event of unresolved disagreement.
This review systematically identifies and appraises measures that intend to assess the specific construct of workers’ wellbeing. The results will elucidate the existing specific measures of workers’ wellbeing and the quality of their measurement properties. This is an important first step to support future workers’ wellbeing researchers to identify and select the most appropriate instruments for effectiveness evaluations.
Ethical approval is not required for this systematic review that identifies, appraises, and synthesises published data.
Findings from this review will be published in a peer-reviewed journal.
Availability of data and materials
The datasets generated and/or analysed during the current study are available from the corresponding author on reasonable request.
RJ conceived and drafted the protocol. MR, RS and JKM reviewed, revised and edited the protocol. All authors read and approved the final manuscript.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
- 1.Carlson J, Geisinger K, Jonson J, editors. Twentieth Mental Measurements Yearbook. Nebraska: University of Nebraska Press; 2017.Google Scholar
- 5.Hamling K, Jarden A, Schofield G. Recipes for occupational wellbeing: an investigation of the associations with wellbeing in New Zealand workers. N Z J Hum Resour Manage. 2015;15:151–73.Google Scholar
- 8.Dewe P, Kompier M. Foresight mental capital and wellbeing project. In: Wellbeing at work: future challenges. London: The Government Office for Science; 2008.Google Scholar
- 10.Lawrie E, Tuckey M, Dollard M. Job design for mindful work: the boosting effect of psychosocial safety climate. 2018;23:483–95.Google Scholar
- 12.Easton S, Van Laar D. User manual for the work-related quality of life (WRQoL) scale: a measure of quality of working life. 2018 edition. Portsmouth: University of Portsmouth; 2018.Google Scholar
- 13.Jarden A, Jarden R. Positive psychological assessment for the workplace. In: Oades L, Steger M, Della-Fave A, Passmore J, editors. The Wiley Blackwell Handbook of the psychology of positivity and strengths-based approaches at work. New York: Wiley; 2016. p. 415–37.Google Scholar
- 14.Page K. Subjective wellbeing in the workplace. Unpublished honours thesis. Melbourne: Deakin University; 2005.Google Scholar
- 18.Health and Safety at Work Act 2015. No. 70. New Zealand: New Zealand Parliamentary Counsel Office; 2015.Google Scholar
- 21.Hone L, Schofield G, Jarden A. Conceptualizations of wellbeing: insights from a prototype analysis on New Zealand workers. N Z J Hum Resour Manage. 2015;15:97–118.Google Scholar
- 22.Fisher C. Conceptualizing and measuring wellbeing at work. In: Cooper CL, editor. Wellbeing: a complete reference guide. Hoboken: Wiley; 2014.Google Scholar
- 24.Moher D, Liberati A, Tetzlaff J, Altman DG, Altman D, Antes G, Atkins D, Barbour V, Barrowman N, Berlin JA, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:1-6.Google Scholar
- 25.Mokkink L, Prinsen C, Patrick D, Alonso J, Bouter L, de Vet H, Terwee C. COSMIN methodology for systematic reviews of Patient-Reported Outcome Measures (PROMs): user manual; 2018.Google Scholar
- 28.Prinsen C, Vohra S, Rose M, Boers M, Tugwell P, Clarke M, Williamson P, Terwee C. How to select outcome measurement instruments for outcomes included in a “Core Outcome Set” - a practical guideline. BMC Trials. 2016;17:449–59.Google Scholar
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.