Abstract
This prospective study aimed to determine the patient acceptable symptom state (PASS) cut-off for the patient reported outcome measure shoulder pain and disability index (SPADI), and evaluate predictors of PASS achievement following standard shoulder care. Patients with shoulder pain, referred for shoulder ultrasound were recruited from a community cohort. Patients completed both SPADI (scored 0–130) and a question on symptom state and followed-up at 6 months. PASS was calculated from Rasch-transformed scores using 2 methods: the 75th percentile of the cumulative response curve and the receiver operating characteristic curve (ROC). Logistic regression was used to identify factors associated with PASS. 304 participants (169 females, mean age 57.2 years) were included. At 6 months, 193 (63%) reported PASS. The association between SPADI at 6 months and PASS depended on baseline SPADI (interaction p = 0.036). Those with higher baseline scores had higher 6 months PASS cut-offs. Using the 75th percentile method, the 6 months total SPADI cut-off was 49.2 in those starting in the highest tertile at baseline compared to 39.4 in the lowest tertile: 46.4 vs. 36.7 for pain, 46.8 vs. 25.1 for disability. The ROC method yielded similar results. We have shown for the first time that the PASS cut-off for SPADI is dependent on baseline severity scores. Understanding the SPADI PASS threshold is important for clinical research to allow standardised reporting of shoulder intervention success at the patient level.
Avoid common mistakes on your manuscript.
Introduction
Shoulder pain is common and often poorly treated [1]. Patient-reported outcome measures (PROMs) [2] are used to assess symptomatic and functional change in shoulder disorders. For example, the shoulder pain and disability index (SPADI) is commonly used in clinical trials, comprising five pain and eight disability questions with severity recorded on ordinal scales [3]. It is reliable, responsive and has good construct validity. While analysis of ordinal PROM scores may provide discriminatory ability between treatments at group level, this may not reflect therapeutic success for the individual. The patient acceptable symptom state (PASS) cut-off is the value on a PROM beyond which the patient feels good enough to continue in that state [4, 5]. Establishing PROM cut-offs for PASS would indicate therapeutic success at the individual level and provide a tool for standardising responder rates in clinical trials.
There are limited data for PASS values for shoulder outcomes [6] with existing studies assessing the cross-sectional relationship between PASS and PROMs, including the American shoulder and elbow surgeons (ASES) score, the simple shoulder test (SST), and the Visual Analog Scale (VAS) pain score [6, 7], with only one assessing SPADI in a shoulder pain cohort awaiting surgery [8].
There are no prospective longitudinal studies investigating SPADI PASS cut-offs. We aimed to determine PASS cut-offs using two different methods and assess whether these values were influenced by time or other covariates.
Materials and methods
Study population
Patients care with shoulder symptoms are initially managed in the community, but if they continue to have problems, they are referred for a diagnostic ultrasound scan. Patients attending a hospital, outpatients’ clinic, for their first shoulder scan were recruited from 2016 to 2017. Inclusion criteria were: age > 18 years, ability to provide informed consent, first ever ultrasound scan of shoulder (determined by clinical records). Exclusion criteria were: inflammatory arthritis; previous fracture/dislocation/surgery of the affected shoulder; steroid injection or physiotherapy for target shoulder within prior 6 weeks (to exclude effects of treatment, which are short term [9], on baseline symptoms); or complex regional pain syndrome. To avoid recruitment bias, quota sampling was used in four categories: gender (male/female) and age (younger/older split at the median); target proportions of these categories were determined from a previous study [10]: males aged < 54 (25%), males aged ≥ 54 (22%), females aged < 54 (25%), females aged ≥ 54 (28%).
Data collection
Patients completed standardised questionnaires at baseline and 6 months, assessing age (years), gender, body mass index (BMI), duration of symptoms, pain status (yes/no), other painful body joints, hospital anxiety and depression scale (HADS) [11] and SPADI. Treatments received (physiotherapy, corticosteroid injections and/or surgery) prior to initial scan were recorded at baseline and treatments received during the study were collected at 6 months. Ultrasound findings were collected to understand the aetiology of shoulder pain.
To determine PASS, patients were asked “in the next few months, if you were to remain as you were during the last 48 h, would this be acceptable or unacceptable to you?” This is the wording recommended by an Outcome Measures in Rheumatology (OMERACT) special interest group [12] and dichotomises each participant into PASS positive (PASSpos) or negative (PASSneg).
Statistical analysis
SPADI scores were transformed to interval scaling by assessing fit to the Rasch measurement model in RUMM 2030 software [13, 14].This process formally tests that the data meet minimum requirements for measurement then allows transformation from ordinal to interval linear scaling [15, 16], facilitating parametric analysis [17]. Comparisons were made between included and excluded patients using Student’s t, Mann–Whitney U or Chi-square tests according to data type and distribution; visual inspection of approximate fit to the normal distribution was performed for continuous variables. PASS cut-offs were calculated using the 75th percentile of the cumulative percentage curve of patients who were PASSpos [5] and the point on the receiver operating characteristic (ROC) curve, which maximised the Youden index [18, 19]. The diagnostic accuracy of each method was reported. We simulated some data to investigate potential differences between the 75th percentile and ROC methods (see Online Resources 3 for more details).
The association between PASSpos/PASSneg and SPADI was assessed using binary logistic regression. An interaction was added between baseline and 6 month SPADI to evaluate if 6 month PASS cut-off depended on baseline score. Interactions with other baseline covariates were also explored; main effects and interactions with 6 month SPADI scores were added for age, gender, baseline painful joint count and baseline HADS scores. If an interaction was not statistically significant, both the interaction and the main effect were removed. To illustrate the influence of baseline score on PASS at 6 months, cut-offs were calculated separately within tertiles of baseline SPADI. SPADI reliability was evaluated using standard error of measurement (SEM) and smallest detectable change (SDC) calculated using Cronbach’s alpha.
Results
SPADI showed acceptable fit to the Rasch model (Online Resource 1). To create a unidimensional total score combining the pain and disability subscales, 7% of the unique variance was lost in a bi-factor equivalent solution. There was no evidence of substantive differential item functioning by age, sex, shoulder side or visit, the latter indicating that the measurement scale remained invariant over time.
Table 1 shows baseline characteristics of patients included in or excluded from the analysis. Questionnaires were returned by 336/500 participants (67%) at 6 months; 304/500 participants (60%) had full data at both time points. Patients excluded from longitudinal analysis were younger with higher HADS, but other covariates were similar. Bursitis was the most common finding on ultrasound at baseline (71%), followed by impingement (63%), acromio-clavicular joint degeneration (53%) and rotator cuff tendinopathy (34%). Amongst the patients included in the analysis, at baseline 69/304 (23%) were PASSpos. At 6 months, 193/304 (63%) were PASSpos; mean ± SD 6 month SPADI scores were total 39.6 ± 19.1, pain 36.5 ± 20.0, disability 33.6 ± 20.3.
Reliability results for SPADI scores are presented in Table S1, Online Resource 2. Most of those reporting improvement > SDC were PASSpos at 6 months: SPADI total 95/106 (90%), pain 100/119 (84%) and disability 113/130 (87%). At baseline the PASS cut-offs for total SPADI were 49.8 and 46.9 using the 75th percentile and ROC method, respectively (Table 2). At 6 months, PASS cut-offs did not differ from baseline to an extent that exceeded measurement error.
The association between 6 month SPADI and PASS did not vary by age, gender, painful joint count or HADS scores (all interactions p > 0.05, data not shown), but at higher baseline SPADI, the coefficient for the association between 6 month SPADI and PASS was larger (p = 0.036; Fig. 1a). This was driven by disability (Fig. 1b), rather than pain (Fig. 1c). Patients with more severe disability at baseline were willing to accept a comparatively greater degree of disability at 6 months. In contrast, there was an upper limit of pain that was acceptable at 6 months, irrespective of pain at baseline.
When baseline SPADI scales were divided into tertiles, the more symptomatic a patient was at baseline (higher score), the higher their SPADI PASS cut-off was at 6 months (Fig. 2). Differences in pain cut-off did not exceed SDD; however, the effect was more pronounced (> SDD) for disability. On the native (ordinal) scale, differences in cut-off were greater than on the Rasch-transformed interval scale, emphasising the importance of using interval scaling (Table S2, Figure S1, Online Resource 2).
Discussion
This is the first study to evaluate the PASS threshold for SPADI longitudinally in a community-based cohort. We found that PASS cut-offs varied between the ROC and 75th percentile methods. PASS cut-offs did not change substantively over time. However, the PASS cut-off on the total SPADI score at 6 months was dependent on the baseline score; this was primarily driven by the disability subscale, rather than the pain subscale.
Our SPADI PASS cut-offs differed from the only other study assessing PASS and SPADI, although that cross-sectional cohort recruited post-surgical patients [10]. Our cut-offs also varied according to the methods used; the optimum methods for calculating PASS are unknown and may depend on the purpose [18]. The 75th percentile method may be inappropriate for determining PASS changes longitudinally, because the scores within those reporting PASS may decrease, affecting the location of the 75th percentile. PASS provides useful information on therapeutic success at the individual level, but does not reflect the desire to improve: patients may define their state as acceptable, but want to feel better. Given the negative and positive predictive values from our study (Table 2), the SPADI PASS cut-off may be more useful as a descriptive outcome for trials rather than a treatment target.
Age and gender did not modify the association between 6 month SPADI and 6 month PASS after adjusting for baseline score; however, these tests may have lacked power as testing for these interactions would require higher number of participants. Previous studies using different PROMs found age and gender affected PASS cut-off, but did not test for interactions or adjust for other covariates or baseline score [20, 21].
Disease adaptation may alter interpretation of a questionnaire, thus changing a patient’s individual PASS threshold over time. This response shift can only be assessed if the measurement properties of the questionnaire itself are time invariant, which we confirmed for SPADI. Patients with worse symptoms at baseline reported higher PASS cut-offs at 6 months. Existing research, using different scales, found that those with worse initial activity limitation in acute rotator cuff syndrome were shown to have higher PASS cut-offs at follow-up, possibly due to adaptation, whilst their pain cut-offs did not differ by baseline score, consistent with our findings [22]. Another study reported data consistent with this trend in those undergoing hip replacement [23]. Studies have found PASS cut-offs for other scales to be stable over time [20, 24], whereas another found thresholds declined [21]. PASS cut-off was reported to be stable after 6 months from knee replacement [23]. This variation in findings may be due to treatment expectation or PASS thresholds may be disease or treatment specific. Another source of variation might be the use of the 75th percentile method to identify changes in PASS cut-off over time, which, as we have shown in Online Resource 3, can give variable results depending on the level of change in symptoms, even if there is actually no change in ‘true’ PASS cut-off.
We did not adjust our analysis for treatments received between baseline and follow-up, because we needed to account for the total effect of baseline SPADI on the association between SPADI and PASS at 6 months. The level of baseline symptoms would potentially influence treatments received post-baseline; therefore, including such treatments in the model may distort this association.
This paper has several strengths. While previous studies used different wordings and time anchors for PASS [7, 10] we used the timeframe of “next few months” recommended by OMERACT [12]. Standardisation of the wording is important for comparison across studies and variations may influence results [18]. SPADI underwent Rasch transformation enabling inferential testing by converting ordinal scores to interval scaling and our measures showed very good reliability. Other studies did not transform their data; therefore, they may have underestimated or overestimated the extent of cut-off dependence on initial values due to the nonlinearity of estimates obtained from these PROMs.
There are limitations to this study. The 6 months response rate was 67%. However, this is better than a published overall rate for 1607 surveys of 48.3% [25]. Our non-responders were on average younger with more anxiety and depression, consistent with published predictors of response to surveys [26]. However, the baseline SPADI and PASS were similar between responders and non-responders. Pain duration, BMI, injections received and sex were also similar between the two groups. This study involved self-completion of questionnaires, which may introduce recall bias.
This study has shown for the first time that the PASS cut-off for SPADI varies depending on calculation method and symptom severity at baseline. Understanding the PASS threshold is important for clinical research as the relationship between SPADI and objective disease parameters is ill-defined. Further work in different centres is required to substantiate these findings.
Availability of data and materials
The data that support the findings of this study are available from the corresponding author upon reasonable request.
References
Artus M, van der Windt DA, Afolabi EK, Buchbinder R, Chesterton LS, Hall A et al (2017) Management of shoulder pain by UK general practitioners (GPs): a national survey. BMJ Open 7(6):e015711
Weldring T, Smith SMS (2013) Patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs). Health Serv Insights 6:61–68
Roach KE, Budiman-Mak E, Songsiridej N, Lertratanakul Y (1991) Development of a shoulder pain and disability index. Arthritis Care Res 4:143–149
Tubach F, Ravaud P, Baron G, Falissard B, Logeart I, Bellamy N et al (2005) Evaluation of clinically relevant states in patient reported outcomes in knee and hip osteoarthritis: the patient acceptable symptom state. Ann Rheum Dis 64:34–37
Kvien TK, Heiberg T, Hagen KB (2007) Minimal clinically important improvement/difference (MCII/MCID) and patient acceptable symptom state (PASS): what do these concepts mean? Ann Rheum Dis 66(Suppl 3):iii40–iii41
Tashjian RZ, Deloach J, Porucznik CA, Powell AP (2009) Minimal clinically important differences (MCID) and patient acceptable symptomatic state (PASS) for visual analog scales (VAS) measuring pain in patients treated for rotator cuff disease. J Shoulder Elb Surg 18:927–932
Chamberlain AM, Hung M, Chen W, Keener JD, McAllister J, Ebersole G et al (2017) Determining the patient acceptable symptomatic state for the ASES SST, and VAS pain after total shoulder arthroplasty. J Shoulder Elbow Arthroplasty. https://doi.org/10.1177/2471549217720042
Christie A, Dagfinrud H, Garratt AM, Ringen Osnes H, Hagen KB (2011) Identification of shoulder-specific patient acceptable symptom state in patients with rheumatic diseases undergoing shoulder surgery. J Hand Ther 24:53–61
van der Windt DA, Koes BW, Devillé W, Boeke AJ, de Jong BA, Bouter LM (1998) Effectiveness of corticosteroid injections versus physiotherapy for treatment of painful stiff shoulder in primary care: randomised trial. BMJ 317(7168):1292–1296
Tran G, Hensor EM, Ray A, Kingsbury SR, O'Connor P, Conaghan PG (2017) Ultrasound-detected pathologies cluster into groups with different clinical outcomes: data from 3000 community referrals for shoulder pain. Arthritis Res Ther 19:30
Zigmond AS, Snaith RP (1983) The hospital anxiety and depression scale. Acta Psychiatr Scand 67(6):361–370
Tubach F, Ravaud P, Beaton D, Boers M, Bombardier C, Felson DT et al (2007) Minimal clinically important improvement and patient acceptable symptom state for subjective outcome measures in rheumatic disorders. J Rheumatol 34:1188–1193
Rasch G (1980) Probabilistic models for some intelligence and attainment test. Expanded edition with foreword and afterword by B.D. Wright. The University of Chicago Press, Chicago
Andrich D, Sheridan B, Luo G (2015) RUMM2030: An MS Windows computer program for the analysis of data according to Rasch unidimensional models for measurement. RUMM Laboratory, Perth
Rasch G (1960) Studies in mathematical psychology: I. Probabilistic models for some intelligence and attainment tests. Nielsen & Lydiche, Oxford, England
Tennant A, Conaghan PG (2007) The Rasch measurement model in rheumatology: what is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis Rheum 57:1358–1362
Svensson E (2001) Guidelines to statistical evaluation of data from rating scales and questionnaires. J Rehabil Med 33:47–48
Tubach F, Wells GA, Ravaud P, Dougados M (2005) Minimal clinically important difference, low disease activity state, and patient acceptable symptom state: methodological issues. J Rheumatol 32:2025–2029
Maksymowych WP, Richardson R, Mallon C, van der Heijde D, Boonen A (2007) Evaluation and validation of the patient acceptable symptom state (PASS) in patients with ankylosing spondylitis. Arthritis Rheum 57:133–139
Heiberg T, Kvien TK, Mowinckel P, Aletaha D, Smolen JS, Hagen KB (2008) Identification of disease activity and health status cut-off points for the symptom state acceptable to patients with rheumatoid arthritis. Ann Rheum Dis 67:967–971
Maksymowych WP, Gooch K, Dougados M, Wong RL, Chen N, Kupper H et al (2010) Thresholds of patient-reported outcomes that define the patient acceptable symptom state in ankylosing spondylitis vary over time and by treatment and patient characteristics. Arthritis Care Res 62:826–834
Tubach F, Dougados M, Falissard B, Baron G, Logeart I, Ravaud P (2006) Feeling good rather than feeling better matters more to patients. Arthritis Rheum 55:526–530
Escobar A, Gonzalez M, Quintana JM, Vrotsou K, Bilbao A, Herrera-Espineira C et al (2012) Patient acceptable symptom state and OMERACT-OARSI set of responder criteria in joint replacement. Identification of cut-off values. Osteoarthr Cartil 20:87–92
Tubach F, Pham T, Skomsvoll JF, Mikkelsen K, Bjorneboe O, Ravaud P et al (2006) Stability of the patient acceptable symptomatic state over time in outcome criteria in ankylosing spondylitis. Arthritis Rheum 55:960–963
Baruch Y, Holtom BC (2008) Survey response rate levels and trends in organizational research. Hum Relat 61(8):1139–1160
Koloski NA, Jones M, Eslick G, Talley NJ (2013) Predictors of response rates to a long term follow-up mail out survey. PLoS ONE 8:e79179
Acknowledgements
This article presents independent research funded by the National Institute for Health Research (NIHR) Leeds Biomedical Research Centre (BRC) and by a NIHR Doctoral Research Fellowship (GT; DRF-2016-09-159). The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care. This study was also part-funded through the Arthritis Research UK Leeds Experimental Osteoarthritis Treatment Centre (20083).
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by GT, BD and EMAH. The first draft of the manuscript was written by GT and BD and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare they have no conflicts of interest.
Ethical approval
Ethics approval was obtained from North East-Newcastle and North Tyneside 2 Research Ethics Committee (ethics approval protocol number: 16/NE/0108) and informed consent was obtained from all participants for publication.
Human and animal rights
All human studies have been approved by the appropriate ethics committee and have therefore been performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Tran, G., Dube, B., Kingsbury, S.R. et al. Investigating the patient acceptable symptom state cut-offs: longitudinal data from a community cohort using the shoulder pain and disability index. Rheumatol Int 40, 599–605 (2020). https://doi.org/10.1007/s00296-019-04486-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00296-019-04486-3