Translation procedure
The validated English ATRS was translated into the Dutch language according to the guidelines of cross-cultural adaptation [5]. Forward translations were performed by two native English speakers who fluently spoke Dutch, and the backward translation was performed by two Dutch native speakers who fluently spoke English. A diverse group of ten volunteers checked for clarity of the wording and meaning of the questions. If there were discrepancies, this was dissolved with discussion.
Reliability and validity evaluation
Patients
Patients treated for a total Achilles tendon rupture from 1 January 2012 to 31 December 2014 in the Academic Medical Center or Onze Lieve Vrouwe Gasthuis were recruited. The eligibility criteria were: age above 18 years, isolated unilateral Achilles tendon rupture without other serious lower limb injury and the ability to read, write and understand Dutch. Patients were invited by mail. To optimize the response rate, patients were given the choice to fill in the questionnaire sent to them by mail or by email. Two weeks after completing the first questionnaire, patients again received a questionnaire by mail or by email and were asked to complete the ATRS questionnaire second time. Only complete ATRS questionnaires were included in the analysis.
Outcome measures
All questionnaires contained the in Dutch translated version of the ATRS, a validated Dutch FAOS, VISA-A and Numeric Rating Scale for Pain in rest and during activity (NRS). For the second questionnaire, one anchor question was added, to determine whether the status of the Achilles tendon complaints had changed. We used the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist for validation of PROMs developed by the COSMIN Initiative [11, 12, 19].
The Achilles tendon Total rupture score (ATRS) is a patient-reported instrument and disease-specific tool designed to evaluate symptoms and physical activity in patients with Achilles tendon rupture. The ATRS is a self-administered instrument and contains ten items, each an 11-points Likert scale (0–10). A maximal score of 100 indicates no symptoms and full function, whereas a minimum score of 0 indicates severe symptoms and major limitations [13]. However, like the English version, the numbers were changed just to not make the patients confused, since they are used to ten being the worst, meaning that a minimum score of 0 indicates no symptoms and full function, whereas a maximum score of 100 indicates severe symptoms and major limitations.
The Foot and Ankle Outcome Score (FAOS) is a 42-items self-administered questionnaire originally designed to evaluate patients with ankle ligament injuries [15]. The FAOS has thus far been used in patients with lateral ankle instability, plantar fasciitis and Achilles tendon rupture, but was not specifically developed for Achilles tendon pathologies. The FAOS consists of five subscales: pain, other symptoms, activity in daily living (ADL), recreational and sport activities and foot and ankle-related quality of life (QOL). In 2014, the FAOS was validated for the Dutch language [16]. Each question in the FAOS is answered on a five-point Likert scale ranging from 0 to 4. A normalized score (100 indicating no symptoms and 0 indicating extreme symptoms) was calculated for each subscale. If there were one or two answers missing in the questionnaire, it was allowed to substitute the missing value by the mean value of the subscale [15].
The Victorian Institute of Sports Assessment-Achilles questionnaire (VISA-A-NL) is a PROM consisting of eight questions, validated for evaluation of pain, symptoms and their effect on physical activity specifically in patients with Achilles tendinopathy [20]. The score ranges from 0 to 100, where 0 represent the worst score and 100 the best score.
The Numeric Rating Scale for pain (NRS) is a common and practical method for assessing pain severity in rest and during activity such as running. In this study, a 11-point numeric rating scale was used, where patients are requested to quantify the intensity of their pain on a scale from 0 to 10 with 0 indicating no pain and 10 indicating the worst pain imaginable [7]. We assessed the NRS in rest and during running.
Reliability
In this study, the reliability was assessed by the internal consistency, the reproducibility (test–retest) and the measurement error [12].
Internal consistency was defined as the degree of the interrelatedness among the items of the ATRS [12]. To measure interrelatedness among items, we used the Cronbach’s alpha with 0.7–0.95 assigned as good interrelatedness [18].
The test–retest reliability was defined as the ability of the Dutch ATRS to measure the same outcome twice in patients with an unchanged state of condition of the complaints [12]. Patients who reported a change in their state were excluded from the test–retest analysis. We assessed the test–retest reliability by calculation of the intraclass correlation coefficients (ICC-agreement, type 3 two-way mixed model) [3]. An ICC of 0 indicated there is no agreement between the two questionnaires, and an ICC of 1 means there was a perfect agreement. An ICC > 0.7 was considered as good agreement [23].
Measurement error was defined as the systematic and random error of a patient’s score that is not attributed to true changes in the construct of the ATRS [12]. Measurement error was calculated as the standard error of measurement (SEM) which was calculated as standard deviation (SD) × √(1 − reliability coefficient) [23]. From this SEM, the minimal detectable change (MDC) at individual level was calculated as 1.96 × √2 × SEM, and the MDC at group level was calculated by dividing the MDC at individual level by √n [23].
Construct validity
Construct validity was defined as the degree to which the scores of the Dutch ATRS were consistent with the hypotheses stated below [12]. Due to the lack of a ‘golden standard’, the construct validity of the Dutch ATRS was assessed in terms of consistency to the subscales of the FAOS, VISA-A-NL and NRS in rest and during running [12]. The ATRS was compared with the FAOS subscales, VISA-A-NL and NRS in rest and during running by analysing means of Spearman’s correlation coefficients. Correlation coefficients between 0.4 and 0.7 (or between −0.4 and −0.7) were defined as a moderate correlation, coefficients lower than 0.4 or higher than −0.4 unconnected or measuring dissimilar constructs and coefficients above 0.7 or lower than −0.7 as strong correlation [18].
A priori hypotheses on correlation between the Dutch ATRS and the Dutch subscales of FAOS, VISA-A-NL and NRS were formulated to evaluate the construct validity. It was hypothesized that the Dutch ATRS would correlate strongly with the VISA-A-NL, the Dutch FAOS symptoms, FAOS function, FAOS pain, FAOS ADL and NRS during running, because the ATRS is a disease-specific tool designed to evaluate symptoms and physical activity and should measure similar construct. Therefore, it was hypothesized that the Dutch ATRS would correlate moderately with the FAOS QOL and NRS in rest since the QOL domain is not disease specific and the NRS in rest is not comparable to a status of activity. The construct validity was defined sufficient if at least 75 % of the results were in correspondence with these hypotheses [18].
Interpretability
Interpretability was defined as the degree to which qualitative meaning can be assigned to the Dutch ATRS quantitative scores or changes in scores [12]. Interpretability was assessed by the distribution and occurrence of ceiling and floor effects. Floor or ceiling effects were considered to be present if more than 15 % of respondents achieved the lowest or highest possible score.
Statistical analysis
Variables with a normal distribution were presented as the mean and standard deviation. Variables with a non-normal distribution were presented as the median and interquartile range, and the Kolmogorov–Smirnov test was used for data distribution assessment. Clinimetric properties were calculated as described above. All statistical analyses were performed with Statistical Package for Social Sciences (SPSS) version 22.0 (SPSS Inc. Chicago, IL).