Introduction

The estimated global incidence of spinal cord injury (SCI) is between 40 and 80 persons per million per annum. The ratio between traumatic and non-traumatic SCI varies across studies and regions, and the most frequent etiologies are transport accidents and falls, as well as neoplastic tumors and degenerative conditions of the spinal column [1]. SCI is a life-disrupting condition, with a long-term impact on individuals’ physical function [2], mental health, life satisfaction [3], and financial situation [4]. A number of outcome measures are available to evaluate the function and health of people with SCI, e.g., Spinal Cord Independence Measure and Spinal Cord Injury-Secondary Conditions Scale [5, 6]. However, none of those cover the whole spectrum of functional status, secondary problems, mood, and social consequences with a single scale, and did not reflect a comparison of current functioning with the pre-injury situation.

The AO Spine Knowledge Forum Trauma initiated a project to develop an instrument for the entire spine trauma population. Based on the Core Set development methodology of the International Classification of Functioning, Disability, and Health (ICF) of the World Health Organization, the Patient Reported Outcome Spine Trauma (AO Spine PROST) was developed [7, 8]. Studies have shown that the AO Spine PROST is reliable and valid among people with spine trauma, with no, transient or mild neurology (American Spinal Injury Association Impairment Scale (AIS) C, D, and E), showing an excellent internal consistency and concurrent validity [9,10,11]. However, data on its applicability for use in people sustaining AIS A and B and non-traumatic causes of injury are unavailable to date. Therefore, the purpose of this study was to examine the applicability of the AO Spine PROST in people with motor-complete traumatic or non-traumatic SCI, by investigating its comprehensibility, non-offensiveness, relevance, and completeness from the patients’ perspective. Specifically, an advice will be given about any adjustments to the AO Spine PROST. In accordance with a pilot study performed in the development phase [9], we hypothesized that no major changes to the tool would be required.

Methods

Study design and setting

The present study is part of a multistep approach to extend the use of AO Spine PROST to people living with motor-complete SCI. Cognitive interviews (CIs) were conducted.

Participants

People were eligible for enrollment if they: (1) were diagnosed with a traumatic or non-traumatic SCI; (2) had an AIS score A or B at the time of admission to the rehabilitation center; (3) were within 2 years post-onset of their injury; (4) ≥18 years of age, (5) able to provide informed consent; and (6) able to complete the AO Spine PROST questionnaire in Dutch. People were excluded if they had cognitive deficits; were not able to read and communicate in Dutch; or people with SCI with a slow-progressive pattern of onset (>1 month). The sample size was determined by identifying groups of people most relevant to this study [12]. The aim was to include 15 participants with traumatic and 15 with non-traumatic etiology.

Study procedures

The study was conducted during the period June 2020 to May 2021. Participants were recruited from two rehabilitation centers in the Netherlands: De Hoogstraat Rehabilitation (Utrecht) and Sint Maartenskliniek (Nijmegen). Eligible former rehabilitants were identified by screening the medical records. All eligible participants received a letter to explain the rationale, goals, and methodology of the study and were invited to participate. After written informed consent, the AO Spine PROST was filled out in a CI, by thinking aloud and verbal probing. CI is an established method for the identification and correction of problems with survey items, through the evaluation of participants’ interpretation and comprehension. “Think-aloud” minimizes the interviewer’s role in eliciting feedback by asking participants to vocalize their thought processes while answering each survey question. “Verbal probing” uses selected probing questions to focus data collection on the relevant issues [12,13,14]. The interviews were performed either using web-based technologies or at the outpatient clinic. The study was performed according to the principles of the Declaration of Helsinki of the World Medical Association. The study received approval from the local Research Ethics Committees of both institutions.

Instruments

Each CI began with an introduction of the study and a few questions on participant characteristics, e.g., age, cause of SCI, and affected body parts. Thereafter, the participant was asked to complete the AO Spine PROST (Supplementary File 1), consisting of 19 questions on aspects of functioning, e.g., household activities, changing posture, neck pain, and bowel function. Each question can be answered on a 0–100 numeric rating scale, with 0 indicating no function at all and 100 indicating functioning at the same level as before the accident, no matter how high or low this prior level was. The total score thereby reflects the individual’s experienced change in the level of functioning as compared to their situation before the onset of the SCI. The questions were shown one by one and the participant was asked to read the question aloud and to score the question while thinking aloud. Probed questions for each item were: (1) elaborations on how they constructed their answer, (2) difficulties in answering, and (3) other aspects which influenced their answer [12]. After completion of the AO Spine PROST, four evaluative questions were presented which could be answered on a 5-point Likert scale (ranging from 1 (Totally disagree) to 5 (Totally agree)): (1) “All questions are easy to understand without further explanation”, (2) “There are no questions that I experience as obtrusive or that I would prefer not to answer”, (3) “All questions are relevant to my current condition/life situation”, and (4) “Together, these questions provide all information necessary to understand how I experience my life with the SCI”. At the end, participants could indicate any additional comments. The average duration of the CI was 40 min (range 17–80 min).

Analyses

The four statements about the AO Spine PROST (comprehensibility, non-offensiveness, relevance, and completeness) were analyzed by computing frequency tables using SPSS. The participants were expected to agree/strongly agree with each statement. Mann–Whitney tests were performed to compare these scores in the subgroups of traumatic and non-traumatic SCI. During the CI, observational notes were made. The interviews were recorded and transcribed. Qualitative data analysis software was used (MaxQDA (2018)) to organize, manage and analyze the data. Theoretical thematic analysis was performed, which led to a detailed analysis of the predefined statements [15]. Therefore, a coding scheme was developed based on the 19 questions and the 4 statements about the AO Spine PROST [16]. Two authors (AJH, SD) first read the interview transcripts to gain an overall sense of the content and quality of the interview data. After that these authors identified and linked codes in the data to the coding schema. Consensus was reached by repeated discussions among four authors (AJH, TvD, MWMP, SS). These qualitative results were used to validate the results of the four statements.

Results

Participant characteristics

A total of 51 people were invited to participate, of which 29 (56.9%) provided informed consent. Characteristics of the participants are presented in Table 1. The majority had a traumatic SCI (71.4%), mostly due to a (high) fall. In all participants, bowel, bladder, and sexual functions were affected. The mean (SD) total score of the AO Spine PROST was 48.9 (12.16). The lowest mean (SD) score was provided for Q5 (walking, 12.2 (26.8)), followed by Q11 (urinating, 19.0 (19.0)) and Q12 (bowel movement, 21.2 (19.5)). The highest mean (SD) scores were seen for Q16 (sleep, 86.8 (15.9)), Q4 (social life, 85.0 (23.6)), and Q14 (emotional function, 83.9 (22.6)). A full overview of scores is presented in Supplementary File 2.

Table 1 Patient characteristics.
Full size table

Evaluation questions

The participants generally agreed with each evaluation question concerning comprehensiveness, non-offensiveness, relevance, and completeness. No statistical differences were found between the traumatic and non-traumatic groups (Table 2). In the following subheadings results are explained, overall, and per question.

Table 2 Evaluation questions AO Spine PROST.
Full size table

Comprehensiveness

Regards many questions, participants commented that their current functioning was completely different compared to before the SCI, making it difficult to compare the two. During the interviews, it became clear that participants had difficulties answering some questions. Some participants answered Q7 (changing posture) in a more general way, about posture, and only after they reached Q8 (maintaining posture) they realized that Q7 was about changing their posture and Q8 about maintaining their posture. Notable is further that most participants referred to the examples laying down and sitting, and not to standing. In Q10 (personal care) some participants interpreted the example “using the toilet” as referring to the anatomical function of the bladder/bowel evacuation: “Quite a turnaround compared to the past because my body no longer works in terms of bowel and bladder function, while it can no longer relieve itself. Especially by defecating and catheterizing 5 times a day… is a very big difference”. In the following Q11 (urinating) and Q12 (bowel movement), they specified this. For some participants, Q18 (loss of strength in your arms and/or legs) was difficult to answer, because participants would have liked to provide separate scores with respect to arms and legs. They mostly solved this by averaging their score. Q19 (back and/or neck pain) was interpreted by some participants as a reflection of pain severity, or they experienced difficulties in how to exclude other types of pain (especially neuropathic pain) in their answers.

The use of an assistive device (e.g., adapted car or cutlery), and how this should weigh in the score was a recurring topic with respect to Q1 (household activities), Q6 (travel), Q7 (changing posture), Q10 (personal care), and Q11 (urinating). An example of a comment on Q11: “As stated in the question the score would be 0, because I do not have any bladder function anymore, but with a catheter I have to say I feel my bladder function is recovered to 80–90 percent”.

The examples mentioned in brackets after the questions were often used to answer the questions. For example, in Q3 (recreation and leisure) participants started their answer by telling about “hobbies or sports” instead of the broader “recreation and leisure”. Some participants suggested adding some examples to Q13 (sexual function) to clarify this concept. They proposed various concepts, e.g., fertility, orgasm, or cuddling.

Furthermore, some participants flagged the combination of examples that they would score differently within one question. This raised doubt about how to score these items, particularly items Q2 (work/study), Q3 (recreation and leisure), Q6 (travel), Q8 (maintaining posture), and Q14 (emotional function). Again, participants mostly solved this by averaging their scores. A participant answered on Q6 (travel) the following: “…Self driving, transport… this one is very varied, I find it difficult to express in numbers, because there are different things in this question”.

Relevance

Most questions were rated as relevant. Only Q5 (walking) was indicated as not relevant for their own current situation. Walking (Q5) was not seen as comparable with the use of a wheelchair and, therefore, not included in the answer: “and of course, with a wheelchair… I can go to other places, but it isn’t walking, absolutely not”. Participants noted that the introduction and rating scale suggested that everyone has had an accident.

Non-offensiveness

Participants rated the AO Spine PROST as not offensive (4.6 out of 5.0). The item on sexual function (Q13) was experienced as offensive by one participant in the current interview setting, but this person clarified that it would not be experienced as offensive in a self-report questionnaire.

Completeness

The lowest score was on completeness (3.6). Some participants indicated a lack of items on emotional impact of SCI (n = 3), care for children (n = 2), arranging support, devices etcetera (n = 2), and societal participation (n = 1). However, most participants could not indicate what they were missing.

Participants made some additions per question. Q17 (stiffness of your back and/or neck) did not appear to be complete for the disability in overall performance that participants experienced, as this could also occur in other body parts. For others, it was not clear whether spasms were included in this question. For Q18 (loss of strength in your arms and/or legs), some stated that loss of strength can occur in other body parts (hands or trunk). A participant described: “My arms function well, while my legs don’t, that would make 50 percent for me. But I have limitations in my hands. I can train my arms with dumbbells, however, due to not being able to use my hands, I’m still limited in functioning”. For Q19 (back and/or neck pain), participants indicated experiencing other types of pain, sometimes with more impact on overall performance (neuropathic pain, visceral pain, or spasms). A participant motivated this as follows: “The pain I experience, that is actually that specific nerve pain, especially when sitting. Behind my back… and while sitting, I just feel pain everywhere. Sometimes I can’t even turn around in bed, it hurts that much. But I don’t have any other pain”. Finally, some participants commented that there were areas in which they functioned even better than before the accident, but that this could not be expressed in the score. This was the case for Q4 (social life), Q8 (maintaining posture), and Q14 (emotional function).

Discussion

This study investigated the applicability of the AO Spine PROST in people with motor-complete traumatic or non-traumatic SCI. Despite comments made on specific items of the AO Spine PROST, in general, participants expressed positive opinions on the tool and found the items to be well understood. This was confirmed by their generally positive ratings of this tool as comprehensive, not offensive, relevant, and largely complete. However, the results do show that there are opportunities to make the AO Spine PROST more suitable for people with motor-complete or non-traumatic SCI.

The study sample is comparable with the SCI population worldwide, with respect to gender, age, cause, and level of injury [17,18,19]. Half of the participants rated their control of movement below the level of SCI as partial due to little arm or hand function, or trunk stability.

Most participants scored the questions of the AO Spine PROST as relevant, however, comments were made about Q5 (walking). However, this item was nevertheless found relevant, because walking affects daily functioning and this function may have been altered by the SCI. The lowest score of the evaluation questions was for completeness. Participants experienced in the questionnaire more focus on the physical domain, compared to the mental and social domain. The importance of those domains for quality of life measurements, more than health and physical function, is also described in a study on cancer patients [20]. Despite the presence of an item on emotional function, some participants indicated that they missed a more specific item about the emotional impact of SCI. The best rating was given for the non-offensiveness of the AO Spine PROST. However, this study was conducted in a western country, where gender equality and flexible social norms are pursued. Therefore this feature of the questionnaire can therefore be assessed differently in other cultures [21]. Future studies should examine the cross-cultural validity of the AO Spine PROST worldwide. To make the AO Spine PROST even more suitable for the entire SCI population, some minor adjustments to the instructions can be made in order to: do justice to the possibility of functioning better as before; include the use of assistive devices; make the instructions more appropriate for people with non-traumatic SCI. It also should be considered to indicate more clearly that the score is based on the situation or example where the patient is most disabled. Further, some adjustments can be made to the questions and examples. Some participants missed the item “care for children”. According to the ICF this subject belongs to the first question (household activities) and it is recommended to add it as an example. To clarify the distinction between Q7 (changing posture) and Q8 (maintaining posture), different examples could be given for both questions, in accordance with the description of the ICF [8]. To prevent interpreting the example in Q10 “using the toilet” as the anatomical function, it would be good to remove this example. To clarify the concept “sexual function” in Q13, the examples “sexual activities or fertility” could be added, based on the opinion of participants in this study. Because stiffness can also occur in other limbs and affect function, “neck and back” could be deleted in Q17 in order to make this question more broadly applicable. It is expected that with this change, people will include hindrance or burden from spasms in their answer, if applicable. To emphasize overall performance, rather than the degree of loss of strength, and the distinction between lower and upper extremities, Q18 (loss of strength in your arms and/or legs) could be changed into “loss of strength”. Loss of strength in other body parts would thereby also be included. The same applies to Q19 (back and/or neck pain): by deleting the words “back and/or neck” the question would be more broadly applicable, so that the answer refers to overall performance and not just to the degree of pain. All other pain types would also be included.

In a previous validation study of the Dutch version of the AO Spine PROST among people with AIS C, D and E, there were some difficulties with the question “work/study”, which was not reflected in the results of the current study [10]. This difference can be explained by the higher age of the participants in the previous study (52.5 years (range 20–75)) compared to the current study (42.9 years (range 20–67)), resulting in more of the participants who were (early) retired in the previous study. Besides, the option to skip this question in case of no work or study was added. Another different finding in the current study is that many participants indicated the importance of assistive devices in their lives, which had a major influence on their independence and quality of life. This can be explained by the fact that the current study included participants with a motor-complete SCI, who make more use of assistive devices [1].

The results of the current study support the usefulness of the AO Spine PROST, and also provide clues for further refinement of this tool. With the inclusion of items on common secondary conditions of SCI (including bladder and bowel function), the tool can be used as an SCI-specific global measure of function that covers a wide range of life domains within a single scale. Also, a global measure of the function is reflected with the incorporated items. Using a subjective rating scale for participants’ experience about their functioning could help professionals to better understand their patients [22]. On the other hand, there may also be shortcomings of summed scores, such as that it ignores differences in task difficulty and the possible multifaceted nature of the measured abilities [23, 24]. The internal consistency of the AO Spine PROST total score has been shown to be excellent (Cronbach’s alpha 0.96 and 0.97) [10, 11]. Nevertheless, there is a need to further investigate the dimensionality of the instrument using more advanced psychometric methods, such as item-response theory or confirmative factor analysis.

There are several limitations of this study. The study group consisted of a relatively small group of persons with a non-traumatic SCI. We found no indications that they had different issues with the questionnaire as compared to those with traumatic SCI, but because of this small group, we might have missed such issues. Second, due to the COVID-19 pandemic, most interviews were conducted online. However, studies on the influence of face-to-face compared to web-based interviews proposed that web-based technologies as an alternative to traditional face-to-face interviews yielded similar information [25,26,27]. In our study, two participants were unable to work with web-based technologies, so these interviews were conducted by telephone. Irvine et al. found that interviews by telephone were mostly shorter in comparison with face-to-face interviews, and participants had less speaking time in comparison with the interviewer [28]. Finally, also participants with cauda lesions were included. As a result, there are four participants who were ambulatory despite having a motor-complete lesion. However, including them was useful because they could have yielded new interesting findings.

In conclusion, the AO Spine PROST is a comprehensive, not offensive, relevant, and largely complete questionnaire for adults with a motor-complete traumatic or non-traumatic SCI. Several recommendations for improvement in instructions, terminology, and used examples of the AO Spine PROST were proposed. After adjustments and further validation, the tool could be used as an outcome measure in the entire spine injury population.