Advertisement

European Spine Journal

, Volume 26, Issue 11, pp 2782–2788 | Cite as

Which patient-reported factors predict referral to spinal surgery? A cohort study among 4987 chronic low back pain patients

  • Johanna M. van DongenEmail author
  • Miranda L. van Hooff
  • Maarten Spruit
  • Marinus de Kleuver
  • Raymond W. J. G. Ostelo
Open Access
Original Article

Abstract

Purpose

It is unknown which chronic low back pain (CLBP) patients are typically referred to spinal surgery. The present study, therefore, aimed to explore which patient-reported factors are predictive of spinal surgery referral among CLBP patients.

Methods

CLBP patients were consecutively recruited from a Dutch orthopedic hospital specialized in spine care (n = 4987). The outcome of this study was referral to spinal surgery (yes/no), and was assessed using hospital records. Possible predictive factors were assessed using a screening questionnaire. A prediction model was constructed using logistic regression, with backwards selection and p < 0.10 for keeping variables in the model. The model was internally validated and evaluated using discrimination and calibration measures.

Results

Female gender, previous back surgery, high intensity leg pain, somatization, and positive treatment expectations increased the odds of being referred to spinal surgery, while being obese, having comorbidities, pain in the thoracic spine, increased walking distance, and consultation location decreased the odds. The model’s fit was good (X 2 = 10.5; p = 0.23), its discriminative ability was poor (AUC = 0.671), and its explained variance was low (5.5%). A post hoc analysis indicated that consultation location was significantly associated with spinal surgery referral, even after correcting for case-mix variables.

Conclusion

Some patient-reported factors could be identified that are predictive of spinal surgery referral. Although the identified factors are known as common predictive factors of surgery outcome, they could only partly predict spinal surgery referral.

Keywords

Low back pain Neurosurgery Orthopedic surgery Clinical decision-making 

Introduction

Globally, the burden of low back pain (LBP) for society is larger than that of any other medical condition [1]. The clinical course of a LBP episode is generally favorable and most pain and related disabilities resolve within a few weeks. In a substantial group of patients, however, pain and disability are ongoing, resulting in chronic LBP (CLBP) [2]. Even though not life-threatening, CLBP has serious consequences for the patients’ overall wellbeing (e.g., reduced health-related quality of life) and is associated with a high economic burden. To illustrate the latter, the total cost of back pain to Dutch society was estimated to be €3.5 billion [1, 2, 3].

Spinal surgeries represent a sizeable proportion of the total cost of back pain [3], but spinal surgery rates vary extensively across, and even within, countries [4]. These variations in spinal surgery rates seem to indicate a lack of professional consensus. Moreover, as in many fields of healthcare, conclusive evidence for the long-term (cost-) effectiveness of spinal surgeries is lacking, and risk is inherent to any surgical procedure [5, 6, 7]. Therefore, reducing the number of unnecessary spine surgeries is important for improving patient safety, patient outcomes, and healthcare efficiency [7]. Proper patient referral and selection is expected to improve the outcome of spinal surgery [6, 8, 9] but no classification system currently exists that is supported by sufficient evidence to be broadly implemented [8, 10].

Improved insight into the current spine surgery decision-making process for CLBP patients may clarify some of the aforementioned variations in spinal surgery rates [4], and may ultimately support spine surgery decision-making guidelines [6, 11]. Amongst others, it would be insightful to know which CLBP patients are typically referred to spinal surgery, whether patient-reported factors can be identified that are predictive of spinal surgery referral, and if so, to what extent spinal surgery referral is explained by these factors. Nonetheless, even though numerous studies have investigated predictive factors of spine surgery outcome, little research has focused on identifying patient-reported factors that are predictive of referral to spinal surgery [11, 12].

Van Hooff et al. conducted a modified-Delphi study, in which a diverse panel of CLBP experts identified 47 indicators for a successful treatment outcome (e.g., gender, BMI, co-morbidities, anxiety, treatment expectations) [11]. This study provides some insight into the factors that are considered by CLBP experts when deciding whether to refer a patient to spinal surgery. One should bear in mind, however, that this study aimed to establish patient-reported predictive factors of a successful treatment outcome, rather than predictive factors of referral to spinal surgery. Moreover, predictive factors were established through a formal consensus procedure, and not derived from empirical patient referral data. Using a large set of spine registry patient referral data, the current study, therefore, aimed to explore what patient-reported factors are predictive of referral to spinal surgery among CLBP patients. Please note that the present study hereby aimed to provide insight into the current spinal surgery decision-making process, and not to develop a prediction model for clinical practice.

Methods

Study design

Prior to their first consultation at a Dutch orthopedic hospital specialized in spine care (i.e., Sint Maartenskliniek, Nijmegen/Woerden, the Netherlands), all consecutive low back pain patients received a web-based screening questionnaire between October 2012 and August 2015 (i.e., the Nijmegen Decision Tool for Chronic Low Back Pain; NDT-CLBP). The NDT-CLBP has been implemented in routine practice since 2012, and data have been collected in the institution’s spine registry ever since. After questionnaire completion, patients were invited for a consultation with a spinal surgeon. Patients were eligible for the present study if they experienced low back pain complaints for more than three months (i.e., CLBP) due to degenerative lumbar spine disorders (excluding trauma and tumor), had access to the internet, and were able to read and write Dutch [11].

Outcome measure

The outcome of this study was referral to spinal surgery, meaning that an elective lumbar spine surgery was scheduled (yes/no; coded 1/0). Referral data were derived from hospital records. As registry data are considered the “gold standard” for measuring healthcare use, we perceived hospital records to be the most reliable source for assessing referral to spinal surgery [13].

Potential predictive factors

Based on previous research, 47 potential patient-reported predictive factors of referral to spinal surgery were identified [11]. These factors covered five domains (i.e., socio-demographic, pain, somatic, psychological, and functioning/quality of life) and were measured by the NDT-CLBP with one or more item(s) [10]. A more detailed description of the development and content of the NDT-CLBP can be found elsewhere; [11].

Of the 47 potential predictive factors, four (i.e., self-management of complaints; bulging or protruding disk; influence of rest, mobility and posture; coping) could not be recoded from the NDT-CLBP items, and could, therefore, not be included. Two additional potential predictive factors were added, including consultation location and having health insurance. Consultation location was used as a proxy of variations in referral patterns between healthcare professionals. Having health insurance was added to the model, because insurance coverage was previously found to be a strong predictive factor of referral in various US studies [14, 15]. Eventually, a total of 57 unique potential predictive factors were included in the model. Please note that this number is higher than the number of potential predictive factors described above, because some of the factors were covered by more than one item or questionnaire. An overview of the 57 unique potential predictive factors, as well as their outcome ranges/categories, can be found in Supplementary file 1.

Statistics

Development of the prediction model

The prediction model was developed according to existing guidelines [16]. Potential predictive factors were checked for collinearity. Continuous potential predictive factors were checked for having a linear relationship with the outcome. In case of a non-linear relationship, continuous potential predictive factors were categorized using clinical cut-off scores (e.g., body mass index). First, univariate logistic regression analyses were performed, in which factors with a p < 0.10 were assigned as possible predictive factors and were included in the backwards multivariable logistic regression analysis. In the latter, variables with the highest p value were manually removed one-by-one, until all remaining variables had a p < 0.10 [17].

Performance of the prediction model

Nagelkerke’s R 2 was used to explore the predictability of spinal surgery referral by the covariates fitted in the prediction model. The model’s discriminative ability was explored by the “receiver operating characteristics” area under the curve (AUC). A rough guide for interpreting the AUC is; AUC 0.9–1.0 = Excellent, AUC 0.8–0.9 = good, AUC 0.7–0.8 = fair, AUC 0.6–0.7 = poor, and AUC <0.6 = fail. The Hosmer–Lemeshow goodness-of-fit test was used to explore the model’s calibration. Bootstrapping with 250 replications was used to internally validate the model. Optimism in the model’s regression coefficients, due to overfitting, was estimated by the calibration slope. A slope of 1.0 indicates no optimism by overfitting [17, 18].

Post-hoc analysis

A post hoc analysis was performed to gain preliminary insight into the existence of practice variation. This was explored by conducting a logistic regression analysis, with spinal surgery referral as dependent variable and consultation location as independent variable. First, a crude model was performed that only included the aforementioned variables. Second, an adjusted model was performed in which the relationship between consultation location and spinal surgery referral was corrected for case-mix variables (i.e., age, gender, education, having a job, previous back surgery, interventions in the past, back pain duration, duration radiating leg pain, back pain intensity, leg pain intensity, preceding low back pain episodes, comorbidities).

Except for the model’s internal validation, which was performed in R, analyses were performed in STATA v12.

Results

Participants

5492 low back pain patients completed the web-based screening questionnaire (Response = 97.2%). Of them, 271 had low back pain complaints for less than three months and were, therefore, excluded. Moreover, 51 patients had missing data on body mass index (BMI) and 183 on educational level. As data were complete for all other potential predictive factors and the percentage of patients with missing data was less than 5%, a complete-case analysis was deemed appropriate. Of the 4987 included patients, 330 (6.6%) were referred to surgery and 4657 (93.4%) were not referred to surgery. Supplementary file 2 shows the patients’ characteristics according to referral status.

Development of the prediction model

Gender, BMI, previous back surgery, use of analgesics, having social support, work functioning (i.e., Are you at work despite your back pain complaints?), duration of radiating leg pain, leg pain intensity, comorbidities, numbness in legs/feet, paresthesias in leg/feet, pain in the thoracic spine, somatization, fear of movement/(re)injury, treatment expectations, walking distance, functioning in daily activities, pain-inference daily activities (i.e., Are your back pain complaints bothersome?), and consultation location were univariate predictive factors of spinal surgery referral (Supplementary file 3).

The backwards multivariable logistic regression analysis indicated that gender, BMI, previous back surgery, leg pain intensity, comorbidities, pain in the thoracic spine, somatization, treatment expectations, walking distance, and consultation location were predictive factors of spinal surgery referral (Table 1).
Table 1

Final prediction model for referral to spinal surgery

 

B (SE)

Wald

p

95% CI

Lower bound

Odds ratio

Upper bound

Gender (ref: male)

 Female

0.36 (0.12)

2.90

0.004

1.12

1.44

1.83

Previous back surgery (ref: no.)

 Yes

0.63 (0.12)

5.30

0.000

1.49

1.89

2.38

BMI (ref: BMI < 25)

 25 ≤ BMI < 30

−0.19 (0.13)

−1.51

0.131

0.64

0.83

1.06

 BMI ≥ 30

−0.78 (0.19)

−4.23

0.000

0.32

0.46

0.66

Leg pain intensity (range 0–10)

0.07 (0.22)

3.16

0.002

1.03

1.07

1.12

Comorbidities (ref: none)

 Heart problems

−0.71 (0.33)

−0.21

0.830

0.49

0.93

1.78

 Neurological problems

−0.52 (0.23)

−2.28

0.023

0.38

0.60

0.93

 Cancer

−0.05 (0.76)

−0.07

0.947

0.21

0.95

4.24

 Other disease that influences

−0.85 (0.29)

−2.97

0.003

0.24

0.43

0.75

 walking capabilities

      

 Other disease that induces pain

−0.41 (0.21)

−1.98

0.048

0.44

0.66

0.99

Pain in thoracic spine (ref: no.)

 Yes

−0.43 (0.24)

−1.80

0.071

0.41

0.65

1.03

Somatization—my body is telling me i have something dangerously wrong (ref: strongly disagree)

 Disagree

0.37 (0.19)

1.89

0.059

0.99

1.44

2.11

 Agree

0.38 (0.18)

2.17

0.030

1.04

1.46

2.06

 Strongly agree

0.39 (0.21)

1.83

0.067

0.97

1.47

2.23

Do you expect to be complaint free after treatment? (ref: no.)

 Yes

0.38 (0.12)

3.08

0.002

1.15

1.46

1.86

How far are you able to walk? (ref: <100 m)

 100–500 m

−0.11 (0.17)

−0.63

0.530

0.64

0.90

1.26

 500–1000 m

−0.23 (0.18)

−1.27

0.203

0.56

0.79

1.13

 >1000 m

−0.37 (0.18)

−2.03

0.043

0.48

0.69

0.99

Consultation location (ref: Nijmegen)

 Woerden

−1.45 (0.36)

−3.99

0.000

0.12

0.23

0.48

 Constant (i.e., intercept)

−1.96 (0.48)

−4.10

0.000

0.06

0.14

0.36

Performance and internal validity

The Hosmer–Lemeshow statistic was not significant (X 2 = 10.5; p = 0.23), indicating that the model’s overall fit was good. The model explained 7.5% of the variation in referral to spinal surgery and the model’s AUC was 0.697 (95% CI 0.669–0.724). After internal validation, the model’s explained variance reduced to 5.5% and the AUC to 0.671. The calibration slope was 0.864, indicating relatively little optimism or overfitting of the regression coefficients.

Post-hoc analysis

The crude model indicated that consultation location was statistically significantly associated with spinal surgery referral (OR 0.22; 95% CI 0.11–0.44). After correcting for case-mix variables, the odds ratio remained the same (OR 0.22; 95% CI 0.11–0.44), while the Nagelkerke’s R 2 improved from 1.2 to 4.0%.

Discussion

Main findings

Of the 4987 included patients, only 330 (6.6%) were referred to spinal surgery. Various patient-reported factors were found to be predictive of spinal surgery referral. Of them, female gender, previous back surgery, high intensity leg pain, somatization, and positive treatment expectations were found to increase the odds of being referred to spinal surgery. On the contrary, being obese, having comorbidities, pain in the thoracic spine, increased walking distance, and consultation location (Woerden) were found to decrease the odds of being referred to spinal surgery. The model’s overall fit was good, its discriminative ability was poor, and its explained variance was low (i.e., only 5.5% of the variance in referral was explained by the identified predictive factors). Internal validation had little effect on the model’s performance. Also, a post hoc analysis indicated that consultation location was statistically significantly associated with spinal surgery referral, even after correcting for case-mix variables.

Comparison with the literature

This was the first study to evaluate which patient-reported factors are predictive of spinal surgery referral among CLBP patients using patient referral data. Similar studies using patient referral data are lacking, but Willems et al. evaluated the opinion of 62 Dutch spine surgeons on the use of patient-reported factors (e.g., age, BMI) and predictive tests (e.g., magnetic resonance imaging) in clinical decision-making for spinal fusion [6]. That study showed a lack of professional consensus and indicated that patient-reported factors were not consistently incorporated in the surgeons’ treatment strategy [6]. The lack of professional consensus was underscored by the present finding that consultation location was predictive of spinal surgery referral, the final model’s low explained variance, and the finding that consultation location was statistically significantly associated with spinal surgery referral, even after correcting for case-mix variables.

The finding that gender, BMI, previous back surgery, leg pain intensity, comorbidities, pain in the thoracic spine, somatization, treatment expectations, and walking distance were predictive of spinal surgery referral is in line with previous studies showing that they are also indicative of spinal surgery outcome [12, 19, 20, 21, 22, 23]. This can be interpreted as that surgeons are aware of and adhere to some extent to the current literature and (international) guidelines. In a recent literature review, for example, Gaudin et al. found patient reports of good health and low cardiovascular comorbidity to be the two most powerful preoperative predictors of a better spinal fusion outcome [12]. Also, CLBP patients who expect to be complaint free after treatment were found to be more likely to be referred to spinal surgery than those who still expect to experience complaints. This referral strategy is likely the result of research indicating that positive treatment expectations are predictive of better surgery outcomes [23]. On the contrary, CLBP patients with comorbidities were found to be less likely to be referred to surgery than those without comorbidities. This too suggests that the surgeons referred in line with the current scientific evidence, as comorbidities are an important risk factor for surgery [12, 19, 21, 22].

Some factors were not found to be predictive of referral to spinal surgery, whereas van Hooff et al. found conclusive evidence for their predictive value of treatment outcome [11]. These factors included self-management of complaints, interventions in the past, social support, socioeconomic status, sick leave, litigation, daily course of pain complaints, loss of neurological function, various red flags (e.g., significant trauma, deformities), distress, anxiety, catastrophizing, coping, fear of movement/(re)injury, return-to-work expectations, and health-related physical functioning. Celestin et al., for example, found psychological factors, such as anxiety, depression, and coping, to be important predictive factors of poor response to spinal surgery [20]. Despite the fact that information on these psychological factors was available to the surgeons for all patients, none of them was found to be predictive of referral to spinal surgery. This may be explained by the fact that surgeons typically value medical history and imaging data more than psychosocial screening during their clinical decision-making process [6]. Moreover, the finding that various red flags were not predictive of spinal surgery referral might be explained by the possibility that surgeons base their referral decisions on a combination of red flags, instead of individual red flags (which was evaluated in the present study).

Strengths and limitations

Strengths of the present study are the fact that it was the first to identify predictive patient-reported factors of referral to spinal surgery among a large consecutive cohort of CLBP patients (n = 4987), its use of objective patient referral data, and its high response rate (97.2%).

Some limitations are noteworthy as well. First, possible predictive factors were explored using self-report, which might have caused “recall bias” and/or “social desirability bias”. Second, imaging data were not included, whereas other studies indicate that imaging is valued higher among surgeons than predictive tests, psychological screening, and patient preferences [6]. The lack of imaging data is likely one of the reasons for the final prediction model’s low explained variance and should thus be included in future studies. Third, in the present study, the events per factor (EPV) was approximately 6 (i.e., 330/57). Herewith, the rule of thumb that logistic models should consider at least 10 EPVs was not met in our study [24]. Vittinghoff and McCulloch, however, showed that this rule of thumb can be somewhat relaxed and that with fewer EPVs, only little bias in coefficient estimates can be expected [17, 25]. Fourth, of the 47 indicators for a successful treatment outcome identified by van Hooff et al. [11], four (i.e., the self-management of complaints; bulging or protruding disk; influence of rest, mobility and posture; coping) could not be recoded from the NDT-CLBP items. As a consequence, they could not be included in the present study. Fifth, we have not yet been able to externally validate our model. The lack of external validation, however, does not negate the value of the present findings, as the present study was primarily aimed at exploring predictive factors of spinal surgery referral, instead of developing a prediction model for clinical practice. Also, as the present study was conducted at one orthopedic hospital (with two spinal surgery locations), practice variation could only be explored in a post hoc analysis. Future research into the existence of practice variation in spinal surgery is, therefore, warranted.

Implications for research and practice

The identified lack of professional consensus is probably due to differences in personal preferences and clinical and/or scientific uncertainty as to what CLBP patients might benefit most from spinal surgery. Currently, however, reliable tools for triaging CLBP patients to spinal surgery are lacking. As a better patient selection will likely result in reduced waiting time and healthcare costs and improved surgery outcomes [9], future research should focus on identifying subgroups for which spinal surgery is most likely to be effective, including imaging parameters. In the meantime, this study was the first to indicate that some patient-reported predictive factors of spinal surgery referral can be identified, indicating that surgeons are aware and to some extent adhere to the current scientific evidence. Nonetheless, although the identified predictive factors are known as common predictive factors of surgery outcome, they could only partly predict spinal surgery referral. As such, future studies on predictive factors of spinal surgery referral should also include imaging data.

Conclusion

Some patient-reported factors could be identified that are predictive of spinal surgery referral. Although the identified patient-reported predictive factors are known as common predictive factors of surgery outcome, they could only partly predict spinal surgery referral. Future research on predictive factors of spinal surgery referral should, therefore, also include imaging data. Also, as the difference between the two locations studied suggests a lack of professional consensus, researchers are encouraged to develop reliable tools for triaging CLBP patients to spinal surgery.

Notes

Acknowledgements

The authors would like to thank Jolanda Rubrech for her administrative support and Frank Laumen for managing the institution’s spine registry.

Author contributions

MdK and MvH initiated the institutional spine registry. JvD, MvH, MdK, and RO designed the study. JvD and MvH collected and cleaned the data. JvD conducted the analyses and wrote the manuscript. MvH, MS, MdK, and RO helped with interpreting the data and contributed to the drafting of the manuscript.

Compliance with ethical standards

Ethics

This study was approved by the institutional review board of the participating hospital. Ethical approval was not required as the “Dutch Act on Medical Research involving Human Subjects” does not apply to screening questionnaires that are part of routine practice.

Sources of support

This study was funded by a personal grant provided by the EMGO+ Institute for Health and Care Research (to JM van Dongen) as well as a grant received from “Innovatiefonds zorgverzekeraars”.

Conflict of interest

All authors declare that they have no conflict of interest.

Supplementary material

586_2017_5201_MOESM1_ESM.docx (18 kb)
Supplementary material 1 (DOCX 18 kb)
586_2017_5201_MOESM2_ESM.docx (21 kb)
Supplementary material 2 (DOCX 20 kb)
586_2017_5201_MOESM3_ESM.docx (25 kb)
Supplementary material 3 (DOCX 24 kb)

References

  1. 1.
    Hoy D, March L, Brooks P, Blyth F, Woolf A, Bain C et al (2014) The global burden of low back pain: estimates from the Global Burden of disease 2010 study. Ann Rheum Dis 73(6):968–974CrossRefPubMedGoogle Scholar
  2. 2.
    Pengel LH, Herbert RD, Maher CG, Refshauge KM (2003) Acute low back pain: systematic review of its prognosis. BMJ 327(7410):323CrossRefPubMedPubMedCentralGoogle Scholar
  3. 3.
    Lambeek LC, van Tulder MW, Swinkels IC, Koppes LL, Anema JR, van Mechelen W (2011) The trend in total cost of back pain in The Netherlands in the period 2002–2007. Spine 36(13):1050–1058CrossRefPubMedGoogle Scholar
  4. 4.
    Weinstein JN, Lurie JD, Olson P, Bronner KK, Fisher ES, Morgan MTS (2006) United States trends and regional variations in lumbar spine surgery: 1992–2003. Spine 31(23):2707CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Balagué F, Mannion AF, Pellisé F, Cedraschi C (2012) Non-specific low back pain. Lancet 379(9814):482–491CrossRefPubMedGoogle Scholar
  6. 6.
    Willems P, de Bie R, Öner C, Castelein R, de Kleuver M (2011) Clinical decision making in spinal fusion for chronic low back pain. Results of a nationwide survey among spine surgeons. BMJ Open 1(2):e000391CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Deyo RA, Mirza SK (2009) The case for restraint in spinal surgery: does quality management have a role to play? Eur Spine J 18(3):331–337CrossRefPubMedPubMedCentralGoogle Scholar
  8. 8.
    Kamper SJ, Maher CG, Hancock MJ, Koes BW, Croft PR, Hay E (2010) Treatment-based subgroups of low back pain: a guide to appraisal of research studies and a summary of current evidence. Best Prac Res Clin Rheumatol 24(2):181–191CrossRefGoogle Scholar
  9. 9.
    Wilgenbusch CS, Wu AS, Fourney DR (2014) Triage of Spine surgery referrals through a multidisciplinary care pathway: a value-based comparison with conventional referral processes. Spine 39(22S):S129–S135CrossRefPubMedGoogle Scholar
  10. 10.
    Fairbank J, Gwilym SE, France JC, Daffner SD, Dettori J, Hermsmeyer J et al (2011) The role of classification of chronic low back pain. Spine 36:S19–S42CrossRefPubMedGoogle Scholar
  11. 11.
    van Hooff ML, van Loon J, van Limbeek J, de Kleuver M (2014) The Nijmegen decision tool for chronic low back pain. Development of a clinical decision tool for secondary or tertiary spine care specialists. PLoS One 9(8):e104226CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    Gaudin D, Krafcik BM, Mansour TR, Alnemar A (2017) Considerations in spinal fusion surgery for chronic lumbar pain: psychological factors, rating scales, and perioperative patient education—a review of the literature. World Neurosurg 98:21–27CrossRefPubMedGoogle Scholar
  13. 13.
    Drummond MF, Sculper MJ, Claxton K, Stoddart GL, Torrance GW (2015) Methods for the economic evaluation of health care programmes, 4th edn. Oxford University, Oxford United KingdomGoogle Scholar
  14. 14.
    Shea D, Stuart B, Vasey J, Nag S (1999) Medicare physician referral patterns. Health Ser Res 34(1 Pt 2):331Google Scholar
  15. 15.
    Cortés O, Arthur HM (2006) Determinants of referral to cardiac rehabilitation programs in patients with coronary artery disease: a systematic review. Am Heart J 151(2):249–256CrossRefPubMedGoogle Scholar
  16. 16.
    Collins GS, Reitsma JB, Altman DG, Moons GM (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 350:g7594CrossRefPubMedGoogle Scholar
  17. 17.
    Van der Werff S, Junco Díaz R, Reyneveld R, Heymans M, Ponce Campos M, Gorbea Bonet M (2013) Prediction of asthma by common risk factors: a follow-up study in Cuban schoolchildren. J Investig Allergol Clin Immunol 23(6):415–420PubMedGoogle Scholar
  18. 18.
    Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N et al (2010) Assessing the performance of prediction models: a framework for some traditional and novel measures. Epidemiology 21(1):128CrossRefPubMedPubMedCentralGoogle Scholar
  19. 19.
    Aalto TJ, Malmivaara A, Kovacs F, Herno A, Alen M, Salmi L et al (2006) Preoperative predictors for postoperative clinical outcome in lumbar spinal stenosis: systematic review. Spine 31(18):E648–E663CrossRefPubMedGoogle Scholar
  20. 20.
    Celestin J, Edwards RR, Jamison RN (2009) Pretreatment psychosocial variables as predictors of outcomes following lumbar surgery and spinal cord stimulation: a systematic review and literature synthesis. Pain Med 10(4):639–653CrossRefPubMedGoogle Scholar
  21. 21.
    Katz JN, Stucki G, Lipson SJ, Fossel AH, Grobler LJ, Weinstein JN (1999) Predictors of surgical outcome in degenerative lumbar spinal stenosis. Spine 24(21):2229CrossRefPubMedGoogle Scholar
  22. 22.
    Airaksinen O, Herno A, Turunen V, Saari T, Suomlainen O (1997) Surgical outcome of 438 patients treated surgically for lumbar spinal stenosis. Spine 22(19):2278–2282CrossRefPubMedGoogle Scholar
  23. 23.
    Yee A, Adjei N, Do J, Ford M, Finkelstein J (2008) Do patient expectations of spinal surgery relate to functional outcome? Clin Orthop Relat Res 466(5):1154–1161CrossRefPubMedPubMedCentralGoogle Scholar
  24. 24.
    Steyerberg E (2008) Clinical prediction models: a practical approach to development, validation, and updating. Springer Science & Business Media, BerlinGoogle Scholar
  25. 25.
    Vittinghoff E, McCulloch CE (2007) Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol 165(6):710–718CrossRefPubMedGoogle Scholar

Copyright information

© The Author(s) 2017

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • Johanna M. van Dongen
    • 1
    Email author
  • Miranda L. van Hooff
    • 2
  • Maarten Spruit
    • 3
  • Marinus de Kleuver
    • 4
  • Raymond W. J. G. Ostelo
    • 1
    • 5
  1. 1.Department of Health Sciences and EMGO+ Institute for Health and Care Research, Faculty of Earth and Life SciencesVU University AmsterdamAmsterdamThe Netherlands
  2. 2.Department of ResearchSint MaartenskliniekNijmegenThe Netherlands
  3. 3.Department of Orthopedic SurgerySint MaartenskliniekNijmegenThe Netherlands
  4. 4.Department of Orthopedic SurgeryRadboud University Medical CenterNijmegenThe Netherlands
  5. 5.Department of Epidemiology and Biostatistics and the EMGO+ Institute for Health and Care ResearchVU University Medical CenterAmsterdamThe Netherlands

Personalised recommendations