Funnel plots and spine surgery: unit level and surgeon level outcome data
A funnel plot is a scatter plot of treatment effect against a measure of study size. Introduced by Light and Pillemer in 1984 and advanced by Egger et al.  they have been used to assess the existence of publication bias in systematic reviews and meta-analyses. Spiegelhalter  helpfully discussed the use of funnel plots to compare institutional performance and noted the advantages of being readily interpretable, avoidance of spurious institutional ranking and allowance for variability of institutions with small volume and the relationship between outcome and volume to be assessed.
The year 2013 has seen the ‘landmark’ publication by the Healthcare Quality Improvement Partnership (HQIP) entitled ‘Everyone counts: offer 2′ followed by publication of surgeon level data on bariatric, cardiac, colo-rectal, head and neck, orthopaedic, thyroid and endocrine, upper GI, urological and vascular surgery . Surgeons had the option not to consent to data publication, but their names and reasons for not consenting were published on the NHS Choices website . This, unsurprisingly, was followed by publication in the press of surgeons perceived to have high mortality rates. As Orthopaedic and Neurosurgical spine surgeons our time will surely come!
In this edition of the European Spine Journal, Divecha et al. and the British Scoliosis Society (BSS, founded 1975) members have contributed to this area with the publication of 5 years cumulative unit level safety data on children and adults undergoing spine deformity surgery. The patients’ data were submitted anonymously and voluntarily by contributing units and have been presented annually at the BSS scientific meeting. There has been increased participation year on year and information on over 9,000 patients has been collected. Funnel plots of unit level data have been produced for neurological injury, infection and death. Some of us have used this data, when unfavourable, to undertake whole system reviews in our units, changes in practice and reversal of an ‘outlying’ position .
The previous deformity surgery standards have come from publications from the Scoliosis Research Society (SRS) Morbidity and Mortality Committee database. The SRS has increasing sought to encourage (candidate and active) members to contribute their data annually. Most recently this has taken the form of a financial penalty if membership is maintained but data not submitted—‘we want your data, not your money’. The BSS data are remarkably consistent with the SRS data.
But whose responsibility is it to decide what data we are to collect and when, then to collect it, to ensure its accuracy, submit it to our national registry (or another) and then reflect and act, as necessary, on the outcomes and complications identified. There are many fine examples in the UK of units collecting data over many years. Much of this to date has been used effectively for research purposes.
Commissioning should mandate the active collection of outcomes and safety data. This is being actively considered by the D14 Complex Spine Clinical Reference Group (CRG)  who will set the quality dashboard and service specifications  for paediatric and adult deformity surgery. Our three national spine societies all have regional representatives (senators) on the complex spine CRG to ensure our collective input. Our NHS trusts should since mid-2013 be collecting outcomes’ data.
A minimum data set for a patient undergoing corrective surgery for adolescent idiopathic scoliosis (AIS) should probably include a patient-reported outcome measure such as the SRS-22, a simple measure of the trunk deformity such as the angle of trunk inclination (ATI) and a radiographic measure such as the Cobb angle (in spite of its shortcomings) each measured pre-operatively and post-operatively at 1 and 2 years as outcome measures. Safety data should include mortality, neurological deficits, deep infection (early and late), and reoperations. Outcome measures are more challenging to collect in younger patients with spine deformities and will require much longer follow-up.
Effective treatment of spine deformities, especially in children, is often time-specific and it is essential the commissioners, Clinical Commissioning Groups (CCG’s) and CRG’s break down barriers to referral and management. The re-emergence of bracing as an effective treatment of AIS will necessitate prompt referral of patients with small curves so this option can at least be discussed with patients .
The accurate collection of data requires effort and resource. Sell examined the utility of a surgeon-based system and concluded that follow-up collection rates were poor . An HQIP National Joint Registry press release dated 13 Sept 2013 stated 40 % of NHS Trusts and Health Boards ‘red’ rated for submissions to mandatory National Joint Registry showed the difficulty in consistent data collection. Some data collection can be delegated such as patient-reported outcome measures, but other information requires active surgeon input such as operation details and confirmation of complications.
In England, our NHS trusts currently submit data to the NHS which subsequently emerges as HES data. For the period 2010–2011, the annual number of operations for spine deformity according to these data was approximately 1,200 in the 17 years and below age group. The British Spine Registry (BSR) funded by the British Association of Spine Surgeons (BASS) and now the BSS between July 2012 and December 2013 had 320 patients entered on the deformity pathway . At present much of the input relies on surgeon enthusiasm. While spinal cord monitoring is considered mandatory for most paediatric deformity surgeries in the UK, its use is less well defined in adult deformity surgery and the registry will eventually be able to provide useful information on this and other safety measures. The current choice of registry in the UK seems to lie between Spine Tango and the BSR. The ability of these registries to output the desired information to their users will, in the long term, be of key importance.
As a part of revalidation we are now expected to reflect on our individual surgical outcomes and complications and will want to ensure the information we reflect on is accurate. We need some data that are surgeon-specific, particularly so we can assess the effectiveness of particular interventions in our hands. However, the environment in which we undertake major surgical procedures in can have a profound effect on our ability to perform efficiently and effectively. Review of accurate unit level data remains an essential part of our clinical governance activity. In short, we need both surgeon level and unit or institution level data which are accurate to inform our clinical practices and serve our patients. We can be sure that healthcare commissioners, lawyers and the press will also remain interested in our outcome and complication-data collection activities.
Conflict of interest
Our unit has an industry funded part time data collection clerk who since mid-2013 has entered PROMS on to the BSR.
- 5.Lavelle D et al (2013) Reducing deep infections following paediatric spinal deformity surgery. In: Podium presentation at British Scoliosis Society 12–13th Sept 2013Google Scholar
- 9.Sell P (2013) The utility of spine registry. Eur Spine J 22(Supp 1):S68Google Scholar
- 10.Personal communication (2012) L BreakwellGoogle Scholar