Abstract
Background
The Idea, Development, Exploration, Assessment and Long-term study (IDEAL) framework was created to provide a structured way for assessing and evaluating novel surgical techniques and devices.
Objectives
The aim of this paper was to investigate the utilization of the IDEAL framework within neurosurgery, and to identify factors influencing implementation.
Methods
A bibliometric analysis of the 7 key IDEAL papers on Scopus, PubMed, Embase, Web of Science, and Google Scholar databases (2009–2019) was performed. A second journal-specific search then identified additional papers citing the IDEAL framework. Publications identified were screened by two independent reviewers to select neurosurgery-specific articles.
Results
The citation search identified 1336 articles. The journal search identified another 16 articles. Following deduplication and review, 51 relevant articles remained; 14 primary papers (27%) and 37 secondary papers (73%). Of the primary papers, 5 (36%) papers applied the IDEAL framework to their research correctly; two were aligned to the pre-IDEAL stage, one to the Idea and Development stages, and two to the Exploration stage. Of the secondary papers, 21 (57%) explicitly discussed the IDEAL framework. Eighteen (86%) of these were supportive of implementing the framework, while one was not, and two were neutral.
Conclusion
The adoption of the IDEAL framework in neurosurgery has been slow, particularly for early-stage neurosurgical techniques and inventions. However, the largely positive reviews in secondary literature suggest potential for increased use that may be achieved with education and publicity.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
With technological advances leading to rapid development of new devices and operative techniques, it remains imperative that we critically assess novel ideas to ensure they confer true patient benefit. Innovative solutions to unique problems are reliant on creativity and lateral thinking, which can appear at odds with the rules and regulations required to systematically appraise developments. However, it is in the interest of patient safety and the society to ensure that widespread implementation occurs after rigorous assessment and research [18].
The IDEAL framework is a straightforward and structured approach that can guide evaluation and research across all surgical fields, while also allowing enough flexibility to prevent stifling of innovation. First published in 2009 [41], it was developed to provide guidance on the evaluation of surgical techniques and devices from inception to long-term evaluation [7]. It consists of 5 stages: Idea, Development, Exploration, Assessment, and Long-term studies. Each stage includes various recommendations to ensure that high-quality evidence is obtained when studying new ideas, as demonstrated in Table 1.
This framework differs from the traditional model of pharmacological trials to accommodate the distinct way surgical developments occur, particularly since individual operator skill and technique modifications at the early stages can differentially impact outcomes.[46]. A subsequent paper introduced Stage 0, a preclinical stage, for testing involving cadavers, animals, or simulations, while another clarified the use of the IDEAL framework in surgical device development [25, 56]. The IDEAL structure adds value to surgical innovation by ensuring safety and regulation; however, uptake has varied across various countries and surgical specialties. While utilization has increased overall, this has predominantly happened in other surgical fields [28]. A recent review of the IDEAL framework applied within minimally invasive neurosurgical research assessed historical studies involved in the development of the endoscopic endonasal approach for skull base meningiomas and the Woven EndoBridge (WEB) device for endovascular treatment of intracranial aneurysms [48]. The authors assessed the quality of the research that contributed to these developments by measuring the adherence of various studies to IDEAL guidelines. Their search revealed a total of four clinical papers that could be aligned with any stage of the IDEAL framework.
The aim of this bibliometric review is to investigate the adoption of the IDEAL framework in neurosurgical literature as a whole and reflect on factors influencing implementation since its inception 10 years ago.
Methodology
We performed individual citation searches for the seven main IDEAL papers [13, 18, 19, 25, 41, 42, 56] on Scopus, PubMed, Embase, Web of Science, and Google Scholar databases (September 2009 to August 2019). Articles were identified and analyzed by two independent reviewers, CO and BS. These searches were completed on each database individually. Papers identified were stored and deduplicated in Mendeley reference manager. The titles and abstracts of all papers were then screened for relevance to the study by the two independent reviewers, followed by full text review; the final selection was reviewed by a third independent reviewer to ensure suitability for inclusion (AA). Any disagreements were resolved following discussion with the senior author (AK). All papers with a focus on neurosurgical techniques or devices published between September 2009 and August 2019 were included. Non-English papers and non-journal text (such as book chapters) were excluded. This methodology was first used by Tradewell et al., investigating the use of the IDEAL framework in urological research [57].
For completeness, a second search using the terms “IDEAL Collaboration” and “IDEAL framework” was performed in the following major neurosurgical journals: Acta Neurochirurgica, British Journal of Neurosurgery, Child’s Nervous System, Clinical Neurology and Neurosurgery, Journal of Clinical Neuroscience, Journal of Korean Neurosurgical Society, Journal of Neurology, Neurosurgery and Psychiatry, Journal of Neurosurgery, Journal of Neurosurgery: Pediatrics, Journal of Neurosurgery: Spine, Journal of Neurosurgical Sciences, Neurosurgery, Neurosurgery Clinics of North America and Clinical Neurosurgery, Neurosurgical Focus, Neurosurgical Review, Pediatric Neurosurgery, Stereotactic and Functional Neurosurgery, Turkish Neurosurgery, and World Neurosurgery.
Primary papers were defined as original research publications assessing new surgical techniques or devices, and were linked to the most relevant stage of the IDEAL protocol (many self-identified) and assessed for adherence to criteria for that stage. Secondary papers included systematic reviews, opinion pieces, and letters to the editor; these were assessed with regard to their support for and evaluation of the IDEAL papers and framework.
Results
The citation search identified 1336 articles, while the secondary search identified another 16 (see Fig. 1). Following deduplication and abstract review, 51 relevant articles were identified. Thereafter remained fourteen primary papers [8, 17, 20, 26, 33, 34, 37, 39, 46, 51, 55, 58,59,60], and 37 secondary papers [1,2,3,4,5, 9,10,11,12, 14,15,16, 23, 27, 29,30,31,32, 35, 36, 38, 40, 43,44,45, 47,48,49,50, 52,53,54, 57, 62, 64,65,66].
Articles were published from 2010 to July 2019 (date of citation search), with a peak in 2018. The top 3 journals for publications that cited the IDEAL articles were Acta Neurochirurgica (n = 8), Journal of Neurosurgery (n = 5), and World Neurosurgery (n = 5).
Primary publication analysis
All of the fourteen primary publications identified were from European groups. Papers were published between 2013 and 2019 and spanned a number of sub-specialties including skull base, neuro-oncology, and spinal neurosurgery. Ten papers (71%) explicitly mention the IDEAL framework within their text [8, 20, 26, 33, 34, 37, 39, 55, 58, 60]. Six (43%) papers aimed to align their research with a specific stage of the IDEAL framework [8, 20, 33, 34, 39, 60]. Of these six, five (83%) papers appropriately applied the IDEAL framework to their research; one publication claiming to align to stage 1/2a was an animal study [8]. Animal studies are technically stage 0, the preclinical stage of the IDEAL framework [33].
Of the five papers appropriately staged, their adherence to IDEAL recommendation including study size, ethical oversight, and outcome measures are presented in Table 2.
The paper by Belotti et al. aligned with Stage 0, the preclinical stage of the IDEAL framework [8]. This study involved the use of 10 specimens for testing in one clinical center, another two for filming in a second center. The purpose of the study was to improve safety and effectiveness of the transsphenoidal approach by categorizing different approaches, aligning with Stage 0. Marcus et al. also aligned their study to Stage 0, within the IDEAL-D framework specific to devices; comparing the use of computer-assisted planning with manual planning in stereotactic brain biopsy [39]. The study by Versteeg et al. that investigated spinal stabilization following radiotherapy had two arms. Each arm was applied appropriately to the Idea (1) and Development (2a) stages with samples sizes of 3 and 10, respectively [60]. Adverse events were evaluated and classified according to the common toxicity criteria adverse events during a median clinical follow-up time of 13 months. The study by Majovsky et al. was stage 2a and assessed the use of burr holes in evacuation of chronic subdural hematomas [34]. The paper by Flores et al. was also aligned to stage 2a, investigating the use of syringes as dilators and retractors in spinal surgery in 30 patients [20]. None of the studies included from the citation search identified as stage 2b, 3, or 4 (exploration, assessment, long-term monitoring).
Secondary publication analysis
Thirty-seven secondary papers were identified. Of them, 13 (35%) were narrative reviews [9, 11, 15, 27, 29,30,31, 43,44,45, 52, 53, 64]. Eleven (30%) were systematic reviews [1, 5, 10, 14, 16, 38, 40, 46, 48, 49, 66].
The remaining papers identified are consist of 10 letters to the editor (27%) [2,3,4, 12, 23, 32, 36, 47, 50, 54], 2 questionnaire-based analyses (one exploring views on the use of evidence-based medicine principles in neurosurgery, the other seeking to understand the neurosurgical definition of “innovation”) [62, 65], and 1 article outlining the work of the British Neurosurgical Trainee Collaborative [57].
Of the secondary publications, 21 (57%) explicitly mentioned the IDEAL framework [1, 3, 5, 10,11,12, 14, 15, 29, 32, 38, 43, 47,48,49,50, 52, 53, 57, 60, 65], while the rest cited the IDEAL publications, but did not discuss them. Of the 21 papers that discussed the IDEAL framework, 18 (86%) were supportive of the use of the IDEAL framework [3, 5, 10,11,12, 14, 15, 29, 32, 38, 43, 47, 49, 50, 52, 53, 57, 60], while 1 paper was not [1], and 2 were neutral [48, 65]. Six of these publications evaluated the applicability of the IDEAL framework in neurosurgical innovation [38, 47, 48, 50, 64, 65].
Discussion
The aim of this study was to explore the adoption of the IDEAL framework within neurosurgery by reviewing citations of the key IDEAL papers. The citation search allowed us to assess awareness of the framework, and examine the way in which it is utilized. This methodology was informed by the work of Tradewell et al. who also used a citation search to review the use of the IDEAL framework in urological literature [21]. Given the large number of citations of these papers in urological literature, it is clear that the framework has impacted thinking in surgical development; however, this does not seem to be reflected particularly in neurosurgery, evident by the small number of papers that specifically cited the framework, or were aligned to a specific stage of the framework. Encouragingly, the number of neurosurgical papers referencing the framework has shown a year on year increase; however, numbers remain relatively small.
There are several factors that could contribute to the limited uptake of the IDEAL framework in neurosurgery. A potential cause is a lack of awareness. The IDEAL framework was developed in the UK, and initially published in 2009 [41]. Though there are a number of articles that outline the IDEAL framework and its recommendations, it is possible that those involved in current research and surgical development are either not aware, or not particularly well versed in the framework. This is supported by our finding that all primary papers were of European origin. It is also important to consider that many researchers may not have considered the need for a framework specifically designed for surgical innovation, and therefore did not seek one out. Given the low numbers of primary papers that cited and applied the IDEAL framework to their research, and given that some of these papers did so incorrectly or incompletely, it is clear that education on the IDEAL framework is still actively required to guide researchers and authors, a problem also identified by Khachane et al. [28]. The IDEAL council has identified this potential issue, and encourages surgeons from all specialties and any country to join the collaboration, particularly through the use of a diverse group of council advisors and specialty leads from a number of different countries and specialties, who help to develop and promote the IDEAL framework. The IDEAL council view the framework as ever-evolving in response to pragmatic need and a requirement to learn from IDEAL’s end-users. The update paper was also published in the Annals of Surgery, a US Journal, to further increase the reach of the framework [25].
It is also important to consider how applicable the framework is to neurosurgical research. Surgical innovation and technical development differ between specialties and even sub-specialties, as illustrated in Muskens et al.’s examination of innovation within endoscopic endonasal surgery for skull base meningiomas versus the Woven EndoBridge device for endovascular treatment of intracranial aneurysms. When they explored mapping those developments to the IDEAL framework, only four of the 21 papers included could be matched to any stage of the IDEAL framework: two papers to Stage 2A and two to 2B [48]. Looking specifically at the lack of papers that align with stage 2B, as replicated by our own results, a potential reason for the low numbers of this type of study is that neurosurgical research still largely follows the traditional model of a phase 2 randomized trial that may or may not be followed by a phase 3 randomized trial. A recent example is the MISTIE II and MISTIE III trials (phase 2 and phase 3 trials), which evaluated a new technique for evacuation of intracerebral hemorrhage (ICH) [22, 63]. Given these studies fulfill much of the criteria for IDEAL stages 2b and 3, it is clear that potential to align these studies is the IDEAL framework is present; however, tradition or simply lack of awareness of the IDEAL framework may have prevented this.
Another potential reason is that, as is often the case with surgical research, there is an overreliance on retrospective “case series.” This is likely because case series are easy to perform, require less resources, can be conducted at a single center, and, for many surgeons, are means to showcase their surgical skills and outcomes [61]. Furthermore these studies afford a flexibility that may not be afforded by adhering strictly to the IDEAL framework. Some surgeons would argue that limiting this flexibility stifles innovation. However, the true aim of the framework is to facilitate the conduct of well-designed and well-executed studies in order to facilitate the adoption of innovative techniques, if found to be effective. A recent example is the use of middle meningeal artery (MMA) embolization for chronic subdural hematomas (CSDH), where the majority of conducted studies are case series [6]. Although this procedure is clearly innovative, many of the studies reported use of MMA embolization for atypical indications (e.g., asymptomatic patients, as “prophylaxis” after surgery) outside the context of ethically approved research.
The lack of IDEAL stage 3 studies, which are typically a definitive, multi-center randomized trial, can be explained by the fact that these studies are usually identified as phase 3 trials or simply randomized trials, despite technically meeting the criteria of a stage 3 IDEAL study. A recent review identified 401 published randomized trials on brain and spine conditions treated by neurosurgeons from 2003 to 2016 [24]. Given these numbers, it appears that randomized trials are possible within neurosurgical innovation, and therefore the potential for stage 2b and stage 3 IDEAL studies is much greater than what is currently present. In order to investigate this further, qualitative feedback should be sought from neurosurgeons on the applicability of the IDEAL framework to their research, in order to identify factors that have limited or prevented use. This information may in turn be used to guide future updates of the framework, and educational materials used within the neurosurgical community.
As most developments occur in incremental improvements upon techniques or devices, the appropriateness of large randomized controlled trials for each small, additive change is questionable. However, this would not apply to innovations, such as MMA embolization or the MISTIE procedure, which are entirely different to the usual method of treating CSDH and ICH, respectively [21, 22, 61]. There are also ethical considerations; if a new technique or surgical device displays a substantial, unequivocal benefit over past standard (e.g. the introduction of microscope in micro-neurosurgery), the lack of clinical equipoise precludes conducting a trial that may expose patients to suboptimal treatment.
There are also ongoing concerns that neurosurgical randomized trials are often not feasible or impractical [64]. Some authors suggest that routinely collected or observational data can lead to robust conclusions regarding the comparative effectiveness of treatment; however, this is a relatively new field with ongoing methodological challenges. As an example, a recent study found that observational studies based on routinely collected health data could give different answers from subsequent randomized trials on the same clinical questions and may substantially overestimate treatment effects [6]. Reasons for this are likely multiple, and while this difference could reflect a difference in validity achieved by the different study methodologies, it is important to consider the impact the highly selective populations used for randomized trials could have on research outcomes, in comparison with the broader populations that usually contribute to observational studies. Another option, which is gaining traction in recent years, is the use of pragmatic large randomized trials that have broad inclusion criteria in order to reflect real-world practice. The IDEAL collaboration is working on developing guidance on the use of real-world evidence for the purposes of comparative effectiveness research.
The most recent IDEAL publication, a follow up to the first paper, has elaborated more on the recommendations and how to apply them for each stage [25]. It is hoped that this new update of the IDEAL framework will improve the understanding of where the framework can fit within neurosurgery and therefore improve uptake. Further projects include stage-specific reporting guidelines developed using Delphi methodology. Given that the majority of secondary papers that cited the IDEAL framework were supportive of its use, it would seem that education, promotion, and room for specialty-specific nuance within recommendations could largely improve uptake, and in turn help guide neurosurgical development to produce a high-quality evidence base for our practice.
Limitations of this paper
This paper reviewed articles that cited the IDEAL framework key papers. It is possible that there are studies that have adhered to and referenced the framework, but have not cited these papers and are therefore excluded from this review. The secondary journal specific search aimed to negate this limitation; however, this solely identified articles within the selection of journals searched (listed above). Papers published in other journals will not have been identified in this search. It is also possible that there is a research that has unintentionally adhered to the IDEAL framework but has not been included, again as they have not cited the IDEAL papers. Evaluation of the papers and their adherence to the IDEAL framework recommendation (primary papers) or support of the IDEAL framework (secondary papers) was subjective, as based on the opinion and understanding of the authors of this paper.
Conclusion
Ultimately, in order to fully evaluate the potential for the IDEAL framework in neurosurgical research, it is necessary that more primary research studies attempt to follow the recommendations. Feedback highlighting neurosurgery-specific limitations can be generated and incorporated into future iterations of the framework. This will ensure that it is able to support and work with nuances and specialty-specific concerns that are causing limited use of the IDEAL framework thus far.
References
Almutairi R, Muskens I, Cote D, Dijkman M, Kavouridis V, Crocker E, Ghazawi K, Broekman M, Smith T, Mekary R, Zaidi H (2018) Gross total resection of pituitary adenomas after endoscopic vs. microscopic transsphenoidal surgery: a meta-analysis. Acta Neurochir 160(5):1005–1021
Al-Shahi Salman R, Whittle I (2012) There is still Hope for surgery for spontaneous supratentorial intracerebral hemorrhage. Stroke 43(6):1460–1461
Ammirati M (2017) Innovation in neurosurgery response to: “Ideal”, the operating microscope, and the parachute. Acta Neurochir 160(2):371–371
Ammirati M, Ercan S (2017) Letter to the editor. Keyhole and standard subtemporal approaches. J Neurosurg 127(1):228–229
Ammirati M, Wei L, Ciric I (2012) Short-term outcome of endoscopic versus microscopic pituitary adenoma surgery: a systematic review and meta-analysis. J Neurol Neurosurg Psychiatry 84(8):843–849
Azad T, Veeravagu A, Mittal V, Esparza R, Johnson E, Ioannidis J, Grant G (2017) Neurosurgical randomized controlled trials—distance travelled. Neurosurgery 82(5):604–612
Barkun J, Aronson J, Feldman L, Maddern G, Strasberg S (2009) Evaluation and stages of surgical innovations. Lancet 374(9695):1089–1096
Belotti F, Doglietto F, Schreiber A, Ravanelli M, Ferrari M, Lancini D, Rampinelli V, Hirtler L, Buffoli B, Bolzoni Villaret A, Maroldi R, Rodella L, Nicolai P, Fontanella M (2018) Modular classification of endoscopic endonasal transsphenoidal approaches to sellar region: anatomic quantitative study. World Neurosurgery 109:e281–e291
Brastianos P, Galanis E, Butowski N, Chan J, Dunn I, Goldbrunner R, Herold-Mende C, Ippen F, Mawrin C, McDermott M, Sloan A, Snyder J, Tabatabai G, Tatagiba M, Tonn J, Wen P, Aldape K, Nassiri F, Zadeh G, Jenkinson M, Raleigh D, Aldape K, Au K, Barnhartz-Sloan J, Bi W, Brastianos P, Butowski N, Carlotti C, Cusimano M, DiMeco F, Drummond K, Dunn I, Galanis E, Giannini C, Goldbrunner R, Griffith B, Hashizume R, Hanemann C, Herold-Mende C, Horbinski C, Huang R, James D, Jenkinson M, Jungk C, Kaufman T, Krischek B, Lachance D, Lafougère C, Lee I, Liu J, Mamatjan Y, Mansouri A, Mawrin C, McDermott M, Munoz D, Nassiri F, Noushmehr H, Ng H, Perry A, Pirouzmand F, Poisson L, Pollo B, Raleigh D, Sahm F, Saladino A, Santarius T, Schichor C, Schultz D, Schmidt N, Selman W, Sloan A, Spears J, Snyder J, Suppiah S, Tabatabai G, Tatagiba M, Tirapelli D, Tonn J, Tsang D, Vogelbaum M, Deimling A, Wen P, Walbert T, Westphal M, Workewych A, Zadeh G (2019) Advances in multidisciplinary therapy for meningiomas. Neuro-oncology 21(Supplement_1):i18–i31
Chari A, Kolias A, Santarius T, Bond S, Hutchinson P (2014) Twist-drill craniostomy with hollow screws for evacuation of chronic subdural hematoma. J Neurosurg 121(1):176–183
Chari A, Hentall I, Papadopoulos M, Pereira E (2017) Surgical neurostimulation for spinal cord injury. Brain Sciences 7(12):18
Coffey R (2013) Editorial: thalamic DBS and neuropathic pain. Neurosurg Focus 35(3):E8
Cook J, McCulloch P, Blazeby J, Beard D, Marinac-Dabic D, Sedrakyan A (2013) IDEAL framework for surgical innovation 3: randomised controlled trials in the assessment stage and evaluations in the long term study stage. BMJ 346(jun18 3):f2820–f2820
Corbett M, South E, Harden M, Eldabe S, Pereira E, Sedki I, Hall N, Woolacott N (2018) Brain and spinal stimulation therapies for phantom limb pain: a systematic review. Health Technol Assess 22(62):1–94
Doglietto F, Maira G (2013) Endoscopic Skull Base surgery: probably not the time for meta-analyses but certainly for prospectively collected data. World Neurosurgery 80(6):784–786
Doglietto F, Radovanovic I, Ravichandiran M, Agur A, Zadeh G, Qiu J, Kucharczyk W, Fernandez E, Fontanella M, Gentili F (2016) Quantification and comparison of neurosurgical approaches in the preclinical setting: literature review. Neurosurg Rev 39(3):357–368
Doglietto F, Qiu J, Ravichandiran M, Radovanovic I, Belotti F, Agur A, Zadeh G, Fontanella M, Kucharczyk W, Gentili F (2017) Quantitative comparison of cranial approaches in the anatomy laboratory: a neuronavigation based research method. World Journal of Methodology 7(4):139–147
Ergina P, Cook J, Blazeby J, Boutron I, Clavien P, Reeves B, Seiler C (2009) Challenges in evaluating surgical innovation. Lancet 374(9695):1097–1104
Ergina P, Barkun J, McCulloch P, Cook J, Altman D (2013) IDEAL framework for surgical innovation 2: observational studies in the exploration and assessment stages. BMJ 346(jun18 3):f3011–f3011
Flores R, Beltrán J, Ogando-Rivas E (2019) An affordable and feasible technique for minimally invasive tubular lumbar discectomy. World Neurosurgery 129:378–385
Hanley D, Thompson R, Muschelli J, Rosenblum M, McBee N, Lane K, Bistran-Hall A, Mayo S, Keyl P, Gandhi D, Morgan T, Ullman N, Mould W, Carhuapoma J, Kase C, Ziai W, Thompson C, Yenokyan G, Huang E, Broaddus W, Graham R, Aldrich E, Dodd R, Wijman C, Caron J, Huang J, Camarata P, Mendelow A, Gregson B, Janis S, Vespa P, Martin N, Awad I, Zuccarello M (2016) Safety and efficacy of minimally invasive surgery plus alteplase in intracerebral haemorrhage evacuation (MISTIE): a randomised, controlled, open-label, phase 2 trial. The Lancet Neurology 15(12):1228–1237
Hanley D, Thompson R, Rosenblum M, Yenokyan G, Lane K, McBee N, Mayo S, Bistran-Hall A, Gandhi D, Mould W, Ullman N, Ali H, Carhuapoma J, Kase C, Lees K, Dawson J, Wilson A, Betz J, Sugar E, Hao Y, Avadhani R, Caron J, Harrigan M, Carlson A, Bulters D, LeDoux D, Huang J, Cobb C, Gupta G, Kitagawa R, Chicoine M, Patel H, Dodd R, Camarata P, Wolfe S, Stadnik A, Money P, Mitchell P, Sarabia R, Harnof S, Barzo P, Unterberg A, Teitelbaum J, Wang W, Anderson C, Mendelow A, Gregson B, Janis S, Vespa P, Ziai W, Zuccarello M, Awad I, Abdul-Rahim A, Abou-Hamden A, Abraham M, Ahmed A, Alba C, Aldrich E, Altschul D, Amin-Hanjani S, Anderson D, Ansari S, Antezana D, Ardelt A, Arikan F, Baguena M, Baker A, Barrer S, Becker K, Bergman T, Boström A, Braun J, Brindley P, Broaddus W, Brown R, Buki A, Cao B, Cao Y, Carrion-Penagos J, Chalela J, Chang T, Chorro I, Chowdhry S, Corral L, Csiba L, Davies J, Díaz A, Derdeyn C, Diringer M, Dlugash R, Ecker R, Economas T, Enriquez P, Ezer E, Fan Y, Feng H, Franz D, Freeman W, Fusco M, Galicich W, Gelea M, Goldstein J, Gonzalez A, Grabarits C, Greenberg S, Gress D, Gu E, Hall C, Hernandez F, Hoesch R, Hoh B, Houser J, Hu R, Huang Y, Hussain M, Insinga S, Jadhav A, Jaffe J, Jahromi B, Jallo J, James M, James R, Jankowitz B, Jeon E, Jichici D, Jonczak K, Jonker B, Karlen N, Keric N, Kerz T, Knopman J, Koenig C, Krishnamurthy S, Kumar A, Kureshi I, Laidlaw J, Lakhanpal A, Latorre J, Leifer D, Leiphart J, Lenington S, Li Y, Lopez G, Lovick D, Lumenta C, Luo J, Maas M, MacDonald J, MacKenzie L, Madan V, Majkowski R, Major O, Malhorta R, Malkoff M, Mangat H, Maswadeh A, Matouk C, McArthur K, McCaul S, Medow J, Mezey G, Mighty J, Miller D, Mohan K, Muir K, Muñoz L, Nakaji P, Nee A, Nekoovaght-Tak S, Nyquist P, O'Kane R, Okasha M, O'Kelly C, Ostapkovich N, Pandey A, Parry-Jones A, Perla K, Pollack A, Polster S, Pouratian N, Quinn T, Rajajee V, Reddy K, Rehman M, Reimer R, Rincon F, Rybinnik I, Sanchez B, Sansing L, Schneck M, Schuerer L, Schul D, Schweitzer J, Seder D, Seyfried D, Sheth K, Spiotta A, Stechison M, Szabo K, Tamayo G, Tanczos K, Taussky P, Terry J, Testai F, Thomas K, Thompson C, Thompson G, Torner J, Tran H, Tucker K, Ungar L, Varelas P, Vargas N, Vatter H, Venkatasubramanian C, Vermillion K, Vollmer D, Wang Y, Wang Y, Wen J, Whitworth L, Willis B, Wrencher M, Wright S, Xu Y, Yanase L, Yi X, Yu Z, Zomorodi A (2019) Efficacy and safety of minimally invasive surgery with thrombolysis in intracerebral haemorrhage evacuation (MISTIE III): a randomised, controlled, open-label, blinded endpoint phase 3 trial. Lancet 393(10175):1021–1032
Helmy A, Timofeev I, Santarius T, Hutchinson P (2010) The utility of randomised control trials in neurosurgery. A response to “equipoise and randomisation in surgery”. Br J Neurosurg 24(1):98–99
Hemkens L, Contopoulos-Ioannidis D, Ioannidis J (2016) Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey. BMJ:i493
Hirst A, Philippou Y, Blazeby J, Campbell B, Campbell M, Feinberg J, Rovers M, Blencowe N, Pennell C, Quinn T, Rogers W, Cook J, Kolias A, Agha R, Dahm P, Sedrakyan A, McCulloch P (2019) No surgical innovation without evaluation: evolution and further development of the IDEAL framework and recommendations. Ann Surg 269(2):211–220
Hyam J, Pereira E, McCulloch P, Javed S, Plaha P, Mooney L, Forrow B, Joint C, Whone A, Gill S, Glasziou P, Aziz T, Green A (2015) Implementing novel trial methods to evaluate surgery for essential tremor. Br J Neurosurg 29(3):334–339
Kelly M, Malone D, Okun M, Booth J, Machado A (2014) Barriers to investigator-initiated deep brain stimulation and device research. Neurology 82(16):1465–1473
Khachane A, Philippou Y, Hirst A, McCulloch P (2018) Appraising the uptake and use of the IDEAL framework and recommendations: a review of the literature. Int J Surg 57:84–90
Kolias A, Chari A, Santarius T, Hutchinson P (2014) Chronic subdural haematoma: modern management and emerging therapies. Nat Rev Neurol 10(10):570–578
Kolias A, Adams H, Timofeev I, Czosnyka M, Corteen E, Pickard J, Turner C, Gregson B, Kirkpatrick P, Murray G, Menon D, Hutchinson P (2016) Decompressive craniectomy following traumatic brain injury: developing the evidence base. Br J Neurosurg 30(2):246–250
Kolias A, Viaroli E, Rubiano A, Adams H, Khan T, Gupta D, Adeleye A, Iaccarino C, Servadei F, Devi B, Hutchinson P (2018) The current status of decompressive craniectomy in traumatic brain injury. Current Trauma Reports 4(4):326–332
Kolias A, Marcus H, Broekman M, Hutchinson P, McCulloch P (2019) Surgery for intracerebral haemorrhage. The Lancet 394(10199):e21
Latouche E, Arena C, Ivey J, Garcia P, Pancotto T, Pavlisko N, Verbridge S, Davalos R, Rossmeisl J (2018) High-frequency irreversible electroporation for intracranial meningioma: a feasibility study in a spontaneous canine tumor model. Technology in Cancer Research & Treatment 17:153303381878528
Májovský M, Netuka D, Beneš V, Kucera P (2019) Burr-hole evacuation of chronic subdural hematoma: biophysically and evidence-based technique improvement. Journal of Neurosciences in Rural Practice 10(1):113
Marcus H, Hughes-Hallett A, Kwasnicki R, Darzi A, Yang G, Nandi D (2015) Technological innovation in neurosurgery: a quantitative study. J Neurosurg 123(1):174–181
Marcus H, Choi D, Dorward N (2017) Letter to the editor. da Vinci robot-assisted transoral surgery for sellar tumors. J Neurosurg 127(4):961–962
Marcus H, Williams S, Hughes-Hallett A, Camp S, Nandi D, Thorne L (2017) Predicting surgical outcome in patients with glioblastoma multiforme using pre-operative magnetic resonance imaging: development and preliminary validation of a grading system. Neurosurg Rev 40(4):621–631
Marcus H, Vakharia V, Ourselin S, Duncan J, Tisdall M, Aquilina K (2018) Robot-assisted stereotactic brain biopsy: systematic review and bibliometric analysis. Childs Nerv Syst 34(7):1299–1309
Marcus H, Vakharia V, Sparks R, Rodionov R, Kitchen N, McEvoy A, Miserocchi A, Thorne L, Ourselin S, Duncan J (2019) Computer-assisted versus manual planning for stereotactic brain biopsy: retrospective comparative pilot study. J Neurol Neurosurg Psychiatry 90(3):e5.3–e5e4
Martin E, Muskens I, Senders J, DiRisio A, Karhade A, Zaidi H, Moojen W, Peul W, Smith T, Broekman M (2019) Randomized controlled trials comparing surgery to non-operative management in neurosurgery: a systematic review. Acta Neurochir 161(4):627–634
McCulloch P, Altman D, Campbell W, Flum D, Glasziou P, Marshall J, Nicholl J (2009) No surgical innovation without evaluation: the IDEAL recommendations. Lancet 374(9695):105–1112
McCulloch P, Cook J, Altman D, Heneghan C, Diener M (2013) IDEAL framework for surgical innovation 1: the idea and development stages. BMJ 346(jun18 3):f3012–f3012
Moojen W, Peul W (2015) Minimally invasive surgery for lumbar spinal stenosis. BMJ 350(apr01 5):h1664
Moojen W, Bredenoord A, Viergever R, Peul W (2014) Scientific evaluation of spinal implants. Spine 39(26):2115–2118
Mortini P (2014) Cons: endoscopic endonasal transsphenoidal pituitary surgery is not superior to microscopic transsphenoidal surgery for pituitary adenomas. Endocrine 47(2):415–420
Mortini P, Barzaghi L, Albano L, Panni P, Losa M (2017) Microsurgical therapy of pituitary adenomas. Endocrine 59(1):72–81
Muskens I, Broekman M (2017) Innovation in neurosurgery—response to: “IDEAL”, the operating microscope, and the parachute. Acta Neurochir 160(2):369–370
Muskens I, Diederen S, Senders J, Zamanipoor Najafabadi A, van Furth W, May A, Smith T, Bredenoord A, Broekman M (2017) Innovation in neurosurgery: less than IDEAL? A systematic review. Acta Neurochir 159(10):1957–1966
Muskens I, Senders J, Dasenbrock H, Smith T, Broekman M (2017) The Woven EndoBridge device for treatment of intracranial aneurysms: a systematic review. World Neurosurgery 98:809–817.e1
Muskens I, Broekman M, Lycklama à Nijeholt G, Moojen W (2019) Letter: Food and drug administration approval of the Woven EndoBridge device: what do the numbers tell us? Neurosurgery 85(3):E619–E620
Nerland U, Jakola A, Solheim O, Weber C, Rao V, Lonne G, Solberg T, Salvesen O, Carlsen S, Nygaard O, Gulati S (2015) Minimally invasive decompression versus open laminectomy for central stenosis of the lumbar spine: pragmatic comparative effectiveness study. BMJ 350(apr01 1):h1603
Radic J, Illes J, McDonald P (2019) Fetal repair of open neural tube defects: ethical, legal, and social issues. Camb Q Healthc Ethics 28(3):476–487
Rushton P, Siddique I, Crawford R, Birch N, Gibson M, Hutton M (2017) Magnetically controlled growing rods in the treatment of early-onset scoliosis. The Bone & Joint Journal 99-B(6):708–713
Sala F, Skinner S, Arle J, Constantini S, Deletis V, Kothbauer K, MacDonald D, Shils J, Soto F, Szelenyi A (2018) Letter: guidelines for the use of electrophysiological monitoring for surgery of the human spinal column and spinal cord. Neurosurgery 83(2):E82–E84
Sciacca S, Smith J, Akram H, Asim A, Matharu M, Watkins L (2014) Rechargeable occipital nerve stimulator systems: a patient satisfaction study. Br J Neurosurg 28(5):645–649
Sedrakyan A, Campbell B, Merino J, Kuntz R, Hirst A, McCulloch P (2016) IDEAL-D: a rational framework for evaluating and regulating the use of medical devices. BMJ:2372
Tradewell M, Albersheim J, Dahm P (2019) Use of the IDEAL framework in the urological literature: where are we in 2018? BJU Int 123(6):1078–1085
Vakharia V, Rodionov R, McEvoy A, Miserocchi A, Sparks R, O’Keeffe A, Ourselin S, Duncan J (2018) Improving patient safety during introduction of novel medical devices through cumulative summation analysis. J Neurosurg 130(1):213–219
van Essen T, den Boogert H, Cnossen M, de Ruiter G, Haitsma I, Polinder S, Steyerberg E, Menon D, Maas A, Lingsma H, Peul W (2018) Variation in neurosurgical management of traumatic brain injury: a survey in 68 centers participating in the CENTER-TBI study. Acta Neurochir 161(3):435–449
Versteeg A, van der Velden J, Hes J, Eppinga W, Kasperts N, Verkooijen H, Oner F, Seravalli E, Verlaan J (2018) Stereotactic radiotherapy followed by surgical stabilization within 24 h for unstable spinal metastases; a stage I/IIa study according to the IDEAL framework. Frontiers in Oncology:8
Waqas M, Vakhari K, Weimer P, Hashmi E, Davies J, Siddiqui A (2019) Safety and effectiveness of embolization for chronic subdural hematoma: systematic review and case series. World Neurosurgery 126:228–236
Weber C, Jakola A, Gulati S, Nygaard Ø, Solheim O (2013) Evidence-based clinical management and utilization of new technology in European neurosurgery. Acta Neurochir 155(4):747–754
Weil R (2004) The future of surgical research. PLoS Med 1(1):e13
Williams M (2011) No endovascular innovation without evaluation in chronic cerebrospinal venous insufficiency. Arch Neurol 68(12):1510
Zaki M, Cote D, Muskens I, Smith T, Broekman M (2018) Defining innovation in neurosurgery: results from an international survey. World Neurosurgery 114:e1038–e1048
Zakrzewska J, Akram H (2011) Neurosurgical interventions for the treatment of classical trigeminal neuralgia. Cochrane Database Syst Rev
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
PH is supported by a Research Professorship from the National Institute for Health Research (NIHR), the NIHR Cambridge Biomedical Research Centre, an NIHR Senior Investigator award, a European Union Seventh Framework Program grant (CENTER-TBI; grant no. 602150), and the Royal College of Surgeons of England. AK is supported by a Clinical Lectureship, School of Clinical Medicine, University of Cambridge and the Royal College of Surgeons of England.
All other authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers’ bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements) in the subject matter or materials discussed in this manuscript. A number of authors are affiliated with the IDEAL collaboration, and therefore have a professional relationship with an organization with non-financial interest in this subject matter.
Ethical approval
For this type of study formal consent is not required.
Additional information
Comments
The IDEAL framework represents a set of steps whose goal is that of generating sound innovations in Surgery while assuring adequate ethics and patient awareness compliance. These recommendations were first published in 2009 following a series of meetings at Balliol College, Oxford, during 2007 to 2009 to discuss the specific challenges of evaluating surgical innovation (1).
A review of the evolving IDEAL recommendations is provided by Hirst et al. in 2019 (2). The paper by CU Ota et al analyzes the penetration of the Ideal framework in Neurosurgical publications. It identifies primary papers where an attempt is made to follow Ideal recommendations in the construction of the study and secondary papers in which the Ideal recommendations are discussed in the text or simply mentioned in the Reference section.
The results show a low penetration of the IDEAL instrument with 14 primary papers and 37 secondary papers representing 4% of a total of 1336 articles evaluated.
The authors suggest a number of reasons for this low uptake including overreliance on time-honored retrospective case series and lack of awareness of the IDEAL concept, including a few studies which in practicality conform to IDEAL recommendations but fail to mention them. While these explanations may play a more or less relevant role, a perhaps bigger role may be linked to the over-enthusiasm of the surgical innovator for her/his innovation that might not be validated if a more stringent evaluation, such as the IDEAL one, was to be sought.
What to do about the current situation? Demand more accountability on the proposed innovations, highlighting what the proposed innovation is going to ameliorate and what is the process chosen to validate it.
In this context, a more widespread use of the Ideal recommendations could be very useful.
We fully agree with the authors that a more robust education of the Neurosurgical community on the concept of the IDEAL recommendations is warranted and, we add, should be actively pursued by Neurosurgical Societies worldwide.
Mario Ammirati
PA, USA
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article is part of the Topical Collection on Neurosurgical technique evaluation
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Ota, H.C.U., Smith, B.G., Alamri, A. et al. The IDEAL framework in neurosurgery: a bibliometric analysis. Acta Neurochir 162, 2939–2947 (2020). https://doi.org/10.1007/s00701-020-04477-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00701-020-04477-5