Abstract
Background
Although the provision of performance feedback to healthcare professionals based on data from quality registries is common practice in many fields of medicine, observational studies of its effect on the quality of care have shown mixed results. The objective of this study is to evaluate the effect of performance feedback on the quality of care for acute ischemic stroke.
Methods
PERFEQTOS is a stepped wedge cluster randomized trial in 13 hospitals in the Netherlands providing endovascular thrombectomy for ischemic stroke. The primary outcome is the hospital’s door-to-groin time. The study starts with a 6-month period in which none of the hospitals receives the performance feedback intervention. Subsequently, every 6 months, three or four hospitals are randomized to cross over from the control to the intervention conditions, until all hospitals receive the feedback intervention. The feedback intervention consists of a dashboard with quarterly reports on patient characteristics, structure, process, and outcome indicators related to patients with ischemic stroke treated with endovascular thrombectomy. Hospitals can compare their present performance with their own performance in the past and with other hospitals. The performance feedback is provided to local quality improvement teams in each hospital, who define their own targets on specific indicators and develop performance improvement plans. The impact of the performance feedback and improvement plans will be evaluated by comparing the primary outcome before and after the intervention.
Discussion
This study will provide evidence on the effectiveness of performance feedback to healthcare providers. The results will be actively disseminated through peer-reviewed journals, conference presentations, and various stakeholder engagement activities.
Trial registration
Netherlands Trial Register NL9090. Registered on December 3, 2020
Similar content being viewed by others
Introduction
Background and rationale
In 2019, more than 30,000 ischemic stroke patients were admitted to Dutch hospitals and this number is increasing every year [1]. Ischemic stroke is a major cause of death and long-term disability [2]. Randomized trials showed that early endovascular thrombectomy (EVT) substantially improves the 3-month functional outcome with an absolute risk reduction in terms of death or permanent disability of 19.5% [3,4,5]. With EVT, the neuro-interventionalist advances a catheter through the femoral artery up to the occluded cerebral artery to remove the occlusion of blood clots. The effectiveness of EVT is highly time-dependent. Every hour delay in the initiation of EVT results in death or permanent disability in 1 out of every 19 patients [6]. Several trials and observational studies noted an association between an efficient workflow to achieve fast recanalization and a stronger treatment effect of EVT resulting in better clinical outcomes [7,8,9,10,11]. Therefore, having the right infrastructure and an efficient process of care is of utmost importance to be able to treat every patient as fast as possible.
Performance feedback has been defined as “a summary of clinical performance on a specific indicator, e.g. process measure, with benchmarking against performance of other providers over a specified period of time with or without recommendations for action” [12,13,14,15]. Providing performance feedback regarding process indicators to healthcare professionals has become quite common in healthcare [12, 16]. However, performance feedback has no firm empirical basis and consensus on how this feedback is best to be provided is lacking [12]. A Cochrane systematic review of 140 studies on the effectiveness of audit and performance feedback on the quality of care reported a median 4.3% absolute improvement in patient outcomes (interquartile range 0.5–16%) [12]. However, effects tend to be very heterogeneous, and success factors for the design and delivery of effective performance feedback have not been identified. Understanding heterogeneity of effect has been limited in part by the lack of appropriate design of the feedback interventions and the absence of a quantitative evaluation of effectiveness [17,18,19,20]. The aim of this study is to evaluate the effect of providing performance feedback to healthcare providers in individual EVT hospitals to improve the quality of stroke care.
Objectives
Since stroke logistics and time metrics vary considerably between centers and within centers, and are strongly associated with outcome [21], this study aims to assess the extent to which performance feedback to healthcare providers in individual hospitals providing EVT for ischemic stroke, resulting in action plans and targets based on this feedback, reduces time from arrival at the hospital to initiation of EVT and thereby improves the quality of care.
Methods
Trial design
We designed a stepped wedge cluster randomized trial (CRT) of performance feedback on the quality of care in hospitals performing EVT for ischemic stroke (PERFEQTOS) to determine the impact of a performance feedback intervention accompanying hospital-specific action plans and improvement targets. This is a specific form of CRT in which initially all clusters (in this study: hospitals) serve as controls. The intervention is rolled out sequentially but randomly at different time points, such that at the end of the study all clusters have crossed over to the intervention condition (Fig. 1) [22]. In the present study, the design includes an initial 6-month period in which none of the hospitals receives performance feedback. Subsequently, every 6 months, three or four randomly chosen hospitals will cross over from control to intervention conditions, which implies a total study duration of 30 months given that 13 hospitals participate. The Standard Protocol Items: Recommendation for Interventional Trials (SPIRIT) checklist [23] is shown in Additional file 1.
Study setting and participants
The study will be carried out in specialized neuro-intervention hospitals performing EVT for ischemic stroke. In the Netherlands, EVT is concentrated in 17 such specialized hospitals [24]. All EVT hospitals are invited to participate in this trial with no specific inclusion or exclusion criteria. Thirteen out of these 17 hospitals agreed to participate in the study (Fig. 2). The participating hospitals include all admitted adult patients with acute ischemic stroke who underwent EVT.
Randomization and intervention allocation
Randomization will be performed according to a lottery method of sampling by the Study Coordinator in the presence of the Principal Investigator and an independent observer and will be videotaped. Once the hospitals provided consent to participate in the study, each enrolled hospital will start in the control condition. Every 6 months, three to four hospitals will be randomized to cross over to the intervention condition. All other hospitals stay in the control condition. This procedure continues every 6 months until all hospitals are crossed over to the intervention condition. Participating hospitals will be blinded to the allocation sequence, and those hospitals not yet in the intervention condition will not be aware of the time at which they will cross over to the intervention condition. When a hospital is crossed over to the intervention, this is communicated to all participating hospitals.
Intervention
Dashboard design
When it comes to performance feedback, it is necessary to visualize and communicate the information content in such a way that care providers can use this information for improvement and to maximally reduce the risk of misinterpretation of the results [25]. With this in mind, we developed a dashboard containing quality of care measures per hospital, benchmarked against the average/median performance of the other hospitals for the same time period as well as their own performance over time. Prior to designing this dashboard, we reviewed the general empirical literature on dashboard design [25,26,27,28,29,30,31,32]. Based on the insights from previous studies, we composed a dashboard containing a compositional qualitative and quantitative (graphical display with textual explanation) visualization of quality of care data. Specifically, the PERFEQTOS dashboard provides an overview of results on quality of care indicators and patient characteristics (Fig. 3). The performance feedback is presented in a way that clearly highlights the key message (i.e., improvement is recommended or not), limiting the amount of extra information to increase actionability while still allowing the recipient hospital to view more detailed comparative information.
Indicator measurement for the dashboard
Evaluation of healthcare providers’ quality performance is commonly done using a combination of structure, process, and outcome indicators (Table 1) [33, 34]. Our dashboard follows that approach.
Structure indicators are the total number of patients treated with EVT in each hospital (henceforth referred to as “center volume”), the number of direct (i.e., non-transferred) and transferred stroke patients from other hospitals, and completeness of data.
Process indicators are time from arrival at the emergency department of the intervention hospital to start of the intravenous thrombolytics (IVT) administration (door-to-needle time) for non-transferred patients and time from arrival at the emergency department of the intervention hospital to initiation of EVT (door-to-groin time) for all patients. In case a patient has an ischemic stroke while already admitted to the hospital, then the arrival time is defined as the time the neurologist first assessed the patient. We stratified the indicators by whether patients are transferred or non-transferred from another hospital.
Outcome indicators are post-EVT reperfusion grade of the Extended Thrombolysis in Cerebral Infarction (eTICI) score, ranging from 0 (no reperfusion) to 3 (full reperfusion), and the National Institute of Health Stroke Scale (NIHSS) score at 24 h (± 12 h), which quantifies the neurological deficit caused by a stroke (range 0–42). In addition, the modified Rankin Scale (mRS) score is used as a measure of patients’ functional outcome after acute ischemic stroke, ranging from 0 (no symptoms) to 6 (death). The mRS score is assessed 3 months after admission. Since the outcome is strongly associated with baseline patient characteristics [35], patients’ age, sex, NIHSS score on arrival, location of the proximal intracranial occlusion, and time from onset to arrival at the intervention hospital (onset-to-door time) are also collected. Stroke onset is defined as the time point when stroke symptoms were first noticed by the patient or an observer. In cases the time of first symptoms is unknown, onset is defined as the moment the patient was last seen well. Additionally, we adjusted the outcomes per hospital for differences in these baseline characteristics.
Strategy for quality improvement
The performance improvement cycle is depicted in Fig. 4 to reflect the procedures through which hospitals aim to improve their clinical performance. Each hospital randomized to the intervention will install a local quality improvement team. This team consists at least of a neurologist, an interventionalist, a resident in neurology, and a (stroke) nurse. It can be expanded with representatives of other relevant disciplines. Each quality improvement team is trained by one of the PERFEQTOS investigators to explain the dashboard functionalities, interpretation of data, and how action plans can be developed. Quality improvement teams of hospitals in the intervention cluster periods receive quarterly performance feedback reports, which they can use to set improvement targets pertaining to specific indicators and develop performance improvement plans to achieve those targets. The assumption is that if a center’s performance is below that of the comparator, an improvement target will be set accordingly, and subsequent improvement actions would be aimed at reaching the target and remove the discrepancy [16, 36, 37]. The impact of these actions can then be evaluated based on the next feedback report(s).
In addition, during workshops organized every 6 months for those hospitals randomized to the intervention, both the best performing hospital and most improved hospital share their best practices with the other hospitals.
Data collection and management
Data on performance indicators and patient characteristics are routinely collected in each hospital and reported to the Dutch Acute Stroke Audit (DASA) from the Dutch Institute of Clinical Auditing (DICA). DASA is the main prospective clinical auditing tool for stroke in The Netherlands since 2014, with the aim to assure the quality of patient care and to aid in improving outcomes [38]. DICA is an independent organization, founded by medical specialists, that facilitates national audits for various medical professions, including the DASA. Hospitals are free to decide who carries out the data registration (e.g., nurses, data managers, neurologists), but the final responsible person is a neurologist. Medical Research Data Management (MRDM) processor, a trusted third party, is involved to pseudonymize the data to comply with privacy legislation. Hospitals can provide the collected data to MRDM in three ways. First, an online survey through a secured web environment is available for hospitals to record the data. Second, hospitals can provide the data in batches, i.e., data files in which large amounts of data can be transferred directly to MRDM. Third, to minimize the registration burden, some hospitals took the initiative to implement data linkage, i.e., extracting the data from their individual electronic patient health record to be automatically forwarded to MRDM [38]. Participating hospitals approved with a written agreement that MRDM send pseudo raw data about the patients treated in their hospital to the PERFEQTOS Study Coordinator at Erasmus MC. The Study Coordinator then aggregates the data into hospital-level medians and percentages. Data collection continues throughout the study, so each hospital contributes to both control and intervention cluster periods, which will be compared. Next, the aggregated data are summarized in a report and fed back via the dashboard. All hospitals randomized to the intervention group will receive a performance feedback report on a quarterly basis and will start and re-iterate the performance improvement cycle (Fig. 4).
Study outcomes
The primary outcome is door-to-groin time. Secondary outcomes are door-to-needle time, eTICI score, NIHSS at 24 h, and mRS at 3 months (Table 1). These outcomes are collected for every individual patient (Table 1).
Power calculation
The power calculation was based on mean differences in door-to-groin time as a primary outcome. We used a parametric power estimation methodology for stepped wedge designs put forward by Hemming and Girling [39] in the Stata function stepped wedge, derived from Hussy and Hughes [40]. We assumed 13 EVT centers were randomized in four clusters per time step (three clusters of three hospitals and one cluster of four hospitals) that treated an average of 30 patients per center per time period of 3 months [21]. We used a mean door-to-groin time of 77 min (standard deviation 47 min) and ICC 0.37, both obtained from the MR CLEAN Registry data [21, 24]. Assuming a significance level of 5%, this would result in 88% power to detect a clinically relevant reduction of door-to-groin time of 10 min.
Statistical methods
The effect of the intervention on the primary and secondary outcomes will be assessed by comparing median/percentage outcomes between control and intervention cluster periods and also in the time trends of intervention cluster periods using a non-parametric Kruskal-Wallis test and Pearson’s chi-square statistic, respectively.
Generalized linear mixed modeling at the individual patient level will be used for analyzing the effect of the performance feedback intervention on the primary outcome (door-to-groin time). This will allow us to appropriately account for the different observation cluster periods (both control and intervention) and the hierarchical structure of the data [40]. The model will contain fixed effects for intervention (yes/no), calendar time (month) to account for autonomous time trends, patient characteristics (i.e., patients’ age, sex, NIHSS score at arrival, location of the proximal intracranial occlusion, and onset-to-door time) [40], and a random effect for the hospital. This model enables us to estimate the variance of the outcome of interest (door-to-groin time) at the hospital level (inter-cluster variation) and individual level (intra-cluster variation), which in turn enables us to estimate the total effect of the performance feedback intervention. The increase in the variance due to the clustering will be quantified by the variance inflation factor. In sensitivity analyses, we will estimate the impact of the duration of exposure to the performance feedback intervention, the effect of moving from control to the intervention condition, and/or interaction of both factors on the primary outcome [41].
Multivariable normal models will be used for imputation of missing values, using available data on patient characteristics, structure, and care processes.
Oversight and data monitoring
The trial Executive Committee consists of one Principal Investigator, one junior Study Coordinator, two researchers affiliated to DICA, three vascular neurologists, one neuro-interventionalist, one radiologist, and two trial methodologists. The Trial Steering Committee is the main decision-making body. It consists of members of the Executive Committee and all local Principal Investigators. The Steering Committee meets at least once a year.
All incoming data are reviewed by the Study Coordinator at the central trial office. All pseudo data from DASA are stored on the Erasmus MC’s secure server, which is only accessible by the Study Coordinator and Principal Investigator.
This study will be conducted in accordance with the principles of Good Clinical Practice, the Dutch Agreement on Medical Treatment Act (WGBO), and the European General Data Protection Regulation. All patients will receive the best medical treatment according to national and local guidelines and current insights. We use data collection exclusively for improving quality of care purposes. To protect patients’ privacy, the trusted third party MRDM processes the data on behalf of the healthcare providers in such a way that the Study Coordinator receives only pseudonymized patient data, meaning that personal data has been processed in such a way that they can no longer be linked to a specific person. These additional data are stored separately, and technical and organizational measures are taken to ensure that personal data can never be linked to a specific person.
This trial does focus on (performance feedback about) the treatment of acute ischemic stroke. No reason to assume that the ancillary or post-trial care varies between participating hospitals or is influenced by the intervention tested in this trial. Therefore, we do not expect this to confound the trial’s results.
Dissemination policy
The Executive Committee forms the Writing Committee for the trial. Publications will be made on behalf of all investigators. The main study results will be disseminated via publication in an international peer-reviewed journal and presentation at international conferences for health provider specialists. Representatives of the participating hospitals will be given the opportunity to comment on the manuscript and participate as co-authors. We plan to disseminate the results of the planned secondary analyses on the feasibility and effectiveness of performance feedback in one or more separate papers.
Discussion
PERFEQTOS is a stepped wedge cluster randomized trial about the effect of performance feedback on the quality of acute ischemic stroke care. This performance feedback consists of a multifaceted intervention, including the implementation of a quality indicator dashboard, quality improvement teams, and performance improvement plans. The primary outcome is a process measure (door-to-groin time) which is actionable and strongly associated with clinical outcomes [6].
Previously, it has been noticed that a stepped wedge CRT design is more efficient when the intra-cluster correlation (ICC) is moderate or high (ICC ≥ 0.1) [42, 43]. Generally, ICC tends to be higher for process indicators in comparison to clinical outcomes, which is related to the fact that processes can be more easily influenced by providers than outcomes [43]. Therefore, since our main focus is on improving care processes through performance feedback, a stepped wedge CRT design is an efficient design for this study. Another strength of the stepped wedge CRT design is that it provides an opportunity to measure the effect of duration of exposure to the intervention as well as of underlying temporal changes. In order to take advantage of this strength, the longitudinal time intervals should be sufficiently broad [40]. We hypothesize 6 months is long enough to achieve an effect based on treatment outcome measurements (e.g., functional outcome at 3 months after stroke). This design helps us also to tackle difficulties of implementation of the intervention at all hospitals at once. Additionally, the intervention effect in all hospitals can be evaluated with different time exposures (6–18 months).
The (design of the) current study has some limitations. First, given that our intervention is multifaceted, it may be difficult to disentangle the relative impact of the different aspects of the intervention. However, an intervention with a combination of different strategies is more effective than approaches using a single intervention [44]. Second, the quality improvement policies of hospitals in the control phase are heterogeneous and this may influence the effect of our intervention. To facilitate the interpretation of our results, we will use semi-annual questionnaires to obtain insight into the quality improvement policies of the hospitals in the control arm. Third, outcome assessment is not blinded in our trial. However, the primary outcome (door-to-groin time) is an objective measure and therefore unlikely to bias the trial results. The assessment of the secondary outcomes NIHSS at 24 h and mRS at 3 months is not blinded as well, but because these outcomes are routinely collected through a structured algorithm, we expect the risk of bias to be limited.
Findings from this study will provide insight into the feasibility and effectiveness of structured performance feedback in reducing the time to treatment of patients with ischemic stroke treated with endovascular thrombectomy and, thereby, in improving patient outcomes. If the expected results are realized, the developed method for performance feedback is ready for wider implementation on a national and international level and can be adapted for use in other diseases.
Trial status
Protocol version number: 2, August 2020. PERFEQTOS has started in January 2020, and the anticipated study duration is 30 months.
Availability of data and materials
Data cannot be made available, as no patient approval has been obtained for sharing coded data. However, syntax files and output of statistical analyses will be made available upon request after the trial is finalized.
Abbreviations
- CRT:
-
Cluster randomized trial
- DASA:
-
Dutch Acute Stroke Audit
- DICA:
-
Dutch Institute of Clinical Auditing
- eTICI:
-
Extended thrombolysis in cerebral infarction
- EVT:
-
Endovascular thrombectomy
- ICC:
-
Intra-cluster correlation coefficient
- IVT:
-
Intravenous thrombolytics
- MRDM:
-
Medical Research Data Management
- mRS:
-
Modified Rankin Scale
- NIHSS:
-
National Institute of Health Stroke Scale
References
De Boer AR, Van Dis I, Wimmers RH, Vaartjes I, Bots ML. Cardiovascular diseases in the Netherlands. Dutch Heart Foundation. 2020. https://nederlandsehartregistratie.nl/hartenvaatcijfers/wp-content/uploads/2020/12/HVZ_in_NL_2020.pdfatie.nl.
Donkor ES. Stroke in the 21(st) century: a snapshot of the burden, epidemiology, and quality of life. Stroke Res Treat. 2018;2018:3238165.
Vidale S, Agostoni E. Endovascular treatment of ischemic stroke: an updated meta-analysis of efficacy and safety. Vasc Endovascular Surg. 2017;51(4):215–9. https://doi.org/10.1177/1538574417698905.
Goyal M, Menon BK, van Zwam WH, Dippel DW, Mitchell PJ, Demchuk AM, et al. Endovascular thrombectomy after large-vessel ischaemic stroke: a meta-analysis of individual patient data from five randomised trials. Lancet (London, England). 2016;387(10029):1723–31.
Berkhemer OA, Fransen PS, Beumer D, van den Berg LA, Lingsma HF, Yoo AJ, et al. A randomized trial of intraarterial treatment for acute ischemic stroke. N Engl J Med. 2015;372(1):11–20. https://doi.org/10.1056/NEJMoa1411587.
Saver JL, Goyal M, van der Lugt A, Menon BK, Majoie CB, Dippel DW, et al. Time to treatment with endovascular thrombectomy and outcomes from ischemic stroke: a meta-analysis. Jama. 2016;316(12):1279–88. https://doi.org/10.1001/jama.2016.13647.
Goyal M, Almekhlafi MA, Fan L, Menon BK, Demchuk AM, Yeatts SD, et al. Evaluation of interval times from onset to reperfusion in patients undergoing endovascular therapy in the Interventional Management of Stroke III trial. Circulation. 2014;130(3):265–72. https://doi.org/10.1161/CIRCULATIONAHA.113.007826.
Menon BK, Almekhlafi MA, Pereira VM, Gralla J, Bonafe A, Davalos A, et al. Optimal workflow and process-based performance measures for endovascular therapy in acute ischemic stroke: analysis of the Solitaire FR thrombectomy for acute revascularization study. Stroke. 2014;45(7):2024–9. https://doi.org/10.1161/STROKEAHA.114.005050.
Demchuk AM, Goyal M, Menon BK, Eesa M, Ryckborst KJ, Kamal N, et al. Endovascular treatment for Small Core and Anterior circulation Proximal occlusion with Emphasis on minimizing CT to recanalization times (ESCAPE) trial: methodology. Int J Stroke. 2015;10(3):429–38. https://doi.org/10.1111/ijs.12424.
Mulder M, Jansen IGH, Goldhoorn RB, Venema E, Chalos V, Compagne KCJ, et al. Time to endovascular treatment and outcome in acute ischemic stroke: MR CLEAN registry results. Circulation. 2018;138(3):232–40. https://doi.org/10.1161/CIRCULATIONAHA.117.032600.
Fransen PS, Berkhemer OA, Lingsma HF, Beumer D, van den Berg LA, Yoo AJ, et al. Time to reperfusion and treatment effect for acute ischemic stroke: a randomized clinical trial. JAMA Neurol. 2016;73(2):190–6. https://doi.org/10.1001/jamaneurol.2015.3886.
Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6:CD000259. https://doi.org/10.1002/14651858.CD000259.pub3.
Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. Jama. 2006;296(9):1094–102. https://doi.org/10.1001/jama.296.9.1094.
Porter ME. What is value in health care? N Engl J Med. 2010;363(26):2477–81. https://doi.org/10.1056/NEJMp1011024.
Porter ME, Baron JF, Chacko JM, Tang RJ. The UCLA Medical Center: kidney transplantation. Harvard: Business School Case 9-711-410 Boston: Harvard Business School Publishing. 2012.
van der Veer SN, de Keizer NF, Ravelli AC, Tenkink S, Jager KJ. Improving quality of care. A systematic review on how medical registries provide information feedback to health care providers. Int J Med Inform. 2010;79(5):305–23. https://doi.org/10.1016/j.ijmedinf.2010.01.011.
Colquhoun HL, Brehaut JC, Sales A, Ivers N, Grimshaw J, Michie S, et al. A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci. 2013;8:66.
Sales A, Smith J, Curran G, Kochevar L. Models, strategies, and tools. Theory in implementing evidence-based findings into health care practice. J Gen Intern Med. 2006;21(Suppl 2):S43–9.
Ivers NM, Sales A, Colquhoun H, Michie S, Foy R, Francis JJ, et al. No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci. 2014;9:14.
Gude WT, Brown B, van der Veer SN, Colquhoun HL, Ivers NM, Brehaut JC, et al. Clinical performance comparators in audit and feedback: a review of theory and evidence. Implement Sci. 2019;14(1):39.
Amini M, van Leeuwen N, Eijkenaar F, MJHL M, Schonewille W, Lycklama à Nijeholt G, et al. Improving quality of stroke care through benchmarking center performance: why focusing on outcomes is not enough. BMC Health Serv Res. 2020;20(1):998.
Brown CA, Lilford RJ. The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006;6(1):54. https://doi.org/10.1186/1471-2288-6-54.
Chan A-W, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. Br Med J. 2013;346:e7586. https://doi.org/10.1136/bmj.e7586.
Jansen IGH, Mulder M, Goldhoorn RB, investigators MCR. Endovascular treatment for acute ischaemic stroke in routine clinical practice: prospective, observational cohort study (MR CLEAN Registry). BMJ (Clinical research ed). 2018;360:k949.
Brust-Renck PG, Royer CE, Reyna VF. Communicating numerical risk: human factors that aid understanding in health c. Rev Hum Factors Ergon. 2013;8(1):235–76. https://doi.org/10.1177/1557234X13492980.
Bantug ET, Coles T, Smith KC, Snyder CF, Rouette J, Brundage MD, et al. Graphical displays of patient-reported outcomes (PRO) for use in clinical practice: what makes a pro picture worth a thousand words? Patient Educ Couns. 2016;99(4):483–90. https://doi.org/10.1016/j.pec.2015.10.027.
Garcia-Retamero R, Cokely ET. Designing visual aids that promote risk literacy: a systematic review of health research and evidence-based design heuristics. Hum Factors. 2017;59(4):582–627. https://doi.org/10.1177/0018720817690634.
Kurtzman ET, Greene J. Effective presentation of health care performance information for consumer decision making: a systematic review. Patient Educ Couns. 2016;99(1):36–43. https://doi.org/10.1016/j.pec.2015.07.030.
Okan Y, Galesic M, Garcia-Retamero R. How people with low and high graph literacy process health graphs: evidence from eye-tracking. J Behav Decis Mak. 2016;29(2-3):271–94. https://doi.org/10.1002/bdm.1891.
Okan Y, Garcia-Retamero R, Galesic M, Cokely ET. When higher bars are not larger quantities: on individual differences in the use of spatial information in graph comprehension. Spatial Cognition Computation. 2012;12(2-3):195–218. https://doi.org/10.1080/13875868.2012.659302.
Stone ER, Gabard AR, Groves AE, Lipkus IM. Effects of numerical versus foreground-only icon displays on understanding of risk magnitudes. J Health Commun. 2015;20(10):1230–41. https://doi.org/10.1080/10810730.2015.1018594.
Hallgreen CE, Mt-Isa S, Lieftucht A, Phillips LD, Hughes D, Talbot S, et al. Literature review of visual representation of the results of benefit-risk assessments of medicinal products. Pharmacoepidemiol Drug Saf. 2016;25(3):238–50. https://doi.org/10.1002/pds.3880.
Lilford R, Mohammed MA, Spiegelhalter D, Thomson R. Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma. Lancet (London, England). 2004;363(9415):1147–54.
Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260(12):1743–8. https://doi.org/10.1001/jama.1988.03410120089033.
Lingsma HF, Dippel DW, Hoeks SE, Steyerberg EW, Franke CL, van Oostenbrugge RJ, et al. Variation between hospitals in patient outcome after stroke is only partly explained by differences in quality of care: results from the Netherlands Stroke Survey. J Neurol Neurosurg Psychiatry. 2008;79(8):888–94. https://doi.org/10.1136/jnnp.2007.137059.
Carver CS, Scheier MF. Control theory: a useful conceptual framework for personality-social, clinical, and health psychology. Psychol Bull. 1982;92(1):111–35. https://doi.org/10.1037/0033-2909.92.1.111.
Gardner B, Whittington C, McAteer J, Eccles MP, Michie S. Using theory to synthesise evidence from behaviour change interventions: the example of audit and feedback. Soc Sci Med. 2010;70(10):1618–25. https://doi.org/10.1016/j.socscimed.2010.01.039.
Kuhrij LS, Wouters MWJM, van den Berg-Vos RM, de Leeuw F-E, Nederkoorn PJ. The Dutch Acute Stroke Audit: benchmarking acute stroke care in the Netherlands. Eur Stroke J. 2018;3(4):361–8. https://doi.org/10.1177/2396987318787695.
Hemming K, Girling A. A menu-driven facility for power and detectable-difference calculations in stepped-wedge cluster-randomized trials. Stata J. 2014;14(2):363–80. https://doi.org/10.1177/1536867X1401400208.
Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28(2):182–91. https://doi.org/10.1016/j.cct.2006.05.007.
Nickless A, Voysey M, Geddes J, Yu LM, Fanshawe TR. Mixed effects approach to the analysis of the stepped wedge cluster randomised trial-Investigating the confounding effect of time through simulation. PLoS One. 2018;13(12):e0208876. https://doi.org/10.1371/journal.pone.0208876.
Campbell MK, Fayers PM, Grimshaw JM. Determinants of the intracluster correlation coefficient in cluster randomized trials: the case of implementation research. Clin Trials (London, England). 2005;2(2):99–107.
Hemming K, Taljaard M. Sample size calculations for stepped wedge and cluster randomised trials: a unified approach. J Clin Epidemiol. 2016;69:137–46. https://doi.org/10.1016/j.jclinepi.2015.08.015.
Kilkenny MF, Bravata DM. Quality improvement. Stroke. 2021;52(5):1866–70. https://doi.org/10.1161/STROKEAHA.121.033451.
Acknowledgements
This study protocol was published on behalf of the PERFEQTOS Investigators: Bob Roozenbeek; Sanne J. den Hartog; Diederik W.J. Dippel; Aad van der Lugt; Hester F. Lingsma; Nikki van Leeuwen; Laurien S. Kuhrij, Lotte J. Stolze; Paul J. Nederkoorn; Adriaan C.G.M. van Es; Ido R. van den Wijngaard; Paula M. Janssen; Pieter-Jan van Doormaal; Yvo B.W.E.M. Roos; Bart J. Emmer; Suzanne M. Silvis; Wouter Dinkelaar; Michel J.M. Remmers; Douwe L.D. Vos; Rob A.R. Gons; Lonneke S.F. Yo; Julia H. van Tuijl; Hans Kortman; Jelis Boiten, Geert J. Lycklama à Nijeholt; Jurgen R. Piet; Wouter Stomp; Heleen M. den Hertog; H. Zwenneke Flach; Nyika D. Kruyt; Marianne A.A. van Walderveen; Wim H. van Zwam; Robert J. van Oostenbrugge; Jasper M. Martens, Jeannette Hofmeijer; H. Bart van der Worp; Rob T.H. Lo.
Sponsor contact information
Trial sponsor: Erasmus MC, University Medical Center Rotterdam
Contact name: Bob Roozenbeek
Address: Department of Neurology
Erasmus MC, University Medical Center Rotterdam
Dr. Molewaterplein 40
3015 GD Rotterdam, the Netherlands
Email: b.roozenbeek@erasmusmc.nl
Funding
This work is supported by a grant from the Erasmus University Rotterdam, The Netherlands (Erasmus Initiative “Smarter Choices for Better Health,” Action Line “Value-Based Health Care”). Award/Grant number: not applicable.
This funding source had no role in the design of this study and will not have any role during its execution, analyses, interpretation of the data, or decision to submit results.
Author information
Authors and Affiliations
Consortia
Contributions
BR and HFL initially conceptualized the methodology. MA, SJdH, NvL, FE, LSK, LS, PJN, AMvE, IvdW, AvdL, and DWJD aided the development of the methodology. SJdH is collecting data and undertaking data analysis. MA and SJdH contributed to writing this manuscript. All authors contributed to the refinement of the study protocol and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
The Institutional Review Board of Erasmus MC University Medical Center has reviewed the study protocol and confirmed that the Dutch Medical Research Involving Human Subjects Act (WMO) is not applicable.
Consent for publication
Not applicable.
Competing interests
The authors and principal investigators of the trial declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Additional file 1.
SPIRIT Checklist.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Amini, M., den Hartog, S.J., van Leeuwen, N. et al. Performance feedback on the quality of care in hospitals performing thrombectomy for ischemic stroke (PERFEQTOS): protocol of a stepped wedge cluster randomized trial. Trials 22, 870 (2021). https://doi.org/10.1186/s13063-021-05819-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13063-021-05819-z