1 Introduction

Multidisciplinary tumor boards (TBs) provide an interdisciplinary approach for decision-making in cancer care [1]. TBs have evolved to become an integral part of cancer treatment planning [2] and are widely considered the “gold standard” of cancer care delivery [3]. In the United Kingdom (UK), TBs were mandated by the National Cancer Plan in 2000 [4]. However, the cancer care landscape has changed significantly due to increased patient numbers, more sophisticated diagnostic testing and development of personalized treatments [4]. Legacy approaches to preparing and conducting TBs are impractical and there are needs for solutions that can improve operational effectiveness and standardize processes [5].

Implementation of TBs require the joint effort of multiple hospital staff members [6,7,8]. Studies demonstrated that effective TBs depend on multiple inputs (individuals, teams, environment, and patients) and processes (interactions, tests, results) [3, 9]. Studies have identified practices to ensure effectiveness of TBs [3]. Best practices included good relationships between team members and effective conflict management; incorporating patient choice, psychosocial factors & comorbidities into decision-making; ensuring equality and inclusiveness of team participation (e.g., nursing staff); and, rotating chair duties within and between disciplines. Correspondingly, non-recommended practices included basing decisions primarily on biomedical information and unequal participation during discussions. Consistent with these academic recommendations, empirical evidence provided by the National Cancer Action Team in England [9] suggested additional factors, such as level of expertise and specialization; infrastructure (e.g., appropriate meeting room, availability of technology); organization (e.g. regular meetings); efficient logistics (e.g. preparation and scheduling); patient-centred clinical decision-making; and, robust team governance.

The delivery of TBs, however, often falls below expected best practices with common challenges, as well as specific issues related to hospital type and healthcare payer system. One challenge is insufficient time for case discussions, particularly for more complex patients, which can result in postponement of other patient cases [4, 10, 11]. It has been recently reported that half of all patients presented at UK TBs were discussed for 2 min only [4, 11]. Other studies reported average discussion time per case of between 1.8 and 7.6 min, with variations across different TBs (Table S3). With such limited time, it is questionable whether the full patient case (clinical history, comorbidities, psychosocial factors, pathology reports, radiological imaging, patient preference, disease stage and available clinical trials) can be considered by the TB. In addition, such fast & dense decision-making can lead to mental fatigue and decreased attention of clinicians [12].

Factors that influence decision-making include: lack of necessary information & incomplete data [13]; lack of consideration of patient comorbidities, choices, and disease progression [13]; non-attendance of key TB members [13]; unequal participation [10]; and, technological problems and switching between different IT systems [13]. Empirical studies reported that clinicians found existing approaches to TB conduction ineffective, wasteful and prone to error [14]. For example, paper handouts could be out-of-date and represented an information governance risk, whereas, electronic medical records (EMR) allowed real-time access to patient information, but additional time was required to find and navigate between key information [14].

In regions where TBs are mandated, it has become a priority to create adequate time for discussion, improve efficiency, enhance consistency, and to enable transparency of decision-making, in order to ensure the best use of clinical time [4]. Even in countries where TBs are not mandated the need for systems for effective coordination of cancer care have been identified [15]. Digital solutions are well positioned to meet these challenges.

The FDA considers digital health as a broad scope, which includes mobile health, health information technology, wearable devices, telehealth and telemedicine, and personalized medicine [16]. We follow this definition and consider solutions in any of these categories as digital health solutions. Indeed, digital solutions have recently been implemented for TB meeting preparation and conduction [17,18,19]. The NAVIFY Tumor Board (NTB), is a cloud-based workflow that facilitates multidisciplinary meetings by integrating all relevant clinical data into a single source [5, 6]. It assists with coordinating, scheduling, preparing, presenting and documenting decisions for TBs [5, 6]. Several recent publications have examined legacy approaches to TBs to identify unmet needs, and demonstrated how NTB could improve efficiency [5, 6], by reducing clinician preparation time for TBs [6, 20, 21].

2 Aim

Little evidence exists on how digital technologies can address challenges for TB conduction. The specific objectives for this study were to test if NTB decreased average patient case discussion time and decreased postponement of patient cases. Impacts were examined at four different tumor boards: Breast, Gastrointestinal (GI – gastrointestinal and hepatopancreatobiliary), Ear, Nose and Throat (ENT) and Hematopathology.

3 Methods

3.1 Study design

A prospective cohort observational study evaluated TBs conduction before and after the implementation of NTB at the University of Missouri Healthcare (MU). The study was reviewed and approved by the local research and ethics committee (IRB #2005046-QI).

Four TBs were evaluated; all used NTB for routine clinical practice (Breast, ENT, and GI TBs); however, the Hematopathology TB was involved in ongoing development. Participants were physically co-located in the same room during TBs; however, NTB can also support remote virtual participation [22].

Case discussion time was recorded and compared during four phases of NTB implementation (Fig. 1a):

  • Phase 1: Before NTB implementation (legacy approach)

  • Phase 2: Manual NTB (no integration with EMR; users manually uploaded patient data into NTB)

  • Phase 3: Partial Integration (gradual integration with EMR; this permitted ordering of TB case discussion via EMR, and subsequent flow of patient information from EMR to NTB; details in the Software section)

  • Phase 4: Stable integration (Post initial integration)

Fig. 1
figure 1

a Study design showing detailed timeline of the phased rollout of NTB for all TBs, as presented in a separate manuscript focusing on case preparation. Notably, ENT TB started phase 1 relatively later than other TBs as we followed snowball recruitment process, and decided to skip phase 2 and start phase 3 right away to keep a similar timeline with other TBs. (b–e) Weekly mean TB discussion time/case for b breast, c GI, d ENT, and e hematopathology. Significant decreases in case discussion time observed for breast and GI TBs. No significant changes were observed for ENT and hematopathology. Y-axis represents average case discussion time (minutes) for each TB. X-axis represents the weeks when TBs were conducted. Black dots represent the average discussing time in the corresponding week. Vertical lines delineate the week of the launch of the NTB application, after initial and stable integration as indicated

Full size image

Within each TB, comparisons were conducted, as follows (Table S1):

  • Phase 1 vs. Phases 2–4: effect of pre- vs. post-NTB.

  • Phase 1 vs. Phase 3&4: pre-NTB compared to integrated version.

  • Phase 1 vs. Phase 4: pre-NTB compared to stable integration.

  • Phase 2 vs. Phase 3&4: manual vs. integrated.

  • Phase 2 vs. Phase 4: manual vs. stable integration.

In addition, comparison was conducted on postponement for pre-NTB (phase 1) and post-NTB implementation (phase 2-4 combined; Table S1).

3.2 Software

The “manual” version of NTB (Roche Molecular Systems, Inc., Belmont, CA) was implemented through a phased rollout to each TB throughout 2018 (Fig. 1a). The integration phases involved partial integration with the hospital EMR. This permitted ordering of a TB case discussion via EMR, and subsequent flow of patient information from EMR to NTB. In the initial phase patient demographic data (name, age, sex, date-of-birth, and medical record number) were automatically incorporated into NTB. Pathology report integration was introduced on November 7, 2018 and finalized on April 9, 2019. The phase from April 9 to end of July 2019 was considered stable after integration (Phase 4). NTB integration is ongoing with the ultimate objective of full integration of all necessary data sources.

3.3 Data collection on case discussion time and postponement

Discussion time was recorded by a Nurse Navigator for each case, at each TB. Case postponements were recorded as comments (e.g. “discuss next week”). More detailed postponement reasons were not collected because of the challenge for the nurse navigator to record details whilst concurrently participating in the meeting given the fast pace of case discussions.

3.4 Training

Participants (all healthcare workers preparing and presenting cases at TBs, incl., nurse navigators, pathology residents, etc.) received comprehensive NTB training and were familiarized with all NTB versions prior to initiation of study phases. One key nurse navigator received initial training (remote and on-site trainings), and then trained all new users. This included, general features and functionalities, as well as on specific areas related to a given user(s) role and contribution.

3.5 Statistical analysis

For comparison of case discussion time, a student’s t-test was performed, where data met the assumption of normality, and Mann–Whitney (non-parametric) U test otherwise (Table S1). The postponements of cases in pre- and post-NTB were examined with Chi-square or Fisher's exact test (Table S1). Fisher 's exact test was used when sample size was small (at least one cell with expected size <5). A p-value of < 0.05 was considered statistically significant. Average discussion time/case was calculated for each week. Interquartile range (IQR) and standard deviation (SD) of the averages were calculated for the meetings of: Pre-NTB (Phase 1); Post-NTB (Phases 2–4); Manual (Phase 2); Post-Integration (Phase 3 & 4); and, Integration Stable (Phase 4).

4 Results

4.1 Breast cancer TB

Sixty-four breast TB meetings, involving 441 individual case discussions, were evaluated (Table 1). Users spent 40 h (2406 min) discussing patient cases. Average discussion time/case significantly decreased from 6.0 (SD = 1.01) min in phase 1 to 4.4 (SD = 0.57) mins (p<0.05; Table 1; Fig. 1b; Table S1), representing a 27% reduction (Table S2). Additionally, significant decrease in average discussion time/case was observed in the manual version (phase 2) compared to integrated version (phase 4), 5.7 vs. 4.4 min (p<0.05; Table 1, S1). From Fig. 1b, we observed a trend that such time-saving improvements were sustained and became more evident over time (Table 1). Postponement rate was 2% pre-NTB, considered low with limited space for improvement, and remained at 2% post-NTB (Table 2). Statistical tests for all comparisons are reported in Table S1.

Table 1 Breast, GI, ENT and Hematopathology TB case discussion times
Full size table
Table 2 Breast, GI, ENT and Hematopathology TB case postponement
Full size table

4.2 Ear, Nose & Throat (ENT) TB

Seventy-four ENT TBs, involving 740 case discussions, were evaluated (Table 1). Users spent 68 h (4093 min) discussing cases. No reduction in average discussion time/case or SD was observed between phase 1 and 4 (mean, 5.6 min vs. 5.8 min; SD, 0.73 vs. 0.78 [p>0.1; Table 1; Fig. 1d]). Importantly, the postponement rate reduced from 23% pre-NTB to 10% post-NTB, representing a significant decrease (Table 2; Table S1). It is noteworthy that the average numbers of patient discussed per TB remained unchanged; 10 cases for both pre- and post-NTB (Table 2). Instead, the drop in the postponement rate was related to the different number of cases planned to be discussed (Table 2). For example, on average pre-NTB clinicians proposed an average of 13 patient cases per meeting, as such 3 cases were postponed. Post-NTB implementation, only 11 cases were proposed for discussion. Statistical tests for all comparisons are reported in Table S1.

4.3 Gastrointestinal cancer TB

Seventy-five GI TB meetings, involving 648 case discussion, were evaluated (Table 1). Users spent 64 h (3850 min) discussing cases. Average discussion time/case significantly decreased from 6.4 (SD = 2.13) min in phase 1 to 5.1 (SD = 1.43) mins (p<0.05; Table 1; Fig. 1c; Table S1), representing a 20% reduction (Table S2). Additionally, significant decrease in average discussion time/case was observed in the manual version (phase 2) compared to integrated version (phase 4), 6.0 vs. 5.1 min (p<0.05; Table 1, S1). We observed a trend that such time-saving improvements were sustained become more evident over time (e.g. decreasing pattern but not statistically tested; see Table 1). Postponement rate was 4% for both pre- and post-NTB (Table 2). Statistical tests for all comparisons are reported in Table S1.

4.4 Hematopathology cancer TB

Seventy-two Hematopathology TBs, involving 483 case discussion, were evaluated (Table 1). Users spent 33 h (1967 min) discussing cases. No reduction in average discussion time/case was observed between Phase 1 and 4 (mean, 3.6 min vs. 3.8 min; SD, 1.26 vs. 0.8 [p>0.1; Table 1; Fig. 1e; Table S2]). The postponement rate was 0% for pre-NTB, and 1% in post-NTB (Table 2). Statistical tests for all comparisons are reported in Table S1.

5 Discussion

TBs have been widely implemented as the gold standard for cancer care decision-making. This has placed significant strain on cancer service providers because it is time-consuming, cancer incidence is rising, and preparation and conduction of TBs requires coordinated efforts from multiple staff, across multiple locations. In addition, the complexity of diagnostic information and treatment options has increased, necessitating solutions to facilitate the structured codification, visualization, and interpretation of complex clinical data. In response, there is a growing body of literature on ways to improve TB implementation from a variety of academic disciplines, such as psychology, improvement science and organizational science [3]. These studies have identified characteristics of effective TBs [9], and have sought to test behavioural interventions to improve decision-making. One such study from the UK demonstrated that simply taking a break during a 3 hr long TB meeting improved performance [12], however, it remains largely unknown how digital solutions could support best practice.

To our knowledge this study is the first to examine the impacts of a digital solution on TB meeting conduction, and specifically on case discussion time and postponement rates.

5.1 Case discussion time during TBs

Our results showed that the average discussion time/case was between 3.6 and 6.4 min pre-NTB across four TBs (Table 1), which, to our knowledge, provides the first report of case discussion time/case during TBs in the US. Limited published data exists about case discussion time, and almost all studies were conducted in the UK NHS (Table S3), notwithstanding, our results were comparable. For example, at the American College of Surgeons (ACOS) accredited breast cancer TB, where all cases must be discussed, the average discussion time was 6.0 min; comparable to previously published work (range 0.5-9 min) [12, 23]. It should be noted, however, that UK and US TBs are not directly comparable, since UK meetings can last up to 3.5 h [12], as compared with 1 h in this study, the increased duration may result in shorter discussion time/ per case (Table S1). This seems to be supported by our data, which demonstrated that median discussion time was between 3.5 and 6.1 min across four TBs (Table 1), compared with the UK, which reported that half of cases were discussed within two minutes.

Our results showed average case discussion time decreased significantly in pre-NTB vs. post-NTB for the Breast and GI TBs. In addition, a significant decrease was observed between the integrated vs. manual version. Most importantly, the improvements realized were sustained and became more evident over time (Figs. 1b–d). No decrease in case discussion time was seen for hematopathology TBs. It should be noted that the hematopathology TB was the co-creation partner and the initial implementer of the NTB, as such, results may be confounded by activities related to ongoing co-creation work. Similarly, no time decrease was seen for ENT TBs, however, important benefits were observed in decreased postponement rates.

5.2 Variance of case discussion time

We observed a trend that variance (SD and IQR) of case discussion time decreased across three TBs (breast, GI and hematopathology; Table 1; Fig. 1). Despite not statistically tested, we want to highlight that this pattern was observed consistently across TBs, with a percentage decrease in SD of 39% (range 34%-45%). Future studies with larger sample size are required to examine the impact on variance statistically.

Variance can be considered a surrogate for process standardization [24], and is critical for increased uniformity of practice [24], to ensure consistency across patient treatments [25], facilitate resource planning [25], increase efficiency and reduce cost [25], enhance patient safety and decrease the risks induced by variation [24]. Understanding the impacts of digital solutions, with respect to operational efficiency and variance, is key for quality improvement (QI) programs that monitor, analyse, and improve hospital processes to effectively implement change. [17]

Our results suggest that NTB standardized the process of meeting conduction, likely through integration with the EMR and a well-designed workflow. This has important ramifications for decreasing administrative burden of TBs, protecting physicians against EMR related burnout, and supporting audit and governance.

5.3 Postponement

The postponement rate for the ENT TB decreased significantly from 23 to 10% (Table S1), but importantly, average number of patients discussed remained unchanged. Instead, the drop was related to the different number of case discussions planned. For example, pre-NTB clinicians proposed 13 patient cases per meeting, resulting in 3 postponements. Post-NTB, only 11 cases were proposed, likely due to the increased scheduling transparency (e.g. NTB displayed scheduled and prepared cases). Other TBs included in this study already had low postponement rates (below 5%) and therefore, had limited space for improvements. Postponement of discussion results in an unnecessary delay, which can be distressing for patients and potentially affect their wait to start treatment.

This study examined operational factors related to TB case discussion. Future work is required to investigate the impact of digital solutions on the quality of discussions [26], mental fatigue, treatment decisions, and the sustainability of benefits. It has been shown that the effectiveness of behavioural interventions alone varies and can be difficult to replicate [27,28,29]. In addition, general fatigue has been recognised by World Health Organization (WHO) as a leading contributor to medical error [30], but fatigue arising from intensity and complexity of work-load in healthcare delivery [31,32,33] has not received adequate investigation or recognition [12]. It remains to be assessed whether digital solutions in combination with behavioural interventions could deliver even more sustainable benefits.

5.4 Challenges of TBs conduction and opportunity areas for technologies

The National Cancer Action Team in England have identified 6 key dimensions for an effective TB including: An appropriate team; reliable infrastructure; well-organized; efficient logistics; patient-centered clinical decision-making; and robust governance. Digital solutions can improve infrastructure, facilitate better TB organization, streamline preparation and conduction [21], enable greater consideration of patient preference, and facilitate evidence-based decisions by integrating additional clinical decision support into the TB workflow. In addition, digital solutions can provide real-time access to data on performance metrics for audit.

5.5 Challenges and limitations

As with all studies of real-world clinical practice [34, 35] there were several challenges and limitations in this research study.

First, the case discussion time were manual recorded in real-time during TB meetings (which are often ran in a fast pace) by nurse navigators, as such, times could have been logged with rounding or incorrectly. This should have been mitigated by the large numbers of observations. Real-time data collection in general is a key challenge in studies evaluating digital health solutions, as at the moment it often requires manual data collection, and places high requirements for the data collectors/assessors (e.g., with deep clinical knowledge, able to capture the clinical discussions and document in time during fast-paced TB meetings). Digital solutions and/or features built in these solutions that can support data collection are highly needed.

Second, there was limited time for nurse navigators to record postponement reasons, as such, it was not possible for us to evaluate the root cause (e.g., missing pathology reports, incomplete patient information etc.) for these postponements. Similar to the previous point, it is in general challenging for data collectors to document these in detail and with high accuracy in real-time during the TB meetings. Digital solutions and/or features that support these data capturing can offer opportunities in studies evaluating digital health interventions.

Third, as summarized in the background section, case discussion time is only one aspect of effective TB meeting conduction, and our study hasn’t evaluated the impacts of the digital solution on other aspects, such as discussion quality (e.g., equal contributions from team members of different disciplines, whether decisions on treatment plans reached, rates of decisions at TB meetings are implemented in real practice). We started with the conduction time of TB meetings, mainly because that, by the time of the study design, there were no widely used and well-validated questionnaires to measure the quality of TB meetings. A few recent studies [3, 12, 23, 26, 36] offer great potential and we are in preparation for new studies to apply these methods.

Fourth, we observed that the impacts of the digital solution vary across four types of TBs in our study. Therefore, the generalization of our findings to other cancer TBs and various types of healthcare providers warrants further investigation in future studies.

6 Conclusions

To our knowledge, this is the first prospective study to demonstrate significant benefits of a digital solution for TB meeting conduction. This study demonstrated that NTB significantly decreased average discussion time per case at Breast and GI TBs, and decreased postponement rates in ENT TBs. These benefits reflect NTB’s generalizability to different TBs. Adoption of such solutions could improve the efficiency of TBs which could have direct economic benefits to a hospital.