Case Selection
All patient case data sets were real-world patients who were undergoing routine guideline-advocated surveillance for recurrent non-muscle-invasive UC by US physicians. Cases were selected from a database of 1036 US patients enrolled in previous prospective clinical studies of Cxbladder Monitor who were undergoing surveillance for recurrent UC (Clinicaltrials.gov identifier: NCT02700659). The 30 cases selected were representative of the database in terms of disease grade, gender and with respect to the existence of recurrent UC as determined by cystoscopy or pyelography.
Each patient case was assigned a random case number, and all patient identifiable information was redacted as a condition of inclusion in this study.
All procedures performed in this study were in accordance with the ethical standards of the institutional and national research committees and with the 1964 Helsinki declaration and its later amendments. All cases in the present study were patients who had given informed consent to the anonymous use of their urine sample and clinical information for the evaluation of clinical utility.
Physicians
All 18 participating physicians (see Supplementary Material, Table S1) had experience in use of Cxbladder Monitor and included physicians from community practices, academic institutions and large urology practices. Selection of physicians with previous experience using Cxbladder Monitor was intended to minimize the potential for bias due to unfamiliarity with the test. Each physician was provided with a description of the study framework (Supplementary Material, Clinical Utility Assessment of Cxbladder Monitor) and a completed example question to illustrate the study process. Each physician individually evaluated real world patient data sets, with each patient dataset presented to each participating physician in the same sequence, under the supervision of a study coordinator. A standardised questionnaire format was used to collect all clinical data and assessments to enable each physician’s evaluation of each patients case and selected procedures and tests that they would use to evaluate the patient to be collected consistently.
Mode of Assessment
There were 828 diagnostic decisions (physician–patient decisions, hereafter referred to as “interactions”) made by 18 physicians on 46 real-world data sets from 30 patients undergoing surveillance for recurrence of UC. The 828 interactions comprised the clinical utility assessment dataset for this study. All 30 patients had one clinical visit and a corresponding Cxbladder Monitor test, giving rise to 540 interactions. A subset, 16 of the 30 patients, had a second clinical visit 3–6 months later with a corresponding Cxbladder Monitor test providing an additional 288 interactions, for a total of 828 physician–patient interactions. The additional 288 interactions provide a longitudinal perspective of the impact of Cxbladder Monitor results over two consecutive clinical visits.
Each physician–patient interaction data set included all clinical information and decisions made before and after disclosure of the Cxbladder Monitor data, using a previously validated experimental design [21, 22, and Supplementary Material, Clinical Utility Assessment of Cxbladder Monitor]. The clinical information contained data on the patient’s gender, age, ethnicity and risk factors including smoking history, any pre-existing conditions and a timeline summary of all available clinical history data (Supplementary Material, Timelines).
The evaluation of the clinical information comprised four steps. Firstly, following review of the clinical information from each patient, the participating physicians were asked to make an initial recommendation of whether the patient required an investigation (workup) for UC (hereafter referred to as “workup recommendation”). Secondly, following a workup recommendation to investigate for UC, the baseline data set was defined as the tests and procedures ordered for the workup, as well as the timing of those tests and procedures ordered by each physician before disclosure of the Cxbladder Monitor test results.
Thirdly, upon completion of the baseline data set for each patient, each participating physician was provided with the patient’s Cxbladder Monitor result (Supplementary Material, Example Report and Interpretation), an updated clinical history and timeline summary, and then asked to make a second decision on a workup recommendation.
Fourthly, if a workup recommendation was made to investigate the patient for UC, data on tests and procedures ordered was similarly collected on the standard assessment form. Cxbladder Monitor results for each patient were presented to each physician in the same test report format as the commercially available test [20] and Supplementary Material, Example Report and Interpretation.
This evaluation and data collection process was repeated for each of the 30 patients and again, separately and consecutively, for the 16 patient case subset where a second clinical visit had been scheduled and undertaken.
Test and Procedure Classification
Tests and procedures were selected by the participating physicians from a provided list of AUA guideline recommended procedures and tests for the surveillance of UC, with options to add alternative tests if required. For the purposes of this study, cystoscopy (flexible and rigid), computed tomography (CT) scans (contrast and non-contrast), retrograde pyelogram and preparation for biopsy were defined as invasive procedures. Cxbladder Monitor, urinalysis, urine cytology, ultrasound, UroVysion® FISH and cytology reflexive to FISH were defined as non-invasive tests.
Statistical Procedures
For each of the 828 physician–patient interactions, arising from this study, analysis included the average number of all procedures, invasive and non-invasive procedures, and each individual procedure ordered, as well as the average length in weeks for the future scheduling of flexible cystoscopy and CT scan (contrast). This data was used to determine the change between the baseline number of procedures ordered and the future schedule of procedures in weeks before and after disclosure of the Cxbladder Monitor results. All changes were analysed using a 95% t test confidence interval and change was considered statistically significant when the confidence interval did not include 0 (zero).
Heatmap Data Graphic
Graphical representation, to provide active visualisation of the results, at the level of the physician–patient interaction, have been presented as heatmaps. The heatmaps depict the total count of procedures ordered at each of the interactions at baseline and the change, relative to baseline, in the number of procedures ordered following the disclosure of Cxbladder Monitor results. The heatmaps have been drawn with columns representing individual physicians and rows represent individual patients. Each row and column intersection represents a physician–patient interaction and are colour-coded based on either an increase or decrease from baseline with green representing a decrease and red an increase. Patients (rows) are grouped by Cxbladder Monitor result, patient ID and test result. The number at each intersection on the baseline heatmap is the total number of tests and procedures ordered, while the number at each intersection of the “change” heatmap is the change in number of tests and procedures ordered (i.e. “0” means the same number of tests were ordered as at baseline and “− 3” means 3 fewer tests were ordered).
Study Endpoints
The co-primary endpoints were to determine: (1) the change in the number of total procedures ordered and (2) the change in the number of invasive procedures (including cystoscopy) ordered by the physician for each patient. Secondary endpoints included the number of tests or procedures added or avoided based on changes in the scheduling of future cystoscopies and CT scans as a result of the inclusion of the Cxbladder Monitor test results.
Primary and secondary endpoints were evaluated for the 540 interactions, to determine change in physician test ordering after the first Cxbladder Monitor test results were disclosed, compared to the pre-disclosure baseline. The same primary and secondary endpoints were evaluated for the additional 288 interactions, representing the subset where two successive Cxbladder Monitor tests were disclosed.