Introduction

Gynecologic cancers are some of the most frequently diagnosed cancers in women, accounting for 13.3% of cancer incidences in 20201. High dose-rate (HDR) brachytherapy is a common form of internal radiation therapy used during the treatment process of gynecologic cancers owing to the increased precision associated with the delivery of very high dose, short-range radiation to cervical and endometrial malignancies2,3. This method is known to reduce the risk of normal tissue toxicity while delivering therapeutic doses with curative intent, as the radiation dose is conformal within the cancerous region thus sparing the surrounding healthy tissues from exposure to unnecessary radiation4. A wide range of applicator systems exist for delivering short-range radiation; however, intracavitary applicators such as Vienna/ring-and-tandem (with or without interstitial needles) or Fletcher/tandem-and-ovoids applicators are commonly used (Fig. 2)5,6,7. In treating cervical or endometrial cancers where the uterus remains intact, the applicator is positioned within the vagina, extending through the cervix and into the uterus. Interstitial needles may be added for additional targeting and are inserted through the patient’s perineum. Precise and accurate placement of the applicator and needles is vital for successful treatment, as well as the protection of the surrounding organs at risk (OARs), including the bladder and rectum8,9,10.

Identifying the intracavitary applicator and interstitial needle tips is critical to ensure effective placement and treatment dose conformity, as the radioactive source dwells within the lumens of these apparatus11,12. Medical imaging is used intra-procedurally to ensure the appropriate localization of applicator/needle placement, helping to contribute to a lower incidence of radiotherapy side effects in cervical cancer patients12,13,14,15. The current standard-of-care modalities for HDR gynecologic brachytherapy dosimetric treatment planning are computed tomography (CT) and magnetic resonance (MR) imaging16,17,18. These advanced imaging modalities are widely used after applicator placement for image-based treatment planning due to their soft tissue contrast and three-dimensional (3D) imaging capabilities. Many centers also use imaging to provide 3D visualization of the applicators and needles intraoperatively. While CT and MR images provide accurate 3D visualization of the pelvic region and its anatomical structures, they are expensive and their use is time-consuming, with some MR protocols taking up to 45 min, hindering treatment at overburdened or cost-constrained healthcare facilities19,20. The most widely available intraoperative imaging technique is two-dimensional (2D) ultrasound (US), which is commonly used trans-abdominally to guide applicator insertion and ensure appropriate placement14. The feasibility of combining intraoperative 2D US images with advanced modalities has been previously investigated; however, 2D US imaging alone does not provide the volumetric information necessary for evaluating the applicator placement relative to the surrounding anatomy in 3D space21,22,23,24,25. Gynecologic brachytherapy dosimetry does not rely on the Hounsfield units of CT imaging and, instead, assumes the region to be water-equivalent, enabling the possibility of US-based dose planning. Thus, an alternative 3D imaging modality is required for the identification of the applicator and needle placement during gynecologic brachytherapy treatments to improve treatment quality and allow for accessible intraoperative imaging methods with the potential for dose planning.

Volumetric 3D US imaging is a technique whereby 2D US images are acquired in succession over a spatial interval and reconstructed to provide a 3D image26. Previous studies have demonstrated the application of 3D US imaging to gynecologic brachytherapy treatments. Our group formerly showed that 3D trans-vaginal or trans-rectal ultrasound (TRUS) provided clear visualization of needles and small geometric imaging errors when used intraoperatively during interstitial brachytherapy treatments27,28. The addition of 3D trans-abdominal ultrasound (TAUS) information to CT images was found to improve cervical brachytherapy dose planning when compared to advanced modality imaging alone29. However, using 3D TAUS or 3D TRUS imaging in isolation has a reduced viewing area and results in uncertainty in needle tip visualization or obfuscation of the anterior region of the anatomy due to shadowing from the applicator. A recent study overcame this limitation by demonstrating the feasibility of combining two 3D US images of the female pelvis; however, the technique has yet to be demonstrated clinically and a lack of information exists regarding the method’s robustness with applicators that include interstitial needles30. Therefore, there exists a clinical need for a novel 3D US intraoperative image guidance system that combines 3D TAUS and 3D TRUS images for full visualization of the pelvic region. This paper builds on these previous studies to describe the development of a method to register and fuse 3D TAUS and 3D TRUS images with high accuracy, demonstrating the results in a clinical setting.

Methods

Three-dimensional ultrasound systems

Two custom 3D US systems were developed for 3D TAUS and 3D TRUS image acquisition purposes. The systems are outlined in detail in the following sections.

Three-dimensional TAUS system

We developed a dedicated 3D TAUS system for use during HDR gynecologic brachytherapy procedures. Our system (Fig. 1a) consists of four components: (1) a motorized US scanner attached to a mechatronic stabilizer, (2) a 3D-printed transducer cradle, (3) any commercially available US machine, and (4) a computer system for image reconstruction and visualization31. The system was designed to be customizable to accommodate any commercial US transducer by using a 3D-printed resin cradle, which matches the transducer casing. For all phantom studies and clinical experiments, our 3D TAUS system used a point-of-care Philips CX50 (Philips Healthcare, Amsterdam, The Netherlands) US machine and a C5-1 curvilinear transducer operating at 3 MHz. For image acquisition, the transducer was secured in the transducer cradle and connected to the motorized scan head. The motorized scanner moves the transducer through a hybrid 40 mm translation and 60° of tilt motion over the course of 10 s to acquire images over a large field-of-view (FOV) pelvic region. While the transducer moves, 2D US images are acquired into a computer at a fixed translation spatial interval of 0.25 mm and a tilt angular interval of 0.5º using an onboard Matrox Clarity UHD frame grabber (Matrox Graphics Inc., Quebec, Canada)32. As they are acquired, 2D TAUS images are reconstructed into a 3D TAUS image which is available for viewing using our custom workstation software33,34.

Figure 1
figure 1

Illustrations of custom 3D US mechatronic systems for use in HDR gynecologic brachytherapy. (a) 3D TAUS system including relevant components. During acquisition, the scan plane (grey) tilts 60° in- and out-of-page. (b) 3D TRUS system mounted on a standard operating room bed. The rotational motion of the transducer is controlled by the motorized scanner.

Full size image

Three-dimensional TRUS system

Our 3D TRUS system is a modified version of the system previously described by Rodgers et al. (Fig. 1b) 28. The system is compatible with endocavity transducers from any commercial US machine using customized 3D-printed cradles. Using the same portable US machine, a C10-3v endocavity transducer was used during all phantom and clinical imaging procedures. The endocavity transducer is conventionally used endovaginally; however, we used the micro-convex offset nature of the transducer to acquire 3D TRUS images, leveraging the expanded FOV for the pelvic anatomy. The images generated by the C10-3v transducer have an approximate 5° offset from the central axis of the transducer, allowing the reconstruction of a 3D TRUS image that is tilted anteriorly to provide a 3D view of the applicator in relation to the cervix and uterus. These 3D TRUS images were generated from equally spaced 2D TRUS images acquired every 0.5º during a 180° rotation around the transducer’s central axis (Fig. 1b). Image viewing and reconstruction were completed in a similar manner as the 3D TAUS images31,35. The time for the full range of transducer rotation is 15 s, at which time the 3D TRUS image is immediately available for viewing.

Anthropomorphic pelvic phantom

A tri-modal female pelvic imaging phantom, built on previous work by Rodgers et al., was designed and fabricated within a custom template box that included removable walls for imaging (Fig. 2)30. The phantom replicated the female pelvic anatomy by the inclusion of a cervix, uterus, tumor, bladder, and rectal canal. The cervix, uterus, and tumor were created using 3D-printed molds of patient contours from a diagnostic MR image. To mimic the soft tissue contrast in US images and the speed-of-sound in tissue, the phantom was composed of agar powder (Sigma Aldrich Co., St. Louis, MO, USA), glycerol (Sigma Aldrich Co., St. Louis, MO, USA) and distilled water. SigmaCell cellulose (Sigma Aldrich Co., St. Louis, MO, USA) and tungsten powder were used in various internal structures to provide appropriate levels of contrast in MR and CT images. Qualitative experiments were performed to determine the ideal fiducial material to embed within the phantom for apparent visualization in US, CT, and MR. Consequently, 2 mm brass fiducial spheres were placed within the phantom, on the surfaces of the uterus and cervix, for image registration analysis.

Figure 2
figure 2

Depictions of the custom phantom boxes. Female pelvic anatomy (cervix, uterus, bladder, rectal canal) is shown along with an embedded: (a) Vienna/ring-and-tandem applicator and an example interstitial needle, and (b) Fletcher/tandem-and-ovoids applicator.

Full size image

Phantom study

Phantom experiments were performed with the two types of intracavitary HDR gynecologic brachytherapy applicators available for clinical use at our institution. A CT-compatible, MR-conditional, Vienna-style ring-and-tandem applicator system (Varian Medical Systems, Palo Alto, CA, USA) was embedded within a pelvic phantom alongside the internal structures (Fig. 2a). A 60 mm uterine tandem length (past the cervix) with a 30° angle of the central tandem was chosen to represent a standard patient setup. The ring cap was placed at the external os of the cervix and four interstitial needles were inserted through the ring and into the tumor to simulate the clinical workflow.

A CT-compatible, MR-conditional, titanium Fletcher-style tandem-and-ovoids applicator (Varian Medical Systems, Palo Alto, CA, USA) was inserted into a second phantom (Fig. 2b). A 30° central tandem and 25 mm diameter ovoids were chosen as a representative geometric scenario. An acetal cervical stop was placed 60 mm from the tandem tip, corresponding to the external os of the cervix and the active length of the ovoids was placed just beyond the cervical stop.

3D TAUS imaging was performed on the pelvic phantoms at a depth of 16 cm, which visualized the full extent of the bladder, uterus, and cervix. The reconstructed image size was 177 × 215 × 262 mm3 with a voxel size of 0.29 × 0.29 × 1.48 mm3. Complementary 3D TRUS images were acquired at a 9-cm depth, visualizing a portion of the uterus that included the full extent of the intrauterine tandem and much of the cervix. The lack of full visualization of the region-of-interest is typical for endorectal imaging; however, our use of the 5º offset transducer improved the FOV by angling the transducer view towards the uterus when compared to standard end-fire US imaging. The reconstructed image size was 123 × 157 × 157 mm3 with a voxel size of 0.18 × 0.18 × 0.39 mm3. To expand the FOV, a second 3D TRUS image was acquired 10 mm inferiorly to the original image using a method similar to the procedure described by Rodgers et al30. All 3D US images were acquired such that the phantom tumor was fully visualized in all views.

Comparison to gold-standard CT and MR images was performed for both pelvic phantoms. CT images were acquired with a Canon Aquilion ONE system (Canon Medical Systems, Tochigi, Japan) using an axial helical acquisition (mA = 124, kVp = 120, slice thickness = 0.5 mm). A 3 T GE Discovery MR750 (GE Healthcare, Chicago, IL, USA) was used to obtain MR images of the entire phantom box, using a 32-channel cardiac coil (GE Healthcare, Chicago, IL, USA). A T2-weighted fast spin-echo sequence was used (TR = 7.2 s, TE = 0.1 s, 3 mm slice thickness).

Registration and fusion

Registration and fusion procedures were performed for images of both phantoms using our in-house developed software containing modules for image acquisition, 3D visualization, 3D manipulation, image registration, and image fusion31. Rigid registration was first performed for the two 3D TRUS images by applying a matrix translation in the sagittal direction corresponding to the 10 mm transducer pull-back distance. The registered images were subsequently fused using the custom developed fusion function described below, resulting in a final 3D US image with updated, combined voxel intensity values. Although previous proof of concept work used an intensity averaging algorithm to combine two 3D US images, we found this to be insufficient in cases where significant shadowing from the ring or ovoids was present in one of the US views (Fig. 3)30. To ensure that the images were merged such that shadowing artifacts in the fused image were substantially reduced, we devised a new custom weighting function, \(w\), to sum the voxel intensities \(I\) of the two 3D US images, \({I}_{TAUS}\left(x, y, z\right)\) and \({I}_{TRUS}\left(x, y, z\right)\), into a final combined 3D TAUS-3D TRUS image with fused voxel intensities \({I}_{F}\), as follows.

$${I}_{F}\left(x, y, z\right)={I}_{TAUS}\left(x, y, z\right)\left(\frac{\pi }{4}w\left({I}_{TAUS}, {I}_{TRUS}\right)+\frac{\pi }{4}\right) +{I}_{TRUS}\left(x, y, z\right)\left(\frac{\pi }{4}w\left({I}_{TAUS}, {I}_{TRUS}\right)+\frac{\pi }{4}\right)$$
(1)

where

$$w\left({I}_{TAUS}, {I}_{TRUS}\right)=\frac{{I}_{TAUS}\left(x, y, z\right)-{I}_{TRUS}\left(x, y, z\right)}{{I}_{TAUS}\left(x, y, z\right)+{I}_{TRUS}\left(x, y, z\right)}$$
(2)

and the following conditions were imposed,

Figure 3
figure 3

Fused 3D US images of a Vienna ring-and-tandem phantom. (a) and (c) depict sagittal and coronal views of images fused using an averaging function, respectively. Arrows indicate regions of shadowing or reflection artifacts that are present. (b) and (d) depict sagittal and coronal views of images fused using our updated weighting function. Note the lack of image artifacts in regions corresponding to the arrows in the left figure panels.

Full size image
  1. i.

    If \({I}_{TAUS}\left(x, y, z\right) = 0\) then \({I}_{F}\left(x, y, z\right) = {I}_{TRUS}\left(x, y, z\right)\)

  2. ii.

    If \({I}_{TRUS}\left(x, y, z\right) = 0\) then \({I}_{F}\left(x, y, z\right) = {I}_{TAUS} \left(x, y, z\right)\)

  3. iii.

    If \({I}_{TAUS} \left(x, y, z\right) = {I}_{TRUS} \left(x, y, z\right)\) then

  4. iv.

    \(0 \le w\left({I}_{TAUS}, {I}_{TRUS}\right)\le 1\)

The combined 3D TRUS image was then registered to the 3D TAUS image using a manual rigid registration of the brachytherapy applicator model. To achieve this, a 3D rendered computer-aided design (CAD) model was constructed for each intracavitary applicator, resembling the library applicators commonly found in treatment planning systems. This involved measuring the physical components of each applicator and creating a corresponding model based on the physical dimensions. Subsequently, the applicator models were imported into the 3D US images using our custom workstation software. Manual adjustments based on the shadows and reflections present in the 3D US images were then completed to ensure accurate placement of the applicator model within the images. Identifiable rigid landmarks, such as the tandem tip, ovoids, or ring cap were used to aid in this process. Fiducial markers and anatomical features were deliberately avoided for matching purposes to prevent any potential cyclic bias between the registration and validation procedures. Once the applicator models were successfully positioned, the two 3D US images were rigidly registered. This was accomplished by performing a landmark registration of the applicator models, utilizing the point cloud generated from the 3D rendering.

Any overlapping voxels were fused using Eq. (1). Following the 3D TAUS—3D TRUS image fusion, the resultant combined 3D US was treated as a distinct image. To register fused 3D US with CT or MR images, the applicator model was positioned in each image and rigidly registered using the same point cloud matching procedure.

Image analysis

To ensure the volumetric accuracy of the 3D TAUS—3D TRUS fused image, segmentations of the bladder and uterus were performed by a trained observer and compared to the known volumes of the molds used to create the phantom structures. To evaluate the 3D TAUS- 3D TRUS image rigid registration, fiducial markers external to the applicator model were localized in each of the 3D US images prior to the registration. For the Vienna ring-and-tandem applicator, four interstitial needle tips were selected as fiducial points using the bright reflections from the tip. For the Fletcher tandem-and-ovoids phantom, three brass fiducial spheres located on the uterine or cervical surface were used for registration analysis. Using these fiducials, the target registration error (TRE) and fiducial localization error (FLE) were calculated36.

The mean TRE was calculated to assess the Euclidean distance errors between corresponding fiducials in two images upon registration.

$$TRE=\parallel T\left({p}_{2}\right)-{p}_{1}\parallel$$
(3)
$$<TRE> = \sqrt{\frac{1}{N}{\sum }_{ i=1}^{ N}TR{E}_{i}^{2}}$$
(4)

where T is the rigid body transformation that includes the rotated and translated point \({p}_{2}\left(x, y, z\right)\) to the corresponding point \({p}_{1}\left(x, y, z\right)\) after registration and \(N\) is the number of calculation repetitions.

For individual fiducials, the mean FLE was calculated to quantify the intra-observer point localization error at three time points, a minimum of 24 h apart.

$$FLE=\sqrt{{\sigma }^{2}\left(x\right)+{\sigma }^{2}\left(y\right)+{\sigma }^{2}\left(z\right)}$$
(5)
$$<FLE> = \sqrt{\frac{1}{N}{\sum }_{ N}FL{E}^{2}}$$
(6)

where \({\sigma }^{2}\)(i) is the variance in the individual Euclidean directions.

Clinical proof of concept study

The 3D US system and related fusion techniques were approved for a proof of concept clinical study (NCT# 04705467) at Western University (London, ON, Canada) in accordance with our institution’s Research Ethics Board and Health Canada’s investigational testing authorization (ITA) guidelines and regulations. Two patients who were previously diagnosed with a malignant neoplasm of the cervix uteri and were scheduled to receive HDR gynecologic brachytherapy were enrolled in the study and provided written informed consent. The first patient (P1) was a 47-year-old female who received two insertions of a ring-and-tandem applicator (no interstitial needles). The second patient (P2) was a 53-year-old female who received two insertions of a tandem-and-ovoids applicator. Apart from the addition of 3D US imaging, the clinical workflow adhered to our institution’s standard of care. In both cases, with the patient in the lithotomy position and after the administration of general anesthesia, our 3D TRUS mover and stabilizer arms were attached to the operating room bed (Fig. 1b). As is the usual practice of our institution, a Foley catheter was inserted into the patient’s bladder, and the bladder was filled with approximately 100 mL of fluid. Next, the brachytherapy applicator was placed within the patient under 2D US guidance by the attending radiation oncologist. The TRUS system was placed into the rectum, and both 3D TRUS and 3D TAUS imaging were performed with the applicator in place. The pull-back method described in Section “Phantom Study” was used during the patient procedures to ensure a full FOV with the endocavity transducer. Both patients were imaged using sterilized US coupling gel, with 3D TAUS depth settings of 13 cm and 15 cm for P1 and P2, respectively, and 3D TRUS depth settings of 8 cm and 10 cm for P1 and P2, respectively. The total time for image acquisition using both 3D US systems ranged from approximately 1–5 min, depending on factors such as the number of images captured and the duration of review by the radiation oncologist. After image acquisition, both 3D US systems were removed from the operating room prior to patient recovery, thus ending the intraoperative imaging portion of our study. Next, typical CT and MR simulation imaging were performed for dose planning purposes approximately 2 h after the applicator insertion, with the patient’s legs lowered into a supine position. The images from the planning modalities were de-identified and used as gold-standard comparisons for the fusion study. Offline, the 3D US images were fused, analyzed, and compared to the gold standard in the manner described in Section “Registration and Fusion” and Section “Image Analysis” by two observers who received image analysis training from multiple clinical medical physicists. Specifically, three points were identified on both the 3D TAUS and 3D TRUS images and used as landmarks for fiducial analysis as no physical fiducial markers were placed within the patient during these procedures.

Statistical analyses

All statistical analyses were performed using SPSS 25 (IBM, Chicago, USA). The normality of all registration error metrics (TRE and FLE) was calculated using the Shapiro–Wilk test and the variance of the mean TRE and FLE distributions across the three analysis sessions was determined with two-tailed independent sample t-tests, post confirmation of normality. Significance levels for both tests were p < 0.05, representing a < 5% chance of committing a Type I error. Specifically, the distributions represented the mean TRE and FLE values for each modality comparison and for two observers (n = 48) to determine the geometric accuracy of image registrations. Inter-user reliability analysis was performed by calculating the intraclass correlation of the TRE and FLE values between observers and across all modality comparisons. All statistical analyses were performed for both the phantom and the clinical dataset.

Results

Fusion method

A comparison of the previous voxel intensity averaging technique and our novel fusion function is presented in Fig. 3. It is evident that areas of 3D TAUS-3D TRUS image overlap that previously contained shadow or reflection artifacts are substantially improved when combined using the updated function. This improvement provides clearer visualization of the internal pelvic structures, vital for ensuring accurate intracavitary applicator insertion. Moreover, the mitigation of tandem shadowing on the anterior aspect of the uterus may aid in the prevention of uterine perforation.

Phantom study

Representative imaging results of the Vienna ring-and-tandem phantom and Fletcher tandem-and-ovoids phantom are shown in Fig. 4 and Fig. 5, respectively. For both applicator phantoms, the 3D TAUS image allowed visualization of the embedded structures, intracavitary applicator, and any interstitial needles. The cervix, a portion of the uterus, the majority of the intracavitary applicator, the tumor, and the rectal canal were visible in the 3D TRUS image FOV. The fused 3D US images provided a full visualization of all relevant structures within the phantom. Segmentation metrics for the bladder and uterus resulted in volumes of 52.78 cm3 and 102.2 cm3, respectively, indicating minor deviations from the known mold volumes of 50 cm3 and 100 cm3. The segmented uterus volume meets the AAPM Task Group 128 recommendations for volume measurements on brachytherapy phantom structures of a 5% error from the nominal target volume, while the bladder segmentation deviates by 5.6%37. The quantitative registration and fusion metrics for the combined 3D US and gold-standard imaging modalities of both applicator phantoms are shown in Fig. 8. The FLE values for both phantoms were consistently calculated to be greater for Observer 1 than Observer 2. The maximum of the TRE measurements was 5.4 mm, with the vast majority of errors reported below 5 mm. Observer 1 reported a mean ± standard deviation (coefficient of variation) of TRE values across all modalities of 3.7 ± 0.6 mm (16.2%) while Observer 2 reported values of 4.2 ± 0.3 mm (7.1%). The Shapiro–Wilk test of the TRE and FLE values found no significant difference from normality in all cases. There was no significant difference between observer errors, with the intraclass correlation indicating an 88% agreement for all TRE and FLE values.

Figure 4
figure 4

US, CT, and MR imaging results of the Vienna/ring-and-tandem phantom. (a) Axial view of the registered and fused 3D TAUS and 3D TRUS images. The bladder, uterus, and rectal canal are visible, along with the tandem, tandem shadowing, and needle tips. Reflections of the phantom box contents appear outside of the hyperechoic box outline. (b) Axial view of a 3D TAUS image that localizes the tumor. (c) Sagittal view of a 3D TRUS image, which demonstrates the visibility of the tandem and rectal canal. Tandem shadowing is also apparent above the tandem. (d) Sagittal view of the CT image, with visualization of the applicator and needles. (e) Coronal view of the MR image, which includes relevant anatomy and applicator placement.

Full size image
Figure 5
figure 5

US, CT, and MR imaging results of the Fletcher/tandem-and-ovoids phantom. (a) Sagittal and (b) axial views of the registered and fused 3D TAUS and 3D TRUS images, with bladder, uterus, rectal canal, and tandem visible. (c) Sagittal view of the 3D TAUS image. (d) Sagittal view of the 3D TRUS image. The uterus, tandem, and most of the cervix are visible, as well as some shadowing caused by the ovoids. Pullback artifacts from the 10 mm inferior-superior image acquisitions are also apparent at the edge of each of the spaced scans, but do not impact visibility. Sagittal slices of the (e) CT and (f) MR images also show the relevant anatomy and the ovoid’s appearance in these modalities. In (e) and (f), the left image depicts an image slice of the phantom along the central tandem axis while the right image depicts and image slice of the phantom along a lateral ovoid axis.

Full size image

Clinical feasibility study

Representative imaging results for patients P1 and P2 are shown in Fig. 6 and Fig. 7, respectively. For both patients, the 3D TAUS images showed the majority of the bladder, including the Foley catheter balloon and tip, the full extent of the uterus and cervix, and the central tandem of the intracavitary applicator. The 3D TRUS images depicted a portion of the rectal canal, partial visualization of the bladder and Foley catheter, and views of approximately half of the uterus. Shadowing from the ovoids was visible in the 3D TRUS image of P2, however, the ovoids themselves were not directly visualized. In all scenarios, the resultant fused 3D US image depicted all structures that were present in the single 3D US image views, with improved visualization of regions that were previously shadowed, indicating the need for image fusion to leverage the corresponding shadow/non-shadow areas from each view. The quantitative registration and fusion metrics for the combined 3D US and gold-standard imaging modalities are shown in Fig. 8. Observer 1 reported a mean ± standard deviation (coefficient of variation) of TRE values across all modalities of 3.5 ± 0.7 mm (20.0%) while Observer 2 reported values of 4.6 ± 0.4 mm (8.7%). Similar to the phantom study, the Shapiro–Wilk test of the TRE and FLE values found no significant difference from normality. No significant difference was demonstrated between observer errors, with the intraclass correlation indicating an 82% agreement between observers.

Figure 6
figure 6

Imaging results of P1 with the Vienna/ring-and-tandem applicator inserted. (a) Central axis sagittal view of the registered and fused 3D TAUS and 3D TRUS images, with bladder, uterus, cervix, tandem, and Foley balloon visible. Shadows due to the applicator are mitigated in the fused image. (b) Sagittal view of the 3D TAUS image, with ring position made apparent by its posterior shadowing. (c) Fused 3D TRUS images. Pullback artifacts appear, but the quality of organ and applicator visibility is maintained, with shadowing appearing anteriorly from the applicator. Sagittal views of the (d) CT and (e) MR images demonstrate differences in the appearance of the anatomy and applicator on these modalities.

Full size image
Figure 7
figure 7

Imaging results of P2 with the Fletcher/tandem-and-ovoid applicator inserted. Sagittal views of the registered and fused 3D TAUS and 3D TRUS images show the (a) central axis with bladder, uterus, cervix, tandem, and Foley balloon visible, and (b) the lateral view with ovoid shadowing, (c) sagittal view of the 3D TAUS image. (d) sagittal view of the 3D TRUS image, with pullback artifacts, rectal canal, and tandem visible. Sagittal slices of the (e) CT and (f) MR images also show the relevant anatomy and applicator appearance in these modalities.

Full size image
Figure 8
figure 8

Bar plots of registration error metrics for 3D TAUS-3D TRUS (dark grey), fused 3D US-CT (medium grey), and fused 3D US-MR (light grey) across applicators for both phantoms and patients. Mean errors (n = 6) are presented for both trained observers (plain versus hatched bars) in millimeters for TRE and FLE, with error bars representing one standard deviation. Analysis was completed for (a) the Vienna/ring-and-tandem phantom, (b) the Fletcher/tandem-and-ovoids phantom, (c) P1 (Vienna applicator), and (d) P2 (Fletcher applicator).

Full size image

Discussion

This work investigated the clinical application of 3D TAUS-3D TRUS image fusion for the visualization of interstitial needle tips and intracavitary applicators during HDR gynecologic brachytherapy procedures. Combining 3D US images using the novel fusion methods from this study expands the conventional FOV of clinical US imaging, thereby enabling accurate applicator localization in relation to nearby OARs during gynecologic brachytherapy procedures. Optimization of the fusion algorithm enhances the benefits of this method by minimizing shadowing effects from the applicators in regions of image overlap. Specifically, we leverage the less shadowed corresponding regions on each image view (i.e. the 3D TAUS image has fewer shadows anteriorly while the 3D TRUS image has fewer shadows posteriorly) to mitigate shadowing upon image fusion. We have shown that our imaging approach can be incorporated into clinical procedures that place intracavitary applicators, with a future extension to treatments with interstitial needles. The enhanced 3D US visualization provided by our fusion method has the potential to benefit gynecologic brachytherapy procedures through the verification of applicator placement, the provision of 3D US imaging for dose planning purposes, and the ability to conveniently decouple the transducer to perform 2D US imaging, if required. Intraoperative visualization with 3D US imaging increases the accessibility of 3D imaging modalities to healthcare systems with limited access to advanced imaging techniques such as CT and MR.

The phantom study validated the use of applicator model-based registration by assessing the registration errors associated with fiducial landmarks. We created the model by measuring the diameters and angles of the physical applicator using digital calipers and micrometers, as a full applicator library was unavailable for use in our fusion software. The estimated maximum error associated with modeling the applicator is 0.2 mm based on the errors associated with the measurement tools. This error will have minimal impact on the alignment of the applicator models and the registration accuracy of the images. In the future, the models of the applicators may be obtained from the manufacturers for improved accuracy. Our approach for registration was based on manual selection of salient features on the models and corresponding images. Future development of a semi-automated or automated registration technique would be based on the selection of a small number of salient and appropriately spaced features on the visualized intracavitary applicator to provide an efficient model overlay and subsequent registration. A rigid registration technique was shown to be accurate when placing a model of the Vienna ring-and-tandem applicator, due to its immutable physical nature. However, further work into the validity of using a model overlay for the Fletcher tandem-and-ovoids applicator must be performed due to the range of motion and related dynamic geometry of its structure. Depending on the applicator position, ovoid placement may heavily shadow nearby OARs, yet our initial studies suggest that the optimized fusion algorithm will overcome the related reduced visibility by providing image information for the posterior aspect of the applicator.

Registration errors were consistently greater in the Fletcher tandem-and-ovoids phantom, likely due to the potential mismatch between the applicator model and actual positioning within the agar. The registration of combined 3D US images to either CT or MR standards showed the qualitative accuracy of our fusion method for matching the applicator models within an expanded 3D US view. These comparisons could be improved by applying the same imaging sequences used for clinical patient imaging to our phantom studies. We are presently working to program a 3D MR sequence with reduced slice thickness into our research scanner for future work. Compared to previous work, the registration errors reported in this work are larger, potentially reducing the resolution of the fused 3D US images when compared to their individual image components. These errors provide uncertainties above clinically relevant metrics for the context of dose planning8,30. We ascribe the increased errors to the manual nature of the applicator overlay and expect that improvements to our system will substantially reduce this effect. In future, we aim to develop a corresponding 3D TAUS arm that will enable automated, tracked, and inherently registered images to be acquired from both a 3D TAUS and 3D TRUS view, leaving only the fusion process to be completed after 3D image reconstruction. This will mitigate the need for manual matching of the applicator model and will reduce the 3D US registration errors to the precision level of the tracking device. The 3D TAUS arm will also improve the reproducibility of registration results. In all cases, the FLE values were substantially lower than the TRE metrics, suggesting consistent localization capabilities and clear fiducial points, including the interstitial needle tips. Future work will investigate the effects of variability in the number of needles used during a hybrid interstitial-intracavitary procedure on the 3D US images.

Imaging two cervical cancer patients resulted in similar TRE and FLE values to the phantom study across all modalities’ rigid registrations, suggesting reliable landmark and boundary visualization in human images. The inability to decrease TRE values below 2 mm is attributed to the lack of placed fiducial markers within the patients. Future work will include placing temporary surgical clips within the patient during the applicator insertion procedure to improve the ease of fiducial identification for TRE and FLE reporting. However, while increased image artifacts from the applicator within the vaginal canal were present in the patient images, the Foley catheter proved to be an invaluable landmark when manually placing the applicator model into the 3D US images. While no interstitial needles were present for P1, analysis of future patients within our consented cohort will demonstrate the generalizability of our technique to hybrid treatment configurations and alternate applicator geometries. One limitation of this patient trial is the deformation of OARs and tissues when comparing the 3D US images, which are acquired with the patient in lithotomy, to the CT or MR gold standard images that are acquired with the patient supine. Although we have yet to quantify the shifts in applicator or needle placement during the transition of the patient’s leg position, we expect that the cross-modality registration errors reported here are due to the rigid registration. It is important to reduce the registration errors, as misalignments on the order of 3–4 mm will have large dosimetric impacts. In the future, the employment of deformable image registration, along with a comprehensive analysis of anatomical changes during the imaging and treatment planning process will augment our results. Furthermore, a thorough analysis of patient OAR contours has yet to be completed and is a limitation of this study. In future work, geometric accuracy and volumetric comparisons of OAR segmentation will be completed by trained observers and analyzed in the context of dose planning using 3D US images, including the step size and number of dwell positions of the HDR brachytherapy plan.

Conclusions

The development of a 3D TAUS-3D TRUS image fusion technique for use during HDR gynecologic brachytherapy treatments has been shown to be promising for improving intraoperative visualization and localization of intracavitary applicators, needle tips, and surrounding anatomy. Phantom studies were conducted to evaluate the suitability of rigid applicator-based registration and fusion while a clinical proof of concept study indicated the robustness of our methods intraoperatively. The techniques developed in this study show promise for intraoperative imaging during applicator placement, potentially mitigating the need for alternative imaging and reducing the overall treatment time. Our 3D US image fusion is an appealing alternative to advanced intraoperative imaging, providing inexpensive, effective, and readily accessible visualization that can be easily translated to underserved healthcare centers.